Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add text-to-speech using Web Speech API #5889

Closed
wants to merge 8 commits into from
Closed

Add text-to-speech using Web Speech API #5889

wants to merge 8 commits into from

Conversation

neubig
Copy link
Contributor

@neubig neubig commented Dec 29, 2024

This PR adds text-to-speech functionality to the chat interface using the Web Speech API. Features:

  • Uses browser built-in Web Speech API for text-to-speech
  • Only speaks assistant messages (not user messages)
  • Can be toggled on/off with a button in the chat interface
  • Automatically stops speaking when disabled
  • Removes markdown formatting before speaking
  • Tries to use a good English voice if available
  • Cancels any ongoing speech when a new message arrives

The implementation is simple and works directly in the browser without additional API calls or dependencies.

All speech-related tests are passing:

  • Chat message speech tests
  • Speech slice tests
  • Toggle speech button tests

To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:e95ad75-nikolaik   --name openhands-app-e95ad75   docker.all-hands.dev/all-hands-ai/openhands:e95ad75

@neubig neubig changed the title Fix React context providers and click outside hook Add text-to-speech using Web Speech API Dec 29, 2024
@neubig neubig self-assigned this Dec 29, 2024
@neubig neubig marked this pull request as draft December 29, 2024 01:00
@neubig
Copy link
Contributor Author

neubig commented Dec 29, 2024

This is not super-high priority and I'm having trouble getting it to work, so I'll close this for now.

@neubig neubig closed this Dec 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants