Skip to content

Add voice channel adapter with WebSocket server#798

Open
pbranchu wants to merge 3 commits intoRightNow-AI:mainfrom
pbranchu:feat/voice-adapter
Open

Add voice channel adapter with WebSocket server#798
pbranchu wants to merge 3 commits intoRightNow-AI:mainfrom
pbranchu:feat/voice-adapter

Conversation

@pbranchu
Copy link
Copy Markdown
Contributor

Summary

  • Add VoiceAdapter in openfang-channels that runs a WebSocket server accepting voice clients (mobile apps, Meet bots, web clients)
  • Clients handle STT/TTS directly with providers; the adapter exchanges text using a JSON protocol with utterance, cancel, end (client) and response, backchannel, status, error (server) message types
  • Add emotion tag parsing ([amused], [concerned], etc.) with unknown tag stripping and abbreviation-aware sentence splitting for natural TTS delivery
  • Add VoiceConfig to openfang-types with listen address, default agent, and channel overrides
  • Wire voice adapter registration in channel_bridge.rs with override support

Test plan

  • cargo build -p openfang-channels -p openfang-api --lib compiles cleanly
  • cargo clippy -p openfang-channels -p openfang-api --lib -- -D warnings passes
  • All 10 voice adapter unit tests pass (emotion parsing, sentence splitting, adapter creation)
  • Live test: connect WebSocket client to ws://localhost:4201/voice, send utterance, verify response

🤖 Generated with Claude Code

@jaberjaber23
Copy link
Copy Markdown
Member

Reviewed and approved. This PR has merge conflicts with recent changes on main. Please rebase onto current main and we will merge it.

@pbranchu pbranchu force-pushed the feat/voice-adapter branch 7 times, most recently from b0e71d5 to 72724af Compare March 27, 2026 16:04
@jaberjaber23
Copy link
Copy Markdown
Member

Reviewed and approved. Has merge conflicts with recent changes on main. Please rebase and we will merge immediately.

@pbranchu pbranchu force-pushed the feat/voice-adapter branch from 72724af to e6fb60a Compare March 27, 2026 19:17
Introduces a new voice channel adapter that provides a WebSocket server
for voice clients (mobile apps, Meet bots, web clients). Clients handle
STT/TTS directly; this adapter exchanges text with emotion tag parsing
and sentence-by-sentence delivery.

- New VoiceAdapter in openfang-channels with WebSocket protocol
- VoiceConfig in openfang-types with listen address and default agent
- Channel bridge registration and override wiring in openfang-api
- Emotion tag parsing (amused, concerned, formal, warm, etc.)
- Abbreviation-aware sentence splitting for natural TTS delivery
- 10 unit tests for parsing and adapter creation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@pbranchu pbranchu force-pushed the feat/voice-adapter branch from e6fb60a to 34a21ed Compare March 27, 2026 23:15
Philippe Branchu and others added 2 commits March 28, 2026 04:14
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants