Speech to speech: Mute functionality

[pranavjoshi001]

Changelog Entry

• Added mute/unmute functionality for speech-to-speech with silent chunks to keep server connection alive, in PR #5688, by @pranavjoshi

Description

This PR adds mute/unmute functionality for the Speech-to-Speech (S2S) feature. When muted, the microphone is turned off (browser indicator disappears) but silent audio chunks continue to be sent to keep the server connection alive. This prevents connection timeouts while allowing users to temporarily disable their microphone input.

Design

The mute functionality works at multiple levels:

1. AudioWorklet Level:

   • Added MUTE and UNMUTE commands
   • When muted, the processor generates silent (all zeros) Int16 audio chunks instead of real audio data
   • This keeps chunks flowing at the same interval to maintain the server connection

2. useRecorder Hook (useRecorder.ts):

   • Added mute() function that:
     • Sends MUTE command to the worklet
     • Disconnects the source node from the audio graph
     • Stops all MediaStream tracks (turns off browser mic indicator)
   • Returns an unmute() function that:
     • Sends UNMUTE command to the worklet
     • Re-acquires the microphone via getUserMedia
     • Reconnects the source node to the audio graph

3. VoiceRecorderBridge (VoiceRecorderBridge.tsx):

   • Wires the mute function to the voice state machine
   • When voice state transitions to muted, calls mute() and stores the unmute function
   • When voice state transitions back to listening, calls the stored unmute function

Specific Changes

• Added MUTE and UNMUTE command handling to audio-recorder-worklet.ts to generate silent chunks when muted
• Added mute function to useRecorder.ts hook that disconnects audio and stops MediaStream while continuing to send silent chunks
• Updated VoiceRecorderBridge.tsx to handle mute/unmute based on voice state changes
• Added unit tests for mute functionality in useRecorder.spec.tsx:
  • should return mute function
  • should send MUTE command and stop media stream when mute is called
  • should return unmute function from mute() that sends UNMUTE and restarts media stream
• Added E2E test mute.functionality.html to verify mute/unmute flow in the full Web Chat context

• I have added tests and executed them locally
• I have updated CHANGELOG.md
• I have updated documentation

Review Checklist

This section is for contributors to review your work.

• Accessibility reviewed (tab order, content readability, alt text, color contrast)
• Browser and platform compatibilities reviewed
• CSS styles reviewed (minimal rules, no z-index)
• Documents reviewed (docs, samples, live demo)
• Internationalization reviewed (strings, unit formatting)
• package.json and package-lock.json reviewed
• Security reviewed (no data URIs, check for nonce leak)
• Tests reviewed (coverage, legitimacy)