Skip to content
Merged
57 changes: 57 additions & 0 deletions fern/customization/custom-transcriber/gradium.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
---
title: Gradium
subtitle: Use Gradium speech-to-text as a custom transcriber in Vapi
description: Stream Vapi call audio to Gradium's real-time speech-to-text over a WebSocket bridge.
slug: customization/custom-transcriber/gradium
---

[Gradium](https://gradium.ai) builds real-time audio models, including streaming speech-to-text with semantic voice activity detection. You can connect it through a [custom transcriber](/customization/custom-transcriber) endpoint: a small WebSocket bridge that forwards call audio to Gradium and returns transcripts to Vapi.

<Note>A Gradium account and API key are required.</Note>

<Steps>
<Step title="Build the bridge">
Gradium's [Vapi integration guide](https://docs.gradium.ai/integrations/agent-frameworks/vapi) has the complete bridge, including end-of-turn detection and reconnecting on long calls.

Two things matter most: forward **only channel 0** of the interleaved audio, or the agent transcribes its own speech; and send `partial` transcripts as well as `final` ones, which Vapi needs for barge-in.
</Step>
<Step title="Point your assistant at it">
Host the bridge somewhere Vapi can reach over `wss`, then reference it with
a [custom credential](/server-url/server-authentication) so Vapi can
authenticate to it:

```json
{
"transcriber": {
"provider": "custom-transcriber",
"server": {
"url": "wss://your-server.com/vapi/transcriber",
"credentialId": "cred_transcriber_auth_123"
}
}
}
```

Start a call and your assistant transcribes with Gradium.
</Step>
</Steps>

## Troubleshooting

| Symptom | Fix |
| --- | --- |
| The agent transcribes its own speech | Both channels are being forwarded. Send only channel 0 from the interleaved PCM |
| Turns get swallowed, barge-in stops working | Send `partial` transcripts as well as `final` |
| Final transcripts lose the last word or two | Finalize on Gradium's flush acknowledgement, not on the VAD threshold crossing. See [Gradium's Vapi guide](https://docs.gradium.ai/integrations/agent-frameworks/vapi) |
| The assistant replies before the caller finishes | Vapi's endpointing is racing Gradium's VAD. Tune [start and stop speaking plans](/customization/voice-pipeline-configuration), or hand the decision to Gradium with a [custom endpointing model](https://docs.gradium.ai/integrations/agent-frameworks/vapi) |

## Related

<CardGroup cols={2}>
<Card title="Gradium Vapi guide" icon="book" href="https://docs.gradium.ai/integrations/agent-frameworks/vapi">
The full bridge, tuning, and configuration reference.
</Card>
<Card title="Gradium text-to-speech" icon="volume-high" href="/customization/custom-tts/gradium">
Use Gradium voices as a custom voice.
</Card>
</CardGroup>
56 changes: 56 additions & 0 deletions fern/customization/custom-tts/gradium.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
---
title: Gradium
subtitle: Use Gradium text-to-speech as a custom voice in Vapi
description: Connect the Gradium text-to-speech API to Vapi through a custom voice endpoint.
slug: customization/custom-tts/gradium
---

[Gradium](https://gradium.ai) builds real-time audio models, including streaming text-to-speech with instant voice cloning. You connect it through a [custom voice](/customization/custom-voices/custom-tts) endpoint: a small server that receives the text Vapi wants spoken and streams raw PCM back.

<Note>A Gradium account and API key are required.</Note>

<Steps>
<Step title="Build the endpoint">
Gradium's [Vapi integration guide](https://docs.gradium.ai/integrations/agent-frameworks/vapi) has the complete endpoint, including the SDK setup and streaming loop.

Read `message.sampleRate` on each request and synthesize at that rate. Vapi expects raw 16-bit little-endian mono PCM at exactly the rate it asked for, streamed as it is produced.
</Step>
<Step title="Point your assistant at it">
Host the endpoint somewhere Vapi can reach over HTTPS, then reference it
with a [custom credential](/server-url/server-authentication) so Vapi can
authenticate to it:

```json
{
"voice": {
"provider": "custom-voice",
"server": {
"url": "https://your-server.com/vapi/tts",
"credentialId": "cred_tts_auth_123"
}
}
}
```

Start a call and your assistant speaks with your Gradium voice.
</Step>
</Steps>

## Troubleshooting

| Symptom | Fix |
| --- | --- |
| Audio plays at the wrong speed or pitch | Synthesize at the rate in `message.sampleRate`, and return mono 16-bit little-endian PCM with no WAV header |
| Voice or pronunciation edits don't take effect | Vapi caches custom voice audio. Set `"cachingEnabled": false` on the `voice` object while iterating |
| Long pause before the first word | A new connection per sentence. Pool them as shown in [Gradium's Vapi guide](https://docs.gradium.ai/integrations/agent-frameworks/vapi) |

## Related

<CardGroup cols={2}>
<Card title="Gradium Vapi guide" icon="book" href="https://docs.gradium.ai/integrations/agent-frameworks/vapi">
The full endpoint, tuning, and configuration reference.
</Card>
<Card title="Gradium speech-to-text" icon="wave-square" href="/customization/custom-transcriber/gradium">
Use Gradium as a custom transcriber.
</Card>
</CardGroup>
10 changes: 8 additions & 2 deletions fern/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -289,12 +289,18 @@ navigation:
path: customization/custom-voices/cartesia.mdx
- page: Gandr
path: customization/custom-voices/gandr.mdx
- page: Custom transcriber
- section: Custom transcriber
path: customization/custom-transcriber.mdx
icon: fa-light fa-microphone
- page: Custom TTS
contents:
- page: Gradium
path: customization/custom-transcriber/gradium.mdx
- section: Custom TTS
path: customization/custom-tts.mdx
icon: fa-light fa-volume-high
contents:
- page: Gradium
path: customization/custom-tts/gradium.mdx
- section: Custom LLMs
icon: fa-light fa-brain-circuit
contents:
Expand Down
Loading