Every account can use it. Try it without code in the console playground (the Live 1 tab).
Building on a voice agent framework? Use the ready-made packages instead of writing a client: Pipecat (
pipecat-aircaps) and LiveKit Agents (livekit-plugins-aircaps). The rest of this page is the protocol they use.
For captions of meetings and conversations with speaker labels, in nine languages, see A5S Streaming v1 instead.
Session flow
1
Look up the WebSocket address
Ask the API where to connect, once before every session. See Get the WebSocket address.
2
Connect and send start
Open the address with your API key in the handshake header, then send a
start message that names the model.3
Wait for ready
Status messages can arrive first. Do not send audio before
ready.4
Stream audio and read transcripts
Send 20 ms PCM frames in real time. Read
transcript messages while you send.5
End turns
Send
finalize when the speaker stops, or let the model detect it with end_of_turn.6
Finish
Send
finish. A last final and finished follow, then the socket closes.1. Get the WebSocket address
websocket_url exactly as returned, with the same Authorization: Bearer aircaps_sk_... header.
Look the address up once before every session. The answer is the server that gives your client the lowest latency among those that can take a session right now:
- The host can differ from one session to the next, and from one of your machines to another. Do not hard-code it, cache it across sessions, or share one answer between machines in different places.
- When a server is taken out of service, the lookup stops handing it out at once. A client that looks the address up moves to another server on its next session with nothing to change.
- As servers are added in more regions, your clients are sent to the nearest one without a change on your side.
5xx), retry it once or twice; only if it still fails, connect to the last address you were given.
If your network restricts outbound connections, allow HTTPS and secure WebSockets on port 443 to api.aircaps.com and to *.aircaps.ai.
wss://api.aircaps.com/v1/realtime also accepts AirCaps Live 1 sessions, with the same messages. The looked-up address has lower latency, so use it for voice agents.2. Start a session
Send this as the first message, within 45 seconds of connecting:ready. provider_status messages can arrive before it; ignore them. An error message means the session was refused; the socket closes after it.
start accepts only the documented fields. Key terms (custom_vocabulary), speaker labels (diarization) and speaker names (speaker_identification) are not part of this model: a start that asks for them is refused with invalid_start.
3. Stream audio
Send audio as binary frames of raw PCM (16 kHz, mono, 16-bit little-endian), at real-time pace. A 20 ms frame is 320 samples, 640 bytes. WAV headers, MP3 and other containers are not accepted: decode, resample and downmix first.
Read from the socket while you send; do not wait until the audio is finished.
finalize and finish do different jobs. finalize asks for the transcript and keeps the session open. finish closes the session. A finish sent after a finalize with no new audio in between returns the same final transcript a second time: it adds nothing, and it arrives one extra network round trip later. Take your transcript, and your timing, from the final that answers finalize.
4. Read transcripts
Text becomes final in two ways. While audio streams, the model settles earlier words on its own and
final_chars grows. And when a turn ends, everything up to that point becomes final at once.
Punctuation. A final ends with its closing punctuation (., ? or !), and the text after it starts with a capital letter, so finals read as complete sentences when you join them. Sometimes the model settles on the closing mark just after it has sent the final. The mark is then added to the end of the final text, and final_chars covers it. That is the only addition ever made to text that is already final. If you take each turn’s text as what was added since the previous final, ignore an addition that is punctuation only.
The server sends a transcript only when something changes. During silence nothing arrives; use ping to check that the connection is alive.
For long sessions, set "transcript_text": "incremental" in start so messages stay small. Each transcript then carries an edit instead of text: remove text_delete characters from the end of the text you have, then append text_append.
Other messages:
5. Ending turns
Choose one of two ways. Both can be used in the same session.You decide (default)
Send{"type": "finalize"} when your own voice activity or turn detector says the user stopped speaking. The final transcript for the turn comes back right away.
The model does not finalize on pauses by itself in this mode. If you never send finalize, the only final comes at finish, while the text keeps settling as you go (final_chars grows).
End-of-turn detection
Let the model decide when the speaker has finished. Addend_of_turn to the start message:
Modes
A mode is a preset that sets every value. Start withbalanced.
Settings
Any value you give beside a mode overrides that one value. Without a mode, give your own.ready reports the settings the session runs with:
Changing settings during a session
Sendconfigure with the same fields. It applies from that moment and is answered with configured:
{"type": "configure", "end_of_turn": false} turns detection off; configured then carries "end_of_turn": false. For example, use max_accuracy while the user reads out a card number, then go back to balanced.
finalize still works with detection on: it returns a final at once, with "end_of_turn": false and "reason": "manual".
6. Measuring latency
The figure that matters for a voice agent is the time from the end of the user’s speech to the final transcript.- You end turns: measure from the moment you send
finalizeto the firstis_final: truetranscript after it. Stop the clock there. If you then sendfinish, the same final comes back once more beforefinished; a clock that stops at the last final of the session counts that repeat and an extra network round trip, not the model. - The model ends turns: measure from the end of speech in your audio to the final that carries
"end_of_turn": true.
Pricing and limits
Your account’s own numbers are in the console and in
GET /v1/account (models["aircaps-live-1"]). For more sessions at once or volume pricing, talk to sales.
Errors
Every session that ends abnormally sends anerror message with a code, then closes with the matching WebSocket close code.
After a dropped connection, look up the address again (the answer may be a different server) and start a new session. Audio and text do not carry over. A session that ends early counts only the audio you streamed.
Python example
websockets older than version 14, use extra_headers= in place of additional_headers=.
To let the model end turns instead, add "end_of_turn": True, "end_of_turn_mode": "balanced" to the start message and drop the finalize line: each turn’s final arrives by itself with "end_of_turn": true.
Questions or odd transcripts: contact support with the time, your account email and, if you can, the audio.
