> ## Documentation Index
> Fetch the complete documentation index at: https://aircaps.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# AirCaps Live 1

> Low-latency real-time English speech-to-text for voice agents, with end-of-turn detection, over WebSocket.

AirCaps Live 1 is a low-latency, real-time English speech-to-text model for voice agents. You stream audio over a WebSocket and get the transcript back as the person speaks, with final text the moment a turn ends.

| | |
| - | - |
| Model id | `aircaps-live-1` |
| Transport | WebSocket |
| Audio in | Raw PCM, 16 kHz, mono, signed 16-bit little-endian |
| Language | English |
| Turn endings | You signal them (`finalize`), or the model detects them (`end_of_turn`) |
| Not included | Key terms, speaker labels, speaker names |
| Price | \$0.20 per audio hour, after your free hours |
| Sessions at once | 1 by default |
| Session length | Up to 1 hour |

Every account can use it. Try it without code in the [console playground](https://console.aircaps.ai/playground) (the **Live 1** tab).

Building on a voice agent framework? Use the ready-made packages instead of writing a client: [Pipecat](https://aircaps.ai/docs/integrations/pipecat.md) (`pipecat-aircaps`) and [LiveKit Agents](https://aircaps.ai/docs/integrations/livekit.md) (`livekit-plugins-aircaps`). The rest of this page is the protocol they use.

For captions of meetings and conversations with speaker labels, in nine languages, see [A5S Streaming v1](https://aircaps.ai/docs/guides/streaming.md) instead.

## Session flow

**1. Look up the WebSocket address**

Ask the API where to connect, once before every session. See [Get the WebSocket address](#1-get-the-websocket-address).

**2. Connect and send start**

Open the address with your API key in the handshake header, then send a `start` message that names the model.

**3. Wait for ready**

Status messages can arrive first. Do not send audio before `ready`.

**4. Stream audio and read transcripts**

Send 20 ms PCM frames in real time. Read `transcript` messages while you send.

**5. End turns**

Send `finalize` when the speaker stops, or let the model detect it with `end_of_turn`.

**6. Finish**

Send `finish`. A last final and `finished` follow, then the socket closes.

## 1. Get the WebSocket address

```bash theme={null}
curl --fail-with-body \
  -H "Authorization: Bearer $AIRCAPS_API_KEY" \
  'https://api.aircaps.com/v1/realtime/endpoint?model=aircaps-live-1'
```

```json theme={null}
{
  "websocket_url": "wss://rt1.aircaps.ai/v1/realtime",
  "model": "aircaps-live-1"
}
```

Open `websocket_url` exactly as returned, with the same `Authorization: Bearer aircaps_sk_...` header.

**Look the address up once before every session.** The answer is the server that gives your client the lowest latency among those that can take a session right now:

* The host can differ from one session to the next, and from one of your machines to another. Do not hard-code it, cache it across sessions, or share one answer between machines in different places.
* When a server is taken out of service, the lookup stops handing it out at once. A client that looks the address up moves to another server on its next session with nothing to change.
* As servers are added in more regions, your clients are sent to the nearest one without a change on your side.

The lookup takes a few tens of milliseconds and is not part of a session's latency. If it fails (a network error or a `5xx`), retry it once or twice; only if it still fails, connect to the last address you were given.

If your network restricts outbound connections, allow HTTPS and secure WebSockets on port 443 to `api.aircaps.com` and to `*.aircaps.ai`.

> [!NOTE]
>
> `wss://api.aircaps.com/v1/realtime` also accepts AirCaps Live 1 sessions, with the same messages. The looked-up address has lower latency, so use it for voice agents.

Keep your API key on your servers. Never put it in browser or mobile client code.

## 2. Start a session

Send this as the first message, within 45 seconds of connecting:

```json theme={null}
{
  "type": "start",
  "model": "aircaps-live-1",
  "encoding": "pcm_s16le",
  "sample_rate": 16000,
  "channels": 1
}
```

Then read messages until `ready`. `provider_status` messages can arrive before it; ignore them. An `error` message means the session was refused; the socket closes after it.

```json theme={null}
{"type": "ready", "model": "aircaps-live-1", "sample_rate": 16000, "encoding": "pcm_s16le", "session_limit_ms": 3600000, ...}
```

`start` accepts only the documented fields. Key terms (`custom_vocabulary`), speaker labels (`diarization`) and speaker names (`speaker_identification`) are not part of this model: a `start` that asks for them is refused with `invalid_start`.

## 3. Stream audio

Send audio as binary frames of raw PCM (16 kHz, mono, 16-bit little-endian), at real-time pace. A 20 ms frame is 320 samples, 640 bytes. WAV headers, MP3 and other containers are not accepted: decode, resample and downmix first.

| You send | Meaning |
| - | - |
| binary frame | Audio |
| `{"type": "finalize"}` | The speaker's turn has ended: return the final transcript now. `{"type": "commit"}` is the same |
| `{"type": "configure", ...}` | Change [end-of-turn settings](#end-of-turn-detection) during the session |
| `{"type": "ping"}` | Keep-alive; answered with `{"type": "pong"}` |
| `{"type": "finish"}` | End the session. A last final follows, then `finished` |

Read from the socket while you send; do not wait until the audio is finished.

`finalize` and `finish` do different jobs. `finalize` asks for the transcript and keeps the session open. `finish` closes the session. A `finish` sent after a `finalize` with no new audio in between returns the same final transcript a second time: it adds nothing, and it arrives one extra network round trip later. Take your transcript, and your timing, from the final that answers `finalize`.

## 4. Read transcripts

```json theme={null}
{"type": "transcript", "text": "Hi, I would like to book a table", "final_chars": 14, "is_final": false}
```

| Field | Meaning |
| - | - |
| `text` | The whole transcript of the session so far. Replace what you show with it; do not append |
| `final_chars` | The first `final_chars` characters of `text` will never change. The rest may still be revised |
| `is_final` | `true` after a `finalize`, `commit`, `finish` or a detected end of turn: all of `text` is final |

Text becomes final in two ways. While audio streams, the model settles earlier words on its own and `final_chars` grows. And when a turn ends, everything up to that point becomes final at once.

**Punctuation.** A final ends with its closing punctuation (`.`, `?` or `!`), and the text after it starts with a capital letter, so finals read as complete sentences when you join them. Sometimes the model settles on the closing mark just after it has sent the final. The mark is then added to the end of the final text, and `final_chars` covers it. That is the only addition ever made to text that is already final. If you take each turn's text as what was added since the previous final, ignore an addition that is punctuation only.

The server sends a `transcript` only when something changes. During silence nothing arrives; use `ping` to check that the connection is alive.

For long sessions, set `"transcript_text": "incremental"` in `start` so messages stay small. Each `transcript` then carries an edit instead of `text`: remove `text_delete` characters from the end of the text you have, then append `text_append`.

Other messages:

| You receive | Meaning |
| - | - |
| `provider_status` | Startup progress, before `ready` |
| `ready` | Send audio now |
| `configured` | The answer to `configure`: the end-of-turn settings now in force |
| `session_limit` | The session length or your usage was reached; a last final and `finished` follow |
| `finished` | Session over: `audio_ms` streamed and `billed_ms` counted toward usage |
| `error` | `code` and `message`; the session was refused or ended, and the socket then closes |
| `pong` | Reply to `ping` |

## 5. Ending turns

Choose one of two ways. Both can be used in the same session.

### You decide (default)

Send `{"type": "finalize"}` when your own voice activity or turn detector says the user stopped speaking. The final transcript for the turn comes back right away.

The model does not finalize on pauses by itself in this mode. If you never send `finalize`, the only final comes at `finish`, while the text keeps settling as you go (`final_chars` grows).

### End-of-turn detection

Let the model decide when the speaker has finished. Add `end_of_turn` to the `start` message:

```json theme={null}
{
  "type": "start",
  "model": "aircaps-live-1",
  "encoding": "pcm_s16le",
  "sample_rate": 16000,
  "channels": 1,
  "end_of_turn": true,
  "end_of_turn_mode": "balanced"
}
```

The model then ends a turn when the speaker pauses and what was said sounds complete, or when the pause lasts long enough that it must be over. Finals arrive without being asked:

```json theme={null}
{"type": "transcript", "text": "The quick brown fox jumps over the lazy dog.", "is_final": true, "final_chars": 44,
 "end_of_turn": true, "reason": "model", "end_of_turn_confidence": 0.802}
```

| Field | Meaning |
| - | - |
| `end_of_turn` | `true` when the model ended the turn here. `false` on a final you asked for with `finalize` or `finish` |
| `reason` | `model`: the speaker paused and the turn sounded complete. `max_silence`: the pause reached `max_turn_silence_ms`. `manual`: you asked for the final |
| `end_of_turn_confidence` | How sure the model was that the turn had ended, 0 to 1. Present when the model ended the turn |

#### Modes

A mode is a preset that sets every value. Start with `balanced`.

| `end_of_turn_mode` | Use it when | `end_of_turn_threshold` | `min_turn_silence_ms` | `max_turn_silence_ms` |
| - | - | - | - | - |
| `min_latency` | The fastest reply matters most. Ends turns on short pauses, so it can cut in while someone is thinking | 0.3 | 40 | 2560 |
| `balanced` | Most voice agents | 0.5 | 80 | 2560 |
| `max_accuracy` | Interrupting the speaker is costly (long answers, numbers, addresses). Waits longer to be sure | 0.6 | 400 | 2560 |

#### Settings

Any value you give beside a mode overrides that one value. Without a mode, give your own.

| Setting | Values | Meaning |
| - | - | - |
| `end_of_turn` | `true` or `false` | Turns detection on. Off by default; the other settings do nothing without it |
| `end_of_turn_mode` | `min_latency`, `balanced`, `max_accuracy` | The preset |
| `end_of_turn_threshold` | 0.2 to 0.9 | How sure the model must be that the turn is complete. Higher waits for more certainty |
| `min_turn_silence_ms` | 0 to 1000 | The shortest pause that can end a turn |
| `max_turn_silence_ms` | 0, or 500 to 5000 | The pause that always ends a turn, however incomplete it sounds. 0: no such cap |
| `vad_threshold` | 0.3 to 0.9 | How sure the voice detector must be that it hears speech. Raise it in noisy rooms |

`ready` reports the settings the session runs with:

```json theme={null}
{"type": "ready", "model": "aircaps-live-1", ...,
 "end_of_turn": {"mode": "balanced", "threshold": 0.5, "min_turn_silence_ms": 80, "max_turn_silence_ms": 2560, "vad_threshold": 0.65}}
```

#### Changing settings during a session

Send `configure` with the same fields. It applies from that moment and is answered with `configured`:

```json theme={null}
{"type": "configure", "end_of_turn_mode": "max_accuracy"}
```

```json theme={null}
{"type": "configured", "end_of_turn": {"mode": "max_accuracy", "threshold": 0.6, "min_turn_silence_ms": 400, "max_turn_silence_ms": 2560, "vad_threshold": 0.65}}
```

`{"type": "configure", "end_of_turn": false}` turns detection off; `configured` then carries `"end_of_turn": false`. For example, use `max_accuracy` while the user reads out a card number, then go back to `balanced`.

`finalize` still works with detection on: it returns a final at once, with `"end_of_turn": false` and `"reason": "manual"`.

## 6. Measuring latency

The figure that matters for a voice agent is the time from the end of the user's speech to the final transcript.

* **You end turns:** measure from the moment you send `finalize` to the first `is_final: true` transcript after it. Stop the clock there. If you then send `finish`, the same final comes back once more before `finished`; a clock that stops at the last final of the session counts that repeat and an extra network round trip, not the model.
* **The model ends turns:** measure from the end of speech in your audio to the final that carries `"end_of_turn": true`.

Send audio at real-time pace when you measure. Audio sent faster than real time is refused, and a session that is paced unevenly measures your client, not the model. The network round trip between your client and the address from step 1 is part of every figure, so measure it separately if you want the model's share.

## Pricing and limits

| Limit | Default |
| - | - |
| Price | \$0.20 per audio hour, billed per millisecond of audio streamed, after your 5 free hours (shared by all models) |
| Sessions at the same time | 1. A second one is refused with `quota_unavailable` (close code 4409); retry when the first ends |
| Session length | 1 hour (`session_limit`); open a new session to continue |
| Start message | Within 45 seconds of connecting |
| Idle | No message (audio or `ping`) for 45 seconds ends the session with `idle_timeout` |
| Pace | Real time; sending faster (beyond a 2-second burst) ends the session with `realtime_rate_exceeded` |
| Audio frame | At most 64 KB |

Your account's own numbers are in the console and in [`GET /v1/account`](https://aircaps.ai/docs/api-reference/account) (`models["aircaps-live-1"]`). For more sessions at once or volume pricing, [talk to sales](https://aircaps.ai/contact).

## Errors

Every session that ends abnormally sends an `error` message with a `code`, then closes with the matching WebSocket close code.

| What you see | Close code | What to do |
| - | - | - |
| HTTP 401 on the lookup, or `unauthorized` | 4401 | The key is missing or wrong. Send `Authorization: Bearer aircaps_sk_...` on both the lookup and the WebSocket |
| `unsupported_model` | 4400 | Check the model id: `aircaps-live-1`. An account limited to specific models that leaves this one out also gets it |
| `invalid_start` | 4400 | A field in `start` is not accepted; the message names it. Do not send key terms or speaker options |
| `invalid_message` | 4400 | A text message that isn't `finalize`, `commit`, `configure`, `ping` or `finish`, or a `configure` with a value out of range |
| `audio_before_ready`, `invalid_audio_frame` | 4400 | Wait for `ready`; send non-empty frames with an even number of bytes |
| `idle_timeout` | 4408 | Send audio or `ping` at least every 45 seconds |
| `quota_unavailable` | 4409 | A session is already running on this account (1 at a time by default), or its usage limit is used up |
| `insufficient_balance` | 4409 | Free hours and credits are used up: add credits in the console |
| `realtime_rate_exceeded`, `frame_too_large` | 4409 | Send audio in real time, in frames of at most 64 KB |
| `streaming_unavailable`, `upstream_unavailable` | 1013 | Temporarily unavailable, for example at capacity: retry with a bounded number of attempts and a growing wait |
| `internal_error` | 1011 | Server error: retry |

After a dropped connection, look up the address again (the answer may be a different server) and start a new session. Audio and text do not carry over. A session that ends early counts only the audio you streamed.

## Python example

```bash theme={null}
pip install requests websockets soundfile
```

```python theme={null}
import asyncio, json, os, time
import requests, soundfile as sf, websockets

KEY = os.environ["AIRCAPS_API_KEY"]
AUTH = {"Authorization": f"Bearer {KEY}"}


def endpoint():
    r = requests.get("https://api.aircaps.com/v1/realtime/endpoint",
                     params={"model": "aircaps-live-1"}, headers=AUTH, timeout=30)
    r.raise_for_status()
    return r.json()["websocket_url"]


async def main(path):
    pcm, sr = sf.read(path, dtype="int16")        # a 16 kHz mono WAV file
    assert sr == 16000 and pcm.ndim == 1
    async with websockets.connect(endpoint(), additional_headers=AUTH) as ws:
        await ws.send(json.dumps({"type": "start", "model": "aircaps-live-1", "encoding": "pcm_s16le",
                                  "sample_rate": 16000, "channels": 1}))
        while True:                               # status messages may come before "ready"
            m = json.loads(await ws.recv())
            if m["type"] == "ready":
                break
            if m["type"] == "error":
                raise RuntimeError(m)

        async def reader():
            async for raw in ws:
                m = json.loads(raw)
                if m["type"] == "transcript":
                    print("FINAL  " if m["is_final"] else "interim", m["text"])
                elif m["type"] in ("finished", "error"):
                    print(m)
                    return

        task = asyncio.create_task(reader())
        t0 = time.perf_counter()
        for i in range(0, len(pcm), 320):         # 20 ms frames, paced in real time
            await ws.send(pcm[i:i + 320].tobytes())
            await asyncio.sleep(max(0, t0 + (i + 320) / 16000 - time.perf_counter()))
        await ws.send(json.dumps({"type": "finalize"}))   # the turn ended: final transcript now
        await asyncio.sleep(1)
        await ws.send(json.dumps({"type": "finish"}))     # only closes the session: it repeats that final
        await task

asyncio.run(main("sample_16k_mono.wav"))
```

With `websockets` older than version 14, use `extra_headers=` in place of `additional_headers=`.

To let the model end turns instead, add `"end_of_turn": True, "end_of_turn_mode": "balanced"` to the `start` message and drop the `finalize` line: each turn's final arrives by itself with `"end_of_turn": true`.

Questions or odd transcripts: [contact support](https://console.aircaps.ai/support) with the time, your account email and, if you can, the audio.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.