> ## Documentation Index
> Fetch the complete documentation index at: https://aircaps.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Realtime stream

> Live transcription with A5S Small V2 Streaming over a WebSocket.

**WSS** `wss://api.aircaps.com/v1/realtime`

Requires access to A5S Small V2 Streaming. For a walkthrough, see [Live streaming](https://aircaps.ai/docs/guides/streaming.md).

## Connection

**Authorization** `string` *required*

`Bearer aircaps_sk_...`, sent with the WebSocket handshake.

The flow: connect, send `start`, wait for `ready`, stream binary audio, send `finish`, then receive `finished`. The server then closes with `1000`.

## Client messages

### start

The first message, as a JSON text frame, within 45 seconds of connecting. Unknown fields are rejected.

**type** `string` *required*

`"start"`

**language** `string` (default: en)

Language of the audio: `en` (A5S Small V2 Streaming transcribes English).

**encoding** `string` (default: pcm\_s16le)

Only `pcm_s16le` (signed 16-bit little-endian PCM).

**sample\_rate** `integer` (default: 16000)

Only `16000`.

**channels** `integer` (default: 1)

Only `1` (mono).

**providers** `string[]` (default: \["a5sv2"])

The streaming engine. Only `"a5sv2"`.

**custom\_vocabulary\_enabled** `boolean` (default: false)

Turns on key terms: biases recognition toward the entries in `custom_vocabulary`.

**custom\_vocabulary** `object[]`

Key terms: up to 64. See [Key terms](https://aircaps.ai/docs/guides/streaming.md#key-terms).

**properties**

**text** `string` *required*

A word or phrase, up to 100 characters.

**strength** `number` *required*

Bias strength, 0 to 12.

**diarization** `boolean` (default: false)

Label who said what. Each `transcript` then also carries `utterances` with a speaker on every utterance and word, and `speakers`. See [Speaker labels](https://aircaps.ai/docs/guides/streaming.md#speaker-labels).

**transcript\_text** `string` (default: full)

`"full"`: every `transcript` carries the complete transcript so far in `text`. `"incremental"`: every `transcript` instead carries an edit of the text you have: `text_delete` and `text_append` (below). Use `"incremental"` for long sessions: in `"full"` mode each message grows with the transcript.

**speaker\_identification** `boolean | object` (default: false)

Put names on the speakers with AirCaps Speaker ID-v1 (requires access to that model, and `"diarization": true`). `true` uses the defaults. Sends a `speaker_identification` message (below) for each stretch of a speaker's speech, and adds `speaker_id` and `speaker_name` to utterances. See [Speaker names](https://aircaps.ai/docs/guides/streaming.md#speaker-names).

**properties**

**threshold** `number` (default: 0.6)

A speaker gets a name when their best match scores above this (0 to 1) and is clearly ahead of the next-best.

**speaker\_ids** `string[]`

Compare with these enrolled speakers only. Default: every enrolled speaker on the account.

```json Example theme={null}
{"type": "start", "language": "en", "encoding": "pcm_s16le", "sample_rate": 16000, "channels": 1,
 "diarization": true,
 "custom_vocabulary_enabled": true, "custom_vocabulary": [{"text": "AirCaps", "strength": 6}]}
```

### Audio

Binary frames of raw PCM in the `start` format: non-empty, an even number of bytes, at most 64 KB (larger frames end the session with `frame_too_large`; frames over 1 MB close the connection without a message). Send them after `ready`, in real time; a 2-second burst is allowed. 20 ms frames (640 bytes) are recommended.

### ping

`{"type": "ping"}`. The server answers `pong`. Any message, audio or `ping`, resets the 45-second idle timer.

### finish

`{"type": "finish"}`. The server flushes the final transcript, sends `finished`, and closes.

## Server messages

Each is a JSON text frame with a `type`.

### provider\_status

Startup progress while the session starts.

**message** `string`

For example `"Getting things ready, please hold on"`.

### ready

Start sending audio.

**session\_limit\_ms** `integer`

Maximum audio for this session: your session length (30 minutes unless your account has a longer one, up to 4 hours), or less if little free usage and credit remain.

**transcript\_text** `string`

`"full"` or `"incremental"`, as requested in `start`.

**quota\_ms** `integer`

The account's free usage, in milliseconds.

**used\_ms** `integer`

Free usage used so far, in milliseconds.

**language** `string`

`"en-US"`

**sample\_rate** `integer`

`16000`

**encoding** `string`

`"pcm_s16le"`

**diarization** `boolean`

Whether speaker labels are on for this session.

**speaker\_identification** `object`

Only with `speaker_identification`: `model` (`"aircaps-speaker-id-v1"`), `threshold`, `speaker_ids` (`null` for every enrolled speaker) and `speakers_compared`.

**warnings** `object[]`

Only when `start` named a deprecated model id (`a5s-v2-streaming`, now `a5s-small-v2-streaming`): `{code, message}` with `code` `"model_deprecated"`. The session runs normally.

### transcript

**text** `string`

With `transcript_text``"full"` (the default): the complete transcript so far, punctuated and cased. Replace what you display instead of appending.

**text\_delete** `integer`

With `transcript_text``"incremental"`: remove this many characters from the end of the text you have (recent words the engine revised; often 0).

**text\_append** `string`

With `"incremental"`: then append this. The result is the complete transcript so far, as `text` would have been.

**is\_final** `boolean`

`true` when the current segment is settled.

**provider** `string`

`"a5sv2"`

**utterances** `object[]`

Only with `diarization`. The utterances that became final since the previous `transcript` (`is_final: true`; each is sent once and never changes), followed by every pending utterance (`is_final: false`; these may still change, so replace the previous pending ones). Keep the final ones and show them with the latest pending ones, ordered by `start`.

**properties**

**speaker** `string | null`

`speaker_0`, `speaker_1`, … in order of first speech, as in async transcripts. `null` on a pending utterance before any speaker has been recognized.

**start** `number`

Seconds from the start of the session.

**end** `number`

Seconds from the start of the session.

**text** `string`

Punctuated and cased.

**is\_final** `boolean`

`true` once the utterance will not change.

**words** `object[]`

`{text, start, end, speaker}` for each word.

**speaker\_id** `string | null`

Only with `speaker_identification`. The enrolled speaker named by the `speaker_identification` decision for this `speaker` that overlaps the utterance most, as known when it is sent; `null` until then or when unnamed.

**speaker\_name** `string | null`

Only with `speaker_identification`. That speaker's name.

**speakers** `string[]`

Only with `diarization`. The speaker labels used so far in final utterances.

```json Example theme={null}
{"type": "transcript", "provider": "a5sv2", "text": "Good afternoon, everybody.", "is_final": true}
```

```json Example with diarization theme={null}
{"type": "transcript", "provider": "a5sv2", "text": "Shall we start? Sure, go ahead.", "is_final": false,
 "utterances": [
   {"speaker": "speaker_0", "start": 0.4, "end": 1.36, "text": "Shall we start?", "is_final": true,
    "words": [{"text": "Shall", "start": 0.4, "end": 0.64, "speaker": "speaker_0"}, "..."]},
   {"speaker": "speaker_1", "start": 1.92, "end": 2.88, "text": "Sure, go ahead.", "is_final": false,
    "words": ["..."]}],
 "speakers": ["speaker_0"]}
```

### speaker\_identification

Only with `speaker_identification`. One decision for a stretch of one speaker's speech, usually sent about a second after it ends, named or not.

**speaker** `string`

The speaker label, as in utterances.

**start** `number`

Start of the stretch, in seconds from the start of the session.

**end** `number`

End of the stretch, in seconds from the start of the session.

**speaker\_id** `string | null`

The enrolled speaker it was named as, or `null`.

**name** `string | null`

That speaker's name, or `null`.

**score** `number | null`

The best match's similarity score, -1 to 1.

**margin** `number | null`

How far the best score is above the next-best; `null` with one enrolled speaker.

**matches** `object[]`

Up to 3 closest enrolled speakers, best first: `{speaker_id, name, score}`.

**compared\_duration** `number`

Seconds of speech compared.

**reason** `string`

`accepted`, `score` (best score not above `threshold`), `margin` (too close to the next-best), or `error` (this stretch could not be compared; the session goes on).

```json Example theme={null}
{"type": "speaker_identification", "speaker": "speaker_0", "start": 12.48, "end": 13.29,
 "speaker_id": "spk_01k73a9x8w7v6u5t4s3r2q1p0n", "name": "Ana García", "score": 0.781, "margin": 0.512,
 "compared_duration": 4.81, "reason": "accepted",
 "matches": [{"speaker_id": "spk_01k73a9x8w7v6u5t4s3r2q1p0n", "name": "Ana García", "score": 0.781},
             {"speaker_id": "spk_01k73a7m6n5b4v3c2x1z0l9k8j", "name": "Ben Ito", "score": 0.269}]}
```

### session\_limit

The session reached its session length (30 minutes unless your account has a longer one) or the account's usage limit. The final transcript and `finished` follow, then close code 1000.

**message** `string`

Why the session ended.

### finished

**audio\_ms** `integer`

Audio received.

**billed\_ms** `integer`

Audio counted toward usage.

**used\_ms** `integer`

Free usage used after this session.

**quota\_ms** `integer`

The account's free usage.

### provider\_error

Streaming failed during the session. An `error` follows.

**message** `string`

What failed.

### error

Sent before the server ends the session abnormally; the connection then closes with the matching close code.

**code** `string`

One of the codes below.

**message** `string`

Human-readable detail.

| code | Close | Meaning |
| - | - | - |
| `invalid_start` | 4400 | The first message is missing, late or invalid (including `speaker_identification` without `diarization`) |
| `unsupported_language` | 4400 | `language` is not English |
| `unknown_provider` | 4400 | `providers` names an unknown engine |
| `speaker_not_found`, `speaker_not_enrolled`, `no_enrolled_speakers` | 4400 | `speaker_identification.speaker_ids` names an unknown speaker or one with no enrollments, or the account has no enrolled speakers |
| `audio_before_ready` | 4400 | Audio sent before `ready` |
| `invalid_audio_frame` | 4400 | An empty or odd-length audio frame |
| `invalid_message` | 4400 | A text message other than `ping` or `finish` |
| `unauthorized` | 4401 | Missing or invalid API key |
| `account_disabled` | 4403 | The account is disabled |
| `model_access_denied` | 4403 | No access to A5S Small V2 Streaming, or to AirCaps Speaker ID-v1 for `speaker_identification` ([talk to sales](https://aircaps.ai/contact)) |
| `origin_not_allowed` | 4403 | Browser origin not allowed |
| `idle_timeout` | 4408 | No messages for 45 seconds |
| `quota_unavailable` | 4409 | Usage limit used up, or too many live sessions ([talk to sales](https://aircaps.ai/contact)) |
| `insufficient_balance` | 4409 | Free usage and credits are used up: add credits in the console |
| `realtime_rate_exceeded` | 4409 | Audio sent faster than real time |
| `frame_too_large` | 4409 | An audio frame over 64 KB |
| `a5sv2_unavailable`, `upstream_unavailable` | 1013 | Temporarily unavailable, at start or during the session; open a new session with backoff |
| `diarization_unavailable` | 1013 | Speaker labels are temporarily unavailable; retry, or start without `diarization` |
| `speaker_identification_unavailable` | 1013 | Speaker names are temporarily unavailable; retry, or start without `speaker_identification` |
| `internal_error` | 1011 | Server error |

### pong

The reply to `ping`.

## Close codes

| Code | Meaning |
| - | - |
| 1000 | Normal close after `finished` |
| 1011 | Server error |
| 1013 | Temporarily unavailable; open a new session with backoff (for example 1 s, 2 s, 4 s … up to 30 s) |
| 4400 | Invalid message or frame |
| 4401 | Authentication failed |
| 4403 | Forbidden: account disabled, no access to the model, or browser origin not allowed |
| 4408 | Idle timeout |
| 4409 | Limit reached at start (usage, sessions), audio faster than real time, or a frame over 64 KB |


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.