> ## Documentation Index
> Fetch the complete documentation index at: https://aircaps.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# LiveKit

> Use AirCaps Live 1 as the speech-to-text of a LiveKit Agents voice agent with the livekit-plugins-aircaps plugin.

[LiveKit Agents](https://docs.livekit.io/agents/) is an open-source framework for real-time voice agents that join LiveKit rooms. `livekit-plugins-aircaps` adds [AirCaps Live 1](https://aircaps.ai/docs/guides/live-1.md) to it as a speech-to-text plugin.

`aircaps.STT` streams the user's audio to AirCaps Live 1, sends interim transcripts while the user speaks, and returns the turn's final transcript the moment the turn ends. Turns end in one of two ways:

* **LiveKit decides (default).** The session's own turn handling (its VAD or turn detector) ends the turn, and the final transcript comes back right away.
* **AirCaps decides.** With `end_of_turn=True` and `turn_detection="stt"`, the model detects the end of each turn itself.

| | |
| - | - |
| Package | [`livekit-plugins-aircaps`](https://pypi.org/project/livekit-plugins-aircaps/) · [source](https://github.com/aircaps/livekit-plugins-aircaps) |
| Plugin | `livekit.plugins.aircaps.STT` |
| Model | `aircaps-live-1`: English, 16 kHz, streaming recognition |
| Requires | `livekit-agents` 1.8.13 or later, Python 3.10 or later |

## Prerequisites

* An AirCaps API key: sign in to the [AirCaps console](https://console.aircaps.ai), open **API keys** and create one. AirCaps Live 1 is available to every account.
* A LiveKit project ([LiveKit Cloud](https://cloud.livekit.io) or your own server) with its URL, API key and API secret.

> [!NOTE]
>
> Each agent conversation holds one AirCaps Live 1 session. Accounts run **one session at a time by default**, so a second simultaneous call is refused. For more, [talk to sales](https://aircaps.ai/contact). See [Pricing and limits](https://aircaps.ai/docs/guides/live-1.md#pricing-and-limits).

## Installation

```bash theme={null}
pip install livekit-plugins-aircaps
```

or `uv add livekit-plugins-aircaps`.

## Authentication

The plugin reads your key from the `AIRCAPS_API_KEY` environment variable. You can also pass it as `aircaps.STT(api_key=...)`.

## Quickstart

**1. Install the packages**

The plugin, plus LiveKit Agents with Silero VAD:

```bash theme={null}
pip install livekit-plugins-aircaps "livekit-agents[silero]" python-dotenv
```

**2. Set your API keys**

Put them in a `.env` file beside your agent:

```bash .env theme={null}
AIRCAPS_API_KEY=aircaps_sk_...
LIVEKIT_URL=wss://your-project.livekit.cloud
LIVEKIT_API_KEY=...
LIVEKIT_API_SECRET=...
```

**3. Use AirCaps in the AgentSession**

```python theme={null}
from livekit.agents import AgentSession
from livekit.plugins import aircaps, silero

session = AgentSession(
    stt=aircaps.STT(),
    vad=silero.VAD.load(),
    # ... llm, tts, turn_handling, etc.
)
```

LiveKit flushes the stream at the end of each turn, and the final transcript comes back right away. Without a flush, words stay interim until the stream ends, so keep a VAD (or a turn detector) in this mode.

**4. Run and talk to it**

Save the complete agent below as `agent.py`, then:

```bash theme={null}
python agent.py console
```

`console` runs the agent in your terminal with your microphone and speakers. `python agent.py dev` connects it to your LiveKit project instead, so you can join it from a room. Ask a question, interrupt the answer, and pause mid-sentence to hear how turns end.

> [accordion] Complete agent: agent.py
>
> ```python agent.py theme={null}
> import os
>
> from dotenv import load_dotenv
>
> from livekit.agents import (
>     Agent,
>     AgentServer,
>     AgentSession,
>     JobContext,
>     TurnHandlingOptions,
>     cli,
>     inference,
> )
> from livekit.plugins import aircaps, silero
>
> load_dotenv()
>
> END_OF_TURN = os.environ.get("AIRCAPS_END_OF_TURN") == "1"
>
>
> class Assistant(Agent):
>     def __init__(self) -> None:
>         super().__init__(
>             instructions="You are a helpful assistant in a voice conversation. Keep your answers short, "
>             "and do not use emojis, markdown or other formatting that cannot be spoken."
>         )
>
>     async def on_enter(self) -> None:
>         self.session.generate_reply(instructions="Greet the user and offer your help.")
>
>
> server = AgentServer()
>
>
> @server.rtc_session()
> async def entrypoint(ctx: JobContext) -> None:
>     session = AgentSession(
>         stt=aircaps.STT(end_of_turn=END_OF_TURN),
>         llm=inference.LLM("openai/gpt-4.1-mini"),
>         tts=inference.TTS("cartesia/sonic-3", voice="9626c31c-bec5-4cca-baa8-f8ba9e84c8bc"),
>         vad=silero.VAD.load(),
>         # With AirCaps end of turn, the model's decision already includes the user's pause:
>         # LiveKit adds no delay of its own.
>         turn_handling=TurnHandlingOptions(turn_detection="stt", endpointing={"min_delay": 0})
>         if END_OF_TURN
>         else TurnHandlingOptions(turn_detection="vad"),
>     )
>     await session.start(agent=Assistant(), room=ctx.room)
>
>
> if __name__ == "__main__":
>     cli.run_app(server)
> ```

This is LiveKit's basic voice agent with AirCaps as its speech-to-text; the LLM and the voice come from LiveKit Inference, so swap in any LLM and TTS plugin you use. Set `AIRCAPS_END_OF_TURN=1` to switch it to model-detected end of turn. The same file is in the plugin's repository as [`examples/agent.py`](https://github.com/aircaps/livekit-plugins-aircaps/blob/main/examples/agent.py).

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `model` | `str` | `"aircaps-live-1"` | The AirCaps model |
| `end_of_turn` | `bool` | `False` | Let the model detect the end of each turn. See [Turn detection](#turn-detection) |
| `end_of_turn_mode` | `str` | `"balanced"` | `"min_latency"`, `"balanced"` or `"max_accuracy"`. A mode sets the four values below; any of them given beside it overrides that one |
| `end_of_turn_threshold` | `float` | the mode's | How sure the model must be that the turn ended, 0.2 to 0.9 |
| `min_turn_silence_ms` | `int` | the mode's | The shortest pause that can end a turn, 0 to 1000 ms |
| `max_turn_silence_ms` | `int` | the mode's | A pause this long always ends the turn: 500 to 5000 ms, or 0 for no limit |
| `vad_threshold` | `float` | the model's | How sure the model's voice detector must be that a sound is speech, 0.3 to 0.9 |
| `api_key` | `str` | `AIRCAPS_API_KEY` | Your AirCaps API key |
| `base_url` | `str` | `"https://api.aircaps.com"` | The AirCaps API address |
| `http_session` | `aiohttp.ClientSession` | LiveKit's shared session | An HTTP session to use |

Audio is resampled to 16 kHz mono before it is sent.

## Turn detection

### LiveKit decides (default)

The session's turn handling ends each turn: its VAD (`turn_detection="vad"`), LiveKit's turn detector model, or a manual commit. LiveKit then flushes the stream, the plugin sends `finalize` to AirCaps, and the turn's final transcript comes straight back.

```python theme={null}
from livekit.agents import AgentSession, TurnHandlingOptions
from livekit.plugins import aircaps, silero

session = AgentSession(
    stt=aircaps.STT(),
    vad=silero.VAD.load(),
    turn_handling=TurnHandlingOptions(turn_detection="vad"),
    # ... llm, tts
)
```

Use this mode when you already tune turns in LiveKit, or want the same turn behaviour across STT providers.

### AirCaps decides

Turn on `end_of_turn` and tell the session to follow the STT:

```python theme={null}
from livekit.agents import AgentSession, TurnHandlingOptions
from livekit.plugins import aircaps, silero

session = AgentSession(
    stt=aircaps.STT(end_of_turn=True, end_of_turn_mode="balanced"),
    vad=silero.VAD.load(),
    turn_handling=TurnHandlingOptions(turn_detection="stt", endpointing={"min_delay": 0}),
    # ... llm, tts
)
```

The model decides when the user has finished, from what was said as well as from the pause.

> [!WARNING]
>
> Set `endpointing={"min_delay": 0}` in this mode. The model's end of turn has already waited out the user's pause, and LiveKit's default adds another 0.5 seconds to every turn.

Keep the VAD: LiveKit still uses it to detect interruptions while the agent speaks.

### End-of-turn settings

A mode is a preset. Start with `balanced`.

| Mode | Use it when |
| - | - |
| `min_latency` | The fastest reply matters most. Ends turns on short pauses, so it can cut in while someone is thinking |
| `balanced` (default) | Most voice agents |
| `max_accuracy` | Interrupting the speaker is costly (long answers, numbers, addresses). Waits longer to be sure |

Any setting given beside a mode overrides that one value.

| Setting | Values | Meaning | `min_latency` | `balanced` | `max_accuracy` |
| - | - | - | - | - | - |
| `end_of_turn_threshold` | 0.2 to 0.9 | How sure the model must be that the turn is complete. Higher waits for more certainty | 0.3 | 0.5 | 0.6 |
| `min_turn_silence_ms` | 0 to 1000 | The shortest pause that can end a turn | 40 | 80 | 400 |
| `max_turn_silence_ms` | 0, or 500 to 5000 | The pause that always ends a turn. 0: no such cap | 2560 | 2560 | 2560 |
| `vad_threshold` | 0.3 to 0.9 | How sure the model's voice detector must be that it hears speech. Raise it in noisy rooms | the same in every mode | | |

```python theme={null}
stt = aircaps.STT(end_of_turn=True, end_of_turn_mode="balanced", min_turn_silence_ms=300)
```

### Changing settings during a call

`update_options` changes the end-of-turn settings for new streams and for the ones already open:

```python theme={null}
stt.update_options(end_of_turn_mode="max_accuracy")
```

For example, switch to `max_accuracy` while the user reads out a card number, then back to `balanced`.

## How it works

* Before each session the plugin asks AirCaps where to connect ([`GET /v1/realtime/endpoint`](https://aircaps.ai/docs/api-reference/realtime-endpoint)), then opens that WebSocket, so audio goes to the server with the lowest latency for your agent.
* A flush sends `finalize`; the turn's final transcript comes straight back.
* A dropped or ended session (sessions last up to an hour) is reopened, with a new address lookup. A turn cut short by the drop ends with its text so far.

Not supported by AirCaps Live 1: languages other than English, key terms and speaker labels. The model supports streaming recognition only.

## Troubleshooting

| Issue | Cause | What to do |
| - | - | - |
| Replies feel about half a second late with `end_of_turn=True` | LiveKit's own endpointing delay is added to the model's | Set `endpointing={"min_delay": 0}` in `TurnHandlingOptions` |
| The agent never answers; only interim transcripts appear | Nothing ends the turn: no VAD or turn detector, and `end_of_turn` is off | Add `vad=silero.VAD.load()`, or use `end_of_turn=True` with `turn_detection="stt"` |
| `end_of_turn=True` seems to do nothing | The session is not following the STT | Set `turn_detection="stt"` in `TurnHandlingOptions` |
| `quota_unavailable` when a second call starts | The account's AirCaps Live 1 sessions are all in use (1 at a time by default) | End the other session, or [talk to sales](https://aircaps.ai/contact) for more |
| `unauthorized`, or the connection closes at once | The API key is missing or wrong | Check `AIRCAPS_API_KEY`; create a key in the [console](https://console.aircaps.ai) |
| `insufficient_balance` | Free hours and credits are used up | Add credits in the console |
| The agent answers before the user has finished | Turns end on short pauses | Use `max_accuracy`, or raise `min_turn_silence_ms` |
| Connections fail from a locked-down network | Outbound traffic is restricted | Allow HTTPS and secure WebSockets on port 443 to `api.aircaps.com` and `*.aircaps.ai` |

The error codes are those of the [AirCaps Live 1 protocol](https://aircaps.ai/docs/guides/live-1.md#errors).

## Resources

* [`livekit-plugins-aircaps` on PyPI](https://pypi.org/project/livekit-plugins-aircaps/) and [on GitHub](https://github.com/aircaps/livekit-plugins-aircaps): source, changelog and the example agent
* [AirCaps Live 1 guide](https://aircaps.ai/docs/guides/live-1.md): the model, its protocol, pricing and limits
* [LiveKit Agents documentation](https://docs.livekit.io/agents/): sessions, turn handling and deployment
* [Pipecat integration](https://aircaps.ai/docs/integrations/pipecat.md): the same model in Pipecat
* Problems: open an [issue](https://github.com/aircaps/livekit-plugins-aircaps/issues) or [contact support](https://console.aircaps.ai/support)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.