> ## Documentation Index
> Fetch the complete documentation index at: https://aircaps.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Pipecat

> Use AirCaps Live 1 as the speech-to-text of a Pipecat voice agent with the pipecat-aircaps package.

[Pipecat](https://docs.pipecat.ai) is an open-source Python framework for real-time voice agents: a pipeline of a transport, speech-to-text, an LLM and text-to-speech. `pipecat-aircaps` adds [AirCaps Live 1](https://aircaps.ai/docs/guides/live-1.md) to it as a speech-to-text service.

`AirCapsLiveSTTService` streams the user's audio to AirCaps Live 1, pushes interim transcripts while the user speaks, and returns the turn's final transcript the moment the turn ends. Turns end in one of two ways:

* **Pipecat decides (default).** When Pipecat's VAD reports the end of speech, the service asks AirCaps for the turn's final transcript at once.
* **AirCaps decides.** With `end_of_turn=True` the model detects the end of each turn itself.

| | |
| - | - |
| Package | [`pipecat-aircaps`](https://pypi.org/project/pipecat-aircaps/) · [source](https://github.com/aircaps/pipecat-aircaps) |
| Service | `pipecat_aircaps.AirCapsLiveSTTService` |
| Model | `aircaps-live-1`: English, 16 kHz |
| Requires | `pipecat-ai` 1.16.0 or later, Python 3.11 or later |

## Prerequisites

* An AirCaps API key: sign in to the [AirCaps console](https://console.aircaps.ai), open **API keys** and create one. AirCaps Live 1 is available to every account.
* Keys for the rest of your pipeline. The example below uses OpenAI for the LLM and Cartesia for the voice; any Pipecat LLM and TTS service works.

> [!NOTE]
>
> Each agent conversation holds one AirCaps Live 1 session. Accounts run **one session at a time by default**, so a second simultaneous call is refused. For more, [talk to sales](https://aircaps.ai/contact). See [Pricing and limits](https://aircaps.ai/docs/guides/live-1.md#pricing-and-limits).

## Installation

```bash theme={null}
pip install pipecat-aircaps
```

or `uv add pipecat-aircaps`.

## Quickstart

**1. Install the packages**

The service itself, plus the Pipecat extras this example uses (a WebRTC transport, Silero VAD, OpenAI and Cartesia):

```bash theme={null}
pip install pipecat-aircaps "pipecat-ai[cartesia,openai,runner,silero,webrtc]" python-dotenv
```

**2. Set your API keys**

Put them in a `.env` file beside your bot:

```bash .env theme={null}
AIRCAPS_API_KEY=aircaps_sk_...
OPENAI_API_KEY=...
CARTESIA_API_KEY=...
```

**3. Add AirCaps to the pipeline**

Create the service and put it where any Pipecat STT service goes, right after the transport's input:

```python theme={null}
import os

from pipecat_aircaps import AirCapsLiveSTTService

stt = AirCapsLiveSTTService(api_key=os.environ["AIRCAPS_API_KEY"])

pipeline = Pipeline(
    [
        transport.input(),
        stt,
        user_aggregator,
        llm,
        tts,
        transport.output(),
        assistant_aggregator,
    ]
)
```

The service needs VAD frames to know when a turn ends. The user aggregator provides them when it has a VAD analyzer:

```python theme={null}
user_aggregator, assistant_aggregator = LLMContextAggregatorPair(
    context,
    user_params=LLMUserAggregatorParams(vad_analyzer=SileroVADAnalyzer()),
)
```

**4. Run and talk to it**

Save the complete bot below as `bot.py`, then:

```bash theme={null}
python bot.py
```

Open [http://localhost:7860/client](http://localhost:7860/client) and speak. Ask a question, interrupt the answer, and pause mid-sentence to hear how turns end.

> [accordion] Complete bot: bot.py
>
> ```python bot.py theme={null}
> import os
>
> from dotenv import load_dotenv
> from loguru import logger
> from pipecat.audio.vad.silero import SileroVADAnalyzer
> from pipecat.evals.transport import EvalTransportParams
> from pipecat.frames.frames import LLMRunFrame
> from pipecat.pipeline.pipeline import Pipeline
> from pipecat.pipeline.worker import PipelineParams, PipelineWorker, ProcessorUnusablePolicy
> from pipecat.processors.aggregators.llm_context import LLMContext
> from pipecat.processors.aggregators.llm_response_universal import (
>     LLMContextAggregatorPair,
>     LLMUserAggregatorParams,
> )
> from pipecat.runner.types import RunnerArguments
> from pipecat.runner.utils import create_transport
> from pipecat.services.cartesia.tts import CartesiaTTSService
> from pipecat.services.openai.llm import OpenAILLMService
> from pipecat.transports.base_transport import BaseTransport, TransportParams
> from pipecat.workers.runner import WorkerRunner
>
> from pipecat_aircaps import AirCapsLiveSTTService
>
> load_dotenv(override=True)
>
> transport_params = {
>     "eval": lambda: EvalTransportParams(audio_in_enabled=True, audio_out_enabled=True),
>     "webrtc": lambda: TransportParams(audio_in_enabled=True, audio_out_enabled=True),
> }
>
>
> async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
>     logger.info("Starting bot")
>
>     stt = AirCapsLiveSTTService(api_key=os.environ["AIRCAPS_API_KEY"])
>
>     tts = CartesiaTTSService(
>         api_key=os.environ["CARTESIA_API_KEY"],
>         settings=CartesiaTTSService.Settings(voice="86e30c1d-714b-4074-a1f2-1cb6b552fb49"),
>     )
>
>     llm = OpenAILLMService(
>         api_key=os.environ["OPENAI_API_KEY"],
>         settings=OpenAILLMService.Settings(
>             system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
>         ),
>     )
>
>     context = LLMContext()
>     user_aggregator, assistant_aggregator = LLMContextAggregatorPair(
>         context,
>         user_params=LLMUserAggregatorParams(vad_analyzer=SileroVADAnalyzer()),
>     )
>
>     pipeline = Pipeline(
>         [
>             transport.input(),
>             stt,
>             user_aggregator,
>             llm,
>             tts,
>             transport.output(),
>             assistant_aggregator,
>         ]
>     )
>
>     worker = PipelineWorker(
>         pipeline,
>         params=PipelineParams(enable_metrics=True, enable_usage_metrics=True),
>         idle_timeout_secs=runner_args.pipeline_idle_timeout_secs,
>         processor_unusable_policy=ProcessorUnusablePolicy.END,
>     )
>
>     runner = WorkerRunner(handle_sigint=runner_args.handle_sigint)
>     await runner.add_workers(worker)
>
>     @transport.event_handler("on_client_connected")
>     async def on_client_connected(transport, client):
>         logger.info("Client connected")
>         context.add_message(
>             {"role": "developer", "content": "Please introduce yourself to the user."}
>         )
>         await worker.queue_frames([LLMRunFrame()])
>
>     @transport.event_handler("on_client_disconnected")
>     async def on_client_disconnected(transport, client):
>         logger.info("Client disconnected")
>         await runner.cancel()
>
>     await runner.run()
>
>
> async def bot(runner_args: RunnerArguments):
>     """Main bot entry point compatible with Pipecat Cloud."""
>     transport = await create_transport(runner_args, transport_params)
>     await run_bot(transport, runner_args)
>
>
> if __name__ == "__main__":
>     from pipecat.runner.run import main
>
>     main()
> ```

This is Pipecat's standard voice agent with AirCaps as its speech-to-text. The same file is in the package's repository as [`examples/voice-agent.py`](https://github.com/aircaps/pipecat-aircaps/blob/main/examples/voice-agent.py).

## Configuration

### Constructor arguments

| Argument | Type | Default | Description |
| - | - | - | - |
| `api_key` | `str` | required | Your AirCaps API key |
| `base_url` | `str` | `https://api.aircaps.com` | The AirCaps API address |
| `sample_rate` | `int` | the pipeline's input rate | Input audio rate. Audio is resampled to 16 kHz before it is sent |
| `end_of_turn` | `bool` | `False` | Let the model detect the end of each turn. See [Turn detection](#turn-detection) |
| `should_interrupt` | `bool` | `True` | With `end_of_turn`: whether a new user turn interrupts the bot |
| `settings` | `AirCapsLiveSTTService.Settings` | see below | Model, language and end-of-turn settings |
| `ttfs_p99_latency` | `float` | `0.25` | P99 seconds from the end of speech to the final transcript, for Pipecat's turn timing. Measure your own with [stt-benchmark](https://github.com/pipecat-ai/stt-benchmark) |

### Settings

Pass them as `AirCapsLiveSTTService.Settings(...)`.

| Setting | Type | Default | Description |
| - | - | - | - |
| `model` | `str` | `"aircaps-live-1"` | The AirCaps model |
| `language` | `Language` | `Language.EN` | AirCaps Live 1 transcribes English |
| `end_of_turn_mode` | `str` | `"balanced"` | `"min_latency"`, `"balanced"` or `"max_accuracy"`. Used only with `end_of_turn=True` |
| `end_of_turn_threshold` | `float` | the mode's | Overrides one value of the mode. See [End-of-turn settings](#end-of-turn-settings) |
| `min_turn_silence_ms` | `int` | the mode's | Overrides one value of the mode |
| `max_turn_silence_ms` | `int` | the mode's | Overrides one value of the mode |
| `vad_threshold` | `float` | the model's | The model's own voice detector |

## Turn detection

### Pipecat decides (default)

Pipecat's VAD, turn analyzer and user turn strategies own the turn. When VAD reports that the user stopped speaking (`VADUserStoppedSpeakingFrame`), the service sends `finalize` to AirCaps, and the turn's final transcript comes back as one finalized `TranscriptionFrame`.

Use this mode when you already tune turns in Pipecat (Smart Turn, your own strategies), or want the same turn behaviour across STT providers. It needs a VAD analyzer on the user aggregator, as in the quickstart.

### AirCaps decides

```python theme={null}
stt = AirCapsLiveSTTService(
    api_key=os.environ["AIRCAPS_API_KEY"],
    end_of_turn=True,
    settings=AirCapsLiveSTTService.Settings(end_of_turn_mode="balanced"),
)
```

With `end_of_turn=True` the model decides when the user has finished, from what was said as well as from the pause, instead of Pipecat's VAD and turn analyzer:

* When a turn's first words arrive, the service pushes `ProposedUserStartedSpeakingFrame`.
* When the model ends the turn, the service pushes the turn's final `TranscriptionFrame` (finalized), then `ProposedUserStoppedSpeakingFrame`.
* The service recommends `ExternalUserTurnStrategies`, so the user aggregator follows these frames. You don't need a VAD analyzer on the user aggregator for turn ends. If you pass your own `user_turn_strategies`, they win.
* `should_interrupt=False` stops a new user turn from interrupting the bot.

### End-of-turn settings

A mode is a preset. Start with `balanced`.

| Mode | Use it when |
| - | - |
| `min_latency` | The fastest reply matters most. Ends turns on short pauses, so it can cut in while someone is thinking |
| `balanced` (default) | Most voice agents |
| `max_accuracy` | Interrupting the speaker is costly (long answers, numbers, addresses). Waits longer to be sure |

Any setting given beside a mode overrides that one value; leave the others unset to keep the mode's.

| Setting | Values | Meaning | `min_latency` | `balanced` | `max_accuracy` |
| - | - | - | - | - | - |
| `end_of_turn_threshold` | 0.2 to 0.9 | How sure the model must be that the turn is complete. Higher waits for more certainty | 0.3 | 0.5 | 0.6 |
| `min_turn_silence_ms` | 0 to 1000 | The shortest pause that can end a turn | 40 | 80 | 400 |
| `max_turn_silence_ms` | 0, or 500 to 5000 | The pause that always ends a turn. 0: no such cap | 2560 | 2560 | 2560 |
| `vad_threshold` | 0.3 to 0.9 | How sure the model's voice detector must be that it hears speech. Raise it in noisy rooms | the same in every mode | | |

```python theme={null}
settings = AirCapsLiveSTTService.Settings(end_of_turn_mode="balanced", min_turn_silence_ms=300)
```

All of them can be changed during a call with Pipecat's `STTUpdateSettingsFrame`; the AirCaps session reopens to apply them.

## How it works

* Before each connection the service asks AirCaps where to connect ([`GET /v1/realtime/endpoint`](https://aircaps.ai/docs/api-reference/realtime-endpoint)), then opens that WebSocket, so audio goes to the server with the lowest latency for your agent.
* Interim transcripts are pushed as `InterimTranscriptionFrame`s while the user speaks; each turn ends with one finalized `TranscriptionFrame`, and Pipecat reports STT TTFB from it.
* If words arrive that VAD did not hear (a short or quiet word), the service asks for their final itself one second after the last new words, so the user turn they started still ends.
* A session the server ends (sessions last up to an hour) is reopened automatically.
* While no audio is sent (for example when the STT is muted), the service pings the session so it stays open.

Not supported by AirCaps Live 1: languages other than English, key terms and speaker labels.

## Troubleshooting

| Issue | Cause | What to do |
| - | - | - |
| The bot never answers; only interim transcripts appear | No VAD frames reach the service, so no turn ever ends | Give the user aggregator a VAD analyzer (`LLMUserAggregatorParams(vad_analyzer=SileroVADAnalyzer())`), or use `end_of_turn=True` |
| `quota_unavailable` when a second call starts | The account's AirCaps Live 1 sessions are all in use (1 at a time by default) | End the other session, or [talk to sales](https://aircaps.ai/contact) for more |
| `unauthorized`, or the connection closes at once | The API key is missing or wrong | Check `AIRCAPS_API_KEY`; create a key in the [console](https://console.aircaps.ai) |
| `insufficient_balance` | Free hours and credits are used up | Add credits in the console |
| The agent answers before the user has finished | Turns end on short pauses | With `end_of_turn=True`: use `max_accuracy`, or raise `min_turn_silence_ms`. Otherwise tune Pipecat's VAD stop time or turn analyzer |
| The agent waits too long to answer | Turns wait for long pauses | With `end_of_turn=True`: use `min_latency`. Otherwise shorten Pipecat's VAD stop time |
| Connections fail from a locked-down network | Outbound traffic is restricted | Allow HTTPS and secure WebSockets on port 443 to `api.aircaps.com` and `*.aircaps.ai` |

The error codes are those of the [AirCaps Live 1 protocol](https://aircaps.ai/docs/guides/live-1.md#errors).

## Resources

* [`pipecat-aircaps` on PyPI](https://pypi.org/project/pipecat-aircaps/) and [on GitHub](https://github.com/aircaps/pipecat-aircaps): source, changelog and the example agent
* [AirCaps Live 1 guide](https://aircaps.ai/docs/guides/live-1.md): the model, its protocol, pricing and limits
* [Pipecat documentation](https://docs.pipecat.ai): transports, turn strategies and deployment
* [LiveKit integration](https://aircaps.ai/docs/integrations/livekit.md): the same model in LiveKit Agents
* Problems: open an [issue](https://github.com/aircaps/pipecat-aircaps/issues) or [contact support](https://console.aircaps.ai/support)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.