Demonstrating live transcription using a local microphone

The live demo client is found in the inference/live_demo_client directory.

This script connects to a running CAIMAN-ASR server and streams audio from your microphone to the server for low-latency transcription.

Step 1: Set up the CAIMAN-ASR server

Follow the instructions provided in Inference Flow to set up the ASR server.

Step 2: Set up the client

Install PyAudio’s system dependencies and set up the client environment:

sudo apt install portaudio19-dev python3-dev # dependencies of PyAudio
cd inference/live_demo_client
uv sync --locked

This installs the dependencies listed in pyproject.toml at the versions pinned in uv.lock.

Then run uv run --locked live_client.py --host localhost --port 3030. Set the host and port for your server. If --prefix is omitted, the latest server uses the model’s declared default prefix.

Prefix (punctuation/casing and language)

Use --prefix to condition the RNN-T prediction network:

<pnc>: punctuated and cased output
<nopnc>: lowercase only (slightly more accurate)

The accepted prefixes depend on the model’s training. Choose the prefix supported by your model:

Models trained with punctuation tags only: use --prefix='<pnc>' or --prefix='<nopnc>', without a language tag.
Models trained with punctuation and language tags: use both, with punctuation first, e.g. --prefix='<pnc><lang_en>' for English.

See Prefix for details.

For English using a multilingual model:

uv run --locked live_client.py --host localhost --port 3030 --prefix='<pnc><lang_en>'

Quote the prefix and leave no spaces between tags. See Prefix for supported language tags, including <lang_detect> for automatic detection.

CAIMAN-ASR supports keyword boosting of up to 100 keywords. Keyword boosting is a technique to improve the recognition of domain-specific words/phrases and proper nouns, by boosting the probability of specific tokens at inference time.

To provide keywords to the server, use the --keywords argument with a space-separated list of keywords e.g.

uv run --locked live_client.py --keywords caiman asr myrtle \
  --keyword-c0 0.3 --keyword-beta 0.9 --keyword-context-score 1.5

The three boost controls take numeric values and require a non-empty --keywords list. Omitted controls use server defaults. --keyword-context-score is an overall multiplier: increase it for stronger boosting. Settings apply to this connection. These controls override the server-wide values per connection. See Keyword boosting hyperparameters for what c0, beta, and context_score do and tuning recommendations.

Flush

Add --manual-flush to enable Enter-to-flush in an interactive terminal:

uv run --locked live_client.py --manual-flush

While recording, press Enter to finalize the current segment and continue on the same connection. The client displays [Flush complete: manual] when acknowledged. Prefix and keyword settings are retained. Press Ctrl+C to finish recording and save the audio. Flush is disabled by default.

Troubleshooting

If the client raises OSError: [Errno -9997] Invalid sample rate, you may need to use a different audio input device:

Run uv run --locked print_input_devices.py to list available input devices
Try each device using (for example) uv run --locked live_client.py --input_device 5

CAIMAN-ASR