Conditional Decoding

Having trained a conditional model by following the instructions in the conditional training docs, you can select the output style at validation/inference time by providing a prefix.

What it does

  • Initialize the prediction network with control tokens before decoding.
  • Supported prefixes:
    • pnc → use <pnc> (punctuation + casing)
    • nopnc → use <nopnc> (lowercase, no punctuation)
    • lang_XX → use <lang_XX> (language code XX, e.g. lang_en, lang_fr, etc.)
  • Works with both greedy and beam decoders.

Note

Default: If --pnc_prefix and --val_lang_prefix are not set, no prefix is used and decoding behaves exactly as before (backwards compatible with existing models).

Usage

pnc

./scripts/val.sh --pnc_prefix=pnc

nopnc

./scripts/val.sh --pnc_prefix=nopnc

Language prefix

./scripts/val.sh --pnc_prefix=pnc --val_lang_prefix=lang_fr

A prefix with more than one language lists its language tokens sorted by language code, in plain string order, e.g. <pnc><lang_de><lang_en> instead of <pnc><lang_en><lang_de>. This is the order the model is trained on, and the checkpoint’s valid_prefix accepts, so anything that builds a prefix, such as an inference server, must sort the languages the same way. --val_lang_prefix sorts them for you, so they can be given in any order:

./scripts/val.sh --pnc_prefix=pnc --val_lang_prefix lang_fr lang_en

Note

The --(lang|pnc)_prefix options are available in both val.sh and train.sh (for on-the-fly validation during training).

Start of audio

With soa: "<SOA>" in the config’s user_tokens, every prefix ends with <SOA>, in training and when decoding, e.g. <pnc><lang_en><lang_fr><SOA>. Whatever the prefix’s length, the encoder’s first audio frame and the first prediction then follow the same token. The checkpoint’s default_prefix and valid_prefix include it.

Training requires the tokenizer to have <SOA>. Validation and checkpoint averaging of a checkpoint whose tokenizer predates <SOA> drop it with a warning, so the checkpoint is evaluated as it was trained.

Notes

  • Ensure your model config .yaml includes the control tokens in user_tokens, e.g. <pnc> and <nopnc>.
    • And that your tokenizer includes these user symbols and maps them to valid token IDs.
  • The model must have been trained with conditional training; otherwise decoding with a prefix will fail.