input: audio: sample_rate: 16000 channels: 1 format: [flac, wav, mp3, ogg] # any soundfile-readable format, resampled to 16kHz mono features: shape: [1, 80, 3000] dtype: float32 extraction: log-mel spectrogram via WhisperProcessor feature_extractor (80 mel bins, 30s window, padded/truncated) preprocessing: - load audio (soundfile.read), convert to mono by averaging channels if stereo - resample to 16000 Hz via linear interpolation if source sample_rate != 16000 - "compute log-mel spectrogram: processor.feature_extractor(waveform, sampling_rate=16000).input_features" decode_signature: inputs: - decoder_token: { shape: [1, 1], dtype: int64 } - input_pos: { shape: [1], dtype: int64 } - self_attn_mask: { shape: [1, 1, 1, 448], dtype: float32 } - self_kv_cache: { shape: "48 x [1, 448, 16, 64]", dtype: float32 } # 24 layers x (k, v) - cross_kv_cache: { shape: "48 x [1, 1500, 16, 64]", dtype: float32 } # from encode outputs output: format: greedy-decoded token sequence -> text string decoding: method: greedy (argmax over logits), autoregressive, self-attention + cross-attention KV-cache accelerated signatures: "encode (mel features [1,80,3000] -> cross-attention KV cache, 48 tensors), decode (single-step: decoder_token [1,1] int64, input_pos [1] int64, self_attn_mask [1,1,1,448] float32, self/cross KV cache -> logits [1,1,51865] + updated self-attention KV cache)" prompt: [decoder_start_token_id, "<|en|>", "<|transcribe|>", "<|notimestamps|>"] # language=en, task=transcribe max_new_tokens: 128 stop_condition: eos_token_id (50257) suppressed_tokens: model.generation_config.suppress_tokens applied every step; begin_suppress_tokens applied only at the first generated step postprocessing: - tokenizer.decode(token_ids, skip_special_tokens=True) - strip() surrounding whitespace