Translation
Transformers
PyTorch
Safetensors
IndicTrans
text2text-generation
indictrans2
ai4bharat
multilingual
custom_code
Instructions to use ai4bharat/indictrans2-en-indic-1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ai4bharat/indictrans2-en-indic-1B with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "translation" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("translation", model="ai4bharat/indictrans2-en-indic-1B", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForSeq2SeqLM model = AutoModelForSeq2SeqLM.from_pretrained("ai4bharat/indictrans2-en-indic-1B", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Support transformers 4.48β5.x: fix ONNX import, KV cache and tokenizer errors; SDPA by default
#19
by VarunGumma - opened
This PR updates the remote code so the model works with current transformers releases, without changing its outputs.
Errors fixed
ModuleNotFoundError: No module named 'transformers.onnx'(transformers v5): the ONNX export config is only defined whentransformers.onnxexists.AttributeError: ... _special_tokens_mapwhen loading the tokenizer (transformers v5).AttributeError: 'NoneType' object has no attribute 'shape'ingenerate(transformers >= 4.54): KV caching now usesEncoderDecoderCache.from_pretrained(..., attn_implementation=...)was ignored,config.hidden_sizeraised, tokenizersave_pretrainedβfrom_pretrainedfailed,get_tgt_vocab()raised.
Changes
modeling_indictrans.py: KV cache throughCacheobjects; attention throughALL_ATTENTION_FUNCTIONS(eager,sdpa,flash_attention_2); weight tying and buffer initialisation compatible with transformers v5; beam search no longer reorders the cross-attention cache (it is identical across the beams of a sentence).configuration_indictrans.py,tokenization_indictrans.py: the fixes above.config.json: drop"attn_implementation": "eager", so SDPA is used by default. Passattn_implementation="eager"or"flash_attention_2"to choose explicitly.
Compatibility
- Same outputs: with eager attention the forward pass is bit-identical to the previous code in fp32 and fp16 (encoder, teacher forcing, KV-cache decoding, training with dropout, gradients), and greedy, beam and sampled generations are token-identical. SDPA only differs by floating-point rounding.
- Tested with transformers 4.53.2, 4.57.6 and 5.21.0 on CPU and Apple MPS (smoke-tested with 4.48.3);
flash_attention_2was not tested (no CUDA GPU). Requires transformers >= 4.48; to keep an older version, load withrevision=set to the currentmaincommit. - Throughput (dist-200M, MPS fp16, 64 sentences, batch 32): greedy +32%, 5-beam search 11Γ faster on transformers 5.21 than on 4.53.2. Beam search in transformers 4.53.2 also appended runaway tokens to some batched outputs; 4.57 and later do not.
Related: https://github.com/VarunGumma/IndicTransToolkit/issues/28, https://github.com/VarunGumma/IndicTransToolkit/issues/29, https://github.com/AI4Bharat/IndicTrans2/issues/128, toolkit fix https://github.com/VarunGumma/IndicTransToolkit/pull/31