---
license: mit
language:
- hi
tags:
- transliteration
- hindi
- hinglish
- ime
- input-method
pipeline_tag: text2text-generation
---

# Hinglish → Devanagari Transliteration Model (v3)

15M-param character-level seq2seq Transformer for romanized Hindi (Hinglish)
→ Devanagari transliteration, built for a Chinese-smart-pinyin-style Hindi IME.
Trained on [`IB-Emper/hinglish-translit-v2`](https://huggingface.co/datasets/IB-Emper/hinglish-translit-v2)
(548,943 pairs) plus the Dakshina lexicon and a capped Aksharantar split.

## Files

- `best.pt` — fp32 checkpoint, ~60MB
- `best_int8.pt` — dynamically INT8-quantized checkpoint (feedforward layers only;
  attention projections stay fp32 — PyTorch's dynamic quantization does not touch
  `nn.MultiheadAttention` projections), ~37MB
- `model.py` — architecture + tokenizer + checkpoint load/save (needed to load either file)

## Architecture

d_model=384, nhead=6, 4 encoder + 4 decoder layers, dim_feedforward=1280,
dropout=0.1, max_len=64, norm_first (pre-LN), weight-tied embedding/output.
Char-level tokenizer, frozen vocab stored in the checkpoint. 15.13M params.

## Usage

```python
from model import load_checkpoint, BOS, EOS
import torch

model, tok, extra = load_checkpoint("best.pt", map_location="cpu")
# beam search / decoding left to the caller -- see the project repo for a
# reference implementation (KV-cached beam search decoder).
```

Loading `best_int8.pt` requires reconstructing the model, applying
`torch.quantization.quantize_dynamic(model, {torch.nn.Linear}, dtype=torch.qint8)`,
*then* calling `load_state_dict` on the result — the packed quantized weights
won't load directly into a plain fp32 model.

## Eval (Dakshina lexicon, Hindi test set, beam+LM rescoring)

| | acc@1 | acc@3 |
|---|---|---|
| ALL | 62.5% | 77.0% |
| FREQ tier (own corpus-frequency tiering) | 84.2% | — |

For comparison, IndicXlit (AI4Bharat/Microsoft, 11M params split across 21
languages) reports 60.5% acc@1 / 81.9% acc@3 unreranked, 72.4% acc@1 with
their own reranking, on the same official Dakshina test set. This model is
competitive on acc@1, not superior — it loses to IndicXlit's reranked top-1
and to their top-3 even before reranking. It was built and tuned for casual/
code-switched chat-style input specifically, which IndicXlit was not trained
or evaluated on; that comparison has not been run head-to-head.

## Known limitations

- Confusable near-homophone pairs (मेरा/मारा, क्या/किया-type errors) may or may not always have a correct prediction.
- ~13% of training pairs retain inline Latin code-switch words on the
  Devanagari side by design (realistic chat register), not a bug.
- Not yet quantized beyond dynamic INT8;