---
language: kk
license: apache-2.0
tags:
- bert
- kazakh
- question-answering
- extractive-qa
- tokenizer-comparison
datasets:
- IS2AI/KazQAD
base_model: Abzalbek89/kk-bert-small-morph-bpe
metrics:
- exact_match
- f1
---

# kk-bert-small-morph-bpe-kazqad

BERT-Small finetuned on **KazQAD** (Kazakh Reading Comprehension) starting from
[`Abzalbek89/kk-bert-small-morph-bpe`](https://huggingface.co/Abzalbek89/kk-bert-small-morph-bpe) — pretrained with the **morphbpe** tokenizer.

Part of a comparative study of subword tokenizers for Kazakh small language models.

## Test set results
| metric | value |
|---|---|
| **Exact Match** | **1.8798** |
| **F1** (token-level) | **8.6526** |

## Training
| | |
|---|---|
| Base model | [Abzalbek89/kk-bert-small-morph-bpe](https://huggingface.co/Abzalbek89/kk-bert-small-morph-bpe) |
| Dataset | [IS2AI/KazQAD](https://github.com/IS2AI/KazQAD) reading-comprehension v1.0 |
| Examples | 2,713 test, ~10K train |
| Epochs | 10 |
| Batch size | 12 |
| Max length | 384 (stride 128) |
| Learning rate | 3e-5 (warmup 10%, weight decay 0.01) |
| Precision | fp16 |
| Hardware | RTX A4000 |
| Time | 1.3 min |
| Seed | 42 |

## Usage
```python
from transformers import AutoTokenizer, AutoModelForQuestionAnswering, pipeline
tok = AutoTokenizer.from_pretrained("Abzalbek89/kk-bert-small-morph-bpe-kazqad")
model = AutoModelForQuestionAnswering.from_pretrained("Abzalbek89/kk-bert-small-morph-bpe-kazqad")
qa = pipeline("question-answering", model=model, tokenizer=tok)
qa(question="Кардиннің туған жылы қашан?",
   context="Belcalis Marlenis Almánzar 1992 жылы 11 қазанда дүниеге келген...")
```

## Companion artifacts
- Pretrained base: [`Abzalbek89/kk-bert-small-morph-bpe`](https://huggingface.co/Abzalbek89/kk-bert-small-morph-bpe)
- KazNERD finetune: [`Abzalbek89/kk-bert-small-morph-bpe-kaznerd`](https://huggingface.co/Abzalbek89/kk-bert-small-morph-bpe-kaznerd)
- Aggregate metrics: [`Abzalbek89/kk-tokenizer-fertility-baseline`](https://huggingface.co/datasets/Abzalbek89/kk-tokenizer-fertility-baseline)