OpenRouter Leaderboard: 425 Models by Price, Measured Speed and Korean Quality
What this leaderboard is
It ranks every model listed on OpenRouter on the things that decide a purchase: price per million tokens, the providers serving each model and at what precision and uptime, context length, response speed, and language quality.
Price, provider and traffic data come from OpenRouter's public API, which this project builds on and credits. Two columns are ours, because they did not exist: measured response speed, and a Korean-language grade.
The Korean column is the reason this exists. Request-language statistics will tell you that Korean requests arrive. They will not tell you whether the Korean that comes back is usable in a Korean office — and for a team putting an API in front of Korean customers, that is the only question that matters. So we measured it, on 330 models.
Three languages, three currencies, and a stated boundary between borrowed data and measured data.
What "good Korean" breaks down into
A single "Korean score" would hide the thing that matters. Korean fails in specific, separable ways, so we grade seven axes independently:
| Axis | What it tests |
|---|---|
| Register | Holding a consistent formal or informal level through a whole answer |
| Knowledge of Korean institutions | Leave accrual law, tax categories, public-agency procedure — facts, not fluency |
| Format compliance | Obeying an explicit instruction like "answer in exactly two sentences" |
| Honorifics | Subject and object honorifics, including the errors native speakers notice instantly |
| Summary accuracy | Preserving figures and named entities when compressing |
| Terminology | Professional vocabulary in finance, law, engineering |
| Freedom from translationese | Whether it reads as Korean or as translated English |
Each axis is graded A/B/C/F, averaged, and folded into eight bands from A+++ down to F.
What the table shows
The winner is a different model in every column
Ask "which model is best at Korean" and you get three different answers depending on what you meant:
| Winner | Value | |
|---|---|---|
| Quality | Google: Nano Banana 2 (Gemini 3.1 Flash Image) | A+++ |
| Value (grade ÷ price) | Mistral: Mistral Nemo | $0.049 / 1M, grade C |
| Speed (measured first token) | NVIDIA: Nemotron 3 Nano Omni (free) | 128ms, grade F |
This pattern repeats on every axis we track — intelligence, coding, agentic, arena Elo. The quality leader is essentially never the value leader, and almost never the speed leader. A leaderboard with a single ranked column is answering a question nobody actually asked.
Five axes, fifteen cells, and the same model almost never holds two columns in a row.
Two axes collapse, and they are the two that matter most in an office
| Axis | A rate | F rate |
|---|---|---|
| Honorifics | 8.5% | 51.8% |
| Knowledge of Korean institutions | 9.4% | 54.7% |
| Terminology | 31.2% | 44.2% |
| Format compliance | 47.6% | 34.2% |
| Summary accuracy | 49.2% | 37.4% |
| Freedom from translationese | 53.1% | 33.4% |
| Register | 53.3% | 32.7% |
The spread is the finding. Models that read as fluent Korean — clean register, no translationese — fail on honorifics and don't signal that they failed. A model that writes beautiful, natural Korean and then applies an honorific to an inanimate object produces text that is fluent and wrong at the same time, which is worse than obviously broken output because nobody catches it in review.
The institutional-knowledge axis fails for a different reason: it is not a language problem at all. Korean labour law and tax categories are facts, and a model that never saw them fluently invents them in perfect Korean.
Overall distribution
| Band | Models | Share |
|---|---|---|
| A+++ | 5 | 1.5% |
| A++ | 2 | 0.6% |
| A+ | 18 | 5.5% |
| B+ | 55 | 16.7% |
| B | 58 | 17.6% |
| C | 52 | 15.8% |
| D | 29 | 8.8% |
| F | 111 | 33.6% |
A third of the field fails outright. Under 8% reach any A band.
Recency and size do not predict Korean
Among the five models holding a perfect 3.00 is gpt-3.5-turbo-16k, a model from 2023, sitting
alongside 2026 flagships. gpt-3.5-turbo-0613 takes A++.
The conclusion is narrow and practical: a model's Korean cannot be inferred from its release date, its parameter count or its English benchmark scores. It has to be measured, per model.
Measuring speed instead of inheriting it
The latency field in the public model data is null. So we called 329 models on a paid API with an identical prompt and recorded first-token latency and tokens per second.
We show that measurement next to OpenRouter's p50–p99 percentiles, which aggregate thousands of real requests. The two answer different questions — theirs describes steady-state behaviour across traffic, ours is one controlled call with a known prompt, provider and moment. They are labelled separately and shown side by side rather than merged, so each can be checked against the other.
This matters because our single call also tells you something theirs cannot: which provider served it, at what precision, at that moment.
How the grading was kept honest
An LLM judge is only worth as much as the controls around it.
- The judge never sees the candidate model's name. No brand priors.
- It runs only after scoring every known-answer control correctly. The control set is fixed in advance and the run is blocked, not warned, when a control is missed.
- Character contamination is counted mechanically, not judged. Han characters, kana and the Hangul ratio are computed in code. This is arithmetic; delegating arithmetic to a language model adds variance for nothing.
- Fewer than five of seven axes graded means no grade at all, flagged partial. A model cannot reach the top band by acing one question.
The supply side
Of the 107 providers serving these models, 55 are headquartered in the US, 6 in Singapore, 6 in China, and a scattering elsewhere. 33 do not disclose a headquarters.
Among the 74 that do, zero are Korean.
We also surface each provider's headquarters country and data-centre locations in the model detail view, because it is the first question Korean enterprise procurement asks — where does the data physically go — and no public leaderboard shows it.
The same weights at $2.05 or $5.80, at fp4 or fp8, from the US, China, Singapore, Sweden or France.
The best-value pick states its reasoning rather than asserting a rank.
Use it
Everything is live at the Space in English, Korean and Chinese, with prices in USD, KRW and CNY. Pick a starting point (coding agents, long documents, high volume, real-time, Korean work, multimodal) or combine filters, compare two to four models side by side with all seven Korean axes expanded, and share the exact filtered view as a link.
Korean grade sits next to price and latency, so a cheap model that breaks in Korean is visible at a
glance instead of two tabs away.
The data is open and needs no key. On the Space's app URL:
GET /api/korean # grades, per-axis detail included
GET /api/speed # measured first-token latency and tokens per second
GET /api/models # prices, context, modalities
Refreshed daily at 08:00 KST.
Feedback
If a grade looks wrong for a model you know well, tell us with the prompt. The seven axes are a proposal, not a standard.
Built by GiniGEN AI. Price, provider and traffic data from OpenRouter's public API. Korean grades and latency measurements our own.
