---
license: apache-2.0
language:
  - en
  - es
tags:
  - mixture-of-experts
  - decision-model
  - typed-decisions
  - classification
  - english
  - spanish
datasets:
  - HuggingFaceFW/fineweb
  - HuggingFaceFW/fineweb-edu
  - HuggingFaceFW/fineweb-2
  - common-pile/wikimedia_filtered
  - common-pile/doab_filtered
  - common-pile/peS2o_filtered
  - common-pile/python_enhancement_proposals_filtered
  - common-pile/stackexchange_filtered
  - common-pile/stackv2_edu_filtered
  - open-web-math/open-web-math
  - LocalLLaMA/typed-decisions
  - telepatia-ai/typed-decisions-pt-es
  - samatv256/jev-decisions-v1
  - tasksource/tasksource-jev-typed-decisions
  - n4ze3m/typed-decisions-synth
  - helmo/synthetic-typed-decisions
---

# Ines-1

> **Licences — what covers what.**
> - **Apache-2.0** ([LICENSE](LICENSE), [NOTICE](NOTICE)): the model weights (`model.safetensors`), `config.json` and
>   `tokenizer/`; the documentation (`README.md` and the other `.md` files, `paper/`, `assets/`, `tasksource_license_audit.csv`); the
>   training records (`training/*.json`); the model's own evaluation outputs in `eval/`; `SHA256SUMS`.
> - **MIT** ([LICENSE-CODE](LICENSE-CODE)): the code — `mini_v41/`, `mini_v41_jev/`, `scripts/`, `examples/`,
>   `training/code/`, `eval/btzsc22/code/` — and the environment and build files: `Dockerfile`, `.dockerignore`, `requirements.txt`,
>   `requirements.lock`, `.gitattributes`, `.gitignore`.
> - **Third-party terms, not relicensed:** the gold labels and teacher distributions of the public test sets that
>   `eval/items/`, `eval/reference_rows.json` and `eval/btzsc22/` (BTZSC label texts and targets) include keep their
>   datasets' licences. The training data is not
>   redistributed and remains under its own terms, including attribution, share-alike and copyleft ones: see
>   [THIRD_PARTY_DATA_NOTICE.md](THIRD_PARTY_DATA_NOTICE.md) and [LICENSING_NOTES.md](LICENSING_NOTES.md).

**Ines-1** is a **typed-decision model pretrained from scratch**: a 1.59B-parameter mixture-of-experts causal
encoder–decoder (405M parameters active per token) with hashed n-gram memory (Engram), pretrained on about 10B
public tokens on one H200, then fine-tuned to answer **one typed question at a time** — `choice`, `score` or `noul`
(yes/no) — with a probability for every option, in English and Spanish. The request shape follows the typed-decision
`POST /v1/systemone` interface popularised by TypeSafe's Jev. **Ines-1 does not reproduce Jev's architecture or its
full-case (multi-question) protocol.**

Development codename: mini-v41 / mini-v41-Decisions. The Python packages keep that name (`mini_v41`,
`mini_v41_jev`), and so do the provenance records in `config.json` and `training/`.

## Architecture

![Ines-1 architecture](assets/architecture.png)

Causal encoder–decoder (CED) mixture-of-experts with Engram n-gram memory: 8 encoder layers with full causal attention,
then 8 decoder layers that attend, in one softmax, to their own sliding-window K/V (128 tokens) and to a global K/V
projected once from the encoder output and shared by every decoder layer. All 16 layers use a MoE FFN (16 routed experts,
top-2, plus 1 shared expert, SwiGLU 1536); Engram is added to the residual before encoder layers 1 and 5. Context is
2,048 tokens: raw tokens reach every position through the encoder → global K/V path, while features computed inside the
decoder travel 128 + 7·127 = 1,017 positions. Vector source: [assets/architecture.svg](assets/architecture.svg).

## Supported decision interface

- **One typed question per model prompt.** The answer is the softmax over the option letters (at most 26) at the
  first reply position.
- **Several questions of a case can be API-batched**: the server (and `Decider.decide`) splits a case into one prompt
  per question and scores them together. **This is batching, not multi-question modelling.**
- **The model was not trained to condition jointly on several typed questions in one prompt**, and it does not do so
  well: with the whole case in the prompt its accuracy falls from 76.4 % to **37–42 %** (*Full-context stress tests*).

## Intended use

- Typed **single-question** decisions over a piece of state (text or JSON): classification, routing and scoring over
  a **closed, caller-defined option set**, with a probability per option.
- A starting checkpoint for **further domain specialisation** (fine-tuning on your own decisions).

Out of scope, or not supported by our evidence: use as a chat or general assistant; retrieval or fact lookup in long
documents; native multi-question (Jev full-case) compatibility; strong zero-shot performance on a new domain; any
safety-critical decision.

| | |
|---|---|
| Parameters | 1,590,066,240 total; 405,052,224 active per token; weights in bf16 |
| Layers | 8 encoder + 8 decoder in one causal stack; hidden 1024; 16 heads / 4 KV heads, head dim 64 |
| Decoder | sliding window of 128 + one global K/V projected from the encoder output; receptive field 1,017 positions |
| MoE | every layer: 16 routed experts (FFN 1536), top-2, + 1 shared expert |
| Engram | hashed 2/3/4-gram embeddings at encoder layers 1 and 5 (2 × 10⁶ rows, 128M parameters) |
| Context | 2,048 tokens; longer cases are trimmed in the middle |
| Tokenizer | byte-level BPE, 65,536 tokens (identity sha256 `859e5c63…74469`, checked at load) |

## Quickstart

```bash
pip install -r requirements.txt   # requirements.lock pins the canonical environment (ENVIRONMENT.md)
```

```python
import sys
from huggingface_hub import snapshot_download

path = snapshot_download("Endikavi/Ines-1")   # the whole repository: weights, tokenizer and reader code
sys.path.insert(0, path)
from mini_v41_jev import Decider

decider = Decider(path)                       # CUDA if available (bf16 + autocast), otherwise CPU (fp32)
question = {"type": "choice",
            "instructions": "Which department should handle this ticket?",
            "criteria": {"billing": "payments and invoices", "tech": "outages and bugs", "legal": "disputes"}}
state = "The invoice was charged twice and the customer wants the money back today."

answer = decider.decide(state, {"dept": question}, lang="en")["dept"]
print(answer["choice"])          # argmax option
print(answer["probabilities"])   # probability of every option
```

`score` questions return `probabilities` over levels `"0".."n-1"`; `noul` questions return `{"noul": P(true)}`.
`lang="es"` uses Spanish prompts. [examples/quickstart.py](examples/quickstart.py) runs the same from a local copy.

HTTP server with the same request shape (`POST /v1/systemone`, `GET /v1/models`, `GET /health`; CUDA and `aiohttp`),
batching concurrent questions into one prefill: `python scripts/serve_jev.py --port 30040`.

## Lineage

`Pretrain-V3` (9.96B tokens) → `Decision-FT Stage 1` (typed-decisions + jev-decisions; 8 epochs scheduled, **7
effective**: the epoch chosen on validation is the checkpoint kept) → `Decision-Mix Stage 2` (balanced public mix,
2 epochs) → **Ines-1**. Each stage's parent-weights hash matches the previous stage's output (table under *Training*).

## Results

All numbers measured by us on the released weights in the canonical environment ([ENVIRONMENT.md](ENVIRONMENT.md):
bf16 weights under bf16 autocast). Per-item outputs in `eval/items/`, reports in `eval/`. Accuracy = the
highest-probability option equals the gold label; intervals are bootstrap over cases (the five questions of a case
are not independent).

### typed-decisions — single-question specialist protocol

**Single-question specialist protocol, after training on the benchmark's training split**: one prompt per question;
the model was trained on typed-decisions `train` (same four workflows, with the dataset's teacher distributions).
This is not a zero-shot result.

| test (400 cases) | correct / n | accuracy | 95 % CI | choice | score | noul | Brier | KL(gold‖p) | log loss |
|---|---:|---:|---|---:|---:|---:|---:|---:|---:|
| English | 1528 / 2000 | **76.4** | 74.2–78.5 | 71.5 | 74.5 | 83.8 | 0.065 | 0.121 | 0.635 |
| Spanish (our machine translation) | 1496 / 1985 | 75.4 | 73.2–77.4 | 69.9 | 73.9 | 82.7 | 0.065 | 0.119 | 0.639 |
| Spanish (independent translation) | 1526 / 2000 | 76.3 | 74.2–78.5 | 73.0 | 73.8 | 83.0 | 0.064 | 0.119 | 0.639 |

Brier and KL are against the teacher's gold distribution and follow the dataset card's definitions (they reproduce
its *Uniform* reference row exactly: KL 0.444, Brier 0.238). Our ECE (10 equal-width bins of top-1 confidence; 0.132
on English) is reported as `our-ECE` and is **not comparable** with published ECE values, whose code is not public.
By teacher agreement (`label_agreement.argmax_agree`): 86.4 % where the teacher's samples agree (n = 1,188),
61.7 % where they do not (n = 812).

### typed-decisions — full-context stress tests

The dataset's public runs send the whole case (state + its five questions) in one request, and its card notes that
request shape can change results. Ines-1 has no readout that answers several questions in one forward, so we ran
two stress tests instead (English; Spanish behaves the same):

| readout | what the model sees and how answers are read | correct / n | accuracy | 95 % CI | Brier |
|---|---|---:|---:|---|---:|
| single question (above) | the state and one question | 1528 / 2000 | 76.4 | 74.2–78.5 | 0.065 |
| `context` | the state and **all five** questions, ending "Answer question k"; **one forward per question** | 831 / 2000 | **41.5** | 39.5–43.6 | 0.266 |
| `joint` | **one prompt** with all five questions; answers read in sequence as "k: letter", each conditioned on the model's previous answers | 742 / 2000 | **37.1** | 34.9–39.3 | 0.246 |
| *dataset reference: Prior* | *train label frequencies, ignoring the input* | | *47.0* | | *0.189* |

Neither stress test reproduces the public protocol or a Jev-style single forward over five independent decisions;
**neither is a DecisionEval score.** Both show strong sensitivity to request shape. Decision-FT Stage 1 drops the same
way (75.5 → 42.8 %, `context`), so this is not caused by Stage 2. **Task competence does not imply interface
generalisation.**

### What else the public tests measure

- **tasksource held-out** (2,206 questions, 350 sources): **new examples of tasks seen in training.** Every source of
  this test is also in the Stage 2 mix (from its train split); no example group is shared. Accuracy 49.5 % [47.4–51.6]
  (Stage 1: 38.5 %; paired p = 1e-14). It is not an evaluation on tasks absent from training; for that, see BTZSC-22 below.
- No typed-decisions test case, state, normalised state text or state structure appears in any training file.

### External figures, kept apart (no leaderboard ranking)

General / zero-shot models, as published (measured by others, whole case per request):

| model | accuracy | Brier | source |
|---|---:|---:|---|
| TypeSafe Jev 1.13 | 74.0 [72.1–75.9] | 0.148 | [DecisionEval](https://decisioneval.dev/models/typesafe-jev/), independent run, test frozen 2026-09-20 |

**Ines-1 is not ranked against Jev.** Its 76.4 % is a single-question specialist result; Jev's 74.0 % is zero-shot on
whole cases. Specialists trained on typed-decisions `train` have a separate table in the dataset card (mostly
self-reported, 0.483 to 0.801); the card also notes that scores above its teacher self-agreement (0.735) indicate
learning the teacher's quirks.

### External zero-shot classification: BTZSC-22

We evaluated Ines-1 on 22 public BTZSC classification datasets (100 examples per dataset), using an extension of the
jev-benchmarks protocol. Fifteen of the 22 datasets are absent from Ines-1's decision fine-tuning mixture.

Under the model-specific classification track, mean macro-F1 across datasets was 0.474 for Ines-1, 0.525 for Julia-1,
0.530 for laya-multilingual, and 0.537 for GLiNER2.5. Using a paired hierarchical bootstrap with datasets as the unit of
inference, the difference relative to Ines-1 was resolved for GLiNER2.5 (+0.063, 95% CI +0.014 to +0.107), but not for
Julia-1 (+0.051, 95% CI −0.026 to +0.126) or laya-multilingual (+0.056, 95% CI −0.040 to +0.163). On the 15 datasets
absent from Ines-1's decision fine-tuning mixture, none of the pairwise differences was resolved.

For datasets with at most 20 labels, native ECE was 0.127 for Ines-1, 0.298 for Julia-1, 0.093 for laya-multilingual,
and 0.159 for GLiNER2.5.

We also tested alternative decompositions of multiclass decisions. A common one-vs-rest protocol did not improve Ines-1
(mean macro-F1 0.457 overall, and 0.489 → 0.283 on the 3–20-label subset). In contrast, a descriptive pairwise-knockout
ablation improved Ines-1 on 6 of 9 multiclass datasets (accuracy). This suggests that Ines-1's current interface benefits
more from comparing a small set of competing alternatives than from independently scoring each label as a yes/no
decision. The knockout result is order-dependent and is reported as an ablation rather than a general classification
protocol.

Full protocol, manifests, prediction hashes, per-dataset results, calibration details, and reproducibility notes are
available in [`eval/btzsc22/`](eval/btzsc22/).

### Private domain benchmark

On an internal held-out domain benchmark (9 Spanish subtasks, 1,026 questions; not released), generic typed-decision
training does **not** transfer uniformly zero-shot. Macro balanced accuracy: Pretrain-V3 0.280, Stage 1 0.447, Stage 2
from pretraining 0.543, **Ines-1 zero-shot 0.558**, an untuned 322M multilingual encoder 0.553, an untuned 144M encoder
0.485. On one imbalanced yes/no subtask no untuned checkpoint separates the classes (AUROC 0.48–0.55).

**Fine-tuning this exact released checkpoint** on that domain (recipe of the earlier domain checkpoints, only the
parent changed; epoch 3 chosen on validation) raised macro balanced accuracy **0.558 → 0.790** (micro accuracy 0.788,
macro-F1 0.780; Stage 1 + the same domain fine-tuning: 0.784); on the yes/no subtask AUROC 0.541 → 0.889;
organisations never seen in training scored 0.860 accuracy (0.678 balanced) against 0.841 (0.665) for seen ones.
Cost: 21,423 domain cases, 23,608 questions per epoch, 28.5M prompt tokens, 2.25 H200 GPU-hours. Specialisation cost
**6–9 points of typed-decisions accuracy**: EN 0.764 → 0.698, ES 0.754 → 0.689, independent ES 0.763 → 0.675
(tasksource 0.495 → 0.480). **The domain-adapted model is private and is not included in this repository.**

### Ablations (public tests, same questions, exact McNemar)

| comparison | typed EN | typed ES (ours) | typed ES (indep.) | tasksource |
|---|---|---|---|---|
| Stage 1 → release | 75.5 → 76.4 (p = 0.24) | 74.7 → 75.4 (p = 0.32) | 74.6 → 76.3 (p = 0.023) | 38.5 → 49.5 (p = 1e-14) |
| release, Engram removed at inference | 76.4 → 74.7 (p = 0.018) | 75.4 → 72.8 (p = 0.001) | 76.3 → 73.5 (p = 3e-4) | 49.5 → 47.0 (p = 0.007) |
| Stage 2 from pretraining (instead of from Stage 1) | 72.6 | 70.4 | 71.2 | 50.7 |

"Engram removed at inference" = every Engram module returns its input (gate 0) on this checkpoint, which was trained
with Engram: it measures this model's reliance on Engram, not Engram's contribution to training.

### Speed

[eval/speed.md](eval/speed.md) has the full protocol and both runs. One H200, bf16, HTTP closed loop, same requests and
load generator for every model. Two regimes, kept apart:

| | single request p50 / p95 | max throughput (questions/s) |
|---|---:|---:|
| short request (2 questions, ~60 prompt tokens each) — Ines-1, 1 process | 9.6 / 9.7 ms | ~1,900 |
| same, 144M / 322M encoders, 4 processes each | 12.7 / 21–26 ms | ~1,700–1,850 |
| typed-decisions case (5 questions, ~315 tokens each) — Ines-1 | 29.6 / 38.3 ms | ~540 |
| same, 144M / 322M encoders | 15–16 / 87–88 ms | ~760–1,110 |

On ~300-token questions **the encoders are 1.4–2.1× faster**.

## Training

| stage | parent weights (sha256) | data | epochs | optimizer steps | output weights (fp32, sha256) |
|---|---|---|---|---|---|
| Pretrain-V3 | — | 9.96B tokens processed from the ~10B-token corpus below | 1 pass | 152,588 × 65,536 tokens | `dd3a794d…57f9` |
| Decision-FT Stage 1 | `dd3a794d…` | typed-decisions train EN + our ES translation + 9.5k jev-decisions: 20,233 questions | 8 scheduled, **7 effective** | 8,852 of 10,117 | `d5c76527…8fbf` |
| Decision-Mix Stage 2 (Ines-1) | `d5c76527…` | balanced mix, 86,436 questions | 2 | 10,805 | `5657d3ee…e8fb` |

The released `model.safetensors` is the Stage 2 fp32 checkpoint cast to bf16.

Recipe for both decision stages: full fine-tuning, AdamW lr 1e-5 (β 0.9/0.95, no weight decay, clip 1.0), accumulation
16 (one question per forward), linear warm-up over 5 % of the scheduled steps then linear decay to 0, cross-entropy of
the letter distribution against the teacher distribution when present (one-hot otherwise) + 0.01 × MoE auxiliary loss,
choice options shuffled per epoch, epoch chosen on the mean of one validation file per data family, seed 7. The
typed-decisions training questions are seen in both stages (7 + 2 epochs).

**Code provenance, stated as it happened:** Stage 2 ran from commit `0b5e63e` with uncommitted files (`-dirty`: the
mix builder `prepare_mix.py` was not yet committed). It was committed afterwards (`039b99c`) and rebuilding the mix from
that clean commit reproduces all 16 data files byte for byte. Bundle code comes only from commits (see `config.json`).

## Training data

Full provenance, identifiers and licence notes: [THIRD_PARTY_DATA_NOTICE.md](THIRD_PARTY_DATA_NOTICE.md); per-source
licences of the tasksource questions used: [TASKSOURCE_LICENSE_AUDIT.md](TASKSOURCE_LICENSE_AUDIT.md); how our Spanish
translation was made: [training/TRANSLATION_PROVENANCE.md](training/TRANSLATION_PROVENANCE.md).
No training data is redistributed in this repository.

### Pretraining (Pretrain-V3, ~10B tokens, realised mix)

| domain | share | sources |
|---|---:|---|
| general English | 46.5 % | FineWeb-Edu, FineWeb |
| Spanish | 20.0 % | FineWeb-2 (`spa_Latn`) |
| reference | 15.0 % | Common Pile: Wikimedia, DOAB, peS2o, Python Enhancement Proposals |
| math | 9.0 % | OpenWebMath, Common Pile StackExchange (math sites) |
| code | 8.5 % | Common Pile Stack-Edu |
| structured | 1.0 % | 75M tokens of JSON/YAML/XML mined from Stack-Edu + 25M synthetic |

**Approximately 25M pretraining tokens (~0.25% of the corpus) were synthetic structured documents** (JSON-schema
style documents generated for this corpus; not redistributed). Their contribution was not isolated.

### Decision fine-tuning (Stage 1 and Stage 2)

Stage 1: typed-decisions train (English) + our Spanish translation of it + 9.5k jev-decisions questions. Stage 2
mix (no family above ~29 %):

| family | questions | source |
|---|---:|---|
| typed-decisions EN | 5,420 | LocalLLaMA/typed-decisions @ `c76749ec` (data files identical to the 2026-09-18 revision) |
| typed-decisions ES (ours) | 5,335 | our machine translation of the above (text values only; keys and structure checked; [provenance](training/TRANSLATION_PROVENANCE.md)) |
| typed-decisions ES (independent) | 5,415 | telepatia-ai/typed-decisions-pt-es @ `8c85a4d2`, Spanish only; 117 cases matching our validation split removed |
| tool selection | 17,898 | samatv256/jev-decisions-v1 @ `c12aadf1`, general-clean-50k (CC-BY-4.0) |
| tasksource | 24,959 | tasksource/tasksource-jev-typed-decisions @ `8173a06c`: rows whose metadata says `license_use=commercial`, ≤ 80 per source, 366 sources, none derived from typed- or jev-decisions |
| synthetic, teacher soft labels | 18,001 | n4ze3m/typed-decisions-synth @ `5ece89a2` |
| synthetic | 9,408 | helmo/synthetic-typed-decisions @ `1827dc0d` |

## Limitations

- **One-question interface.** With the whole case in one prompt, accuracy falls to 37–42 % (below the dataset's
  Prior); API batching does not change that the model sees one question at a time.
- **Specialist exposure to the benchmark.** The typed-decisions results are in-distribution for a model trained on
  its train split; they are not comparable with zero-shot results.
- **Limited evaluation on tasks absent from training.** tasksource measures new examples of tasks seen in training;
  BTZSC-22 includes 15 datasets absent from the decision fine-tuning mixture (100 examples each; pretraining exposure
  not excluded), where no difference against the three classifiers is resolved.
- **Uneven zero-shot domain transfer** (0.558 macro balanced accuracy on a private domain), including a class-prior
  shift on yes/no questions over long texts after Stage 2.
- **Specialisation forgetting.** Domain fine-tuning cost 6–9 points of typed-decisions accuracy.
- **No retrieval.** 2,048-token context; it fails a passkey retrieval test even at 1k tokens: it decides, it does not
  look things up.
- **Engram reliance was measured only by removal at inference**; no model was trained without Engram.
- **Pretrain-V2 → V3 is not a pure token-scaling ablation**: corpus, horizon and schedule changed together (and V3 is
  the first corpus with synthetic documents).
- **Soft vs hard labels not ablated**; the contribution of teacher distributions is not isolated.
- **Synthetic contribution not isolated**, in pretraining or in fine-tuning.
- **our-ECE is not comparable** with published ECE values; probabilities are calibrated by training, not certified.
- Weaker on `choice` than on `score` and `noul`. Spanish results are on machine translations (two independent
  translations agree within 1 point for every model). Not evaluated for safety-critical decisions.

## Reproducibility

- **Environment**: [ENVIRONMENT.md](ENVIRONMENT.md) (GPU, driver, Python, PyTorch, settings), `requirements.txt`
  (minimum), `requirements.lock` (exact versions of the evaluation environment), `Dockerfile` (that environment as an
  image; built and smoke-tested, see ENVIRONMENT.md). File hashes: `SHA256SUMS`;
  `model.safetensors` sha256 `7fa38ab3adf37530c5b150b0527d8cc9c24350b402cd2d8bf727e736b78ac555`.
- **Canonical inference** (every number in this card): bf16 weights under `torch.autocast("cuda", dtype=torch.bfloat16)`,
  one prompt per question, `prefill(num_logits=1)` — the default path of `Decider.decide`.
- **Canonical evaluator**: `training/code/eval_items.py` (per-item outputs) and `training/code/metrics.py`; their
  outputs for this model are in `eval/items/` and `eval/report.json`.
- **HTTP / batched path** (`scripts/serve_jev.py`, `Decider.decide(..., batched=True)`): right-padded rows in one
  prefill, with CUDA graphs on the server. It is **not bit-identical** to the canonical path and may change a few
  answers because of batching and bf16 numerics (8 of 400 on a probe). Through HTTP the English test scored 1527 to
  1531 / 2000 in three runs of the same server (1975–1976 of 2000 answers equal to the canonical ones; concurrent
  batching changes from run to run), vs 1528 / 2000 canonical.
- PyTorch 2.13 and 2.14 gave bit-identical outputs with the same code; autocast on/off and the attention backend each
  changed 8 of 400 answers (ENVIRONMENT.md).

## Future work (hypotheses, not results)

Branched / tree attention or a shared-state prefill so that several questions are answered independently in one
forward; training explicitly on multi-question cases; and measuring whether that restores full-case behaviour without
sacrificing single-question performance.

## Files

`model.safetensors` (bf16), `config.json` (configuration + provenance), `tokenizer/`, `mini_v41/` (modelling code),
`mini_v41_jev/` (reading decisions, `Decider`), `scripts/` (HTTP server, load test), `examples/`, `training/` (recipe,
manifests, data audit, scripts), `eval/` (reports, per-item outputs, speed protocol, `btzsc22/` external classification evaluation), [ENVIRONMENT.md](ENVIRONMENT.md),
`requirements.txt`, `requirements.lock`, `Dockerfile`, [CHANGELOG_CLAIMS.md](CHANGELOG_CLAIMS.md),
[THIRD_PARTY_DATA_NOTICE.md](THIRD_PARTY_DATA_NOTICE.md), [LICENSING_NOTES.md](LICENSING_NOTES.md),
[TASKSOURCE_LICENSE_AUDIT.md](TASKSOURCE_LICENSE_AUDIT.md) + `tasksource_license_audit.csv`,
`assets/` (architecture diagram: PNG + SVG source), `paper/` (technical report: PDF + LaTeX sources), [LICENSE](LICENSE) (Apache-2.0), [NOTICE](NOTICE), [LICENSE-CODE](LICENSE-CODE) (MIT), `SHA256SUMS`.

## Paper

The technical report is available in this repository:

[Ines-1: Training a 1.6B Mixture-of-Experts Model for Typed Decisions from Scratch](./paper/Ines-1.pdf)

An arXiv submission is planned; the citation will be updated once an arXiv identifier is available. The LaTeX sources
(`paper/main.tex`, `paper/references.bib`, `paper/figures/`) build the same PDF with `pdflatex` + `bibtex`.

## Citation

Preprint; the identifier will be added when it exists.

```bibtex
@misc{endikavi2026ines1,
  title  = {Ines-1: Training a 1.6B Mixture-of-Experts Model for Typed Decisions from Scratch},
  author = {Endikavi},
  year   = {2026},
  note   = {Preprint}
}
```