---
language: el
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
base_model: swiss-ai/Apertus-8B-2509
datasets:
- glossAPI/apertus-8b-greek-cpt-modern-greek-train
tags:
- apertus
- greek
- glossapi
- continued-pretraining
model-index:
- name: apertus-8b-greek-cpt main
  results:
  - task:
      type: text-generation
    dataset:
      name: GreekMMLU (decontaminated, n=16,159)
      type: dascim/GreekMMLU
    metrics:
    - type: accuracy
      value: 54.85
      verified: false
  - task:
      type: text-generation
    dataset:
      name: ASEP MCQA (strict contamination-filtered)
      type: fffoivos/native-greek-suite
    metrics:
    - type: accuracy
      value: 55.08
      verified: false
  - task:
      type: text-generation
    dataset:
      name: DemosQA (strict contamination-filtered)
      type: fffoivos/native-greek-suite
    metrics:
    - type: accuracy
      value: 46.58
      verified: false
  - task:
      type: text-generation
    dataset:
      name: GPCR (strict contamination-filtered)
      type: fffoivos/native-greek-suite
    metrics:
    - type: accuracy
      value: 62.89
      verified: false
  - task:
      type: text-generation
    dataset:
      name: Medical MCQA (strict contamination-filtered)
      type: fffoivos/native-greek-suite
    metrics:
    - type: accuracy
      value: 38.42
      verified: false
  - task:
      type: text-generation
    dataset:
      name: OYXOY metaphor (strict contamination-filtered)
      type: fffoivos/native-greek-suite
    metrics:
    - type: accuracy
      value: 33.89
      verified: false
  - task:
      type: text-generation
    dataset:
      name: OYXOY NLI (strict contamination-filtered)
      type: fffoivos/native-greek-suite
    metrics:
    - type: accuracy
      value: 38.73
      verified: false
  - task:
      type: text-generation
    dataset:
      name: OYXOY WiC (strict contamination-filtered)
      type: fffoivos/native-greek-suite
    metrics:
    - type: accuracy
      value: 33.64
      verified: false
  - task:
      type: text-generation
    dataset:
      name: OYXOY WSD (strict contamination-filtered)
      type: fffoivos/native-greek-suite
    metrics:
    - type: accuracy
      value: 38.06
      verified: false
  - task:
      type: text-generation
    dataset:
      name: Apertus Table-14 retention macro
      type: apertus-pretraining-suite
    metrics:
    - type: accuracy
      value: 62.95
      verified: false
---

# Apertus 8B Greek CPT — full checkpoint trajectory

Continued pretraining on 76.685B active tokens. Base models, not instruction tuned.

## Training

79% Modern Greek, 20% foreign replay, 1% Old Greek. Extended vocabulary: 148,992 tokens. 18 training checkpoints and two checkpoint averages. `main`: terminal checkpoint.

## Recommended checkpoint

`base-avg30B-50B`: GreekMMLU 56.78% (best single checkpoint: 56.81%). Apertus pretraining-suite macro: 64.58% (terminal: 62.95%).

## Repositories and checkpoints

| Resource | Links |
|---|---|
| Original Apertus | [Repository](https://huggingface.co/swiss-ai/Apertus-8B-2509), [Revision](https://huggingface.co/swiss-ai/Apertus-8B-2509/tree/3162c99675aa588097cecd4a24b9aa1f712af477) |
| Instruction models | [Repository](https://huggingface.co/glossAPI/greek-apertus-8b-instruct), [Stage 1](https://huggingface.co/glossAPI/greek-apertus-8b-instruct/tree/stage1-sft-S1), [Greek maths](https://huggingface.co/glossAPI/greek-apertus-8b-instruct/tree/stage2-sft-recipe3), [Conversation](https://huggingface.co/glossAPI/greek-apertus-8b-instruct/tree/stage3-sft-chat-P) |
| Greek base | [base-avg30B-50B](https://huggingface.co/glossAPI/apertus-8b-greek-cpt/tree/base-avg30B-50B), [17-step18284-tokens77B](https://huggingface.co/glossAPI/apertus-8b-greek-cpt/tree/17-step18284-tokens77B), [base-avg30B-50B](https://huggingface.co/glossAPI/apertus-8b-greek-cpt/tree/18-avg-uniform5-tokens30B-50B), [CHECKPOINTS.md](https://huggingface.co/glossAPI/apertus-8b-greek-cpt/blob/main/CHECKPOINTS.md), [checkpoint-index.json](https://huggingface.co/glossAPI/apertus-8b-greek-cpt/raw/main/checkpoint-index.json) |
| Pre-training data | [Repository](https://huggingface.co/datasets/glossAPI/apertus-8b-greek-cpt-modern-greek-train), [Revision](https://huggingface.co/datasets/fffoivos/apertus-8b-greek-cpt-modern-greek-train/tree/c72f02513aa617f9229b7ca3986fce69421165eb) |
| SFT data | [Stage 1](https://huggingface.co/datasets/glossAPI/greek-apertus-sft-data/tree/main/stage1-sft-S1), [Repository](https://huggingface.co/datasets/glossAPI/greek-apertus-sft-data) |
| Post-training data | [Greek maths](https://huggingface.co/datasets/glossAPI/greek-apertus-post-training-data/tree/main/second_stage_sft/stage2-sft-recipe3), [Conversation](https://huggingface.co/datasets/glossAPI/greek-apertus-post-training-data/tree/main/second_stage_sft/stage3-sft-chat-P), [Repository](https://huggingface.co/datasets/glossAPI/greek-apertus-post-training-data) |
| Tokenizer | [Revision](https://huggingface.co/fffoivos/apertus-tokenizer-extension/tree/fcd33ec09fb7d86bc072b3a4b3e890efa6473b66), [Repository](https://huggingface.co/fffoivos/apertus-tokenizer-extension) |
| Benchmark audit | [Repository](https://huggingface.co/datasets/fffoivos/glossapi-greek-nanochat-pretraining-dataset-v2) |

## Acknowledgements

This work was implemented thanks to a grant by Swiss AI for compute on CSCS.

Collection: [Greek Apertus 8B](https://huggingface.co/collections/glossAPI/greek-apertus-8b-v05-preview-6abecae3e0362dff3324b647).