--- library_name: transformers license: apache-2.0 base_model: Qwen/Qwen3.8-27B base_model_relation: finetune pipeline_tag: image-text-to-text language: - en tags: - reasoning-gym - reinforcement-learning - grpo - full-finetune - caesar-cipher --- # Caesar Cipher — Qwen3.8-27B Adam expert A full-weight reinforcement-learning expert trained by Baseten on **Caesar Cipher** using GRPO with Adam. Decode Caesar-cipher text. This repository contains the final inference checkpoint after **100 optimizer updates**. - Base model: [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B/tree/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0), pinned revision `1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0`. - Environment source: [Caesar Cipher](https://github.com/open-thought/reasoning-gym/blob/49b07130b3fcd12f2d064bba7c43869543a0e7e7/reasoning_gym/algorithmic/caesar_cipher.py) from [Reasoning Gym](https://github.com/open-thought/reasoning-gym), pinned revision `49b07130b3fcd12f2d064bba7c43869543a0e7e7`. The experiment used a frozen task bank and a completed-answer binary reward adapter derived from this environment; these results are not an official upstream benchmark score. - Training record: training run `lo5clpo9`. - Machine-readable settings, results, and checkpoint lineage: [expert-provenance.json](expert-provenance.json). ## Evaluation | Checkpoint | Accuracy | Correct / scored completions | Truncation | |---|---:|---:|---:| | Pinned base model, before environment training | 39.00% | 156 / 400 | 0.25% | | Final expert, update 100 | **95.25%** | **381 / 400** | **3.25%** | Accuracy improved by **56.25 percentage points** from the pinned base model. Each evaluation uses **50 held-out tasks × 8 sampled completions = 400 scored completions**. Accuracy is the mean binary correctness across all completions, not best-of-eight or pass-at-eight. Truncated responses remain in the denominator and count as failures. The reported endpoint is the final checkpoint, not an earlier peak. The bank contains 114 training plaintexts from the finite approved corpus, with 50 separate validation plaintexts and no train/validation plaintext overlap. Validation uses a dedicated inference role every 10 optimizer updates with fixed seeds `[17, 43, 101, 137, 211, 307, 401, 503]`. All scheduled evaluations and final inference/full-state archival were verified against native receipts. The same validation bank was inspected during training; there was no separate final confirmation set. Results establish performance on held-out cases within this environment and do not establish cross-environment transfer or broad general reasoning improvement. ## Training and generation settings | Setting | Value | |---|---| | Training | Full-weight GRPO; Adam | | Learning rate | Constant `1e-6` | | Adam betas / weight decay | `(0.9, 0.98)` / `0.1` | | Policy clip epsilon / upper epsilon | `0.2` / `0.28` | | KL / entropy coefficients | `0.0` / `0.0` | | Batch per optimizer update | 8 questions × 8 completions = 64 responses | | Total consumed training responses | 6,400 | | Precision | BF16 model training and sampling | | Thinking | Enabled; `reasoning_effort="medium"` | | Sampling | Temperature `1.0`, top-p `0.95`, top-k `20` | | Maximum generated tokens | `16384` | | Context limit used in this experiment | `32768` | | Checkpoint / validation interval | Every 10 optimizer updates | | Async consumption | Ordered batches; maximum completed-update lag of 1 | For comparable text generation, use the supplied tokenizer and chat template, keep thinking enabled with `reasoning_effort="medium"`, and retain the sampling and output limits above. The original runtime pinned Transformers `5.12.1`; detailed runtime revisions are recorded in the provenance JSON. This expert was trained and evaluated on text inputs. Vision or video performance and retention of unrelated base-model capabilities were not evaluated. ## Provenance and reporting The source inference snapshot is `miles-qwen3.8-27b-06448b429f21/run-06448b429f21/training/eval-snapshots/step_99`. It was emitted at optimizer update 100 and verified complete in durable platform storage. The native final evaluation completed at `2026-10-19T13:25:51.973586+00:00`. This release contains inference weights and required inference assets; resumable optimizer state and raw training/evaluation traces remain in the experiment archives. W&B's final validation point is missing: its recorded validation history stops at update 90 (95.00%). An existing-run update permission restriction prevents reconciliation. The final **95.25% at update 100** reported here comes from verified native evaluation receipts and archived rollout evidence; the W&B gap does not change that measured endpoint. ## Intended use and limitations This is a specialized research checkpoint for text reasoning in the named environment. Its results depend on the frozen bank, prompt and grading contract, decoding settings, and token budget. Broader task transfer, production safety, multimodal behavior, and performance under different decoding settings have not been established by these experiments. ## License Apache-2.0, following the pinned base model. See the accompanying license and the [pinned base-model card](https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/README.md). ## Training and validation curves The plots below show the recorded training trajectory and verified validation results for this environment. Training uses a running average of 10 with the original series visible; validation is unsmoothed. No training, evaluation, or interpolation was performed to create these images. Training metric: `rollout/raw_reward` against `rollout/step`. Validation metric: `eval/validation` against completed optimizer updates. Both published experts contain 100 optimizer updates. Their training curves use the full native axis 0–99; validation includes every scheduled point from 0 through 100. Restored runs retain their original checkpoint ancestry and exclude abandoned branches. The overlay compares these recorded trajectories; it does not establish an optimizer-only causal effect. See each model’s settings and limitations above. ### ADAM training and validation [Final ADAM weights](https://huggingface.co/baseten/caesar-cipher-adam-qwen38_27b-expert) ![Caesar Cipher ADAM training](assets/environment-curves/20261024/adam-training.png) ![Caesar Cipher ADAM validation](assets/environment-curves/20261024/adam-validation.png) ### SGD training and validation [Final SGD weights](https://huggingface.co/baseten/caesar-cipher-sgd-qwen38_27b-expert) ![Caesar Cipher SGD training](assets/environment-curves/20261026/sgd-training.png) ![Caesar Cipher SGD validation](assets/environment-curves/20261024/sgd-validation.png) ### Adam and SGD comparison Adam is blue; SGD is pink. ![Caesar Cipher Adam and SGD training](assets/environment-curves/20261024/comparison-training.png) ![Caesar Cipher Adam and SGD validation](assets/environment-curves/20261024/comparison-validation.png) [Chart provenance and settings](assets/environment-curves/20261026/provenance.json). The original weight commit is `e4119e628b198e7da0bc188f047a4015e37b3579`.