arturoornelasb/triadic-gpt-40m

GitHub: arturoornelasb/triadic-microgpt | Paper: TriadicGPT (2026) | PyPI: triadic-head

TriadicGPT (40M parameters + 72-bit triadic head)

A 40M-parameter GPT language model trained end-to-end with a triadic projection head that produces discrete prime-factor signatures alongside standard next-token predictions.

Unlike the GPT-2 Medium variant, this model learns triadic representations end-to-end -- both the language model and the triadic head are trained jointly.

Checkpoint

Variant	Bit Accuracy	Dead Bits	Description
model_best.pt	93.0%	26	Best checkpoint (63-bit XL v2)

Architecture

• Model: Custom GPT (12 layers, d_model=512, ~40M params)
• Triadic head: Linear projection 512 -> 72 bits
• Activation: iFSQ (2*sigmoid(1.6x) - 1)

Training

• Data: WikiText-103 (~100M tokens)
• Loss: Language + triadic (supervised + subsumption + diversity + InfoNCE)
• Steps: 50,000
• Hardware: NVIDIA RTX 4060 Ti 16GB

Key Results

• Bit accuracy: 93.0%
• Analogy verification: 98%
• Signature uniqueness: 100%
• Subsumption: 98.3% (higher than 355M model at 76.9%)

Intended Use

Research on neurosymbolic AI, discrete representation learning, and interpretable semantic structures. Demonstrates that a small language model can learn algebraic prime-factor signatures end-to-end without catastrophic forgetting.

Limitations

• Trained on WikiText-103 only; generalization to other corpora not yet evaluated.
• Higher subsumption rate (98.3%) compared to the larger 355M model (76.9%) suggests a scale-algebra tradeoff that is not yet understood.
• D-A-N classifications are self-evaluated; independent rater validation pending.

Files

model_best.pt            # Checkpoint (153MB)
training_log.csv         # Per-step metrics
results.json             # Evaluation results
tokenizer.json           # Tokenizer
data/
  primitivos.json        # 72 semantic primitives (DAG, primes, bits)
  anclas.json            # Training anchors
  anclas_v2.json         # Extended anchors
LICENSE                  # BUSL-1.1

Related Models

• triadic-gpt2-medium -- Frozen GPT-2 Medium 355M + triadic head

Related Papers

1. Prime Factorization as a Neurosymbolic Bridge (Ornelas Brand, 2026a) -- Core algebraic framework
2. TriadicGPT (Ornelas Brand, 2026b) -- End-to-end learning (this model)
3. reptimeline (Ornelas Brand, 2026c) -- Representation lifecycle tracking
4. Emergent Duality (Ornelas Brand, 2026d) -- 14+ dualities across 6 algebraic layers (in progress)

Usage

import torch
checkpoint = torch.load("model_best.pt", map_location="cpu")
# Contains full model state dict (language model + triadic head)

For the full training/inference pipeline and desktop UI, see triadic-microgpt.

Citation

@article{ornelas2026triadicgpt,
  title={TriadicGPT: End-to-End Prime Factorization in a Generative Language Model},
  author={Ornelas Brand, J. Arturo},
  year={2026}
}

License

BUSL-1.1 -- free for individuals, academic, and non-profit use. Commercial production use requires a participation agreement. See COMMERCIAL.md.