--- language: - ko - en license: apache-2.0 library_name: aster pipeline_tag: text-generation base_model: Qwen/Qwen3.8-27B tags: - aster - causal-lm - hybrid-ssm - state-space-model - attention - knowledge-distillation - instruction-following - custom-runtime - safetensors - experimental --- # Aster-1.0-Instruct **A hybrid SSM–Attention language model by [Caramella Inc.](https://huggingface.co/CaramellaInc)** ([한국어](./README_ko.md)) | ([ENG](./README.md)) Aster-1.0-Instruct is an instruction-oriented checkpoint in the Aster language model family. Its architecture combines **independently initialized and trained State Space Model (SSM) mixers** with **Attention, token embedding, and LM head weights inherited from [Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B)**. Aster-1.0 is an experimental, partially pretrained hybrid language model. It is neither an unchanged Qwen fine-tune nor a model pretrained entirely from random initialization. ## Project Vision Aster is a language model research project pursuing progressively independent architectures, training methods, and learned representations. At the beginning of the project, **Qwen3.8-27B was selected as the reference architecture**, providing a high-capability model whose layer organization, representation dimensions, and pretrained parameters could serve as a foundation for architectural experimentation. The intention was not to reproduce Qwen as the final product, but to establish a technically grounded starting point for developing Aster's own language models. This approach follows three principles: - **Architectural reference:** Start from an established high-capability model design to make early experiments more controlled and comparable. - **Representation bootstrap:** Retain selected pretrained parameters so new SSM components can learn alongside an existing language representation framework. - **Progressive independence:** Develop and evaluate new sequence mixers, training methods, and representations while reducing reliance on inherited components over successive generations. The inherited parameters provide a starting point, not a guarantee that the teacher model's reasoning, language capabilities, or internal states are preserved. ## Model Overview | Property | Description | | --- | --- | | Developer | Caramella Inc. | | Model | Aster-1.0-Instruct | | Model family | Aster 1.0 | | Type | Decoder-only causal language model | | Architecture | Hybrid SSM–Attention | | Parameters | Approximately 28.9B | | Backbone | 48 SSM mixer layers + 16 Attention layers (64 layers total) | | Languages | Korean, English | | Input / output | Text → text | | Weight format | Safetensors | | Inference runtime | Custom Aster runtime (Rust) | | License | Apache-2.0 | ## Architecture Aster-1.0 replaces the reference model's Gated DeltaNet mixers with **SSM mixers initialized and trained as new Aster parameters**. The model retains selected pretrained components to provide a shared representational starting point. ```text Input tokens │ ▼ Token Embedding (inherited from Qwen3.8-27B) │ ▼ ┌─────────────────────────────────────────┐ │ Aster-1.0 Hybrid Backbone │ │ │ │ 48 × independently trained SSM mixers │ │ 16 × inherited Attention layers │ │ (64 layers total) │ └─────────────────────────────────────────┘ │ ▼ LM Head (inherited from Qwen3.8-27B) │ ▼ Next-token logits ``` The diagram summarizes the model's components rather than their precise execution order. **Aster-1.0 uses SSM mixers; GDN-2 is a separate direction for subsequent Aster research.** ### Parameter Provenance | Component | Initialization | | --- | --- | | SSM mixers | Newly initialized and trained for Aster-1.0 | | Attention layers | Pretrained Qwen3.8-27B weights | | Token Embedding | Pretrained Qwen3.8-27B weights | | LM Head | Pretrained Qwen3.8-27B weights | Other components are not assigned a provenance in this table. ## Training Approach Aster-1.0 combines **weight transfer** with **cross-architecture knowledge distillation** to investigate whether newly initialized SSM mixers can learn effectively within a backbone containing pretrained language-model components. - **Weight transfer** initializes compatible components from a pretrained checkpoint. - **Knowledge distillation** trains a student model using signals derived from a teacher, such as intermediate representations or output distributions. - **Independent SSM training** means Aster's SSM mixer parameters were not copied from Qwen's Gated DeltaNet mixers. The project draws on ideas from **MOHAWK**, which studies transferring capabilities across sequence-model architectures. The Instruct checkpoint represents an instruction-oriented release of this research line. The resulting model is intended as a foundation for studying architectural adaptation and the longer-term development of Aster's own models, rather than as evidence that the teacher's full capabilities have been preserved. ## Download and Runtime Download the checkpoint using the Hugging Face CLI: ```bash hf download CaramellaInc/Aster-1.0-instruct \ --local-dir ./Aster-1.0-instruct ``` Aster-1.0 uses a **custom Rust inference runtime** and a hybrid recurrent architecture. Standard Qwen loading paths in Hugging Face Transformers or vLLM are not interchangeable with the Aster runtime without specific architecture support. Inference should use the tokenizer, model configuration, and chat formatting associated with the Aster checkpoint and its runtime. ## Intended Use - Research on hybrid SSM–Attention language models - Experiments in cross-architecture knowledge transfer and language-model adaptation - Korean and English text generation and instruction-following research - Inspection and reproduction of Aster checkpoints using the compatible runtime ## Limitations Aster-1.0-Instruct is a research checkpoint. Its architecture and parameter lineage do not, by themselves, establish improvements in language understanding, reasoning, long-context retrieval, latency, or memory efficiency. Generated outputs may be inaccurate or misleading. Use human review and independent testing before applying the model in high-stakes settings. The model does not claim persistent semantic memory, online continual learning, or automatic consolidation of recurrent state into long-term weights. ## Model Lineage and License Aster-1.0-Instruct is developed by **Caramella Inc.** with selected pretrained parameters inherited from **[Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B)**. The model is distributed under **Apache-2.0**, consistent with the repository's license metadata and the referenced upstream model's license. Aster's SSM mixers are independently initialized and trained, while the inherited Attention, token embedding, and LM head remain part of the model's documented lineage. Applicable upstream copyright, license, attribution, and modification-notice requirements continue to apply. ### References - [Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) - [Aster-1.0-Base](https://huggingface.co/CaramellaInc/Aster-1.0-base) - [Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Models (MOHAWK)](https://arxiv.org/abs/2408.10189) ## Citation ```bibtex @misc{caramella_aster_1_0_instruct_2026, title = {Aster-1.0-Instruct: A Hybrid SSM-Attention Language Model}, author = {{Caramella Inc.}}, year = {2026}, howpublished = {Hugging Face model repository}, url = {https://huggingface.co/CaramellaInc/Aster-1.0-instruct} } ```