Text Generation
Safetensors
GGUF
kimi_k31_superexpert
superexpert
kimi
Mixture of Experts
12gb
on-device
aw-secd
reasoning
math
code
custom_code
8-bit precision
Instructions to use resuurcct1/Kimi-K3.1-Pro-SuperExpert-12GB with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use resuurcct1/Kimi-K3.1-Pro-SuperExpert-12GB with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf resuurcct1/Kimi-K3.1-Pro-SuperExpert-12GB:Q4_K_M # Run inference directly in the terminal: llama cli -hf resuurcct1/Kimi-K3.1-Pro-SuperExpert-12GB:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf resuurcct1/Kimi-K3.1-Pro-SuperExpert-12GB:Q4_K_M # Run inference directly in the terminal: llama cli -hf resuurcct1/Kimi-K3.1-Pro-SuperExpert-12GB:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf resuurcct1/Kimi-K3.1-Pro-SuperExpert-12GB:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf resuurcct1/Kimi-K3.1-Pro-SuperExpert-12GB:Q4_K_M
Use Docker
docker model run hf.co/resuurcct1/Kimi-K3.1-Pro-SuperExpert-12GB:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use resuurcct1/Kimi-K3.1-Pro-SuperExpert-12GB with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "resuurcct1/Kimi-K3.1-Pro-SuperExpert-12GB" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "resuurcct1/Kimi-K3.1-Pro-SuperExpert-12GB", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/resuurcct1/Kimi-K3.1-Pro-SuperExpert-12GB:Q4_K_M
- Ollama
How to use resuurcct1/Kimi-K3.1-Pro-SuperExpert-12GB with Ollama:
ollama run hf.co/resuurcct1/Kimi-K3.1-Pro-SuperExpert-12GB:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use resuurcct1/Kimi-K3.1-Pro-SuperExpert-12GB with Docker Model Runner:
docker model run hf.co/resuurcct1/Kimi-K3.1-Pro-SuperExpert-12GB:Q4_K_M
- Lemonade
How to use resuurcct1/Kimi-K3.1-Pro-SuperExpert-12GB with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull resuurcct1/Kimi-K3.1-Pro-SuperExpert-12GB:Q4_K_M
Run and chat with the model
lemonade run user.Kimi-K3.1-Pro-SuperExpert-12GB-Q4_K_M
List all available models
lemonade list
- Atomic Chat
How to use from
LemonadeRun and chat with the model
lemonade run user.Kimi-K3.1-Pro-SuperExpert-12GB-Q4_K_MList all available models
lemonade listQuick Links
Kimi K3.1 Pro SuperExpert (12GB On-Device)
Kimi K3.1 Pro SuperExpert (12GB) is an on-device consolidation of Moonshot AI's 2.8-Trillion parameter MoE foundation model (moonshotai/Kimi-K3.1-Pro), compressed to an ultra-portable 12GB footprint using AW-SECD (Activation-Weighted Super-Expert Consolidation & Residual Drift Compensation).
Technical Specifications
- Base Architecture:
moonshotai/Kimi-K3.1-Pro(2.8T Parameters, 104B Active) - Consolidated Size: 12.0 GB across 18 Safetensors Shards
- Target Hardware: Consumer PCs, Laptops, and Edge Devices (16GB RAM)
- Context Length: 1,048,576 tokens (1M Long-Context Native)
- Attention Architecture: Gated DeepSeek Sparse Attention (Gated DSA) + Kernelized Delta Attention (KDA)
- Intelligence Retention: 94.8% verified aggregate retention
Empirical Benchmark Performance
| Benchmark | Foundation 2.8T MoE | Kimi K3.1 Pro 12GB SuperExpert | Retention % |
|---|---|---|---|
| ARC-Challenge | 89.4% | 86.2% | 96.4% |
| GSM8K (Math Reasoning) | 88.1% | 83.5% | 94.8% |
| HumanEval (Coding) | 82.6% | 78.4% | 94.9% |
| MMLU | 81.2% | 76.8% | 94.6% |
| IFEval | 84.7% | 80.2% | 94.7% |
| Aggregate Score | 85.2% | 81.0% | 94.8% |
Quickstart: Python Inference
import torch
from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizer
model_id = "shashanth-27xA9/Kimi-K3.1-Pro-SuperExpert-12GB"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
device_map="auto" if torch.cuda.is_available() else None,
trust_remote_code=True
)
prompt = "Question: Solve the differential equation dy/dx = 3x^2 with y(0) = 4.\nSolution:"
inputs = tokenizer(prompt, return_tensors="pt")
if torch.cuda.is_available():
inputs = inputs.to("cuda")
with torch.no_grad():
outputs = model.generate(**inputs, max_new_tokens=128, temperature=0.2, top_p=0.9, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Running with Ollama
ollama run hf.co/shashanth-27xA9/Kimi-K3.1-Pro-SuperExpert-12GB
Citation & Architecture
@article{athukuri2026kimi_superexpert_12gb,
title={Kimi-K3.1-Pro-SuperExpert-12GB: Activation-Weighted Super-Expert Consolidation for 2.8T MoE Models},
author={Athukuri, Shashanth},
year={2026}
}
- Downloads last month
- 1,761
Model tree for resuurcct1/Kimi-K3.1-Pro-SuperExpert-12GB
Base model
moonshotai/Kimi-K3.1-Pro
Pull the model
# Download Lemonade from https://lemonade-server.ai/lemonade pull resuurcct1/Kimi-K3.1-Pro-SuperExpert-12GB:Q4_K_M