iCoder-27B-Q4_K_M (GGUF)

Quantized GGUF version of iCoder-27B by the i-Coder team.
Original model: i-Coder/iCoder-27B · Base: Qwen/Qwen3.6-27B

This repository contains the GGUF quantized weights of iCoder-27B, a 27B-parameter vision-language model for industrial coding (RTL design, GPU kernel optimization). The model was quantized to Q4_K_M (main model) and F16 (vision projector) for efficient local inference on consumer GPUs.

---

Original Model

iCoder-27B is a 27B-parameter model fine-tuned from Qwen3.6-27B. It uses a hybrid linear-attention architecture (Gated Delta Net) with 64 text layers and a separate vision encoder. The model was developed through an agent-driven pipeline combining supervised fine-tuning, self-distillation, and reinforcement learning with verifiable rewards.

Original authors: Cheng Yang, Jiayang Lyu, Shangyuan Liu, Guibin Zhang, Jiong Lin, Xinlei Yu, Junchi Yan, Shuicheng Yan, Weinan E, Linfeng Zhang, Qibing Ren.

Paper: arXiv:2609.29626

Original repo: i-Coder/iCoder-27B

---

Quantization Details

File	Params	Precision	Notes
icoder-27b-q4_k_m.gguf	27B	Q4_K_M	Main model (text + vision fusion)
icoder-mmproj-f16.gguf	~927M	F16	Vision projector (Qwen3VL)

• Quantization method: llama.cpp quantize with Q4_K_M scheme (4-bit mixed precision with higher precision for attention/embd layers).
• Imatrix: Custom importance matrix collected from representative coding prompts.
• Vision projector: Kept at F16 to preserve image understanding quality.
• No MTP head: This model has no speculative decoding head (mtp_num_hidden_layers = 0), converted with --no-mtp.

---

Usage (llama.cpp)

llama-server \
  -m icoder-27b-q4_k_m.gguf \
  --mmproj icoder-mmproj-f16.gguf \
  --port 8090 \
  --ctx-size 65536 \
  --n-gpu-layers 99 \
  --flash-attn on \
  --cache-type-k q4_0 \
  --cache-type-v q4_0 \
  --kv-unified \
  --no-mmproj-offload \
  --spec-default

• VRAM: ~18–20 GB (RTX 3090 compatible)
• Context: 64K tokens (KV cache in q4_0)
• Vision: --no-mmproj-offload keeps the 927 MB projector in system RAM, freeing VRAM for the main model.

---

Benchmarks (Original Model)

Benchmark	Metric	iCoder-27B	Qwen3.6-27B	DeepSeek-V4-Pro
RTLLM	Functional avg@4	68.0	49.6	67.5
TritonBench-G	Correctness pass@1	20.1	11.4	19.0
KernelBench L1	Correct (%)	61	32	32
KernelBench L2	Correct (%)	74	28	40

Full benchmark table in the original model card.

---

Citation

@article{yang2026icoder,
  title   = {iCoder-27B: Recursive AI-Led Development of Frontier Industrial Coding Model},
  author  = {Cheng Yang and Jiayang Lyu and Shangyuan Liu and Guibin Zhang and
             Jiong Lin and Xinlei Yu and Junchi Yan and Shuicheng Yan and
             Weinan E and Linfeng Zhang and Qibing Ren},
  journal = {arXiv preprint arXiv:2609.29626},
  year    = {2026}
}

---

License

Apache-2.0 (inherited from the original iCoder-27B / Qwen3.6-27B).

---

Acknowledgements

• Original model and training: i-Coder team
• Base model: Qwen3.6-27B by Qwen
• Quantization tools: llama.cpp