Update README.md

README.md

@@ -1,6 +1,604 @@
 ---
+
 license: apache-2.0
 base_model:
-- Qwen/Qwen3.8-27B
+- trohrbaugh/Qwen3.8-27B-heretic-ara
+tags:
+- qwen
+- qwen3.8
+- qwen3.8-27b
+- gguf
+- rocm
+- rocmfpx
+- activefpx
+- promptforge
+- amd
+- strix-halo
+- gfx1151
+- mtp
+- heretic
+- uncensored
+
 ---
-Based on https://huggingface.co/jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge for inference on Strix Halo 128 GB systems. Base model is from https://huggingface.co/trohrbaugh/Qwen3.8-27B-heretic-ara.
+
+# Qwen3.8-27B-Heretic-ROCmFPX-STRIX
+
+ROCmFPX / ActiveFPX build of [`trohrbaugh/Qwen3.8-27B-heretic-ara`](https://huggingface.co/trohrbaugh/Qwen3.8-27B-heretic-ara), targeting AMD Strix Halo / Radeon 8060S (`gfx1151`) and the specialized [CIRU ROCmFPX](https://github.com/ciru-ai/ROCmFPX) + PromptForge inference stack.
+
+This repository contains:
+
+* a compact ROCmFPX / ActiveFPX GGUF using the CIRU Qwen3.8 v3-style precision allocation;
+* a model-specific full-W8 PromptForge FFN sidecar;
+* a model-specific full-W8 PromptForge Gated DeltaNet sidecar.
+
+The GGUF can be used on its own with a compatible ROCmFPX-capable llama.cpp runtime. The `.pfs` files are optional serving-time compute views for CIRU PromptForge and trade additional memory for substantially faster supported prompt-processing paths.
+
+## Model provenance
+
+### Original model
+
+The architecture and original model weights originate from:
+
+[`Qwen/Qwen3.8-27B`](https://huggingface.co/Qwen/Qwen3.8-27B)
+
+Qwen3.8-27B is a dense 27B model with:
+
+* 64 language-model layers;
+* hidden dimension 5120;
+* FFN intermediate dimension 17,408;
+* a repeating hybrid Gated DeltaNet / gated-attention layout;
+* native MTP support;
+* a 248,320-token padded vocabulary;
+* native 262,144-token context;
+* native multimodal architecture.
+
+### Heretic / ARA source
+
+The full-precision source used for this repository is:
+
+[`trohrbaugh/Qwen3.8-27B-heretic-ara`](https://huggingface.co/trohrbaugh/Qwen3.8-27B-heretic-ara)
+
+That model is a behavior-modified Qwen3.8-27B produced using the [Heretic](https://github.com/p-e-w/heretic) project and its Arbitrary-Rank Ablation (ARA) approach.
+
+This repository does **not** perform an additional fine-tune. The ROCmFPX GGUF and PromptForge sidecars were generated from the Heretic-Ara BF16 weights.
+
+## Quantization and inference provenance
+
+The quantization format and specialized inference implementation come from:
+
+* [CIRU ROCmFPX](https://github.com/ciru-ai/ROCmFPX)
+* [CIRU Qwen3.8 ActiveFPX + PromptForge reference release](https://huggingface.co/jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge)
+* [llama.cpp](https://github.com/ggml-org/llama.cpp)
+* [AMD Composable Kernel](https://github.com/ROCm/composable_kernel)
+
+CIRU's Qwen3.8 work adds model-specific `gfx1151` paths including:
+
+* fused FFN gate/up processing;
+* accelerated SwiGLU/down projection;
+* merged QKV/Z projection for Gated DeltaNet layers;
+* prepacked W8 PromptForge compute views;
+* small-row W8 execution;
+* ROCmFPX low-bit decode paths;
+* native Qwen3.8 MTP support.
+
+## About this v3-style quant
+
+The compact GGUF in this repository was generated from the Heretic-Ara BF16 source using the **per-tensor precision allocation from CIRU's Qwen3.8 v3 release** as the reference.
+
+That includes the important v3 storage decisions such as:
+
+* Q8 output projection;
+* selected LateQ6 precision islands;
+* the remaining mixed ROCmFPX tensor allocation.
+
+### FFNEQ note
+
+CIRU's official v3 model is named:
+
+`Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-v3-Q8-FFNEQ-LateQ6.gguf`
+
+The `FFNEQ` part refers to an FFN-equalization treatment in CIRU's model-production process.
+
+This Heretic build reproduces the **v3 tensor-type / precision allocation**, but does **not claim to reproduce CIRU's unpublished FFNEQ weight transformation** unless explicitly stated otherwise.
+
+In other words:
+
+```text
+Heretic-Ara BF16
+       |
+       +--> CIRU v3-style Q8 / LateQ6 / ROCmFPX allocation
+       |         |
+       |         +--> compact GGUF
+       |
+       +--> BF16-derived PromptForge W8 views
+                 |
+                 +--> FFN.pfs
+                 +--> GDN.pfs
+```
+
+## PromptForge sidecars
+
+The `.pfs` files are **not independent models** and should not be mixed with another Qwen3.8 checkpoint.
+
+They are prepacked W8 compute views generated from this model's BF16 weights.
+
+### FFN sidecar
+
+Contains W8 prompt-time views of:
+
+```text
+ffn_gate
+ffn_up
+ffn_down
+```
+
+for all 64 language-model layers.
+
+The fixed Qwen3.8 PromptForge FFN ABI uses a file size of:
+
+```text
+17,123,004,416 bytes
+```
+
+### GDN sidecar
+
+Contains the merged W8:
+
+```text
+attn_qkv + attn_gate/Z
+```
+
+views used by the 48 Gated DeltaNet layers.
+
+The fixed GDN sidecar size is:
+
+```text
+4,029,685,760 bytes
+```
+
+## Output-K8 status
+
+CIRU's official v3 release also includes a third sidecar:
+
+```text
+PromptForge-Output-K8.pfs
+```
+
+That file provides a draft-only top-8 output shortlist used by the v3 MTP path before authoritative Q8 reranking.
+
+**This repository does not currently include a Heretic-specific Output-K8 proxy.**
+
+Do not use CIRU's Output-K8 sidecar with this model: it was generated from CIRU's output weights, not the Heretic-Ara weights.
+
+The FFN and GDN PromptForge paths work independently of Output-K8.
+
+---
+
+# Running with the CIRU PromptForge runtime
+
+The optimized path is intended primarily for AMD Strix Halo / Radeon 8060S (`gfx1151`).
+
+CIRU developed and validated the Qwen3.8 PromptForge work with a pinned TheRock ROCm 7.15 environment and a pinned Composable Kernel revision.
+
+A sufficiently compatible ROCm toolchain with working `gfx1151` HIP support is required.
+
+## 1. Download this repository
+
+Install the Hugging Face CLI if necessary:
+
+```bash
+python3 -m pip install -U huggingface_hub
+```
+
+Download the repository:
+
+```bash
+export MODEL_DIR="$HOME/models/Qwen3.8-27B-Heretic-ROCmFPX-STRIX"
+
+mkdir -p "$MODEL_DIR"
+
+hf download \
+  tonyrishwain/Qwen3.8-27B-Heretic-ROCmFPX-STRIX \
+  --local-dir "$MODEL_DIR"
+```
+
+Find the artifacts:
+
+```bash
+export MODEL="$(find "$MODEL_DIR" -maxdepth 1 -type f -name '*.gguf' | head -n1)"
+export FFN="$(find "$MODEL_DIR" -maxdepth 1 -type f -iname '*ffn*.pfs' | head -n1)"
+export GDN="$(find "$MODEL_DIR" -maxdepth 1 -type f -iname '*gdn*.pfs' | head -n1)"
+
+printf 'MODEL=%s\nFFN=%s\nGDN=%s\n' \
+  "$MODEL" "$FFN" "$GDN"
+```
+
+Verify the PFS sizes:
+
+```bash
+stat -c '%n %s' "$FFN" "$GDN"
+```
+
+Expected:
+
+```text
+FFN: 17123004416
+GDN: 4029685760
+```
+
+## 2. Install build dependencies
+
+### Fedora
+
+```bash
+sudo dnf install -y \
+  @development-tools \
+  cmake \
+  ninja-build \
+  git \
+  python3 \
+  libcurl-devel
+```
+
+You also need a ROCm/HIP installation capable of compiling and running `gfx1151`.
+
+CIRU's published v3 measurements use ROCm 7.15. Distribution-provided ROCm versions may differ.
+
+Verify your GPU:
+
+```bash
+rocminfo | grep -m1 gfx1151
+```
+
+You should see `gfx1151`.
+
+## 3. Download CIRU's v3 runtime patch
+
+The v3 runtime is built from CIRU's public v2.3 tag plus the v3 Output-K8 patch.
+
+Even though this model does not currently use Output-K8, using the current patched runtime keeps the execution stack aligned with CIRU v3.
+
+```bash
+export CIRU_PATCH="$HOME/ciru-qwen38-v3"
+
+mkdir -p "$CIRU_PATCH"
+
+hf download \
+  jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge \
+  runtime/qwen38-v3-output-k8-runtime.patch \
+  --local-dir "$CIRU_PATCH"
+```
+
+## 4. Clone ROCmFPX
+
+```bash
+mkdir -p "$HOME/src"
+cd "$HOME/src"
+
+git clone https://github.com/ciru-ai/ROCmFPX.git
+cd ROCmFPX
+
+git checkout qwen3.8-activefpx-promptforge-v2.3
+
+git am \
+  "$CIRU_PATCH/runtime/qwen38-v3-output-k8-runtime.patch"
+```
+
+## 5. Clone the pinned Composable Kernel
+
+```bash
+cd "$HOME/src"
+
+git clone https://github.com/ROCm/composable_kernel.git
+
+git -C composable_kernel checkout \
+  fdf4bb7fcc984811cef48ce817d89aac064b984a
+```
+
+Set:
+
+```bash
+export ROCMFPX="$HOME/src/ROCmFPX"
+export CK="$HOME/src/composable_kernel"
+```
+
+## 6. Configure ROCm
+
+If your ROCm installation is under `/opt/rocm`:
+
+```bash
+export ROCM_PATH=/opt/rocm
+export PATH="$ROCM_PATH/bin:$ROCM_PATH/llvm/bin:$PATH"
+export LD_LIBRARY_PATH="$ROCM_PATH/lib:$ROCM_PATH/lib/llvm/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
+```
+
+Check:
+
+```bash
+"$ROCM_PATH/bin/rocminfo" | grep -m1 gfx1151
+```
+
+On Fedora or a custom TheRock installation, `ROCM_PATH` may be somewhere else. Adjust the path accordingly.
+
+## 7. Build the PromptForge server
+
+```bash
+cd "$ROCMFPX"
+
+cmake -S . -B build-promptforge -G Ninja \
+  -DCMAKE_BUILD_TYPE=Release \
+  -DCMAKE_C_COMPILER="$ROCM_PATH/llvm/bin/clang" \
+  -DCMAKE_CXX_COMPILER="$ROCM_PATH/llvm/bin/clang++" \
+  -DCMAKE_HIP_COMPILER="$ROCM_PATH/llvm/bin/clang++" \
+  -DCMAKE_PREFIX_PATH="$ROCM_PATH" \
+  -DGGML_HIP=ON \
+  -DGGML_CUDA=OFF \
+  -DGGML_VULKAN=OFF \
+  -DGGML_HIP_FORCE_MMQ=ON \
+  -DGGML_HIP_GRAPHS=ON \
+  -DGGML_HIP_MMQ_MFMA=ON \
+  -DGGML_HIP_NO_VMM=ON \
+  -DGGML_HIP_ROCWMMA_FATTN=OFF \
+  -DGGML_NATIVE=ON \
+  -DAMDGPU_TARGETS=gfx1151 \
+  -DGPU_BUILD_TARGETS=gfx1151 \
+  -DPROMPTFORGE_CK_ROOT="$CK" \
+  -DGGML_BUILD_TESTS=OFF \
+  -DLLAMA_BUILD_TESTS=OFF \
+  -DLLAMA_BUILD_SERVER=ON
+```
+
+Build:
+
+```bash
+cmake --build build-promptforge \
+  --target llama-server \
+  -j"$(nproc)"
+```
+
+## 8. Configure PromptForge for this model
+
+Enable the two model-specific sidecars:
+
+```bash
+export PROMPTFORGE_SIDECAR="$FFN"
+export PROMPTFORGE_GDN_SIDECAR="$GDN"
+
+export PROMPTFORGE_MODE=m2048_fused_tail1476
+export PROMPTFORGE_ENABLE_SMALLM_W8=1
+
+export GGML_CUDA_GRAPH_OPT=0
+export HIP_VISIBLE_DEVICES=0
+export HSA_OVERRIDE_GFX_VERSION=11.5.1
+```
+
+Because this repository does **not** contain a Heretic-specific Output-K8 proxy, explicitly disable that path:
+
+```bash
+unset PROMPTFORGE_MTP_OUTPUT_K8
+unset PROMPTFORGE_MTP_OUTPUT_K8_VALIDATE
+unset PROMPTFORGE_MTP_OUTPUT_K8_PROXY
+```
+
+Also keep the experimental IU4/ngram paths disabled:
+
+```bash
+unset PROMPTFORGE_IU4_SIDECAR
+unset PROMPTFORGE_GDN_IU4_SIDECAR
+unset PROMPTFORGE_ENABLE_NGRAM_MOD
+unset PROMPTFORGE_ENABLE_NGRAM_M65_IU4
+```
+
+For the pre-Output-K8 MTP selection path:
+
+```bash
+export LLAMA_MTP_CPU_ARGMAX_FASTPATH=1
+unset LLAMA_MTP_BACKEND_GREEDY
+```
+
+Set runtime library paths:
+
+```bash
+export LD_LIBRARY_PATH="$ROCMFPX/build-promptforge/bin:$ROCM_PATH/lib:$ROCM_PATH/lib/llvm/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
+```
+
+## 9. Start the server
+
+```bash
+cd "$ROCMFPX"
+
+./build-promptforge/bin/llama-server \
+  -m "$MODEL" \
+  --alias heretic \
+  --host 127.0.0.1 \
+  --port 8080 \
+  --jinja \
+  -fit off \
+  -dev ROCm0 \
+  -ngl 999 \
+  -c 262144 \
+  -b 2048 \
+  -ub 2048 \
+  -fa on \
+  -ctk f16 \
+  -ctv f16 \
+  -t 16 \
+  -tb 16 \
+  -np 1 \
+  -ctxcp 0 \
+  --cache-ram 0 \
+  --no-cache-prompt \
+  --no-cache-idle-slots \
+  --timeout 3600 \
+  --metrics \
+  --spec-type draft-mtp \
+  --spec-draft-device ROCm0 \
+  --spec-draft-ngl 999 \
+  --spec-draft-type-k f16 \
+  --spec-draft-type-v f16 \
+  --spec-draft-n-max 4 \
+  --spec-draft-n-min 0 \
+  --spec-draft-p-min 0.0 \
+  --spec-draft-p-split 0.10 \
+  --spec-draft-backend-sampling \
+  --temp 0.7 \
+  --top-p 0.95 \
+  --reasoning off \
+  --reasoning-format none \
+  --reasoning-budget -1
+```
+
+The OpenAI-compatible API will be available at:
+
+```text
+http://127.0.0.1:8080/v1
+```
+
+## 10. Test the server
+
+```bash
+curl http://127.0.0.1:8080/v1/chat/completions \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "model": "heretic",
+    "messages": [
+      {
+        "role": "user",
+        "content": "Explain why the sky appears blue."
+      }
+    ],
+    "max_tokens": 256,
+    "temperature": 0.7
+  }'
+```
+
+## Smaller context configurations
+
+The CIRU v3 reference profile uses:
+
+```text
+-c 262144
+```
+
+which is appropriate for a high-memory Strix Halo configuration but consumes substantial unified memory.
+
+If you do not need the full context window, you can reduce it, for example:
+
+```text
+-c 65536
+```
+
+or:
+
+```text
+-c 131072
+```
+
+This does not change the GGUF or PFS files; it only changes runtime memory allocation.
+
+---
+
+# GGUF-only operation
+
+The `.pfs` files are optional.
+
+To run the compact model without PromptForge:
+
+```bash
+unset PROMPTFORGE_SIDECAR
+unset PROMPTFORGE_GDN_SIDECAR
+unset PROMPTFORGE_MODE
+unset PROMPTFORGE_ENABLE_SMALLM_W8
+unset PROMPTFORGE_MTP_OUTPUT_K8
+unset PROMPTFORGE_MTP_OUTPUT_K8_PROXY
+```
+
+Then start a compatible llama.cpp / ROCmFPX server with the GGUF normally.
+
+The GGUF remains the authoritative model representation. PromptForge sidecars are alternate compute views used only on qualified execution paths.
+
+---
+
+# Vision support
+
+Qwen3.8-27B is natively multimodal, but this repository currently focuses on the language-model GGUF and PromptForge language-model sidecars.
+
+No vision projector is assumed by the launch command above.
+
+If adding multimodal support, use a compatible Qwen3.8 projector whose provenance has been verified against the source checkpoint rather than blindly mixing unrelated model artifacts.
+
+---
+
+# Why the PFS files are model-specific
+
+PromptForge does not simply contain generic kernels.
+
+The sidecars contain quantized/prepacked views of the actual model weights.
+
+Therefore:
+
+```text
+Heretic GGUF + Heretic PFS     correct
+
+CIRU GGUF + CIRU PFS           correct
+
+Heretic GGUF + CIRU PFS        incorrect
+```
+
+Mixing sidecars between different fine-tunes or ablations may silently cause PromptForge execution to use weights from a different model.
+
+Always keep the GGUF and `.pfs` files from the same repository/build together.
+
+---
+
+# Hardware target
+
+Primary target:
+
+```text
+AMD Ryzen AI Max+ 395
+Radeon 8060S
+gfx1151
+Strix Halo
+```
+
+The ROCmFPX formats themselves may work on other AMD hardware, but PromptForge contains specialized model- and kernel-level paths and should not be assumed portable without testing.
+
+CIRU's published Qwen3.8 performance work was developed specifically around `gfx1151`.
+
+---
+
+# Notes on "uncensored"
+
+This repository inherits the behavioral modification of the Heretic-Ara source model.
+
+"Uncensored" or "decensored" describes the model-building goal; it is not a guarantee that every prompt will produce a particular response, nor that the model is more accurate than the original Qwen checkpoint.
+
+Quantization can also introduce small numerical differences relative to BF16.
+
+Users should evaluate the model for their own workloads.
+
+---
+
+# Credits
+
+Model:
+
+* [Qwen / Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B)
+* [trohrbaugh / Qwen3.8-27B-heretic-ara](https://huggingface.co/trohrbaugh/Qwen3.8-27B-heretic-ara)
+* [p-e-w / Heretic](https://github.com/p-e-w/heretic)
+
+Quantization and inference:
+
+* [CIRU ROCmFPX](https://github.com/ciru-ai/ROCmFPX)
+* [CIRU Qwen3.8 ActiveFPX + PromptForge](https://huggingface.co/jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge)
+* [llama.cpp](https://github.com/ggml-org/llama.cpp)
+* [AMD Composable Kernel](https://github.com/ROCm/composable_kernel)
+
+Thanks to the Qwen team, the Heretic contributors, CIRU Inference Lab, llama.cpp contributors, and AMD ROCm/Composable Kernel contributors.
+
+## License
+
+The upstream Qwen3.8 model is released under Apache-2.0. This derived model repository follows the applicable upstream model license.
+
+Runtime projects and third-party components retain their own respective licenses.