Instructions to use huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF:IQ1_S # Run inference directly in the terminal: llama cli -hf huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF:IQ1_S
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF:IQ1_S # Run inference directly in the terminal: llama cli -hf huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF:IQ1_S
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF:IQ1_S # Run inference directly in the terminal: ./build/bin/llama cli -hf huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF:IQ1_S
Use Docker
docker model run hf.co/huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF:IQ1_S
- Llama new
- LM Studio
- Jan
- vLLM
How to use huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF:IQ1_S
- SGLang
How to use huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Ollama
How to use huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF with Ollama:
ollama run hf.co/huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF:IQ1_S
- Unsloth Desktop
- Pi
How to use huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF:IQ1_S
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF:IQ1_S" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF with Docker Model Runner:
docker model run hf.co/huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF:IQ1_S
- Lemonade
How to use huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF:IQ1_S
Run and chat with the model
lemonade run user.Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF-IQ1_S
List all available models
lemonade list
- Hermes Agent
How to use huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF:IQ1_S
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF:IQ1_S
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF:IQ1_S
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF:IQ1_S" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Download .gitattributes from huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF: direct link, hf CLI and curl.
- Browser
- Download file 5.04 kB
-
https://huggingface.co/huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF/resolve/main/.gitattributes
- Command line
-
hf download hf://huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF/.gitattributes
-
curl -L -o .gitattributes https://huggingface.co/huihui-ai/Huihui-DeepSeek-V4.1-Flash-1017-abliterated-GGUF/resolve/main/.gitattributes
| *.7z filter=lfs diff=lfs merge=lfs -text | |
| *.arrow filter=lfs diff=lfs merge=lfs -text | |
| *.bin filter=lfs diff=lfs merge=lfs -text | |
| *.bz2 filter=lfs diff=lfs merge=lfs -text | |
| *.ckpt filter=lfs diff=lfs merge=lfs -text | |
| *.ftz filter=lfs diff=lfs merge=lfs -text | |
| *.gz filter=lfs diff=lfs merge=lfs -text | |
| *.h5 filter=lfs diff=lfs merge=lfs -text | |
| *.joblib filter=lfs diff=lfs merge=lfs -text | |
| *.lfs.* filter=lfs diff=lfs merge=lfs -text | |
| *.mlmodel filter=lfs diff=lfs merge=lfs -text | |
| *.model filter=lfs diff=lfs merge=lfs -text | |
| *.msgpack filter=lfs diff=lfs merge=lfs -text | |
| *.npy filter=lfs diff=lfs merge=lfs -text | |
| *.npz filter=lfs diff=lfs merge=lfs -text | |
| *.onnx filter=lfs diff=lfs merge=lfs -text | |
| *.ot filter=lfs diff=lfs merge=lfs -text | |
| *.parquet filter=lfs diff=lfs merge=lfs -text | |
| *.pb filter=lfs diff=lfs merge=lfs -text | |
| *.pickle filter=lfs diff=lfs merge=lfs -text | |
| *.pkl filter=lfs diff=lfs merge=lfs -text | |
| *.pt filter=lfs diff=lfs merge=lfs -text | |
| *.pth filter=lfs diff=lfs merge=lfs -text | |
| *.rar filter=lfs diff=lfs merge=lfs -text | |
| *.safetensors filter=lfs diff=lfs merge=lfs -text | |
| saved_model/**/* filter=lfs diff=lfs merge=lfs -text | |
| *.tar.* filter=lfs diff=lfs merge=lfs -text | |
| *.tar filter=lfs diff=lfs merge=lfs -text | |
| *.tflite filter=lfs diff=lfs merge=lfs -text | |
| *.tgz filter=lfs diff=lfs merge=lfs -text | |
| *.wasm filter=lfs diff=lfs merge=lfs -text | |
| *.xz filter=lfs diff=lfs merge=lfs -text | |
| *.zip filter=lfs diff=lfs merge=lfs -text | |
| *.zst filter=lfs diff=lfs merge=lfs -text | |
| *tfevents* filter=lfs diff=lfs merge=lfs -text | |
| mmproj-BF16.gguf filter=lfs diff=lfs merge=lfs -text | |
| UD-IQ1_M/DeepSeek-V4.1-Flash-1017-UD-IQ1_M-00001-of-00005.gguf filter=lfs diff=lfs merge=lfs -text | |
| UD-IQ1_S/DeepSeek-V4.1-Flash-1017-UD-IQ1_S-00001-of-00005.gguf filter=lfs diff=lfs merge=lfs -text | |
| UD-IQ1_M/DeepSeek-V4.1-Flash-1017-UD-IQ1_M-00002-of-00005.gguf filter=lfs diff=lfs merge=lfs -text | |
| UD-IQ1_M/DeepSeek-V4.1-Flash-1017-UD-IQ1_M-00003-of-00005.gguf filter=lfs diff=lfs merge=lfs -text | |
| UD-IQ1_M/DeepSeek-V4.1-Flash-1017-UD-IQ1_M-00004-of-00005.gguf filter=lfs diff=lfs merge=lfs -text | |
| UD-IQ1_M/DeepSeek-V4.1-Flash-1017-UD-IQ1_M-00005-of-00005.gguf filter=lfs diff=lfs merge=lfs -text | |
| UD-IQ1_S/DeepSeek-V4.1-Flash-1017-UD-IQ1_S-00002-of-00005.gguf filter=lfs diff=lfs merge=lfs -text | |
| UD-IQ1_S/DeepSeek-V4.1-Flash-1017-UD-IQ1_S-00003-of-00005.gguf filter=lfs diff=lfs merge=lfs -text | |
| UD-IQ1_S/DeepSeek-V4.1-Flash-1017-UD-IQ1_S-00004-of-00005.gguf filter=lfs diff=lfs merge=lfs -text | |
| UD-IQ1_S/DeepSeek-V4.1-Flash-1017-UD-IQ1_S-00005-of-00005.gguf filter=lfs diff=lfs merge=lfs -text | |
| Strata-deepseek-v4.1-flash-IST-DAS/DeepSeek-V4.1-Flash-1017-GSQ-RCO-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text | |
| Maya/DeepSeek-V4.1-Flash-1017-Maya-XS-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text | |
| unsloth/patch-UD-IQ1_M/DeepSeek-V4.1-Flash-1017-UD-IQ1_M-patch.gguf filter=lfs diff=lfs merge=lfs -text | |
| unsloth/patch-UD-IQ1_S/DeepSeek-V4.1-Flash-1017-UD-IQ1_S-patch.gguf filter=lfs diff=lfs merge=lfs -text | |
| unsloth/patch-UD-IQ2_XXS/DeepSeek-V4.1-Flash-1017-UD-IQ2_XXS-patch.gguf filter=lfs diff=lfs merge=lfs -text | |
| unsloth/patch-UD-IQ3_S/DeepSeek-V4.1-Flash-1017-UD-IQ3_S-patch.gguf filter=lfs diff=lfs merge=lfs -text | |
| unsloth/patch-UD-IQ3_XXS/DeepSeek-V4.1-Flash-1017-UD-IQ3_XXS-patch.gguf filter=lfs diff=lfs merge=lfs -text | |
| unsloth/patch-UD-Q2_K_XL/DeepSeek-V4.1-Flash-1017-UD-Q2_K_XL-patch.gguf filter=lfs diff=lfs merge=lfs -text | |
| patch-unsloth-Q2/patch-unsloth-Q2_1/Huihui-DeepSeek-V4.1-Flash-1017-patch-Q2-1.gguf filter=lfs diff=lfs merge=lfs -text | |
| patch-Strata-GSQ-RCO-Coder/patch-Strata-GSQ-RCO-Coder/Huihui-DeepSeek-V4.1-Flash-1017-patch-GSQ-RCO-Coder-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text | |
| patch-unsloth-Q2_2/Huihui-DeepSeek-V4.1-Flash-1017-patch-Q2-2-0.gguf filter=lfs diff=lfs merge=lfs -text | |
| patch-unsloth-Q2_2/Huihui-DeepSeek-V4.1-Flash-1017-patch-Q2-2-1.gguf filter=lfs diff=lfs merge=lfs -text | |
| patch-unsloth-Q2_2/Huihui-DeepSeek-V4.1-Flash-1017-patch-Q2-2-2.gguf filter=lfs diff=lfs merge=lfs -text | |
| patch-unsloth-Q2_2/Huihui-DeepSeek-V4.1-Flash-1017-patch-Q2-2-3.gguf filter=lfs diff=lfs merge=lfs -text | |
| patch-unsloth-Q2_2/Huihui-DeepSeek-V4.1-Flash-1017-patch-Q2-2-4.gguf filter=lfs diff=lfs merge=lfs -text | |
| patch-unsloth-Q2_2/Huihui-DeepSeek-V4.1-Flash-1017-patch-Q2-2-5.gguf filter=lfs diff=lfs merge=lfs -text | |
| patch-unsloth-Q2/patch-unsloth-Q2_2/Huihui-DeepSeek-V4.1-Flash-1017-patch-Q2-2-0.gguf filter=lfs diff=lfs merge=lfs -text | |
| patch-unsloth-Q2/patch-unsloth-Q2_2/Huihui-DeepSeek-V4.1-Flash-1017-patch-Q2-2-1.gguf filter=lfs diff=lfs merge=lfs -text | |
| patch-unsloth-Q2/patch-unsloth-Q2_2/Huihui-DeepSeek-V4.1-Flash-1017-patch-Q2-2-2.gguf filter=lfs diff=lfs merge=lfs -text | |
| patch-unsloth-Q2/patch-unsloth-Q2_2/Huihui-DeepSeek-V4.1-Flash-1017-patch-Q2-2-3.gguf filter=lfs diff=lfs merge=lfs -text | |
| patch-unsloth-Q2/patch-unsloth-Q2_2/Huihui-DeepSeek-V4.1-Flash-1017-patch-Q2-2-4.gguf filter=lfs diff=lfs merge=lfs -text | |
| patch-unsloth-Q2/patch-unsloth-Q2_2/Huihui-DeepSeek-V4.1-Flash-1017-patch-Q2-2-5.gguf filter=lfs diff=lfs merge=lfs -text | |