Instructions to use fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090")
model = AutoModelForMultimodalLM.from_pretrained("fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090

SGLang

How to use fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090 with Docker Model Runner:
```
docker model run hf.co/fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090
```

Qwen3.8-27B Uncensored RTX 5090 NVFP4 + DSpark release

by fleetml - opened Sep 1

Discussion

fleetml

Owner Sep 1

•

edited Sep 1

Hugging Face CLI

I am releasing an RTX 5090 optimized Qwen3.8-27B checkpoint pair built from the original OrcaRouter BF16 safetensors.

The release includes a ModelOpt NVFP4 target with native MTP support and a BF16 DSpark drafter trained against the exact quantized target.

The combined release is 24.31 GB on disk: 20.59 GB for the target and 3.71 GB for the drafter. The optimized serving profile uses approximately 29.39 GB of GPU memory and leaves 3.22 GB free on the RTX 5090.

The model architecture supports a 262,144-token context window. The RTX 5090 speed profile is configured for a 65,536-token context limit with a 16,384-token active pool.

The trained DSpark profile reached 139.32 tks/s median decode on one RTX 5090. This delivered 2.79x the throughput of standard decoding, 1.41x native MTP depth 5, and 1.18x the public DSpark warm start.

The repositories include SGLang launch settings, checkpoint hashes, source attribution, conversion records, and training details.

Target: https://huggingface.co/fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090

DSpark drafter: https://huggingface.co/fleetml/Qwen3.8-27B-Uncensored-DSpark-RTX5090

License: Apache 2.0

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment