Instructions to use towardsinnovationlab/Qwen3-8B-medical with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use towardsinnovationlab/Qwen3-8B-medical with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("unsloth/Qwen3-8B")
model = PeftModel.from_pretrained(base_model, "towardsinnovationlab/Qwen3-8B-medical")

Transformers

How to use towardsinnovationlab/Qwen3-8B-medical with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="towardsinnovationlab/Qwen3-8B-medical")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("towardsinnovationlab/Qwen3-8B-medical", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use towardsinnovationlab/Qwen3-8B-medical with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "towardsinnovationlab/Qwen3-8B-medical"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "towardsinnovationlab/Qwen3-8B-medical",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/towardsinnovationlab/Qwen3-8B-medical

SGLang

How to use towardsinnovationlab/Qwen3-8B-medical with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "towardsinnovationlab/Qwen3-8B-medical" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "towardsinnovationlab/Qwen3-8B-medical",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "towardsinnovationlab/Qwen3-8B-medical" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "towardsinnovationlab/Qwen3-8B-medical",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Desktop
Docker Model Runner
How to use towardsinnovationlab/Qwen3-8B-medical with Docker Model Runner:
```
docker model run hf.co/towardsinnovationlab/Qwen3-8B-medical
```

Qwen3-8B-medical / README.md

towardsinnovationlab

Update README.md

e7fe4ab verified 7 months ago

preview code

Raw

History Blame Contribute Delete

3.86 kB

metadata

license: apache-2.0
base_model: unsloth/Qwen3-8B
library_name: peft
pipeline_tag: text-generation
tags:
  - base_model:adapter:unsloth/Qwen3-8B
  - lora
  - sft
  - transformers
  - trl
  - unsloth
datasets:
  - OpenMed/Medical-Reasoning-SFT-GPT-OSS-120B
language:
  - en

Model Adapter Description

This is a LoRA adapter for Qwen/Qwen3-8B, trained using the Unsloth library for parameter-efficient fine-tuning (PEFT). When loaded via the transformers pipeline or PeftModel, the base model Qwen/Qwen3-8B is automatically fetched, and the adapter is applied on top.

The adapter was trained on the following dataset:

-OpenMed/Medical-Reasoning-SFT-GPT-OSS-120B - for enhancing medical reasoning skills

Model Adapter Details

Developed by: Claudio Giorgio Giancaterino
Language(s) (NLP): English
License: Apache 2.0

Uses

Direct Use

This adapter can be used as support in healthcare applications, medical research, and clinical text generation.

Downstream Use

It can be integrated into educational chatbots for medical reasoning conversations.

Out-of-Scope Use

It is not suitable for high-level decision-making.

Bias, Risks, and Limitations

Conversational quality may degrade with complex or multi-turn inputs.

How to Get Started with the Model Adapter

Use the code below to get started with the model.

-Using the pipeline:

# Use a pipeline as a high-level helper
from transformers import pipeline
import re

pipe = pipeline("text-generation", model="towardsinnovationlab/Qwen3-8B-medical")

messages = [
    {"role": "user", "content": "What are the main symptoms of heart disease?
    Please provide your answer in bullet points."},
]
result = pipe(messages)
# Extract only the assistant's response
assistant_response = result[0]['generated_text'][-1]['content']
# Remove the <think> tags and their content
clean_response = re.sub(r'<think>.*?</think>', '',
assistant_response, flags=re.DOTALL).strip()
print(clean_response)

-Loading the model:

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
import re

tokenizer = AutoTokenizer.from_pretrained("towardsinnovationlab/Qwen3-8B-medical")
model = AutoModelForCausalLM.from_pretrained("towardsinnovationlab/Qwen3-8B-medical")
messages = [
    {"role": "user", "content": "What are the main symptoms of heart disease?
    Please provide your answer in bullet points."},
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(
    **inputs, 
    max_new_tokens=512, 
    temperature=0.7, 
    top_p=0.8, 
    top_k=20,
    do_sample=True  
)

# Extract assistant's response
assistant_response = tokenizer.decode(
    outputs[0][inputs["input_ids"].shape[-1]:], 
    skip_special_tokens=True
)

# Remove <think> tags and their content
clean_response = re.sub(r'<think>.*?</think>', '', assistant_response,
flags=re.DOTALL).strip()

print(clean_response)

Training Details

Training Data

-OpenMed/Medical-Reasoning-SFT-GPT-OSS-120B with 200,193 synthetic medical conversations.

Training Procedure

-Colab with NVIDIA A100 GPU

-per_device_train_batch_size = 4,

-gradient_accumulation_steps = 8,

-warmup_steps = 5,

-max_steps = 30,

-learning_rate = 2e-5,

-logging_steps = 100,

-save_steps=500,

-optim = "adamw_torch",

-weight_decay = 0.001,

-lr_scheduler_type = "linear"

Results

Loss Value >> 1.1232

Citation

@misc{towardsinnovationlab_2025,
    author       = { Claudio Giorgio Giancaterino },
    title        = { Qwen3-8B-medical },
    year         = 2025,
    url          = { https://huggingface.co/towardsinnovationlab/Qwen3-8B-medical },
    publisher    = { Hugging Face }
}

Framework versions

PEFT 0.18.0