Instructions to use baseten/math-sgd-qwen38_27b-expert with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use baseten/math-sgd-qwen38_27b-expert with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="baseten/math-sgd-qwen38_27b-expert")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("baseten/math-sgd-qwen38_27b-expert")
model = AutoModelForMultimodalLM.from_pretrained("baseten/math-sgd-qwen38_27b-expert", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use baseten/math-sgd-qwen38_27b-expert with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "baseten/math-sgd-qwen38_27b-expert"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "baseten/math-sgd-qwen38_27b-expert",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/baseten/math-sgd-qwen38_27b-expert

SGLang

How to use baseten/math-sgd-qwen38_27b-expert with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "baseten/math-sgd-qwen38_27b-expert" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "baseten/math-sgd-qwen38_27b-expert",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "baseten/math-sgd-qwen38_27b-expert" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "baseten/math-sgd-qwen38_27b-expert",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use baseten/math-sgd-qwen38_27b-expert with Docker Model Runner:
```
docker model run hf.co/baseten/math-sgd-qwen38_27b-expert
```

math-sgd-qwen38_27b-expert

File size: 6,257 Bytes

5d653bb

series,completed_updates,value,source_metric,complete,samples,scored,unscored,correct,truncated,completed,format_valid,format_valid_among_completed,mean_output_tokens,correctness_lower_bound,correctness_upper_bound
Adam,0,0.165,validation/reward_mean,True,600,600,,99,,,,,,,
Adam,5,0.18166666666666667,validation/reward_mean,True,600,600,,109,,,,,,,
Adam,10,0.19,validation/reward_mean,True,600,600,,114,,,,,,,
Adam,15,0.21666666666666667,validation/reward_mean,True,600,600,,130,,,,,,,
Adam,20,0.25,validation/reward_mean,True,600,600,,150,,,,,,,
Adam,25,0.265,validation/reward_mean,True,600,600,,159,,,,,,,
Adam,30,0.2783333333333333,validation/reward_mean,True,600,600,,167,,,,,,,
Adam,35,0.29833333333333334,validation/reward_mean,True,600,600,,179,,,,,,,
Adam,40,0.31333333333333335,validation/reward_mean,True,600,600,,188,,,,,,,
Adam,45,0.30333333333333334,validation/reward_mean,True,600,600,,182,,,,,,,
Adam,50,0.32,validation/reward_mean,True,600,600,,192,,,,,,,
Adam,55,0.33666666666666667,validation/reward_mean,True,600,600,,202,,,,,,,
Adam,60,0.355,validation/reward_mean,True,600,600,,213,,,,,,,
Adam,65,0.36666666666666664,validation/reward_mean,True,600,600,,220,,,,,,,
Adam,70,0.38,validation/reward_mean,True,600,600,,228,,,,,,,
Adam,75,0.38333333333333336,validation/reward_mean,True,600,600,,230,,,,,,,
Adam,80,0.395,validation/reward_mean,True,600,600,,237,,,,,,,
Adam,85,0.4083333333333333,validation/reward_mean,True,600,600,,245,,,,,,,
Adam,90,0.425,validation/reward_mean,True,600,600,,255,,,,,,,
Adam,95,0.42833333333333334,validation/reward_mean,True,600,600,,257,,,,,,,
Adam,100,0.45666666666666667,validation/reward_mean,True,600,600,,274,,,,,,,
Adam,105,0.485,validation/reward_mean,True,600,600,,291,,,,,,,
Adam,110,0.5033333333333333,validation/reward_mean,True,600,600,,302,,,,,,,
Adam,115,0.49666666666666665,validation/reward_mean,True,600,600,,298,,,,,,,
Adam,120,0.4716666666666667,validation/reward_mean,True,600,600,,283,,,,,,,
Adam,125,0.5416666666666666,validation/reward_mean,True,600,600,,325,,,,,,,
Adam,130,0.525,validation/reward_mean,True,600,600,,315,,,,,,,
Adam,135,0.5366666666666666,validation/reward_mean,True,600,600,,322,,,,,,,
Adam,140,0.54,validation/reward_mean,True,600,600,,324,,,,,,,
Adam,145,0.57,validation/reward_mean,True,600,600,,342,,,,,,,
Adam,150,0.565,validation/reward_mean,True,600,600,,339,,,,,,,
Adam,155,0.57,validation/reward_mean,True,600,600,,342,,,,,,,
Adam,160,0.5883333333333334,validation/reward_mean,True,600,600,,353,,,,,,,
Adam,165,0.5833333333333334,validation/reward_mean,True,600,600,,350,,,,,,,
Adam,170,0.595,validation/reward_mean,True,600,600,,357,,,,,,,
Adam,175,0.6283333333333333,validation/reward_mean,True,600,600,,377,,,,,,,
Adam,180,0.62,validation/reward_mean,True,600,600,,372,,,,,,,
Adam,185,0.625,validation/reward_mean,True,600,600,,375,,,,,,,
Adam,190,0.655,validation/reward_mean,True,600,600,,393,,,,,,,
Adam,195,0.6433333333333333,validation/reward_mean,True,600,600,,386,,,,,,,
Adam,200,0.66,validation/reward_mean,True,600,600,,396,,,,,,,
SGD + format reward,0,0.17333333333333334,eval/validation,True,600,600,0,104,329,271,132,0.4870848708487085,25873.241666666665,0.17333333333333334,0.17333333333333334
SGD + format reward,10,0.215,eval/validation,True,600,600,0,129,361,239,152,0.6359832635983264,26313.566666666666,0.215,0.215
SGD + format reward,20,0.3016666666666667,eval/validation,True,600,600,0,181,366,234,200,0.8547008547008547,25271.135,0.3016666666666667,0.3016666666666667
SGD + format reward,30,0.31,eval/validation,True,600,600,0,186,387,213,198,0.9295774647887324,25177.31,0.31,0.31
SGD + format reward,40,0.38,eval/validation,True,600,600,0,228,338,262,245,0.9351145038167938,24372.07,0.38,0.38
SGD + format reward,50,0.3933333333333333,eval/validation,True,600,600,0,236,310,290,266,0.9172413793103448,23544.863333333335,0.3933333333333333,0.3933333333333333
SGD + format reward,60,0.4083333333333333,eval/validation,True,600,600,0,245,300,300,281,0.9366666666666666,22972.725,0.4083333333333333,0.4083333333333333
SGD + format reward,70,0.435,eval/validation,True,600,600,0,261,289,311,284,0.9131832797427653,22796.366666666665,0.435,0.435
SGD + format reward,80,0.5466666666666666,eval/validation,True,600,600,0,328,216,384,349,0.9088541666666666,21417.52666666667,0.5466666666666666,0.5466666666666666
SGD + format reward,90,0.5316666666666666,eval/validation,True,600,600,0,319,223,377,344,0.9124668435013262,21325.703333333335,0.5316666666666666,0.5316666666666666
SGD + format reward,100,0.5583333333333333,eval/validation,True,600,600,0,335,224,376,351,0.9335106382978723,21334.083333333332,0.5583333333333333,0.5583333333333333
SGD + format reward,110,0.5433333333333333,eval/validation,True,600,600,0,326,220,380,351,0.9236842105263158,20928.245,0.5433333333333333,0.5433333333333333
SGD + format reward,120,0.5483333333333333,eval/validation,True,600,600,0,329,212,388,357,0.9201030927835051,20669.585,0.5483333333333333,0.5483333333333333
SGD + format reward,130,0.58,eval/validation,True,600,600,0,348,203,397,370,0.9319899244332494,20450.351666666666,0.58,0.58
SGD + format reward,140,0.5766666666666667,eval/validation,True,600,600,0,346,216,384,366,0.953125,20709.033333333333,0.5766666666666667,0.5766666666666667
SGD + format reward,150,0.57,eval/validation,True,600,600,0,342,201,399,366,0.9172932330827067,20427.183333333334,0.57,0.57
SGD + format reward,160,0.5466666666666666,eval/validation,True,600,600,0,328,205,395,353,0.8936708860759494,20703.773333333334,0.5466666666666666,0.5466666666666666
SGD + format reward,170,0.5416666666666666,eval/validation,True,600,600,0,325,140,460,425,0.9239130434782609,19961.003333333334,0.5416666666666666,0.5416666666666666
SGD + format reward,180,0.48580968280467446,eval/validation,False,600,599,1,291,82,518,483,0.9324324324324325,18022.326666666668,0.485,0.4866666666666667
SGD + format reward,190,0.5216666666666666,eval/validation,True,600,600,0,313,66,534,505,0.9456928838951311,17293.948333333334,0.5216666666666666,0.5216666666666666
SGD + format reward,200,0.48833333333333334,eval/validation,True,600,600,0,293,29,571,528,0.9246935201401051,14941.75,0.48833333333333334,0.48833333333333334