van Es
Mels
webbrain-one-626934's profile picture
cHNAINA's profile picture
webbrain-one-221250's profile picture·
AI & ML interests
None yet
Recent Activity
liked a model 1 day ago
vllm-sr/d3-mini liked a Space 1 day ago
tardellirs/typed-decisions-explorer reacted to tardellirs's post with 🔥 1 day ago
I really like the LocalLLaMA/typed-decisions benchmark, so I built a small Space to make its leaderboard easier to explore: https://huggingface.co/spaces/tardellirs/typed-decisions-explorer
It reads the dataset card on every visit, so new submissions show up as soon as they are merged. It has a sortable ranking and two Pareto charts: accuracy × size and accuracy × latency.
A few things it makes easy to see in the zero-shot table today:
- Only three models are within noise of the top: Quyet-1.0-Large (0.791), meraGPT Decider 1 and Baseten Mammoth 1 (0.768).
- The smallest model above Jev 1.13 (0.727) is d3-mini, at 4B.
- The fastest local run above Jev is FastBrain-9B: 50 ms on an RTX PRO 6000.
- 15 of 41 models score above the teacher's self-agreement (0.735), so the top of the table measures agreement with the teacher more than correctness.
Congrats to @codelion and @sandeep85, the maintainers, for a great benchmark. Feedback welcome!Organizations
None yet