Instructions to use FiShota/hinomoto-1b-continued-pt-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use FiShota/hinomoto-1b-continued-pt-v1 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="FiShota/hinomoto-1b-continued-pt-v1")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("FiShota/hinomoto-1b-continued-pt-v1") model = AutoModelForCausalLM.from_pretrained("FiShota/hinomoto-1b-continued-pt-v1", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use FiShota/hinomoto-1b-continued-pt-v1 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "FiShota/hinomoto-1b-continued-pt-v1" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FiShota/hinomoto-1b-continued-pt-v1", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/FiShota/hinomoto-1b-continued-pt-v1
- SGLang
How to use FiShota/hinomoto-1b-continued-pt-v1 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "FiShota/hinomoto-1b-continued-pt-v1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FiShota/hinomoto-1b-continued-pt-v1", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "FiShota/hinomoto-1b-continued-pt-v1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FiShota/hinomoto-1b-continued-pt-v1", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use FiShota/hinomoto-1b-continued-pt-v1 with Docker Model Runner:
docker model run hf.co/FiShota/hinomoto-1b-continued-pt-v1
HinoMoto-1B v1 Continued-PT (Continued Pretraining with Bench Mix)
HinoMoto-1B v1 Phase 2 Full に 継続事前学習 (corpus 95% + bench 5%) を 1500 step 適用した派生.
「LoRA Cultural SFT」 では echo collapse (= 同 token 連発) が起きてしまった失敗を回避するために考案. mass corpus を anchor として bench 表現を 自然に内化 させる別アプローチ.
なぜ continued pretraining か
SFT v1 (LoRA r=32, lr=2e-4, 5 epoch) で 1B base + bench v0.6 train (551 items) を学習させたところ、 全 3 軸で base より大幅に悪化:
| metric | base full | SFT v1 (失敗) |
|---|---|---|
| family score /12 | 5.08 | 3.68 (-27%) |
| family degenerate | 9.1% | 31.8% (+250%) |
| keigo pass | 11.4% | 7.1% (-38%) |
| keigo degenerate | 5.7% | 34.3% (+500%) |
| silence degenerate | 6.0% | 46.0% |
→ mode collapse: model が prompt template (= 等) を echo loop. 1B base が WSD final で stable point に深く着地 + 小データで mode crush.
代替案: continued pretraining
Training summary
| Item | Value |
|---|---|
| Base | HinoMoto-1B Phase 2 Full (50,000 step) |
| Steps | 1,500 (continued) |
| Optimizer | AdamW (lr 1e-5, cosine schedule, warmup 100) |
| Batch | 1 × grad_accum 8 |
| Seq len | 512 |
| dtype | bf16 mixed |
| Corpus / bench ratio | 95% / 5% |
| Hardware | RTX 3090 24GB |
| Wall-clock | ~21 hr (= GPU 共有による slowdown) |
Bench results (HinoMoto-Bench-ja v0.6)
| metric | base full | Continued-PT v1 | Δ |
|---|---|---|---|
| family score /12 | 5.08 | 5.45 | +0.37 ✅ |
| family degenerate | 9.1% | 5.5% | -3.6% ✅ |
| keigo pass | 11.4% | 11.4% | 0 |
| keigo degenerate | 5.7% | 5.7% | 0 |
| silence pass | 6.0% | 2.0% | -4.0% |
| silence degenerate | 6.0% | 6.0% | 0 |
→ family 軸 改善 (+ score, -degenerate). keigo / silence は base 同等. → SFT v1/v2 と異なり mode collapse 発生せず.
Usage
import torch from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained( "FiShota/hinomoto-1b-continued-pt-v1", dtype=torch.bfloat16, ).to("cuda") tok = AutoTokenizer.from_pretrained("FiShota/hinomoto-1b-continued-pt-v1")
License
CC BY 4.0. Use for any purpose with attribution.
Limitations
- 1 seed (= seed 0) のみ. variance 未測定
- bench 軸 で base 同等 (劇的改善ではない). 構造的問題解決 (= echo collapse 回避) が本質
- 200MB corpus 由来の知識制限 + no safety alignment
Related
- base: https://huggingface.co/FiShota/hinomoto-1b-v1-phase2-full
- BitNet quant: https://huggingface.co/FiShota/hinomoto-1b-bitnet158-v1
- 350M cultural SFT: https://huggingface.co/FiShota/hinomoto-350m-cultural-sft-v1
- bench: https://github.com/FIshota/hinomoto-bench-ja
- training repo: https://github.com/FIshota/hinomoto-model
Citation
@misc{hinomoto-1b-continued-pt-v1-2026, author = {ryu (FIshota)}, title = {HinoMoto-1B v1 Continued-PT (Continued Pretraining with Bench Mix, mode collapse-free)}, year = {2026}, publisher = {Hugging Face}, }
Generated: 2026-05-20 (HinoMoto/ryu + Claude)
- Downloads last month
- 19
Model tree for FiShota/hinomoto-1b-continued-pt-v1
Base model
FiShota/hinomoto-1b-v1-phase2-full
docker model run hf.co/FiShota/hinomoto-1b-continued-pt-v1