Instructions to use fpadovani/hin-deva-100mb-ppt-Dp-100mb_seed10 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use fpadovani/hin-deva-100mb-ppt-Dp-100mb_seed10 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="fpadovani/hin-deva-100mb-ppt-Dp-100mb_seed10")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("fpadovani/hin-deva-100mb-ppt-Dp-100mb_seed10")
model = AutoModelForCausalLM.from_pretrained("fpadovani/hin-deva-100mb-ppt-Dp-100mb_seed10", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use fpadovani/hin-deva-100mb-ppt-Dp-100mb_seed10 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "fpadovani/hin-deva-100mb-ppt-Dp-100mb_seed10"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fpadovani/hin-deva-100mb-ppt-Dp-100mb_seed10",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/fpadovani/hin-deva-100mb-ppt-Dp-100mb_seed10

SGLang

How to use fpadovani/hin-deva-100mb-ppt-Dp-100mb_seed10 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "fpadovani/hin-deva-100mb-ppt-Dp-100mb_seed10" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fpadovani/hin-deva-100mb-ppt-Dp-100mb_seed10",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "fpadovani/hin-deva-100mb-ppt-Dp-100mb_seed10" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fpadovani/hin-deva-100mb-ppt-Dp-100mb_seed10",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use fpadovani/hin-deva-100mb-ppt-Dp-100mb_seed10 with Docker Model Runner:
```
docker model run hf.co/fpadovani/hin-deva-100mb-ppt-Dp-100mb_seed10
```

fpadovani commited on Jul 1

Commit

ea06ef7

verified ·

1 Parent(s): a6d71c1

Training checkpoint: step 1500

Browse files

Files changed (10) hide show

checkpoint-1500/added_tokens.json +1202 -0
checkpoint-1500/config.json +34 -0
checkpoint-1500/generation_config.json +9 -0
checkpoint-1500/model.safetensors +3 -0
checkpoint-1500/rng_state.pth +3 -0
checkpoint-1500/special_tokens_map.json +1249 -0
checkpoint-1500/spiece.model +3 -0
checkpoint-1500/tokenizer_config.json +0 -0
checkpoint-1500/trainer_state.json +3034 -0
checkpoint-1500/training_args.bin +3 -0

checkpoint-1500/added_tokens.json ADDED Viewed

	@@ -0,0 +1,1202 @@

+{
+  "<pad>": 50002,
+  "[CLS]": 50000,
+  "[MASK]": 50003,
+  "[SEP]": 50001,
+  "[XXXXX0]": 50771,
+  "[XXXXX1000]": 50549,
+  "[XXXXX1001]": 50657,
+  "[XXXXX1002]": 51158,
+  "[XXXXX1003]": 51044,
+  "[XXXXX1004]": 50688,
+  "[XXXXX1005]": 50689,
+  "[XXXXX1006]": 50410,
+  "[XXXXX1007]": 50724,
+  "[XXXXX1008]": 50171,
+  "[XXXXX1009]": 51077,
+  "[XXXXX100]": 50091,
+  "[XXXXX1010]": 50639,
+  "[XXXXX1011]": 51094,
+  "[XXXXX1012]": 50392,
+  "[XXXXX1013]": 50626,
+  "[XXXXX1014]": 50991,
+  "[XXXXX1015]": 50529,
+  "[XXXXX1016]": 51193,
+  "[XXXXX1017]": 51073,
+  "[XXXXX1018]": 51104,
+  "[XXXXX1019]": 50775,
+  "[XXXXX101]": 50139,
+  "[XXXXX1020]": 50157,
+  "[XXXXX1021]": 50934,
+  "[XXXXX1022]": 51159,
+  "[XXXXX1023]": 50048,
+  "[XXXXX1024]": 51123,
+  "[XXXXX1025]": 50084,
+  "[XXXXX1026]": 50813,
+  "[XXXXX1027]": 50725,
+  "[XXXXX1028]": 50931,
+  "[XXXXX1029]": 50233,
+  "[XXXXX102]": 50213,
+  "[XXXXX1030]": 51023,
+  "[XXXXX1031]": 50098,
+  "[XXXXX1032]": 50802,
+  "[XXXXX1033]": 50061,
+  "[XXXXX1034]": 51139,
+  "[XXXXX1035]": 51110,
+  "[XXXXX1036]": 50204,
+  "[XXXXX1037]": 50023,
+  "[XXXXX1038]": 50142,
+  "[XXXXX1039]": 50328,
+  "[XXXXX103]": 50087,
+  "[XXXXX1040]": 50435,
+  "[XXXXX1041]": 50489,
+  "[XXXXX1042]": 50216,
+  "[XXXXX1043]": 50144,
+  "[XXXXX1044]": 50832,
+  "[XXXXX1045]": 50566,
+  "[XXXXX1046]": 50922,
+  "[XXXXX1047]": 50796,
+  "[XXXXX1048]": 50533,
+  "[XXXXX1049]": 50433,
+  "[XXXXX104]": 50478,
+  "[XXXXX1050]": 50953,
+  "[XXXXX1051]": 50497,
+  "[XXXXX1052]": 50655,
+  "[XXXXX1053]": 50881,
+  "[XXXXX1054]": 50461,
+  "[XXXXX1055]": 50052,
+  "[XXXXX1056]": 50219,
+  "[XXXXX1057]": 50789,
+  "[XXXXX1058]": 50121,
+  "[XXXXX1059]": 51002,
+  "[XXXXX105]": 51192,
+  "[XXXXX1060]": 50158,
+  "[XXXXX1061]": 50352,
+  "[XXXXX1062]": 50637,
+  "[XXXXX1063]": 50440,
+  "[XXXXX1064]": 50590,
+  "[XXXXX1065]": 51107,
+  "[XXXXX1066]": 50758,
+  "[XXXXX1067]": 51011,
+  "[XXXXX1068]": 50985,
+  "[XXXXX1069]": 51175,
+  "[XXXXX106]": 50532,
+  "[XXXXX1070]": 50790,
+  "[XXXXX1071]": 50394,
+  "[XXXXX1072]": 50852,
+  "[XXXXX1073]": 50869,
+  "[XXXXX1074]": 50126,
+  "[XXXXX1075]": 50783,
+  "[XXXXX1076]": 50806,
+  "[XXXXX1077]": 50882,
+  "[XXXXX1078]": 50726,
+  "[XXXXX1079]": 51010,
+  "[XXXXX107]": 50714,
+  "[XXXXX1080]": 50168,
+  "[XXXXX1081]": 50425,
+  "[XXXXX1082]": 50848,
+  "[XXXXX1083]": 50014,
+  "[XXXXX1084]": 50341,
+  "[XXXXX1085]": 50734,
+  "[XXXXX1086]": 50664,
+  "[XXXXX1087]": 51007,
+  "[XXXXX1088]": 51151,
+  "[XXXXX1089]": 50269,
+  "[XXXXX108]": 50996,
+  "[XXXXX1090]": 50826,
+  "[XXXXX1091]": 51105,
+  "[XXXXX1092]": 50731,
+  "[XXXXX1093]": 50856,
+  "[XXXXX1094]": 50369,
+  "[XXXXX1095]": 50065,
+  "[XXXXX1096]": 50309,
+  "[XXXXX1097]": 50838,
+  "[XXXXX1098]": 50451,
+  "[XXXXX1099]": 50384,
+  "[XXXXX109]": 50543,
+  "[XXXXX10]": 50325,
+  "[XXXXX1100]": 50318,
+  "[XXXXX1101]": 50764,
+  "[XXXXX1102]": 51025,
+  "[XXXXX1103]": 50642,
+  "[XXXXX1104]": 50665,
+  "[XXXXX1105]": 50855,
+  "[XXXXX1106]": 51063,
+  "[XXXXX1107]": 50025,
+  "[XXXXX1108]": 50122,
+  "[XXXXX1109]": 50470,
+  "[XXXXX110]": 50062,
+  "[XXXXX1110]": 51019,
+  "[XXXXX1111]": 51124,
+  "[XXXXX1112]": 50146,
+  "[XXXXX1113]": 51125,
+  "[XXXXX1114]": 50434,
+  "[XXXXX1115]": 50818,
+  "[XXXXX1116]": 50960,
+  "[XXXXX1117]": 50258,
+  "[XXXXX1118]": 50596,
+  "[XXXXX1119]": 50083,
+  "[XXXXX111]": 51064,
+  "[XXXXX1120]": 50915,
+  "[XXXXX1121]": 50835,
+  "[XXXXX1122]": 51056,
+  "[XXXXX1123]": 50172,
+  "[XXXXX1124]": 50972,
+  "[XXXXX1125]": 50275,
+  "[XXXXX1126]": 50430,
+  "[XXXXX1127]": 50743,
+  "[XXXXX1128]": 50551,
+  "[XXXXX1129]": 50439,
+  "[XXXXX112]": 50819,
+  "[XXXXX1130]": 50536,
+  "[XXXXX1131]": 50622,
+  "[XXXXX1132]": 50623,
+  "[XXXXX1133]": 50117,
+  "[XXXXX1134]": 50251,
+  "[XXXXX1135]": 50678,
+  "[XXXXX1136]": 50316,
+  "[XXXXX1137]": 50905,
+  "[XXXXX1138]": 50992,
+  "[XXXXX1139]": 50851,
+  "[XXXXX113]": 50443,
+  "[XXXXX1140]": 50499,
+  "[XXXXX1141]": 50629,
+  "[XXXXX1142]": 50944,
+  "[XXXXX1143]": 50373,
+  "[XXXXX1144]": 50123,
+  "[XXXXX1145]": 50173,
+  "[XXXXX1146]": 50891,
+  "[XXXXX1147]": 50471,
+  "[XXXXX1148]": 50560,
+  "[XXXXX1149]": 50541,
+  "[XXXXX114]": 50631,
+  "[XXXXX1150]": 50033,
+  "[XXXXX1151]": 50063,
+  "[XXXXX1152]": 50767,
+  "[XXXXX1153]": 50897,
+  "[XXXXX1154]": 50568,
+  "[XXXXX1155]": 51131,
+  "[XXXXX1156]": 50009,
+  "[XXXXX1157]": 50417,
+  "[XXXXX1158]": 50477,
+  "[XXXXX1159]": 50941,
+  "[XXXXX115]": 50294,
+  "[XXXXX1160]": 50679,
+  "[XXXXX1161]": 51108,
+  "[XXXXX1162]": 50332,
+  "[XXXXX1163]": 50424,
+  "[XXXXX1164]": 50473,
+  "[XXXXX1165]": 50793,
+  "[XXXXX1166]": 50453,
+  "[XXXXX1167]": 50807,
+  "[XXXXX1168]": 50128,
+  "[XXXXX1169]": 50039,
+  "[XXXXX116]": 50759,
+  "[XXXXX1170]": 50735,
+  "[XXXXX1171]": 50366,
+  "[XXXXX1172]": 50808,
+  "[XXXXX1173]": 50426,
+  "[XXXXX1174]": 50076,
+  "[XXXXX1175]": 50943,
+  "[XXXXX1176]": 50252,
+  "[XXXXX1177]": 50999,
+  "[XXXXX1178]": 50264,
+  "[XXXXX1179]": 50607,
+  "[XXXXX117]": 51036,
+  "[XXXXX1180]": 50986,
+  "[XXXXX1181]": 50744,
+  "[XXXXX1182]": 50186,
+  "[XXXXX1183]": 50555,
+  "[XXXXX1184]": 51076,
+  "[XXXXX1185]": 50849,
+  "[XXXXX1186]": 51128,
+  "[XXXXX1187]": 50858,
+  "[XXXXX1188]": 50656,
+  "[XXXXX1189]": 50064,
+  "[XXXXX118]": 50081,
+  "[XXXXX1190]": 51197,
+  "[XXXXX1191]": 50104,
+  "[XXXXX1192]": 50045,
+  "[XXXXX1193]": 50404,
+  "[XXXXX1194]": 50690,
+  "[XXXXX1195]": 50495,
+  "[XXXXX119]": 50108,
+  "[XXXXX11]": 50976,
+  "[XXXXX120]": 51081,
+  "[XXXXX121]": 50464,
+  "[XXXXX122]": 50829,
+  "[XXXXX123]": 50766,
+  "[XXXXX124]": 50732,
+  "[XXXXX125]": 50221,
+  "[XXXXX126]": 50810,
+  "[XXXXX127]": 50130,
+  "[XXXXX128]": 50751,
+  "[XXXXX129]": 50662,
+  "[XXXXX12]": 50602,
+  "[XXXXX130]": 50035,
+  "[XXXXX131]": 50715,
+  "[XXXXX132]": 50340,
+  "[XXXXX133]": 50563,
+  "[XXXXX134]": 50993,
+  "[XXXXX135]": 50304,
+  "[XXXXX136]": 50131,
+  "[XXXXX137]": 50578,
+  "[XXXXX138]": 50757,
+  "[XXXXX139]": 50582,
+  "[XXXXX13]": 50105,
+  "[XXXXX140]": 50085,
+  "[XXXXX141]": 50553,
+  "[XXXXX142]": 50749,
+  "[XXXXX143]": 50351,
+  "[XXXXX144]": 51024,
+  "[XXXXX145]": 51144,
+  "[XXXXX146]": 50042,
+  "[XXXXX147]": 50145,
+  "[XXXXX148]": 50066,
+  "[XXXXX149]": 50876,
+  "[XXXXX14]": 51006,
+  "[XXXXX150]": 50280,
+  "[XXXXX151]": 50093,
+  "[XXXXX152]": 50638,
+  "[XXXXX153]": 50684,
+  "[XXXXX154]": 50820,
+  "[XXXXX155]": 51196,
+  "[XXXXX156]": 50317,
+  "[XXXXX157]": 50284,
+  "[XXXXX158]": 50288,
+  "[XXXXX159]": 50814,
+  "[XXXXX15]": 50608,
+  "[XXXXX160]": 50004,
+  "[XXXXX161]": 50008,
+  "[XXXXX162]": 50377,
+  "[XXXXX163]": 50214,
+  "[XXXXX164]": 50708,
+  "[XXXXX165]": 50390,
+  "[XXXXX166]": 50741,
+  "[XXXXX167]": 50445,
+  "[XXXXX168]": 50936,
+  "[XXXXX169]": 50896,
+  "[XXXXX16]": 50442,
+  "[XXXXX170]": 50109,
+  "[XXXXX171]": 50140,
+  "[XXXXX172]": 50427,
+  "[XXXXX173]": 51074,
+  "[XXXXX174]": 50162,
+  "[XXXXX175]": 51015,
+  "[XXXXX176]": 50524,
+  "[XXXXX177]": 50673,
+  "[XXXXX178]": 50054,
+  "[XXXXX179]": 50805,
+  "[XXXXX17]": 50482,
+  "[XXXXX180]": 50253,
+  "[XXXXX181]": 50493,
+  "[XXXXX182]": 50421,
+  "[XXXXX183]": 51050,
+  "[XXXXX184]": 50521,
+  "[XXXXX185]": 50188,
+  "[XXXXX186]": 50147,
+  "[XXXXX187]": 50201,
+  "[XXXXX188]": 50973,
+  "[XXXXX189]": 50409,
+  "[XXXXX18]": 50115,
+  "[XXXXX190]": 50051,
+  "[XXXXX191]": 51020,
+  "[XXXXX192]": 51047,
+  "[XXXXX193]": 50669,
+  "[XXXXX194]": 50643,
+  "[XXXXX195]": 50159,
+  "[XXXXX196]": 50649,
+  "[XXXXX197]": 51178,
+  "[XXXXX198]": 50043,
+  "[XXXXX199]": 51147,
+  "[XXXXX19]": 51127,
+  "[XXXXX1]": 51114,
+  "[XXXXX200]": 50840,
+  "[XXXXX201]": 50365,
+  "[XXXXX202]": 50232,
+  "[XXXXX203]": 50241,
+  "[XXXXX204]": 50548,
+  "[XXXXX205]": 50781,
+  "[XXXXX206]": 50923,
+  "[XXXXX207]": 51115,
+  "[XXXXX208]": 50344,
+  "[XXXXX209]": 50299,
+  "[XXXXX20]": 51031,
+  "[XXXXX210]": 51179,
+  "[XXXXX211]": 51163,
+  "[XXXXX212]": 50562,
+  "[XXXXX213]": 50094,
+  "[XXXXX214]": 50257,
+  "[XXXXX215]": 50967,
+  "[XXXXX216]": 50523,
+  "[XXXXX217]": 51132,
+  "[XXXXX218]": 51146,
+  "[XXXXX219]": 50411,
+  "[XXXXX21]": 50354,
+  "[XXXXX220]": 50853,
+  "[XXXXX221]": 50886,
+  "[XXXXX222]": 50600,
+  "[XXXXX223]": 50873,
+  "[XXXXX224]": 50416,
+  "[XXXXX225]": 50675,
+  "[XXXXX226]": 50583,
+  "[XXXXX227]": 50151,
+  "[XXXXX228]": 50927,
+  "[XXXXX229]": 50609,
+  "[XXXXX22]": 50589,
+  "[XXXXX230]": 50238,
+  "[XXXXX231]": 50242,
+  "[XXXXX232]": 51016,
+  "[XXXXX233]": 50406,
+  "[XXXXX234]": 50908,
+  "[XXXXX235]": 50024,
+  "[XXXXX236]": 50270,
+  "[XXXXX237]": 50132,
+  "[XXXXX238]": 50800,
+  "[XXXXX239]": 50236,
+  "[XXXXX23]": 50007,
+  "[XXXXX240]": 50005,
+  "[XXXXX241]": 50894,
+  "[XXXXX242]": 50694,
+  "[XXXXX243]": 51071,
+  "[XXXXX244]": 51017,
+  "[XXXXX245]": 50068,
+  "[XXXXX246]": 50682,
+  "[XXXXX247]": 50913,
+  "[XXXXX248]": 51028,
+  "[XXXXX249]": 50037,
+  "[XXXXX24]": 51169,
+  "[XXXXX250]": 50747,
+  "[XXXXX251]": 50947,
+  "[XXXXX252]": 50329,
+  "[XXXXX253]": 50612,
+  "[XXXXX254]": 50110,
+  "[XXXXX255]": 50824,
+  "[XXXXX256]": 50558,
+  "[XXXXX257]": 50969,
+  "[XXXXX258]": 50729,
+  "[XXXXX259]": 50125,
+  "[XXXXX25]": 51160,
+  "[XXXXX260]": 50040,
+  "[XXXXX261]": 51038,
+  "[XXXXX262]": 50347,
+  "[XXXXX263]": 50740,
+  "[XXXXX264]": 51195,
+  "[XXXXX265]": 50879,
+  "[XXXXX266]": 50770,
+  "[XXXXX267]": 50178,
+  "[XXXXX268]": 50804,
+  "[XXXXX269]": 50906,
+  "[XXXXX26]": 50150,
+  "[XXXXX270]": 51191,
+  "[XXXXX271]": 50047,
+  "[XXXXX272]": 50195,
+  "[XXXXX273]": 50554,
+  "[XXXXX274]": 50833,
+  "[XXXXX275]": 50346,
+  "[XXXXX276]": 50950,
+  "[XXXXX277]": 50864,
+  "[XXXXX278]": 50259,
+  "[XXXXX279]": 50671,
+  "[XXXXX27]": 50468,
+  "[XXXXX280]": 50082,
+  "[XXXXX281]": 50565,
+  "[XXXXX282]": 50984,
+  "[XXXXX283]": 50650,
+  "[XXXXX284]": 50975,
+  "[XXXXX285]": 50603,
+  "[XXXXX286]": 50027,
+  "[XXXXX287]": 50181,
+  "[XXXXX288]": 50981,
+  "[XXXXX289]": 50885,
+  "[XXXXX28]": 50249,
+  "[XXXXX290]": 50522,
+  "[XXXXX291]": 50050,
+  "[XXXXX292]": 50313,
+  "[XXXXX293]": 50503,
+  "[XXXXX294]": 51176,
+  "[XXXXX295]": 51177,
+  "[XXXXX296]": 50379,
+  "[XXXXX297]": 50635,
+  "[XXXXX298]": 50723,
+  "[XXXXX299]": 50200,
+  "[XXXXX29]": 50892,
+  "[XXXXX2]": 51119,
+  "[XXXXX300]": 51026,
+  "[XXXXX301]": 50528,
+  "[XXXXX302]": 51149,
+  "[XXXXX303]": 50044,
+  "[XXXXX304]": 50209,
+  "[XXXXX305]": 50988,
+  "[XXXXX306]": 50518,
+  "[XXXXX307]": 50088,
+  "[XXXXX308]": 50733,
+  "[XXXXX309]": 51066,
+  "[XXXXX30]": 50506,
+  "[XXXXX310]": 50285,
+  "[XXXXX311]": 50859,
+  "[XXXXX312]": 50263,
+  "[XXXXX313]": 50594,
+  "[XXXXX314]": 51103,
+  "[XXXXX315]": 50871,
+  "[XXXXX316]": 50300,
+  "[XXXXX317]": 50760,
+  "[XXXXX318]": 50055,
+  "[XXXXX319]": 50389,
+  "[XXXXX31]": 50212,
+  "[XXXXX320]": 50444,
+  "[XXXXX321]": 50994,
+  "[XXXXX322]": 50281,
+  "[XXXXX323]": 50179,
+  "[XXXXX324]": 50455,
+  "[XXXXX325]": 50970,
+  "[XXXXX326]": 50666,
+  "[XXXXX327]": 50604,
+  "[XXXXX328]": 50627,
+  "[XXXXX329]": 50475,
+  "[XXXXX32]": 50277,
+  "[XXXXX330]": 50667,
+  "[XXXXX331]": 50348,
+  "[XXXXX332]": 50772,
+  "[XXXXX333]": 50163,
+  "[XXXXX334]": 50569,
+  "[XXXXX335]": 50018,
+  "[XXXXX336]": 50883,
+  "[XXXXX337]": 50067,
+  "[XXXXX338]": 51184,
+  "[XXXXX339]": 50997,
+  "[XXXXX33]": 50403,
+  "[XXXXX340]": 51013,
+  "[XXXXX341]": 50957,
+  "[XXXXX342]": 51106,
+  "[XXXXX343]": 51174,
+  "[XXXXX344]": 50391,
+  "[XXXXX345]": 50948,
+  "[XXXXX346]": 50180,
+  "[XXXXX347]": 50811,
+  "[XXXXX348]": 50401,
+  "[XXXXX349]": 50911,
+  "[XXXXX34]": 50223,
+  "[XXXXX350]": 51089,
+  "[XXXXX351]": 50777,
+  "[XXXXX352]": 50189,
+  "[XXXXX353]": 50492,
+  "[XXXXX354]": 50205,
+  "[XXXXX355]": 50374,
+  "[XXXXX356]": 51152,
+  "[XXXXX357]": 50695,
+  "[XXXXX358]": 51117,
+  "[XXXXX359]": 50573,
+  "[XXXXX35]": 50752,
+  "[XXXXX360]": 50955,
+  "[XXXXX361]": 50349,
+  "[XXXXX362]": 51137,
+  "[XXXXX363]": 50641,
+  "[XXXXX364]": 51148,
+  "[XXXXX365]": 50255,
+  "[XXXXX366]": 50310,
+  "[XXXXX367]": 50314,
+  "[XXXXX368]": 50971,
+  "[XXXXX369]": 50916,
+  "[XXXXX36]": 50437,
+  "[XXXXX370]": 51096,
+  "[XXXXX371]": 50557,
+  "[XXXXX372]": 51095,
+  "[XXXXX373]": 50382,
+  "[XXXXX374]": 50119,
+  "[XXXXX375]": 50958,
+  "[XXXXX376]": 50624,
+  "[XXXXX377]": 50483,
+  "[XXXXX378]": 50605,
+  "[XXXXX379]": 50017,
+  "[XXXXX37]": 50706,
+  "[XXXXX380]": 50246,
+  "[XXXXX381]": 50588,
+  "[XXXXX382]": 51008,
+  "[XXXXX383]": 50801,
+  "[XXXXX384]": 50290,
+  "[XXXXX385]": 50899,
+  "[XXXXX386]": 50206,
+  "[XXXXX387]": 50716,
+  "[XXXXX388]": 51082,
+  "[XXXXX389]": 50177,
+  "[XXXXX38]": 50736,
+  "[XXXXX390]": 50282,
+  "[XXXXX391]": 51183,
+  "[XXXXX392]": 50286,
+  "[XXXXX393]": 50659,
+  "[XXXXX394]": 50378,
+  "[XXXXX395]": 50709,
+  "[XXXXX396]": 50028,
+  "[XXXXX397]": 50235,
+  "[XXXXX398]": 51014,
+  "[XXXXX399]": 50880,
+  "[XXXXX39]": 50305,
+  "[XXXXX3]": 50077,
+  "[XXXXX400]": 50155,
+  "[XXXXX401]": 50710,
+  "[XXXXX402]": 50507,
+  "[XXXXX403]": 50860,
+  "[XXXXX404]": 50661,
+  "[XXXXX405]": 51083,
+  "[XXXXX406]": 50888,
+  "[XXXXX407]": 50069,
+  "[XXXXX408]": 50691,
+  "[XXXXX409]": 51055,
+  "[XXXXX40]": 50337,
+  "[XXXXX410]": 50137,
+  "[XXXXX411]": 50791,
+  "[XXXXX412]": 50136,
+  "[XXXXX413]": 50730,
+  "[XXXXX414]": 50267,
+  "[XXXXX415]": 50012,
+  "[XXXXX416]": 50681,
+  "[XXXXX417]": 50942,
+  "[XXXXX418]": 50720,
+  "[XXXXX419]": 50844,
+  "[XXXXX41]": 50387,
+  "[XXXXX420]": 50111,
+  "[XXXXX421]": 51003,
+  "[XXXXX422]": 50785,
+  "[XXXXX423]": 50149,
+  "[XXXXX424]": 50422,
+  "[XXXXX425]": 51086,
+  "[XXXXX426]": 50862,
+  "[XXXXX427]": 50576,
+  "[XXXXX428]": 51049,
+  "[XXXXX429]": 50630,
+  "[XXXXX42]": 50799,
+  "[XXXXX430]": 50289,
+  "[XXXXX431]": 50184,
+  "[XXXXX432]": 50319,
+  "[XXXXX433]": 51116,
+  "[XXXXX434]": 50519,
+  "[XXXXX435]": 50496,
+  "[XXXXX436]": 50517,
+  "[XXXXX437]": 50865,
+  "[XXXXX438]": 50737,
+  "[XXXXX439]": 50356,
+  "[XXXXX43]": 50359,
+  "[XXXXX440]": 50898,
+  "[XXXXX441]": 50113,
+  "[XXXXX442]": 50527,
+  "[XXXXX443]": 50762,
+  "[XXXXX444]": 50540,
+  "[XXXXX445]": 50925,
+  "[XXXXX446]": 50711,
+  "[XXXXX447]": 50143,
+  "[XXXXX448]": 51170,
+  "[XXXXX449]": 50413,
+  "[XXXXX44]": 50097,
+  "[XXXXX450]": 50375,
+  "[XXXXX451]": 50846,
+  "[XXXXX452]": 50217,
+  "[XXXXX453]": 50699,
+  "[XXXXX454]": 50412,
+  "[XXXXX455]": 50418,
+  "[XXXXX456]": 51143,
+  "[XXXXX457]": 51141,
+  "[XXXXX458]": 51157,
+  "[XXXXX459]": 50825,
+  "[XXXXX45]": 50203,
+  "[XXXXX460]": 50194,
+  "[XXXXX461]": 50148,
+  "[XXXXX462]": 50276,
+  "[XXXXX463]": 50271,
+  "[XXXXX464]": 50836,
+  "[XXXXX465]": 50854,
+  "[XXXXX466]": 50164,
+  "[XXXXX467]": 50484,
+  "[XXXXX468]": 51097,
+  "[XXXXX469]": 50182,
+  "[XXXXX46]": 50505,
+  "[XXXXX470]": 50670,
+  "[XXXXX471]": 50595,
+  "[XXXXX472]": 50237,
+  "[XXXXX473]": 50057,
+  "[XXXXX474]": 50431,
+  "[XXXXX475]": 50738,
+  "[XXXXX476]": 51145,
+  "[XXXXX477]": 50247,
+  "[XXXXX478]": 50029,
+  "[XXXXX479]": 50917,
+  "[XXXXX47]": 50579,
+  "[XXXXX480]": 50702,
+  "[XXXXX481]": 50831,
+  "[XXXXX482]": 50753,
+  "[XXXXX483]": 50598,
+  "[XXXXX484]": 51136,
+  "[XXXXX485]": 50092,
+  "[XXXXX486]": 50202,
+  "[XXXXX487]": 50106,
+  "[XXXXX488]": 50924,
+  "[XXXXX489]": 51113,
+  "[XXXXX48]": 50842,
+  "[XXXXX490]": 50761,
+  "[XXXXX491]": 50577,
+  "[XXXXX492]": 50692,
+  "[XXXXX493]": 51130,
+  "[XXXXX494]": 50932,
+  "[XXXXX495]": 50837,
+  "[XXXXX496]": 50893,
+  "[XXXXX497]": 50773,
+  "[XXXXX498]": 50360,
+  "[XXXXX499]": 50900,
+  "[XXXXX49]": 51062,
+  "[XXXXX4]": 50396,
+  "[XXXXX500]": 51034,
+  "[XXXXX501]": 50472,
+  "[XXXXX502]": 50660,
+  "[XXXXX503]": 50815,
+  "[XXXXX504]": 50939,
+  "[XXXXX505]": 50419,
+  "[XXXXX506]": 50324,
+  "[XXXXX507]": 50428,
+  "[XXXXX508]": 50274,
+  "[XXXXX509]": 50311,
+  "[XXXXX50]": 50459,
+  "[XXXXX510]": 50674,
+  "[XXXXX511]": 50580,
+  "[XXXXX512]": 50956,
+  "[XXXXX513]": 50165,
+  "[XXXXX514]": 50320,
+  "[XXXXX515]": 50648,
+  "[XXXXX516]": 50386,
+  "[XXXXX517]": 50058,
+  "[XXXXX518]": 51045,
+  "[XXXXX519]": 50103,
+  "[XXXXX51]": 50979,
+  "[XXXXX520]": 50530,
+  "[XXXXX521]": 50907,
+  "[XXXXX522]": 50260,
+  "[XXXXX523]": 50632,
+  "[XXXXX524]": 50926,
+  "[XXXXX525]": 50687,
+  "[XXXXX526]": 50160,
+  "[XXXXX527]": 51053,
+  "[XXXXX528]": 50652,
+  "[XXXXX529]": 51041,
+  "[XXXXX52]": 50272,
+  "[XXXXX530]": 51111,
+  "[XXXXX531]": 51009,
+  "[XXXXX532]": 51079,
+  "[XXXXX533]": 50487,
+  "[XXXXX534]": 51059,
+  "[XXXXX535]": 50335,
+  "[XXXXX536]": 50380,
+  "[XXXXX537]": 50301,
+  "[XXXXX538]": 51067,
+  "[XXXXX539]": 51052,
+  "[XXXXX53]": 51048,
+  "[XXXXX540]": 50822,
+  "[XXXXX541]": 51040,
+  "[XXXXX542]": 50490,
+  "[XXXXX543]": 50448,
+  "[XXXXX544]": 50812,
+  "[XXXXX545]": 50099,
+  "[XXXXX546]": 51199,
+  "[XXXXX547]": 50226,
+  "[XXXXX548]": 50538,
+  "[XXXXX549]": 50342,
+  "[XXXXX54]": 50701,
+  "[XXXXX550]": 50544,
+  "[XXXXX551]": 50398,
+  "[XXXXX552]": 51102,
+  "[XXXXX553]": 50919,
+  "[XXXXX554]": 50615,
+  "[XXXXX555]": 50769,
+  "[XXXXX556]": 50940,
+  "[XXXXX557]": 50070,
+  "[XXXXX558]": 50680,
+  "[XXXXX559]": 50611,
+  "[XXXXX55]": 50574,
+  "[XXXXX560]": 50381,
+  "[XXXXX561]": 50350,
+  "[XXXXX562]": 50175,
+  "[XXXXX563]": 50584,
+  "[XXXXX564]": 50703,
+  "[XXXXX565]": 50989,
+  "[XXXXX566]": 50877,
+  "[XXXXX567]": 51039,
+  "[XXXXX568]": 51018,
+  "[XXXXX569]": 50452,
+  "[XXXXX56]": 51099,
+  "[XXXXX570]": 50161,
+  "[XXXXX571]": 50921,
+  "[XXXXX572]": 50676,
+  "[XXXXX573]": 50326,
+  "[XXXXX574]": 50987,
+  "[XXXXX575]": 50361,
+  "[XXXXX576]": 50071,
+  "[XXXXX577]": 50803,
+  "[XXXXX578]": 50400,
+  "[XXXXX579]": 51091,
+  "[XXXXX57]": 50501,
+  "[XXXXX580]": 50887,
+  "[XXXXX581]": 50839,
+  "[XXXXX582]": 51138,
+  "[XXXXX583]": 50816,
+  "[XXXXX584]": 50339,
+  "[XXXXX585]": 50463,
+  "[XXXXX586]": 50399,
+  "[XXXXX587]": 51070,
+  "[XXXXX588]": 50457,
+  "[XXXXX589]": 50828,
+  "[XXXXX58]": 50183,
+  "[XXXXX590]": 50080,
+  "[XXXXX591]": 50550,
+  "[XXXXX592]": 50129,
+  "[XXXXX593]": 50273,
+  "[XXXXX594]": 50388,
+  "[XXXXX595]": 50491,
+  "[XXXXX596]": 50278,
+  "[XXXXX597]": 50011,
+  "[XXXXX598]": 50254,
+  "[XXXXX599]": 50302,
+  "[XXXXX59]": 50353,
+  "[XXXXX5]": 50010,
+  "[XXXXX600]": 50704,
+  "[XXXXX601]": 51118,
+  "[XXXXX602]": 50049,
+  "[XXXXX603]": 51190,
+  "[XXXXX604]": 51142,
+  "[XXXXX605]": 50707,
+  "[XXXXX606]": 50937,
+  "[XXXXX607]": 50875,
+  "[XXXXX608]": 50564,
+  "[XXXXX609]": 50345,
+  "[XXXXX60]": 50713,
+  "[XXXXX610]": 50949,
+  "[XXXXX611]": 50978,
+  "[XXXXX612]": 50330,
+  "[XXXXX613]": 51035,
+  "[XXXXX614]": 50402,
+  "[XXXXX615]": 50176,
+  "[XXXXX616]": 51133,
+  "[XXXXX617]": 50222,
+  "[XXXXX618]": 50118,
+  "[XXXXX619]": 50196,
+  "[XXXXX61]": 50298,
+  "[XXXXX620]": 50668,
+  "[XXXXX621]": 50462,
+  "[XXXXX622]": 50218,
+  "[XXXXX623]": 51101,
+  "[XXXXX624]": 50362,
+  "[XXXXX625]": 50779,
+  "[XXXXX626]": 50513,
+  "[XXXXX627]": 50746,
+  "[XXXXX628]": 50141,
+  "[XXXXX629]": 50479,
+  "[XXXXX62]": 50056,
+  "[XXXXX630]": 51180,
+  "[XXXXX631]": 50872,
+  "[XXXXX632]": 50315,
+  "[XXXXX633]": 50397,
+  "[XXXXX634]": 51165,
+  "[XXXXX635]": 50046,
+  "[XXXXX636]": 50795,
+  "[XXXXX637]": 50959,
+  "[XXXXX638]": 50786,
+  "[XXXXX639]": 50240,
+  "[XXXXX63]": 51061,
+  "[XXXXX640]": 50601,
+  "[XXXXX641]": 50904,
+  "[XXXXX642]": 50376,
+  "[XXXXX643]": 50663,
+  "[XXXXX644]": 50613,
+  "[XXXXX645]": 50467,
+  "[XXXXX646]": 50546,
+  "[XXXXX647]": 50465,
+  "[XXXXX648]": 50395,
+  "[XXXXX649]": 50287,
+  "[XXXXX64]": 51162,
+  "[XXXXX650]": 50474,
+  "[XXXXX651]": 50727,
+  "[XXXXX652]": 50243,
+  "[XXXXX653]": 51100,
+  "[XXXXX654]": 50705,
+  "[XXXXX655]": 51012,
+  "[XXXXX656]": 50447,
+  "[XXXXX657]": 50719,
+  "[XXXXX658]": 50712,
+  "[XXXXX659]": 50476,
+  "[XXXXX65]": 50587,
+  "[XXXXX660]": 51054,
+  "[XXXXX661]": 51166,
+  "[XXXXX662]": 50494,
+  "[XXXXX663]": 50230,
+  "[XXXXX664]": 50621,
+  "[XXXXX665]": 50809,
+  "[XXXXX666]": 50265,
+  "[XXXXX667]": 50645,
+  "[XXXXX668]": 50817,
+  "[XXXXX669]": 50030,
+  "[XXXXX66]": 51029,
+  "[XXXXX670]": 51004,
+  "[XXXXX671]": 50685,
+  "[XXXXX672]": 50625,
+  "[XXXXX673]": 50890,
+  "[XXXXX674]": 50383,
+  "[XXXXX675]": 50929,
+  "[XXXXX676]": 50116,
+  "[XXXXX677]": 51072,
+  "[XXXXX678]": 51126,
+  "[XXXXX679]": 50755,
+  "[XXXXX67]": 50717,
+  "[XXXXX680]": 50303,
+  "[XXXXX681]": 50504,
+  "[XXXXX682]": 51075,
+  "[XXXXX683]": 50261,
+  "[XXXXX684]": 50861,
+  "[XXXXX685]": 50306,
+  "[XXXXX686]": 50016,
+  "[XXXXX687]": 50101,
+  "[XXXXX688]": 51098,
+  "[XXXXX689]": 50827,
+  "[XXXXX68]": 50863,
+  "[XXXXX690]": 50640,
+  "[XXXXX691]": 50031,
+  "[XXXXX692]": 50768,
+  "[XXXXX693]": 50013,
+  "[XXXXX694]": 51032,
+  "[XXXXX695]": 50166,
+  "[XXXXX696]": 50127,
+  "[XXXXX697]": 50414,
+  "[XXXXX698]": 50742,
+  "[XXXXX699]": 51042,
+  "[XXXXX69]": 51112,
+  "[XXXXX6]": 50593,
+  "[XXXXX700]": 50292,
+  "[XXXXX701]": 50174,
+  "[XXXXX702]": 50547,
+  "[XXXXX703]": 50498,
+  "[XXXXX704]": 51078,
+  "[XXXXX705]": 50133,
+  "[XXXXX706]": 50041,
+  "[XXXXX707]": 51181,
+  "[XXXXX708]": 50585,
+  "[XXXXX709]": 50776,
+  "[XXXXX70]": 50053,
+  "[XXXXX710]": 50336,
+  "[XXXXX711]": 50765,
+  "[XXXXX712]": 50224,
+  "[XXXXX713]": 50599,
+  "[XXXXX714]": 50207,
+  "[XXXXX715]": 51043,
+  "[XXXXX716]": 51090,
+  "[XXXXX717]": 50006,
+  "[XXXXX718]": 50901,
+  "[XXXXX719]": 51001,
+  "[XXXXX71]": 50954,
+  "[XXXXX720]": 50646,
+  "[XXXXX721]": 50075,
+  "[XXXXX722]": 51120,
+  "[XXXXX723]": 50215,
+  "[XXXXX724]": 50415,
+  "[XXXXX725]": 50154,
+  "[XXXXX726]": 50935,
+  "[XXXXX727]": 50545,
+  "[XXXXX728]": 51085,
+  "[XXXXX729]": 50256,
+  "[XXXXX72]": 50363,
+  "[XXXXX730]": 50700,
+  "[XXXXX731]": 50262,
+  "[XXXXX732]": 50930,
+  "[XXXXX733]": 50333,
+  "[XXXXX734]": 50884,
+  "[XXXXX735]": 51030,
+  "[XXXXX736]": 50231,
+  "[XXXXX737]": 50405,
+  "[XXXXX738]": 50962,
+  "[XXXXX739]": 50458,
+  "[XXXXX73]": 50019,
+  "[XXXXX740]": 50878,
+  "[XXXXX741]": 50910,
+  "[XXXXX742]": 50170,
+  "[XXXXX743]": 50718,
+  "[XXXXX744]": 50616,
+  "[XXXXX745]": 50358,
+  "[XXXXX746]": 50152,
+  "[XXXXX747]": 50525,
+  "[XXXXX748]": 51167,
+  "[XXXXX749]": 50653,
+  "[XXXXX74]": 50952,
+  "[XXXXX750]": 50500,
+  "[XXXXX751]": 50446,
+  "[XXXXX752]": 50995,
+  "[XXXXX753]": 50945,
+  "[XXXXX754]": 51173,
+  "[XXXXX755]": 50823,
+  "[XXXXX756]": 51060,
+  "[XXXXX757]": 50748,
+  "[XXXXX758]": 50020,
+  "[XXXXX759]": 50197,
+  "[XXXXX75]": 50502,
+  "[XXXXX760]": 50291,
+  "[XXXXX761]": 50821,
+  "[XXXXX762]": 50190,
+  "[XXXXX763]": 50745,
+  "[XXXXX764]": 50778,
+  "[XXXXX765]": 50556,
+  "[XXXXX766]": 51161,
+  "[XXXXX767]": 50466,
+  "[XXXXX768]": 50792,
+  "[XXXXX769]": 50208,
+  "[XXXXX76]": 50169,
+  "[XXXXX770]": 50647,
+  "[XXXXX771]": 50610,
+  "[XXXXX772]": 51140,
+  "[XXXXX773]": 50441,
+  "[XXXXX774]": 50870,
+  "[XXXXX775]": 50368,
+  "[XXXXX776]": 50026,
+  "[XXXXX777]": 50677,
+  "[XXXXX778]": 51168,
+  "[XXXXX779]": 50481,
+  "[XXXXX77]": 50515,
+  "[XXXXX780]": 50225,
+  "[XXXXX781]": 50408,
+  "[XXXXX782]": 50090,
+  "[XXXXX783]": 50951,
+  "[XXXXX784]": 50015,
+  "[XXXXX785]": 50658,
+  "[XXXXX786]": 50617,
+  "[XXXXX787]": 50721,
+  "[XXXXX788]": 51084,
+  "[XXXXX789]": 50830,
+  "[XXXXX78]": 50100,
+  "[XXXXX790]": 50606,
+  "[XXXXX791]": 50124,
+  "[XXXXX792]": 50078,
+  "[XXXXX793]": 51188,
+  "[XXXXX794]": 50750,
+  "[XXXXX795]": 50138,
+  "[XXXXX796]": 50239,
+  "[XXXXX797]": 50385,
+  "[XXXXX798]": 50780,
+  "[XXXXX799]": 50683,
+  "[XXXXX79]": 50436,
+  "[XXXXX7]": 50211,
+  "[XXXXX800]": 51185,
+  "[XXXXX801]": 50250,
+  "[XXXXX802]": 50456,
+  "[XXXXX803]": 50508,
+  "[XXXXX804]": 51194,
+  "[XXXXX805]": 50021,
+  "[XXXXX806]": 50307,
+  "[XXXXX807]": 50199,
+  "[XXXXX808]": 51129,
+  "[XXXXX809]": 50686,
+  "[XXXXX80]": 50370,
+  "[XXXXX810]": 50343,
+  "[XXXXX811]": 50614,
+  "[XXXXX812]": 50636,
+  "[XXXXX813]": 51198,
+  "[XXXXX814]": 50788,
+  "[XXXXX815]": 50120,
+  "[XXXXX816]": 50135,
+  "[XXXXX817]": 50293,
+  "[XXXXX818]": 50153,
+  "[XXXXX819]": 50516,
+  "[XXXXX81]": 50581,
+  "[XXXXX820]": 50586,
+  "[XXXXX821]": 50509,
+  "[XXXXX822]": 50220,
+  "[XXXXX823]": 51068,
+  "[XXXXX824]": 50946,
+  "[XXXXX825]": 50965,
+  "[XXXXX826]": 50032,
+  "[XXXXX827]": 50156,
+  "[XXXXX828]": 50449,
+  "[XXXXX829]": 50531,
+  "[XXXXX82]": 51164,
+  "[XXXXX830]": 50486,
+  "[XXXXX831]": 50357,
+  "[XXXXX832]": 51022,
+  "[XXXXX833]": 50633,
+  "[XXXXX834]": 50754,
+  "[XXXXX835]": 51134,
+  "[XXXXX836]": 50193,
+  "[XXXXX837]": 51088,
+  "[XXXXX838]": 50722,
+  "[XXXXX839]": 50756,
+  "[XXXXX83]": 50977,
+  "[XXXXX840]": 50022,
+  "[XXXXX841]": 51186,
+  "[XXXXX842]": 50575,
+  "[XXXXX843]": 50079,
+  "[XXXXX844]": 50644,
+  "[XXXXX845]": 51033,
+  "[XXXXX846]": 50454,
+  "[XXXXX847]": 50095,
+  "[XXXXX848]": 50902,
+  "[XXXXX849]": 50597,
+  "[XXXXX84]": 50167,
+  "[XXXXX850]": 51051,
+  "[XXXXX851]": 50696,
+  "[XXXXX852]": 50966,
+  "[XXXXX853]": 50034,
+  "[XXXXX854]": 50889,
+  "[XXXXX855]": 50843,
+  "[XXXXX856]": 50963,
+  "[XXXXX857]": 50227,
+  "[XXXXX858]": 51058,
+  "[XXXXX859]": 50850,
+  "[XXXXX85]": 50192,
+  "[XXXXX860]": 50228,
+  "[XXXXX861]": 50570,
+  "[XXXXX862]": 50488,
+  "[XXXXX863]": 50539,
+  "[XXXXX864]": 51182,
+  "[XXXXX865]": 50990,
+  "[XXXXX866]": 50542,
+  "[XXXXX867]": 51069,
+  "[XXXXX868]": 50480,
+  "[XXXXX869]": 50847,
+  "[XXXXX86]": 50874,
+  "[XXXXX870]": 50308,
+  "[XXXXX871]": 50510,
+  "[XXXXX872]": 50423,
+  "[XXXXX873]": 50086,
+  "[XXXXX874]": 50591,
+  "[XXXXX875]": 50728,
+  "[XXXXX876]": 50372,
+  "[XXXXX877]": 50903,
+  "[XXXXX878]": 50868,
+  "[XXXXX879]": 50763,
+  "[XXXXX87]": 50571,
+  "[XXXXX880]": 50096,
+  "[XXXXX881]": 51150,
+  "[XXXXX882]": 50460,
+  "[XXXXX883]": 51087,
+  "[XXXXX884]": 51093,
+  "[XXXXX885]": 50295,
+  "[XXXXX886]": 51037,
+  "[XXXXX887]": 51171,
+  "[XXXXX888]": 50794,
+  "[XXXXX889]": 50628,
+  "[XXXXX88]": 50520,
+  "[XXXXX890]": 51172,
+  "[XXXXX891]": 51046,
+  "[XXXXX892]": 50918,
+  "[XXXXX893]": 50998,
+  "[XXXXX894]": 50371,
+  "[XXXXX895]": 51155,
+  "[XXXXX896]": 50866,
+  "[XXXXX897]": 50229,
+  "[XXXXX898]": 51057,
+  "[XXXXX899]": 50867,
+  "[XXXXX89]": 50537,
+  "[XXXXX8]": 50784,
+  "[XXXXX900]": 50739,
+  "[XXXXX901]": 51080,
+  "[XXXXX902]": 50038,
+  "[XXXXX903]": 50321,
+  "[XXXXX904]": 50774,
+  "[XXXXX905]": 50974,
+  "[XXXXX906]": 50920,
+  "[XXXXX907]": 50469,
+  "[XXXXX908]": 51122,
+  "[XXXXX909]": 51092,
+  "[XXXXX90]": 50514,
+  "[XXXXX910]": 50798,
+  "[XXXXX911]": 50245,
+  "[XXXXX912]": 50367,
+  "[XXXXX913]": 50618,
+  "[XXXXX914]": 50072,
+  "[XXXXX915]": 50450,
+  "[XXXXX916]": 51189,
+  "[XXXXX917]": 50485,
+  "[XXXXX918]": 50283,
+  "[XXXXX919]": 50845,
+  "[XXXXX91]": 50968,
+  "[XXXXX920]": 50429,
+  "[XXXXX921]": 50364,
+  "[XXXXX922]": 50782,
+  "[XXXXX923]": 50534,
+  "[XXXXX924]": 50059,
+  "[XXXXX925]": 50102,
+  "[XXXXX926]": 50185,
+  "[XXXXX927]": 50634,
+  "[XXXXX928]": 50567,
+  "[XXXXX929]": 50895,
+  "[XXXXX92]": 50210,
+  "[XXXXX930]": 50561,
+  "[XXXXX931]": 51021,
+  "[XXXXX932]": 50107,
+  "[XXXXX933]": 50407,
+  "[XXXXX934]": 50654,
+  "[XXXXX935]": 50268,
+  "[XXXXX936]": 51153,
+  "[XXXXX937]": 50552,
+  "[XXXXX938]": 50526,
+  "[XXXXX939]": 50322,
+  "[XXXXX93]": 50912,
+  "[XXXXX940]": 50296,
+  "[XXXXX941]": 50572,
+  "[XXXXX942]": 50393,
+  "[XXXXX943]": 50312,
+  "[XXXXX944]": 50592,
+  "[XXXXX945]": 50982,
+  "[XXXXX946]": 50089,
+  "[XXXXX947]": 50331,
+  "[XXXXX948]": 50279,
+  "[XXXXX949]": 51005,
+  "[XXXXX94]": 50198,
+  "[XXXXX950]": 50114,
+  "[XXXXX951]": 50074,
+  "[XXXXX952]": 50248,
+  "[XXXXX953]": 50857,
+  "[XXXXX954]": 50841,
+  "[XXXXX955]": 51065,
+  "[XXXXX956]": 50234,
+  "[XXXXX957]": 50834,
+  "[XXXXX958]": 50535,
+  "[XXXXX959]": 51109,
+  "[XXXXX95]": 50651,
+  "[XXXXX960]": 50797,
+  "[XXXXX961]": 50914,
+  "[XXXXX962]": 50112,
+  "[XXXXX963]": 50073,
+  "[XXXXX964]": 50559,
+  "[XXXXX965]": 50933,
+  "[XXXXX966]": 50619,
+  "[XXXXX967]": 51187,
+  "[XXXXX968]": 50327,
+  "[XXXXX969]": 50698,
+  "[XXXXX96]": 50420,
+  "[XXXXX970]": 50060,
+  "[XXXXX971]": 50432,
+  "[XXXXX972]": 50620,
+  "[XXXXX973]": 50693,
+  "[XXXXX974]": 50938,
+  "[XXXXX975]": 50983,
+  "[XXXXX976]": 50334,
+  "[XXXXX977]": 50438,
+  "[XXXXX978]": 51154,
+  "[XXXXX979]": 50964,
+  "[XXXXX97]": 50672,
+  "[XXXXX980]": 50511,
+  "[XXXXX981]": 50244,
+  "[XXXXX982]": 50266,
+  "[XXXXX983]": 50134,
+  "[XXXXX984]": 50338,
+  "[XXXXX985]": 51121,
+  "[XXXXX986]": 50323,
+  "[XXXXX987]": 50909,
+  "[XXXXX988]": 51000,
+  "[XXXXX989]": 50036,
+  "[XXXXX98]": 50787,
+  "[XXXXX990]": 50512,
+  "[XXXXX991]": 50191,
+  "[XXXXX992]": 50187,
+  "[XXXXX993]": 51156,
+  "[XXXXX994]": 51135,
+  "[XXXXX995]": 50355,
+  "[XXXXX996]": 51027,
+  "[XXXXX997]": 50297,
+  "[XXXXX998]": 50928,
+  "[XXXXX999]": 50697,
+  "[XXXXX99]": 50980,
+  "[XXXXX9]": 50961
+}

checkpoint-1500/config.json ADDED Viewed

	@@ -0,0 +1,34 @@

+{
+  "activation_function": "gelu",
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.1,
+  "bos_token_id": 50000,
+  "dtype": "bfloat16",
+  "embd_pdrop": 0.1,
+  "eos_token_id": 50001,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_ctx": 512,
+  "n_embd": 768,
+  "n_head": 12,
+  "n_inner": 3072,
+  "n_layer": 12,
+  "n_positions": 512,
+  "pad_token_id": 50002,
+  "prefix": "[CLS]",
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.1,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "transformers_version": "4.56.2",
+  "use_cache": true,
+  "vocab_size": 51200
+}

checkpoint-1500/generation_config.json ADDED Viewed

	@@ -0,0 +1,9 @@

+{
+  "_from_model_config": true,
+  "bos_token_id": 50000,
+  "eos_token_id": [
+    50001
+  ],
+  "pad_token_id": 50002,
+  "transformers_version": "4.56.2"
+}

checkpoint-1500/model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2feb190a14a2eb591bb0bbe86f62bc6bf6970aa013895213f2dead7173cdc652
+size 249556672

checkpoint-1500/rng_state.pth ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1edf0db1e702f91f430837b8241902813d4a3b0b06d7f371598ebd4cf2964f8c
+size 14645

checkpoint-1500/special_tokens_map.json ADDED Viewed

	@@ -0,0 +1,1249 @@

+{
+  "additional_special_tokens": [
+    "[XXXXX160]",
+    "[XXXXX240]",
+    "[XXXXX717]",
+    "[XXXXX23]",
+    "[XXXXX161]",
+    "[XXXXX1156]",
+    "[XXXXX5]",
+    "[XXXXX597]",
+    "[XXXXX415]",
+    "[XXXXX693]",
+    "[XXXXX1083]",
+    "[XXXXX784]",
+    "[XXXXX686]",
+    "[XXXXX379]",
+    "[XXXXX335]",
+    "[XXXXX73]",
+    "[XXXXX758]",
+    "[XXXXX805]",
+    "[XXXXX840]",
+    "[XXXXX1037]",
+    "[XXXXX235]",
+    "[XXXXX1107]",
+    "[XXXXX776]",
+    "[XXXXX286]",
+    "[XXXXX396]",
+    "[XXXXX478]",
+    "[XXXXX669]",
+    "[XXXXX691]",
+    "[XXXXX826]",
+    "[XXXXX1150]",
+    "[XXXXX853]",
+    "[XXXXX130]",
+    "[XXXXX989]",
+    "[XXXXX249]",
+    "[XXXXX902]",
+    "[XXXXX1169]",
+    "[XXXXX260]",
+    "[XXXXX706]",
+    "[XXXXX146]",
+    "[XXXXX198]",
+    "[XXXXX303]",
+    "[XXXXX1192]",
+    "[XXXXX635]",
+    "[XXXXX271]",
+    "[XXXXX1023]",
+    "[XXXXX602]",
+    "[XXXXX291]",
+    "[XXXXX190]",
+    "[XXXXX1055]",
+    "[XXXXX70]",
+    "[XXXXX178]",
+    "[XXXXX318]",
+    "[XXXXX62]",
+    "[XXXXX473]",
+    "[XXXXX517]",
+    "[XXXXX924]",
+    "[XXXXX970]",
+    "[XXXXX1033]",
+    "[XXXXX110]",
+    "[XXXXX1151]",
+    "[XXXXX1189]",
+    "[XXXXX1095]",
+    "[XXXXX148]",
+    "[XXXXX337]",
+    "[XXXXX245]",
+    "[XXXXX407]",
+    "[XXXXX557]",
+    "[XXXXX576]",
+    "[XXXXX914]",
+    "[XXXXX963]",
+    "[XXXXX951]",
+    "[XXXXX721]",
+    "[XXXXX1174]",
+    "[XXXXX3]",
+    "[XXXXX792]",
+    "[XXXXX843]",
+    "[XXXXX590]",
+    "[XXXXX118]",
+    "[XXXXX280]",
+    "[XXXXX1119]",
+    "[XXXXX1025]",
+    "[XXXXX140]",
+    "[XXXXX873]",
+    "[XXXXX103]",
+    "[XXXXX307]",
+    "[XXXXX946]",
+    "[XXXXX782]",
+    "[XXXXX100]",
+    "[XXXXX485]",
+    "[XXXXX151]",
+    "[XXXXX213]",
+    "[XXXXX847]",
+    "[XXXXX880]",
+    "[XXXXX44]",
+    "[XXXXX1031]",
+    "[XXXXX545]",
+    "[XXXXX78]",
+    "[XXXXX687]",
+    "[XXXXX925]",
+    "[XXXXX519]",
+    "[XXXXX1191]",
+    "[XXXXX13]",
+    "[XXXXX487]",
+    "[XXXXX932]",
+    "[XXXXX119]",
+    "[XXXXX170]",
+    "[XXXXX254]",
+    "[XXXXX420]",
+    "[XXXXX962]",
+    "[XXXXX441]",
+    "[XXXXX950]",
+    "[XXXXX18]",
+    "[XXXXX676]",
+    "[XXXXX1133]",
+    "[XXXXX618]",
+    "[XXXXX374]",
+    "[XXXXX815]",
+    "[XXXXX1058]",
+    "[XXXXX1108]",
+    "[XXXXX1144]",
+    "[XXXXX791]",
+    "[XXXXX259]",
+    "[XXXXX1074]",
+    "[XXXXX696]",
+    "[XXXXX1168]",
+    "[XXXXX592]",
+    "[XXXXX127]",
+    "[XXXXX136]",
+    "[XXXXX237]",
+    "[XXXXX705]",
+    "[XXXXX983]",
+    "[XXXXX816]",
+    "[XXXXX412]",
+    "[XXXXX410]",
+    "[XXXXX795]",
+    "[XXXXX101]",
+    "[XXXXX171]",
+    "[XXXXX628]",
+    "[XXXXX1038]",
+    "[XXXXX447]",
+    "[XXXXX1043]",
+    "[XXXXX147]",
+    "[XXXXX1112]",
+    "[XXXXX186]",
+    "[XXXXX461]",
+    "[XXXXX423]",
+    "[XXXXX26]",
+    "[XXXXX227]",
+    "[XXXXX746]",
+    "[XXXXX818]",
+    "[XXXXX725]",
+    "[XXXXX400]",
+    "[XXXXX827]",
+    "[XXXXX1020]",
+    "[XXXXX1060]",
+    "[XXXXX195]",
+    "[XXXXX526]",
+    "[XXXXX570]",
+    "[XXXXX174]",
+    "[XXXXX333]",
+    "[XXXXX466]",
+    "[XXXXX513]",
+    "[XXXXX695]",
+    "[XXXXX84]",
+    "[XXXXX1080]",
+    "[XXXXX76]",
+    "[XXXXX742]",
+    "[XXXXX1008]",
+    "[XXXXX1123]",
+    "[XXXXX1145]",
+    "[XXXXX701]",
+    "[XXXXX562]",
+    "[XXXXX615]",
+    "[XXXXX389]",
+    "[XXXXX267]",
+    "[XXXXX323]",
+    "[XXXXX346]",
+    "[XXXXX287]",
+    "[XXXXX469]",
+    "[XXXXX58]",
+    "[XXXXX431]",
+    "[XXXXX926]",
+    "[XXXXX1182]",
+    "[XXXXX992]",
+    "[XXXXX185]",
+    "[XXXXX352]",
+    "[XXXXX762]",
+    "[XXXXX991]",
+    "[XXXXX85]",
+    "[XXXXX836]",
+    "[XXXXX460]",
+    "[XXXXX272]",
+    "[XXXXX619]",
+    "[XXXXX759]",
+    "[XXXXX94]",
+    "[XXXXX807]",
+    "[XXXXX299]",
+    "[XXXXX187]",
+    "[XXXXX486]",
+    "[XXXXX45]",
+    "[XXXXX1036]",
+    "[XXXXX354]",
+    "[XXXXX386]",
+    "[XXXXX714]",
+    "[XXXXX769]",
+    "[XXXXX304]",
+    "[XXXXX92]",
+    "[XXXXX7]",
+    "[XXXXX31]",
+    "[XXXXX102]",
+    "[XXXXX163]",
+    "[XXXXX723]",
+    "[XXXXX1042]",
+    "[XXXXX452]",
+    "[XXXXX622]",
+    "[XXXXX1056]",
+    "[XXXXX822]",
+    "[XXXXX125]",
+    "[XXXXX617]",
+    "[XXXXX34]",
+    "[XXXXX712]",
+    "[XXXXX780]",
+    "[XXXXX547]",
+    "[XXXXX857]",
+    "[XXXXX860]",
+    "[XXXXX897]",
+    "[XXXXX663]",
+    "[XXXXX736]",
+    "[XXXXX202]",
+    "[XXXXX1029]",
+    "[XXXXX956]",
+    "[XXXXX397]",
+    "[XXXXX239]",
+    "[XXXXX472]",
+    "[XXXXX230]",
+    "[XXXXX796]",
+    "[XXXXX639]",
+    "[XXXXX203]",
+    "[XXXXX231]",
+    "[XXXXX652]",
+    "[XXXXX981]",
+    "[XXXXX911]",
+    "[XXXXX380]",
+    "[XXXXX477]",
+    "[XXXXX952]",
+    "[XXXXX28]",
+    "[XXXXX801]",
+    "[XXXXX1134]",
+    "[XXXXX1176]",
+    "[XXXXX180]",
+    "[XXXXX598]",
+    "[XXXXX365]",
+    "[XXXXX729]",
+    "[XXXXX214]",
+    "[XXXXX1117]",
+    "[XXXXX278]",
+    "[XXXXX522]",
+    "[XXXXX683]",
+    "[XXXXX731]",
+    "[XXXXX312]",
+    "[XXXXX1178]",
+    "[XXXXX666]",
+    "[XXXXX982]",
+    "[XXXXX414]",
+    "[XXXXX935]",
+    "[XXXXX1089]",
+    "[XXXXX236]",
+    "[XXXXX463]",
+    "[XXXXX52]",
+    "[XXXXX593]",
+    "[XXXXX508]",
+    "[XXXXX1125]",
+    "[XXXXX462]",
+    "[XXXXX32]",
+    "[XXXXX596]",
+    "[XXXXX948]",
+    "[XXXXX150]",
+    "[XXXXX322]",
+    "[XXXXX390]",
+    "[XXXXX918]",
+    "[XXXXX157]",
+    "[XXXXX310]",
+    "[XXXXX392]",
+    "[XXXXX649]",
+    "[XXXXX158]",
+    "[XXXXX430]",
+    "[XXXXX384]",
+    "[XXXXX760]",
+    "[XXXXX700]",
+    "[XXXXX817]",
+    "[XXXXX115]",
+    "[XXXXX885]",
+    "[XXXXX940]",
+    "[XXXXX997]",
+    "[XXXXX61]",
+    "[XXXXX209]",
+    "[XXXXX316]",
+    "[XXXXX537]",
+    "[XXXXX599]",
+    "[XXXXX680]",
+    "[XXXXX135]",
+    "[XXXXX39]",
+    "[XXXXX685]",
+    "[XXXXX806]",
+    "[XXXXX870]",
+    "[XXXXX1096]",
+    "[XXXXX366]",
+    "[XXXXX509]",
+    "[XXXXX943]",
+    "[XXXXX292]",
+    "[XXXXX367]",
+    "[XXXXX632]",
+    "[XXXXX1136]",
+    "[XXXXX156]",
+    "[XXXXX1100]",
+    "[XXXXX432]",
+    "[XXXXX514]",
+    "[XXXXX903]",
+    "[XXXXX939]",
+    "[XXXXX986]",
+    "[XXXXX506]",
+    "[XXXXX10]",
+    "[XXXXX573]",
+    "[XXXXX968]",
+    "[XXXXX1039]",
+    "[XXXXX252]",
+    "[XXXXX612]",
+    "[XXXXX947]",
+    "[XXXXX1162]",
+    "[XXXXX733]",
+    "[XXXXX976]",
+    "[XXXXX535]",
+    "[XXXXX710]",
+    "[XXXXX40]",
+    "[XXXXX984]",
+    "[XXXXX584]",
+    "[XXXXX132]",
+    "[XXXXX1084]",
+    "[XXXXX549]",
+    "[XXXXX810]",
+    "[XXXXX208]",
+    "[XXXXX609]",
+    "[XXXXX275]",
+    "[XXXXX262]",
+    "[XXXXX331]",
+    "[XXXXX361]",
+    "[XXXXX561]",
+    "[XXXXX143]",
+    "[XXXXX1061]",
+    "[XXXXX59]",
+    "[XXXXX21]",
+    "[XXXXX995]",
+    "[XXXXX439]",
+    "[XXXXX831]",
+    "[XXXXX745]",
+    "[XXXXX43]",
+    "[XXXXX498]",
+    "[XXXXX575]",
+    "[XXXXX624]",
+    "[XXXXX72]",
+    "[XXXXX921]",
+    "[XXXXX201]",
+    "[XXXXX1171]",
+    "[XXXXX912]",
+    "[XXXXX775]",
+    "[XXXXX1094]",
+    "[XXXXX80]",
+    "[XXXXX894]",
+    "[XXXXX876]",
+    "[XXXXX1143]",
+    "[XXXXX355]",
+    "[XXXXX450]",
+    "[XXXXX642]",
+    "[XXXXX162]",
+    "[XXXXX394]",
+    "[XXXXX296]",
+    "[XXXXX536]",
+    "[XXXXX560]",
+    "[XXXXX373]",
+    "[XXXXX674]",
+    "[XXXXX1099]",
+    "[XXXXX797]",
+    "[XXXXX516]",
+    "[XXXXX41]",
+    "[XXXXX594]",
+    "[XXXXX319]",
+    "[XXXXX165]",
+    "[XXXXX344]",
+    "[XXXXX1012]",
+    "[XXXXX942]",
+    "[XXXXX1071]",
+    "[XXXXX648]",
+    "[XXXXX4]",
+    "[XXXXX633]",
+    "[XXXXX551]",
+    "[XXXXX586]",
+    "[XXXXX578]",
+    "[XXXXX348]",
+    "[XXXXX614]",
+    "[XXXXX33]",
+    "[XXXXX1193]",
+    "[XXXXX737]",
+    "[XXXXX233]",
+    "[XXXXX933]",
+    "[XXXXX781]",
+    "[XXXXX189]",
+    "[XXXXX1006]",
+    "[XXXXX219]",
+    "[XXXXX454]",
+    "[XXXXX449]",
+    "[XXXXX697]",
+    "[XXXXX724]",
+    "[XXXXX224]",
+    "[XXXXX1157]",
+    "[XXXXX455]",
+    "[XXXXX505]",
+    "[XXXXX96]",
+    "[XXXXX182]",
+    "[XXXXX424]",
+    "[XXXXX872]",
+    "[XXXXX1163]",
+    "[XXXXX1081]",
+    "[XXXXX1173]",
+    "[XXXXX172]",
+    "[XXXXX507]",
+    "[XXXXX920]",
+    "[XXXXX1126]",
+    "[XXXXX474]",
+    "[XXXXX971]",
+    "[XXXXX1049]",
+    "[XXXXX1114]",
+    "[XXXXX1040]",
+    "[XXXXX79]",
+    "[XXXXX36]",
+    "[XXXXX977]",
+    "[XXXXX1129]",
+    "[XXXXX1063]",
+    "[XXXXX773]",
+    "[XXXXX16]",
+    "[XXXXX113]",
+    "[XXXXX320]",
+    "[XXXXX167]",
+    "[XXXXX751]",
+    "[XXXXX656]",
+    "[XXXXX543]",
+    "[XXXXX828]",
+    "[XXXXX915]",
+    "[XXXXX1098]",
+    "[XXXXX569]",
+    "[XXXXX1166]",
+    "[XXXXX846]",
+    "[XXXXX324]",
+    "[XXXXX802]",
+    "[XXXXX588]",
+    "[XXXXX739]",
+    "[XXXXX50]",
+    "[XXXXX882]",
+    "[XXXXX1054]",
+    "[XXXXX621]",
+    "[XXXXX585]",
+    "[XXXXX121]",
+    "[XXXXX647]",
+    "[XXXXX767]",
+    "[XXXXX645]",
+    "[XXXXX27]",
+    "[XXXXX907]",
+    "[XXXXX1109]",
+    "[XXXXX1147]",
+    "[XXXXX501]",
+    "[XXXXX1164]",
+    "[XXXXX650]",
+    "[XXXXX329]",
+    "[XXXXX659]",
+    "[XXXXX1158]",
+    "[XXXXX104]",
+    "[XXXXX629]",
+    "[XXXXX868]",
+    "[XXXXX779]",
+    "[XXXXX17]",
+    "[XXXXX377]",
+    "[XXXXX467]",
+    "[XXXXX917]",
+    "[XXXXX830]",
+    "[XXXXX533]",
+    "[XXXXX862]",
+    "[XXXXX1041]",
+    "[XXXXX542]",
+    "[XXXXX595]",
+    "[XXXXX353]",
+    "[XXXXX181]",
+    "[XXXXX662]",
+    "[XXXXX1195]",
+    "[XXXXX435]",
+    "[XXXXX1051]",
+    "[XXXXX703]",
+    "[XXXXX1140]",
+    "[XXXXX750]",
+    "[XXXXX57]",
+    "[XXXXX75]",
+    "[XXXXX293]",
+    "[XXXXX681]",
+    "[XXXXX46]",
+    "[XXXXX30]",
+    "[XXXXX402]",
+    "[XXXXX803]",
+    "[XXXXX821]",
+    "[XXXXX871]",
+    "[XXXXX980]",
+    "[XXXXX990]",
+    "[XXXXX626]",
+    "[XXXXX90]",
+    "[XXXXX77]",
+    "[XXXXX819]",
+    "[XXXXX436]",
+    "[XXXXX306]",
+    "[XXXXX434]",
+    "[XXXXX88]",
+    "[XXXXX184]",
+    "[XXXXX290]",
+    "[XXXXX216]",
+    "[XXXXX176]",
+    "[XXXXX747]",
+    "[XXXXX938]",
+    "[XXXXX442]",
+    "[XXXXX301]",
+    "[XXXXX1015]",
+    "[XXXXX520]",
+    "[XXXXX829]",
+    "[XXXXX106]",
+    "[XXXXX1048]",
+    "[XXXXX923]",
+    "[XXXXX958]",
+    "[XXXXX1130]",
+    "[XXXXX89]",
+    "[XXXXX548]",
+    "[XXXXX863]",
+    "[XXXXX444]",
+    "[XXXXX1149]",
+    "[XXXXX866]",
+    "[XXXXX109]",
+    "[XXXXX550]",
+    "[XXXXX727]",
+    "[XXXXX646]",
+    "[XXXXX702]",
+    "[XXXXX204]",
+    "[XXXXX1000]",
+    "[XXXXX591]",
+    "[XXXXX1128]",
+    "[XXXXX937]",
+    "[XXXXX141]",
+    "[XXXXX273]",
+    "[XXXXX1183]",
+    "[XXXXX765]",
+    "[XXXXX371]",
+    "[XXXXX256]",
+    "[XXXXX964]",
+    "[XXXXX1148]",
+    "[XXXXX930]",
+    "[XXXXX212]",
+    "[XXXXX133]",
+    "[XXXXX608]",
+    "[XXXXX281]",
+    "[XXXXX1045]",
+    "[XXXXX928]",
+    "[XXXXX1154]",
+    "[XXXXX334]",
+    "[XXXXX861]",
+    "[XXXXX87]",
+    "[XXXXX941]",
+    "[XXXXX359]",
+    "[XXXXX55]",
+    "[XXXXX842]",
+    "[XXXXX427]",
+    "[XXXXX491]",
+    "[XXXXX137]",
+    "[XXXXX47]",
+    "[XXXXX511]",
+    "[XXXXX81]",
+    "[XXXXX139]",
+    "[XXXXX226]",
+    "[XXXXX563]",
+    "[XXXXX708]",
+    "[XXXXX820]",
+    "[XXXXX65]",
+    "[XXXXX381]",
+    "[XXXXX22]",
+    "[XXXXX1064]",
+    "[XXXXX874]",
+    "[XXXXX944]",
+    "[XXXXX6]",
+    "[XXXXX313]",
+    "[XXXXX471]",
+    "[XXXXX1118]",
+    "[XXXXX849]",
+    "[XXXXX483]",
+    "[XXXXX713]",
+    "[XXXXX222]",
+    "[XXXXX640]",
+    "[XXXXX12]",
+    "[XXXXX285]",
+    "[XXXXX327]",
+    "[XXXXX378]",
+    "[XXXXX790]",
+    "[XXXXX1179]",
+    "[XXXXX15]",
+    "[XXXXX229]",
+    "[XXXXX771]",
+    "[XXXXX559]",
+    "[XXXXX253]",
+    "[XXXXX644]",
+    "[XXXXX811]",
+    "[XXXXX554]",
+    "[XXXXX744]",
+    "[XXXXX786]",
+    "[XXXXX913]",
+    "[XXXXX966]",
+    "[XXXXX972]",
+    "[XXXXX664]",
+    "[XXXXX1131]",
+    "[XXXXX1132]",
+    "[XXXXX376]",
+    "[XXXXX672]",
+    "[XXXXX1013]",
+    "[XXXXX328]",
+    "[XXXXX889]",
+    "[XXXXX1141]",
+    "[XXXXX429]",
+    "[XXXXX114]",
+    "[XXXXX523]",
+    "[XXXXX833]",
+    "[XXXXX927]",
+    "[XXXXX297]",
+    "[XXXXX812]",
+    "[XXXXX1062]",
+    "[XXXXX152]",
+    "[XXXXX1010]",
+    "[XXXXX690]",
+    "[XXXXX363]",
+    "[XXXXX1103]",
+    "[XXXXX194]",
+    "[XXXXX844]",
+    "[XXXXX667]",
+    "[XXXXX720]",
+    "[XXXXX770]",
+    "[XXXXX515]",
+    "[XXXXX196]",
+    "[XXXXX283]",
+    "[XXXXX95]",
+    "[XXXXX528]",
+    "[XXXXX749]",
+    "[XXXXX934]",
+    "[XXXXX1052]",
+    "[XXXXX1188]",
+    "[XXXXX1001]",
+    "[XXXXX785]",
+    "[XXXXX393]",
+    "[XXXXX502]",
+    "[XXXXX404]",
+    "[XXXXX129]",
+    "[XXXXX643]",
+    "[XXXXX1086]",
+    "[XXXXX1104]",
+    "[XXXXX326]",
+    "[XXXXX330]",
+    "[XXXXX620]",
+    "[XXXXX193]",
+    "[XXXXX470]",
+    "[XXXXX279]",
+    "[XXXXX97]",
+    "[XXXXX177]",
+    "[XXXXX510]",
+    "[XXXXX225]",
+    "[XXXXX572]",
+    "[XXXXX777]",
+    "[XXXXX1135]",
+    "[XXXXX1160]",
+    "[XXXXX558]",
+    "[XXXXX416]",
+    "[XXXXX246]",
+    "[XXXXX799]",
+    "[XXXXX153]",
+    "[XXXXX671]",
+    "[XXXXX809]",
+    "[XXXXX525]",
+    "[XXXXX1004]",
+    "[XXXXX1005]",
+    "[XXXXX1194]",
+    "[XXXXX408]",
+    "[XXXXX492]",
+    "[XXXXX973]",
+    "[XXXXX242]",
+    "[XXXXX357]",
+    "[XXXXX851]",
+    "[XXXXX999]",
+    "[XXXXX969]",
+    "[XXXXX453]",
+    "[XXXXX730]",
+    "[XXXXX54]",
+    "[XXXXX480]",
+    "[XXXXX564]",
+    "[XXXXX600]",
+    "[XXXXX654]",
+    "[XXXXX37]",
+    "[XXXXX605]",
+    "[XXXXX164]",
+    "[XXXXX395]",
+    "[XXXXX401]",
+    "[XXXXX446]",
+    "[XXXXX658]",
+    "[XXXXX60]",
+    "[XXXXX107]",
+    "[XXXXX131]",
+    "[XXXXX387]",
+    "[XXXXX67]",
+    "[XXXXX743]",
+    "[XXXXX657]",
+    "[XXXXX418]",
+    "[XXXXX787]",
+    "[XXXXX838]",
+    "[XXXXX298]",
+    "[XXXXX1007]",
+    "[XXXXX1027]",
+    "[XXXXX1078]",
+    "[XXXXX651]",
+    "[XXXXX875]",
+    "[XXXXX258]",
+    "[XXXXX413]",
+    "[XXXXX1092]",
+    "[XXXXX124]",
+    "[XXXXX308]",
+    "[XXXXX1085]",
+    "[XXXXX1170]",
+    "[XXXXX38]",
+    "[XXXXX438]",
+    "[XXXXX475]",
+    "[XXXXX900]",
+    "[XXXXX263]",
+    "[XXXXX166]",
+    "[XXXXX698]",
+    "[XXXXX1127]",
+    "[XXXXX1181]",
+    "[XXXXX763]",
+    "[XXXXX627]",
+    "[XXXXX250]",
+    "[XXXXX757]",
+    "[XXXXX142]",
+    "[XXXXX794]",
+    "[XXXXX128]",
+    "[XXXXX35]",
+    "[XXXXX482]",
+    "[XXXXX834]",
+    "[XXXXX679]",
+    "[XXXXX839]",
+    "[XXXXX138]",
+    "[XXXXX1066]",
+    "[XXXXX116]",
+    "[XXXXX317]",
+    "[XXXXX490]",
+    "[XXXXX443]",
+    "[XXXXX879]",
+    "[XXXXX1101]",
+    "[XXXXX711]",
+    "[XXXXX123]",
+    "[XXXXX1152]",
+    "[XXXXX692]",
+    "[XXXXX555]",
+    "[XXXXX266]",
+    "[XXXXX0]",
+    "[XXXXX332]",
+    "[XXXXX497]",
+    "[XXXXX904]",
+    "[XXXXX1019]",
+    "[XXXXX709]",
+    "[XXXXX351]",
+    "[XXXXX764]",
+    "[XXXXX625]",
+    "[XXXXX798]",
+    "[XXXXX205]",
+    "[XXXXX922]",
+    "[XXXXX1075]",
+    "[XXXXX8]",
+    "[XXXXX422]",
+    "[XXXXX638]",
+    "[XXXXX98]",
+    "[XXXXX814]",
+    "[XXXXX1057]",
+    "[XXXXX1070]",
+    "[XXXXX411]",
+    "[XXXXX768]",
+    "[XXXXX1165]",
+    "[XXXXX888]",
+    "[XXXXX636]",
+    "[XXXXX1047]",
+    "[XXXXX960]",
+    "[XXXXX910]",
+    "[XXXXX42]",
+    "[XXXXX238]",
+    "[XXXXX383]",
+    "[XXXXX1032]",
+    "[XXXXX577]",
+    "[XXXXX268]",
+    "[XXXXX179]",
+    "[XXXXX1076]",
+    "[XXXXX1167]",
+    "[XXXXX1172]",
+    "[XXXXX665]",
+    "[XXXXX126]",
+    "[XXXXX347]",
+    "[XXXXX544]",
+    "[XXXXX1026]",
+    "[XXXXX159]",
+    "[XXXXX503]",
+    "[XXXXX583]",
+    "[XXXXX668]",
+    "[XXXXX1115]",
+    "[XXXXX112]",
+    "[XXXXX154]",
+    "[XXXXX761]",
+    "[XXXXX540]",
+    "[XXXXX755]",
+    "[XXXXX255]",
+    "[XXXXX459]",
+    "[XXXXX1090]",
+    "[XXXXX689]",
+    "[XXXXX589]",
+    "[XXXXX122]",
+    "[XXXXX789]",
+    "[XXXXX481]",
+    "[XXXXX1044]",
+    "[XXXXX274]",
+    "[XXXXX957]",
+    "[XXXXX1121]",
+    "[XXXXX464]",
+    "[XXXXX495]",
+    "[XXXXX1097]",
+    "[XXXXX581]",
+    "[XXXXX200]",
+    "[XXXXX954]",
+    "[XXXXX48]",
+    "[XXXXX855]",
+    "[XXXXX419]",
+    "[XXXXX919]",
+    "[XXXXX451]",
+    "[XXXXX869]",
+    "[XXXXX1082]",
+    "[XXXXX1185]",
+    "[XXXXX859]",
+    "[XXXXX1139]",
+    "[XXXXX1072]",
+    "[XXXXX220]",
+    "[XXXXX465]",
+    "[XXXXX1105]",
+    "[XXXXX1093]",
+    "[XXXXX953]",
+    "[XXXXX1187]",
+    "[XXXXX311]",
+    "[XXXXX403]",
+    "[XXXXX684]",
+    "[XXXXX426]",
+    "[XXXXX68]",
+    "[XXXXX277]",
+    "[XXXXX437]",
+    "[XXXXX896]",
+    "[XXXXX899]",
+    "[XXXXX878]",
+    "[XXXXX1073]",
+    "[XXXXX774]",
+    "[XXXXX315]",
+    "[XXXXX631]",
+    "[XXXXX223]",
+    "[XXXXX86]",
+    "[XXXXX607]",
+    "[XXXXX149]",
+    "[XXXXX566]",
+    "[XXXXX740]",
+    "[XXXXX265]",
+    "[XXXXX399]",
+    "[XXXXX1053]",
+    "[XXXXX1077]",
+    "[XXXXX336]",
+    "[XXXXX734]",
+    "[XXXXX289]",
+    "[XXXXX221]",
+    "[XXXXX580]",
+    "[XXXXX406]",
+    "[XXXXX854]",
+    "[XXXXX673]",
+    "[XXXXX1146]",
+    "[XXXXX29]",
+    "[XXXXX496]",
+    "[XXXXX241]",
+    "[XXXXX929]",
+    "[XXXXX169]",
+    "[XXXXX1153]",
+    "[XXXXX440]",
+    "[XXXXX385]",
+    "[XXXXX499]",
+    "[XXXXX718]",
+    "[XXXXX848]",
+    "[XXXXX877]",
+    "[XXXXX641]",
+    "[XXXXX1137]",
+    "[XXXXX269]",
+    "[XXXXX521]",
+    "[XXXXX234]",
+    "[XXXXX987]",
+    "[XXXXX741]",
+    "[XXXXX349]",
+    "[XXXXX93]",
+    "[XXXXX247]",
+    "[XXXXX961]",
+    "[XXXXX1120]",
+    "[XXXXX369]",
+    "[XXXXX479]",
+    "[XXXXX892]",
+    "[XXXXX553]",
+    "[XXXXX906]",
+    "[XXXXX571]",
+    "[XXXXX1046]",
+    "[XXXXX206]",
+    "[XXXXX488]",
+    "[XXXXX445]",
+    "[XXXXX524]",
+    "[XXXXX228]",
+    "[XXXXX998]",
+    "[XXXXX675]",
+    "[XXXXX732]",
+    "[XXXXX1028]",
+    "[XXXXX494]",
+    "[XXXXX965]",
+    "[XXXXX1021]",
+    "[XXXXX726]",
+    "[XXXXX168]",
+    "[XXXXX606]",
+    "[XXXXX974]",
+    "[XXXXX504]",
+    "[XXXXX556]",
+    "[XXXXX1159]",
+    "[XXXXX417]",
+    "[XXXXX1175]",
+    "[XXXXX1142]",
+    "[XXXXX753]",
+    "[XXXXX824]",
+    "[XXXXX251]",
+    "[XXXXX345]",
+    "[XXXXX610]",
+    "[XXXXX276]",
+    "[XXXXX783]",
+    "[XXXXX74]",
+    "[XXXXX1050]",
+    "[XXXXX71]",
+    "[XXXXX360]",
+    "[XXXXX512]",
+    "[XXXXX341]",
+    "[XXXXX375]",
+    "[XXXXX637]",
+    "[XXXXX1116]",
+    "[XXXXX9]",
+    "[XXXXX738]",
+    "[XXXXX856]",
+    "[XXXXX979]",
+    "[XXXXX825]",
+    "[XXXXX852]",
+    "[XXXXX215]",
+    "[XXXXX91]",
+    "[XXXXX257]",
+    "[XXXXX325]",
+    "[XXXXX368]",
+    "[XXXXX1124]",
+    "[XXXXX188]",
+    "[XXXXX905]",
+    "[XXXXX284]",
+    "[XXXXX11]",
+    "[XXXXX83]",
+    "[XXXXX611]",
+    "[XXXXX51]",
+    "[XXXXX99]",
+    "[XXXXX288]",
+    "[XXXXX945]",
+    "[XXXXX975]",
+    "[XXXXX282]",
+    "[XXXXX1068]",
+    "[XXXXX1180]",
+    "[XXXXX574]",
+    "[XXXXX305]",
+    "[XXXXX565]",
+    "[XXXXX865]",
+    "[XXXXX1014]",
+    "[XXXXX1138]",
+    "[XXXXX134]",
+    "[XXXXX321]",
+    "[XXXXX752]",
+    "[XXXXX108]",
+    "[XXXXX339]",
+    "[XXXXX893]",
+    "[XXXXX1177]",
+    "[XXXXX988]",
+    "[XXXXX719]",
+    "[XXXXX1059]",
+    "[XXXXX421]",
+    "[XXXXX670]",
+    "[XXXXX949]",
+    "[XXXXX14]",
+    "[XXXXX1087]",
+    "[XXXXX382]",
+    "[XXXXX531]",
+    "[XXXXX1079]",
+    "[XXXXX1067]",
+    "[XXXXX655]",
+    "[XXXXX340]",
+    "[XXXXX398]",
+    "[XXXXX175]",
+    "[XXXXX232]",
+    "[XXXXX244]",
+    "[XXXXX568]",
+    "[XXXXX1110]",
+    "[XXXXX191]",
+    "[XXXXX931]",
+    "[XXXXX832]",
+    "[XXXXX1030]",
+    "[XXXXX144]",
+    "[XXXXX1102]",
+    "[XXXXX300]",
+    "[XXXXX996]",
+    "[XXXXX248]",
+    "[XXXXX66]",
+    "[XXXXX735]",
+    "[XXXXX20]",
+    "[XXXXX694]",
+    "[XXXXX845]",
+    "[XXXXX500]",
+    "[XXXXX613]",
+    "[XXXXX117]",
+    "[XXXXX886]",
+    "[XXXXX261]",
+    "[XXXXX567]",
+    "[XXXXX541]",
+    "[XXXXX529]",
+    "[XXXXX699]",
+    "[XXXXX715]",
+    "[XXXXX1003]",
+    "[XXXXX518]",
+    "[XXXXX891]",
+    "[XXXXX192]",
+    "[XXXXX53]",
+    "[XXXXX428]",
+    "[XXXXX183]",
+    "[XXXXX850]",
+    "[XXXXX539]",
+    "[XXXXX527]",
+    "[XXXXX660]",
+    "[XXXXX409]",
+    "[XXXXX1122]",
+    "[XXXXX898]",
+    "[XXXXX858]",
+    "[XXXXX534]",
+    "[XXXXX756]",
+    "[XXXXX63]",
+    "[XXXXX49]",
+    "[XXXXX1106]",
+    "[XXXXX111]",
+    "[XXXXX955]",
+    "[XXXXX309]",
+    "[XXXXX538]",
+    "[XXXXX823]",
+    "[XXXXX867]",
+    "[XXXXX587]",
+    "[XXXXX243]",
+    "[XXXXX677]",
+    "[XXXXX1017]",
+    "[XXXXX173]",
+    "[XXXXX682]",
+    "[XXXXX1184]",
+    "[XXXXX1009]",
+    "[XXXXX704]",
+    "[XXXXX532]",
+    "[XXXXX901]",
+    "[XXXXX120]",
+    "[XXXXX388]",
+    "[XXXXX405]",
+    "[XXXXX788]",
+    "[XXXXX728]",
+    "[XXXXX425]",
+    "[XXXXX883]",
+    "[XXXXX837]",
+    "[XXXXX350]",
+    "[XXXXX716]",
+    "[XXXXX579]",
+    "[XXXXX909]",
+    "[XXXXX884]",
+    "[XXXXX1011]",
+    "[XXXXX372]",
+    "[XXXXX370]",
+    "[XXXXX468]",
+    "[XXXXX688]",
+    "[XXXXX56]",
+    "[XXXXX653]",
+    "[XXXXX623]",
+    "[XXXXX552]",
+    "[XXXXX314]",
+    "[XXXXX1018]",
+    "[XXXXX1091]",
+    "[XXXXX342]",
+    "[XXXXX1065]",
+    "[XXXXX1161]",
+    "[XXXXX959]",
+    "[XXXXX1035]",
+    "[XXXXX530]",
+    "[XXXXX69]",
+    "[XXXXX489]",
+    "[XXXXX1]",
+    "[XXXXX207]",
+    "[XXXXX433]",
+    "[XXXXX358]",
+    "[XXXXX601]",
+    "[XXXXX2]",
+    "[XXXXX722]",
+    "[XXXXX985]",
+    "[XXXXX908]",
+    "[XXXXX1024]",
+    "[XXXXX1111]",
+    "[XXXXX1113]",
+    "[XXXXX678]",
+    "[XXXXX19]",
+    "[XXXXX1186]",
+    "[XXXXX808]",
+    "[XXXXX493]",
+    "[XXXXX1155]",
+    "[XXXXX217]",
+    "[XXXXX616]",
+    "[XXXXX835]",
+    "[XXXXX994]",
+    "[XXXXX484]",
+    "[XXXXX362]",
+    "[XXXXX582]",
+    "[XXXXX1034]",
+    "[XXXXX772]",
+    "[XXXXX457]",
+    "[XXXXX604]",
+    "[XXXXX456]",
+    "[XXXXX145]",
+    "[XXXXX476]",
+    "[XXXXX218]",
+    "[XXXXX199]",
+    "[XXXXX364]",
+    "[XXXXX302]",
+    "[XXXXX881]",
+    "[XXXXX1088]",
+    "[XXXXX356]",
+    "[XXXXX936]",
+    "[XXXXX978]",
+    "[XXXXX895]",
+    "[XXXXX993]",
+    "[XXXXX458]",
+    "[XXXXX1002]",
+    "[XXXXX1022]",
+    "[XXXXX25]",
+    "[XXXXX766]",
+    "[XXXXX64]",
+    "[XXXXX211]",
+    "[XXXXX82]",
+    "[XXXXX634]",
+    "[XXXXX661]",
+    "[XXXXX748]",
+    "[XXXXX778]",
+    "[XXXXX24]",
+    "[XXXXX448]",
+    "[XXXXX887]",
+    "[XXXXX890]",
+    "[XXXXX754]",
+    "[XXXXX343]",
+    "[XXXXX1069]",
+    "[XXXXX294]",
+    "[XXXXX295]",
+    "[XXXXX197]",
+    "[XXXXX210]",
+    "[XXXXX630]",
+    "[XXXXX707]",
+    "[XXXXX864]",
+    "[XXXXX391]",
+    "[XXXXX338]",
+    "[XXXXX800]",
+    "[XXXXX841]",
+    "[XXXXX967]",
+    "[XXXXX793]",
+    "[XXXXX916]",
+    "[XXXXX603]",
+    "[XXXXX270]",
+    "[XXXXX105]",
+    "[XXXXX1016]",
+    "[XXXXX804]",
+    "[XXXXX264]",
+    "[XXXXX155]",
+    "[XXXXX1190]",
+    "[XXXXX813]",
+    "[XXXXX546]"
+  ],
+  "bos_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-1500/spiece.model ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cfcecb41352908b2a034dc4ec11e511d1a7ed2d594a679147bc9b51bfe6bd14a
+size 1500527

checkpoint-1500/tokenizer_config.json ADDED Viewed

The diff for this file is too large to render. See raw diff

checkpoint-1500/trainer_state.json ADDED Viewed

	@@ -0,0 +1,3034 @@

+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.09653752091646287,
+  "eval_steps": 500,
+  "global_step": 1500,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 10.69194049835205,
+      "epoch": 0.00032179173638820956,
+      "grad_norm": 12.75,
+      "learning_rate": 2e-06,
+      "loss": 10.854,
+      "mean_token_accuracy": 0.0,
+      "num_tokens": 11532.0,
+      "step": 5
+    },
+    {
+      "entropy": 10.691917896270752,
+      "epoch": 0.0006435834727764191,
+      "grad_norm": 16.25,
+      "learning_rate": 4.5e-06,
+      "loss": 10.7683,
+      "mean_token_accuracy": 0.0002785657299682498,
+      "num_tokens": 21733.0,
+      "step": 10
+    },
+    {
+      "entropy": 10.69142770767212,
+      "epoch": 0.0009653752091646286,
+      "grad_norm": 9.8125,
+      "learning_rate": 7e-06,
+      "loss": 10.5426,
+      "mean_token_accuracy": 0.009345025499351322,
+      "num_tokens": 32040.0,
+      "step": 15
+    },
+    {
+      "entropy": 10.687415981292725,
+      "epoch": 0.0012871669455528382,
+      "grad_norm": 6.09375,
+      "learning_rate": 9.5e-06,
+      "loss": 10.2464,
+      "mean_token_accuracy": 0.035014028288424016,
+      "num_tokens": 42645.0,
+      "step": 20
+    },
+    {
+      "entropy": 10.665467548370362,
+      "epoch": 0.0016089586819410479,
+      "grad_norm": 4.125,
+      "learning_rate": 1.2e-05,
+      "loss": 9.9923,
+      "mean_token_accuracy": 0.03743175081908703,
+      "num_tokens": 54374.0,
+      "step": 25
+    },
+    {
+      "entropy": 10.618990993499756,
+      "epoch": 0.0019307504183292573,
+      "grad_norm": 3.09375,
+      "learning_rate": 1.4500000000000002e-05,
+      "loss": 9.8534,
+      "mean_token_accuracy": 0.03550823628902435,
+      "num_tokens": 66590.0,
+      "step": 30
+    },
+    {
+      "entropy": 10.615287399291992,
+      "epoch": 0.002252542154717467,
+      "grad_norm": 3.109375,
+      "learning_rate": 1.7000000000000003e-05,
+      "loss": 9.7196,
+      "mean_token_accuracy": 0.03219117037951946,
+      "num_tokens": 78832.0,
+      "step": 35
+    },
+    {
+      "entropy": 10.627141761779786,
+      "epoch": 0.0025743338911056765,
+      "grad_norm": 2.703125,
+      "learning_rate": 1.95e-05,
+      "loss": 9.6667,
+      "mean_token_accuracy": 0.046536854654550555,
+      "num_tokens": 89158.0,
+      "step": 40
+    },
+    {
+      "entropy": 10.622460842132568,
+      "epoch": 0.002896125627493886,
+      "grad_norm": 2.671875,
+      "learning_rate": 2.2e-05,
+      "loss": 9.6509,
+      "mean_token_accuracy": 0.04088520556688309,
+      "num_tokens": 99546.0,
+      "step": 45
+    },
+    {
+      "entropy": 10.604833221435547,
+      "epoch": 0.0032179173638820957,
+      "grad_norm": 2.46875,
+      "learning_rate": 2.4500000000000003e-05,
+      "loss": 9.6314,
+      "mean_token_accuracy": 0.043538968078792095,
+      "num_tokens": 110770.0,
+      "step": 50
+    },
+    {
+      "entropy": 10.580564498901367,
+      "epoch": 0.003539709100270305,
+      "grad_norm": 2.40625,
+      "learning_rate": 2.7e-05,
+      "loss": 9.5314,
+      "mean_token_accuracy": 0.04419821500778198,
+      "num_tokens": 122202.0,
+      "step": 55
+    },
+    {
+      "entropy": 10.5360502243042,
+      "epoch": 0.0038615008366585145,
+      "grad_norm": 3.015625,
+      "learning_rate": 2.95e-05,
+      "loss": 9.4058,
+      "mean_token_accuracy": 0.04586403630673885,
+      "num_tokens": 134645.0,
+      "step": 60
+    },
+    {
+      "entropy": 10.41978416442871,
+      "epoch": 0.004183292573046725,
+      "grad_norm": 2.375,
+      "learning_rate": 3.2e-05,
+      "loss": 9.3388,
+      "mean_token_accuracy": 0.05442853756248951,
+      "num_tokens": 146962.0,
+      "step": 65
+    },
+    {
+      "entropy": 10.424466228485107,
+      "epoch": 0.004505084309434934,
+      "grad_norm": 2.78125,
+      "learning_rate": 3.4500000000000005e-05,
+      "loss": 9.2536,
+      "mean_token_accuracy": 0.05842369645833969,
+      "num_tokens": 158200.0,
+      "step": 70
+    },
+    {
+      "entropy": 10.4311918258667,
+      "epoch": 0.004826876045823143,
+      "grad_norm": 2.34375,
+      "learning_rate": 3.7e-05,
+      "loss": 9.0956,
+      "mean_token_accuracy": 0.0712333232164383,
+      "num_tokens": 168437.0,
+      "step": 75
+    },
+    {
+      "entropy": 10.301731491088868,
+      "epoch": 0.005148667782211353,
+      "grad_norm": 2.40625,
+      "learning_rate": 3.95e-05,
+      "loss": 9.0239,
+      "mean_token_accuracy": 0.06886589303612708,
+      "num_tokens": 178953.0,
+      "step": 80
+    },
+    {
+      "entropy": 10.311075687408447,
+      "epoch": 0.005470459518599562,
+      "grad_norm": 2.4375,
+      "learning_rate": 4.2000000000000004e-05,
+      "loss": 8.941,
+      "mean_token_accuracy": 0.07178018912672997,
+      "num_tokens": 189910.0,
+      "step": 85
+    },
+    {
+      "entropy": 10.289142322540282,
+      "epoch": 0.005792251254987772,
+      "grad_norm": 2.265625,
+      "learning_rate": 4.45e-05,
+      "loss": 8.8619,
+      "mean_token_accuracy": 0.06988794207572938,
+      "num_tokens": 201355.0,
+      "step": 90
+    },
+    {
+      "entropy": 10.256867218017579,
+      "epoch": 0.006114042991375981,
+      "grad_norm": 2.578125,
+      "learning_rate": 4.7000000000000004e-05,
+      "loss": 8.7513,
+      "mean_token_accuracy": 0.07205671109259129,
+      "num_tokens": 212839.0,
+      "step": 95
+    },
+    {
+      "entropy": 10.217705726623535,
+      "epoch": 0.0064358347277641914,
+      "grad_norm": 2.1875,
+      "learning_rate": 4.9500000000000004e-05,
+      "loss": 8.6829,
+      "mean_token_accuracy": 0.07173232845962048,
+      "num_tokens": 224453.0,
+      "step": 100
+    },
+    {
+      "entropy": 10.12640552520752,
+      "epoch": 0.006757626464152401,
+      "grad_norm": 2.125,
+      "learning_rate": 5.2e-05,
+      "loss": 8.5053,
+      "mean_token_accuracy": 0.07860757485032081,
+      "num_tokens": 236010.0,
+      "step": 105
+    },
+    {
+      "entropy": 10.044542694091797,
+      "epoch": 0.00707941820054061,
+      "grad_norm": 2.796875,
+      "learning_rate": 5.45e-05,
+      "loss": 8.4314,
+      "mean_token_accuracy": 0.07552603259682655,
+      "num_tokens": 247287.0,
+      "step": 110
+    },
+    {
+      "entropy": 10.028972911834718,
+      "epoch": 0.00740120993692882,
+      "grad_norm": 2.328125,
+      "learning_rate": 5.7e-05,
+      "loss": 8.3188,
+      "mean_token_accuracy": 0.0754437144845724,
+      "num_tokens": 258956.0,
+      "step": 115
+    },
+    {
+      "entropy": 9.909952545166016,
+      "epoch": 0.007723001673317029,
+      "grad_norm": 1.9765625,
+      "learning_rate": 5.9499999999999996e-05,
+      "loss": 8.2245,
+      "mean_token_accuracy": 0.08083591908216477,
+      "num_tokens": 270356.0,
+      "step": 120
+    },
+    {
+      "entropy": 9.780113220214844,
+      "epoch": 0.008044793409705238,
+      "grad_norm": 2.21875,
+      "learning_rate": 6.2e-05,
+      "loss": 8.0979,
+      "mean_token_accuracy": 0.08179116398096084,
+      "num_tokens": 281711.0,
+      "step": 125
+    },
+    {
+      "entropy": 9.635583400726318,
+      "epoch": 0.00836658514609345,
+      "grad_norm": 2.140625,
+      "learning_rate": 6.450000000000001e-05,
+      "loss": 7.9786,
+      "mean_token_accuracy": 0.08390471749007702,
+      "num_tokens": 291995.0,
+      "step": 130
+    },
+    {
+      "entropy": 9.590358066558839,
+      "epoch": 0.008688376882481658,
+      "grad_norm": 2.203125,
+      "learning_rate": 6.7e-05,
+      "loss": 7.9706,
+      "mean_token_accuracy": 0.07811799943447113,
+      "num_tokens": 304222.0,
+      "step": 135
+    },
+    {
+      "entropy": 9.40874490737915,
+      "epoch": 0.009010168618869868,
+      "grad_norm": 1.7109375,
+      "learning_rate": 6.950000000000001e-05,
+      "loss": 7.8908,
+      "mean_token_accuracy": 0.0770251952111721,
+      "num_tokens": 316104.0,
+      "step": 140
+    },
+    {
+      "entropy": 9.228489303588868,
+      "epoch": 0.009331960355258077,
+      "grad_norm": 1.828125,
+      "learning_rate": 7.2e-05,
+      "loss": 7.7264,
+      "mean_token_accuracy": 0.0914445884525776,
+      "num_tokens": 327511.0,
+      "step": 145
+    },
+    {
+      "entropy": 9.002948188781739,
+      "epoch": 0.009653752091646286,
+      "grad_norm": 1.96875,
+      "learning_rate": 7.45e-05,
+      "loss": 7.5834,
+      "mean_token_accuracy": 0.0950602874159813,
+      "num_tokens": 338523.0,
+      "step": 150
+    },
+    {
+      "entropy": 8.787215518951417,
+      "epoch": 0.009975543828034497,
+      "grad_norm": 1.8828125,
+      "learning_rate": 7.7e-05,
+      "loss": 7.5805,
+      "mean_token_accuracy": 0.09019249975681305,
+      "num_tokens": 350247.0,
+      "step": 155
+    },
+    {
+      "entropy": 8.719713592529297,
+      "epoch": 0.010297335564422706,
+      "grad_norm": 1.328125,
+      "learning_rate": 7.950000000000001e-05,
+      "loss": 7.4723,
+      "mean_token_accuracy": 0.09186024889349938,
+      "num_tokens": 362539.0,
+      "step": 160
+    },
+    {
+      "entropy": 8.630542182922364,
+      "epoch": 0.010619127300810915,
+      "grad_norm": 1.3359375,
+      "learning_rate": 8.2e-05,
+      "loss": 7.5412,
+      "mean_token_accuracy": 0.08679113686084747,
+      "num_tokens": 374773.0,
+      "step": 165
+    },
+    {
+      "entropy": 8.505796527862548,
+      "epoch": 0.010940919037199124,
+      "grad_norm": 1.703125,
+      "learning_rate": 8.450000000000001e-05,
+      "loss": 7.3633,
+      "mean_token_accuracy": 0.08832342699170112,
+      "num_tokens": 385292.0,
+      "step": 170
+    },
+    {
+      "entropy": 8.319984436035156,
+      "epoch": 0.011262710773587334,
+      "grad_norm": 1.3984375,
+      "learning_rate": 8.7e-05,
+      "loss": 7.3207,
+      "mean_token_accuracy": 0.09274004399776459,
+      "num_tokens": 396662.0,
+      "step": 175
+    },
+    {
+      "entropy": 8.170382118225097,
+      "epoch": 0.011584502509975544,
+      "grad_norm": 1.453125,
+      "learning_rate": 8.95e-05,
+      "loss": 7.3063,
+      "mean_token_accuracy": 0.09627912491559983,
+      "num_tokens": 407836.0,
+      "step": 180
+    },
+    {
+      "entropy": 8.038551759719848,
+      "epoch": 0.011906294246363754,
+      "grad_norm": 1.59375,
+      "learning_rate": 9.2e-05,
+      "loss": 7.1831,
+      "mean_token_accuracy": 0.10445229858160018,
+      "num_tokens": 418626.0,
+      "step": 185
+    },
+    {
+      "entropy": 7.990694236755371,
+      "epoch": 0.012228085982751963,
+      "grad_norm": 1.40625,
+      "learning_rate": 9.45e-05,
+      "loss": 7.2592,
+      "mean_token_accuracy": 0.09324222803115845,
+      "num_tokens": 430714.0,
+      "step": 190
+    },
+    {
+      "entropy": 8.09174370765686,
+      "epoch": 0.012549877719140172,
+      "grad_norm": 1.375,
+      "learning_rate": 9.7e-05,
+      "loss": 7.2672,
+      "mean_token_accuracy": 0.08939176499843597,
+      "num_tokens": 442144.0,
+      "step": 195
+    },
+    {
+      "entropy": 7.912285184860229,
+      "epoch": 0.012871669455528383,
+      "grad_norm": 1.578125,
+      "learning_rate": 9.95e-05,
+      "loss": 7.3035,
+      "mean_token_accuracy": 0.09566279277205467,
+      "num_tokens": 454338.0,
+      "step": 200
+    },
+    {
+      "entropy": 7.9603489398956295,
+      "epoch": 0.013193461191916592,
+      "grad_norm": 1.53125,
+      "learning_rate": 0.000102,
+      "loss": 7.1568,
+      "mean_token_accuracy": 0.10046104565262795,
+      "num_tokens": 465493.0,
+      "step": 205
+    },
+    {
+      "entropy": 7.831574201583862,
+      "epoch": 0.013515252928304801,
+      "grad_norm": 1.6796875,
+      "learning_rate": 0.00010449999999999999,
+      "loss": 7.2202,
+      "mean_token_accuracy": 0.09616154506802559,
+      "num_tokens": 476753.0,
+      "step": 210
+    },
+    {
+      "entropy": 7.802942085266113,
+      "epoch": 0.01383704466469301,
+      "grad_norm": 1.5703125,
+      "learning_rate": 0.000107,
+      "loss": 7.1711,
+      "mean_token_accuracy": 0.10192405954003333,
+      "num_tokens": 488229.0,
+      "step": 215
+    },
+    {
+      "entropy": 7.800758314132691,
+      "epoch": 0.01415883640108122,
+      "grad_norm": 2.265625,
+      "learning_rate": 0.0001095,
+      "loss": 7.1771,
+      "mean_token_accuracy": 0.09801913350820542,
+      "num_tokens": 500060.0,
+      "step": 220
+    },
+    {
+      "entropy": 7.8091026782989506,
+      "epoch": 0.01448062813746943,
+      "grad_norm": 1.578125,
+      "learning_rate": 0.000112,
+      "loss": 7.1403,
+      "mean_token_accuracy": 0.09416564255952835,
+      "num_tokens": 512368.0,
+      "step": 225
+    },
+    {
+      "entropy": 7.6497923851013185,
+      "epoch": 0.01480241987385764,
+      "grad_norm": 1.5703125,
+      "learning_rate": 0.0001145,
+      "loss": 7.0236,
+      "mean_token_accuracy": 0.10454239919781685,
+      "num_tokens": 522546.0,
+      "step": 230
+    },
+    {
+      "entropy": 7.6424744606018065,
+      "epoch": 0.015124211610245849,
+      "grad_norm": 1.421875,
+      "learning_rate": 0.00011700000000000001,
+      "loss": 7.106,
+      "mean_token_accuracy": 0.10524164140224457,
+      "num_tokens": 535441.0,
+      "step": 235
+    },
+    {
+      "entropy": 7.6858056545257565,
+      "epoch": 0.015446003346634058,
+      "grad_norm": 1.515625,
+      "learning_rate": 0.00011949999999999999,
+      "loss": 7.0976,
+      "mean_token_accuracy": 0.10354602560400963,
+      "num_tokens": 547684.0,
+      "step": 240
+    },
+    {
+      "entropy": 7.588476228713989,
+      "epoch": 0.015767795083022267,
+      "grad_norm": 1.390625,
+      "learning_rate": 0.000122,
+      "loss": 7.1511,
+      "mean_token_accuracy": 0.1012592412531376,
+      "num_tokens": 558822.0,
+      "step": 245
+    },
+    {
+      "entropy": 7.707156753540039,
+      "epoch": 0.016089586819410476,
+      "grad_norm": 1.2890625,
+      "learning_rate": 0.0001245,
+      "loss": 7.0529,
+      "mean_token_accuracy": 0.10268613547086716,
+      "num_tokens": 570343.0,
+      "step": 250
+    },
+    {
+      "entropy": 7.550539875030518,
+      "epoch": 0.016411378555798686,
+      "grad_norm": 1.4921875,
+      "learning_rate": 0.000127,
+      "loss": 7.0108,
+      "mean_token_accuracy": 0.10793713703751565,
+      "num_tokens": 581035.0,
+      "step": 255
+    },
+    {
+      "entropy": 7.545463514328003,
+      "epoch": 0.0167331702921869,
+      "grad_norm": 1.5078125,
+      "learning_rate": 0.0001295,
+      "loss": 6.9715,
+      "mean_token_accuracy": 0.103653634339571,
+      "num_tokens": 594185.0,
+      "step": 260
+    },
+    {
+      "entropy": 7.447913885116577,
+      "epoch": 0.017054962028575107,
+      "grad_norm": 1.3671875,
+      "learning_rate": 0.000132,
+      "loss": 6.9392,
+      "mean_token_accuracy": 0.10639498457312584,
+      "num_tokens": 605353.0,
+      "step": 265
+    },
+    {
+      "entropy": 7.561796474456787,
+      "epoch": 0.017376753764963317,
+      "grad_norm": 1.3203125,
+      "learning_rate": 0.00013450000000000002,
+      "loss": 7.0345,
+      "mean_token_accuracy": 0.10664849355816841,
+      "num_tokens": 617477.0,
+      "step": 270
+    },
+    {
+      "entropy": 7.435778045654297,
+      "epoch": 0.017698545501351526,
+      "grad_norm": 1.71875,
+      "learning_rate": 0.00013700000000000002,
+      "loss": 6.9534,
+      "mean_token_accuracy": 0.10435824021697045,
+      "num_tokens": 627578.0,
+      "step": 275
+    },
+    {
+      "entropy": 7.4433777809143065,
+      "epoch": 0.018020337237739735,
+      "grad_norm": 1.5859375,
+      "learning_rate": 0.0001395,
+      "loss": 6.955,
+      "mean_token_accuracy": 0.10273718237876892,
+      "num_tokens": 639559.0,
+      "step": 280
+    },
+    {
+      "entropy": 7.458190488815307,
+      "epoch": 0.018342128974127944,
+      "grad_norm": 1.7578125,
+      "learning_rate": 0.00014199999999999998,
+      "loss": 6.962,
+      "mean_token_accuracy": 0.1059377796947956,
+      "num_tokens": 650407.0,
+      "step": 285
+    },
+    {
+      "entropy": 7.382494831085205,
+      "epoch": 0.018663920710516153,
+      "grad_norm": 1.2421875,
+      "learning_rate": 0.0001445,
+      "loss": 6.8653,
+      "mean_token_accuracy": 0.11828745529055595,
+      "num_tokens": 662931.0,
+      "step": 290
+    },
+    {
+      "entropy": 7.4552978515625,
+      "epoch": 0.018985712446904363,
+      "grad_norm": 1.7421875,
+      "learning_rate": 0.000147,
+      "loss": 6.9431,
+      "mean_token_accuracy": 0.11408839300274849,
+      "num_tokens": 674686.0,
+      "step": 295
+    },
+    {
+      "entropy": 7.3142343044281,
+      "epoch": 0.01930750418329257,
+      "grad_norm": 1.3515625,
+      "learning_rate": 0.0001495,
+      "loss": 6.8417,
+      "mean_token_accuracy": 0.11158959865570069,
+      "num_tokens": 685463.0,
+      "step": 300
+    },
+    {
+      "entropy": 7.456843852996826,
+      "epoch": 0.01962929591968078,
+      "grad_norm": 1.4453125,
+      "learning_rate": 0.000152,
+      "loss": 7.0395,
+      "mean_token_accuracy": 0.11052512302994728,
+      "num_tokens": 697397.0,
+      "step": 305
+    },
+    {
+      "entropy": 7.265558385848999,
+      "epoch": 0.019951087656068994,
+      "grad_norm": 1.515625,
+      "learning_rate": 0.00015450000000000001,
+      "loss": 6.8946,
+      "mean_token_accuracy": 0.11685808449983597,
+      "num_tokens": 708268.0,
+      "step": 310
+    },
+    {
+      "entropy": 7.297895240783691,
+      "epoch": 0.020272879392457203,
+      "grad_norm": 1.296875,
+      "learning_rate": 0.000157,
+      "loss": 6.8953,
+      "mean_token_accuracy": 0.11156268939375877,
+      "num_tokens": 720056.0,
+      "step": 315
+    },
+    {
+      "entropy": 7.366024732589722,
+      "epoch": 0.020594671128845412,
+      "grad_norm": 1.5390625,
+      "learning_rate": 0.0001595,
+      "loss": 6.8679,
+      "mean_token_accuracy": 0.1150714986026287,
+      "num_tokens": 730835.0,
+      "step": 320
+    },
+    {
+      "entropy": 7.369055366516113,
+      "epoch": 0.02091646286523362,
+      "grad_norm": 1.328125,
+      "learning_rate": 0.000162,
+      "loss": 7.0326,
+      "mean_token_accuracy": 0.108493110537529,
+      "num_tokens": 742428.0,
+      "step": 325
+    },
+    {
+      "entropy": 7.274074840545654,
+      "epoch": 0.02123825460162183,
+      "grad_norm": 1.296875,
+      "learning_rate": 0.00016450000000000001,
+      "loss": 6.8866,
+      "mean_token_accuracy": 0.1098675474524498,
+      "num_tokens": 752147.0,
+      "step": 330
+    },
+    {
+      "entropy": 7.4121346950531,
+      "epoch": 0.02156004633801004,
+      "grad_norm": 1.328125,
+      "learning_rate": 0.00016700000000000002,
+      "loss": 6.9166,
+      "mean_token_accuracy": 0.10750498473644257,
+      "num_tokens": 763415.0,
+      "step": 335
+    },
+    {
+      "entropy": 7.304456186294556,
+      "epoch": 0.02188183807439825,
+      "grad_norm": 1.3984375,
+      "learning_rate": 0.00016950000000000003,
+      "loss": 6.8702,
+      "mean_token_accuracy": 0.10661920979619026,
+      "num_tokens": 775175.0,
+      "step": 340
+    },
+    {
+      "entropy": 7.224200105667114,
+      "epoch": 0.022203629810786458,
+      "grad_norm": 1.359375,
+      "learning_rate": 0.00017199999999999998,
+      "loss": 6.8099,
+      "mean_token_accuracy": 0.11928229033946991,
+      "num_tokens": 785211.0,
+      "step": 345
+    },
+    {
+      "entropy": 7.240478086471557,
+      "epoch": 0.022525421547174667,
+      "grad_norm": 1.359375,
+      "learning_rate": 0.00017449999999999999,
+      "loss": 6.869,
+      "mean_token_accuracy": 0.11734312251210213,
+      "num_tokens": 795946.0,
+      "step": 350
+    },
+    {
+      "entropy": 7.1357990264892575,
+      "epoch": 0.02284721328356288,
+      "grad_norm": 1.375,
+      "learning_rate": 0.000177,
+      "loss": 6.7101,
+      "mean_token_accuracy": 0.12463226988911628,
+      "num_tokens": 806529.0,
+      "step": 355
+    },
+    {
+      "entropy": 7.220741224288941,
+      "epoch": 0.02316900501995109,
+      "grad_norm": 1.140625,
+      "learning_rate": 0.0001795,
+      "loss": 6.8095,
+      "mean_token_accuracy": 0.11709680408239365,
+      "num_tokens": 817591.0,
+      "step": 360
+    },
+    {
+      "entropy": 7.153127908706665,
+      "epoch": 0.023490796756339298,
+      "grad_norm": 1.265625,
+      "learning_rate": 0.000182,
+      "loss": 6.8188,
+      "mean_token_accuracy": 0.11239580363035202,
+      "num_tokens": 828765.0,
+      "step": 365
+    },
+    {
+      "entropy": 7.202557897567749,
+      "epoch": 0.023812588492727507,
+      "grad_norm": 1.34375,
+      "learning_rate": 0.0001845,
+      "loss": 6.8802,
+      "mean_token_accuracy": 0.1095006950199604,
+      "num_tokens": 839442.0,
+      "step": 370
+    },
+    {
+      "entropy": 7.075786590576172,
+      "epoch": 0.024134380229115716,
+      "grad_norm": 1.3828125,
+      "learning_rate": 0.000187,
+      "loss": 6.6783,
+      "mean_token_accuracy": 0.11973611861467362,
+      "num_tokens": 850471.0,
+      "step": 375
+    },
+    {
+      "entropy": 7.15129132270813,
+      "epoch": 0.024456171965503926,
+      "grad_norm": 1.3828125,
+      "learning_rate": 0.0001895,
+      "loss": 6.7819,
+      "mean_token_accuracy": 0.11633364856243134,
+      "num_tokens": 861333.0,
+      "step": 380
+    },
+    {
+      "entropy": 7.195761871337891,
+      "epoch": 0.024777963701892135,
+      "grad_norm": 1.3359375,
+      "learning_rate": 0.000192,
+      "loss": 6.8712,
+      "mean_token_accuracy": 0.11771900430321694,
+      "num_tokens": 871915.0,
+      "step": 385
+    },
+    {
+      "entropy": 7.193553447723389,
+      "epoch": 0.025099755438280344,
+      "grad_norm": 1.34375,
+      "learning_rate": 0.0001945,
+      "loss": 6.8448,
+      "mean_token_accuracy": 0.12030173614621162,
+      "num_tokens": 884006.0,
+      "step": 390
+    },
+    {
+      "entropy": 7.1253314971923825,
+      "epoch": 0.025421547174668553,
+      "grad_norm": 1.609375,
+      "learning_rate": 0.00019700000000000002,
+      "loss": 6.7347,
+      "mean_token_accuracy": 0.1115813210606575,
+      "num_tokens": 896393.0,
+      "step": 395
+    },
+    {
+      "entropy": 7.097787714004516,
+      "epoch": 0.025743338911056766,
+      "grad_norm": 1.3125,
+      "learning_rate": 0.00019950000000000002,
+      "loss": 6.833,
+      "mean_token_accuracy": 0.11548972800374031,
+      "num_tokens": 907202.0,
+      "step": 400
+    },
+    {
+      "entropy": 6.983932209014893,
+      "epoch": 0.026065130647444975,
+      "grad_norm": 1.4375,
+      "learning_rate": 0.000202,
+      "loss": 6.513,
+      "mean_token_accuracy": 0.12307731807231903,
+      "num_tokens": 917063.0,
+      "step": 405
+    },
+    {
+      "entropy": 7.058181858062744,
+      "epoch": 0.026386922383833184,
+      "grad_norm": 1.390625,
+      "learning_rate": 0.00020449999999999998,
+      "loss": 6.7534,
+      "mean_token_accuracy": 0.11989552527666092,
+      "num_tokens": 927593.0,
+      "step": 410
+    },
+    {
+      "entropy": 7.128173398971557,
+      "epoch": 0.026708714120221393,
+      "grad_norm": 1.390625,
+      "learning_rate": 0.000207,
+      "loss": 6.7926,
+      "mean_token_accuracy": 0.11310968622565269,
+      "num_tokens": 938942.0,
+      "step": 415
+    },
+    {
+      "entropy": 7.121332836151123,
+      "epoch": 0.027030505856609603,
+      "grad_norm": 1.7421875,
+      "learning_rate": 0.0002095,
+      "loss": 6.8324,
+      "mean_token_accuracy": 0.11872464641928673,
+      "num_tokens": 949912.0,
+      "step": 420
+    },
+    {
+      "entropy": 6.970332765579224,
+      "epoch": 0.02735229759299781,
+      "grad_norm": 1.2265625,
+      "learning_rate": 0.000212,
+      "loss": 6.7019,
+      "mean_token_accuracy": 0.12167602479457855,
+      "num_tokens": 960959.0,
+      "step": 425
+    },
+    {
+      "entropy": 6.912196826934815,
+      "epoch": 0.02767408932938602,
+      "grad_norm": 1.3515625,
+      "learning_rate": 0.0002145,
+      "loss": 6.607,
+      "mean_token_accuracy": 0.12273273840546609,
+      "num_tokens": 972908.0,
+      "step": 430
+    },
+    {
+      "entropy": 7.092562818527222,
+      "epoch": 0.02799588106577423,
+      "grad_norm": 1.34375,
+      "learning_rate": 0.00021700000000000002,
+      "loss": 6.8127,
+      "mean_token_accuracy": 0.10646726936101913,
+      "num_tokens": 984115.0,
+      "step": 435
+    },
+    {
+      "entropy": 7.0991950035095215,
+      "epoch": 0.02831767280216244,
+      "grad_norm": 1.28125,
+      "learning_rate": 0.0002195,
+      "loss": 6.862,
+      "mean_token_accuracy": 0.11290395855903626,
+      "num_tokens": 995697.0,
+      "step": 440
+    },
+    {
+      "entropy": 7.099041700363159,
+      "epoch": 0.02863946453855065,
+      "grad_norm": 1.390625,
+      "learning_rate": 0.000222,
+      "loss": 6.7961,
+      "mean_token_accuracy": 0.11020021885633469,
+      "num_tokens": 1007918.0,
+      "step": 445
+    },
+    {
+      "entropy": 7.085301446914673,
+      "epoch": 0.02896125627493886,
+      "grad_norm": 1.34375,
+      "learning_rate": 0.0002245,
+      "loss": 6.6977,
+      "mean_token_accuracy": 0.12074138224124908,
+      "num_tokens": 1017699.0,
+      "step": 450
+    },
+    {
+      "entropy": 7.032927083969116,
+      "epoch": 0.02928304801132707,
+      "grad_norm": 1.5078125,
+      "learning_rate": 0.00022700000000000002,
+      "loss": 6.7618,
+      "mean_token_accuracy": 0.12060360610485077,
+      "num_tokens": 1029464.0,
+      "step": 455
+    },
+    {
+      "entropy": 7.0487706661224365,
+      "epoch": 0.02960483974771528,
+      "grad_norm": 1.296875,
+      "learning_rate": 0.00022950000000000002,
+      "loss": 6.7003,
+      "mean_token_accuracy": 0.12441192641854286,
+      "num_tokens": 1040241.0,
+      "step": 460
+    },
+    {
+      "entropy": 7.015468263626099,
+      "epoch": 0.02992663148410349,
+      "grad_norm": 1.34375,
+      "learning_rate": 0.00023200000000000003,
+      "loss": 6.7482,
+      "mean_token_accuracy": 0.11376301795244217,
+      "num_tokens": 1052712.0,
+      "step": 465
+    },
+    {
+      "entropy": 6.893531227111817,
+      "epoch": 0.030248423220491698,
+      "grad_norm": 1.3203125,
+      "learning_rate": 0.00023449999999999998,
+      "loss": 6.6977,
+      "mean_token_accuracy": 0.11767899766564369,
+      "num_tokens": 1063569.0,
+      "step": 470
+    },
+    {
+      "entropy": 6.897192907333374,
+      "epoch": 0.030570214956879907,
+      "grad_norm": 1.46875,
+      "learning_rate": 0.000237,
+      "loss": 6.6361,
+      "mean_token_accuracy": 0.12202028259634971,
+      "num_tokens": 1074314.0,
+      "step": 475
+    },
+    {
+      "entropy": 7.062028932571411,
+      "epoch": 0.030892006693268116,
+      "grad_norm": 1.3671875,
+      "learning_rate": 0.0002395,
+      "loss": 6.8464,
+      "mean_token_accuracy": 0.1108027994632721,
+      "num_tokens": 1086084.0,
+      "step": 480
+    },
+    {
+      "entropy": 7.020650625228882,
+      "epoch": 0.031213798429656325,
+      "grad_norm": 1.3671875,
+      "learning_rate": 0.000242,
+      "loss": 6.7314,
+      "mean_token_accuracy": 0.11836976334452629,
+      "num_tokens": 1097116.0,
+      "step": 485
+    },
+    {
+      "entropy": 6.910376405715942,
+      "epoch": 0.031535590166044535,
+      "grad_norm": 1.2421875,
+      "learning_rate": 0.0002445,
+      "loss": 6.6115,
+      "mean_token_accuracy": 0.12779454588890077,
+      "num_tokens": 1108561.0,
+      "step": 490
+    },
+    {
+      "entropy": 6.949523115158081,
+      "epoch": 0.031857381902432744,
+      "grad_norm": 1.25,
+      "learning_rate": 0.000247,
+      "loss": 6.6493,
+      "mean_token_accuracy": 0.12263330891728401,
+      "num_tokens": 1120847.0,
+      "step": 495
+    },
+    {
+      "entropy": 6.9015583992004395,
+      "epoch": 0.03217917363882095,
+      "grad_norm": 1.375,
+      "learning_rate": 0.0002495,
+      "loss": 6.6208,
+      "mean_token_accuracy": 0.1280594363808632,
+      "num_tokens": 1132470.0,
+      "step": 500
+    },
+    {
+      "entropy": 6.869023752212525,
+      "epoch": 0.03250096537520916,
+      "grad_norm": 1.3671875,
+      "learning_rate": 0.000252,
+      "loss": 6.6691,
+      "mean_token_accuracy": 0.1259506419301033,
+      "num_tokens": 1144966.0,
+      "step": 505
+    },
+    {
+      "entropy": 6.998987627029419,
+      "epoch": 0.03282275711159737,
+      "grad_norm": 1.203125,
+      "learning_rate": 0.0002545,
+      "loss": 6.6196,
+      "mean_token_accuracy": 0.12219893932342529,
+      "num_tokens": 1156413.0,
+      "step": 510
+    },
+    {
+      "entropy": 6.722204494476318,
+      "epoch": 0.03314454884798558,
+      "grad_norm": 1.171875,
+      "learning_rate": 0.000257,
+      "loss": 6.5395,
+      "mean_token_accuracy": 0.1305614896118641,
+      "num_tokens": 1167458.0,
+      "step": 515
+    },
+    {
+      "entropy": 6.878732395172119,
+      "epoch": 0.0334663405843738,
+      "grad_norm": 1.234375,
+      "learning_rate": 0.0002595,
+      "loss": 6.6024,
+      "mean_token_accuracy": 0.11557823643088341,
+      "num_tokens": 1179015.0,
+      "step": 520
+    },
+    {
+      "entropy": 6.836273288726806,
+      "epoch": 0.033788132320762006,
+      "grad_norm": 1.296875,
+      "learning_rate": 0.000262,
+      "loss": 6.5532,
+      "mean_token_accuracy": 0.12147759348154068,
+      "num_tokens": 1189616.0,
+      "step": 525
+    },
+    {
+      "entropy": 6.74482798576355,
+      "epoch": 0.034109924057150215,
+      "grad_norm": 1.5390625,
+      "learning_rate": 0.00026450000000000003,
+      "loss": 6.4766,
+      "mean_token_accuracy": 0.12241575941443443,
+      "num_tokens": 1200283.0,
+      "step": 530
+    },
+    {
+      "entropy": 6.877546215057373,
+      "epoch": 0.034431715793538424,
+      "grad_norm": 1.3046875,
+      "learning_rate": 0.00026700000000000004,
+      "loss": 6.6075,
+      "mean_token_accuracy": 0.12261987179517746,
+      "num_tokens": 1211379.0,
+      "step": 535
+    },
+    {
+      "entropy": 6.855060482025147,
+      "epoch": 0.03475350752992663,
+      "grad_norm": 1.2421875,
+      "learning_rate": 0.00026950000000000005,
+      "loss": 6.5896,
+      "mean_token_accuracy": 0.1253580369055271,
+      "num_tokens": 1222790.0,
+      "step": 540
+    },
+    {
+      "entropy": 6.859920072555542,
+      "epoch": 0.03507529926631484,
+      "grad_norm": 1.28125,
+      "learning_rate": 0.00027200000000000005,
+      "loss": 6.5984,
+      "mean_token_accuracy": 0.12788782343268396,
+      "num_tokens": 1233801.0,
+      "step": 545
+    },
+    {
+      "entropy": 6.815193700790405,
+      "epoch": 0.03539709100270305,
+      "grad_norm": 1.1640625,
+      "learning_rate": 0.0002745,
+      "loss": 6.5805,
+      "mean_token_accuracy": 0.12837038785219193,
+      "num_tokens": 1245805.0,
+      "step": 550
+    },
+    {
+      "entropy": 6.745665168762207,
+      "epoch": 0.03571888273909126,
+      "grad_norm": 1.34375,
+      "learning_rate": 0.000277,
+      "loss": 6.5159,
+      "mean_token_accuracy": 0.1349610432982445,
+      "num_tokens": 1257107.0,
+      "step": 555
+    },
+    {
+      "entropy": 6.797294044494629,
+      "epoch": 0.03604067447547947,
+      "grad_norm": 1.1796875,
+      "learning_rate": 0.0002795,
+      "loss": 6.6277,
+      "mean_token_accuracy": 0.12336362823843956,
+      "num_tokens": 1268179.0,
+      "step": 560
+    },
+    {
+      "entropy": 6.868199729919434,
+      "epoch": 0.03636246621186768,
+      "grad_norm": 1.1484375,
+      "learning_rate": 0.00028199999999999997,
+      "loss": 6.555,
+      "mean_token_accuracy": 0.13341980427503586,
+      "num_tokens": 1278442.0,
+      "step": 565
+    },
+    {
+      "entropy": 6.89066162109375,
+      "epoch": 0.03668425794825589,
+      "grad_norm": 1.3828125,
+      "learning_rate": 0.0002845,
+      "loss": 6.6539,
+      "mean_token_accuracy": 0.12870992943644524,
+      "num_tokens": 1288623.0,
+      "step": 570
+    },
+    {
+      "entropy": 6.751723289489746,
+      "epoch": 0.0370060496846441,
+      "grad_norm": 1.3515625,
+      "learning_rate": 0.000287,
+      "loss": 6.5708,
+      "mean_token_accuracy": 0.12620823234319686,
+      "num_tokens": 1300981.0,
+      "step": 575
+    },
+    {
+      "entropy": 6.691731691360474,
+      "epoch": 0.03732784142103231,
+      "grad_norm": 1.3046875,
+      "learning_rate": 0.0002895,
+      "loss": 6.4173,
+      "mean_token_accuracy": 0.1356960915029049,
+      "num_tokens": 1312496.0,
+      "step": 580
+    },
+    {
+      "entropy": 6.689237260818482,
+      "epoch": 0.037649633157420516,
+      "grad_norm": 1.078125,
+      "learning_rate": 0.000292,
+      "loss": 6.4696,
+      "mean_token_accuracy": 0.13737872168421744,
+      "num_tokens": 1323716.0,
+      "step": 585
+    },
+    {
+      "entropy": 6.680966424942016,
+      "epoch": 0.037971424893808725,
+      "grad_norm": 1.40625,
+      "learning_rate": 0.0002945,
+      "loss": 6.4844,
+      "mean_token_accuracy": 0.1304626889526844,
+      "num_tokens": 1335571.0,
+      "step": 590
+    },
+    {
+      "entropy": 6.6498072147369385,
+      "epoch": 0.038293216630196934,
+      "grad_norm": 1.1953125,
+      "learning_rate": 0.000297,
+      "loss": 6.4942,
+      "mean_token_accuracy": 0.13134991824626924,
+      "num_tokens": 1347105.0,
+      "step": 595
+    },
+    {
+      "entropy": 6.765724849700928,
+      "epoch": 0.03861500836658514,
+      "grad_norm": 1.1015625,
+      "learning_rate": 0.0002995,
+      "loss": 6.5692,
+      "mean_token_accuracy": 0.12424814403057098,
+      "num_tokens": 1357640.0,
+      "step": 600
+    },
+    {
+      "entropy": 6.669166326522827,
+      "epoch": 0.03893680010297335,
+      "grad_norm": 1.28125,
+      "learning_rate": 0.000302,
+      "loss": 6.4196,
+      "mean_token_accuracy": 0.13372000828385353,
+      "num_tokens": 1368608.0,
+      "step": 605
+    },
+    {
+      "entropy": 6.75927505493164,
+      "epoch": 0.03925859183936156,
+      "grad_norm": 1.296875,
+      "learning_rate": 0.0003045,
+      "loss": 6.5127,
+      "mean_token_accuracy": 0.1207577109336853,
+      "num_tokens": 1379853.0,
+      "step": 610
+    },
+    {
+      "entropy": 6.664323329925537,
+      "epoch": 0.03958038357574978,
+      "grad_norm": 1.25,
+      "learning_rate": 0.000307,
+      "loss": 6.4645,
+      "mean_token_accuracy": 0.12720839604735373,
+      "num_tokens": 1390630.0,
+      "step": 615
+    },
+    {
+      "entropy": 6.689991140365601,
+      "epoch": 0.03990217531213799,
+      "grad_norm": 1.3671875,
+      "learning_rate": 0.0003095,
+      "loss": 6.4366,
+      "mean_token_accuracy": 0.13198668509721756,
+      "num_tokens": 1401997.0,
+      "step": 620
+    },
+    {
+      "entropy": 6.634037685394287,
+      "epoch": 0.040223967048526196,
+      "grad_norm": 1.1953125,
+      "learning_rate": 0.000312,
+      "loss": 6.536,
+      "mean_token_accuracy": 0.1256448470056057,
+      "num_tokens": 1413611.0,
+      "step": 625
+    },
+    {
+      "entropy": 6.804963159561157,
+      "epoch": 0.040545758784914405,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.0003145,
+      "loss": 6.6033,
+      "mean_token_accuracy": 0.11850515380501747,
+      "num_tokens": 1424639.0,
+      "step": 630
+    },
+    {
+      "entropy": 6.6990772724151615,
+      "epoch": 0.040867550521302615,
+      "grad_norm": 1.2421875,
+      "learning_rate": 0.000317,
+      "loss": 6.5229,
+      "mean_token_accuracy": 0.13084252625703813,
+      "num_tokens": 1434802.0,
+      "step": 635
+    },
+    {
+      "entropy": 6.687566947937012,
+      "epoch": 0.041189342257690824,
+      "grad_norm": 1.1484375,
+      "learning_rate": 0.0003195,
+      "loss": 6.466,
+      "mean_token_accuracy": 0.12602303102612494,
+      "num_tokens": 1446942.0,
+      "step": 640
+    },
+    {
+      "entropy": 6.730192232131958,
+      "epoch": 0.04151113399407903,
+      "grad_norm": 1.2421875,
+      "learning_rate": 0.000322,
+      "loss": 6.4753,
+      "mean_token_accuracy": 0.12809659466147422,
+      "num_tokens": 1458527.0,
+      "step": 645
+    },
+    {
+      "entropy": 6.495309972763062,
+      "epoch": 0.04183292573046724,
+      "grad_norm": 1.25,
+      "learning_rate": 0.00032450000000000003,
+      "loss": 6.3417,
+      "mean_token_accuracy": 0.13759842664003372,
+      "num_tokens": 1469065.0,
+      "step": 650
+    },
+    {
+      "entropy": 6.648279190063477,
+      "epoch": 0.04215471746685545,
+      "grad_norm": 1.2265625,
+      "learning_rate": 0.00032700000000000003,
+      "loss": 6.4067,
+      "mean_token_accuracy": 0.13652142956852914,
+      "num_tokens": 1479425.0,
+      "step": 655
+    },
+    {
+      "entropy": 6.641835165023804,
+      "epoch": 0.04247650920324366,
+      "grad_norm": 1.21875,
+      "learning_rate": 0.00032950000000000004,
+      "loss": 6.4447,
+      "mean_token_accuracy": 0.12913303449749947,
+      "num_tokens": 1490661.0,
+      "step": 660
+    },
+    {
+      "entropy": 6.750196409225464,
+      "epoch": 0.04279830093963187,
+      "grad_norm": 1.4375,
+      "learning_rate": 0.00033200000000000005,
+      "loss": 6.5532,
+      "mean_token_accuracy": 0.12641846686601638,
+      "num_tokens": 1502638.0,
+      "step": 665
+    },
+    {
+      "entropy": 6.594930934906006,
+      "epoch": 0.04312009267602008,
+      "grad_norm": 1.3125,
+      "learning_rate": 0.00033450000000000005,
+      "loss": 6.4584,
+      "mean_token_accuracy": 0.1351857751607895,
+      "num_tokens": 1512899.0,
+      "step": 670
+    },
+    {
+      "entropy": 6.697818374633789,
+      "epoch": 0.04344188441240829,
+      "grad_norm": 1.2421875,
+      "learning_rate": 0.000337,
+      "loss": 6.4577,
+      "mean_token_accuracy": 0.12779420837759972,
+      "num_tokens": 1523914.0,
+      "step": 675
+    },
+    {
+      "entropy": 6.640419149398804,
+      "epoch": 0.0437636761487965,
+      "grad_norm": 1.3203125,
+      "learning_rate": 0.0003395,
+      "loss": 6.4947,
+      "mean_token_accuracy": 0.13016460612416267,
+      "num_tokens": 1535498.0,
+      "step": 680
+    },
+    {
+      "entropy": 6.607001733779907,
+      "epoch": 0.044085467885184706,
+      "grad_norm": 1.296875,
+      "learning_rate": 0.000342,
+      "loss": 6.4335,
+      "mean_token_accuracy": 0.13075666204094888,
+      "num_tokens": 1545516.0,
+      "step": 685
+    },
+    {
+      "entropy": 6.660410928726196,
+      "epoch": 0.044407259621572916,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.00034449999999999997,
+      "loss": 6.5728,
+      "mean_token_accuracy": 0.12807587459683417,
+      "num_tokens": 1556241.0,
+      "step": 690
+    },
+    {
+      "entropy": 6.73777813911438,
+      "epoch": 0.044729051357961125,
+      "grad_norm": 1.2890625,
+      "learning_rate": 0.000347,
+      "loss": 6.4331,
+      "mean_token_accuracy": 0.13530867099761962,
+      "num_tokens": 1567229.0,
+      "step": 695
+    },
+    {
+      "entropy": 6.50676908493042,
+      "epoch": 0.045050843094349334,
+      "grad_norm": 1.1875,
+      "learning_rate": 0.0003495,
+      "loss": 6.4477,
+      "mean_token_accuracy": 0.12948062345385553,
+      "num_tokens": 1578703.0,
+      "step": 700
+    },
+    {
+      "entropy": 6.497309255599975,
+      "epoch": 0.04537263483073754,
+      "grad_norm": 1.28125,
+      "learning_rate": 0.000352,
+      "loss": 6.3204,
+      "mean_token_accuracy": 0.1343964785337448,
+      "num_tokens": 1589839.0,
+      "step": 705
+    },
+    {
+      "entropy": 6.575838232040406,
+      "epoch": 0.04569442656712576,
+      "grad_norm": 1.2734375,
+      "learning_rate": 0.0003545,
+      "loss": 6.3711,
+      "mean_token_accuracy": 0.13568960577249528,
+      "num_tokens": 1600277.0,
+      "step": 710
+    },
+    {
+      "entropy": 6.6485175609588625,
+      "epoch": 0.04601621830351397,
+      "grad_norm": 1.0859375,
+      "learning_rate": 0.000357,
+      "loss": 6.5288,
+      "mean_token_accuracy": 0.12966799288988112,
+      "num_tokens": 1611894.0,
+      "step": 715
+    },
+    {
+      "entropy": 6.500547075271607,
+      "epoch": 0.04633801003990218,
+      "grad_norm": 1.078125,
+      "learning_rate": 0.0003595,
+      "loss": 6.3377,
+      "mean_token_accuracy": 0.13781237751245498,
+      "num_tokens": 1625165.0,
+      "step": 720
+    },
+    {
+      "entropy": 6.593970775604248,
+      "epoch": 0.04665980177629039,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.000362,
+      "loss": 6.4257,
+      "mean_token_accuracy": 0.13297282308340072,
+      "num_tokens": 1636606.0,
+      "step": 725
+    },
+    {
+      "entropy": 6.567788457870483,
+      "epoch": 0.046981593512678596,
+      "grad_norm": 1.125,
+      "learning_rate": 0.0003645,
+      "loss": 6.3967,
+      "mean_token_accuracy": 0.13927650898694993,
+      "num_tokens": 1648528.0,
+      "step": 730
+    },
+    {
+      "entropy": 6.566985034942627,
+      "epoch": 0.047303385249066805,
+      "grad_norm": 1.0703125,
+      "learning_rate": 0.000367,
+      "loss": 6.4528,
+      "mean_token_accuracy": 0.12781663686037065,
+      "num_tokens": 1661540.0,
+      "step": 735
+    },
+    {
+      "entropy": 6.468529415130615,
+      "epoch": 0.047625176985455014,
+      "grad_norm": 1.1484375,
+      "learning_rate": 0.0003695,
+      "loss": 6.4452,
+      "mean_token_accuracy": 0.12587922364473342,
+      "num_tokens": 1673505.0,
+      "step": 740
+    },
+    {
+      "entropy": 6.576410531997681,
+      "epoch": 0.047946968721843224,
+      "grad_norm": 1.1796875,
+      "learning_rate": 0.000372,
+      "loss": 6.3876,
+      "mean_token_accuracy": 0.1358407884836197,
+      "num_tokens": 1685297.0,
+      "step": 745
+    },
+    {
+      "entropy": 6.451522922515869,
+      "epoch": 0.04826876045823143,
+      "grad_norm": 1.3671875,
+      "learning_rate": 0.0003745,
+      "loss": 6.3365,
+      "mean_token_accuracy": 0.1313149116933346,
+      "num_tokens": 1694649.0,
+      "step": 750
+    },
+    {
+      "entropy": 6.586151742935181,
+      "epoch": 0.04859055219461964,
+      "grad_norm": 1.2109375,
+      "learning_rate": 0.000377,
+      "loss": 6.4005,
+      "mean_token_accuracy": 0.1305807001888752,
+      "num_tokens": 1705720.0,
+      "step": 755
+    },
+    {
+      "entropy": 6.542773485183716,
+      "epoch": 0.04891234393100785,
+      "grad_norm": 1.1875,
+      "learning_rate": 0.0003795,
+      "loss": 6.485,
+      "mean_token_accuracy": 0.12867986187338828,
+      "num_tokens": 1717634.0,
+      "step": 760
+    },
+    {
+      "entropy": 6.590516901016235,
+      "epoch": 0.04923413566739606,
+      "grad_norm": 1.359375,
+      "learning_rate": 0.000382,
+      "loss": 6.3945,
+      "mean_token_accuracy": 0.12419472783803939,
+      "num_tokens": 1729954.0,
+      "step": 765
+    },
+    {
+      "entropy": 6.474939203262329,
+      "epoch": 0.04955592740378427,
+      "grad_norm": 1.234375,
+      "learning_rate": 0.0003845,
+      "loss": 6.3725,
+      "mean_token_accuracy": 0.1370876781642437,
+      "num_tokens": 1742173.0,
+      "step": 770
+    },
+    {
+      "entropy": 6.553148174285889,
+      "epoch": 0.04987771914017248,
+      "grad_norm": 1.1640625,
+      "learning_rate": 0.00038700000000000003,
+      "loss": 6.3779,
+      "mean_token_accuracy": 0.13408709540963173,
+      "num_tokens": 1753682.0,
+      "step": 775
+    },
+    {
+      "entropy": 6.405833148956299,
+      "epoch": 0.05019951087656069,
+      "grad_norm": 1.171875,
+      "learning_rate": 0.00038950000000000003,
+      "loss": 6.3635,
+      "mean_token_accuracy": 0.13386630415916442,
+      "num_tokens": 1764358.0,
+      "step": 780
+    },
+    {
+      "entropy": 6.4643104553222654,
+      "epoch": 0.0505213026129489,
+      "grad_norm": 1.3046875,
+      "learning_rate": 0.00039200000000000004,
+      "loss": 6.3553,
+      "mean_token_accuracy": 0.13426898568868637,
+      "num_tokens": 1775997.0,
+      "step": 785
+    },
+    {
+      "entropy": 6.612279558181763,
+      "epoch": 0.050843094349337106,
+      "grad_norm": 1.2734375,
+      "learning_rate": 0.00039450000000000005,
+      "loss": 6.3925,
+      "mean_token_accuracy": 0.12899470925331116,
+      "num_tokens": 1787684.0,
+      "step": 790
+    },
+    {
+      "entropy": 6.357057094573975,
+      "epoch": 0.051164886085725315,
+      "grad_norm": 1.1640625,
+      "learning_rate": 0.00039700000000000005,
+      "loss": 6.3918,
+      "mean_token_accuracy": 0.12932194545865058,
+      "num_tokens": 1799291.0,
+      "step": 795
+    },
+    {
+      "entropy": 6.46887321472168,
+      "epoch": 0.05148667782211353,
+      "grad_norm": 1.1796875,
+      "learning_rate": 0.0003995,
+      "loss": 6.3496,
+      "mean_token_accuracy": 0.1412258230149746,
+      "num_tokens": 1811976.0,
+      "step": 800
+    },
+    {
+      "entropy": 6.483779716491699,
+      "epoch": 0.05180846955850174,
+      "grad_norm": 1.234375,
+      "learning_rate": 0.000402,
+      "loss": 6.3974,
+      "mean_token_accuracy": 0.12525486797094346,
+      "num_tokens": 1824751.0,
+      "step": 805
+    },
+    {
+      "entropy": 6.414471673965454,
+      "epoch": 0.05213026129488995,
+      "grad_norm": 1.1484375,
+      "learning_rate": 0.0004045,
+      "loss": 6.3602,
+      "mean_token_accuracy": 0.12907202765345574,
+      "num_tokens": 1836676.0,
+      "step": 810
+    },
+    {
+      "entropy": 6.464478349685669,
+      "epoch": 0.05245205303127816,
+      "grad_norm": 1.203125,
+      "learning_rate": 0.00040699999999999997,
+      "loss": 6.2536,
+      "mean_token_accuracy": 0.13829391375184058,
+      "num_tokens": 1847803.0,
+      "step": 815
+    },
+    {
+      "entropy": 6.367223501205444,
+      "epoch": 0.05277384476766637,
+      "grad_norm": 1.125,
+      "learning_rate": 0.0004095,
+      "loss": 6.2625,
+      "mean_token_accuracy": 0.12957098931074143,
+      "num_tokens": 1859965.0,
+      "step": 820
+    },
+    {
+      "entropy": 6.430704641342163,
+      "epoch": 0.05309563650405458,
+      "grad_norm": 1.2578125,
+      "learning_rate": 0.000412,
+      "loss": 6.2909,
+      "mean_token_accuracy": 0.13130465224385263,
+      "num_tokens": 1870950.0,
+      "step": 825
+    },
+    {
+      "entropy": 6.414825677871704,
+      "epoch": 0.05341742824044279,
+      "grad_norm": 1.125,
+      "learning_rate": 0.0004145,
+      "loss": 6.279,
+      "mean_token_accuracy": 0.14454965069890022,
+      "num_tokens": 1882501.0,
+      "step": 830
+    },
+    {
+      "entropy": 6.502061080932617,
+      "epoch": 0.053739219976830996,
+      "grad_norm": 1.3828125,
+      "learning_rate": 0.000417,
+      "loss": 6.4967,
+      "mean_token_accuracy": 0.13128656819462775,
+      "num_tokens": 1895554.0,
+      "step": 835
+    },
+    {
+      "entropy": 6.353726434707641,
+      "epoch": 0.054061011713219205,
+      "grad_norm": 1.2421875,
+      "learning_rate": 0.0004195,
+      "loss": 6.3737,
+      "mean_token_accuracy": 0.13272848650813102,
+      "num_tokens": 1905982.0,
+      "step": 840
+    },
+    {
+      "entropy": 6.482131671905518,
+      "epoch": 0.054382803449607414,
+      "grad_norm": 1.203125,
+      "learning_rate": 0.000422,
+      "loss": 6.3661,
+      "mean_token_accuracy": 0.1328873485326767,
+      "num_tokens": 1917468.0,
+      "step": 845
+    },
+    {
+      "entropy": 6.4495148181915285,
+      "epoch": 0.05470459518599562,
+      "grad_norm": 1.1875,
+      "learning_rate": 0.0004245,
+      "loss": 6.3424,
+      "mean_token_accuracy": 0.13536591157317163,
+      "num_tokens": 1927743.0,
+      "step": 850
+    },
+    {
+      "entropy": 6.410049724578857,
+      "epoch": 0.05502638692238383,
+      "grad_norm": 1.2265625,
+      "learning_rate": 0.000427,
+      "loss": 6.2786,
+      "mean_token_accuracy": 0.13671474531292915,
+      "num_tokens": 1937721.0,
+      "step": 855
+    },
+    {
+      "entropy": 6.431154823303222,
+      "epoch": 0.05534817865877204,
+      "grad_norm": 1.125,
+      "learning_rate": 0.0004295,
+      "loss": 6.2665,
+      "mean_token_accuracy": 0.13668931648135185,
+      "num_tokens": 1949130.0,
+      "step": 860
+    },
+    {
+      "entropy": 6.5185517311096195,
+      "epoch": 0.05566997039516025,
+      "grad_norm": 1.0703125,
+      "learning_rate": 0.000432,
+      "loss": 6.4461,
+      "mean_token_accuracy": 0.12062366977334023,
+      "num_tokens": 1961132.0,
+      "step": 865
+    },
+    {
+      "entropy": 6.3538782596588135,
+      "epoch": 0.05599176213154846,
+      "grad_norm": 1.1953125,
+      "learning_rate": 0.0004345,
+      "loss": 6.2272,
+      "mean_token_accuracy": 0.14166060090065002,
+      "num_tokens": 1971904.0,
+      "step": 870
+    },
+    {
+      "entropy": 6.424667167663574,
+      "epoch": 0.05631355386793667,
+      "grad_norm": 1.0703125,
+      "learning_rate": 0.000437,
+      "loss": 6.3388,
+      "mean_token_accuracy": 0.13849016577005385,
+      "num_tokens": 1982246.0,
+      "step": 875
+    },
+    {
+      "entropy": 6.534474086761475,
+      "epoch": 0.05663534560432488,
+      "grad_norm": 1.46875,
+      "learning_rate": 0.0004395,
+      "loss": 6.3182,
+      "mean_token_accuracy": 0.1408639907836914,
+      "num_tokens": 1993687.0,
+      "step": 880
+    },
+    {
+      "entropy": 6.196937131881714,
+      "epoch": 0.05695713734071309,
+      "grad_norm": 1.125,
+      "learning_rate": 0.000442,
+      "loss": 6.2508,
+      "mean_token_accuracy": 0.14266577661037444,
+      "num_tokens": 2004335.0,
+      "step": 885
+    },
+    {
+      "entropy": 6.529156827926636,
+      "epoch": 0.0572789290771013,
+      "grad_norm": 1.234375,
+      "learning_rate": 0.0004445,
+      "loss": 6.4175,
+      "mean_token_accuracy": 0.1389109380543232,
+      "num_tokens": 2014875.0,
+      "step": 890
+    },
+    {
+      "entropy": 6.415221643447876,
+      "epoch": 0.05760072081348951,
+      "grad_norm": 1.0859375,
+      "learning_rate": 0.000447,
+      "loss": 6.3593,
+      "mean_token_accuracy": 0.14172261878848075,
+      "num_tokens": 2025354.0,
+      "step": 895
+    },
+    {
+      "entropy": 6.325968408584595,
+      "epoch": 0.05792251254987772,
+      "grad_norm": 1.109375,
+      "learning_rate": 0.00044950000000000003,
+      "loss": 6.2146,
+      "mean_token_accuracy": 0.14011863470077515,
+      "num_tokens": 2036306.0,
+      "step": 900
+    },
+    {
+      "entropy": 6.427873182296753,
+      "epoch": 0.05824430428626593,
+      "grad_norm": 1.2421875,
+      "learning_rate": 0.00045200000000000004,
+      "loss": 6.3058,
+      "mean_token_accuracy": 0.13970114216208457,
+      "num_tokens": 2046232.0,
+      "step": 905
+    },
+    {
+      "entropy": 6.283337879180908,
+      "epoch": 0.05856609602265414,
+      "grad_norm": 1.1484375,
+      "learning_rate": 0.00045450000000000004,
+      "loss": 6.2769,
+      "mean_token_accuracy": 0.13600695356726647,
+      "num_tokens": 2056876.0,
+      "step": 910
+    },
+    {
+      "entropy": 6.4581481456756595,
+      "epoch": 0.05888788775904235,
+      "grad_norm": 1.0234375,
+      "learning_rate": 0.00045700000000000005,
+      "loss": 6.5089,
+      "mean_token_accuracy": 0.12785363271832467,
+      "num_tokens": 2069419.0,
+      "step": 915
+    },
+    {
+      "entropy": 6.396502590179443,
+      "epoch": 0.05920967949543056,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.00045950000000000006,
+      "loss": 6.2433,
+      "mean_token_accuracy": 0.13423478305339814,
+      "num_tokens": 2079542.0,
+      "step": 920
+    },
+    {
+      "entropy": 6.267205047607422,
+      "epoch": 0.05953147123181877,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.000462,
+      "loss": 6.2204,
+      "mean_token_accuracy": 0.14118367657065392,
+      "num_tokens": 2090022.0,
+      "step": 925
+    },
+    {
+      "entropy": 6.398083829879761,
+      "epoch": 0.05985326296820698,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.0004645,
+      "loss": 6.3334,
+      "mean_token_accuracy": 0.13829256296157838,
+      "num_tokens": 2100572.0,
+      "step": 930
+    },
+    {
+      "entropy": 6.3885338306427,
+      "epoch": 0.060175054704595186,
+      "grad_norm": 1.2578125,
+      "learning_rate": 0.000467,
+      "loss": 6.2758,
+      "mean_token_accuracy": 0.13688874542713164,
+      "num_tokens": 2111868.0,
+      "step": 935
+    },
+    {
+      "entropy": 6.308207416534424,
+      "epoch": 0.060496846440983396,
+      "grad_norm": 1.125,
+      "learning_rate": 0.0004695,
+      "loss": 6.3022,
+      "mean_token_accuracy": 0.13644594624638556,
+      "num_tokens": 2122908.0,
+      "step": 940
+    },
+    {
+      "entropy": 6.246605396270752,
+      "epoch": 0.060818638177371605,
+      "grad_norm": 1.1953125,
+      "learning_rate": 0.000472,
+      "loss": 6.1958,
+      "mean_token_accuracy": 0.1409742645919323,
+      "num_tokens": 2134667.0,
+      "step": 945
+    },
+    {
+      "entropy": 6.289280796051026,
+      "epoch": 0.061140429913759814,
+      "grad_norm": 1.0390625,
+      "learning_rate": 0.0004745,
+      "loss": 6.2301,
+      "mean_token_accuracy": 0.1406254932284355,
+      "num_tokens": 2146183.0,
+      "step": 950
+    },
+    {
+      "entropy": 6.305873441696167,
+      "epoch": 0.06146222165014802,
+      "grad_norm": 1.2265625,
+      "learning_rate": 0.000477,
+      "loss": 6.2783,
+      "mean_token_accuracy": 0.13354712650179862,
+      "num_tokens": 2157846.0,
+      "step": 955
+    },
+    {
+      "entropy": 6.296875238418579,
+      "epoch": 0.06178401338653623,
+      "grad_norm": 1.0859375,
+      "learning_rate": 0.0004795,
+      "loss": 6.2284,
+      "mean_token_accuracy": 0.14504019021987916,
+      "num_tokens": 2168454.0,
+      "step": 960
+    },
+    {
+      "entropy": 6.360051965713501,
+      "epoch": 0.06210580512292444,
+      "grad_norm": 1.1796875,
+      "learning_rate": 0.000482,
+      "loss": 6.3116,
+      "mean_token_accuracy": 0.1352120630443096,
+      "num_tokens": 2179457.0,
+      "step": 965
+    },
+    {
+      "entropy": 6.2297309875488285,
+      "epoch": 0.06242759685931265,
+      "grad_norm": 1.1796875,
+      "learning_rate": 0.0004845,
+      "loss": 6.2111,
+      "mean_token_accuracy": 0.1407732456922531,
+      "num_tokens": 2190780.0,
+      "step": 970
+    },
+    {
+      "entropy": 6.341225433349609,
+      "epoch": 0.06274938859570087,
+      "grad_norm": 1.1875,
+      "learning_rate": 0.000487,
+      "loss": 6.3645,
+      "mean_token_accuracy": 0.12729336619377135,
+      "num_tokens": 2201000.0,
+      "step": 975
+    },
+    {
+      "entropy": 6.179282903671265,
+      "epoch": 0.06307118033208907,
+      "grad_norm": 1.640625,
+      "learning_rate": 0.0004895,
+      "loss": 6.2269,
+      "mean_token_accuracy": 0.1418264776468277,
+      "num_tokens": 2213208.0,
+      "step": 980
+    },
+    {
+      "entropy": 6.357757472991944,
+      "epoch": 0.06339297206847729,
+      "grad_norm": 1.1015625,
+      "learning_rate": 0.000492,
+      "loss": 6.2115,
+      "mean_token_accuracy": 0.13693007826805115,
+      "num_tokens": 2224550.0,
+      "step": 985
+    },
+    {
+      "entropy": 6.2303062915802006,
+      "epoch": 0.06371476380486549,
+      "grad_norm": 1.21875,
+      "learning_rate": 0.0004945,
+      "loss": 6.2138,
+      "mean_token_accuracy": 0.13869421929121017,
+      "num_tokens": 2236341.0,
+      "step": 990
+    },
+    {
+      "entropy": 6.285347557067871,
+      "epoch": 0.0640365555412537,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.000497,
+      "loss": 6.296,
+      "mean_token_accuracy": 0.1306707262992859,
+      "num_tokens": 2247462.0,
+      "step": 995
+    },
+    {
+      "entropy": 6.282100057601928,
+      "epoch": 0.0643583472776419,
+      "grad_norm": 1.578125,
+      "learning_rate": 0.0004995,
+      "loss": 6.1985,
+      "mean_token_accuracy": 0.13407785445451736,
+      "num_tokens": 2259136.0,
+      "step": 1000
+    },
+    {
+      "entropy": 6.265222263336182,
+      "epoch": 0.06468013901403012,
+      "grad_norm": 0.98828125,
+      "learning_rate": 0.000499998026082006,
+      "loss": 6.2387,
+      "mean_token_accuracy": 0.13681457713246345,
+      "num_tokens": 2270064.0,
+      "step": 1005
+    },
+    {
+      "entropy": 6.234803342819214,
+      "epoch": 0.06500193075041832,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.0004999900070995136,
+      "loss": 6.1861,
+      "mean_token_accuracy": 0.1417514629662037,
+      "num_tokens": 2281104.0,
+      "step": 1010
+    },
+    {
+      "entropy": 6.302186393737793,
+      "epoch": 0.06532372248680654,
+      "grad_norm": 1.0546875,
+      "learning_rate": 0.0004999758199023239,
+      "loss": 6.3205,
+      "mean_token_accuracy": 0.1325412757694721,
+      "num_tokens": 2292906.0,
+      "step": 1015
+    },
+    {
+      "entropy": 6.214407348632813,
+      "epoch": 0.06564551422319474,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.0004999554648793858,
+      "loss": 6.2279,
+      "mean_token_accuracy": 0.13934953659772872,
+      "num_tokens": 2304224.0,
+      "step": 1020
+    },
+    {
+      "entropy": 6.445757865905762,
+      "epoch": 0.06596730595958296,
+      "grad_norm": 1.140625,
+      "learning_rate": 0.0004999289425887425,
+      "loss": 6.2842,
+      "mean_token_accuracy": 0.12618643864989282,
+      "num_tokens": 2314904.0,
+      "step": 1025
+    },
+    {
+      "entropy": 6.222538137435913,
+      "epoch": 0.06628909769597116,
+      "grad_norm": 1.0546875,
+      "learning_rate": 0.0004998962537575161,
+      "loss": 6.3476,
+      "mean_token_accuracy": 0.1342654272913933,
+      "num_tokens": 2326352.0,
+      "step": 1030
+    },
+    {
+      "entropy": 6.3435663223266605,
+      "epoch": 0.06661088943235938,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.0004998573992818874,
+      "loss": 6.2911,
+      "mean_token_accuracy": 0.14127109572291374,
+      "num_tokens": 2337950.0,
+      "step": 1035
+    },
+    {
+      "entropy": 6.155977773666382,
+      "epoch": 0.0669326811687476,
+      "grad_norm": 1.125,
+      "learning_rate": 0.0004998123802270715,
+      "loss": 6.083,
+      "mean_token_accuracy": 0.14717964977025985,
+      "num_tokens": 2347847.0,
+      "step": 1040
+    },
+    {
+      "entropy": 6.259258651733399,
+      "epoch": 0.0672544729051358,
+      "grad_norm": 1.265625,
+      "learning_rate": 0.0004997611978272886,
+      "loss": 6.1665,
+      "mean_token_accuracy": 0.1488075628876686,
+      "num_tokens": 2357835.0,
+      "step": 1045
+    },
+    {
+      "entropy": 6.239679956436158,
+      "epoch": 0.06757626464152401,
+      "grad_norm": 1.21875,
+      "learning_rate": 0.0004997038534857298,
+      "loss": 6.2632,
+      "mean_token_accuracy": 0.13494880571961404,
+      "num_tokens": 2369081.0,
+      "step": 1050
+    },
+    {
+      "entropy": 6.193429708480835,
+      "epoch": 0.06789805637791221,
+      "grad_norm": 1.1953125,
+      "learning_rate": 0.0004996403487745194,
+      "loss": 6.2406,
+      "mean_token_accuracy": 0.13763956353068352,
+      "num_tokens": 2380243.0,
+      "step": 1055
+    },
+    {
+      "entropy": 6.336066341400146,
+      "epoch": 0.06821984811430043,
+      "grad_norm": 1.0546875,
+      "learning_rate": 0.000499570685434671,
+      "loss": 6.2294,
+      "mean_token_accuracy": 0.14424258098006248,
+      "num_tokens": 2392514.0,
+      "step": 1060
+    },
+    {
+      "entropy": 6.202878427505493,
+      "epoch": 0.06854163985068863,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.0004994948653760405,
+      "loss": 6.2643,
+      "mean_token_accuracy": 0.1274372950196266,
+      "num_tokens": 2404104.0,
+      "step": 1065
+    },
+    {
+      "entropy": 6.213591384887695,
+      "epoch": 0.06886343158707685,
+      "grad_norm": 1.0078125,
+      "learning_rate": 0.0004994128906772729,
+      "loss": 6.1634,
+      "mean_token_accuracy": 0.14054710865020753,
+      "num_tokens": 2415740.0,
+      "step": 1070
+    },
+    {
+      "entropy": 6.262695741653443,
+      "epoch": 0.06918522332346505,
+      "grad_norm": 1.078125,
+      "learning_rate": 0.000499324763585746,
+      "loss": 6.2432,
+      "mean_token_accuracy": 0.13545622304081917,
+      "num_tokens": 2426952.0,
+      "step": 1075
+    },
+    {
+      "entropy": 6.151621580123901,
+      "epoch": 0.06950701505985327,
+      "grad_norm": 1.125,
+      "learning_rate": 0.0004992304865175085,
+      "loss": 6.1999,
+      "mean_token_accuracy": 0.1342847764492035,
+      "num_tokens": 2438085.0,
+      "step": 1080
+    },
+    {
+      "entropy": 6.251348161697388,
+      "epoch": 0.06982880679624147,
+      "grad_norm": 1.1328125,
+      "learning_rate": 0.0004991300620572138,
+      "loss": 6.1787,
+      "mean_token_accuracy": 0.1454763390123844,
+      "num_tokens": 2448426.0,
+      "step": 1085
+    },
+    {
+      "entropy": 6.1301616668701175,
+      "epoch": 0.07015059853262968,
+      "grad_norm": 1.28125,
+      "learning_rate": 0.0004990234929580494,
+      "loss": 6.1395,
+      "mean_token_accuracy": 0.13527224361896514,
+      "num_tokens": 2459127.0,
+      "step": 1090
+    },
+    {
+      "entropy": 6.3425384044647215,
+      "epoch": 0.07047239026901789,
+      "grad_norm": 1.0234375,
+      "learning_rate": 0.0004989107821416609,
+      "loss": 6.2899,
+      "mean_token_accuracy": 0.133147981017828,
+      "num_tokens": 2470913.0,
+      "step": 1095
+    },
+    {
+      "entropy": 6.121605920791626,
+      "epoch": 0.0707941820054061,
+      "grad_norm": 1.1875,
+      "learning_rate": 0.0004987919326980723,
+      "loss": 6.2032,
+      "mean_token_accuracy": 0.14263079017400743,
+      "num_tokens": 2481214.0,
+      "step": 1100
+    },
+    {
+      "entropy": 6.2000030994415285,
+      "epoch": 0.0711159737417943,
+      "grad_norm": 0.9453125,
+      "learning_rate": 0.0004986669478856011,
+      "loss": 6.1731,
+      "mean_token_accuracy": 0.141845715790987,
+      "num_tokens": 2493050.0,
+      "step": 1105
+    },
+    {
+      "entropy": 6.254453802108765,
+      "epoch": 0.07143776547818252,
+      "grad_norm": 1.046875,
+      "learning_rate": 0.0004985358311307688,
+      "loss": 6.1826,
+      "mean_token_accuracy": 0.14366479367017745,
+      "num_tokens": 2504406.0,
+      "step": 1110
+    },
+    {
+      "entropy": 6.1378173828125,
+      "epoch": 0.07175955721457072,
+      "grad_norm": 1.1015625,
+      "learning_rate": 0.0004983985860282081,
+      "loss": 6.258,
+      "mean_token_accuracy": 0.1386260747909546,
+      "num_tokens": 2514643.0,
+      "step": 1115
+    },
+    {
+      "entropy": 6.324863004684448,
+      "epoch": 0.07208134895095894,
+      "grad_norm": 1.171875,
+      "learning_rate": 0.0004982552163405623,
+      "loss": 6.1865,
+      "mean_token_accuracy": 0.14131532087922097,
+      "num_tokens": 2524924.0,
+      "step": 1120
+    },
+    {
+      "entropy": 6.1816126823425295,
+      "epoch": 0.07240314068734714,
+      "grad_norm": 1.125,
+      "learning_rate": 0.0004981057259983839,
+      "loss": 6.2598,
+      "mean_token_accuracy": 0.14671514555811882,
+      "num_tokens": 2535447.0,
+      "step": 1125
+    },
+    {
+      "entropy": 6.25083646774292,
+      "epoch": 0.07272493242373536,
+      "grad_norm": 0.984375,
+      "learning_rate": 0.0004979501191000262,
+      "loss": 6.1821,
+      "mean_token_accuracy": 0.14264000579714775,
+      "num_tokens": 2547308.0,
+      "step": 1130
+    },
+    {
+      "entropy": 6.189724493026733,
+      "epoch": 0.07304672416012357,
+      "grad_norm": 1.1640625,
+      "learning_rate": 0.0004977883999115311,
+      "loss": 6.1076,
+      "mean_token_accuracy": 0.1410246193408966,
+      "num_tokens": 2557319.0,
+      "step": 1135
+    },
+    {
+      "entropy": 6.084145212173462,
+      "epoch": 0.07336851589651178,
+      "grad_norm": 0.99609375,
+      "learning_rate": 0.0004976205728665113,
+      "loss": 6.096,
+      "mean_token_accuracy": 0.14267066717147828,
+      "num_tokens": 2570099.0,
+      "step": 1140
+    },
+    {
+      "entropy": 6.136931037902832,
+      "epoch": 0.07369030763289999,
+      "grad_norm": 1.1328125,
+      "learning_rate": 0.0004974466425660307,
+      "loss": 6.089,
+      "mean_token_accuracy": 0.14493471682071685,
+      "num_tokens": 2580492.0,
+      "step": 1145
+    },
+    {
+      "entropy": 6.092673969268799,
+      "epoch": 0.0740120993692882,
+      "grad_norm": 1.0625,
+      "learning_rate": 0.0004972666137784759,
+      "loss": 6.1957,
+      "mean_token_accuracy": 0.14505093023180962,
+      "num_tokens": 2591564.0,
+      "step": 1150
+    },
+    {
+      "entropy": 6.166257286071778,
+      "epoch": 0.07433389110567641,
+      "grad_norm": 1.171875,
+      "learning_rate": 0.0004970804914394271,
+      "loss": 6.0627,
+      "mean_token_accuracy": 0.1405116766691208,
+      "num_tokens": 2601611.0,
+      "step": 1155
+    },
+    {
+      "entropy": 6.123493099212647,
+      "epoch": 0.07465568284206461,
+      "grad_norm": 1.1015625,
+      "learning_rate": 0.0004968882806515225,
+      "loss": 6.0904,
+      "mean_token_accuracy": 0.14291229397058486,
+      "num_tokens": 2611945.0,
+      "step": 1160
+    },
+    {
+      "entropy": 6.176377582550049,
+      "epoch": 0.07497747457845283,
+      "grad_norm": 1.0703125,
+      "learning_rate": 0.0004966899866843177,
+      "loss": 6.1528,
+      "mean_token_accuracy": 0.14379807338118553,
+      "num_tokens": 2622520.0,
+      "step": 1165
+    },
+    {
+      "entropy": 6.097160577774048,
+      "epoch": 0.07529926631484103,
+      "grad_norm": 1.0078125,
+      "learning_rate": 0.000496485614974142,
+      "loss": 6.1361,
+      "mean_token_accuracy": 0.14614415317773818,
+      "num_tokens": 2634054.0,
+      "step": 1170
+    },
+    {
+      "entropy": 6.183728504180908,
+      "epoch": 0.07562105805122925,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.0004962751711239492,
+      "loss": 6.2013,
+      "mean_token_accuracy": 0.14067911207675934,
+      "num_tokens": 2645760.0,
+      "step": 1175
+    },
+    {
+      "entropy": 6.19887409210205,
+      "epoch": 0.07594284978761745,
+      "grad_norm": 1.1328125,
+      "learning_rate": 0.0004960586609031636,
+      "loss": 6.1602,
+      "mean_token_accuracy": 0.13677565157413482,
+      "num_tokens": 2657599.0,
+      "step": 1180
+    },
+    {
+      "entropy": 6.193633031845093,
+      "epoch": 0.07626464152400567,
+      "grad_norm": 1.109375,
+      "learning_rate": 0.0004958360902475224,
+      "loss": 6.2417,
+      "mean_token_accuracy": 0.13924393579363822,
+      "num_tokens": 2669962.0,
+      "step": 1185
+    },
+    {
+      "entropy": 6.172778654098511,
+      "epoch": 0.07658643326039387,
+      "grad_norm": 1.0,
+      "learning_rate": 0.0004956074652589125,
+      "loss": 6.1701,
+      "mean_token_accuracy": 0.14136232435703278,
+      "num_tokens": 2682416.0,
+      "step": 1190
+    },
+    {
+      "entropy": 6.083469009399414,
+      "epoch": 0.07690822499678208,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.0004953727922052035,
+      "loss": 6.0961,
+      "mean_token_accuracy": 0.14915101528167723,
+      "num_tokens": 2692095.0,
+      "step": 1195
+    },
+    {
+      "entropy": 6.182906866073608,
+      "epoch": 0.07723001673317029,
+      "grad_norm": 1.078125,
+      "learning_rate": 0.0004951320775200756,
+      "loss": 6.1357,
+      "mean_token_accuracy": 0.1437535248696804,
+      "num_tokens": 2702705.0,
+      "step": 1200
+    },
+    {
+      "entropy": 6.094074058532715,
+      "epoch": 0.0775518084695585,
+      "grad_norm": 1.1640625,
+      "learning_rate": 0.0004948853278028436,
+      "loss": 6.1216,
+      "mean_token_accuracy": 0.14604368135333062,
+      "num_tokens": 2713280.0,
+      "step": 1205
+    },
+    {
+      "entropy": 6.195901679992676,
+      "epoch": 0.0778736002059467,
+      "grad_norm": 0.91015625,
+      "learning_rate": 0.0004946325498182755,
+      "loss": 6.159,
+      "mean_token_accuracy": 0.14376554638147354,
+      "num_tokens": 2725242.0,
+      "step": 1210
+    },
+    {
+      "entropy": 6.108611440658569,
+      "epoch": 0.07819539194233492,
+      "grad_norm": 1.015625,
+      "learning_rate": 0.0004943737504964076,
+      "loss": 6.1032,
+      "mean_token_accuracy": 0.14644696861505507,
+      "num_tokens": 2736895.0,
+      "step": 1215
+    },
+    {
+      "entropy": 6.193878316879273,
+      "epoch": 0.07851718367872312,
+      "grad_norm": 1.0390625,
+      "learning_rate": 0.000494108936932354,
+      "loss": 6.0713,
+      "mean_token_accuracy": 0.15232568681240083,
+      "num_tokens": 2748919.0,
+      "step": 1220
+    },
+    {
+      "entropy": 6.20891547203064,
+      "epoch": 0.07883897541511134,
+      "grad_norm": 1.0234375,
+      "learning_rate": 0.0004938381163861124,
+      "loss": 6.0809,
+      "mean_token_accuracy": 0.14824541360139848,
+      "num_tokens": 2760158.0,
+      "step": 1225
+    },
+    {
+      "entropy": 6.085719776153565,
+      "epoch": 0.07916076715149956,
+      "grad_norm": 1.1015625,
+      "learning_rate": 0.0004935612962823645,
+      "loss": 6.1627,
+      "mean_token_accuracy": 0.1409171588718891,
+      "num_tokens": 2771968.0,
+      "step": 1230
+    },
+    {
+      "entropy": 6.098658227920533,
+      "epoch": 0.07948255888788776,
+      "grad_norm": 1.0078125,
+      "learning_rate": 0.0004932784842102739,
+      "loss": 6.0537,
+      "mean_token_accuracy": 0.14979517236351966,
+      "num_tokens": 2781611.0,
+      "step": 1235
+    },
+    {
+      "entropy": 5.994823932647705,
+      "epoch": 0.07980435062427597,
+      "grad_norm": 1.0,
+      "learning_rate": 0.0004929896879232758,
+      "loss": 6.0589,
+      "mean_token_accuracy": 0.15257512778043747,
+      "num_tokens": 2792800.0,
+      "step": 1240
+    },
+    {
+      "entropy": 6.129728031158447,
+      "epoch": 0.08012614236066418,
+      "grad_norm": 1.0703125,
+      "learning_rate": 0.0004926949153388668,
+      "loss": 6.059,
+      "mean_token_accuracy": 0.14132918566465377,
+      "num_tokens": 2803625.0,
+      "step": 1245
+    },
+    {
+      "entropy": 6.114728021621704,
+      "epoch": 0.08044793409705239,
+      "grad_norm": 1.0390625,
+      "learning_rate": 0.0004923941745383859,
+      "loss": 6.119,
+      "mean_token_accuracy": 0.1411253795027733,
+      "num_tokens": 2815836.0,
+      "step": 1250
+    },
+    {
+      "entropy": 6.101113939285279,
+      "epoch": 0.0807697258334406,
+      "grad_norm": 1.0546875,
+      "learning_rate": 0.000492087473766794,
+      "loss": 6.0744,
+      "mean_token_accuracy": 0.1473376877605915,
+      "num_tokens": 2827590.0,
+      "step": 1255
+    },
+    {
+      "entropy": 6.179189443588257,
+      "epoch": 0.08109151756982881,
+      "grad_norm": 0.97265625,
+      "learning_rate": 0.000491774821432448,
+      "loss": 6.2092,
+      "mean_token_accuracy": 0.14346556589007378,
+      "num_tokens": 2839469.0,
+      "step": 1260
+    },
+    {
+      "entropy": 6.161050510406494,
+      "epoch": 0.08141330930621701,
+      "grad_norm": 1.03125,
+      "learning_rate": 0.0004914562261068693,
+      "loss": 6.1127,
+      "mean_token_accuracy": 0.14351990967988967,
+      "num_tokens": 2851424.0,
+      "step": 1265
+    },
+    {
+      "entropy": 5.995301198959351,
+      "epoch": 0.08173510104260523,
+      "grad_norm": 1.015625,
+      "learning_rate": 0.0004911316965245098,
+      "loss": 5.9972,
+      "mean_token_accuracy": 0.14706434458494186,
+      "num_tokens": 2862818.0,
+      "step": 1270
+    },
+    {
+      "entropy": 6.226345491409302,
+      "epoch": 0.08205689277899343,
+      "grad_norm": 1.0,
+      "learning_rate": 0.000490801241582512,
+      "loss": 6.1326,
+      "mean_token_accuracy": 0.1427297107875347,
+      "num_tokens": 2874795.0,
+      "step": 1275
+    },
+    {
+      "entropy": 6.0090076446533205,
+      "epoch": 0.08237868451538165,
+      "grad_norm": 0.984375,
+      "learning_rate": 0.000490464870340465,
+      "loss": 6.1186,
+      "mean_token_accuracy": 0.14504720717668534,
+      "num_tokens": 2884309.0,
+      "step": 1280
+    },
+    {
+      "entropy": 6.246135234832764,
+      "epoch": 0.08270047625176985,
+      "grad_norm": 0.93359375,
+      "learning_rate": 0.0004901225920201563,
+      "loss": 6.1841,
+      "mean_token_accuracy": 0.13920361548662186,
+      "num_tokens": 2895149.0,
+      "step": 1285
+    },
+    {
+      "entropy": 5.9457801342010494,
+      "epoch": 0.08302226798815807,
+      "grad_norm": 1.046875,
+      "learning_rate": 0.000489774416005319,
+      "loss": 5.953,
+      "mean_token_accuracy": 0.15055820271372794,
+      "num_tokens": 2906311.0,
+      "step": 1290
+    },
+    {
+      "entropy": 6.034160470962524,
+      "epoch": 0.08334405972454627,
+      "grad_norm": 0.9140625,
+      "learning_rate": 0.0004894203518413742,
+      "loss": 6.0461,
+      "mean_token_accuracy": 0.15890783071517944,
+      "num_tokens": 2918706.0,
+      "step": 1295
+    },
+    {
+      "entropy": 6.070583534240723,
+      "epoch": 0.08366585146093448,
+      "grad_norm": 1.0234375,
+      "learning_rate": 0.0004890604092351701,
+      "loss": 6.0141,
+      "mean_token_accuracy": 0.1481314405798912,
+      "num_tokens": 2929569.0,
+      "step": 1300
+    },
+    {
+      "entropy": 5.892015266418457,
+      "epoch": 0.08398764319732269,
+      "grad_norm": 1.0,
+      "learning_rate": 0.000488694598054715,
+      "loss": 5.9925,
+      "mean_token_accuracy": 0.15369523614645003,
+      "num_tokens": 2941286.0,
+      "step": 1305
+    },
+    {
+      "entropy": 6.097052526473999,
+      "epoch": 0.0843094349337109,
+      "grad_norm": 0.9765625,
+      "learning_rate": 0.0004883229283289071,
+      "loss": 6.0232,
+      "mean_token_accuracy": 0.1539250060915947,
+      "num_tokens": 2952490.0,
+      "step": 1310
+    },
+    {
+      "entropy": 6.123910570144654,
+      "epoch": 0.0846312266700991,
+      "grad_norm": 1.0078125,
+      "learning_rate": 0.00048794541024725993,
+      "loss": 6.0977,
+      "mean_token_accuracy": 0.14586264193058013,
+      "num_tokens": 2964381.0,
+      "step": 1315
+    },
+    {
+      "entropy": 6.094405794143677,
+      "epoch": 0.08495301840648732,
+      "grad_norm": 1.109375,
+      "learning_rate": 0.0004875620541596221,
+      "loss": 6.0476,
+      "mean_token_accuracy": 0.14707987383008003,
+      "num_tokens": 2974769.0,
+      "step": 1320
+    },
+    {
+      "entropy": 6.1349348545074465,
+      "epoch": 0.08527481014287554,
+      "grad_norm": 1.0703125,
+      "learning_rate": 0.00048717287057589454,
+      "loss": 6.1471,
+      "mean_token_accuracy": 0.13929664865136146,
+      "num_tokens": 2986333.0,
+      "step": 1325
+    },
+    {
+      "entropy": 6.084645557403564,
+      "epoch": 0.08559660187926374,
+      "grad_norm": 1.0625,
+      "learning_rate": 0.0004867778701657417,
+      "loss": 6.0341,
+      "mean_token_accuracy": 0.14553362131118774,
+      "num_tokens": 2997642.0,
+      "step": 1330
+    },
+    {
+      "entropy": 6.0231305122375485,
+      "epoch": 0.08591839361565196,
+      "grad_norm": 0.98828125,
+      "learning_rate": 0.00048637706375829955,
+      "loss": 6.0573,
+      "mean_token_accuracy": 0.15109685510396959,
+      "num_tokens": 3009847.0,
+      "step": 1335
+    },
+    {
+      "entropy": 6.18050627708435,
+      "epoch": 0.08624018535204016,
+      "grad_norm": 1.0390625,
+      "learning_rate": 0.000485970462341878,
+      "loss": 6.0891,
+      "mean_token_accuracy": 0.1480099990963936,
+      "num_tokens": 3019657.0,
+      "step": 1340
+    },
+    {
+      "entropy": 6.064053153991699,
+      "epoch": 0.08656197708842837,
+      "grad_norm": 0.97265625,
+      "learning_rate": 0.00048555807706366044,
+      "loss": 6.1594,
+      "mean_token_accuracy": 0.14406584575772285,
+      "num_tokens": 3030507.0,
+      "step": 1345
+    },
+    {
+      "entropy": 6.17887053489685,
+      "epoch": 0.08688376882481658,
+      "grad_norm": 0.9609375,
+      "learning_rate": 0.00048513991922939756,
+      "loss": 6.1747,
+      "mean_token_accuracy": 0.14376262128353118,
+      "num_tokens": 3041751.0,
+      "step": 1350
+    },
+    {
+      "entropy": 6.123253107070923,
+      "epoch": 0.08720556056120479,
+      "grad_norm": 1.03125,
+      "learning_rate": 0.00048471600030309744,
+      "loss": 6.0863,
+      "mean_token_accuracy": 0.1487148091197014,
+      "num_tokens": 3052754.0,
+      "step": 1355
+    },
+    {
+      "entropy": 6.192664575576782,
+      "epoch": 0.087527352297593,
+      "grad_norm": 1.1015625,
+      "learning_rate": 0.00048428633190671186,
+      "loss": 6.1462,
+      "mean_token_accuracy": 0.1416320726275444,
+      "num_tokens": 3064731.0,
+      "step": 1360
+    },
+    {
+      "entropy": 6.012926530838013,
+      "epoch": 0.08784914403398121,
+      "grad_norm": 1.1328125,
+      "learning_rate": 0.0004838509258198167,
+      "loss": 5.9544,
+      "mean_token_accuracy": 0.1551859274506569,
+      "num_tokens": 3075093.0,
+      "step": 1365
+    },
+    {
+      "entropy": 6.086758947372436,
+      "epoch": 0.08817093577036941,
+      "grad_norm": 1.0,
+      "learning_rate": 0.00048340979397929,
+      "loss": 6.112,
+      "mean_token_accuracy": 0.1472473606467247,
+      "num_tokens": 3086303.0,
+      "step": 1370
+    },
+    {
+      "entropy": 6.017410135269165,
+      "epoch": 0.08849272750675763,
+      "grad_norm": 0.99609375,
+      "learning_rate": 0.00048296294847898386,
+      "loss": 6.0718,
+      "mean_token_accuracy": 0.14377797544002532,
+      "num_tokens": 3097916.0,
+      "step": 1375
+    },
+    {
+      "entropy": 6.118084335327149,
+      "epoch": 0.08881451924314583,
+      "grad_norm": 1.046875,
+      "learning_rate": 0.0004825104015693934,
+      "loss": 5.997,
+      "mean_token_accuracy": 0.15102456882596016,
+      "num_tokens": 3109168.0,
+      "step": 1380
+    },
+    {
+      "entropy": 5.975411653518677,
+      "epoch": 0.08913631097953405,
+      "grad_norm": 0.9921875,
+      "learning_rate": 0.0004820521656573208,
+      "loss": 6.0083,
+      "mean_token_accuracy": 0.15250723212957382,
+      "num_tokens": 3120739.0,
+      "step": 1385
+    },
+    {
+      "entropy": 6.164347076416016,
+      "epoch": 0.08945810271592225,
+      "grad_norm": 1.0,
+      "learning_rate": 0.00048158825330553505,
+      "loss": 6.0871,
+      "mean_token_accuracy": 0.1393938921391964,
+      "num_tokens": 3131812.0,
+      "step": 1390
+    },
+    {
+      "entropy": 6.07136378288269,
+      "epoch": 0.08977989445231047,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.00048111867723242763,
+      "loss": 6.0846,
+      "mean_token_accuracy": 0.14521935135126113,
+      "num_tokens": 3142430.0,
+      "step": 1395
+    },
+    {
+      "entropy": 5.9969603538513185,
+      "epoch": 0.09010168618869867,
+      "grad_norm": 0.96484375,
+      "learning_rate": 0.0004806434503116637,
+      "loss": 5.9878,
+      "mean_token_accuracy": 0.1448088675737381,
+      "num_tokens": 3154552.0,
+      "step": 1400
+    },
+    {
+      "entropy": 6.107782697677612,
+      "epoch": 0.09042347792508688,
+      "grad_norm": 0.88671875,
+      "learning_rate": 0.0004801625855718296,
+      "loss": 6.0186,
+      "mean_token_accuracy": 0.1484021157026291,
+      "num_tokens": 3167117.0,
+      "step": 1405
+    },
+    {
+      "entropy": 6.196589136123658,
+      "epoch": 0.09074526966147509,
+      "grad_norm": 1.078125,
+      "learning_rate": 0.00047967609619607477,
+      "loss": 6.176,
+      "mean_token_accuracy": 0.14222314432263375,
+      "num_tokens": 3177160.0,
+      "step": 1410
+    },
+    {
+      "entropy": 5.963523435592651,
+      "epoch": 0.0910670613978633,
+      "grad_norm": 1.0078125,
+      "learning_rate": 0.0004791839955217513,
+      "loss": 5.9977,
+      "mean_token_accuracy": 0.15531311482191085,
+      "num_tokens": 3188036.0,
+      "step": 1415
+    },
+    {
+      "entropy": 5.970466613769531,
+      "epoch": 0.09138885313425152,
+      "grad_norm": 1.0078125,
+      "learning_rate": 0.00047868629704004786,
+      "loss": 5.9436,
+      "mean_token_accuracy": 0.15874468088150023,
+      "num_tokens": 3199507.0,
+      "step": 1420
+    },
+    {
+      "entropy": 5.997041034698486,
+      "epoch": 0.09171064487063972,
+      "grad_norm": 1.0859375,
+      "learning_rate": 0.00047818301439561965,
+      "loss": 5.8895,
+      "mean_token_accuracy": 0.1560737043619156,
+      "num_tokens": 3210316.0,
+      "step": 1425
+    },
+    {
+      "entropy": 6.003460884094238,
+      "epoch": 0.09203243660702794,
+      "grad_norm": 1.046875,
+      "learning_rate": 0.00047767416138621454,
+      "loss": 6.058,
+      "mean_token_accuracy": 0.14701004475355148,
+      "num_tokens": 3221794.0,
+      "step": 1430
+    },
+    {
+      "entropy": 6.125612449645996,
+      "epoch": 0.09235422834341614,
+      "grad_norm": 1.0390625,
+      "learning_rate": 0.000477159751962295,
+      "loss": 6.0359,
+      "mean_token_accuracy": 0.1510167494416237,
+      "num_tokens": 3232950.0,
+      "step": 1435
+    },
+    {
+      "entropy": 6.005368185043335,
+      "epoch": 0.09267602007980436,
+      "grad_norm": 1.0546875,
+      "learning_rate": 0.00047663980022665507,
+      "loss": 5.9723,
+      "mean_token_accuracy": 0.15661734193563462,
+      "num_tokens": 3244012.0,
+      "step": 1440
+    },
+    {
+      "entropy": 5.998808765411377,
+      "epoch": 0.09299781181619256,
+      "grad_norm": 1.0390625,
+      "learning_rate": 0.00047611432043403437,
+      "loss": 6.0681,
+      "mean_token_accuracy": 0.1516854725778103,
+      "num_tokens": 3255599.0,
+      "step": 1445
+    },
+    {
+      "entropy": 6.070061779022216,
+      "epoch": 0.09331960355258077,
+      "grad_norm": 1.0078125,
+      "learning_rate": 0.0004755833269907267,
+      "loss": 5.9832,
+      "mean_token_accuracy": 0.1570393592119217,
+      "num_tokens": 3267060.0,
+      "step": 1450
+    },
+    {
+      "entropy": 6.018462991714477,
+      "epoch": 0.09364139528896898,
+      "grad_norm": 1.0546875,
+      "learning_rate": 0.0004750468344541857,
+      "loss": 6.0012,
+      "mean_token_accuracy": 0.15066305920481682,
+      "num_tokens": 3278252.0,
+      "step": 1455
+    },
+    {
+      "entropy": 5.992702293395996,
+      "epoch": 0.09396318702535719,
+      "grad_norm": 1.0078125,
+      "learning_rate": 0.00047450485753262525,
+      "loss": 5.7633,
+      "mean_token_accuracy": 0.15733575001358985,
+      "num_tokens": 3288551.0,
+      "step": 1460
+    },
+    {
+      "entropy": 5.891961193084716,
+      "epoch": 0.0942849787617454,
+      "grad_norm": 1.140625,
+      "learning_rate": 0.00047395741108461633,
+      "loss": 6.0299,
+      "mean_token_accuracy": 0.14577169418334962,
+      "num_tokens": 3300424.0,
+      "step": 1465
+    },
+    {
+      "entropy": 6.028947257995606,
+      "epoch": 0.09460677049813361,
+      "grad_norm": 0.953125,
+      "learning_rate": 0.00047340451011867985,
+      "loss": 5.9416,
+      "mean_token_accuracy": 0.1503148078918457,
+      "num_tokens": 3312224.0,
+      "step": 1470
+    },
+    {
+      "entropy": 5.910190534591675,
+      "epoch": 0.09492856223452181,
+      "grad_norm": 0.95703125,
+      "learning_rate": 0.00047284616979287515,
+      "loss": 5.9129,
+      "mean_token_accuracy": 0.1531186506152153,
+      "num_tokens": 3323262.0,
+      "step": 1475
+    },
+    {
+      "entropy": 5.984435129165649,
+      "epoch": 0.09525035397091003,
+      "grad_norm": 0.9609375,
+      "learning_rate": 0.00047228240541438433,
+      "loss": 5.9634,
+      "mean_token_accuracy": 0.15215728729963302,
+      "num_tokens": 3334016.0,
+      "step": 1480
+    },
+    {
+      "entropy": 6.027677917480469,
+      "epoch": 0.09557214570729823,
+      "grad_norm": 1.0859375,
+      "learning_rate": 0.00047171323243909257,
+      "loss": 6.0225,
+      "mean_token_accuracy": 0.15151964053511618,
+      "num_tokens": 3344026.0,
+      "step": 1485
+    },
+    {
+      "entropy": 6.137910842895508,
+      "epoch": 0.09589393744368645,
+      "grad_norm": 0.93359375,
+      "learning_rate": 0.00047113866647116457,
+      "loss": 6.1397,
+      "mean_token_accuracy": 0.14101183116436006,
+      "num_tokens": 3355453.0,
+      "step": 1490
+    },
+    {
+      "entropy": 5.969770812988282,
+      "epoch": 0.09621572918007465,
+      "grad_norm": 1.1328125,
+      "learning_rate": 0.0004705587232626164,
+      "loss": 5.9421,
+      "mean_token_accuracy": 0.15476928949356078,
+      "num_tokens": 3366719.0,
+      "step": 1495
+    },
+    {
+      "entropy": 5.972850036621094,
+      "epoch": 0.09653752091646287,
+      "grad_norm": 1.0234375,
+      "learning_rate": 0.00046997341871288424,
+      "loss": 5.9765,
+      "mean_token_accuracy": 0.1517869286239147,
+      "num_tokens": 3377689.0,
+      "step": 1500
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 4000,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5299582150656000.0,
+  "train_batch_size": 16,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-1500/training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0e69be2534cda29aa68fbf853cde65983229fedc597c556fd41153df07f5130f
+size 6289