Training in progress, step 140, checkpoint

checkpoint-140/README.md

@@ -0,0 +1,210 @@
+---
+base_model: unsloth/Qwen2.5-32B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:unsloth/Qwen2.5-32B-Instruct
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.0

checkpoint-140/adapter_config.json

@@ -0,0 +1,50 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "Qwen2ForCausalLM",
+    "parent_library": "transformers.models.qwen2.modeling_qwen2",
+    "unsloth_fixed": true
+  },
+  "base_model_name_or_path": "unsloth/Qwen2.5-32B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.0",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "k_proj",
+    "down_proj",
+    "q_proj",
+    "v_proj",
+    "gate_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": true
+}

checkpoint-140/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6628d84ccaa4204790c8219226b70746720bfcd70c0fd0ec9d66d4636cc055b2
+size 1073863208

checkpoint-140/added_tokens.json

@@ -0,0 +1,25 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|PAD_TOKEN|>": 151665,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-140/chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

checkpoint-140/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a3fe1c33eac7896669b474eb7d2e5216ebb0ff7038efd698d3b50cc5d967815
+size 546317671

checkpoint-140/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cde881b24af86d8870de73e5328bf8754c9e262c58bcd875cf451eca0e6671d5
+size 14581

checkpoint-140/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6ea102c9925cfea7de9fb0f8ed1d844670667fbd08afbc9abd88635191a44c86
+size 1465

checkpoint-140/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|PAD_TOKEN|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-140/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fab42efe8d17406525a9154b728cf9e957629a8ed7ce997770efdd71128c6a1a
+size 11422086

checkpoint-140/tokenizer_config.json

@@ -0,0 +1,216 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<|PAD_TOKEN|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 32768,
+  "pad_token": "<|PAD_TOKEN|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-140/trainer_state.json

@@ -0,0 +1,1014 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.4218455743879473,
+  "eval_steps": 500,
+  "global_step": 140,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0030131826741996233,
+      "grad_norm": 34.6861457824707,
+      "learning_rate": 0.0,
+      "loss": 4.7036,
+      "step": 1
+    },
+    {
+      "epoch": 0.006026365348399247,
+      "grad_norm": 33.215126037597656,
+      "learning_rate": 2e-07,
+      "loss": 4.4381,
+      "step": 2
+    },
+    {
+      "epoch": 0.00903954802259887,
+      "grad_norm": 31.91962242126465,
+      "learning_rate": 4e-07,
+      "loss": 4.2738,
+      "step": 3
+    },
+    {
+      "epoch": 0.012052730696798493,
+      "grad_norm": 33.510711669921875,
+      "learning_rate": 6e-07,
+      "loss": 4.5457,
+      "step": 4
+    },
+    {
+      "epoch": 0.015065913370998116,
+      "grad_norm": 33.2039909362793,
+      "learning_rate": 8e-07,
+      "loss": 4.4126,
+      "step": 5
+    },
+    {
+      "epoch": 0.01807909604519774,
+      "grad_norm": 34.26924514770508,
+      "learning_rate": 1e-06,
+      "loss": 4.4619,
+      "step": 6
+    },
+    {
+      "epoch": 0.021092278719397364,
+      "grad_norm": 34.04632568359375,
+      "learning_rate": 9.993957703927493e-07,
+      "loss": 4.1716,
+      "step": 7
+    },
+    {
+      "epoch": 0.024105461393596987,
+      "grad_norm": 32.70482635498047,
+      "learning_rate": 9.987915407854984e-07,
+      "loss": 4.1148,
+      "step": 8
+    },
+    {
+      "epoch": 0.02711864406779661,
+      "grad_norm": 37.01934814453125,
+      "learning_rate": 9.981873111782478e-07,
+      "loss": 4.3351,
+      "step": 9
+    },
+    {
+      "epoch": 0.030131826741996232,
+      "grad_norm": 36.172142028808594,
+      "learning_rate": 9.97583081570997e-07,
+      "loss": 4.1667,
+      "step": 10
+    },
+    {
+      "epoch": 0.03314500941619586,
+      "grad_norm": 40.31679153442383,
+      "learning_rate": 9.969788519637462e-07,
+      "loss": 4.2093,
+      "step": 11
+    },
+    {
+      "epoch": 0.03615819209039548,
+      "grad_norm": 40.15742492675781,
+      "learning_rate": 9.963746223564954e-07,
+      "loss": 4.2268,
+      "step": 12
+    },
+    {
+      "epoch": 0.039171374764595104,
+      "grad_norm": 39.929691314697266,
+      "learning_rate": 9.957703927492447e-07,
+      "loss": 3.9355,
+      "step": 13
+    },
+    {
+      "epoch": 0.04218455743879473,
+      "grad_norm": 39.41320037841797,
+      "learning_rate": 9.951661631419938e-07,
+      "loss": 4.0088,
+      "step": 14
+    },
+    {
+      "epoch": 0.04519774011299435,
+      "grad_norm": 37.93269729614258,
+      "learning_rate": 9.945619335347432e-07,
+      "loss": 3.6819,
+      "step": 15
+    },
+    {
+      "epoch": 0.04821092278719397,
+      "grad_norm": 38.64561462402344,
+      "learning_rate": 9.939577039274925e-07,
+      "loss": 3.7213,
+      "step": 16
+    },
+    {
+      "epoch": 0.051224105461393596,
+      "grad_norm": 33.3484001159668,
+      "learning_rate": 9.933534743202417e-07,
+      "loss": 3.7547,
+      "step": 17
+    },
+    {
+      "epoch": 0.05423728813559322,
+      "grad_norm": 28.220916748046875,
+      "learning_rate": 9.92749244712991e-07,
+      "loss": 3.5524,
+      "step": 18
+    },
+    {
+      "epoch": 0.05725047080979284,
+      "grad_norm": 22.749616622924805,
+      "learning_rate": 9.921450151057401e-07,
+      "loss": 3.5342,
+      "step": 19
+    },
+    {
+      "epoch": 0.060263653483992465,
+      "grad_norm": 15.116613388061523,
+      "learning_rate": 9.915407854984895e-07,
+      "loss": 3.4557,
+      "step": 20
+    },
+    {
+      "epoch": 0.06327683615819209,
+      "grad_norm": 14.70132064819336,
+      "learning_rate": 9.909365558912386e-07,
+      "loss": 3.4788,
+      "step": 21
+    },
+    {
+      "epoch": 0.06629001883239172,
+      "grad_norm": 14.527510643005371,
+      "learning_rate": 9.90332326283988e-07,
+      "loss": 3.3028,
+      "step": 22
+    },
+    {
+      "epoch": 0.06930320150659133,
+      "grad_norm": 12.996399879455566,
+      "learning_rate": 9.89728096676737e-07,
+      "loss": 3.2357,
+      "step": 23
+    },
+    {
+      "epoch": 0.07231638418079096,
+      "grad_norm": 13.773697853088379,
+      "learning_rate": 9.891238670694864e-07,
+      "loss": 3.2755,
+      "step": 24
+    },
+    {
+      "epoch": 0.07532956685499058,
+      "grad_norm": 13.862228393554688,
+      "learning_rate": 9.885196374622357e-07,
+      "loss": 3.3604,
+      "step": 25
+    },
+    {
+      "epoch": 0.07834274952919021,
+      "grad_norm": 12.897477149963379,
+      "learning_rate": 9.879154078549849e-07,
+      "loss": 3.3267,
+      "step": 26
+    },
+    {
+      "epoch": 0.08135593220338982,
+      "grad_norm": 14.752609252929688,
+      "learning_rate": 9.873111782477342e-07,
+      "loss": 3.2435,
+      "step": 27
+    },
+    {
+      "epoch": 0.08436911487758945,
+      "grad_norm": 12.743549346923828,
+      "learning_rate": 9.867069486404833e-07,
+      "loss": 3.1507,
+      "step": 28
+    },
+    {
+      "epoch": 0.08738229755178907,
+      "grad_norm": 11.101383209228516,
+      "learning_rate": 9.861027190332327e-07,
+      "loss": 3.1273,
+      "step": 29
+    },
+    {
+      "epoch": 0.0903954802259887,
+      "grad_norm": 11.95251178741455,
+      "learning_rate": 9.854984894259818e-07,
+      "loss": 3.0316,
+      "step": 30
+    },
+    {
+      "epoch": 0.09340866290018833,
+      "grad_norm": 10.766655921936035,
+      "learning_rate": 9.848942598187312e-07,
+      "loss": 3.1074,
+      "step": 31
+    },
+    {
+      "epoch": 0.09642184557438795,
+      "grad_norm": 11.15977954864502,
+      "learning_rate": 9.842900302114803e-07,
+      "loss": 2.9768,
+      "step": 32
+    },
+    {
+      "epoch": 0.09943502824858758,
+      "grad_norm": 10.29792594909668,
+      "learning_rate": 9.836858006042296e-07,
+      "loss": 3.083,
+      "step": 33
+    },
+    {
+      "epoch": 0.10244821092278719,
+      "grad_norm": 9.040534019470215,
+      "learning_rate": 9.830815709969788e-07,
+      "loss": 2.8304,
+      "step": 34
+    },
+    {
+      "epoch": 0.10546139359698682,
+      "grad_norm": 9.023050308227539,
+      "learning_rate": 9.82477341389728e-07,
+      "loss": 2.6917,
+      "step": 35
+    },
+    {
+      "epoch": 0.10847457627118644,
+      "grad_norm": 10.378137588500977,
+      "learning_rate": 9.818731117824774e-07,
+      "loss": 2.8607,
+      "step": 36
+    },
+    {
+      "epoch": 0.11148775894538607,
+      "grad_norm": 9.073391914367676,
+      "learning_rate": 9.812688821752266e-07,
+      "loss": 2.8967,
+      "step": 37
+    },
+    {
+      "epoch": 0.11450094161958568,
+      "grad_norm": 8.33003044128418,
+      "learning_rate": 9.80664652567976e-07,
+      "loss": 2.6255,
+      "step": 38
+    },
+    {
+      "epoch": 0.11751412429378531,
+      "grad_norm": 8.349893569946289,
+      "learning_rate": 9.80060422960725e-07,
+      "loss": 2.7113,
+      "step": 39
+    },
+    {
+      "epoch": 0.12052730696798493,
+      "grad_norm": 8.032829284667969,
+      "learning_rate": 9.794561933534744e-07,
+      "loss": 2.7488,
+      "step": 40
+    },
+    {
+      "epoch": 0.12354048964218456,
+      "grad_norm": 8.089343070983887,
+      "learning_rate": 9.788519637462235e-07,
+      "loss": 2.5701,
+      "step": 41
+    },
+    {
+      "epoch": 0.12655367231638417,
+      "grad_norm": 8.112906455993652,
+      "learning_rate": 9.782477341389729e-07,
+      "loss": 2.5999,
+      "step": 42
+    },
+    {
+      "epoch": 0.1295668549905838,
+      "grad_norm": 7.398575782775879,
+      "learning_rate": 9.77643504531722e-07,
+      "loss": 2.6078,
+      "step": 43
+    },
+    {
+      "epoch": 0.13258003766478343,
+      "grad_norm": 7.844170093536377,
+      "learning_rate": 9.770392749244713e-07,
+      "loss": 2.5361,
+      "step": 44
+    },
+    {
+      "epoch": 0.13559322033898305,
+      "grad_norm": 7.322074890136719,
+      "learning_rate": 9.764350453172205e-07,
+      "loss": 2.5254,
+      "step": 45
+    },
+    {
+      "epoch": 0.13860640301318267,
+      "grad_norm": 6.919925689697266,
+      "learning_rate": 9.758308157099698e-07,
+      "loss": 2.4676,
+      "step": 46
+    },
+    {
+      "epoch": 0.1416195856873823,
+      "grad_norm": 6.065021514892578,
+      "learning_rate": 9.75226586102719e-07,
+      "loss": 2.338,
+      "step": 47
+    },
+    {
+      "epoch": 0.14463276836158193,
+      "grad_norm": 7.550880432128906,
+      "learning_rate": 9.746223564954683e-07,
+      "loss": 2.4482,
+      "step": 48
+    },
+    {
+      "epoch": 0.14764595103578154,
+      "grad_norm": 6.961904525756836,
+      "learning_rate": 9.740181268882174e-07,
+      "loss": 2.635,
+      "step": 49
+    },
+    {
+      "epoch": 0.15065913370998116,
+      "grad_norm": 6.212646484375,
+      "learning_rate": 9.734138972809667e-07,
+      "loss": 2.3776,
+      "step": 50
+    },
+    {
+      "epoch": 0.1536723163841808,
+      "grad_norm": 6.006916046142578,
+      "learning_rate": 9.728096676737159e-07,
+      "loss": 2.3018,
+      "step": 51
+    },
+    {
+      "epoch": 0.15668549905838042,
+      "grad_norm": 6.0531325340271,
+      "learning_rate": 9.722054380664652e-07,
+      "loss": 2.3164,
+      "step": 52
+    },
+    {
+      "epoch": 0.15969868173258003,
+      "grad_norm": 5.674480438232422,
+      "learning_rate": 9.716012084592146e-07,
+      "loss": 2.3049,
+      "step": 53
+    },
+    {
+      "epoch": 0.16271186440677965,
+      "grad_norm": 6.283177375793457,
+      "learning_rate": 9.709969788519637e-07,
+      "loss": 2.2727,
+      "step": 54
+    },
+    {
+      "epoch": 0.1657250470809793,
+      "grad_norm": 5.819633960723877,
+      "learning_rate": 9.70392749244713e-07,
+      "loss": 2.2122,
+      "step": 55
+    },
+    {
+      "epoch": 0.1687382297551789,
+      "grad_norm": 5.006446838378906,
+      "learning_rate": 9.697885196374622e-07,
+      "loss": 2.2215,
+      "step": 56
+    },
+    {
+      "epoch": 0.17175141242937852,
+      "grad_norm": 5.97379732131958,
+      "learning_rate": 9.691842900302115e-07,
+      "loss": 2.2061,
+      "step": 57
+    },
+    {
+      "epoch": 0.17476459510357814,
+      "grad_norm": 5.025632381439209,
+      "learning_rate": 9.685800604229606e-07,
+      "loss": 2.3006,
+      "step": 58
+    },
+    {
+      "epoch": 0.17777777777777778,
+      "grad_norm": 6.5014472007751465,
+      "learning_rate": 9.6797583081571e-07,
+      "loss": 2.2979,
+      "step": 59
+    },
+    {
+      "epoch": 0.1807909604519774,
+      "grad_norm": 4.874022960662842,
+      "learning_rate": 9.67371601208459e-07,
+      "loss": 2.095,
+      "step": 60
+    },
+    {
+      "epoch": 0.18380414312617702,
+      "grad_norm": 5.257139682769775,
+      "learning_rate": 9.667673716012084e-07,
+      "loss": 2.2371,
+      "step": 61
+    },
+    {
+      "epoch": 0.18681732580037666,
+      "grad_norm": 4.8642377853393555,
+      "learning_rate": 9.661631419939576e-07,
+      "loss": 2.2551,
+      "step": 62
+    },
+    {
+      "epoch": 0.18983050847457628,
+      "grad_norm": 5.601889610290527,
+      "learning_rate": 9.65558912386707e-07,
+      "loss": 2.1867,
+      "step": 63
+    },
+    {
+      "epoch": 0.1928436911487759,
+      "grad_norm": 5.198556900024414,
+      "learning_rate": 9.64954682779456e-07,
+      "loss": 2.0829,
+      "step": 64
+    },
+    {
+      "epoch": 0.1958568738229755,
+      "grad_norm": 5.988768100738525,
+      "learning_rate": 9.643504531722054e-07,
+      "loss": 2.1105,
+      "step": 65
+    },
+    {
+      "epoch": 0.19887005649717515,
+      "grad_norm": 5.178367614746094,
+      "learning_rate": 9.637462235649545e-07,
+      "loss": 2.0064,
+      "step": 66
+    },
+    {
+      "epoch": 0.20188323917137477,
+      "grad_norm": 5.225499153137207,
+      "learning_rate": 9.631419939577038e-07,
+      "loss": 1.9879,
+      "step": 67
+    },
+    {
+      "epoch": 0.20489642184557438,
+      "grad_norm": 5.521583080291748,
+      "learning_rate": 9.625377643504532e-07,
+      "loss": 2.1283,
+      "step": 68
+    },
+    {
+      "epoch": 0.207909604519774,
+      "grad_norm": 5.450455188751221,
+      "learning_rate": 9.619335347432023e-07,
+      "loss": 2.1023,
+      "step": 69
+    },
+    {
+      "epoch": 0.21092278719397364,
+      "grad_norm": 6.377326011657715,
+      "learning_rate": 9.613293051359517e-07,
+      "loss": 2.0945,
+      "step": 70
+    },
+    {
+      "epoch": 0.21393596986817326,
+      "grad_norm": 5.580687046051025,
+      "learning_rate": 9.607250755287008e-07,
+      "loss": 2.0603,
+      "step": 71
+    },
+    {
+      "epoch": 0.21694915254237288,
+      "grad_norm": 5.663399696350098,
+      "learning_rate": 9.601208459214501e-07,
+      "loss": 2.0243,
+      "step": 72
+    },
+    {
+      "epoch": 0.2199623352165725,
+      "grad_norm": 4.719912528991699,
+      "learning_rate": 9.595166163141993e-07,
+      "loss": 2.0211,
+      "step": 73
+    },
+    {
+      "epoch": 0.22297551789077213,
+      "grad_norm": 4.904620170593262,
+      "learning_rate": 9.589123867069486e-07,
+      "loss": 2.0721,
+      "step": 74
+    },
+    {
+      "epoch": 0.22598870056497175,
+      "grad_norm": 5.553138732910156,
+      "learning_rate": 9.583081570996977e-07,
+      "loss": 1.9293,
+      "step": 75
+    },
+    {
+      "epoch": 0.22900188323917137,
+      "grad_norm": 5.154430866241455,
+      "learning_rate": 9.57703927492447e-07,
+      "loss": 1.9106,
+      "step": 76
+    },
+    {
+      "epoch": 0.232015065913371,
+      "grad_norm": 5.0921630859375,
+      "learning_rate": 9.570996978851962e-07,
+      "loss": 1.8061,
+      "step": 77
+    },
+    {
+      "epoch": 0.23502824858757063,
+      "grad_norm": 5.445968151092529,
+      "learning_rate": 9.564954682779455e-07,
+      "loss": 1.8985,
+      "step": 78
+    },
+    {
+      "epoch": 0.23804143126177024,
+      "grad_norm": 4.871007442474365,
+      "learning_rate": 9.558912386706947e-07,
+      "loss": 1.9315,
+      "step": 79
+    },
+    {
+      "epoch": 0.24105461393596986,
+      "grad_norm": 5.1469526290893555,
+      "learning_rate": 9.55287009063444e-07,
+      "loss": 1.8471,
+      "step": 80
+    },
+    {
+      "epoch": 0.2440677966101695,
+      "grad_norm": 5.5420241355896,
+      "learning_rate": 9.546827794561934e-07,
+      "loss": 1.8587,
+      "step": 81
+    },
+    {
+      "epoch": 0.24708097928436912,
+      "grad_norm": 5.493891716003418,
+      "learning_rate": 9.540785498489425e-07,
+      "loss": 1.8932,
+      "step": 82
+    },
+    {
+      "epoch": 0.25009416195856876,
+      "grad_norm": 4.842357635498047,
+      "learning_rate": 9.534743202416918e-07,
+      "loss": 1.7296,
+      "step": 83
+    },
+    {
+      "epoch": 0.25310734463276835,
+      "grad_norm": 5.839240550994873,
+      "learning_rate": 9.528700906344411e-07,
+      "loss": 1.8418,
+      "step": 84
+    },
+    {
+      "epoch": 0.256120527306968,
+      "grad_norm": 6.450507640838623,
+      "learning_rate": 9.522658610271903e-07,
+      "loss": 1.7847,
+      "step": 85
+    },
+    {
+      "epoch": 0.2591337099811676,
+      "grad_norm": 5.740670204162598,
+      "learning_rate": 9.516616314199395e-07,
+      "loss": 1.8536,
+      "step": 86
+    },
+    {
+      "epoch": 0.2621468926553672,
+      "grad_norm": 5.523307800292969,
+      "learning_rate": 9.510574018126888e-07,
+      "loss": 1.8073,
+      "step": 87
+    },
+    {
+      "epoch": 0.26516007532956687,
+      "grad_norm": 4.190532207489014,
+      "learning_rate": 9.50453172205438e-07,
+      "loss": 1.7689,
+      "step": 88
+    },
+    {
+      "epoch": 0.26817325800376646,
+      "grad_norm": 4.71892786026001,
+      "learning_rate": 9.498489425981872e-07,
+      "loss": 1.7927,
+      "step": 89
+    },
+    {
+      "epoch": 0.2711864406779661,
+      "grad_norm": 3.9276342391967773,
+      "learning_rate": 9.492447129909366e-07,
+      "loss": 1.8459,
+      "step": 90
+    },
+    {
+      "epoch": 0.27419962335216574,
+      "grad_norm": 4.415136337280273,
+      "learning_rate": 9.486404833836857e-07,
+      "loss": 1.8626,
+      "step": 91
+    },
+    {
+      "epoch": 0.27721280602636533,
+      "grad_norm": 5.26611852645874,
+      "learning_rate": 9.48036253776435e-07,
+      "loss": 1.6375,
+      "step": 92
+    },
+    {
+      "epoch": 0.280225988700565,
+      "grad_norm": 4.380786895751953,
+      "learning_rate": 9.474320241691842e-07,
+      "loss": 1.7516,
+      "step": 93
+    },
+    {
+      "epoch": 0.2832391713747646,
+      "grad_norm": 4.630937099456787,
+      "learning_rate": 9.468277945619335e-07,
+      "loss": 1.6986,
+      "step": 94
+    },
+    {
+      "epoch": 0.2862523540489642,
+      "grad_norm": 4.607402801513672,
+      "learning_rate": 9.462235649546827e-07,
+      "loss": 1.717,
+      "step": 95
+    },
+    {
+      "epoch": 0.28926553672316385,
+      "grad_norm": 4.747241497039795,
+      "learning_rate": 9.45619335347432e-07,
+      "loss": 1.7993,
+      "step": 96
+    },
+    {
+      "epoch": 0.29227871939736344,
+      "grad_norm": 5.222727298736572,
+      "learning_rate": 9.450151057401811e-07,
+      "loss": 1.7622,
+      "step": 97
+    },
+    {
+      "epoch": 0.2952919020715631,
+      "grad_norm": 4.226417541503906,
+      "learning_rate": 9.444108761329305e-07,
+      "loss": 1.7105,
+      "step": 98
+    },
+    {
+      "epoch": 0.2983050847457627,
+      "grad_norm": 5.662446022033691,
+      "learning_rate": 9.438066465256798e-07,
+      "loss": 1.5551,
+      "step": 99
+    },
+    {
+      "epoch": 0.3013182674199623,
+      "grad_norm": 5.452689170837402,
+      "learning_rate": 9.432024169184289e-07,
+      "loss": 1.6254,
+      "step": 100
+    },
+    {
+      "epoch": 0.30433145009416196,
+      "grad_norm": 6.139257431030273,
+      "learning_rate": 9.425981873111783e-07,
+      "loss": 1.8607,
+      "step": 101
+    },
+    {
+      "epoch": 0.3073446327683616,
+      "grad_norm": 5.731149673461914,
+      "learning_rate": 9.419939577039274e-07,
+      "loss": 1.7846,
+      "step": 102
+    },
+    {
+      "epoch": 0.3103578154425612,
+      "grad_norm": 4.859795093536377,
+      "learning_rate": 9.413897280966767e-07,
+      "loss": 1.7599,
+      "step": 103
+    },
+    {
+      "epoch": 0.31337099811676083,
+      "grad_norm": 4.731764316558838,
+      "learning_rate": 9.407854984894259e-07,
+      "loss": 1.6824,
+      "step": 104
+    },
+    {
+      "epoch": 0.3163841807909605,
+      "grad_norm": 5.090848445892334,
+      "learning_rate": 9.401812688821752e-07,
+      "loss": 1.4966,
+      "step": 105
+    },
+    {
+      "epoch": 0.31939736346516007,
+      "grad_norm": 5.832790374755859,
+      "learning_rate": 9.395770392749243e-07,
+      "loss": 1.6723,
+      "step": 106
+    },
+    {
+      "epoch": 0.3224105461393597,
+      "grad_norm": 4.752500534057617,
+      "learning_rate": 9.389728096676737e-07,
+      "loss": 1.5952,
+      "step": 107
+    },
+    {
+      "epoch": 0.3254237288135593,
+      "grad_norm": 4.700122833251953,
+      "learning_rate": 9.383685800604229e-07,
+      "loss": 1.6605,
+      "step": 108
+    },
+    {
+      "epoch": 0.32843691148775894,
+      "grad_norm": 4.912515640258789,
+      "learning_rate": 9.377643504531722e-07,
+      "loss": 1.7009,
+      "step": 109
+    },
+    {
+      "epoch": 0.3314500941619586,
+      "grad_norm": 4.371939182281494,
+      "learning_rate": 9.371601208459214e-07,
+      "loss": 1.6983,
+      "step": 110
+    },
+    {
+      "epoch": 0.3344632768361582,
+      "grad_norm": 4.993189811706543,
+      "learning_rate": 9.365558912386706e-07,
+      "loss": 1.6657,
+      "step": 111
+    },
+    {
+      "epoch": 0.3374764595103578,
+      "grad_norm": 5.17345666885376,
+      "learning_rate": 9.3595166163142e-07,
+      "loss": 1.6463,
+      "step": 112
+    },
+    {
+      "epoch": 0.34048964218455746,
+      "grad_norm": 4.0338053703308105,
+      "learning_rate": 9.353474320241691e-07,
+      "loss": 1.6468,
+      "step": 113
+    },
+    {
+      "epoch": 0.34350282485875705,
+      "grad_norm": 4.444165229797363,
+      "learning_rate": 9.347432024169184e-07,
+      "loss": 1.6073,
+      "step": 114
+    },
+    {
+      "epoch": 0.3465160075329567,
+      "grad_norm": 5.414663791656494,
+      "learning_rate": 9.341389728096676e-07,
+      "loss": 1.751,
+      "step": 115
+    },
+    {
+      "epoch": 0.3495291902071563,
+      "grad_norm": 4.576066017150879,
+      "learning_rate": 9.335347432024169e-07,
+      "loss": 1.6533,
+      "step": 116
+    },
+    {
+      "epoch": 0.3525423728813559,
+      "grad_norm": 4.185419082641602,
+      "learning_rate": 9.329305135951662e-07,
+      "loss": 1.6802,
+      "step": 117
+    },
+    {
+      "epoch": 0.35555555555555557,
+      "grad_norm": 5.01163911819458,
+      "learning_rate": 9.323262839879154e-07,
+      "loss": 1.6024,
+      "step": 118
+    },
+    {
+      "epoch": 0.35856873822975516,
+      "grad_norm": 5.191035270690918,
+      "learning_rate": 9.317220543806646e-07,
+      "loss": 1.7589,
+      "step": 119
+    },
+    {
+      "epoch": 0.3615819209039548,
+      "grad_norm": 5.3965959548950195,
+      "learning_rate": 9.311178247734139e-07,
+      "loss": 1.5132,
+      "step": 120
+    },
+    {
+      "epoch": 0.36459510357815444,
+      "grad_norm": 5.428668022155762,
+      "learning_rate": 9.305135951661631e-07,
+      "loss": 1.8154,
+      "step": 121
+    },
+    {
+      "epoch": 0.36760828625235403,
+      "grad_norm": 4.524122714996338,
+      "learning_rate": 9.299093655589123e-07,
+      "loss": 1.6294,
+      "step": 122
+    },
+    {
+      "epoch": 0.3706214689265537,
+      "grad_norm": 7.043918132781982,
+      "learning_rate": 9.293051359516616e-07,
+      "loss": 1.6754,
+      "step": 123
+    },
+    {
+      "epoch": 0.3736346516007533,
+      "grad_norm": 5.170762538909912,
+      "learning_rate": 9.287009063444108e-07,
+      "loss": 1.6978,
+      "step": 124
+    },
+    {
+      "epoch": 0.3766478342749529,
+      "grad_norm": 4.266485691070557,
+      "learning_rate": 9.2809667673716e-07,
+      "loss": 1.6359,
+      "step": 125
+    },
+    {
+      "epoch": 0.37966101694915255,
+      "grad_norm": 5.6969685554504395,
+      "learning_rate": 9.274924471299094e-07,
+      "loss": 1.6604,
+      "step": 126
+    },
+    {
+      "epoch": 0.38267419962335214,
+      "grad_norm": 5.008423328399658,
+      "learning_rate": 9.268882175226585e-07,
+      "loss": 1.7061,
+      "step": 127
+    },
+    {
+      "epoch": 0.3856873822975518,
+      "grad_norm": 7.691641330718994,
+      "learning_rate": 9.262839879154078e-07,
+      "loss": 1.627,
+      "step": 128
+    },
+    {
+      "epoch": 0.3887005649717514,
+      "grad_norm": 5.625863552093506,
+      "learning_rate": 9.256797583081571e-07,
+      "loss": 1.5484,
+      "step": 129
+    },
+    {
+      "epoch": 0.391713747645951,
+      "grad_norm": 6.434806823730469,
+      "learning_rate": 9.250755287009063e-07,
+      "loss": 1.7078,
+      "step": 130
+    },
+    {
+      "epoch": 0.39472693032015066,
+      "grad_norm": 6.211691856384277,
+      "learning_rate": 9.244712990936556e-07,
+      "loss": 1.693,
+      "step": 131
+    },
+    {
+      "epoch": 0.3977401129943503,
+      "grad_norm": 5.0158467292785645,
+      "learning_rate": 9.238670694864048e-07,
+      "loss": 1.6616,
+      "step": 132
+    },
+    {
+      "epoch": 0.4007532956685499,
+      "grad_norm": 5.5790114402771,
+      "learning_rate": 9.23262839879154e-07,
+      "loss": 1.5867,
+      "step": 133
+    },
+    {
+      "epoch": 0.40376647834274954,
+      "grad_norm": 5.472513198852539,
+      "learning_rate": 9.226586102719033e-07,
+      "loss": 1.5356,
+      "step": 134
+    },
+    {
+      "epoch": 0.4067796610169492,
+      "grad_norm": 5.892872333526611,
+      "learning_rate": 9.220543806646526e-07,
+      "loss": 1.5713,
+      "step": 135
+    },
+    {
+      "epoch": 0.40979284369114877,
+      "grad_norm": 5.382603645324707,
+      "learning_rate": 9.214501510574017e-07,
+      "loss": 1.5487,
+      "step": 136
+    },
+    {
+      "epoch": 0.4128060263653484,
+      "grad_norm": 5.466237545013428,
+      "learning_rate": 9.208459214501511e-07,
+      "loss": 1.6959,
+      "step": 137
+    },
+    {
+      "epoch": 0.415819209039548,
+      "grad_norm": 6.329081058502197,
+      "learning_rate": 9.202416918429002e-07,
+      "loss": 1.5138,
+      "step": 138
+    },
+    {
+      "epoch": 0.41883239171374764,
+      "grad_norm": 4.847983360290527,
+      "learning_rate": 9.196374622356495e-07,
+      "loss": 1.5952,
+      "step": 139
+    },
+    {
+      "epoch": 0.4218455743879473,
+      "grad_norm": 5.410863876342773,
+      "learning_rate": 9.190332326283987e-07,
+      "loss": 1.5666,
+      "step": 140
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 1660,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 5,
+  "save_steps": 10,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 4.358783882955571e+16,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-140/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9a796d8ca70a9c521b458bc0a0be1a83738727813bac47112049bacec3b51cd8
+size 6417