Update README.md
Browse files
README.md
CHANGED
|
@@ -195,7 +195,7 @@ Stage 3: Reinforcement Learning
|
|
| 195 |
|
| 196 |
NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 model is a result of the above work.
|
| 197 |
|
| 198 |
-
The end-to-end training recipe is available in the [NVIDIA Nemotron Developer Repository](https://github.com/NVIDIA-NeMo/Nemotron). Evaluation results can be replicated using the [NeMo Evaluator SDK](https://github.com/NVIDIA-NeMo/Evaluator). More details on the datasets and synthetic data generation methods can be found in the technical report [NVIDIA Nemotron Nano
|
| 199 |
|
| 200 |
## Input
|
| 201 |
|
|
@@ -400,7 +400,7 @@ For all domains, we apply a unified data filtering pipeline to ensure that only
|
|
| 400 |
|
| 401 |
Alongside the model, we release our final [pre-training](https://huggingface.co/collections/nvidia/nemotron-pre-training-datasets) and [post-training](https://huggingface.co/collections/nvidia/nemotron-post-training-v3) data, as outlined in this section. For ease of analysis, there is a sample set that is ungated. For all remaining code, math and multilingual data, gating and approval is required, and the dataset is permissively licensed for model training purposes.
|
| 402 |
|
| 403 |
-
More details on the datasets and synthetic data generation methods can be found in the technical report [NVIDIA Nemotron Nano
|
| 404 |
|
| 405 |
| Dataset | Collection Period |
|
| 406 |
| :---- | :---- |
|
|
@@ -588,7 +588,7 @@ The GitHub Crawl was collected using the GitHub REST API and the Amazon S3 API.
|
|
| 588 |
|
| 589 |
## Training Dataset
|
| 590 |
|
| 591 |
-
| Dataset | \# of Tokens in Nemotron Nano 2 | \# of Tokens in Nemotron Nano
|
| 592 |
| :---- | :---- | :---- |
|
| 593 |
| English Common Crawl | 3,360,110,334,818 | 3,456,523,212,210 |
|
| 594 |
| English Synthetic CC | 1,949,464,641,123 | 4,340,740,677,920 |
|
|
|
|
| 195 |
|
| 196 |
NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 model is a result of the above work.
|
| 197 |
|
| 198 |
+
The end-to-end training recipe is available in the [NVIDIA Nemotron Developer Repository](https://github.com/NVIDIA-NeMo/Nemotron). Evaluation results can be replicated using the [NeMo Evaluator SDK](https://github.com/NVIDIA-NeMo/Evaluator). More details on the datasets and synthetic data generation methods can be found in the technical report [NVIDIA Nemotron 3 Nano](https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3-Nano-Technical-Report.pdf).
|
| 199 |
|
| 200 |
## Input
|
| 201 |
|
|
|
|
| 400 |
|
| 401 |
Alongside the model, we release our final [pre-training](https://huggingface.co/collections/nvidia/nemotron-pre-training-datasets) and [post-training](https://huggingface.co/collections/nvidia/nemotron-post-training-v3) data, as outlined in this section. For ease of analysis, there is a sample set that is ungated. For all remaining code, math and multilingual data, gating and approval is required, and the dataset is permissively licensed for model training purposes.
|
| 402 |
|
| 403 |
+
More details on the datasets and synthetic data generation methods can be found in the technical report [NVIDIA Nemotron 3 Nano](https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3-Nano-Technical-Report.pdf).
|
| 404 |
|
| 405 |
| Dataset | Collection Period |
|
| 406 |
| :---- | :---- |
|
|
|
|
| 588 |
|
| 589 |
## Training Dataset
|
| 590 |
|
| 591 |
+
| Dataset | \# of Tokens in Nemotron Nano 2 | \# of Tokens in Nemotron 3 Nano |
|
| 592 |
| :---- | :---- | :---- |
|
| 593 |
| English Common Crawl | 3,360,110,334,818 | 3,456,523,212,210 |
|
| 594 |
| English Synthetic CC | 1,949,464,641,123 | 4,340,740,677,920 |
|