gatilin/deepseek_vit_412m.deepseek_v4_1_flash

DeepSeek ViT 412M — V4.1 Flash

Standalone vision backbone extracted from DeepSeek-V4.1-Flash, converted to native timm format without additional training. No language-model weights or trained classification head are included.

The conversion follows timm/deepseek_vit_412m.deepseek_v4_1_flash and the official timm checkpoint mapping. The weights in this repository come from the V4.1-Flash release, pinned to revision 314113f8a7dadb388215396ba5fbee2c610a04d6.

Model details

• Backbone: 32 layers, hidden size 1024, 16 attention heads, MLP intermediate size 2816.
• Patch size: 14 × 14; SwiGLU MLP, RMSNorm and axial 2D RoPE.
• Backbone parameters: 411,842,560; stored in BF16.
• Default input: 546 × 546 RGB. Rectangular inputs with dimensions divisible by 14 are supported.
• forward_features() returns final-normalized NHWC patch features.
• forward() returns average-pooled, affine-free RMS-normalized 1024-dimensional embeddings.
• The original aligner groups 3 × 3 patches and projects to 5120 dimensions. It is retained only in the separately provided original-format checkpoint.

Usage

pip install 'timm==1.0.30' torch torchvision safetensors pillow

import torch
import timm
from PIL import Image

model = timm.create_model('hf-hub:gatilin/deepseek_vit_412m.deepseek_v4_1_flash', pretrained=True).eval()
config = timm.data.resolve_model_data_config(model)
transform = timm.data.create_transform(**config, is_training=False)
x = transform(Image.open('image.jpg').convert('RGB')).unsqueeze(0)

with torch.inference_mode():
    embedding = model(x)                  # [1, 1024]
    features = model.forward_features(x)  # [1, 39, 39, 1024]

The default FP32 model loads the BF16 checkpoint into FP32 parameters. For BF16 inference, explicitly move both model and input to a device that supports the required operations and the same dtype.

Files

• model.safetensors: native timm classifier-wrapper checkpoint, without aligner or classification head (~824 MB).
• config.json: timm Hub configuration.
• vision_and_aligner.safetensors: original vision.* and aligner.* keys (~971 MB); this is not a second timm checkpoint.
• vision.py, image_processor.py, source_config.json, LICENSE: original source files from the pinned upstream revision.
• extract_vit.py: reproducible extraction with validated HTTP range requests; downloads only visual tensors from source shard 1.
• validate_vision.py: strict-loading and numerical-equivalence check against the original ViT.
• smoke_vision.py: default-resolution 546 × 546 forward check.
• extraction.json: provenance, parameter count and SHA256 checksums.
• validation.json: validation results and library versions.

To load original-format weights, instantiate ViT and Aligner from vision.py using the source configuration; load the corresponding keys after removing vision. or aligner.. These modules consume patch tensors and patch-grid dimensions rather than raw image batches.

Conversion and validation

The conversion renames attention/MLP keys, replaces the vision. prefix with encoder., and reshapes the linear patch projection from [1024, 588] to [1024, 3, 14, 14]. Tensor data bytes and BF16 precision are preserved.

Strict key/shape loading passed for both backbone and aligner with timm 1.0.30 and torch 2.14.1. FP32 comparisons against the original implementation covered 28 × 28, 28 × 42, 126 × 126 and 126 × 140 inputs, including rectangular grids and incomplete 3 × 3 aligner groups. Maximum absolute errors across all cases were 5.22e-08 for backbone features, 5.96e-07 for aligner tokens, and 4.77e-06 for pooled embeddings. See validation.json for individual results. A default-resolution 546 × 546 FP32 forward check also passed: features [1,39,39,1024], embedding [1,1024], all values finite. These conversion checks are not downstream accuracy benchmarks.

Preprocessing and limitations

RGB normalization uses mean and standard deviation (0.5, 0.5, 0.5). The timm evaluation transform uses bicubic resize, border padding, and a fixed canvas. The upstream processor uses an adaptive canvas with gray 127 padding; timm uses gray 128. Use the included upstream processor when reproducing the original preprocessing policy.

This is an image feature model. A newly added classification head requires training. The independent backbone does not provide the language-model capabilities of the source multimodal model.

中文说明

本仓库从 DeepSeek-V4.1-Flash 的固定版本重新提取视觉权重，提供可直接通过 timm 加载的独立 ViT，以及保留原始键名的 ViT + Aligner 权重。没有额外训练，没有语言模型权重。主文件为 BF16、411.84M 参数；转换仅涉及键名映射和 patch projection 形状重排。已通过严格加载与四种尺寸输入的数值一致性验证。默认 timm 预处理与上游动态画布策略存在差异，详见上文。

License

MIT, following the included upstream LICENSE. This is an independently extracted checkpoint; no affiliation with or endorsement by DeepSeek-AI or the timm maintainers is implied.

Other requested sources inspected

The published checkpoints for DeepSeek-V4-Flash-0731 (revision 7872f01b1d1fe23eabc4c98b48bffcef5a386062) and DeepSeek-V4-Pro (revision b5968e9190ef611bbf34a7229255be88a0e937c1) have no vision configuration, vision implementation files, or vision/aligner entries in their complete weight indexes (72,317 and 145,116 entries respectively). No ViT checkpoint can be extracted from these published versions. See source_inspection.json for the inspection record.