BLEACH-lc MengqiLei commited on
Commit
9b7acb4
·
0 Parent(s):

Duplicate from MengqiLei/count-anything

Browse files

Co-authored-by: Mengqi Lei <[email protected]>

.gitattributes ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ assets/readme/count_anything_poster.jpg filter=lfs diff=lfs merge=lfs -text
37
+ assets/readme/main_results_cloc_table.png filter=lfs diff=lfs merge=lfs -text
38
+ assets/paper/Count-Anything.pdf filter=lfs diff=lfs merge=lfs -text
39
+ assets/readme/main_results_cloc_table.jpg filter=lfs diff=lfs merge=lfs -text
40
+ assets/readme/count_anything_visualizations.jpg filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,325 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: pytorch
4
+ tags:
5
+ - computer-vision
6
+ - object-counting
7
+ - text-guided-counting
8
+ - open-world-counting
9
+ - count-anything
10
+ - object-detection
11
+ - zero-shot
12
+ - open-vocabulary
13
+ ---
14
+
15
+ <h1 align="center">Count Anything</h1>
16
+
17
+ <p align="center">
18
+ <strong>Text-guided object counting across domains with instance-grounded point predictions</strong>
19
+ </p>
20
+
21
+ <p align="center">
22
+ <a href="https://mengqi-lei.github.io/count-anything-projectpage/"><img src="https://img.shields.io/badge/Project-Page-purple" alt="Project Page"></a>
23
+ <a href="https://huggingface.co/MengqiLei/count-anything"><img src="https://img.shields.io/badge/Hugging%20Face-Model-yellow" alt="Hugging Face Model"></a>
24
+ <a href="https://huggingface.co/spaces/MengqiLei/count-anything-demo"><img src="https://img.shields.io/badge/Hugging%20Face-Demo-orange" alt="Hugging Face Demo"></a>
25
+ <a href="https://arxiv.org/abs/2605.30846"><img src="https://img.shields.io/badge/arXiv-2605.30846-b31b1b" alt="arXiv Paper"></a>
26
+ <a href="assets/paper/Count-Anything.pdf"><img src="https://img.shields.io/badge/Paper-PDF-blue" alt="Paper PDF"></a>
27
+ <a href="mailto:[email protected]"><img src="https://img.shields.io/badge/Contact-Email-green" alt="Contact Email"></a>
28
+ </p>
29
+
30
+ <p align="center">
31
+ <img src="assets/readme/count_anything_poster.jpg" alt="Count Anything poster" width="100%">
32
+ </p>
33
+
34
+ <p align="center">
35
+ <strong>✨Play with the demo: <a href="https://huggingface.co/spaces/MengqiLei/count-anything-demo">🔗Huggingface Demo</a></strong>
36
+ </p>
37
+
38
+ ## Overview
39
+
40
+ This repository introduces **Count Anything**, a generalist model for text-guided object counting across domains. Given an image and a natural-language query, Count Anything returns an instance-grounded set of target points whose cardinality gives the count. This formulation unifies category-conditioned counting with interpretable spatial localization.
41
+
42
+ ### Cross-domain Text-guided Counting
43
+
44
+ - Study text-guided object counting across domains, where users specify the target with a category name or a natural-language query.
45
+ - Construct **CLOC**, a Cross-domain Large-scale Object Counting dataset that reorganizes diverse public data sources into a unified counting benchmark.
46
+ - Cover six visual domains: General Scene, Remote Sensing, Histopathology, Cellular Microscopy, Agriculture, and Microbiology.
47
+
48
+ ### Dual-granularity Instance Enumeration
49
+
50
+ - Adopt discrete instance points as the final prediction form, rather than using density maps as the final output.
51
+ - Use a **Region-level Sparse Counter (RSC)** to provide object-level anchoring for large and sparse targets.
52
+ - Use a **Pixel-level Dense Counter (PDC)** to capture small, crowded, and weakly bounded targets through dense point prediction.
53
+
54
+ ### Point-centric Supervision and Complementary Fusion
55
+
56
+ - Convert heterogeneous annotations, including boxes, points, polygons, masks, rotated boxes, and label maps, into counting points with optional boxes.
57
+ - Use point-centric supervision so every valid instance is supervised by a point, while bounding boxes are used only when reliable box annotations exist.
58
+ - Combine RSC and PDC with **Complementary Count Fusion (CCF)** in a parameter-free manner, suppressing duplicate counts while preserving their complementarity.
59
+
60
+ Count Anything is trained and evaluated on CLOC, which contains about 220K images, 619 categories, and 15M object instances. Extensive experiments show that Count Anything achieves strong counting accuracy and multi-domain generalization, substantially outperforming existing open-world counting methods.
61
+
62
+ ## Main Results
63
+
64
+ ![Main CLOC comparison table from the paper](assets/readme/main_results_cloc_table.jpg)
65
+
66
+ ## Visualizations
67
+
68
+ <p align="center">
69
+ <a href="assets/readme/count_anything_visualizations.jpg">
70
+ <img src="assets/readme/count_anything_visualizations.jpg" alt="Count Anything qualitative visualizations" width="100%">
71
+ </a>
72
+ </p>
73
+
74
+ ## Quick Start
75
+
76
+ ### 1. Environment Setup
77
+
78
+ Create a conda environment and install the required dependencies:
79
+
80
+ ```bash
81
+ conda create -n countanything python=3.12 -y
82
+ conda activate countanything
83
+ pip install -r requirements.txt
84
+ ```
85
+
86
+ The dependency list is intentionally kept minimal. If the default pip resolver does not select the CUDA build you need, please install the PyTorch and torchvision builds that match your local CUDA version.
87
+
88
+ ### 2. Weight Preparation
89
+
90
+ For inference, validation, or test-only reproduction, only the released CountAnything checkpoint is required. Download `count_anything.pt` from [Hugging Face](https://huggingface.co/MengqiLei/count-anything).
91
+
92
+ After downloading, place the file at:
93
+
94
+ ```text
95
+ checkpoints/count_anything.pt
96
+ ```
97
+
98
+ The standalone validation and test configurations load `checkpoints/count_anything.pt` directly.
99
+
100
+ If you want to train or fine-tune CountAnything from the SAM3 initialization, please also download the official SAM3 pretrained weights. Due to license and redistribution restrictions, this repository does not directly provide the official SAM3 pretrained weights. Please visit the official SAM3 Hugging Face page:
101
+
102
+ [Hugging Face](https://huggingface.co/facebook/sam3)
103
+
104
+ Download the SAM3 pretrained weight file `sam3.pt`, and place it at:
105
+
106
+ ```text
107
+ pretrained/sam3.pt
108
+ ```
109
+
110
+ Please make sure to download the **SAM3** weights, not SAM3.1 weights. The expected file name is:
111
+
112
+ ```text
113
+ sam3.pt
114
+ ```
115
+
116
+ By default, the training configuration initializes the model from `pretrained/sam3.pt`.
117
+
118
+ ### 3. Data Preparation
119
+
120
+ This repository uses the CLOC dataset by default. The dataset preparation guide explains how to download the CLOC annotation archive, the distributable augmented-image archive, and the raw images of each source dataset. Please prepare the dataset following that guide before running training or evaluation.
121
+
122
+ 💡 **Dataset availability note.** Because some source datasets used by CLOC are subject to license and redistribution restrictions, we cannot directly release the complete CLOC image set and can only reference those data sources. We release the CLOC annotation files produced by our re-annotation process and the subset of augmented images that can be redistributed. To reproduce the complete CLOC dataset, please download the original source images and follow the preprocessing, rebuilding, and audit steps in [data/README.md](https://github.com/Mengqi-Lei/count-anything/blob/main/data/README.md). **We can also provide a processed, ready-to-use copy of the complete CLOC dataset upon request; please contact us by email for access.**
123
+
124
+ The default configurations expect the train, validation, and test annotations at:
125
+
126
+ ```text
127
+ data/annotations/train_split_expanded_by_class.json
128
+ data/annotations/val_split_expanded_by_class.json
129
+ data/annotations/test_split_expanded_by_class.json
130
+ ```
131
+
132
+ Each sample corresponds to one image-category counting task. The annotation files provide the image path, category text, and the point / bbox annotations for the corresponding category. The actual image location is specified by the `image_path` field in each annotation record.
133
+
134
+ The dataset directory is organized as follows:
135
+
136
+ ```text
137
+ data/
138
+ annotations/ # CLOC train/val/test JSON files
139
+ images/ # Raw dataset download and extraction directories
140
+ augmented/ # Augmented images referenced by CLOC annotations
141
+ tools/ # Data conversion, augmented-image rebuilding, and audit scripts
142
+ README.md # English dataset preparation guide
143
+ ```
144
+
145
+ For the full dataset construction workflow, including raw dataset downloads, format conversion, augmented-image rebuilding, and path auditing, please refer to [data/README.md](https://github.com/Mengqi-Lei/count-anything/blob/main/data/README.md).
146
+
147
+ If your dataset paths differ from the default layout, please edit:
148
+
149
+ ```text
150
+ config/count_anything_train_cloc.yaml
151
+ config/count_anything_val_cloc.yaml
152
+ config/count_anything_test_cloc.yaml
153
+ ```
154
+
155
+ and update `paths.train_annotation_file` and `paths.val_annotation_file` so they point to your local annotation files.
156
+
157
+ ### 4. Training
158
+
159
+ Start training with `train.sh`:
160
+
161
+ ```bash
162
+ CUDA_VISIBLE_DEVICES=0,1,2,3 \
163
+ NUM_GPUS=4 \
164
+ bash train.sh
165
+ ```
166
+
167
+ By default, `train.sh` uses:
168
+
169
+ ```text
170
+ config/count_anything_train_cloc.yaml
171
+ ```
172
+
173
+ This configuration corresponds to the main model setting in the paper: it initializes from SAM3 pretrained weights, enables RSC, PDC, and CCF, and trains the LoRA parameters together with the counting branch. The LoRA learning rate is `1e-3`; the learning rate follows a 30-epoch cosine schedule with `min_lr_ratio=0.1`.
174
+
175
+ Validation after each epoch uses:
176
+
177
+ ```text
178
+ data/annotations/val_split_expanded_by_class.json
179
+ ```
180
+
181
+ The default training parameters include:
182
+
183
+ - `train_batch_size=18`
184
+ - `val_batch_size=40`
185
+ - `max_epochs=30`
186
+ - `val_epoch_freq=1`
187
+ - `visualize_val_every_n_epochs=5`
188
+
189
+ After training starts, logs, visualizations, and checkpoints are saved by default to:
190
+
191
+ ```text
192
+ exp/count_anything_train_cloc/
193
+ ```
194
+
195
+ To change data paths, batch size, number of epochs, or the output directory, edit:
196
+
197
+ ```text
198
+ config/count_anything_train_cloc.yaml
199
+ ```
200
+
201
+ ### 5. Validation
202
+
203
+ Run standalone validation with `val.sh`:
204
+
205
+ ```bash
206
+ CUDA_VISIBLE_DEVICES=0,1,2,3 \
207
+ NUM_GPUS=4 \
208
+ bash val.sh
209
+ ```
210
+
211
+ By default, `val.sh` uses:
212
+
213
+ ```text
214
+ config/count_anything_val_cloc.yaml
215
+ ```
216
+
217
+ This configuration loads:
218
+
219
+ ```text
220
+ checkpoints/count_anything.pt
221
+ ```
222
+
223
+ and evaluates on the CLOC validation split:
224
+
225
+ ```text
226
+ data/annotations/val_split_expanded_by_class.json
227
+ ```
228
+
229
+ Validation logs and prediction statistics are saved by default to:
230
+
231
+ ```text
232
+ exp/count_anything_val_cloc/
233
+ ```
234
+
235
+ To validate another checkpoint or another validation set, edit:
236
+
237
+ ```text
238
+ config/count_anything_val_cloc.yaml
239
+ ```
240
+
241
+ and update the checkpoint path and `paths.val_annotation_file`.
242
+
243
+ ### 6. Testing
244
+
245
+ Evaluate a checkpoint with `test.sh`:
246
+
247
+ ```bash
248
+ CUDA_VISIBLE_DEVICES=0,1,2,3 \
249
+ NUM_GPUS=4 \
250
+ bash test.sh
251
+ ```
252
+
253
+ By default, `test.sh` uses:
254
+
255
+ ```text
256
+ config/count_anything_test_cloc.yaml
257
+ ```
258
+
259
+ This configuration loads:
260
+
261
+ ```text
262
+ checkpoints/count_anything.pt
263
+ ```
264
+
265
+ and evaluates on the CLOC test split:
266
+
267
+ ```text
268
+ data/annotations/test_split_expanded_by_class.json
269
+ ```
270
+
271
+ Evaluation logs and prediction statistics are saved by default to:
272
+
273
+ ```text
274
+ exp/count_anything_test_cloc/
275
+ ```
276
+
277
+ To evaluate another checkpoint or another test set, edit:
278
+
279
+ ```text
280
+ config/count_anything_test_cloc.yaml
281
+ ```
282
+
283
+ and update the checkpoint path and `paths.val_annotation_file`.
284
+
285
+ ## Repository Structure
286
+
287
+ ```text
288
+ CountAnything/
289
+ train.sh # Default training entry point
290
+ val.sh # Default validation entry point
291
+ test.sh # Default test entry point
292
+ requirements.txt # Python dependencies
293
+ config/
294
+ count_anything_train_cloc.yaml # CLOC training configuration
295
+ count_anything_val_cloc.yaml # CLOC validation configuration
296
+ count_anything_test_cloc.yaml # CLOC test configuration
297
+ count_anything/
298
+ model/ # CountAnything model components
299
+ train/ # Trainer, losses, and matcher
300
+ eval/ # Post-processing and counting evaluation
301
+ sam3/ # SAM3-based image-language backbone components
302
+ pretrained/
303
+ sam3.pt # Place the SAM3 pretrained weights here
304
+ checkpoints/
305
+ count_anything.pt # Place the CountAnything checkpoint here
306
+ data/ # CLOC dataset annotations and preparation tools
307
+ exp/ # Training and evaluation outputs
308
+ ```
309
+
310
+ ## Questions and Support
311
+
312
+ If you encounter any difficulty with dataset preparation, model weights, training, validation, or evaluation, please feel free to contact us and we will do our best to help.
313
+
314
+ ## Citation
315
+
316
+ If you find Count Anything useful in your research, please consider citing:
317
+
318
+ ```bibtex
319
+ @article{lei2026count_anything,
320
+ title={Count Anything},
321
+ author={Lei, Mengqi and Cheng, Shuokun and Bao, Wei and Du, Shaoyi and Yong, Jun-Hai and Li, Siqi and Gao, Yue},
322
+ journal={arXiv preprint arXiv:2605.30846},
323
+ year={2026}
324
+ }
325
+ ```
assets/paper/Count-Anything.pdf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9f29856976bb7a7ead3272dda6e4534f3cc69157b2d985646f6c2884c6c22aeb
3
+ size 43395818
assets/readme/count_anything_poster.jpg ADDED

Git LFS Details

  • SHA256: 3e17997460341f5d2b1675fb91951d4509990394a4ea49c765bbed6e99284ee7
  • Pointer size: 132 Bytes
  • Size of remote file: 1.82 MB
assets/readme/count_anything_visualizations.jpg ADDED

Git LFS Details

  • SHA256: 165282db46daad24c60d07e10ae428ae2286a1789073501942393382bb0a24cf
  • Pointer size: 133 Bytes
  • Size of remote file: 12.9 MB
assets/readme/main_results_cloc_table.jpg ADDED

Git LFS Details

  • SHA256: da3bb7bfcdca7d06df1753977b16d7697ad758df671bbf01de114bb7c467023d
  • Pointer size: 131 Bytes
  • Size of remote file: 570 kB
count_anything.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2f84be3c3af5508388b732827e725cb6d6c4dd826ffda7f80de234ebd1ef0f92
3
+ size 3565313633