Add Qwen2.5-VL-7B CLT transcoders with LFS

README.md

@@ -0,0 +1,63 @@
+# Qwen2.5-VL-7B Circuit-Level Transcoders (CLT)
+
+This repository contains Circuit-Level Transcoders (CLTs) for Qwen2.5-VL-7B, trained with TopK sparsity (12% L0 sparsity).
+
+## Model Details
+
+- **Base Model**: Qwen2.5-VL-7B
+- **Type**: Circuit-Level Transcoders (CLT)
+- **Layers**: 27 transcoders (L0-L26)
+- **Sparsity**: 12% L0 (TopK)
+- **File Size**: ~113MB per layer (~3GB total)
+- **Training**: Each layer trained for 5000 steps
+
+## Files
+
+This repository contains 27 transcoder checkpoint files:
+- `transcoder_L0.pt` through `transcoder_L26.pt`
+
+Each file contains the trained transcoder weights for the corresponding layer.
+
+## Usage
+
+```python
+import torch
+
+# Load a specific layer transcoder
+layer_num = 12
+transcoder = torch.load(f'transcoder_L{layer_num}.pt')
+
+# The transcoder can be used for circuit discovery and suppression
+# in conjunction with the Qwen2.5-VL-7B model
+```
+
+## Training Details
+
+- **Optimizer**: AdamW
+- **Training Steps**: 5000 per layer
+- **Sparsity Target**: 12% L0 (TopK)
+- **Validation Frequency**: Every 200 steps
+- **Training Time**: ~5 minutes per layer
+
+## Related Models
+
+- [KokosDev/qwen2p5vl-7b-plt](https://huggingface.co/KokosDev/qwen2p5vl-7b-plt) - Paired Linear Transcoders (PLT) for the same base model
+
+## Citation
+
+If you use these transcoders in your research, please cite:
+
+```bibtex
+@misc{qwen2p5vl-7b-clt,
+  title={Circuit-Level Transcoders for Qwen2.5-VL-7B},
+  author={KokosDev},
+  year={2025},
+  publisher={HuggingFace},
+  howpublished={\url{https://huggingface.co/KokosDev/qwen2p5vl-7b-clt}}
+}
+```
+
+## License
+
+Please refer to the base model's license: [Qwen2.5-VL](https://huggingface.co/Qwen/Qwen2.5-VL-7B)
+

transcoder_L0.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9a523426643bfb7b1ba5fd4e315035947a3124975ab81b17b078cf21e3623e91
+size 117481949

transcoder_L1.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:011da45eec676ccc6ec499a7646e4e9dd33f63d0b30f093a1c2620734a8820a9
+size 117481949

transcoder_L10.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:674f155aa2998527432c8fefa80a7d4d1d46441e6faf79e6d9b41088bcd8e116
+size 117481961

transcoder_L11.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a9dda5894955dbdf4b4ae15ceb15221466024da7d5bda29c176c6a2c896e45e0
+size 117481961

transcoder_L12.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:14fd40ba52d752cc0a39ef8dea39899851dba8bc6642a73caa8d35b32f71c442
+size 117481961

transcoder_L13.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f4f1e16d42611571560f84c1b40bc9c0ff63f187cfc05732ba08bbf14b71c9a7
+size 117481961

transcoder_L14.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5d5e16211dbfc408cdd1633d7299f8a3261a85d9206df9e730a2a2d2bed2fab9
+size 117481961

transcoder_L15.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:90228d9eabf2c68444ca84bfb2e278352400ccaf735ba7fc04f7c1d1b07c1618
+size 117481961

transcoder_L16.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a2d54b51b86723f2d12bf084e0c3609e35285a87f574324900be07de576176f2
+size 117481961

transcoder_L17.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:af541fa055045c78885ce46bb646838330b1f1e88ced882f9a45032abf5ffdd0
+size 117481961

transcoder_L18.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:deca497b56fb9117f24921229cbc2d13ce6cc1218e8d457ce117fc335cad8e50
+size 117481961

transcoder_L19.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c92fed7707738b43f9c2ce8d430e5e7fb003a4352b380a2e6fb9488ee9ebeb1
+size 117481961

transcoder_L2.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:046eb0ad6499828116ca24ddac56d75b2eafe33231549f3f2536c623bdc6b604
+size 117481949

transcoder_L20.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b4da499648fa770a0890d9161368687179914dcbc9e5c54fb6d5b8e927cd84a3
+size 117481961

transcoder_L21.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d1c4a5e6cbcbb544fb37235c462061ac7a927f6613ede5636f92989f4032b00d
+size 117481961

transcoder_L22.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:187e176450f34386b7239f28b85d8f66f27cd52cb20343ff7ea31edb4189a6ed
+size 117481961

transcoder_L23.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4a5049a899afe59374f6ae01056156f2cf25568c5ce31073ec52094fdc1a0fcc
+size 117481961

transcoder_L24.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d28a88349a9709b25322a47f0a7422fca08c024c1e1d05be69ca928ba129c00
+size 117481961

transcoder_L25.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:33874f1bb13cf164d57203c121843b856e4c428bc74f3fbf95f3e263f61b757d
+size 117481961

transcoder_L26.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bea6d92a8d3c5f18ad95c08c10d759a8e00709097e2337797c49d81069622acc
+size 117481961

transcoder_L3.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:efa4d029e87e4a07ebb1ed9317e6e6661365f9627c5baac6e4f3980dec7630ce
+size 117481949

transcoder_L4.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95956064bcab73f1adb052d2115261d9c5a2ad4b4046931366290841777653bb
+size 117481949

transcoder_L5.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c28c60040be909dd0af62b5685595065b7c296b840b649b0a521975e1ba225e
+size 117481949

transcoder_L6.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d09a2f749fd8a208dc5809c10844ba5c1e278819c9cea162835654cd4853dca
+size 117481949

transcoder_L7.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b6422712ae2b8c45d7d08b6ddc9851140da93899d238b73564194a591232ab8
+size 117481949

transcoder_L8.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7cc127463736aa367af6aa8c6f87497204988e0d6a1fb4ba9fe3a8bdf762482e
+size 117481949

transcoder_L9.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a40edc62d6f928e852411a7f802f73c1fa8d34c9b6d2dd57f81be206001f5717
+size 117481949