Freckles kept up without using hidden states. Next up, hidden states.

omega_vs_baseline_output.txt

@@ -0,0 +1,362 @@
+Note: Environment variable`HF_TOKEN` is set and is the current active token independently from the token you've just configured.
+WARNING:huggingface_hub._login:Note: Environment variable`HF_TOKEN` is set and is the current active token independently from the token you've just configured.
+
+======================================================================
+OMEGA PROCESSOR — CIFAR-10 (Freckles features)
+======================================================================
+  Freckles: 2,557,539 params (frozen)
+  Feature dim: 64
+  Classifier: 836,234 params
+  Architecture: d_model=128, heads=4, layers=4
+  CIFAR-10: 50K train, 10K test, 64×64
+  Batch: 128, lr=0.0003, epochs=30
+======================================================================
+100%|██████████| 170M/170M [00:01<00:00, 91.1MB/s]
+Ep 1/30: 100%|████████████████████| 390/390 [00:10<00:00, 38.21it/s, loss=1.8147 acc=23.6%]
+  ep  1 | loss=2.0322 train=23.6% test=33.4% | best=truck=70% worst=bird=1% | 12s
+
+    class             acc
+    ----------------------
+    airplane       44.1% ████████
+    automobile     30.2% ██████
+    bird            1.5% 
+    cat            22.7% ████
+    deer           10.3% ██
+    dog            34.8% ██████
+    frog           54.1% ██████████
+    horse          36.4% ███████
+    ship           30.6% ██████
+    truck          69.7% █████████████
+
+Ep 2/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.18it/s, loss=1.6721 acc=38.5%]
+  ep  2 | loss=1.6980 train=38.5% test=44.8% | best=automobile=74% worst=cat=16% | 11s
+Ep 3/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.22it/s, loss=1.4097 acc=45.5%]
+  ep  3 | loss=1.5104 train=45.5% test=49.5% | best=frog=70% worst=cat=25% | 11s
+Ep 4/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.03it/s, loss=1.4453 acc=50.1%]
+  ep  4 | loss=1.3840 train=50.1% test=54.4% | best=ship=81% worst=cat=25% | 11s
+Ep 5/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.01it/s, loss=1.0432 acc=54.3%]
+  ep  5 | loss=1.2717 train=54.3% test=56.5% | best=ship=80% worst=bird=37% | 11s
+
+    class             acc
+    ----------------------
+    airplane       67.3% █████████████
+    automobile     72.4% ██████████████
+    bird           36.7% ███████
+    cat            40.3% ████████
+    deer           57.8% ███████████
+    dog            39.3% ███████
+    frog           72.9% ██████████████
+    horse          59.3% ███████████
+    ship           80.0% ████████████████
+    truck          39.3% ███████
+
+Ep 6/30: 100%|████████████████████| 390/390 [00:10<00:00, 38.27it/s, loss=1.1928 acc=57.3%]
+  ep  6 | loss=1.1949 train=57.3% test=59.5% | best=ship=81% worst=dog=44% | 11s
+Ep 7/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.01it/s, loss=1.3602 acc=59.9%]
+  ep  7 | loss=1.1243 train=59.9% test=59.7% | best=truck=78% worst=cat=23% | 11s
+Ep 8/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.22it/s, loss=1.2255 acc=61.8%]
+  ep  8 | loss=1.0698 train=61.8% test=62.1% | best=truck=74% worst=cat=44% | 11s
+Ep 9/30: 100%|████████████████████| 390/390 [00:10<00:00, 38.57it/s, loss=1.2281 acc=63.4%]
+  ep  9 | loss=1.0276 train=63.4% test=64.0% | best=automobile=83% worst=cat=35% | 11s
+Ep 10/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.23it/s, loss=0.9413 acc=64.9%]
+  ep 10 | loss=0.9864 train=64.9% test=65.4% | best=ship=81% worst=bird=46% | 11s
+
+    class             acc
+    ----------------------
+    airplane       63.3% ████████████
+    automobile     76.2% ███████████████
+    bird           46.0% █████████
+    cat            51.0% ██████████
+    deer           60.8% ████████████
+    dog            51.8% ██████████
+    frog           73.0% ██████████████
+    horse          73.2% ██████████████
+    ship           81.5% ████████████████
+    truck          77.1% ███████████████
+
+Ep 11/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.06it/s, loss=0.9747 acc=66.2%]
+  ep 11 | loss=0.9497 train=66.2% test=66.0% | best=truck=81% worst=bird=47% | 11s
+Ep 12/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.04it/s, loss=0.9341 acc=67.2%]
+  ep 12 | loss=0.9211 train=67.2% test=65.3% | best=airplane=82% worst=cat=36% | 11s
+Ep 13/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.17it/s, loss=1.0120 acc=68.2%]
+  ep 13 | loss=0.8938 train=68.2% test=67.7% | best=frog=84% worst=cat=44% | 11s
+Ep 14/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.26it/s, loss=0.8321 acc=69.3%]
+  ep 14 | loss=0.8613 train=69.3% test=68.3% | best=automobile=83% worst=cat=50% | 11s
+Ep 15/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.07it/s, loss=0.8031 acc=70.3%]
+  ep 15 | loss=0.8360 train=70.3% test=68.7% | best=automobile=81% worst=cat=39% | 11s
+
+    class             acc
+    ----------------------
+    airplane       76.6% ███████████████
+    automobile     81.1% ████████████████
+    bird           52.9% ██████████
+    cat            39.1% ███████
+    deer           67.7% █████████████
+    dog            60.5% ████████████
+    frog           76.6% ███████████████
+    horse          75.8% ███████████████
+    ship           78.3% ███████████████
+    truck          78.7% ███████████████
+
+Ep 16/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.20it/s, loss=1.0140 acc=71.0%]
+  ep 16 | loss=0.8155 train=71.0% test=69.5% | best=frog=86% worst=cat=43% | 11s
+Ep 17/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.14it/s, loss=0.6555 acc=72.0%]
+  ep 17 | loss=0.7903 train=72.0% test=69.3% | best=automobile=86% worst=cat=43% | 11s
+Ep 18/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.01it/s, loss=0.6629 acc=72.9%]
+  ep 18 | loss=0.7687 train=72.9% test=70.1% | best=ship=84% worst=cat=53% | 11s
+Ep 19/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.17it/s, loss=0.6098 acc=73.3%]
+  ep 19 | loss=0.7525 train=73.3% test=70.9% | best=automobile=82% worst=cat=47% | 11s
+Ep 20/30: 100%|████████████████████| 390/390 [00:10<00:00, 38.60it/s, loss=0.6530 acc=74.3%]
+  ep 20 | loss=0.7280 train=74.3% test=71.1% | best=airplane=85% worst=cat=52% | 11s
+
+    class             acc
+    ----------------------
+    airplane       84.9% ████████████████
+    automobile     82.0% ████████████████
+    bird           55.6% ███████████
+    cat            52.3% ██████████
+    deer           66.9% █████████████
+    dog            59.3% ███████████
+    frog           78.5% ███████████████
+    horse          73.4% ██████████████
+    ship           79.5% ███████████████
+    truck          78.3% ███████████████
+
+Ep 21/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.05it/s, loss=0.7255 acc=74.7%]
+  ep 21 | loss=0.7183 train=74.7% test=70.8% | best=frog=89% worst=cat=52% | 11s
+Ep 22/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.13it/s, loss=0.7117 acc=75.2%]
+  ep 22 | loss=0.7030 train=75.2% test=71.1% | best=automobile=87% worst=cat=49% | 11s
+Ep 23/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.06it/s, loss=0.6705 acc=75.5%]
+  ep 23 | loss=0.6897 train=75.5% test=71.0% | best=truck=83% worst=cat=47% | 11s
+Ep 24/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.19it/s, loss=0.6390 acc=76.0%]
+  ep 24 | loss=0.6786 train=76.0% test=72.0% | best=automobile=82% worst=cat=50% | 11s
+Ep 25/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.14it/s, loss=0.5925 acc=76.5%]
+  ep 25 | loss=0.6657 train=76.5% test=72.0% | best=automobile=86% worst=cat=52% | 11s
+
+    class             acc
+    ----------------------
+    airplane       81.2% ████████████████
+    automobile     86.2% █████████████████
+    bird           56.9% ███████████
+    cat            52.5% ██████████
+    deer           68.3% █████████████
+    dog            58.5% ███████████
+    frog           80.0% ████████████████
+    horse          76.0% ██████��████████
+    ship           80.9% ████████████████
+    truck          79.1% ███████████████
+
+Ep 26/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.24it/s, loss=0.6332 acc=76.5%]
+  ep 26 | loss=0.6614 train=76.5% test=72.1% | best=frog=84% worst=cat=48% | 11s
+Ep 27/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.14it/s, loss=0.6383 acc=76.9%]
+  ep 27 | loss=0.6535 train=76.9% test=72.2% | best=automobile=86% worst=cat=50% | 11s
+Ep 28/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.15it/s, loss=0.7896 acc=77.2%]
+  ep 28 | loss=0.6451 train=77.2% test=72.6% | best=automobile=84% worst=cat=50% | 11s
+Ep 29/30: 100%|████████████████████| 390/390 [00:09<00:00, 39.19it/s, loss=0.6121 acc=77.4%]
+  ep 29 | loss=0.6410 train=77.4% test=72.7% | best=automobile=84% worst=cat=51% | 11s
+Ep 30/30: 100%|████████████████████| 390/390 [00:10<00:00, 38.99it/s, loss=0.7349 acc=77.5%]
+  ep 30 | loss=0.6390 train=77.5% test=72.7% | best=automobile=84% worst=cat=51% | 11s
+
+    class             acc
+    ----------------------
+    airplane       79.0% ███████████████
+    automobile     83.8% ████████████████
+    bird           59.8% ███████████
+    cat            50.9% ██████████
+    deer           68.4% █████████████
+    dog            63.1% ████████████
+    frog           82.0% ████████████████
+    horse          76.9% ███████████████
+    ship           82.2% ████████████████
+    truck          80.6% ████████████████
+
+
+======================================================================
+OMEGA PROCESSOR COMPLETE
+  Best test accuracy: 72.7%
+  Classifier params: 836,234
+  Random chance: 10.0%
+======================================================================
+
+======================================================================
+BASELINE — CIFAR-10 (Raw patches, no Freckles)
+======================================================================
+  Classifier: 867,050 params
+  Architecture: d_model=128, heads=4, layers=4
+  CIFAR-10: 50K train, 10K test, 64×64
+  Batch: 128, lr=0.0003, epochs=30
+======================================================================
+Ep 1/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.28it/s, loss=1.6722 acc=32.2%]
+  ep  1 | loss=1.8088 train=32.2% test=42.3% | best=ship=62% worst=deer=18% | 6s
+
+    class             acc
+    ----------------------
+    airplane       47.6% █████████
+    automobile     59.5% ███████████
+    bird           32.3% ██████
+    cat            21.7% ████
+    deer           17.9% ███
+    dog            32.4% ██████
+    frog           61.7% ████████████
+    horse          41.3% ████████
+    ship           62.4% ████████████
+    truck          46.5% █████████
+
+Ep 2/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.53it/s, loss=1.3286 acc=46.4%]
+  ep  2 | loss=1.4489 train=46.4% test=51.9% | best=frog=66% worst=cat=20% | 6s
+Ep 3/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.29it/s, loss=1.2655 acc=53.2%]
+  ep  3 | loss=1.2781 train=53.2% test=56.3% | best=ship=75% worst=cat=28% | 6s
+Ep 4/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.42it/s, loss=1.1440 acc=57.9%]
+  ep  4 | loss=1.1618 train=57.9% test=60.5% | best=frog=82% worst=bird=34% | 6s
+Ep 5/30: 100%|████████████████████| 390/390 [00:06<00:00, 64.95it/s, loss=1.1496 acc=61.4%]
+  ep  5 | loss=1.0723 train=61.4% test=63.2% | best=horse=79% worst=cat=31% | 6s
+
+    class             acc
+    ----------------------
+    airplane       69.9% █████████████
+    automobile     76.8% ███████████████
+    bird           38.6% ███████
+    cat            30.7% ██████
+    deer           56.0% ███████████
+    dog            62.7% ████████████
+    frog           72.4% ██████████████
+    horse          78.7% ███████████████
+    ship           74.0% ██████████████
+    truck          72.2% ██████████████
+
+Ep 6/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.25it/s, loss=0.8846 acc=64.0%]
+  ep  6 | loss=1.0052 train=64.0% test=62.7% | best=frog=89% worst=bird=32% | 6s
+Ep 7/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.29it/s, loss=0.9618 acc=66.5%]
+  ep  7 | loss=0.9418 train=66.5% test=65.2% | best=truck=87% worst=cat=32% | 6s
+Ep 8/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.03it/s, loss=0.9562 acc=68.3%]
+  ep  8 | loss=0.8913 train=68.3% test=68.0% | best=ship=87% worst=cat=45% | 6s
+Ep 9/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.12it/s, loss=0.8684 acc=69.9%]
+  ep  9 | loss=0.8519 train=69.9% test=68.6% | best=truck=85% worst=cat=40% | 6s
+Ep 10/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.12it/s, loss=0.6724 acc=71.3%]
+  ep 10 | loss=0.8107 train=71.3% test=68.6% | best=ship=84% worst=dog=46% | 6s
+
+    class             acc
+    ----------------------
+    airplane       76.5% ███████████████
+    automobile     64.8% ████████████
+    bird           53.8% ██████████
+    cat            54.7% ██████████
+    deer           72.6% ██████████████
+    dog            45.9% █████████
+    frog           78.4% ███████████████
+    horse          76.2% ███████████████
+    ship           84.2% ████████████████
+    truck          78.7% ███████████████
+
+Ep 11/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.00it/s, loss=0.7053 acc=72.5%]
+  ep 11 | loss=0.7744 train=72.5% test=71.5% | best=automobile=85% worst=cat=47% | 6s
+Ep 12/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.48it/s, loss=0.7007 acc=73.8%]
+  ep 12 | loss=0.7421 train=73.8% test=71.8% | best=truck=88% worst=cat=50% | 6s
+Ep 13/30: 100%|████████████████████| 390/390 [00:06<00:00, 64.77it/s, loss=0.6695 acc=74.9%]
+  ep 13 | loss=0.7130 train=74.9% test=72.3% | best=automobile=90% worst=cat=49% | 6s
+Ep 14/30: 100%|████████████████████| 390/390 [00:06<00:00, 64.57it/s, loss=0.6952 acc=75.6%]
+  ep 14 | loss=0.6917 train=75.6% test=72.6% | best=automobile=90% worst=dog=48% | 6s
+Ep 15/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.06it/s, loss=0.5546 acc=76.7%]
+  ep 15 | loss=0.6617 train=76.7% test=73.7% | best=automobile=91% worst=dog=60% | 6s
+
+    class             acc
+    ----------------------
+    airplane       80.5% ████████████████
+    automobile     91.3% ██████████████████
+    bird           68.1% █████████████
+    cat            60.3% ████████████
+    deer           66.4% █████████████
+    dog            60.0% ████████████
+    frog           79.0% ███████████████
+    horse          76.0% ███████████████
+    ship           85.4% █████████████████
+    truck          70.2% ██████████████
+
+Ep 16/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.05it/s, loss=0.6078 acc=77.6%]
+  ep 16 | loss=0.6364 train=77.6% test=74.0% | best=automobile=88% worst=cat=55% | 6s
+Ep 17/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.26it/s, loss=0.7195 acc=78.4%]
+  ep 17 | loss=0.6122 train=78.4% test=74.8% | best=automobile=89% worst=cat=60% | 6s
+Ep 18/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.46it/s, loss=0.5613 acc=79.2%]
+  ep 18 | loss=0.5941 train=79.2% test=75.1% | best=automobile=90% worst=cat=57% | 6s
+Ep 19/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.23it/s, loss=0.4989 acc=79.7%]
+  ep 19 | loss=0.5762 train=79.7% test=75.1% | best=automobile=90% worst=cat=60% | 6s
+Ep 20/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.22it/s, loss=0.6564 acc=80.5%]
+  ep 20 | loss=0.5553 train=80.5% test=75.8% | best=automobile=87% worst=bird=60% | 6s
+
+    class             acc
+    ----------------------
+    airplane       81.0% ████████████████
+    automobile     87.0% █████████████████
+    bird           59.8% ███████████
+    cat            62.9% ████████████
+    deer           71.4% ██████████████
+    dog            63.7% ████████████
+    frog           85.0% █████████████████
+    horse          79.0% ███████████████
+    ship           85.2% █████████████████
+    truck          82.6% ████████████████
+
+Ep 21/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.23it/s, loss=0.5616 acc=81.1%]
+  ep 21 | loss=0.5364 train=81.1% test=75.2% | best=truck=88% worst=cat=53% | 6s
+Ep 22/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.54it/s, loss=0.4172 acc=81.8%]
+  ep 22 | loss=0.5201 train=81.8% test=76.4% | best=automobile=88% worst=cat=56% | 6s
+Ep 23/30: 100%|████████████████████| 390/390 [00:06<00:00, 64.75it/s, loss=0.6296 acc=82.2%]
+  ep 23 | loss=0.5099 train=82.2% test=76.2% | best=automobile=88% worst=cat=65% | 6s
+Ep 24/30: 100%|████████████████████| 390/390 [00:06<00:00, 64.50it/s, loss=0.4983 acc=82.9%]
+  ep 24 | loss=0.4906 train=82.9% test=76.4% | best=ship=87% worst=cat=60% | 6s
+Ep 25/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.10it/s, loss=0.5062 acc=83.0%]
+  ep 25 | loss=0.4863 train=83.0% test=76.6% | best=automobile=89% worst=cat=56% | 6s
+
+    class             acc
+    ----------------------
+    airplane       80.5% ████████████████
+    automobile     89.4% █████████████████
+    bird           62.5% ████████████
+    cat            55.9% ███████████
+    deer           73.8% ██████████████
+    dog            71.9% ██████████████
+    frog           86.9% █████████████████
+    horse          78.0% ███████████████
+    ship           83.2% ████████████████
+    truck          84.0% ████████████████
+
+Ep 26/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.21it/s, loss=0.4450 acc=83.4%]
+  ep 26 | loss=0.4762 train=83.4% test=77.0% | best=truck=87% worst=cat=58% | 6s
+Ep 27/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.23it/s, loss=0.5349 acc=83.5%]
+  ep 27 | loss=0.4720 train=83.5% test=77.2% | best=automobile=87% worst=cat=59% | 6s
+Ep 28/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.20it/s, loss=0.4860 acc=83.8%]
+  ep 28 | loss=0.4634 train=83.8% test=76.9% | best=automobile=87% worst=cat=59% | 6s
+Ep 29/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.58it/s, loss=0.3822 acc=84.0%]
+  ep 29 | loss=0.4596 train=84.0% test=77.0% | best=automobile=88% worst=cat=59% | 6s
+Ep 30/30: 100%|████████████████████| 390/390 [00:05<00:00, 65.01it/s, loss=0.3785 acc=84.1%]
+  ep 30 | loss=0.4574 train=84.1% test=77.1% | best=automobile=87% worst=cat=59% | 6s
+
+    class             acc
+    ----------------------
+    airplane       79.7% ███████████████
+    automobile     87.2% █████████████████
+    bird           65.3% █████████████
+    cat            58.8% ███████████
+    deer           74.4% ██████████████
+    dog            70.8% ██████████████
+    frog           84.1% ████████████████
+    horse          80.2% ████████████████
+    ship           84.5% ████████████████
+    truck          86.1% █████████████████
+
+
+======================================================================
+BASELINE COMPLETE
+  Best test accuracy: 77.2%
+  Classifier params: 867,050
+  Random chance: 10.0%
+======================================================================
+
+======================================================================
+HEAD-TO-HEAD COMPARISON
+======================================================================
+  Omega Processor (Freckles features): 72.7%
+  Baseline (raw patches):              77.2%
+  Delta:                               -4.5%
+  Random chance:                       10.0%
+======================================================================