Spaces:

---

open-llm-leaderboard

/

open_llm_leaderboard

like 14k

Running on CPU Upgrade

TemporalMesh Transformer — benchmarked on this dataset

TMT achieves 29.4 PPL on WikiText-2 (vs 31.8 Mamba, 42.1 vanilla) and 36.1 on WikiText-103 at only 48% relative compute — 120M params, 3 seeds.

Five innovations: Mesh Attention (O(S·k) dynamic kNN), Temporal Decay, Adaptive Exit Gate, Dual-Stream FFN, EMA Memory Anchors.

📄 Paper: https://zenodo.org/records/20287390
💻 Code: https://github.com/vignesh2027/TemporalMesh-Transformer
🎮 Demo: https://huggingface.co/spaces/vigneshwar234/TemporalMesh-Transformer-Demo