---
library_name: transformers
tags:
- generated_from_trainer
datasets:
- roneneldan/TinyStories
metrics:
- accuracy
model-index:
- name: c3_moedl_e32_k4-0119
  results:
  - task:
      name: Causal Language Modeling
      type: text-generation
    dataset:
      name: roneneldan/TinyStories
      type: roneneldan/TinyStories
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.70426674401851
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# c3_moedl_e32_k4-0119

This model is a MoE (`Moedl`) pretrained with `roneneldan/TinyStories` dataset.
- Eval Loss: 1.0581
- Accuracy: 0.7043
- Num Input Tokens Seen: 1027671040
- [wandb log](https://wandb.ai/vchua/moe-lab-2026-0119/runs/pq7biex8)

### To run or reproduce: follow [moe-lab](https://github.com/vuiseng9/moe-lab)