Instructions to use stefanocarrera/autophagycode_M_meta-llama__Meta-Llama-3.1-8B-Instruct_gen3_TEST with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use stefanocarrera/autophagycode_M_meta-llama__Meta-Llama-3.1-8B-Instruct_gen3_TEST with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3.1-8B-Instruct")
model = PeftModel.from_pretrained(base_model, "stefanocarrera/autophagycode_M_meta-llama__Meta-Llama-3.1-8B-Instruct_gen3_TEST")

Transformers

How to use stefanocarrera/autophagycode_M_meta-llama__Meta-Llama-3.1-8B-Instruct_gen3_TEST with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="stefanocarrera/autophagycode_M_meta-llama__Meta-Llama-3.1-8B-Instruct_gen3_TEST")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("stefanocarrera/autophagycode_M_meta-llama__Meta-Llama-3.1-8B-Instruct_gen3_TEST", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use stefanocarrera/autophagycode_M_meta-llama__Meta-Llama-3.1-8B-Instruct_gen3_TEST with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "stefanocarrera/autophagycode_M_meta-llama__Meta-Llama-3.1-8B-Instruct_gen3_TEST"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "stefanocarrera/autophagycode_M_meta-llama__Meta-Llama-3.1-8B-Instruct_gen3_TEST",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/stefanocarrera/autophagycode_M_meta-llama__Meta-Llama-3.1-8B-Instruct_gen3_TEST

SGLang

How to use stefanocarrera/autophagycode_M_meta-llama__Meta-Llama-3.1-8B-Instruct_gen3_TEST with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "stefanocarrera/autophagycode_M_meta-llama__Meta-Llama-3.1-8B-Instruct_gen3_TEST" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "stefanocarrera/autophagycode_M_meta-llama__Meta-Llama-3.1-8B-Instruct_gen3_TEST",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "stefanocarrera/autophagycode_M_meta-llama__Meta-Llama-3.1-8B-Instruct_gen3_TEST" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "stefanocarrera/autophagycode_M_meta-llama__Meta-Llama-3.1-8B-Instruct_gen3_TEST",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use stefanocarrera/autophagycode_M_meta-llama__Meta-Llama-3.1-8B-Instruct_gen3_TEST with Docker Model Runner:
```
docker model run hf.co/stefanocarrera/autophagycode_M_meta-llama__Meta-Llama-3.1-8B-Instruct_gen3_TEST
```

stefanocarrera commited on Feb 15

Commit

8cdf54d

verified ·

1 Parent(s): fbe1e4d

Upload folder using huggingface_hub

Browse files

Files changed (10) hide show

adapter_config.json +5 -5
adapter_model.safetensors +1 -1
checkpoint-150/adapter_config.json +4 -4
checkpoint-150/adapter_model.safetensors +1 -1
checkpoint-150/optimizer.pt +1 -1
checkpoint-150/trainer_state.json +772 -772
checkpoint-164/adapter_config.json +4 -4
checkpoint-164/adapter_model.safetensors +1 -1
checkpoint-164/optimizer.pt +1 -1
checkpoint-164/trainer_state.json +842 -842

adapter_config.json CHANGED Viewed

@@ -29,13 +29,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "q_proj",
-    "up_proj",
-    "k_proj",
-    "v_proj",
     "o_proj",
     "down_proj",
-    "gate_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

   "rank_pattern": {},
   "revision": null,
   "target_modules": [
     "o_proj",
+    "gate_proj",
+    "k_proj",
+    "q_proj",
     "down_proj",
+    "v_proj",
+    "up_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:99d569edfe594f0c62c62a99823ca2772d86d0bf64c287ec820ae443b07beaa1
 size 83946192

 version https://git-lfs.github.com/spec/v1
+oid sha256:1a30df352e3e5bf7a3be3ccca5e0bf0b3a9b19ac4eb509a3b6c3fbbccdd879fb
 size 83946192

checkpoint-150/adapter_config.json CHANGED Viewed

@@ -29,13 +29,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
     "gate_proj",
-    "up_proj",
-    "q_proj",
-    "v_proj",
     "k_proj",
     "down_proj",
-    "o_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "o_proj",
     "gate_proj",
     "k_proj",
+    "q_proj",
     "down_proj",
+    "v_proj",
+    "up_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

checkpoint-150/adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a44a4988e01e3c390b69e98347d898a87d3cf126ec0cb79df08350fad8ca7faa
 size 83946192

 version https://git-lfs.github.com/spec/v1
+oid sha256:fa9896d8e319bff7d9d35da65aba6a17142562f9de4a13413e0140189cb8db35
 size 83946192

checkpoint-150/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a42bc05db82290a4b40ebe09854f20ca5f421a7db097ca0451c5c9ec80006e2e
 size 85728997

 version https://git-lfs.github.com/spec/v1
+oid sha256:e55538cf9956f93c36ccaecb1516773c281d3971699a863b515afecea63db183
 size 85728997

checkpoint-150/trainer_state.json CHANGED Viewed

@@ -10,1536 +10,1536 @@
   "is_world_process_zero": true,
   "log_history": [
     {
-      "entropy": 0.3660925142467022,
       "epoch": 0.01225114854517611,
-      "grad_norm": 0.0166015625,
       "learning_rate": 0.0002,
-      "loss": 0.0020782470237463713,
-      "mean_token_accuracy": 0.9997171945869923,
-      "num_tokens": 6092.0,
       "step": 1
     },
     {
-      "entropy": 0.34051003493368626,
       "epoch": 0.02450229709035222,
-      "grad_norm": 0.000823974609375,
       "learning_rate": 0.00019878048780487805,
-      "loss": 9.216360922437161e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 11535.0,
       "step": 2
     },
     {
-      "entropy": 0.32960800640285015,
       "epoch": 0.036753445635528334,
-      "grad_norm": 0.0098876953125,
       "learning_rate": 0.0001975609756097561,
-      "loss": 0.0001977928914129734,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 16432.0,
       "step": 3
     },
     {
-      "entropy": 0.33627333864569664,
       "epoch": 0.04900459418070444,
-      "grad_norm": 0.06640625,
       "learning_rate": 0.00019634146341463416,
-      "loss": 0.00977393426001072,
-      "mean_token_accuracy": 0.9985632188618183,
-      "num_tokens": 20507.0,
       "step": 4
     },
     {
-      "entropy": 0.31916058249771595,
       "epoch": 0.06125574272588055,
-      "grad_norm": 0.0003108978271484375,
       "learning_rate": 0.0001951219512195122,
-      "loss": 5.0926646508742124e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 26122.0,
       "step": 5
     },
     {
-      "entropy": 0.3524587769061327,
       "epoch": 0.07350689127105667,
-      "grad_norm": 0.000186920166015625,
       "learning_rate": 0.00019390243902439025,
-      "loss": 4.6155335439834744e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 30847.0,
       "step": 6
     },
     {
-      "entropy": 0.3272323925048113,
       "epoch": 0.08575803981623277,
-      "grad_norm": 0.005859375,
       "learning_rate": 0.0001926829268292683,
-      "loss": 0.000202978597371839,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 36541.0,
       "step": 7
     },
     {
-      "entropy": 0.347023731097579,
       "epoch": 0.09800918836140889,
-      "grad_norm": 0.00072479248046875,
       "learning_rate": 0.00019146341463414633,
-      "loss": 0.00011593783710850403,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 41001.0,
       "step": 8
     },
     {
-      "entropy": 0.376500410027802,
       "epoch": 0.11026033690658499,
-      "grad_norm": 0.09033203125,
       "learning_rate": 0.0001902439024390244,
-      "loss": 0.008863622322678566,
-      "mean_token_accuracy": 0.9979648105800152,
-      "num_tokens": 45467.0,
       "step": 9
     },
     {
-      "entropy": 0.3560014171525836,
       "epoch": 0.1225114854517611,
-      "grad_norm": 0.055419921875,
       "learning_rate": 0.00018902439024390244,
-      "loss": 0.004083322826772928,
-      "mean_token_accuracy": 0.9990039840340614,
-      "num_tokens": 50478.0,
       "step": 10
     },
     {
-      "entropy": 0.3533000349998474,
       "epoch": 0.13476263399693722,
-      "grad_norm": 0.0033721923828125,
       "learning_rate": 0.0001878048780487805,
-      "loss": 0.000252897065365687,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 56181.0,
       "step": 11
     },
     {
-      "entropy": 0.4079158063977957,
       "epoch": 0.14701378254211334,
-      "grad_norm": 0.00110626220703125,
       "learning_rate": 0.00018658536585365856,
-      "loss": 0.00019193078333046287,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 62946.0,
       "step": 12
     },
     {
-      "entropy": 0.4043316235765815,
       "epoch": 0.15926493108728942,
-      "grad_norm": 0.0021209716796875,
       "learning_rate": 0.0001853658536585366,
-      "loss": 0.00025091503630392253,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 68436.0,
       "step": 13
     },
     {
-      "entropy": 0.41207500360906124,
       "epoch": 0.17151607963246554,
-      "grad_norm": 0.00139617919921875,
       "learning_rate": 0.00018414634146341464,
-      "loss": 0.0002536335668992251,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 73603.0,
       "step": 14
     },
     {
-      "entropy": 0.43669185042381287,
       "epoch": 0.18376722817764166,
-      "grad_norm": 0.020751953125,
       "learning_rate": 0.0001829268292682927,
-      "loss": 0.0008837866480462253,
-      "mean_token_accuracy": 0.9994877055287361,
-      "num_tokens": 77845.0,
       "step": 15
     },
     {
-      "entropy": 0.41382858343422413,
       "epoch": 0.19601837672281777,
-      "grad_norm": 0.0145263671875,
       "learning_rate": 0.00018170731707317075,
-      "loss": 0.0006772386841475964,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 82744.0,
       "step": 16
     },
     {
-      "entropy": 0.4243332091718912,
       "epoch": 0.2082695252679939,
-      "grad_norm": 0.001922607421875,
       "learning_rate": 0.0001804878048780488,
-      "loss": 0.00027059210697188973,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 87453.0,
       "step": 17
     },
     {
-      "entropy": 0.4329488482326269,
       "epoch": 0.22052067381316998,
-      "grad_norm": 0.004852294921875,
       "learning_rate": 0.00017926829268292684,
-      "loss": 0.00031758740078657866,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 92321.0,
       "step": 18
     },
     {
-      "entropy": 0.440301101654768,
       "epoch": 0.2327718223583461,
-      "grad_norm": 0.005767822265625,
       "learning_rate": 0.00017804878048780488,
-      "loss": 0.0004065934626851231,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 97146.0,
       "step": 19
     },
     {
-      "entropy": 0.4400939680635929,
       "epoch": 0.2450229709035222,
-      "grad_norm": 0.0023040771484375,
       "learning_rate": 0.00017682926829268295,
-      "loss": 0.00020425915136002004,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 101943.0,
       "step": 20
     },
     {
-      "entropy": 0.4579729177057743,
       "epoch": 0.2572741194486983,
-      "grad_norm": 0.0286865234375,
       "learning_rate": 0.000175609756097561,
-      "loss": 0.0015601275954395533,
-      "mean_token_accuracy": 0.9996448867022991,
-      "num_tokens": 106772.0,
       "step": 21
     },
     {
-      "entropy": 0.40288309939205647,
       "epoch": 0.26952526799387444,
-      "grad_norm": 0.00072479248046875,
       "learning_rate": 0.00017439024390243903,
-      "loss": 9.121054608840495e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 112558.0,
       "step": 22
     },
     {
-      "entropy": 0.4252484003081918,
       "epoch": 0.28177641653905056,
-      "grad_norm": 0.000457763671875,
       "learning_rate": 0.00017317073170731708,
-      "loss": 8.147547487169504e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 117489.0,
       "step": 23
     },
     {
-      "entropy": 0.44810181483626366,
       "epoch": 0.29402756508422667,
-      "grad_norm": 0.007720947265625,
       "learning_rate": 0.00017195121951219512,
-      "loss": 0.0003956289147026837,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 123010.0,
       "step": 24
     },
     {
-      "entropy": 0.4023376125842333,
       "epoch": 0.30627871362940273,
-      "grad_norm": 0.00103759765625,
       "learning_rate": 0.0001707317073170732,
-      "loss": 8.693434210726991e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 127716.0,
       "step": 25
     },
     {
-      "entropy": 0.4007954867556691,
       "epoch": 0.31852986217457885,
-      "grad_norm": 0.00194549560546875,
       "learning_rate": 0.00016951219512195123,
-      "loss": 8.696074655745178e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 132372.0,
       "step": 26
     },
     {
-      "entropy": 0.3759774696081877,
       "epoch": 0.33078101071975496,
-      "grad_norm": 0.003387451171875,
       "learning_rate": 0.00016829268292682927,
-      "loss": 0.00013623938139062375,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 137028.0,
       "step": 27
     },
     {
-      "entropy": 0.40147540159523487,
       "epoch": 0.3430321592649311,
-      "grad_norm": 0.0380859375,
       "learning_rate": 0.00016707317073170731,
-      "loss": 0.005999124608933926,
-      "mean_token_accuracy": 0.9987113401293755,
-      "num_tokens": 142088.0,
       "step": 28
     },
     {
-      "entropy": 0.38656803220510483,
       "epoch": 0.3552833078101072,
-      "grad_norm": 0.0322265625,
       "learning_rate": 0.00016585365853658536,
-      "loss": 0.00021061318693682551,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 147481.0,
       "step": 29
     },
     {
-      "entropy": 0.4059827271848917,
       "epoch": 0.3675344563552833,
-      "grad_norm": 0.00015163421630859375,
       "learning_rate": 0.00016463414634146343,
-      "loss": 3.9411937905242667e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 152973.0,
       "step": 30
     },
     {
-      "entropy": 0.40111804008483887,
       "epoch": 0.37978560490045943,
-      "grad_norm": 0.0003681182861328125,
       "learning_rate": 0.00016341463414634147,
-      "loss": 5.111394784762524e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 156786.0,
       "step": 31
     },
     {
-      "entropy": 0.41568026319146156,
       "epoch": 0.39203675344563554,
-      "grad_norm": 0.00162506103515625,
       "learning_rate": 0.00016219512195121954,
-      "loss": 0.0001103500762837939,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 162859.0,
       "step": 32
     },
     {
-      "entropy": 0.39988269470632076,
       "epoch": 0.40428790199081166,
-      "grad_norm": 0.000518798828125,
       "learning_rate": 0.00016097560975609758,
-      "loss": 6.166221282910556e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 167969.0,
       "step": 33
     },
     {
-      "entropy": 0.3738459562882781,
       "epoch": 0.4165390505359878,
-      "grad_norm": 0.00537109375,
       "learning_rate": 0.00015975609756097562,
-      "loss": 0.00012469613284338266,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 172518.0,
       "step": 34
     },
     {
-      "entropy": 0.40653541777282953,
       "epoch": 0.42879019908116384,
-      "grad_norm": 0.0031280517578125,
       "learning_rate": 0.00015853658536585366,
-      "loss": 0.00010661048872862011,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 177085.0,
       "step": 35
     },
     {
-      "entropy": 0.39361329190433025,
       "epoch": 0.44104134762633995,
-      "grad_norm": 0.08154296875,
       "learning_rate": 0.00015731707317073173,
-      "loss": 0.0010916765313595533,
-      "mean_token_accuracy": 0.9990942031145096,
-      "num_tokens": 181617.0,
       "step": 36
     },
     {
-      "entropy": 0.358949625864625,
       "epoch": 0.45329249617151607,
-      "grad_norm": 0.01080322265625,
       "learning_rate": 0.00015609756097560978,
-      "loss": 0.0010772041277959943,
-      "mean_token_accuracy": 0.9995535723865032,
-      "num_tokens": 186836.0,
       "step": 37
     },
     {
-      "entropy": 0.3930373042821884,
       "epoch": 0.4655436447166922,
-      "grad_norm": 0.000461578369140625,
       "learning_rate": 0.00015487804878048782,
-      "loss": 5.279047036310658e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 191224.0,
       "step": 38
     },
     {
-      "entropy": 0.35740520991384983,
       "epoch": 0.4777947932618683,
-      "grad_norm": 0.000873565673828125,
       "learning_rate": 0.00015365853658536586,
-      "loss": 5.439379674498923e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 195926.0,
       "step": 39
     },
     {
-      "entropy": 0.38909873832017183,
       "epoch": 0.4900459418070444,
-      "grad_norm": 0.0257568359375,
       "learning_rate": 0.0001524390243902439,
-      "loss": 0.0015194097068160772,
-      "mean_token_accuracy": 0.999550361186266,
-      "num_tokens": 200772.0,
       "step": 40
     },
     {
-      "entropy": 0.36850977689027786,
       "epoch": 0.5022970903522205,
-      "grad_norm": 0.1064453125,
       "learning_rate": 0.00015121951219512197,
-      "loss": 0.002955856267362833,
-      "mean_token_accuracy": 0.9993872530758381,
-      "num_tokens": 204499.0,
       "step": 41
     },
     {
-      "entropy": 0.3940112106502056,
       "epoch": 0.5145482388973966,
-      "grad_norm": 0.00885009765625,
       "learning_rate": 0.00015000000000000001,
-      "loss": 0.000253106962190941,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 208814.0,
       "step": 42
     },
     {
-      "entropy": 0.39878340624272823,
       "epoch": 0.5267993874425727,
-      "grad_norm": 0.037841796875,
       "learning_rate": 0.00014878048780487806,
-      "loss": 0.0007202713750302792,
-      "mean_token_accuracy": 0.9995833337306976,
-      "num_tokens": 213907.0,
       "step": 43
     },
     {
-      "entropy": 0.41587444953620434,
       "epoch": 0.5390505359877489,
-      "grad_norm": 0.0004177093505859375,
       "learning_rate": 0.0001475609756097561,
-      "loss": 6.820505223004147e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 218988.0,
       "step": 44
     },
     {
-      "entropy": 0.3888211837038398,
       "epoch": 0.5513016845329249,
-      "grad_norm": 0.007568359375,
       "learning_rate": 0.00014634146341463414,
-      "loss": 0.000737900089006871,
-      "mean_token_accuracy": 0.9995967745780945,
-      "num_tokens": 223595.0,
       "step": 45
     },
     {
-      "entropy": 0.4139576517045498,
       "epoch": 0.5635528330781011,
-      "grad_norm": 0.014892578125,
       "learning_rate": 0.0001451219512195122,
-      "loss": 0.0006043408066034317,
-      "mean_token_accuracy": 0.9995192289352417,
-      "num_tokens": 228244.0,
       "step": 46
     },
     {
-      "entropy": 0.39713083021342754,
       "epoch": 0.5758039816232772,
-      "grad_norm": 0.00046539306640625,
       "learning_rate": 0.00014390243902439025,
-      "loss": 8.217584399972111e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 232606.0,
       "step": 47
     },
     {
-      "entropy": 0.40557617880403996,
       "epoch": 0.5880551301684533,
-      "grad_norm": 0.0009918212890625,
       "learning_rate": 0.0001426829268292683,
-      "loss": 0.00012616875756066293,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 236563.0,
       "step": 48
     },
     {
-      "entropy": 0.43470797687768936,
       "epoch": 0.6003062787136294,
-      "grad_norm": 0.0238037109375,
       "learning_rate": 0.00014146341463414634,
-      "loss": 0.0010796654969453812,
-      "mean_token_accuracy": 0.999465811997652,
-      "num_tokens": 241214.0,
       "step": 49
     },
     {
-      "entropy": 0.4234541580080986,
       "epoch": 0.6125574272588055,
-      "grad_norm": 0.02783203125,
       "learning_rate": 0.00014024390243902438,
-      "loss": 0.0009178520413115621,
-      "mean_token_accuracy": 0.9996565915644169,
-      "num_tokens": 245200.0,
       "step": 50
     },
     {
       "epoch": 0.6125574272588055,
-      "eval_entropy": 0.4022736955380094,
-      "eval_loss": 0.0006544959614984691,
-      "eval_mean_token_accuracy": 0.9998166846192401,
-      "eval_num_tokens": 245200.0,
-      "eval_runtime": 51.0138,
-      "eval_samples_per_second": 1.353,
-      "eval_steps_per_second": 1.353,
       "step": 50
     },
     {
-      "entropy": 0.41674751229584217,
       "epoch": 0.6248085758039816,
-      "grad_norm": 0.00131988525390625,
       "learning_rate": 0.00013902439024390245,
-      "loss": 0.0001285702601308003,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 249761.0,
       "step": 51
     },
     {
-      "entropy": 0.42886597104370594,
       "epoch": 0.6370597243491577,
-      "grad_norm": 0.00171661376953125,
       "learning_rate": 0.0001378048780487805,
-      "loss": 0.00014620381989516318,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 254787.0,
       "step": 52
     },
     {
-      "entropy": 0.4423276912420988,
       "epoch": 0.6493108728943339,
-      "grad_norm": 0.038818359375,
       "learning_rate": 0.00013658536585365856,
-      "loss": 0.003947169054299593,
-      "mean_token_accuracy": 0.9983357414603233,
-      "num_tokens": 260287.0,
       "step": 53
     },
     {
-      "entropy": 0.3989156847819686,
       "epoch": 0.6615620214395099,
-      "grad_norm": 0.0211181640625,
       "learning_rate": 0.0001353658536585366,
-      "loss": 0.00047477131010964513,
-      "mean_token_accuracy": 0.9998249299824238,
-      "num_tokens": 264810.0,
       "step": 54
     },
     {
-      "entropy": 0.4272368475794792,
       "epoch": 0.6738131699846861,
-      "grad_norm": 0.029052734375,
       "learning_rate": 0.00013414634146341464,
-      "loss": 0.00408769678324461,
-      "mean_token_accuracy": 0.9993622452020645,
-      "num_tokens": 270386.0,
       "step": 55
     },
     {
-      "entropy": 0.44703495875000954,
       "epoch": 0.6860643185298622,
-      "grad_norm": 0.01202392578125,
       "learning_rate": 0.0001329268292682927,
-      "loss": 0.00038261126610450447,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 274391.0,
       "step": 56
     },
     {
-      "entropy": 0.4288428146392107,
       "epoch": 0.6983154670750383,
-      "grad_norm": 0.01019287109375,
       "learning_rate": 0.00013170731707317076,
-      "loss": 0.0003242077073082328,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 279716.0,
       "step": 57
     },
     {
-      "entropy": 0.37452960200607777,
       "epoch": 0.7105666156202144,
-      "grad_norm": 0.021728515625,
       "learning_rate": 0.0001304878048780488,
-      "loss": 0.0027725810650736094,
-      "mean_token_accuracy": 0.9994703382253647,
-      "num_tokens": 285404.0,
       "step": 58
     },
     {
-      "entropy": 0.4130611680448055,
       "epoch": 0.7228177641653905,
-      "grad_norm": 0.04541015625,
       "learning_rate": 0.00012926829268292684,
-      "loss": 0.0017543239519000053,
-      "mean_token_accuracy": 0.9995689652860165,
-      "num_tokens": 289992.0,
       "step": 59
     },
     {
-      "entropy": 0.41101630590856075,
       "epoch": 0.7350689127105666,
-      "grad_norm": 0.00078582763671875,
       "learning_rate": 0.00012804878048780488,
-      "loss": 9.316274372395128e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 294861.0,
       "step": 60
     },
     {
-      "entropy": 0.3678157525137067,
       "epoch": 0.7473200612557427,
-      "grad_norm": 0.00058746337890625,
       "learning_rate": 0.00012682926829268293,
-      "loss": 8.83688626345247e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 300355.0,
       "step": 61
     },
     {
-      "entropy": 0.40994635969400406,
       "epoch": 0.7595712098009189,
-      "grad_norm": 0.0015869140625,
       "learning_rate": 0.000125609756097561,
-      "loss": 8.545083983335644e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 305776.0,
       "step": 62
     },
     {
-      "entropy": 0.37295936793088913,
       "epoch": 0.7718223583460949,
-      "grad_norm": 0.000827789306640625,
       "learning_rate": 0.00012439024390243904,
-      "loss": 7.97374959802255e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 310204.0,
       "step": 63
     },
     {
-      "entropy": 0.36804571095854044,
       "epoch": 0.7840735068912711,
-      "grad_norm": 0.0002880096435546875,
       "learning_rate": 0.00012317073170731708,
-      "loss": 6.0703161580022424e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 314205.0,
       "step": 64
     },
     {
-      "entropy": 0.3904844745993614,
       "epoch": 0.7963246554364471,
-      "grad_norm": 0.0019989013671875,
       "learning_rate": 0.00012195121951219512,
-      "loss": 7.91027705417946e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 319157.0,
       "step": 65
     },
     {
-      "entropy": 0.3921838700771332,
       "epoch": 0.8085758039816233,
-      "grad_norm": 0.00177764892578125,
       "learning_rate": 0.00012073170731707318,
-      "loss": 8.364896348211914e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 324681.0,
       "step": 66
     },
     {
-      "entropy": 0.34572961553931236,
       "epoch": 0.8208269525267994,
-      "grad_norm": 0.061767578125,
       "learning_rate": 0.00011951219512195122,
-      "loss": 0.008409281261265278,
-      "mean_token_accuracy": 0.9963545724749565,
-      "num_tokens": 329941.0,
       "step": 67
     },
     {
-      "entropy": 0.3841299172490835,
       "epoch": 0.8330781010719756,
-      "grad_norm": 0.01123046875,
       "learning_rate": 0.00011829268292682926,
-      "loss": 0.00017956709780264646,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 334486.0,
       "step": 68
     },
     {
-      "entropy": 0.39541577361524105,
       "epoch": 0.8453292496171516,
-      "grad_norm": 0.00274658203125,
       "learning_rate": 0.00011707317073170732,
-      "loss": 0.00012585960212163627,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 338183.0,
       "step": 69
     },
     {
-      "entropy": 0.4046988161280751,
       "epoch": 0.8575803981623277,
-      "grad_norm": 0.126953125,
       "learning_rate": 0.00011585365853658536,
-      "loss": 0.007125813513994217,
-      "mean_token_accuracy": 0.9981492757797241,
-      "num_tokens": 342593.0,
       "step": 70
     },
     {
-      "entropy": 0.40994592756032944,
       "epoch": 0.8698315467075038,
-      "grad_norm": 0.0517578125,
       "learning_rate": 0.00011463414634146342,
-      "loss": 0.0006066925125196576,
-      "mean_token_accuracy": 0.9997807033360004,
-      "num_tokens": 347797.0,
       "step": 71
     },
     {
-      "entropy": 0.3796220198273659,
       "epoch": 0.8820826952526799,
-      "grad_norm": 0.006103515625,
       "learning_rate": 0.00011341463414634146,
-      "loss": 0.00017896694771479815,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 352121.0,
       "step": 72
     },
     {
-      "entropy": 0.3931356444954872,
       "epoch": 0.8943338437978561,
-      "grad_norm": 0.0181884765625,
       "learning_rate": 0.00011219512195121953,
-      "loss": 0.0010632644407451153,
-      "mean_token_accuracy": 0.9997568093240261,
-      "num_tokens": 357943.0,
       "step": 73
     },
     {
-      "entropy": 0.36392936669290066,
       "epoch": 0.9065849923430321,
-      "grad_norm": 0.024658203125,
       "learning_rate": 0.00011097560975609757,
-      "loss": 0.0006849091150797904,
-      "mean_token_accuracy": 0.9996345043182373,
-      "num_tokens": 363814.0,
       "step": 74
     },
     {
-      "entropy": 0.3864069525152445,
       "epoch": 0.9188361408882083,
-      "grad_norm": 0.000270843505859375,
       "learning_rate": 0.00010975609756097563,
-      "loss": 5.0294114771531895e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 368870.0,
       "step": 75
     },
     {
-      "entropy": 0.39719677343964577,
       "epoch": 0.9310872894333844,
-      "grad_norm": 0.01519775390625,
       "learning_rate": 0.00010853658536585367,
-      "loss": 0.00048823675024323165,
-      "mean_token_accuracy": 0.999143835157156,
-      "num_tokens": 373670.0,
       "step": 76
     },
     {
-      "entropy": 0.35627279058098793,
       "epoch": 0.9433384379785605,
-      "grad_norm": 0.0074462890625,
       "learning_rate": 0.00010731707317073172,
-      "loss": 0.000174719825736247,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 379037.0,
       "step": 77
     },
     {
-      "entropy": 0.38681978918612003,
       "epoch": 0.9555895865237366,
-      "grad_norm": 0.0181884765625,
       "learning_rate": 0.00010609756097560977,
-      "loss": 0.000976942596025765,
-      "mean_token_accuracy": 0.9992977529764175,
-      "num_tokens": 384252.0,
       "step": 78
     },
     {
-      "entropy": 0.3772548586130142,
       "epoch": 0.9678407350689127,
-      "grad_norm": 0.000904083251953125,
       "learning_rate": 0.00010487804878048781,
-      "loss": 6.608536932617426e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 388347.0,
       "step": 79
     },
     {
-      "entropy": 0.3597776433452964,
       "epoch": 0.9800918836140888,
-      "grad_norm": 0.010986328125,
       "learning_rate": 0.00010365853658536586,
-      "loss": 0.0007963755051605403,
-      "mean_token_accuracy": 0.999015748500824,
-      "num_tokens": 394213.0,
       "step": 80
     },
     {
-      "entropy": 0.3731031287461519,
       "epoch": 0.9923430321592649,
-      "grad_norm": 0.00115966796875,
       "learning_rate": 0.0001024390243902439,
-      "loss": 8.310518751386553e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 399113.0,
       "step": 81
     },
     {
-      "entropy": 0.37349462509155273,
       "epoch": 1.0,
-      "grad_norm": 0.00022125244140625,
       "learning_rate": 0.00010121951219512196,
-      "loss": 4.093759343959391e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 402129.0,
       "step": 82
     },
     {
-      "entropy": 0.38408348336815834,
       "epoch": 1.0122511485451762,
-      "grad_norm": 0.027099609375,
       "learning_rate": 0.0001,
-      "loss": 0.0015746817225590348,
-      "mean_token_accuracy": 0.9996279776096344,
-      "num_tokens": 406760.0,
       "step": 83
     },
     {
-      "entropy": 0.36415083333849907,
       "epoch": 1.0245022970903521,
-      "grad_norm": 0.0032501220703125,
       "learning_rate": 9.878048780487805e-05,
-      "loss": 0.00011362869554432109,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 411366.0,
       "step": 84
     },
     {
-      "entropy": 0.3951573334634304,
       "epoch": 1.0367534456355283,
-      "grad_norm": 0.0018768310546875,
       "learning_rate": 9.75609756097561e-05,
-      "loss": 8.601781155448407e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 417767.0,
       "step": 85
     },
     {
-      "entropy": 0.3533172570168972,
       "epoch": 1.0490045941807045,
-      "grad_norm": 0.00146484375,
       "learning_rate": 9.634146341463415e-05,
-      "loss": 5.874271664652042e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 421737.0,
       "step": 86
     },
     {
-      "entropy": 0.35251205042004585,
       "epoch": 1.0612557427258806,
-      "grad_norm": 6.008148193359375e-05,
       "learning_rate": 9.51219512195122e-05,
-      "loss": 2.1197016394580714e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 426853.0,
       "step": 87
     },
     {
-      "entropy": 0.42304582707583904,
       "epoch": 1.0735068912710566,
-      "grad_norm": 0.000797271728515625,
       "learning_rate": 9.390243902439024e-05,
-      "loss": 6.177897739689797e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 431082.0,
       "step": 88
     },
     {
-      "entropy": 0.39542090706527233,
       "epoch": 1.0857580398162328,
-      "grad_norm": 0.041015625,
       "learning_rate": 9.26829268292683e-05,
-      "loss": 0.0009606232051737607,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 435693.0,
       "step": 89
     },
     {
-      "entropy": 0.37046173214912415,
       "epoch": 1.098009188361409,
-      "grad_norm": 0.000278472900390625,
       "learning_rate": 9.146341463414635e-05,
-      "loss": 4.265129246050492e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 440577.0,
       "step": 90
     },
     {
-      "entropy": 0.3931607408449054,
       "epoch": 1.110260336906585,
-      "grad_norm": 0.035400390625,
       "learning_rate": 9.02439024390244e-05,
-      "loss": 0.004250116180628538,
-      "mean_token_accuracy": 0.9994369372725487,
-      "num_tokens": 445265.0,
       "step": 91
     },
     {
-      "entropy": 0.3917137086391449,
       "epoch": 1.122511485451761,
-      "grad_norm": 0.0419921875,
       "learning_rate": 8.902439024390244e-05,
-      "loss": 0.002317648846656084,
-      "mean_token_accuracy": 0.9992785975337029,
-      "num_tokens": 450020.0,
       "step": 92
     },
     {
-      "entropy": 0.3758338335901499,
       "epoch": 1.1347626339969372,
-      "grad_norm": 0.0196533203125,
       "learning_rate": 8.78048780487805e-05,
-      "loss": 0.0006808089674450457,
-      "mean_token_accuracy": 0.999522902071476,
-      "num_tokens": 455003.0,
       "step": 93
     },
     {
-      "entropy": 0.383782709017396,
       "epoch": 1.1470137825421134,
-      "grad_norm": 0.0034027099609375,
       "learning_rate": 8.658536585365854e-05,
-      "loss": 7.263245788635686e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 459698.0,
       "step": 94
     },
     {
-      "entropy": 0.3821055982261896,
       "epoch": 1.1592649310872893,
-      "grad_norm": 0.0004138946533203125,
       "learning_rate": 8.53658536585366e-05,
-      "loss": 3.771902629523538e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 464337.0,
       "step": 95
     },
     {
-      "entropy": 0.3649219311773777,
       "epoch": 1.1715160796324655,
-      "grad_norm": 0.00872802734375,
       "learning_rate": 8.414634146341464e-05,
-      "loss": 0.0004717935808002949,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 468882.0,
       "step": 96
     },
     {
-      "entropy": 0.3700664434581995,
       "epoch": 1.1837672281776417,
-      "grad_norm": 0.00015544891357421875,
       "learning_rate": 8.292682926829268e-05,
-      "loss": 3.247045970056206e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 473756.0,
       "step": 97
     },
     {
-      "entropy": 0.3915936965495348,
       "epoch": 1.1960183767228179,
-      "grad_norm": 0.05078125,
       "learning_rate": 8.170731707317073e-05,
-      "loss": 0.005024694371968508,
-      "mean_token_accuracy": 0.9996565915644169,
-      "num_tokens": 479061.0,
       "step": 98
     },
     {
-      "entropy": 0.4096358586102724,
       "epoch": 1.2082695252679938,
-      "grad_norm": 0.00144195556640625,
       "learning_rate": 8.048780487804879e-05,
-      "loss": 4.485135286813602e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 484835.0,
       "step": 99
     },
     {
-      "entropy": 0.35138822346925735,
       "epoch": 1.22052067381317,
-      "grad_norm": 0.0038299560546875,
       "learning_rate": 7.926829268292683e-05,
-      "loss": 0.00019770213111769408,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 489546.0,
       "step": 100
     },
     {
       "epoch": 1.22052067381317,
-      "eval_entropy": 0.3780687239722929,
-      "eval_loss": 0.00034746917663142085,
-      "eval_mean_token_accuracy": 0.9999171840971794,
-      "eval_num_tokens": 489546.0,
-      "eval_runtime": 50.9982,
-      "eval_samples_per_second": 1.353,
-      "eval_steps_per_second": 1.353,
       "step": 100
     },
     {
-      "entropy": 0.4012060575187206,
       "epoch": 1.2327718223583461,
-      "grad_norm": 0.000217437744140625,
       "learning_rate": 7.804878048780489e-05,
-      "loss": 3.667730197776109e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 494781.0,
       "step": 101
     },
     {
-      "entropy": 0.37181732058525085,
       "epoch": 1.245022970903522,
-      "grad_norm": 0.0002155303955078125,
       "learning_rate": 7.682926829268293e-05,
-      "loss": 2.923922693298664e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 499861.0,
       "step": 102
     },
     {
-      "entropy": 0.38948795571923256,
       "epoch": 1.2572741194486983,
-      "grad_norm": 6.866455078125e-05,
       "learning_rate": 7.560975609756099e-05,
-      "loss": 3.10177420033142e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 505291.0,
       "step": 103
     },
     {
-      "entropy": 0.3776157572865486,
       "epoch": 1.2695252679938744,
-      "grad_norm": 0.00012874603271484375,
       "learning_rate": 7.439024390243903e-05,
-      "loss": 2.8559963539009914e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 510284.0,
       "step": 104
     },
     {
-      "entropy": 0.3941178657114506,
       "epoch": 1.2817764165390506,
-      "grad_norm": 0.007232666015625,
       "learning_rate": 7.317073170731707e-05,
-      "loss": 0.0008174990070983768,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 514517.0,
       "step": 105
     },
     {
-      "entropy": 0.3697250857949257,
       "epoch": 1.2940275650842268,
-      "grad_norm": 0.003143310546875,
       "learning_rate": 7.195121951219513e-05,
-      "loss": 0.00010880863555939868,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 519535.0,
       "step": 106
     },
     {
-      "entropy": 0.3888526763767004,
       "epoch": 1.3062787136294027,
-      "grad_norm": 0.00054931640625,
       "learning_rate": 7.073170731707317e-05,
-      "loss": 5.111205973662436e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 524397.0,
       "step": 107
     },
     {
-      "entropy": 0.3866258058696985,
       "epoch": 1.318529862174579,
-      "grad_norm": 0.0004100799560546875,
       "learning_rate": 6.951219512195122e-05,
-      "loss": 3.999587715952657e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 528997.0,
       "step": 108
     },
     {
-      "entropy": 0.3921303730458021,
       "epoch": 1.3307810107197549,
-      "grad_norm": 0.000885009765625,
       "learning_rate": 6.829268292682928e-05,
-      "loss": 6.128583481768146e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 533965.0,
       "step": 109
     },
     {
-      "entropy": 0.3705854155123234,
       "epoch": 1.343032159264931,
-      "grad_norm": 0.002960205078125,
       "learning_rate": 6.707317073170732e-05,
-      "loss": 7.792656106175855e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 539387.0,
       "step": 110
     },
     {
-      "entropy": 0.3712622048333287,
       "epoch": 1.3552833078101072,
-      "grad_norm": 0.00089263916015625,
       "learning_rate": 6.585365853658538e-05,
-      "loss": 4.521696246229112e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 543755.0,
       "step": 111
     },
     {
-      "entropy": 0.40867704525589943,
       "epoch": 1.3675344563552834,
-      "grad_norm": 0.023193359375,
       "learning_rate": 6.463414634146342e-05,
-      "loss": 0.003280676668509841,
-      "mean_token_accuracy": 0.9978448264300823,
-      "num_tokens": 548188.0,
       "step": 112
     },
     {
-      "entropy": 0.3910982459783554,
       "epoch": 1.3797856049004595,
-      "grad_norm": 0.0028533935546875,
       "learning_rate": 6.341463414634146e-05,
-      "loss": 0.00015341158723458648,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 553717.0,
       "step": 113
     },
     {
-      "entropy": 0.3753495467826724,
       "epoch": 1.3920367534456355,
-      "grad_norm": 6.866455078125e-05,
       "learning_rate": 6.219512195121952e-05,
-      "loss": 2.554376442276407e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 558501.0,
       "step": 114
     },
     {
-      "entropy": 0.3936616498976946,
       "epoch": 1.4042879019908117,
-      "grad_norm": 0.000774383544921875,
       "learning_rate": 6.097560975609756e-05,
-      "loss": 4.565157360048033e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 563989.0,
       "step": 115
     },
     {
-      "entropy": 0.4080927763134241,
       "epoch": 1.4165390505359878,
-      "grad_norm": 0.000728607177734375,
       "learning_rate": 5.975609756097561e-05,
-      "loss": 5.44461581739597e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 568327.0,
       "step": 116
     },
     {
-      "entropy": 0.36639871448278427,
       "epoch": 1.4287901990811638,
-      "grad_norm": 0.000457763671875,
       "learning_rate": 5.853658536585366e-05,
-      "loss": 3.381741407793015e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 572919.0,
       "step": 117
     },
     {
-      "entropy": 0.4015892669558525,
       "epoch": 1.44104134762634,
-      "grad_norm": 0.00017833709716796875,
       "learning_rate": 5.731707317073171e-05,
-      "loss": 4.158892625127919e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 577916.0,
       "step": 118
     },
     {
-      "entropy": 0.40410150960087776,
       "epoch": 1.4532924961715161,
-      "grad_norm": 0.000621795654296875,
       "learning_rate": 5.6097560975609764e-05,
-      "loss": 2.5736055249581113e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 583152.0,
       "step": 119
     },
     {
-      "entropy": 0.40528898034244776,
       "epoch": 1.4655436447166923,
-      "grad_norm": 0.01953125,
       "learning_rate": 5.487804878048781e-05,
-      "loss": 0.00020874114125035703,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 587880.0,
       "step": 120
     },
     {
-      "entropy": 0.35937592945992947,
       "epoch": 1.4777947932618682,
-      "grad_norm": 0.083984375,
       "learning_rate": 5.365853658536586e-05,
-      "loss": 0.007331337314099073,
-      "mean_token_accuracy": 0.9991379305720329,
-      "num_tokens": 592284.0,
       "step": 121
     },
     {
-      "entropy": 0.3928218297660351,
       "epoch": 1.4900459418070444,
-      "grad_norm": 0.00013446807861328125,
       "learning_rate": 5.2439024390243904e-05,
-      "loss": 2.927147943410091e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 597046.0,
       "step": 122
     },
     {
-      "entropy": 0.3777940608561039,
       "epoch": 1.5022970903522204,
-      "grad_norm": 0.000579833984375,
       "learning_rate": 5.121951219512195e-05,
-      "loss": 6.0145219322294e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 601350.0,
       "step": 123
     },
     {
-      "entropy": 0.39830240048468113,
       "epoch": 1.5145482388973965,
-      "grad_norm": 0.0245361328125,
       "learning_rate": 5e-05,
-      "loss": 0.00029612769139930606,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 606643.0,
       "step": 124
     },
     {
-      "entropy": 0.3925098739564419,
       "epoch": 1.5267993874425727,
-      "grad_norm": 0.0004749298095703125,
       "learning_rate": 4.878048780487805e-05,
-      "loss": 4.631431511370465e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 612405.0,
       "step": 125
     },
     {
-      "entropy": 0.3956710360944271,
       "epoch": 1.5390505359877489,
-      "grad_norm": 0.00634765625,
       "learning_rate": 4.75609756097561e-05,
-      "loss": 8.446360880043358e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 617227.0,
       "step": 126
     },
     {
-      "entropy": 0.430975291877985,
       "epoch": 1.551301684532925,
-      "grad_norm": 0.000518798828125,
       "learning_rate": 4.634146341463415e-05,
-      "loss": 6.132836278993636e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 622353.0,
       "step": 127
     },
     {
-      "entropy": 0.4242272228002548,
       "epoch": 1.5635528330781012,
-      "grad_norm": 0.0025177001953125,
       "learning_rate": 4.51219512195122e-05,
-      "loss": 0.00011561957217054442,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 627267.0,
       "step": 128
     },
     {
-      "entropy": 0.3710012398660183,
       "epoch": 1.5758039816232772,
-      "grad_norm": 0.002777099609375,
       "learning_rate": 4.390243902439025e-05,
-      "loss": 0.00010202911653323099,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 631452.0,
       "step": 129
     },
     {
-      "entropy": 0.35699679516255856,
       "epoch": 1.5880551301684533,
-      "grad_norm": 0.00023651123046875,
       "learning_rate": 4.26829268292683e-05,
-      "loss": 5.903129203943536e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 636500.0,
       "step": 130
     },
     {
-      "entropy": 0.39619251526892185,
       "epoch": 1.6003062787136293,
-      "grad_norm": 0.0230712890625,
       "learning_rate": 4.146341463414634e-05,
-      "loss": 0.0031676713842898607,
-      "mean_token_accuracy": 0.9987796545028687,
-      "num_tokens": 641262.0,
       "step": 131
     },
     {
-      "entropy": 0.40411114878952503,
       "epoch": 1.6125574272588055,
-      "grad_norm": 0.0361328125,
       "learning_rate": 4.0243902439024395e-05,
-      "loss": 0.0015652105212211609,
-      "mean_token_accuracy": 0.999205507338047,
-      "num_tokens": 646375.0,
       "step": 132
     },
     {
-      "entropy": 0.3453770913183689,
       "epoch": 1.6248085758039816,
-      "grad_norm": 8.440017700195312e-05,
       "learning_rate": 3.9024390243902444e-05,
-      "loss": 3.279931843280792e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 650765.0,
       "step": 133
     },
     {
-      "entropy": 0.37724466249346733,
       "epoch": 1.6370597243491578,
-      "grad_norm": 0.00142669677734375,
       "learning_rate": 3.780487804878049e-05,
-      "loss": 5.4958236432867125e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 655167.0,
       "step": 134
     },
     {
-      "entropy": 0.39796170592308044,
       "epoch": 1.649310872894334,
-      "grad_norm": 0.0003986358642578125,
       "learning_rate": 3.6585365853658535e-05,
-      "loss": 3.9815466152504086e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 660288.0,
       "step": 135
     },
     {
-      "entropy": 0.4333613757044077,
       "epoch": 1.66156202143951,
-      "grad_norm": 0.0001544952392578125,
       "learning_rate": 3.5365853658536584e-05,
-      "loss": 4.787950456375256e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 664471.0,
       "step": 136
     },
     {
-      "entropy": 0.41916552372276783,
       "epoch": 1.673813169984686,
-      "grad_norm": 0.0002899169921875,
       "learning_rate": 3.414634146341464e-05,
-      "loss": 4.767990321852267e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 669354.0,
       "step": 137
     },
     {
-      "entropy": 0.3999825790524483,
       "epoch": 1.686064318529862,
-      "grad_norm": 0.0026397705078125,
       "learning_rate": 3.292682926829269e-05,
-      "loss": 0.0001605500146979466,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 674909.0,
       "step": 138
     },
     {
-      "entropy": 0.39421058259904385,
       "epoch": 1.6983154670750382,
-      "grad_norm": 0.005767822265625,
       "learning_rate": 3.170731707317073e-05,
-      "loss": 0.00022102531511336565,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 679690.0,
       "step": 139
     },
     {
-      "entropy": 0.4142182134091854,
       "epoch": 1.7105666156202144,
-      "grad_norm": 0.003631591796875,
       "learning_rate": 3.048780487804878e-05,
-      "loss": 0.00014472004841081798,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 685046.0,
       "step": 140
     },
     {
-      "entropy": 0.3982192352414131,
       "epoch": 1.7228177641653906,
-      "grad_norm": 0.00019168853759765625,
       "learning_rate": 2.926829268292683e-05,
-      "loss": 4.7273264499381185e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 689394.0,
       "step": 141
     },
     {
-      "entropy": 0.4133493732661009,
       "epoch": 1.7350689127105667,
-      "grad_norm": 0.00701904296875,
       "learning_rate": 2.8048780487804882e-05,
-      "loss": 9.296434291172773e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 693187.0,
       "step": 142
     },
     {
-      "entropy": 0.40933855436742306,
       "epoch": 1.7473200612557427,
-      "grad_norm": 0.0019683837890625,
       "learning_rate": 2.682926829268293e-05,
-      "loss": 9.476351988269016e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 697601.0,
       "step": 143
     },
     {
-      "entropy": 0.41714910976588726,
       "epoch": 1.7595712098009189,
-      "grad_norm": 0.045166015625,
       "learning_rate": 2.5609756097560977e-05,
-      "loss": 0.0034146099351346493,
-      "mean_token_accuracy": 0.9998650103807449,
-      "num_tokens": 703048.0,
       "step": 144
     },
     {
-      "entropy": 0.40594901144504547,
       "epoch": 1.7718223583460948,
-      "grad_norm": 0.02587890625,
       "learning_rate": 2.4390243902439026e-05,
-      "loss": 0.001274456619285047,
-      "mean_token_accuracy": 0.999015748500824,
-      "num_tokens": 707860.0,
       "step": 145
     },
     {
-      "entropy": 0.41635255329310894,
       "epoch": 1.784073506891271,
-      "grad_norm": 0.000156402587890625,
       "learning_rate": 2.3170731707317075e-05,
-      "loss": 5.037836672272533e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 712819.0,
       "step": 146
     },
     {
-      "entropy": 0.4038653904572129,
       "epoch": 1.7963246554364471,
-      "grad_norm": 0.0004100799560546875,
       "learning_rate": 2.1951219512195124e-05,
-      "loss": 4.163683479418978e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 718451.0,
       "step": 147
     },
     {
-      "entropy": 0.4069879539310932,
       "epoch": 1.8085758039816233,
-      "grad_norm": 7.104873657226562e-05,
       "learning_rate": 2.073170731707317e-05,
-      "loss": 3.6120818549534306e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 723808.0,
       "step": 148
     },
     {
-      "entropy": 0.4381860624998808,
       "epoch": 1.8208269525267995,
-      "grad_norm": 9.870529174804688e-05,
       "learning_rate": 1.9512195121951222e-05,
-      "loss": 4.433648064150475e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 728124.0,
       "step": 149
     },
     {
-      "entropy": 0.42220813781023026,
       "epoch": 1.8330781010719757,
-      "grad_norm": 0.005401611328125,
       "learning_rate": 1.8292682926829268e-05,
-      "loss": 0.00017107791791204363,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 733915.0,
       "step": 150
     },
     {
       "epoch": 1.8330781010719757,
-      "eval_entropy": 0.4000617520532746,
-      "eval_loss": 0.00023719228920526803,
-      "eval_mean_token_accuracy": 0.9998813841653906,
-      "eval_num_tokens": 733915.0,
-      "eval_runtime": 50.9031,
-      "eval_samples_per_second": 1.356,
-      "eval_steps_per_second": 1.356,
       "step": 150
     }
   ],
@@ -1560,7 +1560,7 @@
       "attributes": {}
     }
   },
-  "total_flos": 3.323255650111488e+16,
   "train_batch_size": 1,
   "trial_name": null,
   "trial_params": null

   "is_world_process_zero": true,
   "log_history": [
     {
+      "entropy": 0.2316489452496171,
       "epoch": 0.01225114854517611,
+      "grad_norm": 1.21875,
       "learning_rate": 0.0002,
+      "loss": 0.1141367182135582,
+      "mean_token_accuracy": 0.962372187525034,
+      "num_tokens": 6133.0,
       "step": 1
     },
     {
+      "entropy": 0.2494401354342699,
       "epoch": 0.02450229709035222,
+      "grad_norm": 0.59765625,
       "learning_rate": 0.00019878048780487805,
+      "loss": 0.07354862987995148,
+      "mean_token_accuracy": 0.9755491837859154,
+      "num_tokens": 12088.0,
       "step": 2
     },
     {
+      "entropy": 0.31152926199138165,
       "epoch": 0.036753445635528334,
+      "grad_norm": 0.306640625,
       "learning_rate": 0.0001975609756097561,
+      "loss": 0.06412772834300995,
+      "mean_token_accuracy": 0.978853102773428,
+      "num_tokens": 17331.0,
       "step": 3
     },
     {
+      "entropy": 0.30638211220502853,
       "epoch": 0.04900459418070444,
+      "grad_norm": 0.8984375,
       "learning_rate": 0.00019634146341463416,
+      "loss": 0.08034519106149673,
+      "mean_token_accuracy": 0.9723691493272781,
+      "num_tokens": 22383.0,
       "step": 4
     },
     {
+      "entropy": 0.3171741934493184,
       "epoch": 0.06125574272588055,
+      "grad_norm": 0.60546875,
       "learning_rate": 0.0001951219512195122,
+      "loss": 0.07083277404308319,
+      "mean_token_accuracy": 0.9742059484124184,
+      "num_tokens": 27930.0,
       "step": 5
     },
     {
+      "entropy": 0.3094687405973673,
       "epoch": 0.07350689127105667,
+      "grad_norm": 0.6796875,
       "learning_rate": 0.00019390243902439025,
+      "loss": 0.08443780243396759,
+      "mean_token_accuracy": 0.9732540361583233,
+      "num_tokens": 33286.0,
       "step": 6
     },
     {
+      "entropy": 0.2914603017270565,
       "epoch": 0.08575803981623277,
+      "grad_norm": 0.265625,
       "learning_rate": 0.0001926829268292683,
+      "loss": 0.06558080017566681,
+      "mean_token_accuracy": 0.9725310951471329,
+      "num_tokens": 39568.0,
       "step": 7
     },
     {
+      "entropy": 0.279434559866786,
       "epoch": 0.09800918836140889,
+      "grad_norm": 0.58984375,
       "learning_rate": 0.00019146341463414633,
+      "loss": 0.07338608056306839,
+      "mean_token_accuracy": 0.9793376848101616,
+      "num_tokens": 44597.0,
       "step": 8
     },
     {
+      "entropy": 0.27481516171246767,
       "epoch": 0.11026033690658499,
+      "grad_norm": 0.3125,
       "learning_rate": 0.0001902439024390244,
+      "loss": 0.06733334064483643,
+      "mean_token_accuracy": 0.9732998013496399,
+      "num_tokens": 49848.0,
       "step": 9
     },
     {
+      "entropy": 0.2752347318455577,
       "epoch": 0.1225114854517611,
+      "grad_norm": 0.4296875,
       "learning_rate": 0.00018902439024390244,
+      "loss": 0.08688339591026306,
+      "mean_token_accuracy": 0.9711812101304531,
+      "num_tokens": 55087.0,
       "step": 10
     },
     {
+      "entropy": 0.23697010707110167,
       "epoch": 0.13476263399693722,
+      "grad_norm": 0.35546875,
       "learning_rate": 0.0001878048780487805,
+      "loss": 0.09419302642345428,
+      "mean_token_accuracy": 0.9671205654740334,
+      "num_tokens": 61901.0,
       "step": 11
     },
     {
+      "entropy": 0.2767820842564106,
       "epoch": 0.14701378254211334,
+      "grad_norm": 0.5078125,
       "learning_rate": 0.00018658536585365856,
+      "loss": 0.09175145626068115,
+      "mean_token_accuracy": 0.9672112688422203,
+      "num_tokens": 68472.0,
       "step": 12
     },
     {
+      "entropy": 0.2712240917608142,
       "epoch": 0.15926493108728942,
+      "grad_norm": 0.43359375,
       "learning_rate": 0.0001853658536585366,
+      "loss": 0.1060388907790184,
+      "mean_token_accuracy": 0.9682641178369522,
+      "num_tokens": 74380.0,
       "step": 13
     },
     {
+      "entropy": 0.2655314621515572,
       "epoch": 0.17151607963246554,
+      "grad_norm": 0.5234375,
       "learning_rate": 0.00018414634146341464,
+      "loss": 0.09543660283088684,
+      "mean_token_accuracy": 0.9580898210406303,
+      "num_tokens": 80297.0,
       "step": 14
     },
     {
+      "entropy": 0.2568928087130189,
       "epoch": 0.18376722817764166,
+      "grad_norm": 0.306640625,
       "learning_rate": 0.0001829268292682927,
+      "loss": 0.05766459181904793,
+      "mean_token_accuracy": 0.9795842878520489,
+      "num_tokens": 85162.0,
       "step": 15
     },
     {
+      "entropy": 0.27691631484776735,
       "epoch": 0.19601837672281777,
+      "grad_norm": 0.359375,
       "learning_rate": 0.00018170731707317075,
+      "loss": 0.0939052402973175,
+      "mean_token_accuracy": 0.9671713933348656,
+      "num_tokens": 90393.0,
       "step": 16
     },
     {
+      "entropy": 0.2810298567637801,
       "epoch": 0.2082695252679939,
+      "grad_norm": 0.26953125,
       "learning_rate": 0.0001804878048780488,
+      "loss": 0.058892831206321716,
+      "mean_token_accuracy": 0.9773643910884857,
+      "num_tokens": 95530.0,
       "step": 17
     },
     {
+      "entropy": 0.2796283131465316,
       "epoch": 0.22052067381316998,
+      "grad_norm": 0.345703125,
       "learning_rate": 0.00017926829268292684,
+      "loss": 0.07744893431663513,
+      "mean_token_accuracy": 0.9721782878041267,
+      "num_tokens": 101234.0,
       "step": 18
     },
     {
+      "entropy": 0.2912421654909849,
       "epoch": 0.2327718223583461,
+      "grad_norm": 0.48828125,
       "learning_rate": 0.00017804878048780488,
+      "loss": 0.07593704760074615,
+      "mean_token_accuracy": 0.9668422974646091,
+      "num_tokens": 107018.0,
       "step": 19
     },
     {
+      "entropy": 0.28678335808217525,
       "epoch": 0.2450229709035222,
+      "grad_norm": 0.337890625,
       "learning_rate": 0.00017682926829268295,
+      "loss": 0.07227691262960434,
+      "mean_token_accuracy": 0.9736582525074482,
+      "num_tokens": 112299.0,
       "step": 20
     },
     {
+      "entropy": 0.296040833927691,
       "epoch": 0.2572741194486983,
+      "grad_norm": 0.33203125,
       "learning_rate": 0.000175609756097561,
+      "loss": 0.07230418920516968,
+      "mean_token_accuracy": 0.9750959761440754,
+      "num_tokens": 117872.0,
       "step": 21
     },
     {
+      "entropy": 0.27195548359304667,
       "epoch": 0.26952526799387444,
+      "grad_norm": 0.3671875,
       "learning_rate": 0.00017439024390243903,
+      "loss": 0.08706101030111313,
+      "mean_token_accuracy": 0.9771376326680183,
+      "num_tokens": 124580.0,
       "step": 22
     },
     {
+      "entropy": 0.29904199205338955,
       "epoch": 0.28177641653905056,
+      "grad_norm": 0.408203125,
       "learning_rate": 0.00017317073170731708,
+      "loss": 0.0653143897652626,
+      "mean_token_accuracy": 0.9760479032993317,
+      "num_tokens": 129745.0,
       "step": 23
     },
     {
+      "entropy": 0.2986137717962265,
       "epoch": 0.29402756508422667,
+      "grad_norm": 0.421875,
       "learning_rate": 0.00017195121951219512,
+      "loss": 0.07193314284086227,
+      "mean_token_accuracy": 0.9698839113116264,
+      "num_tokens": 135543.0,
       "step": 24
     },
     {
+      "entropy": 0.24683671910315752,
       "epoch": 0.30627871362940273,
+      "grad_norm": 0.37890625,
       "learning_rate": 0.0001707317073170732,
+      "loss": 0.07017349451780319,
+      "mean_token_accuracy": 0.9763788469135761,
+      "num_tokens": 141145.0,
       "step": 25
     },
     {
+      "entropy": 0.23581106960773468,
       "epoch": 0.31852986217457885,
+      "grad_norm": 0.349609375,
       "learning_rate": 0.00016951219512195123,
+      "loss": 0.07848861813545227,
+      "mean_token_accuracy": 0.9711455926299095,
+      "num_tokens": 146832.0,
       "step": 26
     },
     {
+      "entropy": 0.19877766259014606,
       "epoch": 0.33078101071975496,
+      "grad_norm": 0.32421875,
       "learning_rate": 0.00016829268292682927,
+      "loss": 0.05964134261012077,
+      "mean_token_accuracy": 0.9766620621085167,
+      "num_tokens": 153062.0,
       "step": 27
     },
     {
+      "entropy": 0.24412551056593657,
       "epoch": 0.3430321592649311,
+      "grad_norm": 0.466796875,
       "learning_rate": 0.00016707317073170731,
+      "loss": 0.10119230300188065,
+      "mean_token_accuracy": 0.9631960429251194,
+      "num_tokens": 159097.0,
       "step": 28
     },
     {
+      "entropy": 0.2634996743872762,
       "epoch": 0.3552833078101072,
+      "grad_norm": 0.376953125,
       "learning_rate": 0.00016585365853658536,
+      "loss": 0.07137235254049301,
+      "mean_token_accuracy": 0.9721279740333557,
+      "num_tokens": 164465.0,
       "step": 29
     },
     {
+      "entropy": 0.2398172626271844,
       "epoch": 0.3675344563552833,
+      "grad_norm": 0.380859375,
       "learning_rate": 0.00016463414634146343,
+      "loss": 0.08367905020713806,
+      "mean_token_accuracy": 0.9688702113926411,
+      "num_tokens": 171131.0,
       "step": 30
     },
     {
+      "entropy": 0.2387447776272893,
       "epoch": 0.37978560490045943,
+      "grad_norm": 0.39453125,
       "learning_rate": 0.00016341463414634147,
+      "loss": 0.07410822808742523,
+      "mean_token_accuracy": 0.9765294268727303,
+      "num_tokens": 175655.0,
       "step": 31
     },
     {
+      "entropy": 0.24556818418204784,
       "epoch": 0.39203675344563554,
+      "grad_norm": 0.361328125,
       "learning_rate": 0.00016219512195121954,
+      "loss": 0.07339000701904297,
+      "mean_token_accuracy": 0.9750400222837925,
+      "num_tokens": 182309.0,
       "step": 32
     },
     {
+      "entropy": 0.23958251252770424,
       "epoch": 0.40428790199081166,
+      "grad_norm": 0.376953125,
       "learning_rate": 0.00016097560975609758,
+      "loss": 0.0825161263346672,
+      "mean_token_accuracy": 0.9695910774171352,
+      "num_tokens": 188122.0,
       "step": 33
     },
     {
+      "entropy": 0.25066179782152176,
       "epoch": 0.4165390505359878,
+      "grad_norm": 0.34765625,
       "learning_rate": 0.00015975609756097562,
+      "loss": 0.0681036114692688,
+      "mean_token_accuracy": 0.9773549512028694,
+      "num_tokens": 193308.0,
       "step": 34
     },
     {
+      "entropy": 0.2489402163773775,
       "epoch": 0.42879019908116384,
+      "grad_norm": 0.33984375,
       "learning_rate": 0.00015853658536585366,
+      "loss": 0.07768924534320831,
+      "mean_token_accuracy": 0.9787707962095737,
+      "num_tokens": 198904.0,
       "step": 35
     },
     {
+      "entropy": 0.25176819786429405,
       "epoch": 0.44104134762633995,
+      "grad_norm": 0.353515625,
       "learning_rate": 0.00015731707317073173,
+      "loss": 0.07323021441698074,
+      "mean_token_accuracy": 0.9740425609052181,
+      "num_tokens": 204184.0,
       "step": 36
     },
     {
+      "entropy": 0.23491865396499634,
       "epoch": 0.45329249617151607,
+      "grad_norm": 0.345703125,
       "learning_rate": 0.00015609756097560978,
+      "loss": 0.06643179059028625,
+      "mean_token_accuracy": 0.9767155349254608,
+      "num_tokens": 210362.0,
       "step": 37
     },
     {
+      "entropy": 0.25266142282634974,
       "epoch": 0.4655436447166922,
+      "grad_norm": 0.50390625,
       "learning_rate": 0.00015487804878048782,
+      "loss": 0.08636192977428436,
+      "mean_token_accuracy": 0.9685244522988796,
+      "num_tokens": 215483.0,
       "step": 38
     },
     {
+      "entropy": 0.24919006042182446,
       "epoch": 0.4777947932618683,
+      "grad_norm": 0.357421875,
       "learning_rate": 0.00015365853658536586,
+      "loss": 0.06912290304899216,
+      "mean_token_accuracy": 0.9728152006864548,
+      "num_tokens": 220437.0,
       "step": 39
     },
     {
+      "entropy": 0.2789237005636096,
       "epoch": 0.4900459418070444,
+      "grad_norm": 0.3671875,
       "learning_rate": 0.0001524390243902439,
+      "loss": 0.07096827030181885,
+      "mean_token_accuracy": 0.9718564338982105,
+      "num_tokens": 225444.0,
       "step": 40
     },
     {
+      "entropy": 0.23915204405784607,
       "epoch": 0.5022970903522205,
+      "grad_norm": 0.35546875,
       "learning_rate": 0.00015121951219512197,
+      "loss": 0.06407603621482849,
+      "mean_token_accuracy": 0.975932989269495,
+      "num_tokens": 230088.0,
       "step": 41
     },
     {
+      "entropy": 0.25953691080212593,
       "epoch": 0.5145482388973966,
+      "grad_norm": 0.365234375,
       "learning_rate": 0.00015000000000000001,
+      "loss": 0.07893452048301697,
+      "mean_token_accuracy": 0.9717175625264645,
+      "num_tokens": 234974.0,
       "step": 42
     },
     {
+      "entropy": 0.25131134409457445,
       "epoch": 0.5267993874425727,
+      "grad_norm": 0.33984375,
       "learning_rate": 0.00014878048780487806,
+      "loss": 0.0724797397851944,
+      "mean_token_accuracy": 0.9746548496186733,
+      "num_tokens": 240695.0,
       "step": 43
     },
     {
+      "entropy": 0.25067666731774807,
       "epoch": 0.5390505359877489,
+      "grad_norm": 0.44921875,
       "learning_rate": 0.0001475609756097561,
+      "loss": 0.06145863234996796,
+      "mean_token_accuracy": 0.9786989763379097,
+      "num_tokens": 246515.0,
       "step": 44
     },
     {
+      "entropy": 0.22192941885441542,
       "epoch": 0.5513016845329249,
+      "grad_norm": 0.4375,
       "learning_rate": 0.00014634146341463414,
+      "loss": 0.06996186822652817,
+      "mean_token_accuracy": 0.9778482280671597,
+      "num_tokens": 252150.0,
       "step": 45
     },
     {
+      "entropy": 0.24868111684918404,
       "epoch": 0.5635528330781011,
+      "grad_norm": 0.392578125,
       "learning_rate": 0.0001451219512195122,
+      "loss": 0.07759839296340942,
+      "mean_token_accuracy": 0.9743853285908699,
+      "num_tokens": 257699.0,
       "step": 46
     },
     {
+      "entropy": 0.2405283828265965,
       "epoch": 0.5758039816232772,
+      "grad_norm": 0.400390625,
       "learning_rate": 0.00014390243902439025,
+      "loss": 0.06918229907751083,
+      "mean_token_accuracy": 0.9726257510483265,
+      "num_tokens": 262974.0,
       "step": 47
     },
     {
+      "entropy": 0.2463641557842493,
       "epoch": 0.5880551301684533,
+      "grad_norm": 0.5078125,
       "learning_rate": 0.0001426829268292683,
+      "loss": 0.08698121458292007,
+      "mean_token_accuracy": 0.9751730673015118,
+      "num_tokens": 267714.0,
       "step": 48
     },
     {
+      "entropy": 0.2611560570076108,
       "epoch": 0.6003062787136294,
+      "grad_norm": 0.3203125,
       "learning_rate": 0.00014146341463414634,
+      "loss": 0.0795765370130539,
+      "mean_token_accuracy": 0.9706047028303146,
+      "num_tokens": 273102.0,
       "step": 49
     },
     {
+      "entropy": 0.24631980434060097,
       "epoch": 0.6125574272588055,
+      "grad_norm": 0.365234375,
       "learning_rate": 0.00014024390243902438,
+      "loss": 0.06434721499681473,
+      "mean_token_accuracy": 0.9787219613790512,
+      "num_tokens": 278414.0,
       "step": 50
     },
     {
       "epoch": 0.6125574272588055,
+      "eval_entropy": 0.25439983627934387,
+      "eval_loss": 0.07568059861660004,
+      "eval_mean_token_accuracy": 0.9709554686062578,
+      "eval_num_tokens": 278414.0,
+      "eval_runtime": 56.679,
+      "eval_samples_per_second": 1.217,
+      "eval_steps_per_second": 1.217,
       "step": 50
     },
     {
+      "entropy": 0.22273720148950815,
       "epoch": 0.6248085758039816,
+      "grad_norm": 0.330078125,
       "learning_rate": 0.00013902439024390245,
+      "loss": 0.06272563338279724,
+      "mean_token_accuracy": 0.9790237173438072,
+      "num_tokens": 284001.0,
       "step": 51
     },
     {
+      "entropy": 0.25650967564433813,
       "epoch": 0.6370597243491577,
+      "grad_norm": 0.3515625,
       "learning_rate": 0.0001378048780487805,
+      "loss": 0.0695340633392334,
+      "mean_token_accuracy": 0.9723741784691811,
+      "num_tokens": 289900.0,
       "step": 52
     },
     {
+      "entropy": 0.27689922973513603,
       "epoch": 0.6493108728943339,
+      "grad_norm": 0.443359375,
       "learning_rate": 0.00013658536585365856,
+      "loss": 0.08247513324022293,
+      "mean_token_accuracy": 0.9751085750758648,
+      "num_tokens": 295774.0,
       "step": 53
     },
     {
+      "entropy": 0.24619914591312408,
       "epoch": 0.6615620214395099,
+      "grad_norm": 0.349609375,
       "learning_rate": 0.0001353658536585366,
+      "loss": 0.06673211604356766,
+      "mean_token_accuracy": 0.9788386225700378,
+      "num_tokens": 300970.0,
       "step": 54
     },
     {
+      "entropy": 0.27198443934321404,
       "epoch": 0.6738131699846861,
+      "grad_norm": 0.4921875,
       "learning_rate": 0.00013414634146341464,
+      "loss": 0.07676997035741806,
+      "mean_token_accuracy": 0.9696366749703884,
+      "num_tokens": 306709.0,
       "step": 55
     },
     {
+      "entropy": 0.2689105300232768,
       "epoch": 0.6860643185298622,
+      "grad_norm": 0.47265625,
       "learning_rate": 0.0001329268292682927,
+      "loss": 0.06719915568828583,
+      "mean_token_accuracy": 0.9702229462563992,
+      "num_tokens": 311650.0,
       "step": 56
     },
     {
+      "entropy": 0.2787257097661495,
       "epoch": 0.6983154670750383,
+      "grad_norm": 0.369140625,
       "learning_rate": 0.00013170731707317076,
+      "loss": 0.07159961760044098,
+      "mean_token_accuracy": 0.9748533591628075,
+      "num_tokens": 317257.0,
       "step": 57
     },
     {
+      "entropy": 0.2522663725540042,
       "epoch": 0.7105666156202144,
+      "grad_norm": 0.31640625,
       "learning_rate": 0.0001304878048780488,
+      "loss": 0.08856096863746643,
+      "mean_token_accuracy": 0.9697616137564182,
+      "num_tokens": 323281.0,
       "step": 58
     },
     {
+      "entropy": 0.24693416617810726,
       "epoch": 0.7228177641653905,
+      "grad_norm": 0.37109375,
       "learning_rate": 0.00012926829268292684,
+      "loss": 0.07423190027475357,
+      "mean_token_accuracy": 0.9705353751778603,
+      "num_tokens": 328551.0,
       "step": 59
     },
     {
+      "entropy": 0.2651137877255678,
       "epoch": 0.7350689127105666,
+      "grad_norm": 0.43359375,
       "learning_rate": 0.00012804878048780488,
+      "loss": 0.0738719031214714,
+      "mean_token_accuracy": 0.9752235859632492,
+      "num_tokens": 334143.0,
       "step": 60
     },
     {
+      "entropy": 0.2281778110191226,
       "epoch": 0.7473200612557427,
+      "grad_norm": 0.2490234375,
       "learning_rate": 0.00012682926829268293,
+      "loss": 0.0633026584982872,
+      "mean_token_accuracy": 0.9741999059915543,
+      "num_tokens": 341092.0,
       "step": 61
     },
     {
+      "entropy": 0.2535929596051574,
       "epoch": 0.7595712098009189,
+      "grad_norm": 0.390625,
       "learning_rate": 0.000125609756097561,
+      "loss": 0.0719546377658844,
+      "mean_token_accuracy": 0.9765410870313644,
+      "num_tokens": 347467.0,
       "step": 62
     },
     {
+      "entropy": 0.25424638390541077,
       "epoch": 0.7718223583460949,
+      "grad_norm": 0.431640625,
       "learning_rate": 0.00012439024390243904,
+      "loss": 0.05535401031374931,
+      "mean_token_accuracy": 0.9780425503849983,
+      "num_tokens": 352164.0,
       "step": 63
     },
     {
+      "entropy": 0.23888325225561857,
       "epoch": 0.7840735068912711,
+      "grad_norm": 0.435546875,
       "learning_rate": 0.00012317073170731708,
+      "loss": 0.07177040725946426,
+      "mean_token_accuracy": 0.9734687805175781,
+      "num_tokens": 357308.0,
       "step": 64
     },
     {
+      "entropy": 0.27028472628444433,
       "epoch": 0.7963246554364471,
+      "grad_norm": 0.30859375,
       "learning_rate": 0.00012195121951219512,
+      "loss": 0.06257087737321854,
+      "mean_token_accuracy": 0.9774579927325249,
+      "num_tokens": 362666.0,
       "step": 65
     },
     {
+      "entropy": 0.2821849435567856,
       "epoch": 0.8085758039816233,
+      "grad_norm": 0.373046875,
       "learning_rate": 0.00012073170731707318,
+      "loss": 0.06471723318099976,
+      "mean_token_accuracy": 0.976191334426403,
+      "num_tokens": 368427.0,
       "step": 66
     },
     {
+      "entropy": 0.22186184907332063,
       "epoch": 0.8208269525267994,
+      "grad_norm": 0.263671875,
       "learning_rate": 0.00011951219512195122,
+      "loss": 0.06329935044050217,
+      "mean_token_accuracy": 0.978707954287529,
+      "num_tokens": 374540.0,
       "step": 67
     },
     {
+      "entropy": 0.23882555402815342,
       "epoch": 0.8330781010719756,
+      "grad_norm": 0.3046875,
       "learning_rate": 0.00011829268292682926,
+      "loss": 0.07082124054431915,
+      "mean_token_accuracy": 0.979393869638443,
+      "num_tokens": 379925.0,
       "step": 68
     },
     {
+      "entropy": 0.2527451729401946,
       "epoch": 0.8453292496171516,
+      "grad_norm": 0.37109375,
       "learning_rate": 0.00011707317073170732,
+      "loss": 0.0804731696844101,
+      "mean_token_accuracy": 0.9763551540672779,
+      "num_tokens": 384279.0,
       "step": 69
     },
     {
+      "entropy": 0.26056139171123505,
       "epoch": 0.8575803981623277,
+      "grad_norm": 0.40234375,
       "learning_rate": 0.00011585365853658536,
+      "loss": 0.09266315400600433,
+      "mean_token_accuracy": 0.9709281474351883,
+      "num_tokens": 389563.0,
       "step": 70
     },
     {
+      "entropy": 0.2919591320678592,
       "epoch": 0.8698315467075038,
+      "grad_norm": 0.423828125,
       "learning_rate": 0.00011463414634146342,
+      "loss": 0.07172521948814392,
+      "mean_token_accuracy": 0.9725044220685959,
+      "num_tokens": 394650.0,
       "step": 71
     },
     {
+      "entropy": 0.2520558973774314,
       "epoch": 0.8820826952526799,
+      "grad_norm": 0.47265625,
       "learning_rate": 0.00011341463414634146,
+      "loss": 0.07857581228017807,
+      "mean_token_accuracy": 0.967189610004425,
+      "num_tokens": 399583.0,
       "step": 72
     },
     {
+      "entropy": 0.2681189738214016,
       "epoch": 0.8943338437978561,
+      "grad_norm": 0.470703125,
       "learning_rate": 0.00011219512195121953,
+      "loss": 0.0883592814207077,
+      "mean_token_accuracy": 0.9760300181806087,
+      "num_tokens": 406224.0,
       "step": 73
     },
     {
+      "entropy": 0.25226688850671053,
       "epoch": 0.9065849923430321,
+      "grad_norm": 0.349609375,
       "learning_rate": 0.00011097560975609757,
+      "loss": 0.06107043847441673,
+      "mean_token_accuracy": 0.9742026180028915,
+      "num_tokens": 412481.0,
       "step": 74
     },
     {
+      "entropy": 0.25610699970275164,
       "epoch": 0.9188361408882083,
+      "grad_norm": 0.4140625,
       "learning_rate": 0.00010975609756097563,
+      "loss": 0.06678957492113113,
+      "mean_token_accuracy": 0.9725399203598499,
+      "num_tokens": 417862.0,
       "step": 75
     },
     {
+      "entropy": 0.2826196616515517,
       "epoch": 0.9310872894333844,
+      "grad_norm": 0.859375,
       "learning_rate": 0.00010853658536585367,
+      "loss": 0.048859648406505585,
+      "mean_token_accuracy": 0.9790267050266266,
+      "num_tokens": 422878.0,
       "step": 76
     },
     {
+      "entropy": 0.23871563002467155,
       "epoch": 0.9433384379785605,
+      "grad_norm": 0.466796875,
       "learning_rate": 0.00010731707317073172,
+      "loss": 0.07596343755722046,
+      "mean_token_accuracy": 0.971769668161869,
+      "num_tokens": 429170.0,
       "step": 77
     },
     {
+      "entropy": 0.2777755409479141,
       "epoch": 0.9555895865237366,
+      "grad_norm": 0.443359375,
       "learning_rate": 0.00010609756097560977,
+      "loss": 0.06630191206932068,
+      "mean_token_accuracy": 0.9747902825474739,
+      "num_tokens": 434323.0,
       "step": 78
     },
     {
+      "entropy": 0.23950364720076323,
       "epoch": 0.9678407350689127,
+      "grad_norm": 0.349609375,
       "learning_rate": 0.00010487804878048781,
+      "loss": 0.057458702474832535,
+      "mean_token_accuracy": 0.980991818010807,
+      "num_tokens": 439539.0,
       "step": 79
     },
     {
+      "entropy": 0.245719694532454,
       "epoch": 0.9800918836140888,
+      "grad_norm": 0.3046875,
       "learning_rate": 0.00010365853658536586,
+      "loss": 0.06474918127059937,
+      "mean_token_accuracy": 0.9749566093087196,
+      "num_tokens": 445548.0,
       "step": 80
     },
     {
+      "entropy": 0.2553516002371907,
       "epoch": 0.9923430321592649,
+      "grad_norm": 0.59375,
       "learning_rate": 0.0001024390243902439,
+      "loss": 0.07626976072788239,
+      "mean_token_accuracy": 0.9740116000175476,
+      "num_tokens": 451007.0,
       "step": 81
     },
     {
+      "entropy": 0.24858922958374025,
       "epoch": 1.0,
+      "grad_norm": 0.4140625,
       "learning_rate": 0.00010121951219512196,
+      "loss": 0.05956536903977394,
+      "mean_token_accuracy": 0.9751910209655762,
+      "num_tokens": 454678.0,
       "step": 82
     },
     {
+      "entropy": 0.22480082791298628,
       "epoch": 1.0122511485451762,
+      "grad_norm": 0.302734375,
       "learning_rate": 0.0001,
+      "loss": 0.03318095952272415,
+      "mean_token_accuracy": 0.9908282831311226,
+      "num_tokens": 460195.0,
       "step": 83
     },
     {
+      "entropy": 0.21941375825554132,
       "epoch": 1.0245022970903521,
+      "grad_norm": 0.322265625,
       "learning_rate": 9.878048780487805e-05,
+      "loss": 0.037562280893325806,
+      "mean_token_accuracy": 0.9899826981127262,
+      "num_tokens": 465814.0,
       "step": 84
     },
     {
+      "entropy": 0.2297668270766735,
       "epoch": 1.0367534456355283,
+      "grad_norm": 0.259765625,
       "learning_rate": 9.75609756097561e-05,
+      "loss": 0.03667337819933891,
+      "mean_token_accuracy": 0.9867987670004368,
+      "num_tokens": 472919.0,
       "step": 85
     },
     {
+      "entropy": 0.1959990761242807,
       "epoch": 1.0490045941807045,
+      "grad_norm": 0.171875,
       "learning_rate": 9.634146341463415e-05,
+      "loss": 0.02224677987396717,
+      "mean_token_accuracy": 0.9947787970304489,
+      "num_tokens": 477926.0,
       "step": 86
     },
     {
+      "entropy": 0.22538460325449705,
       "epoch": 1.0612557427258806,
+      "grad_norm": 0.294921875,
       "learning_rate": 9.51219512195122e-05,
+      "loss": 0.05467130243778229,
+      "mean_token_accuracy": 0.9857094436883926,
+      "num_tokens": 483369.0,
       "step": 87
     },
     {
+      "entropy": 0.2385974396020174,
       "epoch": 1.0735068912710566,
+      "grad_norm": 0.2392578125,
       "learning_rate": 9.390243902439024e-05,
+      "loss": 0.02876465395092964,
+      "mean_token_accuracy": 0.9933567047119141,
+      "num_tokens": 488048.0,
       "step": 88
     },
     {
+      "entropy": 0.2244573337957263,
       "epoch": 1.0857580398162328,
+      "grad_norm": 0.17578125,
       "learning_rate": 9.26829268292683e-05,
+      "loss": 0.022544220089912415,
+      "mean_token_accuracy": 0.9952267222106457,
+      "num_tokens": 492951.0,
       "step": 89
     },
     {
+      "entropy": 0.21164159616455436,
       "epoch": 1.098009188361409,
+      "grad_norm": 0.3671875,
       "learning_rate": 9.146341463414635e-05,
+      "loss": 0.0307400431483984,
+      "mean_token_accuracy": 0.9898485280573368,
+      "num_tokens": 498298.0,
       "step": 90
     },
     {
+      "entropy": 0.22300960402935743,
       "epoch": 1.110260336906585,
+      "grad_norm": 0.25390625,
       "learning_rate": 9.02439024390244e-05,
+      "loss": 0.02349678799510002,
+      "mean_token_accuracy": 0.9937595501542091,
+      "num_tokens": 503013.0,
       "step": 91
     },
     {
+      "entropy": 0.2144601820036769,
       "epoch": 1.122511485451761,
+      "grad_norm": 0.466796875,
       "learning_rate": 8.902439024390244e-05,
+      "loss": 0.025124385952949524,
+      "mean_token_accuracy": 0.9929902292788029,
+      "num_tokens": 507687.0,
       "step": 92
     },
     {
+      "entropy": 0.18067707447335124,
       "epoch": 1.1347626339969372,
+      "grad_norm": 0.462890625,
       "learning_rate": 8.78048780487805e-05,
+      "loss": 0.04210633784532547,
+      "mean_token_accuracy": 0.9874051883816719,
+      "num_tokens": 513217.0,
       "step": 93
     },
     {
+      "entropy": 0.18840790819376707,
       "epoch": 1.1470137825421134,
+      "grad_norm": 0.2578125,
       "learning_rate": 8.658536585365854e-05,
+      "loss": 0.023590605705976486,
+      "mean_token_accuracy": 0.9930241219699383,
+      "num_tokens": 518384.0,
       "step": 94
     },
     {
+      "entropy": 0.16844777530059218,
       "epoch": 1.1592649310872893,
+      "grad_norm": 0.3046875,
       "learning_rate": 8.53658536585366e-05,
+      "loss": 0.02408467046916485,
+      "mean_token_accuracy": 0.9940578565001488,
+      "num_tokens": 523975.0,
       "step": 95
     },
     {
+      "entropy": 0.1988551402464509,
       "epoch": 1.1715160796324655,
+      "grad_norm": 0.25390625,
       "learning_rate": 8.414634146341464e-05,
+      "loss": 0.01896364614367485,
+      "mean_token_accuracy": 0.9935651384294033,
+      "num_tokens": 528838.0,
       "step": 96
     },
     {
+      "entropy": 0.19662938080728054,
       "epoch": 1.1837672281776417,
+      "grad_norm": 0.271484375,
       "learning_rate": 8.292682926829268e-05,
+      "loss": 0.023568641394376755,
+      "mean_token_accuracy": 0.9942812882363796,
+      "num_tokens": 533723.0,
       "step": 97
     },
     {
+      "entropy": 0.18521032202988863,
       "epoch": 1.1960183767228179,
+      "grad_norm": 0.2158203125,
       "learning_rate": 8.170731707317073e-05,
+      "loss": 0.03203809633851051,
+      "mean_token_accuracy": 0.9899982325732708,
+      "num_tokens": 540180.0,
       "step": 98
     },
     {
+      "entropy": 0.18826917372643948,
       "epoch": 1.2082695252679938,
+      "grad_norm": 0.4765625,
       "learning_rate": 8.048780487804879e-05,
+      "loss": 0.03463224321603775,
+      "mean_token_accuracy": 0.9889252111315727,
+      "num_tokens": 546618.0,
       "step": 99
     },
     {
+      "entropy": 0.1889605624601245,
       "epoch": 1.22052067381317,
+      "grad_norm": 0.337890625,
       "learning_rate": 7.926829268292683e-05,
+      "loss": 0.038746241480112076,
+      "mean_token_accuracy": 0.9897148124873638,
+      "num_tokens": 552084.0,
       "step": 100
     },
     {
       "epoch": 1.22052067381317,
+      "eval_entropy": 0.19684839270253113,
+      "eval_loss": 0.08200085908174515,
+      "eval_mean_token_accuracy": 0.9706140955289205,
+      "eval_num_tokens": 552084.0,
+      "eval_runtime": 56.6368,
+      "eval_samples_per_second": 1.218,
+      "eval_steps_per_second": 1.218,
       "step": 100
     },
     {
+      "entropy": 0.18781481962651014,
       "epoch": 1.2327718223583461,
+      "grad_norm": 0.2490234375,
       "learning_rate": 7.804878048780489e-05,
+      "loss": 0.03647669032216072,
+      "mean_token_accuracy": 0.9900195822119713,
+      "num_tokens": 558384.0,
       "step": 101
     },
     {
+      "entropy": 0.182833943516016,
       "epoch": 1.245022970903522,
+      "grad_norm": 0.1708984375,
       "learning_rate": 7.682926829268293e-05,
+      "loss": 0.01754325069487095,
+      "mean_token_accuracy": 0.9952104948461056,
+      "num_tokens": 564025.0,
       "step": 102
     },
     {
+      "entropy": 0.19512099027633667,
       "epoch": 1.2572741194486983,
+      "grad_norm": 0.32421875,
       "learning_rate": 7.560975609756099e-05,
+      "loss": 0.045042332261800766,
+      "mean_token_accuracy": 0.987647294998169,
+      "num_tokens": 569791.0,
       "step": 103
     },
     {
+      "entropy": 0.19775146059691906,
       "epoch": 1.2695252679938744,
+      "grad_norm": 0.287109375,
       "learning_rate": 7.439024390243903e-05,
+      "loss": 0.03481469675898552,
+      "mean_token_accuracy": 0.9876400642096996,
+      "num_tokens": 575432.0,
       "step": 104
     },
     {
+      "entropy": 0.19757689163088799,
       "epoch": 1.2817764165390506,
+      "grad_norm": 0.392578125,
       "learning_rate": 7.317073170731707e-05,
+      "loss": 0.045782968401908875,
+      "mean_token_accuracy": 0.987156193703413,
+      "num_tokens": 580586.0,
       "step": 105
     },
     {
+      "entropy": 0.19568088464438915,
       "epoch": 1.2940275650842268,
+      "grad_norm": 0.271484375,
       "learning_rate": 7.195121951219513e-05,
+      "loss": 0.03614577651023865,
+      "mean_token_accuracy": 0.989520326256752,
+      "num_tokens": 586255.0,
       "step": 106
     },
     {
+      "entropy": 0.18891402333974838,
       "epoch": 1.3062787136294027,
+      "grad_norm": 0.169921875,
       "learning_rate": 7.073170731707317e-05,
+      "loss": 0.018318383023142815,
+      "mean_token_accuracy": 0.9943608231842518,
+      "num_tokens": 591734.0,
       "step": 107
     },
     {
+      "entropy": 0.2118115657940507,
       "epoch": 1.318529862174579,
+      "grad_norm": 0.34375,
       "learning_rate": 6.951219512195122e-05,
+      "loss": 0.02556736022233963,
+      "mean_token_accuracy": 0.9910119064152241,
+      "num_tokens": 596805.0,
       "step": 108
     },
     {
+      "entropy": 0.20146753964945674,
       "epoch": 1.3307810107197549,
+      "grad_norm": 0.251953125,
       "learning_rate": 6.829268292682928e-05,
+      "loss": 0.026423780247569084,
+      "mean_token_accuracy": 0.9911187067627907,
+      "num_tokens": 602469.0,
       "step": 109
     },
     {
+      "entropy": 0.19927682168781757,
       "epoch": 1.343032159264931,
+      "grad_norm": 0.2314453125,
       "learning_rate": 6.707317073170732e-05,
+      "loss": 0.038182880729436874,
+      "mean_token_accuracy": 0.9882474392652512,
+      "num_tokens": 608854.0,
       "step": 110
     },
     {
+      "entropy": 0.18457680894061923,
       "epoch": 1.3552833078101072,
+      "grad_norm": 0.24609375,
       "learning_rate": 6.585365853658538e-05,
+      "loss": 0.025912806391716003,
+      "mean_token_accuracy": 0.9923904649913311,
+      "num_tokens": 614272.0,
       "step": 111
     },
     {
+      "entropy": 0.1993693085387349,
       "epoch": 1.3675344563552834,
+      "grad_norm": 0.291015625,
       "learning_rate": 6.463414634146342e-05,
+      "loss": 0.021378764882683754,
+      "mean_token_accuracy": 0.9953300580382347,
+      "num_tokens": 619446.0,
       "step": 112
     },
     {
+      "entropy": 0.19518085662275553,
       "epoch": 1.3797856049004595,
+      "grad_norm": 0.30078125,
       "learning_rate": 6.341463414634146e-05,
+      "loss": 0.03335938975214958,
+      "mean_token_accuracy": 0.9875492453575134,
+      "num_tokens": 625774.0,
       "step": 113
     },
     {
+      "entropy": 0.20890573505312204,
       "epoch": 1.3920367534456355,
+      "grad_norm": 0.373046875,
       "learning_rate": 6.219512195121952e-05,
+      "loss": 0.036217525601387024,
+      "mean_token_accuracy": 0.9891358688473701,
+      "num_tokens": 630747.0,
       "step": 114
     },
     {
+      "entropy": 0.19118426740169525,
       "epoch": 1.4042879019908117,
+      "grad_norm": 0.251953125,
       "learning_rate": 6.097560975609756e-05,
+      "loss": 0.030090918764472008,
+      "mean_token_accuracy": 0.9934539385139942,
+      "num_tokens": 637405.0,
       "step": 115
     },
     {
+      "entropy": 0.2176859974861145,
       "epoch": 1.4165390505359878,
+      "grad_norm": 0.2373046875,
       "learning_rate": 5.975609756097561e-05,
+      "loss": 0.024563392624258995,
+      "mean_token_accuracy": 0.9921185150742531,
+      "num_tokens": 642328.0,
       "step": 116
     },
     {
+      "entropy": 0.1849509342573583,
       "epoch": 1.4287901990811638,
+      "grad_norm": 0.35546875,
       "learning_rate": 5.853658536585366e-05,
+      "loss": 0.042349379509687424,
+      "mean_token_accuracy": 0.9899747557938099,
+      "num_tokens": 647857.0,
       "step": 117
     },
     {
+      "entropy": 0.19377889391034842,
       "epoch": 1.44104134762634,
+      "grad_norm": 0.279296875,
       "learning_rate": 5.731707317073171e-05,
+      "loss": 0.02413174696266651,
+      "mean_token_accuracy": 0.9931157529354095,
+      "num_tokens": 653805.0,
       "step": 118
     },
     {
+      "entropy": 0.20709845190867782,
       "epoch": 1.4532924961715161,
+      "grad_norm": 0.28125,
       "learning_rate": 5.6097560975609764e-05,
+      "loss": 0.03505600988864899,
+      "mean_token_accuracy": 0.9896740056574345,
+      "num_tokens": 659708.0,
       "step": 119
     },
     {
+      "entropy": 0.20671271299943328,
       "epoch": 1.4655436447166923,
+      "grad_norm": 0.2734375,
       "learning_rate": 5.487804878048781e-05,
+      "loss": 0.02634236589074135,
+      "mean_token_accuracy": 0.9935285076498985,
+      "num_tokens": 665292.0,
       "step": 120
     },
     {
+      "entropy": 0.18826642259955406,
       "epoch": 1.4777947932618682,
+      "grad_norm": 0.2177734375,
       "learning_rate": 5.365853658536586e-05,
+      "loss": 0.022179996594786644,
+      "mean_token_accuracy": 0.9928314089775085,
+      "num_tokens": 670669.0,
       "step": 121
     },
     {
+      "entropy": 0.2311026845127344,
       "epoch": 1.4900459418070444,
+      "grad_norm": 0.267578125,
       "learning_rate": 5.2439024390243904e-05,
+      "loss": 0.025521911680698395,
+      "mean_token_accuracy": 0.9930035471916199,
+      "num_tokens": 675524.0,
       "step": 122
     },
     {
+      "entropy": 0.1890636207535863,
       "epoch": 1.5022970903522204,
+      "grad_norm": 0.22265625,
       "learning_rate": 5.121951219512195e-05,
+      "loss": 0.02293182723224163,
+      "mean_token_accuracy": 0.9917827062308788,
+      "num_tokens": 681083.0,
       "step": 123
     },
     {
+      "entropy": 0.20301904529333115,
       "epoch": 1.5145482388973965,
+      "grad_norm": 0.251953125,
       "learning_rate": 5e-05,
+      "loss": 0.026392869651317596,
+      "mean_token_accuracy": 0.9935696609318256,
+      "num_tokens": 686909.0,
       "step": 124
     },
     {
+      "entropy": 0.18326633982360363,
       "epoch": 1.5267993874425727,
+      "grad_norm": 0.189453125,
       "learning_rate": 4.878048780487805e-05,
+      "loss": 0.03385050222277641,
+      "mean_token_accuracy": 0.9923080727458,
+      "num_tokens": 693716.0,
       "step": 125
     },
     {
+      "entropy": 0.1940352749079466,
       "epoch": 1.5390505359877489,
+      "grad_norm": 0.25,
       "learning_rate": 4.75609756097561e-05,
+      "loss": 0.03128973767161369,
+      "mean_token_accuracy": 0.9904795847833157,
+      "num_tokens": 699231.0,
       "step": 126
     },
     {
+      "entropy": 0.2052145255729556,
       "epoch": 1.551301684532925,
+      "grad_norm": 0.1962890625,
       "learning_rate": 4.634146341463415e-05,
+      "loss": 0.01906367763876915,
+      "mean_token_accuracy": 0.9935221113264561,
+      "num_tokens": 705026.0,
       "step": 127
     },
     {
+      "entropy": 0.22084870096296072,
       "epoch": 1.5635528330781012,
+      "grad_norm": 0.28125,
       "learning_rate": 4.51219512195122e-05,
+      "loss": 0.026771627366542816,
+      "mean_token_accuracy": 0.9931596331298351,
+      "num_tokens": 710155.0,
       "step": 128
     },
     {
+      "entropy": 0.18041892955079675,
       "epoch": 1.5758039816232772,
+      "grad_norm": 0.369140625,
       "learning_rate": 4.390243902439025e-05,
+      "loss": 0.024752795696258545,
+      "mean_token_accuracy": 0.9915198720991611,
+      "num_tokens": 715496.0,
       "step": 129
     },
     {
+      "entropy": 0.1869538608007133,
       "epoch": 1.5880551301684533,
+      "grad_norm": 0.3046875,
       "learning_rate": 4.26829268292683e-05,
+      "loss": 0.03293408453464508,
+      "mean_token_accuracy": 0.990137055516243,
+      "num_tokens": 721491.0,
       "step": 130
     },
     {
+      "entropy": 0.20515098702162504,
       "epoch": 1.6003062787136293,
+      "grad_norm": 0.349609375,
       "learning_rate": 4.146341463414634e-05,
+      "loss": 0.023330464959144592,
+      "mean_token_accuracy": 0.9892629720270634,
+      "num_tokens": 726673.0,
       "step": 131
     },
     {
+      "entropy": 0.18135815067216754,
       "epoch": 1.6125574272588055,
+      "grad_norm": 0.357421875,
       "learning_rate": 4.0243902439024395e-05,
+      "loss": 0.03119005262851715,
+      "mean_token_accuracy": 0.9911304786801338,
+      "num_tokens": 733054.0,
       "step": 132
     },
     {
+      "entropy": 0.20070009911432862,
       "epoch": 1.6248085758039816,
+      "grad_norm": 0.21484375,
       "learning_rate": 3.9024390243902444e-05,
+      "loss": 0.030009731650352478,
+      "mean_token_accuracy": 0.9932212419807911,
+      "num_tokens": 737990.0,
       "step": 133
     },
     {
+      "entropy": 0.18819584511220455,
       "epoch": 1.6370597243491578,
+      "grad_norm": 0.2451171875,
       "learning_rate": 3.780487804878049e-05,
+      "loss": 0.02752860262989998,
+      "mean_token_accuracy": 0.9897669702768326,
+      "num_tokens": 743394.0,
       "step": 134
     },
     {
+      "entropy": 0.18869836069643497,
       "epoch": 1.649310872894334,
+      "grad_norm": 0.240234375,
       "learning_rate": 3.6585365853658535e-05,
+      "loss": 0.03194504603743553,
+      "mean_token_accuracy": 0.9914098270237446,
+      "num_tokens": 749356.0,
       "step": 135
     },
     {
+      "entropy": 0.2093992899172008,
       "epoch": 1.66156202143951,
+      "grad_norm": 0.291015625,
       "learning_rate": 3.5365853658536584e-05,
+      "loss": 0.02633955329656601,
+      "mean_token_accuracy": 0.992473166435957,
+      "num_tokens": 754312.0,
       "step": 136
     },
     {
+      "entropy": 0.1928223273716867,
       "epoch": 1.673813169984686,
+      "grad_norm": 0.2470703125,
       "learning_rate": 3.414634146341464e-05,
+      "loss": 0.035037778317928314,
+      "mean_token_accuracy": 0.9916842468082905,
+      "num_tokens": 760182.0,
       "step": 137
     },
     {
+      "entropy": 0.19663999788463116,
       "epoch": 1.686064318529862,
+      "grad_norm": 0.265625,
       "learning_rate": 3.292682926829269e-05,
+      "loss": 0.03151565045118332,
+      "mean_token_accuracy": 0.9930234625935555,
+      "num_tokens": 766267.0,
       "step": 138
     },
     {
+      "entropy": 0.2058473015204072,
       "epoch": 1.6983154670750382,
+      "grad_norm": 0.2578125,
       "learning_rate": 3.170731707317073e-05,
+      "loss": 0.02509160526096821,
+      "mean_token_accuracy": 0.9920520819723606,
+      "num_tokens": 771135.0,
       "step": 139
     },
     {
+      "entropy": 0.20955495908856392,
       "epoch": 1.7105666156202144,
+      "grad_norm": 0.36328125,
       "learning_rate": 3.048780487804878e-05,
+      "loss": 0.03856905177235603,
+      "mean_token_accuracy": 0.9877506978809834,
+      "num_tokens": 776727.0,
       "step": 140
     },
     {
+      "entropy": 0.17796193715184927,
       "epoch": 1.7228177641653906,
+      "grad_norm": 0.271484375,
       "learning_rate": 2.926829268292683e-05,
+      "loss": 0.03061492368578911,
+      "mean_token_accuracy": 0.9933489374816418,
+      "num_tokens": 782352.0,
       "step": 141
     },
     {
+      "entropy": 0.19299636129289865,
       "epoch": 1.7350689127105667,
+      "grad_norm": 0.2392578125,
       "learning_rate": 2.8048780487804882e-05,
+      "loss": 0.03383423760533333,
+      "mean_token_accuracy": 0.9913677796721458,
+      "num_tokens": 787139.0,
       "step": 142
     },
     {
+      "entropy": 0.2032350143417716,
       "epoch": 1.7473200612557427,
+      "grad_norm": 0.314453125,
       "learning_rate": 2.682926829268293e-05,
+      "loss": 0.03458622097969055,
+      "mean_token_accuracy": 0.9920257851481438,
+      "num_tokens": 792244.0,
       "step": 143
     },
     {
+      "entropy": 0.21589675825089216,
       "epoch": 1.7595712098009189,
+      "grad_norm": 0.27734375,
       "learning_rate": 2.5609756097560977e-05,
+      "loss": 0.029654916375875473,
+      "mean_token_accuracy": 0.9936717823147774,
+      "num_tokens": 797998.0,
       "step": 144
     },
     {
+      "entropy": 0.19791326764971018,
       "epoch": 1.7718223583460948,
+      "grad_norm": 0.1748046875,
       "learning_rate": 2.4390243902439026e-05,
+      "loss": 0.019491517916321754,
+      "mean_token_accuracy": 0.9953687153756618,
+      "num_tokens": 803118.0,
       "step": 145
     },
     {
+      "entropy": 0.19606765313073993,
       "epoch": 1.784073506891271,
+      "grad_norm": 0.2236328125,
       "learning_rate": 2.3170731707317075e-05,
+      "loss": 0.017046257853507996,
+      "mean_token_accuracy": 0.9934666827321053,
+      "num_tokens": 808709.0,
       "step": 146
     },
     {
+      "entropy": 0.17984948493540287,
       "epoch": 1.7963246554364471,
+      "grad_norm": 0.2119140625,
       "learning_rate": 2.1951219512195124e-05,
+      "loss": 0.028008146211504936,
+      "mean_token_accuracy": 0.9918750263750553,
+      "num_tokens": 815053.0,
       "step": 147
     },
     {
+      "entropy": 0.19215012807399035,
       "epoch": 1.8085758039816233,
+      "grad_norm": 0.212890625,
       "learning_rate": 2.073170731707317e-05,
+      "loss": 0.02620745822787285,
+      "mean_token_accuracy": 0.9895812347531319,
+      "num_tokens": 821046.0,
       "step": 148
     },
     {
+      "entropy": 0.1954274857416749,
       "epoch": 1.8208269525267995,
+      "grad_norm": 0.1630859375,
       "learning_rate": 1.9512195121951222e-05,
+      "loss": 0.012469938956201077,
+      "mean_token_accuracy": 0.9970379211008549,
+      "num_tokens": 825773.0,
       "step": 149
     },
     {
+      "entropy": 0.20444792695343494,
       "epoch": 1.8330781010719757,
+      "grad_norm": 0.3671875,
       "learning_rate": 1.8292682926829268e-05,
+      "loss": 0.029102876782417297,
+      "mean_token_accuracy": 0.9916210547089577,
+      "num_tokens": 831944.0,
       "step": 150
     },
     {
       "epoch": 1.8330781010719757,
+      "eval_entropy": 0.20245846825233405,
+      "eval_loss": 0.07568201422691345,
+      "eval_mean_token_accuracy": 0.973983341369076,
+      "eval_num_tokens": 831944.0,
+      "eval_runtime": 56.7259,
+      "eval_samples_per_second": 1.216,
+      "eval_steps_per_second": 1.216,
       "step": 150
     }
   ],
       "attributes": {}
     }
   },
+  "total_flos": 3.767142787075277e+16,
   "train_batch_size": 1,
   "trial_name": null,
   "trial_params": null

checkpoint-164/adapter_config.json CHANGED Viewed

@@ -29,13 +29,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
     "gate_proj",
-    "up_proj",
-    "q_proj",
-    "v_proj",
     "k_proj",
     "down_proj",
-    "o_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "o_proj",
     "gate_proj",
     "k_proj",
+    "q_proj",
     "down_proj",
+    "v_proj",
+    "up_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

checkpoint-164/adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:992684d6ec4153831df4e539107495d771d85ab5ab3a998ef80302393087065c
 size 83946192

 version https://git-lfs.github.com/spec/v1
+oid sha256:1a30df352e3e5bf7a3be3ccca5e0bf0b3a9b19ac4eb509a3b6c3fbbccdd879fb
 size 83946192

checkpoint-164/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7e449096d3a07f08f22b0b85be61c0b047450894cb70e29f590dfe9fce82f726
 size 85728997

 version https://git-lfs.github.com/spec/v1
+oid sha256:0b366dd84c17ff0c0f93bee9c1e5c08d40747cae3b15322a88899d9af7f34b76
 size 85728997

checkpoint-164/trainer_state.json CHANGED Viewed

@@ -10,1676 +10,1676 @@
   "is_world_process_zero": true,
   "log_history": [
     {
-      "entropy": 0.3660925142467022,
       "epoch": 0.01225114854517611,
-      "grad_norm": 0.0166015625,
       "learning_rate": 0.0002,
-      "loss": 0.0020782470237463713,
-      "mean_token_accuracy": 0.9997171945869923,
-      "num_tokens": 6092.0,
       "step": 1
     },
     {
-      "entropy": 0.34051003493368626,
       "epoch": 0.02450229709035222,
-      "grad_norm": 0.000823974609375,
       "learning_rate": 0.00019878048780487805,
-      "loss": 9.216360922437161e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 11535.0,
       "step": 2
     },
     {
-      "entropy": 0.32960800640285015,
       "epoch": 0.036753445635528334,
-      "grad_norm": 0.0098876953125,
       "learning_rate": 0.0001975609756097561,
-      "loss": 0.0001977928914129734,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 16432.0,
       "step": 3
     },
     {
-      "entropy": 0.33627333864569664,
       "epoch": 0.04900459418070444,
-      "grad_norm": 0.06640625,
       "learning_rate": 0.00019634146341463416,
-      "loss": 0.00977393426001072,
-      "mean_token_accuracy": 0.9985632188618183,
-      "num_tokens": 20507.0,
       "step": 4
     },
     {
-      "entropy": 0.31916058249771595,
       "epoch": 0.06125574272588055,
-      "grad_norm": 0.0003108978271484375,
       "learning_rate": 0.0001951219512195122,
-      "loss": 5.0926646508742124e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 26122.0,
       "step": 5
     },
     {
-      "entropy": 0.3524587769061327,
       "epoch": 0.07350689127105667,
-      "grad_norm": 0.000186920166015625,
       "learning_rate": 0.00019390243902439025,
-      "loss": 4.6155335439834744e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 30847.0,
       "step": 6
     },
     {
-      "entropy": 0.3272323925048113,
       "epoch": 0.08575803981623277,
-      "grad_norm": 0.005859375,
       "learning_rate": 0.0001926829268292683,
-      "loss": 0.000202978597371839,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 36541.0,
       "step": 7
     },
     {
-      "entropy": 0.347023731097579,
       "epoch": 0.09800918836140889,
-      "grad_norm": 0.00072479248046875,
       "learning_rate": 0.00019146341463414633,
-      "loss": 0.00011593783710850403,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 41001.0,
       "step": 8
     },
     {
-      "entropy": 0.376500410027802,
       "epoch": 0.11026033690658499,
-      "grad_norm": 0.09033203125,
       "learning_rate": 0.0001902439024390244,
-      "loss": 0.008863622322678566,
-      "mean_token_accuracy": 0.9979648105800152,
-      "num_tokens": 45467.0,
       "step": 9
     },
     {
-      "entropy": 0.3560014171525836,
       "epoch": 0.1225114854517611,
-      "grad_norm": 0.055419921875,
       "learning_rate": 0.00018902439024390244,
-      "loss": 0.004083322826772928,
-      "mean_token_accuracy": 0.9990039840340614,
-      "num_tokens": 50478.0,
       "step": 10
     },
     {
-      "entropy": 0.3533000349998474,
       "epoch": 0.13476263399693722,
-      "grad_norm": 0.0033721923828125,
       "learning_rate": 0.0001878048780487805,
-      "loss": 0.000252897065365687,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 56181.0,
       "step": 11
     },
     {
-      "entropy": 0.4079158063977957,
       "epoch": 0.14701378254211334,
-      "grad_norm": 0.00110626220703125,
       "learning_rate": 0.00018658536585365856,
-      "loss": 0.00019193078333046287,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 62946.0,
       "step": 12
     },
     {
-      "entropy": 0.4043316235765815,
       "epoch": 0.15926493108728942,
-      "grad_norm": 0.0021209716796875,
       "learning_rate": 0.0001853658536585366,
-      "loss": 0.00025091503630392253,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 68436.0,
       "step": 13
     },
     {
-      "entropy": 0.41207500360906124,
       "epoch": 0.17151607963246554,
-      "grad_norm": 0.00139617919921875,
       "learning_rate": 0.00018414634146341464,
-      "loss": 0.0002536335668992251,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 73603.0,
       "step": 14
     },
     {
-      "entropy": 0.43669185042381287,
       "epoch": 0.18376722817764166,
-      "grad_norm": 0.020751953125,
       "learning_rate": 0.0001829268292682927,
-      "loss": 0.0008837866480462253,
-      "mean_token_accuracy": 0.9994877055287361,
-      "num_tokens": 77845.0,
       "step": 15
     },
     {
-      "entropy": 0.41382858343422413,
       "epoch": 0.19601837672281777,
-      "grad_norm": 0.0145263671875,
       "learning_rate": 0.00018170731707317075,
-      "loss": 0.0006772386841475964,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 82744.0,
       "step": 16
     },
     {
-      "entropy": 0.4243332091718912,
       "epoch": 0.2082695252679939,
-      "grad_norm": 0.001922607421875,
       "learning_rate": 0.0001804878048780488,
-      "loss": 0.00027059210697188973,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 87453.0,
       "step": 17
     },
     {
-      "entropy": 0.4329488482326269,
       "epoch": 0.22052067381316998,
-      "grad_norm": 0.004852294921875,
       "learning_rate": 0.00017926829268292684,
-      "loss": 0.00031758740078657866,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 92321.0,
       "step": 18
     },
     {
-      "entropy": 0.440301101654768,
       "epoch": 0.2327718223583461,
-      "grad_norm": 0.005767822265625,
       "learning_rate": 0.00017804878048780488,
-      "loss": 0.0004065934626851231,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 97146.0,
       "step": 19
     },
     {
-      "entropy": 0.4400939680635929,
       "epoch": 0.2450229709035222,
-      "grad_norm": 0.0023040771484375,
       "learning_rate": 0.00017682926829268295,
-      "loss": 0.00020425915136002004,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 101943.0,
       "step": 20
     },
     {
-      "entropy": 0.4579729177057743,
       "epoch": 0.2572741194486983,
-      "grad_norm": 0.0286865234375,
       "learning_rate": 0.000175609756097561,
-      "loss": 0.0015601275954395533,
-      "mean_token_accuracy": 0.9996448867022991,
-      "num_tokens": 106772.0,
       "step": 21
     },
     {
-      "entropy": 0.40288309939205647,
       "epoch": 0.26952526799387444,
-      "grad_norm": 0.00072479248046875,
       "learning_rate": 0.00017439024390243903,
-      "loss": 9.121054608840495e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 112558.0,
       "step": 22
     },
     {
-      "entropy": 0.4252484003081918,
       "epoch": 0.28177641653905056,
-      "grad_norm": 0.000457763671875,
       "learning_rate": 0.00017317073170731708,
-      "loss": 8.147547487169504e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 117489.0,
       "step": 23
     },
     {
-      "entropy": 0.44810181483626366,
       "epoch": 0.29402756508422667,
-      "grad_norm": 0.007720947265625,
       "learning_rate": 0.00017195121951219512,
-      "loss": 0.0003956289147026837,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 123010.0,
       "step": 24
     },
     {
-      "entropy": 0.4023376125842333,
       "epoch": 0.30627871362940273,
-      "grad_norm": 0.00103759765625,
       "learning_rate": 0.0001707317073170732,
-      "loss": 8.693434210726991e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 127716.0,
       "step": 25
     },
     {
-      "entropy": 0.4007954867556691,
       "epoch": 0.31852986217457885,
-      "grad_norm": 0.00194549560546875,
       "learning_rate": 0.00016951219512195123,
-      "loss": 8.696074655745178e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 132372.0,
       "step": 26
     },
     {
-      "entropy": 0.3759774696081877,
       "epoch": 0.33078101071975496,
-      "grad_norm": 0.003387451171875,
       "learning_rate": 0.00016829268292682927,
-      "loss": 0.00013623938139062375,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 137028.0,
       "step": 27
     },
     {
-      "entropy": 0.40147540159523487,
       "epoch": 0.3430321592649311,
-      "grad_norm": 0.0380859375,
       "learning_rate": 0.00016707317073170731,
-      "loss": 0.005999124608933926,
-      "mean_token_accuracy": 0.9987113401293755,
-      "num_tokens": 142088.0,
       "step": 28
     },
     {
-      "entropy": 0.38656803220510483,
       "epoch": 0.3552833078101072,
-      "grad_norm": 0.0322265625,
       "learning_rate": 0.00016585365853658536,
-      "loss": 0.00021061318693682551,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 147481.0,
       "step": 29
     },
     {
-      "entropy": 0.4059827271848917,
       "epoch": 0.3675344563552833,
-      "grad_norm": 0.00015163421630859375,
       "learning_rate": 0.00016463414634146343,
-      "loss": 3.9411937905242667e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 152973.0,
       "step": 30
     },
     {
-      "entropy": 0.40111804008483887,
       "epoch": 0.37978560490045943,
-      "grad_norm": 0.0003681182861328125,
       "learning_rate": 0.00016341463414634147,
-      "loss": 5.111394784762524e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 156786.0,
       "step": 31
     },
     {
-      "entropy": 0.41568026319146156,
       "epoch": 0.39203675344563554,
-      "grad_norm": 0.00162506103515625,
       "learning_rate": 0.00016219512195121954,
-      "loss": 0.0001103500762837939,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 162859.0,
       "step": 32
     },
     {
-      "entropy": 0.39988269470632076,
       "epoch": 0.40428790199081166,
-      "grad_norm": 0.000518798828125,
       "learning_rate": 0.00016097560975609758,
-      "loss": 6.166221282910556e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 167969.0,
       "step": 33
     },
     {
-      "entropy": 0.3738459562882781,
       "epoch": 0.4165390505359878,
-      "grad_norm": 0.00537109375,
       "learning_rate": 0.00015975609756097562,
-      "loss": 0.00012469613284338266,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 172518.0,
       "step": 34
     },
     {
-      "entropy": 0.40653541777282953,
       "epoch": 0.42879019908116384,
-      "grad_norm": 0.0031280517578125,
       "learning_rate": 0.00015853658536585366,
-      "loss": 0.00010661048872862011,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 177085.0,
       "step": 35
     },
     {
-      "entropy": 0.39361329190433025,
       "epoch": 0.44104134762633995,
-      "grad_norm": 0.08154296875,
       "learning_rate": 0.00015731707317073173,
-      "loss": 0.0010916765313595533,
-      "mean_token_accuracy": 0.9990942031145096,
-      "num_tokens": 181617.0,
       "step": 36
     },
     {
-      "entropy": 0.358949625864625,
       "epoch": 0.45329249617151607,
-      "grad_norm": 0.01080322265625,
       "learning_rate": 0.00015609756097560978,
-      "loss": 0.0010772041277959943,
-      "mean_token_accuracy": 0.9995535723865032,
-      "num_tokens": 186836.0,
       "step": 37
     },
     {
-      "entropy": 0.3930373042821884,
       "epoch": 0.4655436447166922,
-      "grad_norm": 0.000461578369140625,
       "learning_rate": 0.00015487804878048782,
-      "loss": 5.279047036310658e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 191224.0,
       "step": 38
     },
     {
-      "entropy": 0.35740520991384983,
       "epoch": 0.4777947932618683,
-      "grad_norm": 0.000873565673828125,
       "learning_rate": 0.00015365853658536586,
-      "loss": 5.439379674498923e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 195926.0,
       "step": 39
     },
     {
-      "entropy": 0.38909873832017183,
       "epoch": 0.4900459418070444,
-      "grad_norm": 0.0257568359375,
       "learning_rate": 0.0001524390243902439,
-      "loss": 0.0015194097068160772,
-      "mean_token_accuracy": 0.999550361186266,
-      "num_tokens": 200772.0,
       "step": 40
     },
     {
-      "entropy": 0.36850977689027786,
       "epoch": 0.5022970903522205,
-      "grad_norm": 0.1064453125,
       "learning_rate": 0.00015121951219512197,
-      "loss": 0.002955856267362833,
-      "mean_token_accuracy": 0.9993872530758381,
-      "num_tokens": 204499.0,
       "step": 41
     },
     {
-      "entropy": 0.3940112106502056,
       "epoch": 0.5145482388973966,
-      "grad_norm": 0.00885009765625,
       "learning_rate": 0.00015000000000000001,
-      "loss": 0.000253106962190941,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 208814.0,
       "step": 42
     },
     {
-      "entropy": 0.39878340624272823,
       "epoch": 0.5267993874425727,
-      "grad_norm": 0.037841796875,
       "learning_rate": 0.00014878048780487806,
-      "loss": 0.0007202713750302792,
-      "mean_token_accuracy": 0.9995833337306976,
-      "num_tokens": 213907.0,
       "step": 43
     },
     {
-      "entropy": 0.41587444953620434,
       "epoch": 0.5390505359877489,
-      "grad_norm": 0.0004177093505859375,
       "learning_rate": 0.0001475609756097561,
-      "loss": 6.820505223004147e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 218988.0,
       "step": 44
     },
     {
-      "entropy": 0.3888211837038398,
       "epoch": 0.5513016845329249,
-      "grad_norm": 0.007568359375,
       "learning_rate": 0.00014634146341463414,
-      "loss": 0.000737900089006871,
-      "mean_token_accuracy": 0.9995967745780945,
-      "num_tokens": 223595.0,
       "step": 45
     },
     {
-      "entropy": 0.4139576517045498,
       "epoch": 0.5635528330781011,
-      "grad_norm": 0.014892578125,
       "learning_rate": 0.0001451219512195122,
-      "loss": 0.0006043408066034317,
-      "mean_token_accuracy": 0.9995192289352417,
-      "num_tokens": 228244.0,
       "step": 46
     },
     {
-      "entropy": 0.39713083021342754,
       "epoch": 0.5758039816232772,
-      "grad_norm": 0.00046539306640625,
       "learning_rate": 0.00014390243902439025,
-      "loss": 8.217584399972111e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 232606.0,
       "step": 47
     },
     {
-      "entropy": 0.40557617880403996,
       "epoch": 0.5880551301684533,
-      "grad_norm": 0.0009918212890625,
       "learning_rate": 0.0001426829268292683,
-      "loss": 0.00012616875756066293,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 236563.0,
       "step": 48
     },
     {
-      "entropy": 0.43470797687768936,
       "epoch": 0.6003062787136294,
-      "grad_norm": 0.0238037109375,
       "learning_rate": 0.00014146341463414634,
-      "loss": 0.0010796654969453812,
-      "mean_token_accuracy": 0.999465811997652,
-      "num_tokens": 241214.0,
       "step": 49
     },
     {
-      "entropy": 0.4234541580080986,
       "epoch": 0.6125574272588055,
-      "grad_norm": 0.02783203125,
       "learning_rate": 0.00014024390243902438,
-      "loss": 0.0009178520413115621,
-      "mean_token_accuracy": 0.9996565915644169,
-      "num_tokens": 245200.0,
       "step": 50
     },
     {
       "epoch": 0.6125574272588055,
-      "eval_entropy": 0.4022736955380094,
-      "eval_loss": 0.0006544959614984691,
-      "eval_mean_token_accuracy": 0.9998166846192401,
-      "eval_num_tokens": 245200.0,
-      "eval_runtime": 51.0138,
-      "eval_samples_per_second": 1.353,
-      "eval_steps_per_second": 1.353,
       "step": 50
     },
     {
-      "entropy": 0.41674751229584217,
       "epoch": 0.6248085758039816,
-      "grad_norm": 0.00131988525390625,
       "learning_rate": 0.00013902439024390245,
-      "loss": 0.0001285702601308003,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 249761.0,
       "step": 51
     },
     {
-      "entropy": 0.42886597104370594,
       "epoch": 0.6370597243491577,
-      "grad_norm": 0.00171661376953125,
       "learning_rate": 0.0001378048780487805,
-      "loss": 0.00014620381989516318,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 254787.0,
       "step": 52
     },
     {
-      "entropy": 0.4423276912420988,
       "epoch": 0.6493108728943339,
-      "grad_norm": 0.038818359375,
       "learning_rate": 0.00013658536585365856,
-      "loss": 0.003947169054299593,
-      "mean_token_accuracy": 0.9983357414603233,
-      "num_tokens": 260287.0,
       "step": 53
     },
     {
-      "entropy": 0.3989156847819686,
       "epoch": 0.6615620214395099,
-      "grad_norm": 0.0211181640625,
       "learning_rate": 0.0001353658536585366,
-      "loss": 0.00047477131010964513,
-      "mean_token_accuracy": 0.9998249299824238,
-      "num_tokens": 264810.0,
       "step": 54
     },
     {
-      "entropy": 0.4272368475794792,
       "epoch": 0.6738131699846861,
-      "grad_norm": 0.029052734375,
       "learning_rate": 0.00013414634146341464,
-      "loss": 0.00408769678324461,
-      "mean_token_accuracy": 0.9993622452020645,
-      "num_tokens": 270386.0,
       "step": 55
     },
     {
-      "entropy": 0.44703495875000954,
       "epoch": 0.6860643185298622,
-      "grad_norm": 0.01202392578125,
       "learning_rate": 0.0001329268292682927,
-      "loss": 0.00038261126610450447,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 274391.0,
       "step": 56
     },
     {
-      "entropy": 0.4288428146392107,
       "epoch": 0.6983154670750383,
-      "grad_norm": 0.01019287109375,
       "learning_rate": 0.00013170731707317076,
-      "loss": 0.0003242077073082328,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 279716.0,
       "step": 57
     },
     {
-      "entropy": 0.37452960200607777,
       "epoch": 0.7105666156202144,
-      "grad_norm": 0.021728515625,
       "learning_rate": 0.0001304878048780488,
-      "loss": 0.0027725810650736094,
-      "mean_token_accuracy": 0.9994703382253647,
-      "num_tokens": 285404.0,
       "step": 58
     },
     {
-      "entropy": 0.4130611680448055,
       "epoch": 0.7228177641653905,
-      "grad_norm": 0.04541015625,
       "learning_rate": 0.00012926829268292684,
-      "loss": 0.0017543239519000053,
-      "mean_token_accuracy": 0.9995689652860165,
-      "num_tokens": 289992.0,
       "step": 59
     },
     {
-      "entropy": 0.41101630590856075,
       "epoch": 0.7350689127105666,
-      "grad_norm": 0.00078582763671875,
       "learning_rate": 0.00012804878048780488,
-      "loss": 9.316274372395128e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 294861.0,
       "step": 60
     },
     {
-      "entropy": 0.3678157525137067,
       "epoch": 0.7473200612557427,
-      "grad_norm": 0.00058746337890625,
       "learning_rate": 0.00012682926829268293,
-      "loss": 8.83688626345247e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 300355.0,
       "step": 61
     },
     {
-      "entropy": 0.40994635969400406,
       "epoch": 0.7595712098009189,
-      "grad_norm": 0.0015869140625,
       "learning_rate": 0.000125609756097561,
-      "loss": 8.545083983335644e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 305776.0,
       "step": 62
     },
     {
-      "entropy": 0.37295936793088913,
       "epoch": 0.7718223583460949,
-      "grad_norm": 0.000827789306640625,
       "learning_rate": 0.00012439024390243904,
-      "loss": 7.97374959802255e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 310204.0,
       "step": 63
     },
     {
-      "entropy": 0.36804571095854044,
       "epoch": 0.7840735068912711,
-      "grad_norm": 0.0002880096435546875,
       "learning_rate": 0.00012317073170731708,
-      "loss": 6.0703161580022424e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 314205.0,
       "step": 64
     },
     {
-      "entropy": 0.3904844745993614,
       "epoch": 0.7963246554364471,
-      "grad_norm": 0.0019989013671875,
       "learning_rate": 0.00012195121951219512,
-      "loss": 7.91027705417946e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 319157.0,
       "step": 65
     },
     {
-      "entropy": 0.3921838700771332,
       "epoch": 0.8085758039816233,
-      "grad_norm": 0.00177764892578125,
       "learning_rate": 0.00012073170731707318,
-      "loss": 8.364896348211914e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 324681.0,
       "step": 66
     },
     {
-      "entropy": 0.34572961553931236,
       "epoch": 0.8208269525267994,
-      "grad_norm": 0.061767578125,
       "learning_rate": 0.00011951219512195122,
-      "loss": 0.008409281261265278,
-      "mean_token_accuracy": 0.9963545724749565,
-      "num_tokens": 329941.0,
       "step": 67
     },
     {
-      "entropy": 0.3841299172490835,
       "epoch": 0.8330781010719756,
-      "grad_norm": 0.01123046875,
       "learning_rate": 0.00011829268292682926,
-      "loss": 0.00017956709780264646,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 334486.0,
       "step": 68
     },
     {
-      "entropy": 0.39541577361524105,
       "epoch": 0.8453292496171516,
-      "grad_norm": 0.00274658203125,
       "learning_rate": 0.00011707317073170732,
-      "loss": 0.00012585960212163627,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 338183.0,
       "step": 69
     },
     {
-      "entropy": 0.4046988161280751,
       "epoch": 0.8575803981623277,
-      "grad_norm": 0.126953125,
       "learning_rate": 0.00011585365853658536,
-      "loss": 0.007125813513994217,
-      "mean_token_accuracy": 0.9981492757797241,
-      "num_tokens": 342593.0,
       "step": 70
     },
     {
-      "entropy": 0.40994592756032944,
       "epoch": 0.8698315467075038,
-      "grad_norm": 0.0517578125,
       "learning_rate": 0.00011463414634146342,
-      "loss": 0.0006066925125196576,
-      "mean_token_accuracy": 0.9997807033360004,
-      "num_tokens": 347797.0,
       "step": 71
     },
     {
-      "entropy": 0.3796220198273659,
       "epoch": 0.8820826952526799,
-      "grad_norm": 0.006103515625,
       "learning_rate": 0.00011341463414634146,
-      "loss": 0.00017896694771479815,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 352121.0,
       "step": 72
     },
     {
-      "entropy": 0.3931356444954872,
       "epoch": 0.8943338437978561,
-      "grad_norm": 0.0181884765625,
       "learning_rate": 0.00011219512195121953,
-      "loss": 0.0010632644407451153,
-      "mean_token_accuracy": 0.9997568093240261,
-      "num_tokens": 357943.0,
       "step": 73
     },
     {
-      "entropy": 0.36392936669290066,
       "epoch": 0.9065849923430321,
-      "grad_norm": 0.024658203125,
       "learning_rate": 0.00011097560975609757,
-      "loss": 0.0006849091150797904,
-      "mean_token_accuracy": 0.9996345043182373,
-      "num_tokens": 363814.0,
       "step": 74
     },
     {
-      "entropy": 0.3864069525152445,
       "epoch": 0.9188361408882083,
-      "grad_norm": 0.000270843505859375,
       "learning_rate": 0.00010975609756097563,
-      "loss": 5.0294114771531895e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 368870.0,
       "step": 75
     },
     {
-      "entropy": 0.39719677343964577,
       "epoch": 0.9310872894333844,
-      "grad_norm": 0.01519775390625,
       "learning_rate": 0.00010853658536585367,
-      "loss": 0.00048823675024323165,
-      "mean_token_accuracy": 0.999143835157156,
-      "num_tokens": 373670.0,
       "step": 76
     },
     {
-      "entropy": 0.35627279058098793,
       "epoch": 0.9433384379785605,
-      "grad_norm": 0.0074462890625,
       "learning_rate": 0.00010731707317073172,
-      "loss": 0.000174719825736247,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 379037.0,
       "step": 77
     },
     {
-      "entropy": 0.38681978918612003,
       "epoch": 0.9555895865237366,
-      "grad_norm": 0.0181884765625,
       "learning_rate": 0.00010609756097560977,
-      "loss": 0.000976942596025765,
-      "mean_token_accuracy": 0.9992977529764175,
-      "num_tokens": 384252.0,
       "step": 78
     },
     {
-      "entropy": 0.3772548586130142,
       "epoch": 0.9678407350689127,
-      "grad_norm": 0.000904083251953125,
       "learning_rate": 0.00010487804878048781,
-      "loss": 6.608536932617426e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 388347.0,
       "step": 79
     },
     {
-      "entropy": 0.3597776433452964,
       "epoch": 0.9800918836140888,
-      "grad_norm": 0.010986328125,
       "learning_rate": 0.00010365853658536586,
-      "loss": 0.0007963755051605403,
-      "mean_token_accuracy": 0.999015748500824,
-      "num_tokens": 394213.0,
       "step": 80
     },
     {
-      "entropy": 0.3731031287461519,
       "epoch": 0.9923430321592649,
-      "grad_norm": 0.00115966796875,
       "learning_rate": 0.0001024390243902439,
-      "loss": 8.310518751386553e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 399113.0,
       "step": 81
     },
     {
-      "entropy": 0.37349462509155273,
       "epoch": 1.0,
-      "grad_norm": 0.00022125244140625,
       "learning_rate": 0.00010121951219512196,
-      "loss": 4.093759343959391e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 402129.0,
       "step": 82
     },
     {
-      "entropy": 0.38408348336815834,
       "epoch": 1.0122511485451762,
-      "grad_norm": 0.027099609375,
       "learning_rate": 0.0001,
-      "loss": 0.0015746817225590348,
-      "mean_token_accuracy": 0.9996279776096344,
-      "num_tokens": 406760.0,
       "step": 83
     },
     {
-      "entropy": 0.36415083333849907,
       "epoch": 1.0245022970903521,
-      "grad_norm": 0.0032501220703125,
       "learning_rate": 9.878048780487805e-05,
-      "loss": 0.00011362869554432109,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 411366.0,
       "step": 84
     },
     {
-      "entropy": 0.3951573334634304,
       "epoch": 1.0367534456355283,
-      "grad_norm": 0.0018768310546875,
       "learning_rate": 9.75609756097561e-05,
-      "loss": 8.601781155448407e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 417767.0,
       "step": 85
     },
     {
-      "entropy": 0.3533172570168972,
       "epoch": 1.0490045941807045,
-      "grad_norm": 0.00146484375,
       "learning_rate": 9.634146341463415e-05,
-      "loss": 5.874271664652042e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 421737.0,
       "step": 86
     },
     {
-      "entropy": 0.35251205042004585,
       "epoch": 1.0612557427258806,
-      "grad_norm": 6.008148193359375e-05,
       "learning_rate": 9.51219512195122e-05,
-      "loss": 2.1197016394580714e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 426853.0,
       "step": 87
     },
     {
-      "entropy": 0.42304582707583904,
       "epoch": 1.0735068912710566,
-      "grad_norm": 0.000797271728515625,
       "learning_rate": 9.390243902439024e-05,
-      "loss": 6.177897739689797e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 431082.0,
       "step": 88
     },
     {
-      "entropy": 0.39542090706527233,
       "epoch": 1.0857580398162328,
-      "grad_norm": 0.041015625,
       "learning_rate": 9.26829268292683e-05,
-      "loss": 0.0009606232051737607,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 435693.0,
       "step": 89
     },
     {
-      "entropy": 0.37046173214912415,
       "epoch": 1.098009188361409,
-      "grad_norm": 0.000278472900390625,
       "learning_rate": 9.146341463414635e-05,
-      "loss": 4.265129246050492e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 440577.0,
       "step": 90
     },
     {
-      "entropy": 0.3931607408449054,
       "epoch": 1.110260336906585,
-      "grad_norm": 0.035400390625,
       "learning_rate": 9.02439024390244e-05,
-      "loss": 0.004250116180628538,
-      "mean_token_accuracy": 0.9994369372725487,
-      "num_tokens": 445265.0,
       "step": 91
     },
     {
-      "entropy": 0.3917137086391449,
       "epoch": 1.122511485451761,
-      "grad_norm": 0.0419921875,
       "learning_rate": 8.902439024390244e-05,
-      "loss": 0.002317648846656084,
-      "mean_token_accuracy": 0.9992785975337029,
-      "num_tokens": 450020.0,
       "step": 92
     },
     {
-      "entropy": 0.3758338335901499,
       "epoch": 1.1347626339969372,
-      "grad_norm": 0.0196533203125,
       "learning_rate": 8.78048780487805e-05,
-      "loss": 0.0006808089674450457,
-      "mean_token_accuracy": 0.999522902071476,
-      "num_tokens": 455003.0,
       "step": 93
     },
     {
-      "entropy": 0.383782709017396,
       "epoch": 1.1470137825421134,
-      "grad_norm": 0.0034027099609375,
       "learning_rate": 8.658536585365854e-05,
-      "loss": 7.263245788635686e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 459698.0,
       "step": 94
     },
     {
-      "entropy": 0.3821055982261896,
       "epoch": 1.1592649310872893,
-      "grad_norm": 0.0004138946533203125,
       "learning_rate": 8.53658536585366e-05,
-      "loss": 3.771902629523538e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 464337.0,
       "step": 95
     },
     {
-      "entropy": 0.3649219311773777,
       "epoch": 1.1715160796324655,
-      "grad_norm": 0.00872802734375,
       "learning_rate": 8.414634146341464e-05,
-      "loss": 0.0004717935808002949,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 468882.0,
       "step": 96
     },
     {
-      "entropy": 0.3700664434581995,
       "epoch": 1.1837672281776417,
-      "grad_norm": 0.00015544891357421875,
       "learning_rate": 8.292682926829268e-05,
-      "loss": 3.247045970056206e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 473756.0,
       "step": 97
     },
     {
-      "entropy": 0.3915936965495348,
       "epoch": 1.1960183767228179,
-      "grad_norm": 0.05078125,
       "learning_rate": 8.170731707317073e-05,
-      "loss": 0.005024694371968508,
-      "mean_token_accuracy": 0.9996565915644169,
-      "num_tokens": 479061.0,
       "step": 98
     },
     {
-      "entropy": 0.4096358586102724,
       "epoch": 1.2082695252679938,
-      "grad_norm": 0.00144195556640625,
       "learning_rate": 8.048780487804879e-05,
-      "loss": 4.485135286813602e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 484835.0,
       "step": 99
     },
     {
-      "entropy": 0.35138822346925735,
       "epoch": 1.22052067381317,
-      "grad_norm": 0.0038299560546875,
       "learning_rate": 7.926829268292683e-05,
-      "loss": 0.00019770213111769408,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 489546.0,
       "step": 100
     },
     {
       "epoch": 1.22052067381317,
-      "eval_entropy": 0.3780687239722929,
-      "eval_loss": 0.00034746917663142085,
-      "eval_mean_token_accuracy": 0.9999171840971794,
-      "eval_num_tokens": 489546.0,
-      "eval_runtime": 50.9982,
-      "eval_samples_per_second": 1.353,
-      "eval_steps_per_second": 1.353,
       "step": 100
     },
     {
-      "entropy": 0.4012060575187206,
       "epoch": 1.2327718223583461,
-      "grad_norm": 0.000217437744140625,
       "learning_rate": 7.804878048780489e-05,
-      "loss": 3.667730197776109e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 494781.0,
       "step": 101
     },
     {
-      "entropy": 0.37181732058525085,
       "epoch": 1.245022970903522,
-      "grad_norm": 0.0002155303955078125,
       "learning_rate": 7.682926829268293e-05,
-      "loss": 2.923922693298664e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 499861.0,
       "step": 102
     },
     {
-      "entropy": 0.38948795571923256,
       "epoch": 1.2572741194486983,
-      "grad_norm": 6.866455078125e-05,
       "learning_rate": 7.560975609756099e-05,
-      "loss": 3.10177420033142e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 505291.0,
       "step": 103
     },
     {
-      "entropy": 0.3776157572865486,
       "epoch": 1.2695252679938744,
-      "grad_norm": 0.00012874603271484375,
       "learning_rate": 7.439024390243903e-05,
-      "loss": 2.8559963539009914e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 510284.0,
       "step": 104
     },
     {
-      "entropy": 0.3941178657114506,
       "epoch": 1.2817764165390506,
-      "grad_norm": 0.007232666015625,
       "learning_rate": 7.317073170731707e-05,
-      "loss": 0.0008174990070983768,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 514517.0,
       "step": 105
     },
     {
-      "entropy": 0.3697250857949257,
       "epoch": 1.2940275650842268,
-      "grad_norm": 0.003143310546875,
       "learning_rate": 7.195121951219513e-05,
-      "loss": 0.00010880863555939868,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 519535.0,
       "step": 106
     },
     {
-      "entropy": 0.3888526763767004,
       "epoch": 1.3062787136294027,
-      "grad_norm": 0.00054931640625,
       "learning_rate": 7.073170731707317e-05,
-      "loss": 5.111205973662436e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 524397.0,
       "step": 107
     },
     {
-      "entropy": 0.3866258058696985,
       "epoch": 1.318529862174579,
-      "grad_norm": 0.0004100799560546875,
       "learning_rate": 6.951219512195122e-05,
-      "loss": 3.999587715952657e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 528997.0,
       "step": 108
     },
     {
-      "entropy": 0.3921303730458021,
       "epoch": 1.3307810107197549,
-      "grad_norm": 0.000885009765625,
       "learning_rate": 6.829268292682928e-05,
-      "loss": 6.128583481768146e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 533965.0,
       "step": 109
     },
     {
-      "entropy": 0.3705854155123234,
       "epoch": 1.343032159264931,
-      "grad_norm": 0.002960205078125,
       "learning_rate": 6.707317073170732e-05,
-      "loss": 7.792656106175855e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 539387.0,
       "step": 110
     },
     {
-      "entropy": 0.3712622048333287,
       "epoch": 1.3552833078101072,
-      "grad_norm": 0.00089263916015625,
       "learning_rate": 6.585365853658538e-05,
-      "loss": 4.521696246229112e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 543755.0,
       "step": 111
     },
     {
-      "entropy": 0.40867704525589943,
       "epoch": 1.3675344563552834,
-      "grad_norm": 0.023193359375,
       "learning_rate": 6.463414634146342e-05,
-      "loss": 0.003280676668509841,
-      "mean_token_accuracy": 0.9978448264300823,
-      "num_tokens": 548188.0,
       "step": 112
     },
     {
-      "entropy": 0.3910982459783554,
       "epoch": 1.3797856049004595,
-      "grad_norm": 0.0028533935546875,
       "learning_rate": 6.341463414634146e-05,
-      "loss": 0.00015341158723458648,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 553717.0,
       "step": 113
     },
     {
-      "entropy": 0.3753495467826724,
       "epoch": 1.3920367534456355,
-      "grad_norm": 6.866455078125e-05,
       "learning_rate": 6.219512195121952e-05,
-      "loss": 2.554376442276407e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 558501.0,
       "step": 114
     },
     {
-      "entropy": 0.3936616498976946,
       "epoch": 1.4042879019908117,
-      "grad_norm": 0.000774383544921875,
       "learning_rate": 6.097560975609756e-05,
-      "loss": 4.565157360048033e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 563989.0,
       "step": 115
     },
     {
-      "entropy": 0.4080927763134241,
       "epoch": 1.4165390505359878,
-      "grad_norm": 0.000728607177734375,
       "learning_rate": 5.975609756097561e-05,
-      "loss": 5.44461581739597e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 568327.0,
       "step": 116
     },
     {
-      "entropy": 0.36639871448278427,
       "epoch": 1.4287901990811638,
-      "grad_norm": 0.000457763671875,
       "learning_rate": 5.853658536585366e-05,
-      "loss": 3.381741407793015e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 572919.0,
       "step": 117
     },
     {
-      "entropy": 0.4015892669558525,
       "epoch": 1.44104134762634,
-      "grad_norm": 0.00017833709716796875,
       "learning_rate": 5.731707317073171e-05,
-      "loss": 4.158892625127919e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 577916.0,
       "step": 118
     },
     {
-      "entropy": 0.40410150960087776,
       "epoch": 1.4532924961715161,
-      "grad_norm": 0.000621795654296875,
       "learning_rate": 5.6097560975609764e-05,
-      "loss": 2.5736055249581113e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 583152.0,
       "step": 119
     },
     {
-      "entropy": 0.40528898034244776,
       "epoch": 1.4655436447166923,
-      "grad_norm": 0.01953125,
       "learning_rate": 5.487804878048781e-05,
-      "loss": 0.00020874114125035703,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 587880.0,
       "step": 120
     },
     {
-      "entropy": 0.35937592945992947,
       "epoch": 1.4777947932618682,
-      "grad_norm": 0.083984375,
       "learning_rate": 5.365853658536586e-05,
-      "loss": 0.007331337314099073,
-      "mean_token_accuracy": 0.9991379305720329,
-      "num_tokens": 592284.0,
       "step": 121
     },
     {
-      "entropy": 0.3928218297660351,
       "epoch": 1.4900459418070444,
-      "grad_norm": 0.00013446807861328125,
       "learning_rate": 5.2439024390243904e-05,
-      "loss": 2.927147943410091e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 597046.0,
       "step": 122
     },
     {
-      "entropy": 0.3777940608561039,
       "epoch": 1.5022970903522204,
-      "grad_norm": 0.000579833984375,
       "learning_rate": 5.121951219512195e-05,
-      "loss": 6.0145219322294e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 601350.0,
       "step": 123
     },
     {
-      "entropy": 0.39830240048468113,
       "epoch": 1.5145482388973965,
-      "grad_norm": 0.0245361328125,
       "learning_rate": 5e-05,
-      "loss": 0.00029612769139930606,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 606643.0,
       "step": 124
     },
     {
-      "entropy": 0.3925098739564419,
       "epoch": 1.5267993874425727,
-      "grad_norm": 0.0004749298095703125,
       "learning_rate": 4.878048780487805e-05,
-      "loss": 4.631431511370465e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 612405.0,
       "step": 125
     },
     {
-      "entropy": 0.3956710360944271,
       "epoch": 1.5390505359877489,
-      "grad_norm": 0.00634765625,
       "learning_rate": 4.75609756097561e-05,
-      "loss": 8.446360880043358e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 617227.0,
       "step": 126
     },
     {
-      "entropy": 0.430975291877985,
       "epoch": 1.551301684532925,
-      "grad_norm": 0.000518798828125,
       "learning_rate": 4.634146341463415e-05,
-      "loss": 6.132836278993636e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 622353.0,
       "step": 127
     },
     {
-      "entropy": 0.4242272228002548,
       "epoch": 1.5635528330781012,
-      "grad_norm": 0.0025177001953125,
       "learning_rate": 4.51219512195122e-05,
-      "loss": 0.00011561957217054442,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 627267.0,
       "step": 128
     },
     {
-      "entropy": 0.3710012398660183,
       "epoch": 1.5758039816232772,
-      "grad_norm": 0.002777099609375,
       "learning_rate": 4.390243902439025e-05,
-      "loss": 0.00010202911653323099,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 631452.0,
       "step": 129
     },
     {
-      "entropy": 0.35699679516255856,
       "epoch": 1.5880551301684533,
-      "grad_norm": 0.00023651123046875,
       "learning_rate": 4.26829268292683e-05,
-      "loss": 5.903129203943536e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 636500.0,
       "step": 130
     },
     {
-      "entropy": 0.39619251526892185,
       "epoch": 1.6003062787136293,
-      "grad_norm": 0.0230712890625,
       "learning_rate": 4.146341463414634e-05,
-      "loss": 0.0031676713842898607,
-      "mean_token_accuracy": 0.9987796545028687,
-      "num_tokens": 641262.0,
       "step": 131
     },
     {
-      "entropy": 0.40411114878952503,
       "epoch": 1.6125574272588055,
-      "grad_norm": 0.0361328125,
       "learning_rate": 4.0243902439024395e-05,
-      "loss": 0.0015652105212211609,
-      "mean_token_accuracy": 0.999205507338047,
-      "num_tokens": 646375.0,
       "step": 132
     },
     {
-      "entropy": 0.3453770913183689,
       "epoch": 1.6248085758039816,
-      "grad_norm": 8.440017700195312e-05,
       "learning_rate": 3.9024390243902444e-05,
-      "loss": 3.279931843280792e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 650765.0,
       "step": 133
     },
     {
-      "entropy": 0.37724466249346733,
       "epoch": 1.6370597243491578,
-      "grad_norm": 0.00142669677734375,
       "learning_rate": 3.780487804878049e-05,
-      "loss": 5.4958236432867125e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 655167.0,
       "step": 134
     },
     {
-      "entropy": 0.39796170592308044,
       "epoch": 1.649310872894334,
-      "grad_norm": 0.0003986358642578125,
       "learning_rate": 3.6585365853658535e-05,
-      "loss": 3.9815466152504086e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 660288.0,
       "step": 135
     },
     {
-      "entropy": 0.4333613757044077,
       "epoch": 1.66156202143951,
-      "grad_norm": 0.0001544952392578125,
       "learning_rate": 3.5365853658536584e-05,
-      "loss": 4.787950456375256e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 664471.0,
       "step": 136
     },
     {
-      "entropy": 0.41916552372276783,
       "epoch": 1.673813169984686,
-      "grad_norm": 0.0002899169921875,
       "learning_rate": 3.414634146341464e-05,
-      "loss": 4.767990321852267e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 669354.0,
       "step": 137
     },
     {
-      "entropy": 0.3999825790524483,
       "epoch": 1.686064318529862,
-      "grad_norm": 0.0026397705078125,
       "learning_rate": 3.292682926829269e-05,
-      "loss": 0.0001605500146979466,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 674909.0,
       "step": 138
     },
     {
-      "entropy": 0.39421058259904385,
       "epoch": 1.6983154670750382,
-      "grad_norm": 0.005767822265625,
       "learning_rate": 3.170731707317073e-05,
-      "loss": 0.00022102531511336565,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 679690.0,
       "step": 139
     },
     {
-      "entropy": 0.4142182134091854,
       "epoch": 1.7105666156202144,
-      "grad_norm": 0.003631591796875,
       "learning_rate": 3.048780487804878e-05,
-      "loss": 0.00014472004841081798,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 685046.0,
       "step": 140
     },
     {
-      "entropy": 0.3982192352414131,
       "epoch": 1.7228177641653906,
-      "grad_norm": 0.00019168853759765625,
       "learning_rate": 2.926829268292683e-05,
-      "loss": 4.7273264499381185e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 689394.0,
       "step": 141
     },
     {
-      "entropy": 0.4133493732661009,
       "epoch": 1.7350689127105667,
-      "grad_norm": 0.00701904296875,
       "learning_rate": 2.8048780487804882e-05,
-      "loss": 9.296434291172773e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 693187.0,
       "step": 142
     },
     {
-      "entropy": 0.40933855436742306,
       "epoch": 1.7473200612557427,
-      "grad_norm": 0.0019683837890625,
       "learning_rate": 2.682926829268293e-05,
-      "loss": 9.476351988269016e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 697601.0,
       "step": 143
     },
     {
-      "entropy": 0.41714910976588726,
       "epoch": 1.7595712098009189,
-      "grad_norm": 0.045166015625,
       "learning_rate": 2.5609756097560977e-05,
-      "loss": 0.0034146099351346493,
-      "mean_token_accuracy": 0.9998650103807449,
-      "num_tokens": 703048.0,
       "step": 144
     },
     {
-      "entropy": 0.40594901144504547,
       "epoch": 1.7718223583460948,
-      "grad_norm": 0.02587890625,
       "learning_rate": 2.4390243902439026e-05,
-      "loss": 0.001274456619285047,
-      "mean_token_accuracy": 0.999015748500824,
-      "num_tokens": 707860.0,
       "step": 145
     },
     {
-      "entropy": 0.41635255329310894,
       "epoch": 1.784073506891271,
-      "grad_norm": 0.000156402587890625,
       "learning_rate": 2.3170731707317075e-05,
-      "loss": 5.037836672272533e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 712819.0,
       "step": 146
     },
     {
-      "entropy": 0.4038653904572129,
       "epoch": 1.7963246554364471,
-      "grad_norm": 0.0004100799560546875,
       "learning_rate": 2.1951219512195124e-05,
-      "loss": 4.163683479418978e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 718451.0,
       "step": 147
     },
     {
-      "entropy": 0.4069879539310932,
       "epoch": 1.8085758039816233,
-      "grad_norm": 7.104873657226562e-05,
       "learning_rate": 2.073170731707317e-05,
-      "loss": 3.6120818549534306e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 723808.0,
       "step": 148
     },
     {
-      "entropy": 0.4381860624998808,
       "epoch": 1.8208269525267995,
-      "grad_norm": 9.870529174804688e-05,
       "learning_rate": 1.9512195121951222e-05,
-      "loss": 4.433648064150475e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 728124.0,
       "step": 149
     },
     {
-      "entropy": 0.42220813781023026,
       "epoch": 1.8330781010719757,
-      "grad_norm": 0.005401611328125,
       "learning_rate": 1.8292682926829268e-05,
-      "loss": 0.00017107791791204363,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 733915.0,
       "step": 150
     },
     {
       "epoch": 1.8330781010719757,
-      "eval_entropy": 0.4000617520532746,
-      "eval_loss": 0.00023719228920526803,
-      "eval_mean_token_accuracy": 0.9998813841653906,
-      "eval_num_tokens": 733915.0,
-      "eval_runtime": 50.9031,
-      "eval_samples_per_second": 1.356,
-      "eval_steps_per_second": 1.356,
       "step": 150
     },
     {
-      "entropy": 0.3884127251803875,
       "epoch": 1.8453292496171516,
-      "grad_norm": 0.0010986328125,
       "learning_rate": 1.707317073170732e-05,
-      "loss": 6.602725625270978e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 738158.0,
       "step": 151
     },
     {
-      "entropy": 0.43312329426407814,
       "epoch": 1.8575803981623276,
-      "grad_norm": 0.00109100341796875,
       "learning_rate": 1.5853658536585366e-05,
-      "loss": 6.0493410273920745e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 743914.0,
       "step": 152
     },
     {
-      "entropy": 0.3950846865773201,
       "epoch": 1.8698315467075037,
-      "grad_norm": 9.393692016601562e-05,
       "learning_rate": 1.4634146341463415e-05,
-      "loss": 3.80194433091674e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 749555.0,
       "step": 153
     },
     {
-      "entropy": 0.41627938114106655,
       "epoch": 1.88208269525268,
-      "grad_norm": 0.00018405914306640625,
       "learning_rate": 1.3414634146341466e-05,
-      "loss": 3.8107638829387724e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 755347.0,
       "step": 154
     },
     {
-      "entropy": 0.44089478626847267,
       "epoch": 1.894333843797856,
-      "grad_norm": 0.00029754638671875,
       "learning_rate": 1.2195121951219513e-05,
-      "loss": 4.713048838311806e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 760505.0,
       "step": 155
     },
     {
-      "entropy": 0.4184252228587866,
       "epoch": 1.9065849923430322,
-      "grad_norm": 0.000301361083984375,
       "learning_rate": 1.0975609756097562e-05,
-      "loss": 5.365146716940217e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 765413.0,
       "step": 156
     },
     {
-      "entropy": 0.41183059848845005,
       "epoch": 1.9188361408882084,
-      "grad_norm": 0.0257568359375,
       "learning_rate": 9.756097560975611e-06,
-      "loss": 0.0007220981642603874,
-      "mean_token_accuracy": 0.9997568093240261,
-      "num_tokens": 770650.0,
       "step": 157
     },
     {
-      "entropy": 0.4214022643864155,
       "epoch": 1.9310872894333844,
-      "grad_norm": 0.000244140625,
       "learning_rate": 8.53658536585366e-06,
-      "loss": 4.793080370291136e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 775650.0,
       "step": 158
     },
     {
-      "entropy": 0.4058344177901745,
       "epoch": 1.9433384379785605,
-      "grad_norm": 0.000362396240234375,
       "learning_rate": 7.317073170731707e-06,
-      "loss": 6.000606663292274e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 779848.0,
       "step": 159
     },
     {
-      "entropy": 0.41280501522123814,
       "epoch": 1.9555895865237365,
-      "grad_norm": 0.00061798095703125,
       "learning_rate": 6.0975609756097564e-06,
-      "loss": 8.753919246373698e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 784522.0,
       "step": 160
     },
     {
-      "entropy": 0.4334367923438549,
       "epoch": 1.9678407350689127,
-      "grad_norm": 0.0003643035888671875,
       "learning_rate": 4.8780487804878055e-06,
-      "loss": 6.043446410330944e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 790217.0,
       "step": 161
     },
     {
-      "entropy": 0.41199295595288277,
       "epoch": 1.9800918836140888,
-      "grad_norm": 0.00101470947265625,
       "learning_rate": 3.6585365853658537e-06,
-      "loss": 9.327918814960867e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 794982.0,
       "step": 162
     },
     {
-      "entropy": 0.43511078506708145,
       "epoch": 1.992343032159265,
-      "grad_norm": 0.000171661376953125,
       "learning_rate": 2.4390243902439027e-06,
-      "loss": 4.208434984320775e-05,
-      "mean_token_accuracy": 1.0,
-      "num_tokens": 800602.0,
       "step": 163
     },
     {
-      "entropy": 0.42249701023101804,
       "epoch": 2.0,
-      "grad_norm": 0.037353515625,
       "learning_rate": 1.2195121951219514e-06,
-      "loss": 0.0015817588428035378,
-      "mean_token_accuracy": 0.9995260655879974,
-      "num_tokens": 804258.0,
       "step": 164
     }
   ],
@@ -1700,7 +1700,7 @@
       "attributes": {}
     }
   },
-  "total_flos": 3.641777239390618e+16,
   "train_batch_size": 1,
   "trial_name": null,
   "trial_params": null

   "is_world_process_zero": true,
   "log_history": [
     {
+      "entropy": 0.2316489452496171,
       "epoch": 0.01225114854517611,
+      "grad_norm": 1.21875,
       "learning_rate": 0.0002,
+      "loss": 0.1141367182135582,
+      "mean_token_accuracy": 0.962372187525034,
+      "num_tokens": 6133.0,
       "step": 1
     },
     {
+      "entropy": 0.2494401354342699,
       "epoch": 0.02450229709035222,
+      "grad_norm": 0.59765625,
       "learning_rate": 0.00019878048780487805,
+      "loss": 0.07354862987995148,
+      "mean_token_accuracy": 0.9755491837859154,
+      "num_tokens": 12088.0,
       "step": 2
     },
     {
+      "entropy": 0.31152926199138165,
       "epoch": 0.036753445635528334,
+      "grad_norm": 0.306640625,
       "learning_rate": 0.0001975609756097561,
+      "loss": 0.06412772834300995,
+      "mean_token_accuracy": 0.978853102773428,
+      "num_tokens": 17331.0,
       "step": 3
     },
     {
+      "entropy": 0.30638211220502853,
       "epoch": 0.04900459418070444,
+      "grad_norm": 0.8984375,
       "learning_rate": 0.00019634146341463416,
+      "loss": 0.08034519106149673,
+      "mean_token_accuracy": 0.9723691493272781,
+      "num_tokens": 22383.0,
       "step": 4
     },
     {
+      "entropy": 0.3171741934493184,
       "epoch": 0.06125574272588055,
+      "grad_norm": 0.60546875,
       "learning_rate": 0.0001951219512195122,
+      "loss": 0.07083277404308319,
+      "mean_token_accuracy": 0.9742059484124184,
+      "num_tokens": 27930.0,
       "step": 5
     },
     {
+      "entropy": 0.3094687405973673,
       "epoch": 0.07350689127105667,
+      "grad_norm": 0.6796875,
       "learning_rate": 0.00019390243902439025,
+      "loss": 0.08443780243396759,
+      "mean_token_accuracy": 0.9732540361583233,
+      "num_tokens": 33286.0,
       "step": 6
     },
     {
+      "entropy": 0.2914603017270565,
       "epoch": 0.08575803981623277,
+      "grad_norm": 0.265625,
       "learning_rate": 0.0001926829268292683,
+      "loss": 0.06558080017566681,
+      "mean_token_accuracy": 0.9725310951471329,
+      "num_tokens": 39568.0,
       "step": 7
     },
     {
+      "entropy": 0.279434559866786,
       "epoch": 0.09800918836140889,
+      "grad_norm": 0.58984375,
       "learning_rate": 0.00019146341463414633,
+      "loss": 0.07338608056306839,
+      "mean_token_accuracy": 0.9793376848101616,
+      "num_tokens": 44597.0,
       "step": 8
     },
     {
+      "entropy": 0.27481516171246767,
       "epoch": 0.11026033690658499,
+      "grad_norm": 0.3125,
       "learning_rate": 0.0001902439024390244,
+      "loss": 0.06733334064483643,
+      "mean_token_accuracy": 0.9732998013496399,
+      "num_tokens": 49848.0,
       "step": 9
     },
     {
+      "entropy": 0.2752347318455577,
       "epoch": 0.1225114854517611,
+      "grad_norm": 0.4296875,
       "learning_rate": 0.00018902439024390244,
+      "loss": 0.08688339591026306,
+      "mean_token_accuracy": 0.9711812101304531,
+      "num_tokens": 55087.0,
       "step": 10
     },
     {
+      "entropy": 0.23697010707110167,
       "epoch": 0.13476263399693722,
+      "grad_norm": 0.35546875,
       "learning_rate": 0.0001878048780487805,
+      "loss": 0.09419302642345428,
+      "mean_token_accuracy": 0.9671205654740334,
+      "num_tokens": 61901.0,
       "step": 11
     },
     {
+      "entropy": 0.2767820842564106,
       "epoch": 0.14701378254211334,
+      "grad_norm": 0.5078125,
       "learning_rate": 0.00018658536585365856,
+      "loss": 0.09175145626068115,
+      "mean_token_accuracy": 0.9672112688422203,
+      "num_tokens": 68472.0,
       "step": 12
     },
     {
+      "entropy": 0.2712240917608142,
       "epoch": 0.15926493108728942,
+      "grad_norm": 0.43359375,
       "learning_rate": 0.0001853658536585366,
+      "loss": 0.1060388907790184,
+      "mean_token_accuracy": 0.9682641178369522,
+      "num_tokens": 74380.0,
       "step": 13
     },
     {
+      "entropy": 0.2655314621515572,
       "epoch": 0.17151607963246554,
+      "grad_norm": 0.5234375,
       "learning_rate": 0.00018414634146341464,
+      "loss": 0.09543660283088684,
+      "mean_token_accuracy": 0.9580898210406303,
+      "num_tokens": 80297.0,
       "step": 14
     },
     {
+      "entropy": 0.2568928087130189,
       "epoch": 0.18376722817764166,
+      "grad_norm": 0.306640625,
       "learning_rate": 0.0001829268292682927,
+      "loss": 0.05766459181904793,
+      "mean_token_accuracy": 0.9795842878520489,
+      "num_tokens": 85162.0,
       "step": 15
     },
     {
+      "entropy": 0.27691631484776735,
       "epoch": 0.19601837672281777,
+      "grad_norm": 0.359375,
       "learning_rate": 0.00018170731707317075,
+      "loss": 0.0939052402973175,
+      "mean_token_accuracy": 0.9671713933348656,
+      "num_tokens": 90393.0,
       "step": 16
     },
     {
+      "entropy": 0.2810298567637801,
       "epoch": 0.2082695252679939,
+      "grad_norm": 0.26953125,
       "learning_rate": 0.0001804878048780488,
+      "loss": 0.058892831206321716,
+      "mean_token_accuracy": 0.9773643910884857,
+      "num_tokens": 95530.0,
       "step": 17
     },
     {
+      "entropy": 0.2796283131465316,
       "epoch": 0.22052067381316998,
+      "grad_norm": 0.345703125,
       "learning_rate": 0.00017926829268292684,
+      "loss": 0.07744893431663513,
+      "mean_token_accuracy": 0.9721782878041267,
+      "num_tokens": 101234.0,
       "step": 18
     },
     {
+      "entropy": 0.2912421654909849,
       "epoch": 0.2327718223583461,
+      "grad_norm": 0.48828125,
       "learning_rate": 0.00017804878048780488,
+      "loss": 0.07593704760074615,
+      "mean_token_accuracy": 0.9668422974646091,
+      "num_tokens": 107018.0,
       "step": 19
     },
     {
+      "entropy": 0.28678335808217525,
       "epoch": 0.2450229709035222,
+      "grad_norm": 0.337890625,
       "learning_rate": 0.00017682926829268295,
+      "loss": 0.07227691262960434,
+      "mean_token_accuracy": 0.9736582525074482,
+      "num_tokens": 112299.0,
       "step": 20
     },
     {
+      "entropy": 0.296040833927691,
       "epoch": 0.2572741194486983,
+      "grad_norm": 0.33203125,
       "learning_rate": 0.000175609756097561,
+      "loss": 0.07230418920516968,
+      "mean_token_accuracy": 0.9750959761440754,
+      "num_tokens": 117872.0,
       "step": 21
     },
     {
+      "entropy": 0.27195548359304667,
       "epoch": 0.26952526799387444,
+      "grad_norm": 0.3671875,
       "learning_rate": 0.00017439024390243903,
+      "loss": 0.08706101030111313,
+      "mean_token_accuracy": 0.9771376326680183,
+      "num_tokens": 124580.0,
       "step": 22
     },
     {
+      "entropy": 0.29904199205338955,
       "epoch": 0.28177641653905056,
+      "grad_norm": 0.408203125,
       "learning_rate": 0.00017317073170731708,
+      "loss": 0.0653143897652626,
+      "mean_token_accuracy": 0.9760479032993317,
+      "num_tokens": 129745.0,
       "step": 23
     },
     {
+      "entropy": 0.2986137717962265,
       "epoch": 0.29402756508422667,
+      "grad_norm": 0.421875,
       "learning_rate": 0.00017195121951219512,
+      "loss": 0.07193314284086227,
+      "mean_token_accuracy": 0.9698839113116264,
+      "num_tokens": 135543.0,
       "step": 24
     },
     {
+      "entropy": 0.24683671910315752,
       "epoch": 0.30627871362940273,
+      "grad_norm": 0.37890625,
       "learning_rate": 0.0001707317073170732,
+      "loss": 0.07017349451780319,
+      "mean_token_accuracy": 0.9763788469135761,
+      "num_tokens": 141145.0,
       "step": 25
     },
     {
+      "entropy": 0.23581106960773468,
       "epoch": 0.31852986217457885,
+      "grad_norm": 0.349609375,
       "learning_rate": 0.00016951219512195123,
+      "loss": 0.07848861813545227,
+      "mean_token_accuracy": 0.9711455926299095,
+      "num_tokens": 146832.0,
       "step": 26
     },
     {
+      "entropy": 0.19877766259014606,
       "epoch": 0.33078101071975496,
+      "grad_norm": 0.32421875,
       "learning_rate": 0.00016829268292682927,
+      "loss": 0.05964134261012077,
+      "mean_token_accuracy": 0.9766620621085167,
+      "num_tokens": 153062.0,
       "step": 27
     },
     {
+      "entropy": 0.24412551056593657,
       "epoch": 0.3430321592649311,
+      "grad_norm": 0.466796875,
       "learning_rate": 0.00016707317073170731,
+      "loss": 0.10119230300188065,
+      "mean_token_accuracy": 0.9631960429251194,
+      "num_tokens": 159097.0,
       "step": 28
     },
     {
+      "entropy": 0.2634996743872762,
       "epoch": 0.3552833078101072,
+      "grad_norm": 0.376953125,
       "learning_rate": 0.00016585365853658536,
+      "loss": 0.07137235254049301,
+      "mean_token_accuracy": 0.9721279740333557,
+      "num_tokens": 164465.0,
       "step": 29
     },
     {
+      "entropy": 0.2398172626271844,
       "epoch": 0.3675344563552833,
+      "grad_norm": 0.380859375,
       "learning_rate": 0.00016463414634146343,
+      "loss": 0.08367905020713806,
+      "mean_token_accuracy": 0.9688702113926411,
+      "num_tokens": 171131.0,
       "step": 30
     },
     {
+      "entropy": 0.2387447776272893,
       "epoch": 0.37978560490045943,
+      "grad_norm": 0.39453125,
       "learning_rate": 0.00016341463414634147,
+      "loss": 0.07410822808742523,
+      "mean_token_accuracy": 0.9765294268727303,
+      "num_tokens": 175655.0,
       "step": 31
     },
     {
+      "entropy": 0.24556818418204784,
       "epoch": 0.39203675344563554,
+      "grad_norm": 0.361328125,
       "learning_rate": 0.00016219512195121954,
+      "loss": 0.07339000701904297,
+      "mean_token_accuracy": 0.9750400222837925,
+      "num_tokens": 182309.0,
       "step": 32
     },
     {
+      "entropy": 0.23958251252770424,
       "epoch": 0.40428790199081166,
+      "grad_norm": 0.376953125,
       "learning_rate": 0.00016097560975609758,
+      "loss": 0.0825161263346672,
+      "mean_token_accuracy": 0.9695910774171352,
+      "num_tokens": 188122.0,
       "step": 33
     },
     {
+      "entropy": 0.25066179782152176,
       "epoch": 0.4165390505359878,
+      "grad_norm": 0.34765625,
       "learning_rate": 0.00015975609756097562,
+      "loss": 0.0681036114692688,
+      "mean_token_accuracy": 0.9773549512028694,
+      "num_tokens": 193308.0,
       "step": 34
     },
     {
+      "entropy": 0.2489402163773775,
       "epoch": 0.42879019908116384,
+      "grad_norm": 0.33984375,
       "learning_rate": 0.00015853658536585366,
+      "loss": 0.07768924534320831,
+      "mean_token_accuracy": 0.9787707962095737,
+      "num_tokens": 198904.0,
       "step": 35
     },
     {
+      "entropy": 0.25176819786429405,
       "epoch": 0.44104134762633995,
+      "grad_norm": 0.353515625,
       "learning_rate": 0.00015731707317073173,
+      "loss": 0.07323021441698074,
+      "mean_token_accuracy": 0.9740425609052181,
+      "num_tokens": 204184.0,
       "step": 36
     },
     {
+      "entropy": 0.23491865396499634,
       "epoch": 0.45329249617151607,
+      "grad_norm": 0.345703125,
       "learning_rate": 0.00015609756097560978,
+      "loss": 0.06643179059028625,
+      "mean_token_accuracy": 0.9767155349254608,
+      "num_tokens": 210362.0,
       "step": 37
     },
     {
+      "entropy": 0.25266142282634974,
       "epoch": 0.4655436447166922,
+      "grad_norm": 0.50390625,
       "learning_rate": 0.00015487804878048782,
+      "loss": 0.08636192977428436,
+      "mean_token_accuracy": 0.9685244522988796,
+      "num_tokens": 215483.0,
       "step": 38
     },
     {
+      "entropy": 0.24919006042182446,
       "epoch": 0.4777947932618683,
+      "grad_norm": 0.357421875,
       "learning_rate": 0.00015365853658536586,
+      "loss": 0.06912290304899216,
+      "mean_token_accuracy": 0.9728152006864548,
+      "num_tokens": 220437.0,
       "step": 39
     },
     {
+      "entropy": 0.2789237005636096,
       "epoch": 0.4900459418070444,
+      "grad_norm": 0.3671875,
       "learning_rate": 0.0001524390243902439,
+      "loss": 0.07096827030181885,
+      "mean_token_accuracy": 0.9718564338982105,
+      "num_tokens": 225444.0,
       "step": 40
     },
     {
+      "entropy": 0.23915204405784607,
       "epoch": 0.5022970903522205,
+      "grad_norm": 0.35546875,
       "learning_rate": 0.00015121951219512197,
+      "loss": 0.06407603621482849,
+      "mean_token_accuracy": 0.975932989269495,
+      "num_tokens": 230088.0,
       "step": 41
     },
     {
+      "entropy": 0.25953691080212593,
       "epoch": 0.5145482388973966,
+      "grad_norm": 0.365234375,
       "learning_rate": 0.00015000000000000001,
+      "loss": 0.07893452048301697,
+      "mean_token_accuracy": 0.9717175625264645,
+      "num_tokens": 234974.0,
       "step": 42
     },
     {
+      "entropy": 0.25131134409457445,
       "epoch": 0.5267993874425727,
+      "grad_norm": 0.33984375,
       "learning_rate": 0.00014878048780487806,
+      "loss": 0.0724797397851944,
+      "mean_token_accuracy": 0.9746548496186733,
+      "num_tokens": 240695.0,
       "step": 43
     },
     {
+      "entropy": 0.25067666731774807,
       "epoch": 0.5390505359877489,
+      "grad_norm": 0.44921875,
       "learning_rate": 0.0001475609756097561,
+      "loss": 0.06145863234996796,
+      "mean_token_accuracy": 0.9786989763379097,
+      "num_tokens": 246515.0,
       "step": 44
     },
     {
+      "entropy": 0.22192941885441542,
       "epoch": 0.5513016845329249,
+      "grad_norm": 0.4375,
       "learning_rate": 0.00014634146341463414,
+      "loss": 0.06996186822652817,
+      "mean_token_accuracy": 0.9778482280671597,
+      "num_tokens": 252150.0,
       "step": 45
     },
     {
+      "entropy": 0.24868111684918404,
       "epoch": 0.5635528330781011,
+      "grad_norm": 0.392578125,
       "learning_rate": 0.0001451219512195122,
+      "loss": 0.07759839296340942,
+      "mean_token_accuracy": 0.9743853285908699,
+      "num_tokens": 257699.0,
       "step": 46
     },
     {
+      "entropy": 0.2405283828265965,
       "epoch": 0.5758039816232772,
+      "grad_norm": 0.400390625,
       "learning_rate": 0.00014390243902439025,
+      "loss": 0.06918229907751083,
+      "mean_token_accuracy": 0.9726257510483265,
+      "num_tokens": 262974.0,
       "step": 47
     },
     {
+      "entropy": 0.2463641557842493,
       "epoch": 0.5880551301684533,
+      "grad_norm": 0.5078125,
       "learning_rate": 0.0001426829268292683,
+      "loss": 0.08698121458292007,
+      "mean_token_accuracy": 0.9751730673015118,
+      "num_tokens": 267714.0,
       "step": 48
     },
     {
+      "entropy": 0.2611560570076108,
       "epoch": 0.6003062787136294,
+      "grad_norm": 0.3203125,
       "learning_rate": 0.00014146341463414634,
+      "loss": 0.0795765370130539,
+      "mean_token_accuracy": 0.9706047028303146,
+      "num_tokens": 273102.0,
       "step": 49
     },
     {
+      "entropy": 0.24631980434060097,
       "epoch": 0.6125574272588055,
+      "grad_norm": 0.365234375,
       "learning_rate": 0.00014024390243902438,
+      "loss": 0.06434721499681473,
+      "mean_token_accuracy": 0.9787219613790512,
+      "num_tokens": 278414.0,
       "step": 50
     },
     {
       "epoch": 0.6125574272588055,
+      "eval_entropy": 0.25439983627934387,
+      "eval_loss": 0.07568059861660004,
+      "eval_mean_token_accuracy": 0.9709554686062578,
+      "eval_num_tokens": 278414.0,
+      "eval_runtime": 56.679,
+      "eval_samples_per_second": 1.217,
+      "eval_steps_per_second": 1.217,
       "step": 50
     },
     {
+      "entropy": 0.22273720148950815,
       "epoch": 0.6248085758039816,
+      "grad_norm": 0.330078125,
       "learning_rate": 0.00013902439024390245,
+      "loss": 0.06272563338279724,
+      "mean_token_accuracy": 0.9790237173438072,
+      "num_tokens": 284001.0,
       "step": 51
     },
     {
+      "entropy": 0.25650967564433813,
       "epoch": 0.6370597243491577,
+      "grad_norm": 0.3515625,
       "learning_rate": 0.0001378048780487805,
+      "loss": 0.0695340633392334,
+      "mean_token_accuracy": 0.9723741784691811,
+      "num_tokens": 289900.0,
       "step": 52
     },
     {
+      "entropy": 0.27689922973513603,
       "epoch": 0.6493108728943339,
+      "grad_norm": 0.443359375,
       "learning_rate": 0.00013658536585365856,
+      "loss": 0.08247513324022293,
+      "mean_token_accuracy": 0.9751085750758648,
+      "num_tokens": 295774.0,
       "step": 53
     },
     {
+      "entropy": 0.24619914591312408,
       "epoch": 0.6615620214395099,
+      "grad_norm": 0.349609375,
       "learning_rate": 0.0001353658536585366,
+      "loss": 0.06673211604356766,
+      "mean_token_accuracy": 0.9788386225700378,
+      "num_tokens": 300970.0,
       "step": 54
     },
     {
+      "entropy": 0.27198443934321404,
       "epoch": 0.6738131699846861,
+      "grad_norm": 0.4921875,
       "learning_rate": 0.00013414634146341464,
+      "loss": 0.07676997035741806,
+      "mean_token_accuracy": 0.9696366749703884,
+      "num_tokens": 306709.0,
       "step": 55
     },
     {
+      "entropy": 0.2689105300232768,
       "epoch": 0.6860643185298622,
+      "grad_norm": 0.47265625,
       "learning_rate": 0.0001329268292682927,
+      "loss": 0.06719915568828583,
+      "mean_token_accuracy": 0.9702229462563992,
+      "num_tokens": 311650.0,
       "step": 56
     },
     {
+      "entropy": 0.2787257097661495,
       "epoch": 0.6983154670750383,
+      "grad_norm": 0.369140625,
       "learning_rate": 0.00013170731707317076,
+      "loss": 0.07159961760044098,
+      "mean_token_accuracy": 0.9748533591628075,
+      "num_tokens": 317257.0,
       "step": 57
     },
     {
+      "entropy": 0.2522663725540042,
       "epoch": 0.7105666156202144,
+      "grad_norm": 0.31640625,
       "learning_rate": 0.0001304878048780488,
+      "loss": 0.08856096863746643,
+      "mean_token_accuracy": 0.9697616137564182,
+      "num_tokens": 323281.0,
       "step": 58
     },
     {
+      "entropy": 0.24693416617810726,
       "epoch": 0.7228177641653905,
+      "grad_norm": 0.37109375,
       "learning_rate": 0.00012926829268292684,
+      "loss": 0.07423190027475357,
+      "mean_token_accuracy": 0.9705353751778603,
+      "num_tokens": 328551.0,
       "step": 59
     },
     {
+      "entropy": 0.2651137877255678,
       "epoch": 0.7350689127105666,
+      "grad_norm": 0.43359375,
       "learning_rate": 0.00012804878048780488,
+      "loss": 0.0738719031214714,
+      "mean_token_accuracy": 0.9752235859632492,
+      "num_tokens": 334143.0,
       "step": 60
     },
     {
+      "entropy": 0.2281778110191226,
       "epoch": 0.7473200612557427,
+      "grad_norm": 0.2490234375,
       "learning_rate": 0.00012682926829268293,
+      "loss": 0.0633026584982872,
+      "mean_token_accuracy": 0.9741999059915543,
+      "num_tokens": 341092.0,
       "step": 61
     },
     {
+      "entropy": 0.2535929596051574,
       "epoch": 0.7595712098009189,
+      "grad_norm": 0.390625,
       "learning_rate": 0.000125609756097561,
+      "loss": 0.0719546377658844,
+      "mean_token_accuracy": 0.9765410870313644,
+      "num_tokens": 347467.0,
       "step": 62
     },
     {
+      "entropy": 0.25424638390541077,
       "epoch": 0.7718223583460949,
+      "grad_norm": 0.431640625,
       "learning_rate": 0.00012439024390243904,
+      "loss": 0.05535401031374931,
+      "mean_token_accuracy": 0.9780425503849983,
+      "num_tokens": 352164.0,
       "step": 63
     },
     {
+      "entropy": 0.23888325225561857,
       "epoch": 0.7840735068912711,
+      "grad_norm": 0.435546875,
       "learning_rate": 0.00012317073170731708,
+      "loss": 0.07177040725946426,
+      "mean_token_accuracy": 0.9734687805175781,
+      "num_tokens": 357308.0,
       "step": 64
     },
     {
+      "entropy": 0.27028472628444433,
       "epoch": 0.7963246554364471,
+      "grad_norm": 0.30859375,
       "learning_rate": 0.00012195121951219512,
+      "loss": 0.06257087737321854,
+      "mean_token_accuracy": 0.9774579927325249,
+      "num_tokens": 362666.0,
       "step": 65
     },
     {
+      "entropy": 0.2821849435567856,
       "epoch": 0.8085758039816233,
+      "grad_norm": 0.373046875,
       "learning_rate": 0.00012073170731707318,
+      "loss": 0.06471723318099976,
+      "mean_token_accuracy": 0.976191334426403,
+      "num_tokens": 368427.0,
       "step": 66
     },
     {
+      "entropy": 0.22186184907332063,
       "epoch": 0.8208269525267994,
+      "grad_norm": 0.263671875,
       "learning_rate": 0.00011951219512195122,
+      "loss": 0.06329935044050217,
+      "mean_token_accuracy": 0.978707954287529,
+      "num_tokens": 374540.0,
       "step": 67
     },
     {
+      "entropy": 0.23882555402815342,
       "epoch": 0.8330781010719756,
+      "grad_norm": 0.3046875,
       "learning_rate": 0.00011829268292682926,
+      "loss": 0.07082124054431915,
+      "mean_token_accuracy": 0.979393869638443,
+      "num_tokens": 379925.0,
       "step": 68
     },
     {
+      "entropy": 0.2527451729401946,
       "epoch": 0.8453292496171516,
+      "grad_norm": 0.37109375,
       "learning_rate": 0.00011707317073170732,
+      "loss": 0.0804731696844101,
+      "mean_token_accuracy": 0.9763551540672779,
+      "num_tokens": 384279.0,
       "step": 69
     },
     {
+      "entropy": 0.26056139171123505,
       "epoch": 0.8575803981623277,
+      "grad_norm": 0.40234375,
       "learning_rate": 0.00011585365853658536,
+      "loss": 0.09266315400600433,
+      "mean_token_accuracy": 0.9709281474351883,
+      "num_tokens": 389563.0,
       "step": 70
     },
     {
+      "entropy": 0.2919591320678592,
       "epoch": 0.8698315467075038,
+      "grad_norm": 0.423828125,
       "learning_rate": 0.00011463414634146342,
+      "loss": 0.07172521948814392,
+      "mean_token_accuracy": 0.9725044220685959,
+      "num_tokens": 394650.0,
       "step": 71
     },
     {
+      "entropy": 0.2520558973774314,
       "epoch": 0.8820826952526799,
+      "grad_norm": 0.47265625,
       "learning_rate": 0.00011341463414634146,
+      "loss": 0.07857581228017807,
+      "mean_token_accuracy": 0.967189610004425,
+      "num_tokens": 399583.0,
       "step": 72
     },
     {
+      "entropy": 0.2681189738214016,
       "epoch": 0.8943338437978561,
+      "grad_norm": 0.470703125,
       "learning_rate": 0.00011219512195121953,
+      "loss": 0.0883592814207077,
+      "mean_token_accuracy": 0.9760300181806087,
+      "num_tokens": 406224.0,
       "step": 73
     },
     {
+      "entropy": 0.25226688850671053,
       "epoch": 0.9065849923430321,
+      "grad_norm": 0.349609375,
       "learning_rate": 0.00011097560975609757,
+      "loss": 0.06107043847441673,
+      "mean_token_accuracy": 0.9742026180028915,
+      "num_tokens": 412481.0,
       "step": 74
     },
     {
+      "entropy": 0.25610699970275164,
       "epoch": 0.9188361408882083,
+      "grad_norm": 0.4140625,
       "learning_rate": 0.00010975609756097563,
+      "loss": 0.06678957492113113,
+      "mean_token_accuracy": 0.9725399203598499,
+      "num_tokens": 417862.0,
       "step": 75
     },
     {
+      "entropy": 0.2826196616515517,
       "epoch": 0.9310872894333844,
+      "grad_norm": 0.859375,
       "learning_rate": 0.00010853658536585367,
+      "loss": 0.048859648406505585,
+      "mean_token_accuracy": 0.9790267050266266,
+      "num_tokens": 422878.0,
       "step": 76
     },
     {
+      "entropy": 0.23871563002467155,
       "epoch": 0.9433384379785605,
+      "grad_norm": 0.466796875,
       "learning_rate": 0.00010731707317073172,
+      "loss": 0.07596343755722046,
+      "mean_token_accuracy": 0.971769668161869,
+      "num_tokens": 429170.0,
       "step": 77
     },
     {
+      "entropy": 0.2777755409479141,
       "epoch": 0.9555895865237366,
+      "grad_norm": 0.443359375,
       "learning_rate": 0.00010609756097560977,
+      "loss": 0.06630191206932068,
+      "mean_token_accuracy": 0.9747902825474739,
+      "num_tokens": 434323.0,
       "step": 78
     },
     {
+      "entropy": 0.23950364720076323,
       "epoch": 0.9678407350689127,
+      "grad_norm": 0.349609375,
       "learning_rate": 0.00010487804878048781,
+      "loss": 0.057458702474832535,
+      "mean_token_accuracy": 0.980991818010807,
+      "num_tokens": 439539.0,
       "step": 79
     },
     {
+      "entropy": 0.245719694532454,
       "epoch": 0.9800918836140888,
+      "grad_norm": 0.3046875,
       "learning_rate": 0.00010365853658536586,
+      "loss": 0.06474918127059937,
+      "mean_token_accuracy": 0.9749566093087196,
+      "num_tokens": 445548.0,
       "step": 80
     },
     {
+      "entropy": 0.2553516002371907,
       "epoch": 0.9923430321592649,
+      "grad_norm": 0.59375,
       "learning_rate": 0.0001024390243902439,
+      "loss": 0.07626976072788239,
+      "mean_token_accuracy": 0.9740116000175476,
+      "num_tokens": 451007.0,
       "step": 81
     },
     {
+      "entropy": 0.24858922958374025,
       "epoch": 1.0,
+      "grad_norm": 0.4140625,
       "learning_rate": 0.00010121951219512196,
+      "loss": 0.05956536903977394,
+      "mean_token_accuracy": 0.9751910209655762,
+      "num_tokens": 454678.0,
       "step": 82
     },
     {
+      "entropy": 0.22480082791298628,
       "epoch": 1.0122511485451762,
+      "grad_norm": 0.302734375,
       "learning_rate": 0.0001,
+      "loss": 0.03318095952272415,
+      "mean_token_accuracy": 0.9908282831311226,
+      "num_tokens": 460195.0,
       "step": 83
     },
     {
+      "entropy": 0.21941375825554132,
       "epoch": 1.0245022970903521,
+      "grad_norm": 0.322265625,
       "learning_rate": 9.878048780487805e-05,
+      "loss": 0.037562280893325806,
+      "mean_token_accuracy": 0.9899826981127262,
+      "num_tokens": 465814.0,
       "step": 84
     },
     {
+      "entropy": 0.2297668270766735,
       "epoch": 1.0367534456355283,
+      "grad_norm": 0.259765625,
       "learning_rate": 9.75609756097561e-05,
+      "loss": 0.03667337819933891,
+      "mean_token_accuracy": 0.9867987670004368,
+      "num_tokens": 472919.0,
       "step": 85
     },
     {
+      "entropy": 0.1959990761242807,
       "epoch": 1.0490045941807045,
+      "grad_norm": 0.171875,
       "learning_rate": 9.634146341463415e-05,
+      "loss": 0.02224677987396717,
+      "mean_token_accuracy": 0.9947787970304489,
+      "num_tokens": 477926.0,
       "step": 86
     },
     {
+      "entropy": 0.22538460325449705,
       "epoch": 1.0612557427258806,
+      "grad_norm": 0.294921875,
       "learning_rate": 9.51219512195122e-05,
+      "loss": 0.05467130243778229,
+      "mean_token_accuracy": 0.9857094436883926,
+      "num_tokens": 483369.0,
       "step": 87
     },
     {
+      "entropy": 0.2385974396020174,
       "epoch": 1.0735068912710566,
+      "grad_norm": 0.2392578125,
       "learning_rate": 9.390243902439024e-05,
+      "loss": 0.02876465395092964,
+      "mean_token_accuracy": 0.9933567047119141,
+      "num_tokens": 488048.0,
       "step": 88
     },
     {
+      "entropy": 0.2244573337957263,
       "epoch": 1.0857580398162328,
+      "grad_norm": 0.17578125,
       "learning_rate": 9.26829268292683e-05,
+      "loss": 0.022544220089912415,
+      "mean_token_accuracy": 0.9952267222106457,
+      "num_tokens": 492951.0,
       "step": 89
     },
     {
+      "entropy": 0.21164159616455436,
       "epoch": 1.098009188361409,
+      "grad_norm": 0.3671875,
       "learning_rate": 9.146341463414635e-05,
+      "loss": 0.0307400431483984,
+      "mean_token_accuracy": 0.9898485280573368,
+      "num_tokens": 498298.0,
       "step": 90
     },
     {
+      "entropy": 0.22300960402935743,
       "epoch": 1.110260336906585,
+      "grad_norm": 0.25390625,
       "learning_rate": 9.02439024390244e-05,
+      "loss": 0.02349678799510002,
+      "mean_token_accuracy": 0.9937595501542091,
+      "num_tokens": 503013.0,
       "step": 91
     },
     {
+      "entropy": 0.2144601820036769,
       "epoch": 1.122511485451761,
+      "grad_norm": 0.466796875,
       "learning_rate": 8.902439024390244e-05,
+      "loss": 0.025124385952949524,
+      "mean_token_accuracy": 0.9929902292788029,
+      "num_tokens": 507687.0,
       "step": 92
     },
     {
+      "entropy": 0.18067707447335124,
       "epoch": 1.1347626339969372,
+      "grad_norm": 0.462890625,
       "learning_rate": 8.78048780487805e-05,
+      "loss": 0.04210633784532547,
+      "mean_token_accuracy": 0.9874051883816719,
+      "num_tokens": 513217.0,
       "step": 93
     },
     {
+      "entropy": 0.18840790819376707,
       "epoch": 1.1470137825421134,
+      "grad_norm": 0.2578125,
       "learning_rate": 8.658536585365854e-05,
+      "loss": 0.023590605705976486,
+      "mean_token_accuracy": 0.9930241219699383,
+      "num_tokens": 518384.0,
       "step": 94
     },
     {
+      "entropy": 0.16844777530059218,
       "epoch": 1.1592649310872893,
+      "grad_norm": 0.3046875,
       "learning_rate": 8.53658536585366e-05,
+      "loss": 0.02408467046916485,
+      "mean_token_accuracy": 0.9940578565001488,
+      "num_tokens": 523975.0,
       "step": 95
     },
     {
+      "entropy": 0.1988551402464509,
       "epoch": 1.1715160796324655,
+      "grad_norm": 0.25390625,
       "learning_rate": 8.414634146341464e-05,
+      "loss": 0.01896364614367485,
+      "mean_token_accuracy": 0.9935651384294033,
+      "num_tokens": 528838.0,
       "step": 96
     },
     {
+      "entropy": 0.19662938080728054,
       "epoch": 1.1837672281776417,
+      "grad_norm": 0.271484375,
       "learning_rate": 8.292682926829268e-05,
+      "loss": 0.023568641394376755,
+      "mean_token_accuracy": 0.9942812882363796,
+      "num_tokens": 533723.0,
       "step": 97
     },
     {
+      "entropy": 0.18521032202988863,
       "epoch": 1.1960183767228179,
+      "grad_norm": 0.2158203125,
       "learning_rate": 8.170731707317073e-05,
+      "loss": 0.03203809633851051,
+      "mean_token_accuracy": 0.9899982325732708,
+      "num_tokens": 540180.0,
       "step": 98
     },
     {
+      "entropy": 0.18826917372643948,
       "epoch": 1.2082695252679938,
+      "grad_norm": 0.4765625,
       "learning_rate": 8.048780487804879e-05,
+      "loss": 0.03463224321603775,
+      "mean_token_accuracy": 0.9889252111315727,
+      "num_tokens": 546618.0,
       "step": 99
     },
     {
+      "entropy": 0.1889605624601245,
       "epoch": 1.22052067381317,
+      "grad_norm": 0.337890625,
       "learning_rate": 7.926829268292683e-05,
+      "loss": 0.038746241480112076,
+      "mean_token_accuracy": 0.9897148124873638,
+      "num_tokens": 552084.0,
       "step": 100
     },
     {
       "epoch": 1.22052067381317,
+      "eval_entropy": 0.19684839270253113,
+      "eval_loss": 0.08200085908174515,
+      "eval_mean_token_accuracy": 0.9706140955289205,
+      "eval_num_tokens": 552084.0,
+      "eval_runtime": 56.6368,
+      "eval_samples_per_second": 1.218,
+      "eval_steps_per_second": 1.218,
       "step": 100
     },
     {
+      "entropy": 0.18781481962651014,
       "epoch": 1.2327718223583461,
+      "grad_norm": 0.2490234375,
       "learning_rate": 7.804878048780489e-05,
+      "loss": 0.03647669032216072,
+      "mean_token_accuracy": 0.9900195822119713,
+      "num_tokens": 558384.0,
       "step": 101
     },
     {
+      "entropy": 0.182833943516016,
       "epoch": 1.245022970903522,
+      "grad_norm": 0.1708984375,
       "learning_rate": 7.682926829268293e-05,
+      "loss": 0.01754325069487095,
+      "mean_token_accuracy": 0.9952104948461056,
+      "num_tokens": 564025.0,
       "step": 102
     },
     {
+      "entropy": 0.19512099027633667,
       "epoch": 1.2572741194486983,
+      "grad_norm": 0.32421875,
       "learning_rate": 7.560975609756099e-05,
+      "loss": 0.045042332261800766,
+      "mean_token_accuracy": 0.987647294998169,
+      "num_tokens": 569791.0,
       "step": 103
     },
     {
+      "entropy": 0.19775146059691906,
       "epoch": 1.2695252679938744,
+      "grad_norm": 0.287109375,
       "learning_rate": 7.439024390243903e-05,
+      "loss": 0.03481469675898552,
+      "mean_token_accuracy": 0.9876400642096996,
+      "num_tokens": 575432.0,
       "step": 104
     },
     {
+      "entropy": 0.19757689163088799,
       "epoch": 1.2817764165390506,
+      "grad_norm": 0.392578125,
       "learning_rate": 7.317073170731707e-05,
+      "loss": 0.045782968401908875,
+      "mean_token_accuracy": 0.987156193703413,
+      "num_tokens": 580586.0,
       "step": 105
     },
     {
+      "entropy": 0.19568088464438915,
       "epoch": 1.2940275650842268,
+      "grad_norm": 0.271484375,
       "learning_rate": 7.195121951219513e-05,
+      "loss": 0.03614577651023865,
+      "mean_token_accuracy": 0.989520326256752,
+      "num_tokens": 586255.0,
       "step": 106
     },
     {
+      "entropy": 0.18891402333974838,
       "epoch": 1.3062787136294027,
+      "grad_norm": 0.169921875,
       "learning_rate": 7.073170731707317e-05,
+      "loss": 0.018318383023142815,
+      "mean_token_accuracy": 0.9943608231842518,
+      "num_tokens": 591734.0,
       "step": 107
     },
     {
+      "entropy": 0.2118115657940507,
       "epoch": 1.318529862174579,
+      "grad_norm": 0.34375,
       "learning_rate": 6.951219512195122e-05,
+      "loss": 0.02556736022233963,
+      "mean_token_accuracy": 0.9910119064152241,
+      "num_tokens": 596805.0,
       "step": 108
     },
     {
+      "entropy": 0.20146753964945674,
       "epoch": 1.3307810107197549,
+      "grad_norm": 0.251953125,
       "learning_rate": 6.829268292682928e-05,
+      "loss": 0.026423780247569084,
+      "mean_token_accuracy": 0.9911187067627907,
+      "num_tokens": 602469.0,
       "step": 109
     },
     {
+      "entropy": 0.19927682168781757,
       "epoch": 1.343032159264931,
+      "grad_norm": 0.2314453125,
       "learning_rate": 6.707317073170732e-05,
+      "loss": 0.038182880729436874,
+      "mean_token_accuracy": 0.9882474392652512,
+      "num_tokens": 608854.0,
       "step": 110
     },
     {
+      "entropy": 0.18457680894061923,
       "epoch": 1.3552833078101072,
+      "grad_norm": 0.24609375,
       "learning_rate": 6.585365853658538e-05,
+      "loss": 0.025912806391716003,
+      "mean_token_accuracy": 0.9923904649913311,
+      "num_tokens": 614272.0,
       "step": 111
     },
     {
+      "entropy": 0.1993693085387349,
       "epoch": 1.3675344563552834,
+      "grad_norm": 0.291015625,
       "learning_rate": 6.463414634146342e-05,
+      "loss": 0.021378764882683754,
+      "mean_token_accuracy": 0.9953300580382347,
+      "num_tokens": 619446.0,
       "step": 112
     },
     {
+      "entropy": 0.19518085662275553,
       "epoch": 1.3797856049004595,
+      "grad_norm": 0.30078125,
       "learning_rate": 6.341463414634146e-05,
+      "loss": 0.03335938975214958,
+      "mean_token_accuracy": 0.9875492453575134,
+      "num_tokens": 625774.0,
       "step": 113
     },
     {
+      "entropy": 0.20890573505312204,
       "epoch": 1.3920367534456355,
+      "grad_norm": 0.373046875,
       "learning_rate": 6.219512195121952e-05,
+      "loss": 0.036217525601387024,
+      "mean_token_accuracy": 0.9891358688473701,
+      "num_tokens": 630747.0,
       "step": 114
     },
     {
+      "entropy": 0.19118426740169525,
       "epoch": 1.4042879019908117,
+      "grad_norm": 0.251953125,
       "learning_rate": 6.097560975609756e-05,
+      "loss": 0.030090918764472008,
+      "mean_token_accuracy": 0.9934539385139942,
+      "num_tokens": 637405.0,
       "step": 115
     },
     {
+      "entropy": 0.2176859974861145,
       "epoch": 1.4165390505359878,
+      "grad_norm": 0.2373046875,
       "learning_rate": 5.975609756097561e-05,
+      "loss": 0.024563392624258995,
+      "mean_token_accuracy": 0.9921185150742531,
+      "num_tokens": 642328.0,
       "step": 116
     },
     {
+      "entropy": 0.1849509342573583,
       "epoch": 1.4287901990811638,
+      "grad_norm": 0.35546875,
       "learning_rate": 5.853658536585366e-05,
+      "loss": 0.042349379509687424,
+      "mean_token_accuracy": 0.9899747557938099,
+      "num_tokens": 647857.0,
       "step": 117
     },
     {
+      "entropy": 0.19377889391034842,
       "epoch": 1.44104134762634,
+      "grad_norm": 0.279296875,
       "learning_rate": 5.731707317073171e-05,
+      "loss": 0.02413174696266651,
+      "mean_token_accuracy": 0.9931157529354095,
+      "num_tokens": 653805.0,
       "step": 118
     },
     {
+      "entropy": 0.20709845190867782,
       "epoch": 1.4532924961715161,
+      "grad_norm": 0.28125,
       "learning_rate": 5.6097560975609764e-05,
+      "loss": 0.03505600988864899,
+      "mean_token_accuracy": 0.9896740056574345,
+      "num_tokens": 659708.0,
       "step": 119
     },
     {
+      "entropy": 0.20671271299943328,
       "epoch": 1.4655436447166923,
+      "grad_norm": 0.2734375,
       "learning_rate": 5.487804878048781e-05,
+      "loss": 0.02634236589074135,
+      "mean_token_accuracy": 0.9935285076498985,
+      "num_tokens": 665292.0,
       "step": 120
     },
     {
+      "entropy": 0.18826642259955406,
       "epoch": 1.4777947932618682,
+      "grad_norm": 0.2177734375,
       "learning_rate": 5.365853658536586e-05,
+      "loss": 0.022179996594786644,
+      "mean_token_accuracy": 0.9928314089775085,
+      "num_tokens": 670669.0,
       "step": 121
     },
     {
+      "entropy": 0.2311026845127344,
       "epoch": 1.4900459418070444,
+      "grad_norm": 0.267578125,
       "learning_rate": 5.2439024390243904e-05,
+      "loss": 0.025521911680698395,
+      "mean_token_accuracy": 0.9930035471916199,
+      "num_tokens": 675524.0,
       "step": 122
     },
     {
+      "entropy": 0.1890636207535863,
       "epoch": 1.5022970903522204,
+      "grad_norm": 0.22265625,
       "learning_rate": 5.121951219512195e-05,
+      "loss": 0.02293182723224163,
+      "mean_token_accuracy": 0.9917827062308788,
+      "num_tokens": 681083.0,
       "step": 123
     },
     {
+      "entropy": 0.20301904529333115,
       "epoch": 1.5145482388973965,
+      "grad_norm": 0.251953125,
       "learning_rate": 5e-05,
+      "loss": 0.026392869651317596,
+      "mean_token_accuracy": 0.9935696609318256,
+      "num_tokens": 686909.0,
       "step": 124
     },
     {
+      "entropy": 0.18326633982360363,
       "epoch": 1.5267993874425727,
+      "grad_norm": 0.189453125,
       "learning_rate": 4.878048780487805e-05,
+      "loss": 0.03385050222277641,
+      "mean_token_accuracy": 0.9923080727458,
+      "num_tokens": 693716.0,
       "step": 125
     },
     {
+      "entropy": 0.1940352749079466,
       "epoch": 1.5390505359877489,
+      "grad_norm": 0.25,
       "learning_rate": 4.75609756097561e-05,
+      "loss": 0.03128973767161369,
+      "mean_token_accuracy": 0.9904795847833157,
+      "num_tokens": 699231.0,
       "step": 126
     },
     {
+      "entropy": 0.2052145255729556,
       "epoch": 1.551301684532925,
+      "grad_norm": 0.1962890625,
       "learning_rate": 4.634146341463415e-05,
+      "loss": 0.01906367763876915,
+      "mean_token_accuracy": 0.9935221113264561,
+      "num_tokens": 705026.0,
       "step": 127
     },
     {
+      "entropy": 0.22084870096296072,
       "epoch": 1.5635528330781012,
+      "grad_norm": 0.28125,
       "learning_rate": 4.51219512195122e-05,
+      "loss": 0.026771627366542816,
+      "mean_token_accuracy": 0.9931596331298351,
+      "num_tokens": 710155.0,
       "step": 128
     },
     {
+      "entropy": 0.18041892955079675,
       "epoch": 1.5758039816232772,
+      "grad_norm": 0.369140625,
       "learning_rate": 4.390243902439025e-05,
+      "loss": 0.024752795696258545,
+      "mean_token_accuracy": 0.9915198720991611,
+      "num_tokens": 715496.0,
       "step": 129
     },
     {
+      "entropy": 0.1869538608007133,
       "epoch": 1.5880551301684533,
+      "grad_norm": 0.3046875,
       "learning_rate": 4.26829268292683e-05,
+      "loss": 0.03293408453464508,
+      "mean_token_accuracy": 0.990137055516243,
+      "num_tokens": 721491.0,
       "step": 130
     },
     {
+      "entropy": 0.20515098702162504,
       "epoch": 1.6003062787136293,
+      "grad_norm": 0.349609375,
       "learning_rate": 4.146341463414634e-05,
+      "loss": 0.023330464959144592,
+      "mean_token_accuracy": 0.9892629720270634,
+      "num_tokens": 726673.0,
       "step": 131
     },
     {
+      "entropy": 0.18135815067216754,
       "epoch": 1.6125574272588055,
+      "grad_norm": 0.357421875,
       "learning_rate": 4.0243902439024395e-05,
+      "loss": 0.03119005262851715,
+      "mean_token_accuracy": 0.9911304786801338,
+      "num_tokens": 733054.0,
       "step": 132
     },
     {
+      "entropy": 0.20070009911432862,
       "epoch": 1.6248085758039816,
+      "grad_norm": 0.21484375,
       "learning_rate": 3.9024390243902444e-05,
+      "loss": 0.030009731650352478,
+      "mean_token_accuracy": 0.9932212419807911,
+      "num_tokens": 737990.0,
       "step": 133
     },
     {
+      "entropy": 0.18819584511220455,
       "epoch": 1.6370597243491578,
+      "grad_norm": 0.2451171875,
       "learning_rate": 3.780487804878049e-05,
+      "loss": 0.02752860262989998,
+      "mean_token_accuracy": 0.9897669702768326,
+      "num_tokens": 743394.0,
       "step": 134
     },
     {
+      "entropy": 0.18869836069643497,
       "epoch": 1.649310872894334,
+      "grad_norm": 0.240234375,
       "learning_rate": 3.6585365853658535e-05,
+      "loss": 0.03194504603743553,
+      "mean_token_accuracy": 0.9914098270237446,
+      "num_tokens": 749356.0,
       "step": 135
     },
     {
+      "entropy": 0.2093992899172008,
       "epoch": 1.66156202143951,
+      "grad_norm": 0.291015625,
       "learning_rate": 3.5365853658536584e-05,
+      "loss": 0.02633955329656601,
+      "mean_token_accuracy": 0.992473166435957,
+      "num_tokens": 754312.0,
       "step": 136
     },
     {
+      "entropy": 0.1928223273716867,
       "epoch": 1.673813169984686,
+      "grad_norm": 0.2470703125,
       "learning_rate": 3.414634146341464e-05,
+      "loss": 0.035037778317928314,
+      "mean_token_accuracy": 0.9916842468082905,
+      "num_tokens": 760182.0,
       "step": 137
     },
     {
+      "entropy": 0.19663999788463116,
       "epoch": 1.686064318529862,
+      "grad_norm": 0.265625,
       "learning_rate": 3.292682926829269e-05,
+      "loss": 0.03151565045118332,
+      "mean_token_accuracy": 0.9930234625935555,
+      "num_tokens": 766267.0,
       "step": 138
     },
     {
+      "entropy": 0.2058473015204072,
       "epoch": 1.6983154670750382,
+      "grad_norm": 0.2578125,
       "learning_rate": 3.170731707317073e-05,
+      "loss": 0.02509160526096821,
+      "mean_token_accuracy": 0.9920520819723606,
+      "num_tokens": 771135.0,
       "step": 139
     },
     {
+      "entropy": 0.20955495908856392,
       "epoch": 1.7105666156202144,
+      "grad_norm": 0.36328125,
       "learning_rate": 3.048780487804878e-05,
+      "loss": 0.03856905177235603,
+      "mean_token_accuracy": 0.9877506978809834,
+      "num_tokens": 776727.0,
       "step": 140
     },
     {
+      "entropy": 0.17796193715184927,
       "epoch": 1.7228177641653906,
+      "grad_norm": 0.271484375,
       "learning_rate": 2.926829268292683e-05,
+      "loss": 0.03061492368578911,
+      "mean_token_accuracy": 0.9933489374816418,
+      "num_tokens": 782352.0,
       "step": 141
     },
     {
+      "entropy": 0.19299636129289865,
       "epoch": 1.7350689127105667,
+      "grad_norm": 0.2392578125,
       "learning_rate": 2.8048780487804882e-05,
+      "loss": 0.03383423760533333,
+      "mean_token_accuracy": 0.9913677796721458,
+      "num_tokens": 787139.0,
       "step": 142
     },
     {
+      "entropy": 0.2032350143417716,
       "epoch": 1.7473200612557427,
+      "grad_norm": 0.314453125,
       "learning_rate": 2.682926829268293e-05,
+      "loss": 0.03458622097969055,
+      "mean_token_accuracy": 0.9920257851481438,
+      "num_tokens": 792244.0,
       "step": 143
     },
     {
+      "entropy": 0.21589675825089216,
       "epoch": 1.7595712098009189,
+      "grad_norm": 0.27734375,
       "learning_rate": 2.5609756097560977e-05,
+      "loss": 0.029654916375875473,
+      "mean_token_accuracy": 0.9936717823147774,
+      "num_tokens": 797998.0,
       "step": 144
     },
     {
+      "entropy": 0.19791326764971018,
       "epoch": 1.7718223583460948,
+      "grad_norm": 0.1748046875,
       "learning_rate": 2.4390243902439026e-05,
+      "loss": 0.019491517916321754,
+      "mean_token_accuracy": 0.9953687153756618,
+      "num_tokens": 803118.0,
       "step": 145
     },
     {
+      "entropy": 0.19606765313073993,
       "epoch": 1.784073506891271,
+      "grad_norm": 0.2236328125,
       "learning_rate": 2.3170731707317075e-05,
+      "loss": 0.017046257853507996,
+      "mean_token_accuracy": 0.9934666827321053,
+      "num_tokens": 808709.0,
       "step": 146
     },
     {
+      "entropy": 0.17984948493540287,
       "epoch": 1.7963246554364471,
+      "grad_norm": 0.2119140625,
       "learning_rate": 2.1951219512195124e-05,
+      "loss": 0.028008146211504936,
+      "mean_token_accuracy": 0.9918750263750553,
+      "num_tokens": 815053.0,
       "step": 147
     },
     {
+      "entropy": 0.19215012807399035,
       "epoch": 1.8085758039816233,
+      "grad_norm": 0.212890625,
       "learning_rate": 2.073170731707317e-05,
+      "loss": 0.02620745822787285,
+      "mean_token_accuracy": 0.9895812347531319,
+      "num_tokens": 821046.0,
       "step": 148
     },
     {
+      "entropy": 0.1954274857416749,
       "epoch": 1.8208269525267995,
+      "grad_norm": 0.1630859375,
       "learning_rate": 1.9512195121951222e-05,
+      "loss": 0.012469938956201077,
+      "mean_token_accuracy": 0.9970379211008549,
+      "num_tokens": 825773.0,
       "step": 149
     },
     {
+      "entropy": 0.20444792695343494,
       "epoch": 1.8330781010719757,
+      "grad_norm": 0.3671875,
       "learning_rate": 1.8292682926829268e-05,
+      "loss": 0.029102876782417297,
+      "mean_token_accuracy": 0.9916210547089577,
+      "num_tokens": 831944.0,
       "step": 150
     },
     {
       "epoch": 1.8330781010719757,
+      "eval_entropy": 0.20245846825233405,
+      "eval_loss": 0.07568201422691345,
+      "eval_mean_token_accuracy": 0.973983341369076,
+      "eval_num_tokens": 831944.0,
+      "eval_runtime": 56.7259,
+      "eval_samples_per_second": 1.216,
+      "eval_steps_per_second": 1.216,
       "step": 150
     },
     {
+      "entropy": 0.18990392005071044,
       "epoch": 1.8453292496171516,
+      "grad_norm": 0.2734375,
       "learning_rate": 1.707317073170732e-05,
+      "loss": 0.019072150811553,
+      "mean_token_accuracy": 0.9943390414118767,
+      "num_tokens": 836732.0,
       "step": 151
     },
     {
+      "entropy": 0.2014783564954996,
       "epoch": 1.8575803981623276,
+      "grad_norm": 0.232421875,
       "learning_rate": 1.5853658536585366e-05,
+      "loss": 0.02772960253059864,
+      "mean_token_accuracy": 0.9943372644484043,
+      "num_tokens": 843548.0,
       "step": 152
     },
     {
+      "entropy": 0.20029952516779304,
       "epoch": 1.8698315467075037,
+      "grad_norm": 0.291015625,
       "learning_rate": 1.4634146341463415e-05,
+      "loss": 0.029696376994252205,
+      "mean_token_accuracy": 0.9905722960829735,
+      "num_tokens": 849264.0,
       "step": 153
     },
     {
+      "entropy": 0.1881282702088356,
       "epoch": 1.88208269525268,
+      "grad_norm": 0.1953125,
       "learning_rate": 1.3414634146341466e-05,
+      "loss": 0.01899532601237297,
+      "mean_token_accuracy": 0.9949756152927876,
+      "num_tokens": 855941.0,
       "step": 154
     },
     {
+      "entropy": 0.20087886042892933,
       "epoch": 1.894333843797856,
+      "grad_norm": 0.28125,
       "learning_rate": 1.2195121951219513e-05,
+      "loss": 0.027130059897899628,
+      "mean_token_accuracy": 0.990227460861206,
+      "num_tokens": 861676.0,
       "step": 155
     },
     {
+      "entropy": 0.18239097949117422,
       "epoch": 1.9065849923430322,
+      "grad_norm": 0.2255859375,
       "learning_rate": 1.0975609756097562e-05,
+      "loss": 0.021481823176145554,
+      "mean_token_accuracy": 0.994240652769804,
+      "num_tokens": 867376.0,
       "step": 156
     },
     {
+      "entropy": 0.19513252703472972,
       "epoch": 1.9188361408882084,
+      "grad_norm": 0.390625,
       "learning_rate": 9.756097560975611e-06,
+      "loss": 0.03814350813627243,
+      "mean_token_accuracy": 0.9888629019260406,
+      "num_tokens": 873357.0,
       "step": 157
     },
     {
+      "entropy": 0.19895873684436083,
       "epoch": 1.9310872894333844,
+      "grad_norm": 0.28515625,
       "learning_rate": 8.53658536585366e-06,
+      "loss": 0.030593648552894592,
+      "mean_token_accuracy": 0.9909784123301506,
+      "num_tokens": 879168.0,
       "step": 158
     },
     {
+      "entropy": 0.19939070381224155,
       "epoch": 1.9433384379785605,
+      "grad_norm": 0.2392578125,
       "learning_rate": 7.317073170731707e-06,
+      "loss": 0.03540084883570671,
+      "mean_token_accuracy": 0.9887162260711193,
+      "num_tokens": 884239.0,
       "step": 159
     },
     {
+      "entropy": 0.19125983119010925,
       "epoch": 1.9555895865237365,
+      "grad_norm": 0.2890625,
       "learning_rate": 6.0975609756097564e-06,
+      "loss": 0.03378206118941307,
+      "mean_token_accuracy": 0.9916012957692146,
+      "num_tokens": 889409.0,
       "step": 160
     },
     {
+      "entropy": 0.21400849102064967,
       "epoch": 1.9678407350689127,
+      "grad_norm": 0.208984375,
       "learning_rate": 4.8780487804878055e-06,
+      "loss": 0.033363211899995804,
+      "mean_token_accuracy": 0.9923242144286633,
+      "num_tokens": 895282.0,
       "step": 161
     },
     {
+      "entropy": 0.22280079126358032,
       "epoch": 1.9800918836140888,
+      "grad_norm": 0.181640625,
       "learning_rate": 3.6585365853658537e-06,
+      "loss": 0.016371803358197212,
+      "mean_token_accuracy": 0.9944233559072018,
+      "num_tokens": 899869.0,
       "step": 162
     },
     {
+      "entropy": 0.21370396204292774,
       "epoch": 1.992343032159265,
+      "grad_norm": 0.279296875,
       "learning_rate": 2.4390243902439027e-06,
+      "loss": 0.024234982207417488,
+      "mean_token_accuracy": 0.9939975440502167,
+      "num_tokens": 905419.0,
       "step": 163
     },
     {
+      "entropy": 0.19501846730709077,
       "epoch": 2.0,
+      "grad_norm": 0.244140625,
       "learning_rate": 1.2195121951219514e-06,
+      "loss": 0.021054470911622047,
+      "mean_token_accuracy": 0.9963582038879395,
+      "num_tokens": 909356.0,
       "step": 164
     }
   ],
       "attributes": {}
     }
   },
+  "total_flos": 4.117673661068083e+16,
   "train_batch_size": 1,
   "trial_name": null,
   "trial_params": null