Hugging Face's logo

Augusto777
/
swinv2-tiny-patch4-window8-256-DMAE-15e

TensorBoard
Safetensors
swinv2
Generated from Trainer
Eval Results (legacy)
Model card Files
xet
 Metrics Community
Augusto777 commited on
Commit
fd4d5a2
·
verified ·
1 Parent(s): eec823c

End of training

Browse files
Files changed (14) hide show
  1. README.md +111 -0
  2. all_results.json +12 -0
  3. config.json +58 -0
  4. eval_results.json +8 -0
  5. model.safetensors +3 -0
  6. preprocessor_config.json +22 -0
  7. runs/Dec02_10-49-51_DESKTOP-SKBE9FB/events.out.tfevents.1733158192.DESKTOP-SKBE9FB.7232.0 +3 -0
  8. runs/Dec02_10-49-51_DESKTOP-SKBE9FB/events.out.tfevents.1733158388.DESKTOP-SKBE9FB.7232.1 +3 -0
  9. runs/Dec02_10-54-50_DESKTOP-SKBE9FB/events.out.tfevents.1733158491.DESKTOP-SKBE9FB.11632.0 +3 -0
  10. runs/Dec02_10-56-42_DESKTOP-SKBE9FB/events.out.tfevents.1733158603.DESKTOP-SKBE9FB.18460.0 +3 -0
  11. runs/Dec02_10-56-42_DESKTOP-SKBE9FB/events.out.tfevents.1733158797.DESKTOP-SKBE9FB.18460.1 +3 -0
  12. train_results.json +7 -0
  13. trainer_state.json +417 -0
  14. training_args.bin +3 -0
README.md ADDED
@@ -0,0 +1,111 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: microsoft/swinv2-tiny-patch4-window8-256
4
+ tags:
5
+ - generated_from_trainer
6
+ datasets:
7
+ - imagefolder
8
+ metrics:
9
+ - accuracy
10
+ model-index:
11
+ - name: swinv2-tiny-patch4-window8-256-DMAE-15e
12
+ results:
13
+ - task:
14
+ name: Image Classification
15
+ type: image-classification
16
+ dataset:
17
+ name: imagefolder
18
+ type: imagefolder
19
+ config: default
20
+ split: validation
21
+ args: default
22
+ metrics:
23
+ - name: Accuracy
24
+ type: accuracy
25
+ value: 0.10869565217391304
26
+ ---
27
+
28
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
29
+ should probably proofread and complete it, then remove this comment. -->
30
+
31
+ # swinv2-tiny-patch4-window8-256-DMAE-15e
32
+
33
+ This model is a fine-tuned version of [microsoft/swinv2-tiny-patch4-window8-256](https://huggingface.co/microsoft/swinv2-tiny-patch4-window8-256) on the imagefolder dataset.
34
+ It achieves the following results on the evaluation set:
35
+ - Loss: 7.9238
36
+ - Accuracy: 0.1087
37
+
38
+ ## Model description
39
+
40
+ More information needed
41
+
42
+ ## Intended uses & limitations
43
+
44
+ More information needed
45
+
46
+ ## Training and evaluation data
47
+
48
+ More information needed
49
+
50
+ ## Training procedure
51
+
52
+ ### Training hyperparameters
53
+
54
+ The following hyperparameters were used during training:
55
+ - learning_rate: 1.5e-05
56
+ - train_batch_size: 16
57
+ - eval_batch_size: 16
58
+ - seed: 42
59
+ - gradient_accumulation_steps: 4
60
+ - total_train_batch_size: 64
61
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
62
+ - lr_scheduler_type: linear
63
+ - num_epochs: 40
64
+
65
+ ### Training results
66
+
67
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy |
68
+ |:-------------:|:-----:|:----:|:---------------:|:--------:|
69
+ | No log | 0.86 | 3 | 7.9238 | 0.1087 |
70
+ | No log | 2.0 | 7 | 7.8746 | 0.1087 |
71
+ | 7.9273 | 2.86 | 10 | 7.8185 | 0.1087 |
72
+ | 7.9273 | 4.0 | 14 | 7.6996 | 0.1087 |
73
+ | 7.9273 | 4.86 | 17 | 7.5876 | 0.1087 |
74
+ | 7.6529 | 6.0 | 21 | 7.4189 | 0.1087 |
75
+ | 7.6529 | 6.86 | 24 | 7.2678 | 0.1087 |
76
+ | 7.6529 | 8.0 | 28 | 7.0489 | 0.1087 |
77
+ | 7.1056 | 8.86 | 31 | 6.8860 | 0.1087 |
78
+ | 7.1056 | 10.0 | 35 | 6.6867 | 0.1087 |
79
+ | 7.1056 | 10.86 | 38 | 6.5549 | 0.1087 |
80
+ | 6.8482 | 12.0 | 42 | 6.3655 | 0.1087 |
81
+ | 6.8482 | 12.86 | 45 | 6.2153 | 0.1087 |
82
+ | 6.8482 | 14.0 | 49 | 6.0311 | 0.1087 |
83
+ | 6.223 | 14.86 | 52 | 5.9038 | 0.1087 |
84
+ | 6.223 | 16.0 | 56 | 5.7400 | 0.1087 |
85
+ | 6.223 | 16.86 | 59 | 5.6276 | 0.1087 |
86
+ | 5.8233 | 18.0 | 63 | 5.4884 | 0.1087 |
87
+ | 5.8233 | 18.86 | 66 | 5.3910 | 0.1087 |
88
+ | 5.5432 | 20.0 | 70 | 5.2690 | 0.1087 |
89
+ | 5.5432 | 20.86 | 73 | 5.1828 | 0.1087 |
90
+ | 5.5432 | 22.0 | 77 | 5.0761 | 0.1087 |
91
+ | 5.2554 | 22.86 | 80 | 5.0026 | 0.1087 |
92
+ | 5.2554 | 24.0 | 84 | 4.9136 | 0.1087 |
93
+ | 5.2554 | 24.86 | 87 | 4.8525 | 0.1087 |
94
+ | 5.0175 | 26.0 | 91 | 4.7792 | 0.1087 |
95
+ | 5.0175 | 26.86 | 94 | 4.7304 | 0.1087 |
96
+ | 5.0175 | 28.0 | 98 | 4.6736 | 0.1087 |
97
+ | 4.7765 | 28.86 | 101 | 4.6371 | 0.1087 |
98
+ | 4.7765 | 30.0 | 105 | 4.5968 | 0.1087 |
99
+ | 4.7765 | 30.86 | 108 | 4.5726 | 0.1087 |
100
+ | 4.7339 | 32.0 | 112 | 4.5483 | 0.1087 |
101
+ | 4.7339 | 32.86 | 115 | 4.5361 | 0.1087 |
102
+ | 4.7339 | 34.0 | 119 | 4.5280 | 0.1087 |
103
+ | 4.6747 | 34.29 | 120 | 4.5274 | 0.1087 |
104
+
105
+
106
+ ### Framework versions
107
+
108
+ - Transformers 4.36.2
109
+ - Pytorch 2.1.2+cu118
110
+ - Datasets 2.16.1
111
+ - Tokenizers 0.15.0
all_results.json ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 34.29,
3
+ "eval_accuracy": 0.10869565217391304,
4
+ "eval_loss": 7.923763275146484,
5
+ "eval_runtime": 0.9442,
6
+ "eval_samples_per_second": 48.718,
7
+ "eval_steps_per_second": 3.177,
8
+ "train_loss": 5.965131664276123,
9
+ "train_runtime": 192.4021,
10
+ "train_samples_per_second": 44.282,
11
+ "train_steps_per_second": 0.624
12
+ }
config.json ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "microsoft/swinv2-tiny-patch4-window8-256",
3
+ "architectures": [
4
+ "Swinv2ForImageClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.0,
7
+ "depths": [
8
+ 2,
9
+ 2,
10
+ 6,
11
+ 2
12
+ ],
13
+ "drop_path_rate": 0.1,
14
+ "embed_dim": 96,
15
+ "encoder_stride": 32,
16
+ "hidden_act": "gelu",
17
+ "hidden_dropout_prob": 0.0,
18
+ "hidden_size": 768,
19
+ "id2label": {
20
+ "0": "avanzada",
21
+ "1": "leve",
22
+ "2": "moderada",
23
+ "3": "no dmae"
24
+ },
25
+ "image_size": 256,
26
+ "initializer_range": 0.02,
27
+ "label2id": {
28
+ "avanzada": 0,
29
+ "leve": 1,
30
+ "moderada": 2,
31
+ "no dmae": 3
32
+ },
33
+ "layer_norm_eps": 1e-05,
34
+ "mlp_ratio": 4.0,
35
+ "model_type": "swinv2",
36
+ "num_channels": 3,
37
+ "num_heads": [
38
+ 3,
39
+ 6,
40
+ 12,
41
+ 24
42
+ ],
43
+ "num_layers": 4,
44
+ "patch_size": 4,
45
+ "path_norm": true,
46
+ "pretrained_window_sizes": [
47
+ 0,
48
+ 0,
49
+ 0,
50
+ 0
51
+ ],
52
+ "problem_type": "single_label_classification",
53
+ "qkv_bias": true,
54
+ "torch_dtype": "float32",
55
+ "transformers_version": "4.36.2",
56
+ "use_absolute_embeddings": false,
57
+ "window_size": 8
58
+ }
eval_results.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 34.29,
3
+ "eval_accuracy": 0.10869565217391304,
4
+ "eval_loss": 7.923763275146484,
5
+ "eval_runtime": 0.9442,
6
+ "eval_samples_per_second": 48.718,
7
+ "eval_steps_per_second": 3.177
8
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:210c09ff57d6f6d1e95232c07f47902171a10512323b61a5d5359942f62d8d52
3
+ size 110356296
preprocessor_config.json ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_normalize": true,
3
+ "do_rescale": true,
4
+ "do_resize": true,
5
+ "image_mean": [
6
+ 0.485,
7
+ 0.456,
8
+ 0.406
9
+ ],
10
+ "image_processor_type": "ViTImageProcessor",
11
+ "image_std": [
12
+ 0.229,
13
+ 0.224,
14
+ 0.225
15
+ ],
16
+ "resample": 3,
17
+ "rescale_factor": 0.00392156862745098,
18
+ "size": {
19
+ "height": 256,
20
+ "width": 256
21
+ }
22
+ }
runs/Dec02_10-49-51_DESKTOP-SKBE9FB/events.out.tfevents.1733158192.DESKTOP-SKBE9FB.7232.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:522791161fe95ff7c01b51fff2b4567538182ee3af2034acacaf57d454daec7d
3
+ size 18095
runs/Dec02_10-49-51_DESKTOP-SKBE9FB/events.out.tfevents.1733158388.DESKTOP-SKBE9FB.7232.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fe0b5b6a5af9f44866073437372ba33cfc5dc2c732208ba23fb4b011de8e94c3
3
+ size 405
runs/Dec02_10-54-50_DESKTOP-SKBE9FB/events.out.tfevents.1733158491.DESKTOP-SKBE9FB.11632.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bfb3d68ce169f1fba1006916ad5570083aec7c9bb40c690b8ddb912b80473e14
3
+ size 10963
runs/Dec02_10-56-42_DESKTOP-SKBE9FB/events.out.tfevents.1733158603.DESKTOP-SKBE9FB.18460.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b3f0fe13e6b239421e54fa159737315acb408f29f317f0e3b5e6e3d30f7df7c8
3
+ size 18095
runs/Dec02_10-56-42_DESKTOP-SKBE9FB/events.out.tfevents.1733158797.DESKTOP-SKBE9FB.18460.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7c33d98c642b9e649a6211c810e00b2fb15b44c252c003c178560d53c2d061ac
3
+ size 405
train_results.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 34.29,
3
+ "train_loss": 5.965131664276123,
4
+ "train_runtime": 192.4021,
5
+ "train_samples_per_second": 44.282,
6
+ "train_steps_per_second": 0.624
7
+ }
trainer_state.json ADDED
@@ -0,0 +1,417 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": 0.10869565217391304,
3
+ "best_model_checkpoint": "swinv2-tiny-patch4-window8-256-DMAE-15e\\checkpoint-3",
4
+ "epoch": 34.285714285714285,
5
+ "eval_steps": 500,
6
+ "global_step": 120,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.86,
13
+ "eval_accuracy": 0.10869565217391304,
14
+ "eval_loss": 7.923763275146484,
15
+ "eval_runtime": 0.7392,
16
+ "eval_samples_per_second": 62.232,
17
+ "eval_steps_per_second": 4.059,
18
+ "step": 3
19
+ },
20
+ {
21
+ "epoch": 2.0,
22
+ "eval_accuracy": 0.10869565217391304,
23
+ "eval_loss": 7.874618053436279,
24
+ "eval_runtime": 0.7127,
25
+ "eval_samples_per_second": 64.546,
26
+ "eval_steps_per_second": 4.21,
27
+ "step": 7
28
+ },
29
+ {
30
+ "epoch": 2.86,
31
+ "learning_rate": 1.375e-05,
32
+ "loss": 7.9273,
33
+ "step": 10
34
+ },
35
+ {
36
+ "epoch": 2.86,
37
+ "eval_accuracy": 0.10869565217391304,
38
+ "eval_loss": 7.818479537963867,
39
+ "eval_runtime": 0.7227,
40
+ "eval_samples_per_second": 63.653,
41
+ "eval_steps_per_second": 4.151,
42
+ "step": 10
43
+ },
44
+ {
45
+ "epoch": 4.0,
46
+ "eval_accuracy": 0.10869565217391304,
47
+ "eval_loss": 7.699648857116699,
48
+ "eval_runtime": 0.7342,
49
+ "eval_samples_per_second": 62.655,
50
+ "eval_steps_per_second": 4.086,
51
+ "step": 14
52
+ },
53
+ {
54
+ "epoch": 4.86,
55
+ "eval_accuracy": 0.10869565217391304,
56
+ "eval_loss": 7.587607383728027,
57
+ "eval_runtime": 0.7147,
58
+ "eval_samples_per_second": 64.365,
59
+ "eval_steps_per_second": 4.198,
60
+ "step": 17
61
+ },
62
+ {
63
+ "epoch": 5.71,
64
+ "learning_rate": 1.25e-05,
65
+ "loss": 7.6529,
66
+ "step": 20
67
+ },
68
+ {
69
+ "epoch": 6.0,
70
+ "eval_accuracy": 0.10869565217391304,
71
+ "eval_loss": 7.418882846832275,
72
+ "eval_runtime": 0.7092,
73
+ "eval_samples_per_second": 64.866,
74
+ "eval_steps_per_second": 4.23,
75
+ "step": 21
76
+ },
77
+ {
78
+ "epoch": 6.86,
79
+ "eval_accuracy": 0.10869565217391304,
80
+ "eval_loss": 7.2678446769714355,
81
+ "eval_runtime": 0.7064,
82
+ "eval_samples_per_second": 65.116,
83
+ "eval_steps_per_second": 4.247,
84
+ "step": 24
85
+ },
86
+ {
87
+ "epoch": 8.0,
88
+ "eval_accuracy": 0.10869565217391304,
89
+ "eval_loss": 7.0488600730896,
90
+ "eval_runtime": 0.8282,
91
+ "eval_samples_per_second": 55.543,
92
+ "eval_steps_per_second": 3.622,
93
+ "step": 28
94
+ },
95
+ {
96
+ "epoch": 8.57,
97
+ "learning_rate": 1.125e-05,
98
+ "loss": 7.1056,
99
+ "step": 30
100
+ },
101
+ {
102
+ "epoch": 8.86,
103
+ "eval_accuracy": 0.10869565217391304,
104
+ "eval_loss": 6.886016368865967,
105
+ "eval_runtime": 0.7082,
106
+ "eval_samples_per_second": 64.957,
107
+ "eval_steps_per_second": 4.236,
108
+ "step": 31
109
+ },
110
+ {
111
+ "epoch": 10.0,
112
+ "eval_accuracy": 0.10869565217391304,
113
+ "eval_loss": 6.686739921569824,
114
+ "eval_runtime": 0.7412,
115
+ "eval_samples_per_second": 62.064,
116
+ "eval_steps_per_second": 4.048,
117
+ "step": 35
118
+ },
119
+ {
120
+ "epoch": 10.86,
121
+ "eval_accuracy": 0.10869565217391304,
122
+ "eval_loss": 6.55491304397583,
123
+ "eval_runtime": 0.7112,
124
+ "eval_samples_per_second": 64.683,
125
+ "eval_steps_per_second": 4.218,
126
+ "step": 38
127
+ },
128
+ {
129
+ "epoch": 11.43,
130
+ "learning_rate": 9.999999999999999e-06,
131
+ "loss": 6.8482,
132
+ "step": 40
133
+ },
134
+ {
135
+ "epoch": 12.0,
136
+ "eval_accuracy": 0.10869565217391304,
137
+ "eval_loss": 6.365451335906982,
138
+ "eval_runtime": 0.7332,
139
+ "eval_samples_per_second": 62.742,
140
+ "eval_steps_per_second": 4.092,
141
+ "step": 42
142
+ },
143
+ {
144
+ "epoch": 12.86,
145
+ "eval_accuracy": 0.10869565217391304,
146
+ "eval_loss": 6.215301990509033,
147
+ "eval_runtime": 0.7252,
148
+ "eval_samples_per_second": 63.433,
149
+ "eval_steps_per_second": 4.137,
150
+ "step": 45
151
+ },
152
+ {
153
+ "epoch": 14.0,
154
+ "eval_accuracy": 0.10869565217391304,
155
+ "eval_loss": 6.0311198234558105,
156
+ "eval_runtime": 0.7212,
157
+ "eval_samples_per_second": 63.785,
158
+ "eval_steps_per_second": 4.16,
159
+ "step": 49
160
+ },
161
+ {
162
+ "epoch": 14.29,
163
+ "learning_rate": 8.750000000000001e-06,
164
+ "loss": 6.223,
165
+ "step": 50
166
+ },
167
+ {
168
+ "epoch": 14.86,
169
+ "eval_accuracy": 0.10869565217391304,
170
+ "eval_loss": 5.903781414031982,
171
+ "eval_runtime": 0.7222,
172
+ "eval_samples_per_second": 63.696,
173
+ "eval_steps_per_second": 4.154,
174
+ "step": 52
175
+ },
176
+ {
177
+ "epoch": 16.0,
178
+ "eval_accuracy": 0.10869565217391304,
179
+ "eval_loss": 5.7399983406066895,
180
+ "eval_runtime": 0.7477,
181
+ "eval_samples_per_second": 61.524,
182
+ "eval_steps_per_second": 4.012,
183
+ "step": 56
184
+ },
185
+ {
186
+ "epoch": 16.86,
187
+ "eval_accuracy": 0.10869565217391304,
188
+ "eval_loss": 5.627634525299072,
189
+ "eval_runtime": 0.7197,
190
+ "eval_samples_per_second": 63.918,
191
+ "eval_steps_per_second": 4.169,
192
+ "step": 59
193
+ },
194
+ {
195
+ "epoch": 17.14,
196
+ "learning_rate": 7.5e-06,
197
+ "loss": 5.8233,
198
+ "step": 60
199
+ },
200
+ {
201
+ "epoch": 18.0,
202
+ "eval_accuracy": 0.10869565217391304,
203
+ "eval_loss": 5.488417148590088,
204
+ "eval_runtime": 0.7112,
205
+ "eval_samples_per_second": 64.683,
206
+ "eval_steps_per_second": 4.218,
207
+ "step": 63
208
+ },
209
+ {
210
+ "epoch": 18.86,
211
+ "eval_accuracy": 0.10869565217391304,
212
+ "eval_loss": 5.391030311584473,
213
+ "eval_runtime": 0.7452,
214
+ "eval_samples_per_second": 61.731,
215
+ "eval_steps_per_second": 4.026,
216
+ "step": 66
217
+ },
218
+ {
219
+ "epoch": 20.0,
220
+ "learning_rate": 6.25e-06,
221
+ "loss": 5.5432,
222
+ "step": 70
223
+ },
224
+ {
225
+ "epoch": 20.0,
226
+ "eval_accuracy": 0.10869565217391304,
227
+ "eval_loss": 5.2689528465271,
228
+ "eval_runtime": 0.7257,
229
+ "eval_samples_per_second": 63.39,
230
+ "eval_steps_per_second": 4.134,
231
+ "step": 70
232
+ },
233
+ {
234
+ "epoch": 20.86,
235
+ "eval_accuracy": 0.10869565217391304,
236
+ "eval_loss": 5.182785511016846,
237
+ "eval_runtime": 0.7282,
238
+ "eval_samples_per_second": 63.173,
239
+ "eval_steps_per_second": 4.12,
240
+ "step": 73
241
+ },
242
+ {
243
+ "epoch": 22.0,
244
+ "eval_accuracy": 0.10869565217391304,
245
+ "eval_loss": 5.076084136962891,
246
+ "eval_runtime": 0.7192,
247
+ "eval_samples_per_second": 63.963,
248
+ "eval_steps_per_second": 4.172,
249
+ "step": 77
250
+ },
251
+ {
252
+ "epoch": 22.86,
253
+ "learning_rate": 4.9999999999999996e-06,
254
+ "loss": 5.2554,
255
+ "step": 80
256
+ },
257
+ {
258
+ "epoch": 22.86,
259
+ "eval_accuracy": 0.10869565217391304,
260
+ "eval_loss": 5.0026044845581055,
261
+ "eval_runtime": 0.8007,
262
+ "eval_samples_per_second": 57.45,
263
+ "eval_steps_per_second": 3.747,
264
+ "step": 80
265
+ },
266
+ {
267
+ "epoch": 24.0,
268
+ "eval_accuracy": 0.10869565217391304,
269
+ "eval_loss": 4.913613796234131,
270
+ "eval_runtime": 0.7312,
271
+ "eval_samples_per_second": 62.913,
272
+ "eval_steps_per_second": 4.103,
273
+ "step": 84
274
+ },
275
+ {
276
+ "epoch": 24.86,
277
+ "eval_accuracy": 0.10869565217391304,
278
+ "eval_loss": 4.852543830871582,
279
+ "eval_runtime": 0.7112,
280
+ "eval_samples_per_second": 64.683,
281
+ "eval_steps_per_second": 4.218,
282
+ "step": 87
283
+ },
284
+ {
285
+ "epoch": 25.71,
286
+ "learning_rate": 3.75e-06,
287
+ "loss": 5.0175,
288
+ "step": 90
289
+ },
290
+ {
291
+ "epoch": 26.0,
292
+ "eval_accuracy": 0.10869565217391304,
293
+ "eval_loss": 4.7791571617126465,
294
+ "eval_runtime": 0.7142,
295
+ "eval_samples_per_second": 64.411,
296
+ "eval_steps_per_second": 4.201,
297
+ "step": 91
298
+ },
299
+ {
300
+ "epoch": 26.86,
301
+ "eval_accuracy": 0.10869565217391304,
302
+ "eval_loss": 4.730408191680908,
303
+ "eval_runtime": 0.7222,
304
+ "eval_samples_per_second": 63.698,
305
+ "eval_steps_per_second": 4.154,
306
+ "step": 94
307
+ },
308
+ {
309
+ "epoch": 28.0,
310
+ "eval_accuracy": 0.10869565217391304,
311
+ "eval_loss": 4.673551559448242,
312
+ "eval_runtime": 0.7282,
313
+ "eval_samples_per_second": 63.173,
314
+ "eval_steps_per_second": 4.12,
315
+ "step": 98
316
+ },
317
+ {
318
+ "epoch": 28.57,
319
+ "learning_rate": 2.4999999999999998e-06,
320
+ "loss": 4.7765,
321
+ "step": 100
322
+ },
323
+ {
324
+ "epoch": 28.86,
325
+ "eval_accuracy": 0.10869565217391304,
326
+ "eval_loss": 4.637141227722168,
327
+ "eval_runtime": 0.7492,
328
+ "eval_samples_per_second": 61.401,
329
+ "eval_steps_per_second": 4.004,
330
+ "step": 101
331
+ },
332
+ {
333
+ "epoch": 30.0,
334
+ "eval_accuracy": 0.10869565217391304,
335
+ "eval_loss": 4.596778392791748,
336
+ "eval_runtime": 0.7372,
337
+ "eval_samples_per_second": 62.401,
338
+ "eval_steps_per_second": 4.07,
339
+ "step": 105
340
+ },
341
+ {
342
+ "epoch": 30.86,
343
+ "eval_accuracy": 0.10869565217391304,
344
+ "eval_loss": 4.572619915008545,
345
+ "eval_runtime": 0.7372,
346
+ "eval_samples_per_second": 62.401,
347
+ "eval_steps_per_second": 4.07,
348
+ "step": 108
349
+ },
350
+ {
351
+ "epoch": 31.43,
352
+ "learning_rate": 1.2499999999999999e-06,
353
+ "loss": 4.7339,
354
+ "step": 110
355
+ },
356
+ {
357
+ "epoch": 32.0,
358
+ "eval_accuracy": 0.10869565217391304,
359
+ "eval_loss": 4.548309326171875,
360
+ "eval_runtime": 0.7537,
361
+ "eval_samples_per_second": 61.034,
362
+ "eval_steps_per_second": 3.98,
363
+ "step": 112
364
+ },
365
+ {
366
+ "epoch": 32.86,
367
+ "eval_accuracy": 0.10869565217391304,
368
+ "eval_loss": 4.536120414733887,
369
+ "eval_runtime": 0.7227,
370
+ "eval_samples_per_second": 63.652,
371
+ "eval_steps_per_second": 4.151,
372
+ "step": 115
373
+ },
374
+ {
375
+ "epoch": 34.0,
376
+ "eval_accuracy": 0.10869565217391304,
377
+ "eval_loss": 4.528027057647705,
378
+ "eval_runtime": 0.7282,
379
+ "eval_samples_per_second": 63.172,
380
+ "eval_steps_per_second": 4.12,
381
+ "step": 119
382
+ },
383
+ {
384
+ "epoch": 34.29,
385
+ "learning_rate": 0.0,
386
+ "loss": 4.6747,
387
+ "step": 120
388
+ },
389
+ {
390
+ "epoch": 34.29,
391
+ "eval_accuracy": 0.10869565217391304,
392
+ "eval_loss": 4.527446746826172,
393
+ "eval_runtime": 0.7392,
394
+ "eval_samples_per_second": 62.232,
395
+ "eval_steps_per_second": 4.059,
396
+ "step": 120
397
+ },
398
+ {
399
+ "epoch": 34.29,
400
+ "step": 120,
401
+ "total_flos": 2.3770905934823424e+17,
402
+ "train_loss": 5.965131664276123,
403
+ "train_runtime": 192.4021,
404
+ "train_samples_per_second": 44.282,
405
+ "train_steps_per_second": 0.624
406
+ }
407
+ ],
408
+ "logging_steps": 10,
409
+ "max_steps": 120,
410
+ "num_input_tokens_seen": 0,
411
+ "num_train_epochs": 40,
412
+ "save_steps": 500,
413
+ "total_flos": 2.3770905934823424e+17,
414
+ "train_batch_size": 16,
415
+ "trial_name": null,
416
+ "trial_params": null
417
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:42e6dd5fe2185c3541871e6a294f52ee5dc548ee7730132acdfe07944e894b8f
3
+ size 4792