Update README.md
Browse files
README.md
CHANGED
|
@@ -118,7 +118,6 @@ In the paper, the method fine-tunes the **Query / Key / Value** projections of s
|
|
| 118 |
- **Adapted modules:** Query / Key / Value projections
|
| 119 |
- **Orthogonality coefficient:** `位 = 1e-4`
|
| 120 |
- **Dropout:** `0.05`
|
| 121 |
-
- **Learning rate:** `3e-4`
|
| 122 |
- **Batch size:** `4` per A100 GPU (40GB)
|
| 123 |
|
| 124 |
Representative D<sup>2</sup>MoRA settings reported in the paper include:
|
|
|
|
| 118 |
- **Adapted modules:** Query / Key / Value projections
|
| 119 |
- **Orthogonality coefficient:** `位 = 1e-4`
|
| 120 |
- **Dropout:** `0.05`
|
|
|
|
| 121 |
- **Batch size:** `4` per A100 GPU (40GB)
|
| 122 |
|
| 123 |
Representative D<sup>2</sup>MoRA settings reported in the paper include:
|