Fix tokenizer (hopefully)
#2
by inflatebot - opened
[SYSTEM_PROMPT] and [/SYSTEM_PROMPT] moved from token IDs 131072/131073 to 17/18, as was originally intended. Extraneous tokens removed from config. Embedding weights and lm_head altered with the Python Transformers toolkit to accommodate these changes.
yeah fuck it yolo
inflatebot changed pull request status to merged