Update alignment heads in gen config
#3
by sanchit-gandhi - opened
Updates the alignment heads in the generation config as per the OpenAI recommendations. A quick inspection of the attention plots shows that these align relatively well with the token ids:
sanchit-gandhi changed pull request title from Upload config to Update alignment heads in gen config
sanchit-gandhi changed pull request status to merged