32k ctx doesn't work on this model for GGUF
#3
by danieloneill - opened
I don't suppose this requires elaboration, but in my tests, anything beyond 4k results in garbage output.
Has anybody else had conflicting success with 8k, 16k, or 32k?
I don't suppose this requires elaboration, but in my tests, anything beyond 4k results in garbage output.
Has anybody else had conflicting success with 8k, 16k, or 32k?