Qwopus3.6-27B-v2-oQ4-fp16-mtp

This repository contains an oMLX/oQ 4-bit quantized MLX version of Jackrong/Qwopus3.6-27B-v2.

This variant uses float16 / fp16 for non-quantized weights and preserves MTP weights.

Model lineage

• Original model: Jackrong/Qwopus3.6-27B-v2
• Quantized model: AbarthJoe/Qwopus3.6-27B-v2-oQ4-fp16-mtp
• Quantization format: MLX / oMLX oQ
• Relationship: Quantized derivative of the original model

Quantization details

• Quantization tool: oMLX / oQ
• Quantization level: oQ4
• Preserve MTP weights: Yes
• Non-quant weight dtype: float16 / fp16
• Output format: MLX
• Target platform: Apple Silicon

About this fp16 variant

This version keeps non-quantized weights in float16 instead of the default bfloat16 path.

The goal is to test whether fp16 improves prefill / prompt processing speed on Apple Silicon while keeping oQ4 generation speed high.

Compared with the default oQ4 MTP version:

• May provide faster prefill in some environments
• May be less numerically stable than bfloat16
• Should be tested carefully with long-context prompts
• May behave differently depending on Apple Silicon generation and MLX/oMLX version

If this fp16 variant shows unstable output, repeated text, degraded reasoning, or unusual long-context behavior, use the default oQ4-mtp version instead.

Expected use case

This is an experimental speed-focused fp16 variant.

It is mainly useful for users who want to benchmark:

• oQ4 generation speed
• fp16 non-quant weight behavior
• MTP-preserved inference
• Apple Silicon local model performance

Benchmark

Tested on MacBook Pro M3 Max 40-core GPU.

Model	Context	Prompt processing	Token generation
oQ4 fp16 + MTP	1k	218.2 tok/s	19.7 tok/s
oQ4 fp16 + MTP	4k	210.9 tok/s	20.9 tok/s

Benchmark results may vary depending on hardware, software version, prompt type, context length, and runtime settings.

Related models

Other oMLX/oQ quantized versions are available in this collection:

Qwopus oMLX oQ Quantized Models for Apple Silicon

Credits

Original model by Jackrong.

This quantized MLX/oQ fp16 variant was created and uploaded by AbarthJoe.

License

The original model is licensed under Apache-2.0.
This quantized version follows the same Apache-2.0 license where applicable.

Disclaimer

This is a community quantized model for research, experimentation, and local inference testing.

It has not been fully safety-evaluated or benchmarked across all tasks.
Please test carefully before using it for production, sensitive, or high-stakes use cases.