Qwen3.5-397B-A17B-RotorQuant-MLX-6bit

6-bit MLX weight-quantized build of Qwen/Qwen3.5-397B-A17B (397B total / 17B active Sparse MoE, multimodal) prepared with RotorQuant learned orthogonal rotors. Optimized for Apple Silicon via MLX.

6-bit RotorQuant is the top fidelity/footprint trade-off in the suite: near-8-bit quality at 25% smaller disk and memory footprint.

Quickstart

from mlx_lm import load, generate

model, tokenizer = load("majentik/Qwen3.5-397B-A17B-RotorQuant-MLX-6bit")

prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Give me 3 tips for debugging MoE routing."}],
    add_generation_prompt=True,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=256, verbose=True))

Model Specs

Property	Value
Base model	Qwen/Qwen3.5-397B-A17B
Architecture	Sparse Mixture-of-Experts (MoE)
Total parameters	397B
Active per token	17B
Modalities	Image + Text → Text (image-text-to-text)
Context window	256K tokens
Weight quantization	6-bit MLX (RotorQuant learned rotors)
Approx. disk footprint	~300 GB
License	Apache 2.0

RotorQuant vs TurboQuant

Aspect	RotorQuant (this repo)	TurboQuant
Rotation	Learned orthogonal rotors (data-calibrated)	Randomized Hadamard (static)
Calibration	~512 sample calibration pass	Zero-shot
Accuracy @ 6-bit	~99.8% of FP16 baseline	~99.6% of FP16 baseline
Best for	Highest fidelity at same bit-width	Fastest turnaround

Memory Estimates (6-bit MLX)

Context	Active memory (approx.)
8K	~308 GB
32K	~318 GB
128K	~348 GB
256K	~378 GB

Hardware Requirements

• Minimum: Apple Silicon with 384 GB+ unified memory (tight)
• Recommended: 512 GB unified memory Mac Studio
• Does not fit on 96 GB / 128 GB / 192 GB / 256 GB Macs

See Also

• RotorQuant MLX: 8-bit · 5-bit · 4-bit · 2-bit
• TurboQuant MLX 6-bit: majentik/Qwen3.5-397B-A17B-TurboQuant-MLX-6bit
• Base model: Qwen/Qwen3.5-397B-A17B