---
base_model: google/gemma-4-31B-it
library_name: mlx
pipeline_tag: text-generation
license: apache-2.0
tags:
- mlx
- jang
- jang-quantized
- JANG_2M
- mixed-precision
- apple-silicon
---

## ⚠️ Low-bit quality warning

This is an aggressive quantization (2-bit average). At this compression level, output quality degrades noticeably — responses may start coherent but degenerate into repetition or garbage tokens toward the end of longer generations. This is expected behavior for 2-bit quantization on this architecture.

**Recommended for:** experimentation, quick testing, extreme memory constraints.
**Not recommended for:** production use, long-form generation, coding tasks.

For reliable output quality, use JANG_4M or higher profiles from this collection.


# gemma-4-31B-it-JANG_2M

JANG adaptive mixed-precision MLX quantization produced via [vmlx / jang-tools](https://github.com/jjang-ai/jangq).

- **Quantization:** 3.75b avg, profile JANG_2M, method mse-all, calibration activations
- **Profile:** JANG_2M
- **Format:** JANG v2 MLX safetensors
- **Compatible with:** vmlx, MLX Studio, oMLX (with JANG patch)

## Usage

### vmlx (recommended)

```bash
pip install 'vmlx[jang]'
vmlx serve bearzi/gemma-4-31B-it-JANG_2M
```

### Python

```python
from jang_tools.loader import load_jang_model
from mlx_lm import generate

model, tokenizer = load_jang_model("bearzi/gemma-4-31B-it-JANG_2M")
messages = [{"role": "user", "content": "Hello"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512, verbose=True))
```

## About JANG

JANG (Jang Adaptive N-bit Grading) assigns different bit widths to different layer types — attention layers get more bits, MLP/expert layers compress harder. This preserves model coherence at aggressive compression levels where uniform quantization breaks down.

See [JANG documentation](https://github.com/jjang-ai/jangq) and scores at [jangq.ai](https://jangq.ai).

Comparative benchmarks and feedback welcome — please open a discussion.