Instructions to use LiconStudio/Gemma-4-31B-it-abliterated-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LiconStudio/Gemma-4-31B-it-abliterated-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="LiconStudio/Gemma-4-31B-it-abliterated-GGUF",
	filename="gemma-4-31B-it-abliterated-F16.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use LiconStudio/Gemma-4-31B-it-abliterated-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf LiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf LiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf LiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf LiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf LiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf LiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf LiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf LiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M

Use Docker

docker model run hf.co/LiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use LiconStudio/Gemma-4-31B-it-abliterated-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LiconStudio/Gemma-4-31B-it-abliterated-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LiconStudio/Gemma-4-31B-it-abliterated-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/LiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M

Ollama
How to use LiconStudio/Gemma-4-31B-it-abliterated-GGUF with Ollama:
```
ollama run hf.co/LiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M
```

Unsloth Studio

How to use LiconStudio/Gemma-4-31B-it-abliterated-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for LiconStudio/Gemma-4-31B-it-abliterated-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for LiconStudio/Gemma-4-31B-it-abliterated-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for LiconStudio/Gemma-4-31B-it-abliterated-GGUF to start chatting

How to use LiconStudio/Gemma-4-31B-it-abliterated-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf LiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "LiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use LiconStudio/Gemma-4-31B-it-abliterated-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf LiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default LiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use LiconStudio/Gemma-4-31B-it-abliterated-GGUF with Docker Model Runner:
```
docker model run hf.co/LiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M
```

Lemonade

How to use LiconStudio/Gemma-4-31B-it-abliterated-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull LiconStudio/Gemma-4-31B-it-abliterated-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Gemma-4-31B-it-abliterated-GGUF-Q4_K_M

List all available models

lemonade list

Overview

We pursue lower rejection rates while exploring lower KL divergence to maximally preserve model intelligence.

We provide two versions to balance censorship removal and capability preservation:

Abliterated Version (Refusal: 4/100, KL: 0.4096) – No refusal scenarios were triggered in manual testing.
Balanced Version (Refusal: 8/100, KL: 0.2446) – May show refusal tendencies on extremely aggressive prompts, but can be corrected via system/user prompts. Theoretically preserves more of the original model's intelligence due to lower KL divergence.

Logic tests show no visibly degraded intelligence compared to the official version. For more stable outputs, system prompts or user prompts can be used for constraints and guidance.

ABLiteration Approach

This model uses the Heretic ABLiteration method for neural direction ablation:

Identify Refusal Direction - Train a LoRA on harmful behavior datasets to identify neural directions controlling "refusal behavior"
Direction Extraction - Extract the "refusal vector" from the trained LoRA
Ablative Removal - Subtract this direction from the original model weights, removing the censorship mechanism

This method only modifies model weights without changing the architecture or adding inference overhead.

For detailed technical principles, refer to: Heretic Abliteration

Data Sources

Purpose	Dataset
Refusal Direction Identification	mlabonne/harmful_behaviors (520 prompts)
KL Evaluation	General prompts (100 prompts)
Refusal Rate Testing	mlabonne/harmful_behaviors (520 prompts)

✅ Recommended Uses

Research and analysis of sensitive topics
Safety testing and red-teaming exercises
Academic research on model alignment
Multi-modal tasks (image + text) with Gemma 4 vision capabilities

❌ Not Recommended For

Production environments requiring content moderation
Applications targeting minors
Scenarios with potential legal risks

Limitations

Minor Capability Loss - KL divergence indicates moderate modification, which may slightly affect performance on complex tasks
User Discretion Required - Users must independently judge the appropriateness of generated outputs
Vision Model Unmodified - The vision encoder remains unchanged from the original Gemma 4

Disclaimer

⚠️ Important: This model is intended for research and educational purposes only.

This model has had its censorship mechanisms removed and may generate harmful, dangerous, or inappropriate content
Users assume all risks associated with usage
Do not use this model for illegal activities, harming others, or any inappropriate purposes
The model authors are not liable for any indirect, incidental, or consequential damages

Acknowledgments

Original Model: google/gemma-4-31B-it
Heretic Method: alignlab/heretic

Downloads last month: 2,567

GGUF

Model size

31B params

Architecture

gemma4

Hardware compatibility

2-bit

3-bit

4-bit

5-bit

8-bit

16-bit

Model tree for LiconStudio/Gemma-4-31B-it-abliterated-GGUF

Base model

google/gemma-4-31B

Finetuned

google/gemma-4-31B-it

Quantized

(237)

this model