Abiray/gemma-4-E4B-it-The-DECKARD-HERETIC-UNCENSORED-Thinking-GGUF

Gemma 4 E4B Instruct - The DECKARD HERETIC UNCENSORED Thinking (GGUF)

This repository contains GGUF format quantizations for DavidAU's Gemma-4-E4B-it-The-DECKARD-HERETIC-UNCENSORED-Thinking.

These files allow for highly efficient local inference on CPUs and Apple Silicon, as well as VRAM-constrained GPUs, utilizing llama.cpp and compatible frontends.

🧠 About the Base Model

The base model is a specialized, heavily fine-tuned version of Google's Gemma 4 E4B (Effective 4B parameters).

• HERETIC / DECKARD Fine-Tune: This model has undergone the "HERETIC" abliteration and fine-tuning process by DavidAU. It is completely uncensored, unrestricted, and heavily calibrated for complex creative writing, roleplay, and deep fiction generation.
• Reasoning & Thinking: It includes augmented reasoning/thinking capabilities, allowing the model to internally chain logic before outputting text, making its responses deeper and more contextually accurate.
• Context Window: The Gemma 4 architecture supports extended context windows (up to 256k in base), making it highly capable for long-form conversations.

📦 Available Quantization Formats

This repository provides multiple levels of quantization to help you balance VRAM/RAM usage, generation speed, and model fidelity.

File Name	Bit Resolution	Recommended Use
Q3_K_M	3-bit	Ultra-low RAM usage. Noticeable perplexity degradation but runs on very constrained hardware.
Q4_K_M	4-bit	Recommended. The sweet spot for local LLMs. Great balance of speed, low memory footprint, and quality.
Q5_K_M	5-bit	Higher precision. Use this if you have the memory to spare and want slightly better reasoning.
Q6_K	6-bit	Very high fidelity. Close to unquantized performance with a larger memory footprint.
Q8_0	8-bit	Extremely close to FP16 baseline. Best for users who want maximum precision and have ample RAM/VRAM.