JusWis/kanana-legal-f16

Kanana Legal FP16 GGUF

This is a FP16 GGUF quantized version of the Korean Legal fine-tuned model based on Kakao's Kanana Nano 2.1B Instruct.

🎯 Model Overview

This ultra-lightweight 2.1B parameter model has been fine-tuned on Korean legal terminology to run efficiently on CPU-only environments without internet connectivity, making it ideal for offline legal document processing in restricted environments.

Key Features

• ✅ CPU-Optimized: Designed for CPU inference without GPU requirements
• ✅ Offline Capable: No internet connection needed for inference
• ✅ Legal Domain: Fine-tuned on 17,484 Korean legal term definitions
• ✅ GGUF Format: Compatible with llama.cpp and other GGUF-compatible tools
• ✅ FP16 Precision: Full 16-bit floating point precision for maximum accuracy

📊 Training Data

The model was fine-tuned on the Korean Legal Terminology dataset:

• Samples: 17,484 legal term definitions
• Format: Instruction-following (input/output pairs)
• Domain: Korean legal terminology, concepts, and definitions
• Language: Korean

🔧 Fine-tuning Details

Base Model

• Model: kakaocorp/kanana-nano-2.1b-instruct
• Size: 2.1B parameters
• Architecture: Transformer-based causal language model

Training Configuration

QLoRA Settings:

• Quantization Type: NF4 (Normal Float 4-bit)
• Compute dtype: bfloat16
• Double Quantization: Enabled
• LoRA Rank (r): 16
• LoRA Alpha: 32
• Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• LoRA Dropout: 0.05
• Trainable Parameters: ~23M (1.95% of total)

Training Hyperparameters:

• Epochs: 3
• Batch Size: 4 (per device)
• Gradient Accumulation Steps: 4
• Effective Batch Size: 16
• Optimizer: Paged AdamW 8-bit
• Learning Rate: 2e-4
• LR Scheduler: Cosine
• Warmup Ratio: 0.03
• Weight Decay: 0.01
• Max Gradient Norm: 0.3
• Mixed Precision: bfloat16
• Max Sequence Length: 2048

Optimization Techniques

• Gradient Checkpointing: Reduces memory during backpropagation
• Paged Optimizers: Efficient memory management for optimizer states
• Mixed Precision: bfloat16 for faster computation
• Gradient Accumulation: Simulates larger batch sizes

💻 Usage

Requirements

# Install llama.cpp or compatible GGUF runtime
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make

Inference Example

# Run with llama.cpp
./main -m kanana-legal-f16.gguf -p "### 질문:\n다음 법률 용어를 설명해줘: 흡수합병\n\n### 답변:" -n 512

# English: "Question: Explain the following legal term: Merger by Absorption"

Python Usage (with llama-cpp-python)

from llama_cpp import Llama

# Load model
llm = Llama(
    model_path="kanana-legal-f16.gguf",
    n_ctx=2048,
    n_threads=8,
)

# Generate response
prompt = """### 질문:
다음 법률 용어를 설명해줘: 흡수합병

### 답변:"""
# English: "Question: Explain the following legal term: Merger by Absorption / Answer:"

output = llm(
    prompt,
    max_tokens=512,
    temperature=0.7,
    top_p=0.9,
)

print(output['choices'][0]['text'])

📝 Prompt Format

The model expects the following instruction format:

### 질문:
다음 법률 용어(한자: 吸收合倂)를 설명해줘: 흡수합병

### 답변:
법률이 정하는 절차에 의하여 2 이상의 법인 전부 또는 그중 1개의 법인이외의 법인이 해산하여...

English Translation:

### Question:
Explain the following legal term (Hanja: 吸收合倂): Merger by Absorption

### Answer:
According to the procedures prescribed by law, when two or more corporations, or all corporations except one, are dissolved...

⚠️ Disclaimer

IMPORTANT: This is an ultra-lightweight 2.1B parameter model designed for CPU-only inference in offline environments. Due to its compact size and training constraints:

• ❌ Legal information may not be accurate or complete
• ❌ Should NOT be used as a substitute for professional legal advice
• ❌ No warranty or liability is provided for any use of this model
• ❌ Users assume all responsibility for verifying information
• ⚠️ For informational and educational purposes only

Always consult qualified legal professionals for actual legal matters.

🔍 Model Variants

This model is available in multiple quantization levels:

Model	Size	Precision	Use Case
kanana-legal-f16	~4.2GB	FP16	Maximum accuracy, higher memory
kanana-legal-q8_0	~2.2GB	Q8_0	Balanced accuracy/size
kanana-legal-q4_k_m	~1.4GB	Q4_K_M	Smallest size, fastest inference

📄 License

Apache 2.0 License - This model inherits the license from the base Kanana model.

🙏 Acknowledgments

• Base Model: Kakao Corp for Kanana Nano 2.1B Instruct
• Dataset: Korean Legal Terminology
• QLoRA: Tim Dettmers et al. for the QLoRA method
• Libraries: HuggingFace Transformers, PEFT, TRL, bitsandbytes
• GGUF: llama.cpp for GGUF format

📚 Related Resources

• Training Code: ko_legal_finetune
• Training Dataset: korean-legal-terminology
• Base Model: kanana-nano-2.1b-instruct

📧 Contact

For questions or issues, please visit the GitHub repository.