Deign86/deped-math-qwen2.5-7b-checkpoint-700-lora

DepEd Math Tutor LoRA (Checkpoint 700)

This repository contains the LoRA adapter checkpoint at training step 700 for a Grade 11-12 mathematics tutoring assistant aligned to Philippine DepEd senior high school topics.

Model Details

Model Description

• Developed by: Deign Lazaro (MathPulse AI)
• Funded by: Self-funded / independent development
• Shared by: Deign86
• Model type: PEFT LoRA adapter (not a standalone merged full model)
• Language(s): English, Filipino (Taglish instructional style)
• License: Apache-2.0 (inherits base model license constraints)
• Finetuned from model: unsloth/Qwen2.5-7B-Instruct-unsloth-bnb-4bit

Model Sources

• Training workflow: Kaggle notebook pipeline with Unsloth + PEFT QLoRA
• Checkpoint source run: deignlazaro/qwen25-deped-lora-production (Kaggle)
• Current artifact: Step-700 adapter snapshot selected as the last stable checkpoint before further-loss instability

Uses

Direct Use

This adapter is intended for:

• Grade 11-12 math tutoring
• Step-by-step worked solutions
• Topic explanation and guided practice prompts

Because this is a LoRA adapter, load it on top of the base model.

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "unsloth/Qwen2.5-7B-Instruct-unsloth-bnb-4bit"
adapter_id = "Deign86/deped-math-qwen2.5-7b-checkpoint-700-lora"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base_model = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")
model = PeftModel.from_pretrained(base_model, adapter_id)

Downstream Use

Recommended downstream use cases:

• Classroom support copilots
• Practice-question explanation bots
• Internal educational QA tools for SHS math review

Out-of-Scope Use

Not intended for:

• High-stakes grading decisions without human review
• Medical, legal, or safety-critical advice
• Formal theorem proving or guaranteed symbolic correctness

Bias, Risks, and Limitations

• May produce incorrect arithmetic or overconfident reasoning.
• Performance is domain-tilted toward SHS math instruction style and may degrade outside that scope.
• Explanations can vary in quality depending on prompt clarity and problem formatting.
• Should be used with human oversight for exams, assessment feedback, and official school outputs.

Recommendations

• Use deterministic decoding for tutoring outputs where consistency matters.
• Add lightweight answer checking (numeric/unit checks) in production.
• Keep fallback routing to stronger/general models for out-of-domain prompts.
• Run periodic red-team prompts for hallucination and curriculum drift checks.

Training Details

Training Data

• DepEd SHS (Grades 11-12) mathematics tutoring-style instruction data
• Curriculum focus includes General Mathematics, Statistics and Probability, Business Math, and Precalculus
• Data formatted for chat-style supervised fine-tuning

Training Procedure

• Method: QLoRA (Unsloth + PEFT)
• Base model: 4-bit quantized Qwen2.5-7B Instruct variant
• Checkpointing: periodic step checkpoints; this card tracks step-700
• Rationale for selection: selected as the last stable checkpoint before loss escalation in continued training

Training Hyperparameters (core)

• LoRA rank (r): 32
• LoRA alpha: 16
• LoRA dropout: 0.05
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• T4-safe sequence length during stable runs: 1024

Compute

• Hardware type: NVIDIA Tesla T4 (Kaggle)
• Precision regime: 4-bit base model + LoRA adapters

Evaluation

Current Status

This release is a checkpoint artifact for integration testing and iterative development. No formal benchmark table is claimed in this card.

Practical Validation Guidance

Before production promotion, validate on:

• Topic coverage by grade-level competencies
• Numerical accuracy on held-out problem sets
• Explanation quality and rubric alignment
• Hallucination/error-rate under ambiguous prompts

Technical Specifications

Architecture and Objective

• Architecture family: Qwen2.5 (causal LM)
• Adaptation method: PEFT LoRA adapter layers
• Objective: instruction-following for educational math tutoring interactions

Files in This Repo

• adapter_model.safetensors
• adapter_config.json
• Tokenizer artifacts (tokenizer.json, tokenizer_config.json, vocab.json, merges.txt, etc.)

Model Card Authors

• Deign Lazaro (Deign86)

Model Card Contact

For feedback, use the Hugging Face Discussions tab on this model repository.