henokyemam/safegate-Qwen2.5-7B-Instruct_lora_multiple_violations_datamix3_rank1

---

                               library_name: accelerate
                               license: apache-2.0
                               tags: [Qwen2.5-7B-Instruct, text-generation, chat]
                               datasets: [lora_multiple_violations_datamix3_rank1]
                               model-index:
                               - name: Qwen2.5-7B-Instruct (lora_multiple_violations_datamix3_rank1)
                               results: []
                               ---

                               # Qwen2.5-7B-Instruct — lora_multiple_violations_datamix3_rank1

                       # Loading the Model

                       This is a LoRA adapter trained on the **[INSERT BASE MODEL HERE: Qwen/Qwen2.5-7B-Instruct]** model.

                       To load the full fine-tuned model for inference, run:

                       ```python
                       import torch
                       from transformers import AutoModelForCausalLM, AutoTokenizer
                       from peft import PeftModel

                       # 1. Load the Base Model
                       base_model_name = "Qwen/Qwen2.5-7B-Instruct"
                       base_model = AutoModelForCausalLM.from_pretrained(
                           base_model_name,
                           torch_dtype=torch.bfloat16 # or appropriate dtype
                       )

                       # 2. Load the Fine-Tuned Adapter
                       repo_id = "henokyemam/safegate-Qwen2.5-7B-Instruct_lora_multiple_violations_datamix3_rank1"
                       model = PeftModel.from_pretrained(base_model, repo_id)
                       model = model.merge_and_unload() # Optional: Merge weights for easier deployment

                       # 3. Load the Tokenizer
                       tokenizer = AutoTokenizer.from_pretrained(repo_id)



                               ## Training hyperparameters
                               ```yaml
                               learning_rate: 0.0005
                               learning_scheduler: <accelerate.scheduler.AcceleratedScheduler object at 0x7f27e31f1ca0>
                               num_training_examples: 30610
                               batch_size_per_device: 1
                               gradient_accumulation_steps: 16
                               effective_batch_size: 16
                               max_steps: 720
                               num_epochs: 3
                               optimizer: adamw
                               num_warmup_steps: 21
                               lr_scheduler_type: linear
                               max_seq_length: 1050
                               precision: bf16
                               gradient_checkpointing: true
                               seed:42
                               ```