thsluck/llm-course-hw3-tinyllama-qlora

QLoRA

Model Details

This repository features a QLoRA fine-tuned model for tweet sentiment classification trained as part of VK's LLM course.

Model Description

PEFT (Parameter-Efficient Fine-Tuning) and QLoRA (Quantized Low-Rank Adaptation) are two techniques used in machine learning to efficiently adapt large pre-trained neural networks to specific tasks without requiring extensive computational resources.

PEFT (Parameter-Efficient Fine-Tuning): PEFT is a technique that focuses on updating only a small subset of the model’s parameters during fine-tuning, rather than the entire network. This approach reduces computational costs and memory usage, making it feasible to adapt large models to new tasks on devices with limited resources. By targeting specific layers or parameters that are most relevant for the task, PEFT achieves significant improvements in efficiency while maintaining performance.

QLoRA (Quantized Low-Rank Adaptation): QLoRA combines quantization with low-rank adaptation to efficiently fine-tune large pre-trained models. It leverages quantization to reduce memory usage by representing model weights with lower precision, alongside low-rank decomposition of weight updates that focuses on crucial components. This approach enables significant reductions in both memory and computational demands during adaptation, maintaining model accuracy while minimizing resource requirements. QLoRA is particularly effective for deploying and adapting models in environments with limited computational resources, ensuring scalable and efficient fine-tuning.

In this case, TinyLlama/TinyLlama-1.1B-Chat-v1.0 is used as the pre-trained model. This model is further fine-tuned using QLoRA to classify tweet's sentiment.

Analysis

References

• Model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
• Dataset: cardiffnlp/tweet_eval
• Original Article: https://arxiv.org/pdf/2305.14314