| --- |
| license: llama3 |
| library_name: peft |
| base_model: unsloth/llama-3-8b-bnb-4bit |
| model-index: |
| - name: Llama3_8B_Odia_Unsloth |
| results: [] |
| --- |
| |
| # Llama3_8B_Odia_Unsloth |
| |
| Llama3_8B_Odia_Unsloth is a fine-tuned Odia large language model with 8 billion parameters, and it is based on Llama3. The model is fine-tuned on a comprehensive [171k Odia instruction set](https://huggingface.co/datasets/OdiaGenAI/all_combined_odia_171k), encompassing domain-specific and cultural nuances. |
|
|
| The fine-tuning process leverages Unsloth, expediting the training process for optimal efficiency. |
|
|
| For more details about the model, data, training procedure, and evaluations, go through the blog [post](https://www.odiagenai.org/blog/odiagenai-releases-llama3-fine-tuned-model-for-the-odia-language). |
|
|
| ## Model Description |
| * Model type: A 8B fine-tuned model |
| * Primary Language(s): Odia and English |
| * License: Llama3 |
|
|
|
|
| ## Inference |
|
|
| Sample inference script. |
|
|
| ### Installation |
| ``` |
| #Install Unsloth |
| %%capture |
| import torch |
| major_version, minor_version = torch.cuda.get_device_capability() |
| # Must install separately since Colab has torch 2.2.1, which breaks packages |
| !pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git" |
| if major_version >= 8: |
| # Use this for new GPUs like Ampere, Hopper GPUs (RTX 30xx, RTX 40xx, A100, H100, L40) |
| !pip install --no-deps packaging ninja einops flash-attn xformers trl peft accelerate bitsandbytes |
| else: |
| # Use this for older GPUs (V100, Tesla T4, RTX 20xx) |
| !pip install --no-deps xformers trl peft accelerate bitsandbytes |
| pass |
| ``` |
|
|
| ### Model loading |
| ``` |
| from unsloth import FastLanguageModel |
| import torch |
| max_seq_length = 2048 |
| dtype = None # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+ |
| load_in_4bit = True # Use 4bit quantization to reduce memory usage. Can be False. |
| |
| model, tokenizer = FastLanguageModel.from_pretrained( |
| model_name = "OdiaGenAI-LLM/Llama3_8B_Odia_Unsloth", |
| max_seq_length = max_seq_length, |
| dtype = dtype, |
| load_in_4bit = load_in_4bit, |
| ) |
| |
| alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. |
| |
| ### Instruction: |
| {} |
| |
| ### Input: |
| {} |
| |
| ### Response: |
| {}""" |
| ``` |
| ### Inference |
| ``` |
| FastLanguageModel.for_inference(model) |
| inputs = tokenizer( |
| [ |
| alpaca_prompt.format( |
| "କୋଭିଡ୍ 19 ର ଲକ୍ଷଣଗୁଡ଼ିକ କ’ଣ?", # instruction |
| "", # input |
| "", # output - leave this blank for generation! |
| ) |
| ], return_tensors = "pt").to("cuda") |
| |
| outputs = model.generate(**inputs, max_new_tokens = 512, use_cache = True) |
| tokenizer.batch_decode(outputs) |
| |
| ``` |
|
|
| ### Citation Information |
|
|
| If you find this model useful, please consider giving 👏 and citing: |
|
|
| ``` |
| @misc{Llama3_8B_Odia_Unsloth, |
| author = {Shantipriya Parida and Sambit Sekhar and Debasish Dhal and Shakshi Panwar}, |
| title = {OdiaGenAI Releases Llama3 Fine-tuned Model for the Odia Language}, |
| year = {2024}, |
| publisher = {Hugging Face}, |
| journal = {Hugging Face repository}, |
| howpublished = {\url{https://huggingface.co/OdiaGenAI}}, |
| } |
| ``` |
|
|
| ### Contributions |
|
|
| - Dr.Shantipriya Parida |
| - Sambit Sekhar |
| - Debasish Dhal |
| - Shakshi Panwar |
