| --- |
| base_model: google/gemma-2b |
| library_name: peft |
| pipeline_tag: text-generation |
| tags: |
| - base_model:adapter:google/gemma-2b |
| - lora |
| - transformers |
| - recipe-generation |
| - unsloth |
| datasets: |
| - corbt/all-recipes |
| license: apache-2.0 |
| language: |
| - en |
| --- |
| |
| # Gemma-2B Recipe Generator (LoRA Adapter) |
|
|
| A LoRA fine-tuned adapter for [google/gemma-2b](https://huggingface.co/google/gemma-2b) trained on recipe data. Given a prompt like "Recipe for chocolate chip cookies:", the model generates structured recipes with ingredients and step-by-step directions. |
|
|
| ## Why fine-tune? |
|
|
| Gemma-2B is a general-purpose model — it knows about recipes but doesn't reliably produce clean, structured output. Fine-tuning on 5,000 recipes teaches it a consistent format and keeps it focused on the task. Here's a side-by-side comparison: |
|
|
| ### Base Gemma-2B (no fine-tuning) |
|
|
| > **Prompt:** `Recipe for beef tacos:` |
|
|
| ``` |
| Recipe for beef tacos: |
| 1 lb ground beef |
| 2 large onion, chopped |
| 1 large tomato, chopped |
| 3-4 fresh jalapeños, sliced |
| Salt and pepper to taste |
| 1 tbsp olive oil |
| 1 can refried beans |
| 1 can tomato sauce |
| 2-3 cups shredded cheese (I used mozzarella) |
| 1 cup sour cream |
| 1 bag of your favorite taco shells |
| |
| Heat the olive oil in a large skillet over medium heat. Brown the ground beef, |
| then drain off excess grease. Add onions, jalapeño, and tomatoes, along with |
| salt and pepper. Cook until vegetables are softened. Stir in the can of refried |
| beans and tomato sauce, then remove from heat. |
| |
| To assemble tacos, place a layer of beef mixture on the bottom of each taco |
| shell. Top with some shredded cheese, followed by sour cream. Serve with your |
| favorite taco toppings, such as lettuce, tomatoes, and salsa! |
| |
| Enjoy! |
| |
| <h2>What is the most popular Mexican dish?</h2> |
| |
| The top five Mexican dishes include tacos, burritos, enchiladas, tamales, and nachos. |
| ``` |
|
|
| ### Fine-tuned model |
|
|
| > **Prompt:** `Recipe for beef tacos:` |
|
|
| ``` |
| Recipe for beef tacos: |
| Ingredients: |
| - 2 lb. ground beef |
| - 1 can (8 oz.) diced tomatoes |
| - 1/4 c. chopped onion |
| - 1/2 tsp. salt |
| - 1/2 tsp. pepper |
| - 1/2 tsp. garlic powder |
| - 1/2 c. water |
| - 1 can (6 oz.) refried beans |
| - 1 c. shredded cheese |
| - 2 flour tortillas |
| |
| Directions: |
| - Brown ground beef in a skillet; drain. |
| - Combine the other ingredients and mix well. |
| - Pour into a 9 x 5 x 3-inch baking dish. |
| - Cover with the refried beans and cheese. |
| - Bake at 350° for 30 minutes. |
| - Serve with sour cream and lettuce. |
| ``` |
|
|
| ### Key differences |
|
|
| | | Base Gemma-2B | Fine-tuned | |
| |---|---|---| |
| | **Format** | No section headers, mixed styles | Consistent Ingredients/Directions structure | |
| | **Bullet style** | None or `•` bullets | Clean `- ` bullets throughout | |
| | **Focus** | Rambles into commentary, HTML tags, FAQ sections | Stays on-task, recipe only | |
| | **Measurements** | Inconsistent (some missing) | Precise and uniform | |
|
|
| ## Usage |
|
|
| ```python |
| from peft import PeftModel |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| base_model = AutoModelForCausalLM.from_pretrained("google/gemma-2b") |
| model = PeftModel.from_pretrained(base_model, "ClaireLee2429/gemma-2b-recipes-lora") |
| tokenizer = AutoTokenizer.from_pretrained("ClaireLee2429/gemma-2b-recipes-lora") |
| |
| prompt = "Recipe for chocolate chip cookies:\n" |
| inputs = tokenizer(prompt, return_tensors="pt") |
| outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7, top_p=0.9, do_sample=True) |
| print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| ``` |
|
|
| ## Sample Output |
|
|
| > **Prompt:** `Recipe for chocolate chip cookies:` |
|
|
| ``` |
| Recipe for chocolate chip cookies: |
| Ingredients: |
| - 1/2 cup butter |
| - 1/4 cup sugar |
| - 1/4 cup packed brown sugar |
| - 3/4 cup flour |
| - 1/2 teaspoon baking soda |
| - 1/2 teaspoon salt |
| - 1 egg |
| - 1 teaspoon vanilla |
| - 1/2 cup chocolate chips |
| |
| Directions: |
| - In a medium bowl, cream together the butter and sugars. |
| - Add in the egg and vanilla, mixing until combined. |
| - In another bowl, whisk together the flour, baking soda and salt. |
| - Add to the creamed mixture alternately with the chocolate chips, ending with the dry ingredients. |
| - Stir in gently using a rubber spatula. |
| - Drop by rounded teaspoonfuls onto ungreased cookie sheets. |
| - Bake at 350 degrees F for 9 minutes. |
| - Cool on wire racks before serving. |
| ``` |
|
|
| > **Prompt:** `Recipe for pasta carbonara:` |
|
|
| ``` |
| Recipe for pasta carbonara: |
| Ingredients: |
| - 100 g spaghetti |
| - 100 g smoked bacon or pancetta |
| - 250 g mushrooms |
| - 2 eggs |
| - 1 tablespoon olive oil |
| - 1 tablespoon white wine |
| - 1 teaspoon freshly grated parmesan cheese |
| |
| Directions: |
| - Cook the pasta according to package instructions. |
| - Meanwhile, brown the bacon in a frying pan with some olive oil. |
| - Add the mushrooms and cook them until they are tender (about 10 minutes). |
| - Add the cooked pasta to the mushroom mixture along with the eggs and stir well. |
| - Stir in the wine and then sprinkle over the grated parmesan cheese. |
| - Season with salt and pepper and serve immediately. |
| ``` |
|
|
| > **Prompt:** `Recipe for tomato soup:` |
|
|
| ``` |
| Recipe for tomato soup: |
| Ingredients: |
| - 1 (28 ounce) can crushed tomatoes |
| - 1 (4 ounce) can tomato paste |
| - 1 tablespoon dried basil |
| - 1/2 tablespoon salt |
| - 1/2 tablespoon sugar |
| - 1/2 teaspoon garlic powder |
| - 1/4 teaspoon red pepper flakes |
| |
| Directions: |
| - Combine all ingredients in a large pot and bring to a boil. |
| - Reduce heat, cover and simmer until thickened, about 1 hour. |
| - Serve over grilled cheese sandwiches or with tortilla chips. |
| ``` |
|
|
| ## Training Details |
|
|
| ### Training Data |
|
|
| [corbt/all-recipes](https://huggingface.co/datasets/corbt/all-recipes) -- a dataset of 2.1M recipes scraped from the web. Each example contains a recipe title, ingredients list, and cooking directions in a single `input` text field. |
|
|
| 5,000 recipes were randomly sampled for training, split into: |
| - **Train**: 4,500 examples |
| - **Validation**: 500 examples |
|
|
| ### Preprocessing |
|
|
| 1. **Cleaning**: Stripped whitespace, removed empty rows, deduplicated |
| 2. **Tokenization**: Tokenized with Gemma-2B's tokenizer, padded/truncated to 512 tokens |
|
|
| ### Training Hyperparameters |
|
|
| | Parameter | Value | |
| |---|---| |
| | **Method** | LoRA (rank 8) | |
| | **Base model** | google/gemma-2b | |
| | **Epochs** | 1 | |
| | **Batch size** | 4 | |
| | **Learning rate** | 2e-4 (linear decay) | |
| | **LoRA rank (r)** | 8 | |
| | **LoRA alpha** | 16 | |
| | **LoRA dropout** | 0.05 | |
| | **Target modules** | all-linear | |
| | **Precision** | float32 (MPS) | |
| | **Training regime** | fp32 on Apple Silicon (MPS) | |
|
|
| ### Training Results |
|
|
| | Metric | Value | |
| |---|---| |
| | **Train loss** | 1.3928 | |
| | **Val loss** | 1.4030 | |
| | **Val perplexity** | 4.07 | |
| | **Training time** | ~1 hr 21 min | |
| | **Train steps** | 1,125 | |
|
|
| ## Infrastructure |
|
|
| - **Hardware**: Apple Silicon (MPS) |
| - **Software**: Transformers, PEFT 0.18.0, PyTorch 2.9.1 |
| - **Orchestration**: [Dagster](https://dagster.io/) pipeline |
|
|
| ## Pipeline |
|
|
| This adapter was produced by an end-to-end [Dagster pipeline](https://github.com/glee2429/recipe-lm): |
|
|
| ``` |
| raw_dataset -> cleaned_dataset -> tokenized_dataset -> train_val_splits -> trained_model |
| ``` |
|
|
| ## Limitations |
|
|
| - Trained on only 5,000 recipes for 1 epoch -- quality improves with more data/epochs |
| - Recipes may contain factual inaccuracies (incorrect quantities, missing steps) |
| - Generation can ramble after the recipe ends -- consider post-processing to truncate |
| - Not suitable for dietary or safety-critical applications |
|
|
| ## License |
|
|
| This adapter inherits the [Gemma license](https://ai.google.dev/gemma/terms) from the base model. |
|
|
| ### Framework versions |
|
|
| - PEFT 0.18.0 |
|
|
