{"id":232,"date":"2024-03-30T06:45:31","date_gmt":"2024-03-30T06:45:31","guid":{"rendered":"https:\/\/www.aerolift.ai\/blog\/?p=232"},"modified":"2024-03-30T06:45:31","modified_gmt":"2024-03-30T06:45:31","slug":"deep-dive-into-fine-tuning-the-tiny-llama-model-on-a-custom-dataset","status":"publish","type":"post","link":"https:\/\/www.aerolift.ai\/blog\/deep-dive-into-fine-tuning-the-tiny-llama-model-on-a-custom-dataset\/","title":{"rendered":"Deep Dive into Fine-Tuning the Tiny-Llama Model on a Custom Dataset"},"content":{"rendered":"\n

In the rapidly evolving landscape of machine learning, the ability to fine-tune models on custom datasets is a game-changer. It allows for the creation of models that are not only powerful but also tailored to specific domains, enhancing their performance and relevance. This article delves into the intricacies of fine-tuning the Tiny-Llama model on a custom dataset, exploring the entire process from data preparation to model training and highlighting advanced training techniques like PEFT and QLORA.<\/p>\n\n\n\n

<\/p>\n\n\n\n

The Power of Fine-Tuning<\/strong><\/span><\/h5>\n\n\n\n

Fine-tuning is a process where a pre-trained model is further trained (i.e., “fine-tuned”) on a smaller, domain-specific dataset. This approach leverages the knowledge gained during the initial training phase and adapts it to the specific needs of the new task. The Tiny-Llama model, a compact and efficient variant of the GPT family, is particularly well-suited for fine-tuning due to its balance between performance and resource efficiency.<\/p>\n\n\n\n

<\/p>\n\n\n\n

Data Prepa<\/strong><\/span>r<\/strong><\/span>a<\/strong><\/span>t<\/strong><\/span>ion: The Foundation of Success<\/strong><\/span><\/h5>\n\n\n\n

The success of fine-tuning hinges on the quality and relevance of the training data. For this tutorial, we’ll explore how to prepare a dataset from PDF documents, converting it into a structured format that can be efficiently used for training.<\/p>\n\n\n\n

<\/p>\n\n\n\n

Extracting Text from PDFs<\/strong><\/span><\/h6>\n\n\n\n

The first step involves extracting meaningful text from PDF documents. This is crucial for generating questions and answers that can be used to fine-tune the model. We use the\u00a0PyPDF2<\/code>\u00a0library, which provides a straightforward way to read PDF files and extract text.<\/p>\n\n\n\n

<\/p>\n\n\n\n

import PyPDF2\r\n\r\ndef extract_text_from_pdf(file_path):\r\n    pdf_file_obj = open(file_path, 'rb')\r\n    pdf_reader = PyPDF2.PdfReader(pdf_file_obj)\r\n    text = ''\r\n    for page_num in range(len(pdf_reader.pages)):\r\n        page_obj = pdf_reader.pages[page_num]\r\n        text += page_obj.extract_text()\r\n    pdf_file_obj.close()\r\n    return text\r\n<\/code><\/pre>\n\n\n\n
<\/p>\n\n\n\n
Generating Questions and Answers<\/strong><\/span><\/h6>\n\n\n\n
Once we have the text, we need to generate questions and answers. This is achieved by sending the text to an API (in this case, using the GPT-4 model from OpenAI) and processing the response to extract the questions and answers.<\/p>\n\n\n\n
<\/p>\n\n\n\n
import openai\r\nimport json\r\nfrom typing import List\r\nfrom tqdm import tqdm\r\n\r\ndef generate_questions_answers(text_chunk):\r\n    messages = [\r\n        {'role': 'system', 'content': 'You are an API that converts bodies of text into a single question and answer into a JSON format. Each JSON \" \\\r\n          \"contains a single question with a single answer. Only respond with the JSON and no additional text. \\n.'},\r\n        {'role': 'user', 'content': 'Text: ' + text_chunk}\r\n    ]\r\n\r\n    response = openai.chat.completions.create(\r\n        model=\"gpt-4\",\r\n        messages=messages,\r\n        max_tokens=2048,\r\n        n=1,\r\n        stop=None,\r\n        temperature=0.7,\r\n    )\r\n\r\n    response_text = response.choices[0].message.content.strip()\r\n    try:\r\n        json_data = json.loads(response_text)\r\n        return json_data\r\n    except json.JSONDecodeError:\r\n        print(\"Error: Response is not valid JSON.\")\r\n        return []\r\n<\/code><\/pre>\n\n\n\n
<\/p>\n\n\n\n
Processing Text and Writing to JSON<\/strong><\/span><\/h6>\n\n\n\n
The final step in data preparation involves processing the text in chunks, generating questions and answers for each chunk, and writing the results to a JSON file. This structured format is essential for the subsequent steps of training.<\/p>\n\n\n\n
def process_text(text: str, chunk_size: int = 4000) -> List[dict]:\r\n    text_chunks = [text[i:i+chunk_size] for i in range(0, len(text), chunk_size)]\r\n    all_responses = []\r\n    for chunk in tqdm(text_chunks, desc=\"Processing chunks\", unit=\"chunk\"):\r\n        responses = generate_questions_answers(chunk)\r\n        all_responses.extend(responses)\r\n    return all_responses\r\n\r\ntext = extract_text_from_pdf('Provide Path to your file')\r\nresponses = {\"responses\": process_text(text)}\r\n\r\nwith open('responses.json', 'w') as f:\r\n    json.dump(responses, f, indent=2)\r\n<\/code><\/pre>\n\n\n\n
<\/p>\n\n\n\n
Converting JSON to CSV for Training<\/strong><\/span><\/h6>\n\n\n\n
With the data prepared, we convert it into a CSV format that is suitable for training. This involves extracting the questions and answers from the JSON and writing them to a CSV file with columns for the prompt, question, and answer.<\/p>\n\n\n\n
import json\r\nimport csv\r\n\r\ninstruction = \"You will answer questions about Machine Learning\"\r\n\r\nwith open('responses.json', 'r', encoding='utf-8') as f:\r\n    responses = json.load(f)\r\n\r\nwith open('responses.csv', 'w', newline='', encoding='utf-8') as csvfile:\r\n    fieldnames = ['prompt', 'question', 'answer']\r\n    writer = csv.DictWriter(csvfile, fieldnames=fieldnames)\r\n    writer.writeheader()\r\n    for response in responses['responses']:\r\n        if 'question' in response and 'answer' in response:\r\n            writer.writerow({'prompt': instruction, 'question': response['question'], 'answer': response['answer']})\r\n<\/code><\/pre>\n\n\n\n
<\/p>\n\n\n\n
Preparing the Data for Training<\/strong><\/span><\/h6>\n\n\n\n
Before training, we need to format the data correctly. This involves reading the CSV file, combining the prompt, question, and answer into a single text column, and then saving the formatted data to a new CSV file.<\/p>\n\n\n\n
import pandas as pd\r\n\r\ndf = pd.read_csv(\"responses.csv\")\r\ndf.fillna(\"\")\r\n\r\ntext_col = []\r\n\r\nfor _, row in df.iterrows():\r\n prompt = \"Below is an instruction which describes a tasks paired with a question that provides further context. Write a response that completes the request.\\n\\n\"\r\n instruction = str(row[\"prompt\"])\r\n input_query = str(row[\"question\"])\r\n response = str(row[\"answer\"])\r\n\r\n if len(input_query.strip()) == 0: \r\n    text = prompt + \"### Instruction:\\n \" + instruction + \"\\n### Response:\\n \" + response\r\n else:\r\n    text = prompt + \"### Instruction:\\n \" + instruction + \"\\n### Input:\" + input_query + \"\\n### Response:\\n \" + response\r\n\r\n text_col.append(text)\r\n\r\ndf.loc[:, \"train\"] = text_col\r\ndf.to_csv(\"train.csv\", index=False)\r\n<\/code><\/pre>\n\n\n\n
<\/p>\n\n\n\n
Training with Auto<\/strong><\/span> <\/strong><\/span>Train Advanced<\/strong><\/span><\/h6>\n\n\n\n
For training, we use Auto Train Advanced, a powerful tool that simplifies the training process. It supports advanced training techniques like PEFT (Process-Efficient Fine-Tuning) and QLORA (Quantized Low-Rank Approximation), which can significantly reduce training time and memory usage.<\/p>\n\n\n\n
Tiny-LLama<\/strong><\/span><\/p>\n\n\n\n
\"\"<\/figure>\n\n\n\n
<\/p>\n\n\n\n
The Tiny-Llama model is a compact and efficient variant of the GPT family, designed for fine-tuning on custom datasets. It leverages the Llama architecture to enhance text generation capabilities, offering a proof of concept for recreating the TinyStories-1M model with improved efficiency and performance. Hosted on Hugging Face, the Tiny-Llama model showcases the potential of fine-tuning and domain-specific modeling in the field of machine learning.<\/p>\n\n\n\n
<\/p>\n\n\n\n
! autotrain llm --train --project-name josh-ops --model Maykeye\/TinyLLama-v0 --data-path \/kaggle\/input\/llamav2 --use-peft --quantization int4 --lr 1e-4 --train-batch-size 15 --epochs 60 --trainer sft\r\n<\/code><\/pre>\n\n\n\n
<\/p>\n\n\n\n
Understanding PEFT and QLORA<\/strong><\/span><\/h6>\n\n\n\n