Fine-Tuning DaVinci-002: A Practical Guide for Enhanced LLM Performance
Introduction:
Fine-tuning DaVinci-002 is a crucial process that significantly boosts the model’s performance for specific tasks and applications. This customization leads to improved results, cost efficiency, reduced latency, and outputs tailored to specific needs. Whether it’s for customer service automation, content generation, or text summarization, fine-tuning DaVinci-002 can substantially enhance its efficacy.
Purpose and Value of Fine-Tuning:
Fine-tuning offers the ability to customize and optimize OpenAI models like DaVinci-002 for specific tasks. This enhancement positively impacts performance across various applications, such as generating content, or assisting with translations.
Preparing Your Dataset:
The training data should be formatted correctly, typically as JSON Lines (JSONL) documents, where each line represents a prompt-completion pair. For models with chat task types, the dataset should include a list of JSON objects representing conversation turns.
Example Format:
{"prompt": "<prompt text>", "completion": "<ideal generated text>"}
{"prompt": "<prompt text>", "completion": "<ideal generated text>"}
{"prompt": "<prompt text>", "completion": "<ideal generated text>"}Dependency Installation dataset conversion:
To get started, make sure to install the required dependencies using the following commands:
pip install openai==0.28
pip install -U scikit-learn
To convert your dataset into the required format, use the provided Python code:
import json
import pandas as pd
from openai import OpenAI
def create_dataset(df):
fine_tuning_data = []
for _, row in df.iterrows():
fine_tuning_data.append (
{"prompt": row['prompt'], "completion": row['completion']}
)
return fine_tuning_data
if __name__ == "__main__":
df = pd.read_csv("instruction_dataset.csv")
converted_data = create_dataset(df)Train and Test Splits:
After collecting the initial dataset, split it into training and test portions using the scikit-learn library:
from sklearn.model_selection import train_test_split
train_data, val_data = train_test_split(
converted_data,
test_size=0.1,
random_state=42
)Creating JSONL Files:
JSONL stands for JSON Lines, which is a format where each line is a valid JSON object. This format is commonly used for storing and exchanging structured data, especially when dealing with datasets for machine learning.
In the context of fine-tuning OpenAI’s models, it’s essential to have the training and validation data in the JSONL format. Each line in the JSONL file represents a prompt-completion pair, defining the input prompt and the corresponding desired output.
def write_to_jsonl(data, file_path):
with open(file_path, 'w') as file:
for entry in data:
json.dump(entry, file)
file.write('\n')
training_file_name = "train.jsonl"
validation_file_name = "val.jsonl"
write_to_jsonl(train_data, training_file_name)
write_to_jsonl(val_data, validation_file_name)Format Check of Data:
Before proceeding with fine-tuning the model, it’s crucial to ensure that the data adheres to the correct format expected by the fine-tuning API. The provided code snippet includes checks for various aspects of the dataset to catch potential errors.
# Format error checks
format_errors = defaultdict(int)
for ex in dataset:
if not isinstance(ex, dict):
format_errors["data_type"] += 1
continue
messages = ex.get("messages", None)
if not messages:
format_errors["missing_messages_list"] += 1
continue
for message in messages:
if "role" not in message or "content" not in message:
format_errors["message_missing_key"] += 1
if any(k not in ("role", "content", "name", "function_call") for k in message):
format_errors["message_unrecognized_key"] += 1
if message.get("role", None) not in ("system", "user", "assistant", "function"):
format_errors["unrecognized_role"] += 1
content = message.get("content", None)
function_call = message.get("function_call", None)
if (not content and not function_call) or not isinstance(content, str):
format_errors["missing_content"] += 1
if not any(message.get("role", None) == "assistant" for message in messages):
format_errors["example_missing_assistant_message"] += 1
if format_errors:
print("Found errors:")
for k, v in format_errors.items():
print(f"{k}: {v}")
else:
print("No errors found")Output: No errors found
Upload Training File:
Once the training and validation data are formatted correctly, the next step is to validate and upload these files using the OpenAI Files API. This process involves interacting with the OpenAI API to create files that will be used for fine-tuning.
from openai import OpenAI
client = OpenAI(api_key="API Key")
training_file = client.files.create(
file=open("train.jsonl", "rb"), purpose="fine-tune"
)
validation_file = client.files.create(
file=open("val.jsonl", "rb"), purpose="fine-tune"
)
print("Training file id:", training_file.id)
print("Validation file id:", validation_file.id)Output:
Training file id: file-hgyEd5d7Jp3iRDsyILNawjke
Validation file id: file-alVIHZgdug6BQYqzJaZ2Gf3g
Fine-Tuning the Model:
The fine-tuning process involves training a pre-existing model, in this case, DaVinci-002, on a specific dataset to improve its performance for a particular task. The provided code initiates the fine-tuning job using the OpenAI API.de:
suffix_name = "instruct_data"
response = client.fine_tuning.jobs.create(
training_file=training_file.id,
validation_file=validation_file.id,
model="davinci-002",
suffix=suffix_name,
)
responseOutput:
{'id': 'ftjob-sBBiGa9D0jEFRbJanxOUZ8Tv',
'created_at': 1700565532,
'error': None,
'fine_tuned_model': None,
'finished_at': None,
'hyperparameters': Hyperparameters(n_epochs=3, batch_size=1, learning_rate_multiplier=2),
'model': 'davinci-002',
'object': 'fine_tuning.job',
'organization_id': 'org-MnEEOPe6VrPVcYcy7Z5oloya',
'result_files': [],
'status': 'running',
'trained_tokens': None,
'training_file': 'file-hgyEd5d7Jp3iRDsyILNawjke',
'validation_file': 'file-alVIHZgdug6BQYqzJaZ2Gf3g'}Get Fine-Tuned Model ID:
After successful fine-tuning, retrieve the ID of the fine-tuned model:
client.fine_tuning.jobs.list(limit=10)Output:
SyncCursorPage[FineTuningJob](data=[FineTuningJob(id='ftjob-sBBiGa9D0jEFRbJanxOUZ8Tv', created_at=1700565532, error=None, fine_tuned_model=None, finished_at=None, hyperparameters=Hyperparameters(n_epochs=3, batch_size=1, learning_rate_multiplier=2), model='davinci-002', object='fine_tuning.job', organization_id='org-MnEEOPe6VrPVcYcy7Z5oloya', result_files=[], status='running', trained_tokens=None, training_file='file-hgyEd5d7Jp3iRDsyILNawjke', validation_file='file-alVIHZgdug6BQYqzJaZ2Gf3g')Test the Fine-Tuned Model:
Once the fine-tuning process is complete, it’s time to test the fine-tuned model by generating responses to specific prompts. The provided code snippet demonstrates how to interact with the fine-tuned model using OpenAI’s API.
YOUR_PROMPT = "Create a 3 turn conversation between a customer and a grocery store clerk - that is, 3 per person. Then tell me what they talked about."
FINE_TUNED_MODEL = "ft:davinci-002:org-limited:instruct-data:8NJ9Rnvc"
openai.api_key = "OpenAI Key"
response = openai.Completion.create(
engine=FINE_TUNED_MODEL,
prompt=YOUR_PROMPT)
print(response.choices[0].text)Conclusion:
In summary, fine-tuning DaVinci-002 is a user-friendly process with significant benefits. By following our guide, users can tailor the model for specific tasks, enhancing performance and achieving outputs aligned with their needs. The integration of scikit-learn and OpenAI’s API simplifies the journey from dataset preparation to model evaluation.
This approach is applicable across diverse domains, from customer service automation to content generation. The provided Python code streamlines dataset conversion and ensures a smooth fine-tuning process. The resulting model, validated and tested with confidence, signifies a step toward the future of AI customization.
In a world where adaptability and precision matter, this guide serves as a valuable resource, offering a clear path to unlocking DaVinci-002’s full potential for enhanced AI performance.
