📊 Creating Synthetic Datasets Using Generative AI
Synthetic datasets have many use cases — from practicing your data analytics and visualization skills to training machine learning models
Do You Need Synthetic Datasets?
Synthetic datasets are computer-made datasets fit to meet specific requirements. They are a quick and cost-effective alternative to collecting and labeling real-world data.
You need synthetic datasets if:
- You need more data than what you currently have.
- Your current data has sensitive information and you don’t want to use it for privacy reasons.
- The data you need requires very specific or rare conditions.
- You’re working on a tight budget but need lots of data.
- You’re in a time crunch and need a specific kind of dataset right away.
Synthetic datasets will not work for you if:
- You are developing applications or extracting insights that rely on real-world data patterns.
- Your work depends on capturing complex, nuanced details from real-world interactions.
When considering synthetic datasets, make sure they align with your project's specific needs. Evaluate whether the synthetic version is enough to represent the complexities of your project.
Remember, while synthetic datasets offer many advantages, they should be used judiciously.
Creating Synthetic Data Using ChatGPT
Picture this: You’re applying as a data analyst at TrendGear, a (fictional) popular online store that sells tech gadgets. They use special marketing techniques and have a lot of data about what customers bought, how they paid, when they shop, preferred shipping method, etc.—data that’s off-limits due to privacy concerns.
To show off your skills, you plan on creating some data visualization projects. However, you want it to be contextually relevant to their business so you decided to make up data that looks like what an online store might have.
Here’s how you can go about it:
1. Create a sample of the dataset you want
Head to the drawing board and list down all of the columns you want to see in your synthetic data. While you’re at it, populate each one with some samples.
- For our purpose, here’s what your sample dataset may look like:
Customer ID: 101, 102, 103, 104
Name: John Doe, Jane Smith, Alice Johnson, Bob Lee
Email: john.doe@example.com, jane.smith@example.com, alice.johnson@example.com, bob.lee@example.com
Product Purchased: Smartphone, Laptop Case, Wireless Headphones, Portable Charger
Purchase Date: 2024–04–25, 2024–04–24, 2024–04–26, 2024–04–22
Amount Spent ($): 299, 25, 120, 30
Age: 28, 34, 22, 45
City: New York, Los Angeles, Chicago, Houston
Payment Method: Credit Card, PayPal, Debit Card, Credit Card
Last Login Date: 2024–05–01, 2024–04–30, 2024–05–03, 2024–05–01
Membership Status: Gold, Silver, Silver, Bronze
Number of Orders: 14, 5, 22, 8
Average Order Value ($): 150, 45, 60, 20
Preferred Shipping Method: Express, Standard, Express, Standard
Account Created Date: 2021–03–15, 2022–07–20, 2020–11–30, 2023–01–25
2. Clean up your sample dataset
The next step is to digitize your sample dataset by typing it as comma-separated values or pipe-separated values. The idea here is to prepare your sample neatly for prompting.
Here’s what the sample dataset would look like as comma-separated values and pipe-separated values:
- Comma-separated values:
Customer ID,Name,Email,Product Purchased,Purchase Date,Amount Spent ($),Age,City,Payment Method,Last Login Date,Membership Status,Number of Orders,Average Order Value ($),Preferred Shipping Method,Account Created Date
101,John Doe,john.doe@example.com,Smartphone,2024-04-25,299,28,New York,Credit Card,2024-05-01,Gold,14,150,Express,2021-03-15
102,Jane Smith,jane.smith@example.com,Laptop Case,2024-04-24,25,34,Los Angeles,PayPal,2024-04-30,Silver,5,45,Standard,2022-07-20
103,Alice Johnson,alice.johnson@example.com,Wireless Headphones,2024-04-26,120,22,Chicago,Debit Card,2024-05-03,Silver,22,60,Express,2020-11-30
104,Bob Lee,bob.lee@example.com,Portable Charger,2024-04-22,30,45,Houston,Credit Card,2024-05-01,Bronze,8,20,Standard,2023-01-25- Pipe-separated values:
| Customer ID | Name | Email | Product Purchased | Purchase Date | Amount Spent ($) | Age | City | Payment Method | Last Login Date | Membership Status | Number of Orders | Average Order Value ($) | Preferred Shipping Method | Account Created Date |
|-------------|---------------|---------------------------|------------------------|---------------|------------------|-----|-------------|----------------|-----------------|-------------------|------------------|-------------------------|--------------------------|----------------------|
| 101 | John Doe | john.doe@example.com | Smartphone | 2024-04-25 | 299 | 28 | New York | Credit Card | 2024-05-01 | Gold | 14 | 150 | Express | 2021-03-15 |
| 102 | Jane Smith | jane.smith@example.com | Laptop Case | 2024-04-24 | 25 | 34 | Los Angeles | PayPal | 2024-04-30 | Silver | 5 | 45 | Standard | 2022-07-20 |
| 103 | Alice Johnson | alice.johnson@example.com | Wireless Headphones | 2024-04-26 | 120 | 22 | Chicago | Debit Card | 2024-05-03 | Silver | 22 | 60 | Express | 2020-11-30 |
| 104 | Bob Lee | bob.lee@example.com | Portable Charger | 2024-04-22 | 30 | 45 | Houston | Credit Card | 2024-05-01 | Bronze | 8 | 20 | Standard | 2023-01-25 |Pro Tip: You can ask GenAI (such as ChatGPT) to clean up your sample data for you, but make sure to check if the output is as intended.
3. Use One-Shot Prompting
In this step, you will take the cleaned up sample dataset and feed it to a generative AI (here, we’ll use ChatGPT). You’ll employ one-shot prompting, an effective prompt-engineering technique for this particular task.
Think of one-shot prompting as giving the model the first few parts of a sequence and allowing it to complete the rest in a similar structure.
You have two ways to get the synthetic dataset:
- Request for a file (straight from the GenAI)
- Request for a Python Script (and run it to get the synthetic dataset)
Method 1: Request for a File
When you request for a file, the idea is to ask the Generative AI to produce a downloadable file. This capability may be limited to more advanced models. In the case of ChatGPT, it works best using GPT 4.0 (accessible through the paid tier).
- Here’s an example prompt:
Instructions:
1. Generate a synthetic dataset following the dataset example below.
2. Compile the synthetic dataset into a CSV file.
3. Add a link to download the CSV file.
Dataset Example:
{{
##replace this with your cleaned sample dataset:
| Customer ID | Name | Email | Product Purchased | Purchase Date | Amount Spent ($) | Age | City | Payment Method | Last Login Date | Membership Status | Number of Orders | Average Order Value ($) | Preferred Shipping Method | Account Created Date |
|-------------|---------------|---------------------------|------------------------|---------------|------------------|-----|-------------|----------------|-----------------|-------------------|------------------|-------------------------|--------------------------|----------------------|
| 101 | John Doe | john.doe@example.com | Smartphone | 2024-04-25 | 299 | 28 | New York | Credit Card | 2024-05-01 | Gold | 14 | 150 | Express | 2021-03-15 |
| 102 | Jane Smith | jane.smith@example.com | Laptop Case | 2024-04-24 | 25 | 34 | Los Angeles | PayPal | 2024-04-30 | Silver | 5 | 45 | Standard | 2022-07-20 |
| 103 | Alice Johnson | alice.johnson@example.com | Wireless Headphones | 2024-04-26 | 120 | 22 | Chicago | Debit Card | 2024-05-03 | Silver | 22 | 60 | Express | 2020-11-30 |
| 104 | Bob Lee | bob.lee@example.com | Portable Charger | 2024-04-22 | 30 | 45 | Houston | Credit Card | 2024-05-01 | Bronze | 8 | 20 | Standard | 2023-01-25 |
}}- Here’s a sample response:
Next, download the dataset by clicking the link provided. Then, open the file using a supported program and there you should see your synthetic dataset.
- Here’s what the synthetic dataset looks like:
Method 2: Request for a Python Script
You also have the option to request for a Python script. The advantage of the second approach is that this works even in GPT 3.5.
This method is also more versatile as it allows you to dabble into the code and tweak it to your liking. However, method 2 involves an extra step of running the code snippet on your own to get the synthetic dataset.
- Here’s an example prompt:
Instructions:
1. Create a Python script that will generate a synthetic dataset following
the structure of the example below.
2. The output of the Python code must be a CSV file containing the
synthetic dataset.
3. The code must automatically download the CSV file into the coding environment.
Dataset Example:
{{
##replace this with your dataset example:
| Customer ID | Name | Email | Product Purchased | Purchase Date | Amount Spent ($) | Age | City | Payment Method | Last Login Date | Membership Status | Number of Orders | Average Order Value ($) | Preferred Shipping Method | Account Created Date |
|-------------|---------------|---------------------------|------------------------|---------------|------------------|-----|-------------|----------------|-----------------|-------------------|------------------|-------------------------|--------------------------|----------------------|
| 101 | John Doe | john.doe@example.com | Smartphone | 2024-04-25 | 299 | 28 | New York | Credit Card | 2024-05-01 | Gold | 14 | 150 | Express | 2021-03-15 |
| 102 | Jane Smith | jane.smith@example.com | Laptop Case | 2024-04-24 | 25 | 34 | Los Angeles | PayPal | 2024-04-30 | Silver | 5 | 45 | Standard | 2022-07-20 |
| 103 | Alice Johnson | alice.johnson@example.com | Wireless Headphones | 2024-04-26 | 120 | 22 | Chicago | Debit Card | 2024-05-03 | Silver | 22 | 60 | Express | 2020-11-30 |
| 104 | Bob Lee | bob.lee@example.com | Portable Charger | 2024-04-22 | 30 | 45 | Houston | Credit Card | 2024-05-01 | Bronze | 8 | 20 | Standard | 2023-01-25 |
}}- Here’s a sample response:
Next, copy the generated code and paste it on your local IDE (Visual Studio, IntelliJ, Notepad++).
If you don’t have a local IDE, you can simply create a Google Colab notebook and run the code there. Click this link and login to Google Colab: https://colab.research.google.com/#create=true
It’s possible to encounter errors when running the code. For example, when I ran the code above, it gave me an error saying the ‘Faker’ library is missing. So I had to add that extra line of code. If you’re unsure of what to do, you can always ask the GenAI or do a quick Google search.
- Here’s the working code:
!pip install faker
import pandas as pd
from faker import Faker
import random
from datetime import datetime, timedelta
# Set up Faker to generate fake data
fake = Faker()
# Define the number of rows for the dataset
num_rows = 1000
# Generate synthetic data
data = []
for _ in range(num_rows):
customer_id = fake.random_int(min=1000, max=9999)
name = fake.name()
email = fake.email()
product_purchased = random.choice(["Smartphone", "Laptop Case", "Wireless Headphones", "Portable Charger"])
purchase_date = fake.date_between(start_date='-30d', end_date='today')
amount_spent = round(random.uniform(20, 500), 2)
age = fake.random_int(min=18, max=80)
city = fake.city()
payment_method = random.choice(["Credit Card", "PayPal", "Debit Card"])
last_login_date = fake.date_between(start_date='-180d', end_date='today')
membership_status = random.choice(["Gold", "Silver", "Bronze"])
number_of_orders = fake.random_int(min=1, max=50)
average_order_value = round(amount_spent / number_of_orders, 2)
preferred_shipping_method = random.choice(["Express", "Standard"])
account_created_date = fake.date_time_between(start_date='-2y', end_date='-1y')
data.append([customer_id, name, email, product_purchased, purchase_date, amount_spent, age, city, payment_method,
last_login_date, membership_status, number_of_orders, average_order_value, preferred_shipping_method,
account_created_date])
# Create a DataFrame
df = pd.DataFrame(data, columns=['Customer ID', 'Name', 'Email', 'Product Purchased', 'Purchase Date',
'Amount Spent ($)', 'Age', 'City', 'Payment Method', 'Last Login Date',
'Membership Status', 'Number of Orders', 'Average Order Value ($)',
'Preferred Shipping Method', 'Account Created Date'])
# Save DataFrame to CSV
df.to_csv('synthetic_dataset.csv', index=False)- Here’s a screenshot of the code on Google Colab:
After running the code, the dataset will be saved somewhere on your computer. In Google Colab, it should be in the ‘files’ tab on the left.
Next, save or find the dataset on your computer and open it using a supported program.
- Here’s what the synthetic dataset looks like:
At this point, you’re already equipped with a contextually relevant dataset to use for your TrendGear project!
4. Review the Dataset
After generating your synthetic dataset, examine it closely to ensure it meets your requirements. Here’s a checklist of things to look at:
- Numbers: Make sure the numbers in your dataset are in the right range. Also, check if the numbers should be whole or with decimals.
- Dates: See if the dates in your dataset make sense. Ensure the dates are logical and consistent with your dataset’s context.
- Other Labels: For categories like “Payment Method,” make sure the options are what you expect. For example, payment methods could be PayPal, Credit Card, or Debit Card, but not things like cheque or cash. This depends on what kind of analysis you’re doing.
Aside from these, you may also need to perform some statistics on your data to ensure they fit what you need.
For our example scenario, our synthetic dataset is sufficient to be used in data visualization projects for our application at TrendGear.
Some Limitations of Using GenAI for Synthetic Dataset Generation
- The ‘request for a file’ method may only work on GPT 4.0 and newer.
- The ‘request for a Python script’ method may not work right away and you would need to troubleshoot code errors.
- Synthetic datasets generated using this method may not capture the subtle nuances and natural variety of the real world.
- The provided sample prompts might need adjustments to function correctly, such as specifying the required number of data points, providing value ranges, etc.
Wrapping Up
In this guide, we looked at how to create synthetic data using Generative AI, specifically ChatGPT.
We considered a fictional scenario where real data isn’t available to us due to privacy issues. We walked through making a dataset from scratch, generating it with AI, and checking if it fits our needs.
Remember, while this method is quick and cheap, it doesn’t perfectly mimic real-world data and might need some tweaking. This approach is handy when you’re short on data and need a flexible solution.
