Two weeks ago, I almost got sick writing an article about how to scrape products from an e-commerce site.
Here's why: I tried to scrape products from an e-commerce site called GymShark with a prompting framework for data classification and editing that I created for the AI assistants in Datablist. At first, it worked, but the data wasn't sorted correctly, which made the entire dataset useless.
That's where I realized different AIs and use cases all need different prompts.
Since when prompting small details can make a huge difference, missing 5% of the details will make your output useless.
Let’s start!
Key Takeaways From This Article:
- Datablist's AI agents can search Google, visit websites, call APIs, and more
- A good prompt makes a huge difference when using an AI agent
- Prompting AI agents is easier when you follow our best practices
- Datablist provides prompting templates, but learning it is still the best way
Datablist’s AI Agents & AI Assistants in Comparison
To understand the differences between prompting for AI agents and prompting for AI assistants, we have to clarify the capabilities of the two.
Here are 2 short and simple overviews of what these tools can do:
AI Assistant
The AI assistants in Datablist are designed for reading, classifying, extracting, editing, and generating text and code
AI Agent
The AI agents in Datablist are AIs equipped with tools that can perform Google searches, render HTML code, navigate through multiple pages, call APIs, visit and extract results from Google News, websites, e-commerce stores and more.
💡 Simple and Easy
Think of AI agents as the system that deals with external data and the AI assistant as the system that deals with the data in your Datablist collection.
Datablist’s AI Agent Capabilities
When it comes to writing an effective prompt for the AI agent, we should list the tools and capabilities of the AI agent again, which are:
- Searching Google
- Searching Google News
- Extracting and generating text or code
- Getting account information
- Visiting a website
- Paginating a website
- Rendering HTML
- Calling APIs
11 Rules for Prompting AI Agents in Datablist
Once you’ve understood the capabilities of the AI agent, you have to follow a few simple but crucial rules to achieve 100% accuracy. These rules are:
- Treat your prompt as a roadmap: This means you should guide the agent with your request and not solely demand something and expect it to work.
- Start always with your goal and provide context about it.
- Tag each section of the prompt to let the agent better interpret what you're writing and to separate the actions/commands.
- Use clear commands to call the tools of the AI agent such as "search Google," "extract text," "get HTML code," etc.
- Tag the outputs and give specific examples, as these will serve as the base for your output configuration.
- Tell it where on the website it can find the information you want to have if you're scraping something specific.
- Use separators between prompt parts (".===") to structure the prompt.
- Tell it which mistakes to avoid so it can prevent them.
- Define how it should deal with errors and missing data.
- Iterate on your prompts and don't blame the AI agent for bad outputs (I used to do it).
- Add the column from which it should take the data from always at the end.
By following these guidelines and best practices for prompting AI agents, you'll save credits and a lot of time iterating on your prompts
📘 Good Prompts Matter
The AI agent will also work if you don't follow any of the rules mentioned above; however, if you follow these rules, your results will be much better, as prompt structure really makes a difference.
AI Agent Prompt Examples: Good Prompt vs. Bad Prompt
AI Agent Prompt: Good Example
First, look at the good prompt and then look at the bad prompt
This is a prompt I wrote recently to scrape products from an e-commerce site.
Let's break down why this is a good prompt:
- It starts with a specific, clearly defined goal and context
- The prompt includes multiple sections which are all tagged — this makes it easy to understand for the AI
- I used clear commands with action-oriented verbs
- The commands follow a clear sequence
- I described what I was looking for and gave it examples
- I told it how to handle data inconsistencies and errors
- It includes specific mistakes that it should avoid
- The sections are clearly separated with ".==="
- The prompt is well structured and guides the AI through my request
- I linked the column with the referring data at the end
AI Agent Prompt: Bad Example
For those of you who think I'm just a tech guy who knows how to write good prompts, here's the truth: first of all, I don't know how to code, and secondly, this is how my prompt looked before:
Yes, some people might still consider this good, but it's not — since the outputs weren't good at all.
In this example, many things went wrong. Although it looks sophisticated, the prompt — and therefore the results — are useless. Here are the things that I did wrong and how you can avoid them:
Bad Prompt Reason #1: Mediocre structure
The prompt lacks a clear flow and organization.
How to avoid it:
Break down your goal into specific steps, like you would explain a task to someone else. For example:
- First, visit the website.
- Then, look for the product section.
- Finally, extract the price.
Bad Prompt Reason #2: Unclear tags
The labels and sections aren't specific enough.
How to avoid it:
Use clear, everyday language to label your sections. Instead of "Data Parameters", say "Product Information to Extract". Instead of "Input", say "Website URLs to Process".
Bad Prompt Reason #3: Unstructured outputs & missing examples
The data format and expected output examples aren't clearly defined.
How to avoid it:
Always include both the data structure (using numbered lists) and 2-3 specific examples of desired outputs.
Example:
1. Product Name: Nike Air Max; Adidas Ultraboost; Puma RS-X;
2. Price: $120; $180; $110;
3. Color: Black; Cloud White; High Risk Red;
4. Size: US 9; US 10.5; US 8;
5. Material: Mesh & Synthetic; Primeknit; Leather & Mesh;
Bad Prompt Reason #4: No clear sequence
The actions aren't in a logical order.
How to avoid it:
Write your steps like you're giving directions to a friend - "First do this, then do that".
Example:
1. Open the website;
2. Find the product section;
3. Extract the data
The 3 Top Mistakes People Make With AI Agents
Since I joined Datablist.com, I've had many calls with customers who wanted to use the AI agent. Throughout these conversations, I've noticed a pattern of mistakes that people tend to make — so make sure you don't make these mistakes
-
Not wanting to prompt: Some people think the AI will understand what they think without them telling the AI what they're thinking. They want to save time, but investing time in saving time sounds kinda ridiculous to them – don’t be like them.
-
Being too vague with instructions: Instead of saying "get product information," specify exactly what data points you need (e.g., "extract the product name, price in USD, and available sizes from the product details section")
-
Overlooking error handling: Your prompt should include instructions for handling missing data or unexpected scenarios (e.g., "if price is not available, mark as 'N/A'")
How Much Time Should You Invest in Prompt Writing
You should think about the value you're receiving from this workflow and how much time it will save you.
I personally spend a lot of time writing prompts for me and to help our users.
When I use Grok, I don't spend 30 minutes writing a prompt — instead, I just make the first ask and iterate since I don't ask it super sophisticated things. But when I'm writing a prompt for email personalization or data cleaning, I take time for it since this saves me time finding issues, iterating on the prompt, and trying again later.
Harrison Chase, Founder of LangChain: "I don’t think we’ve kind of nailed the right way to interact with these agent applications. I think a human in the loop is kind of still necessary because they’re not super reliable. But if it’s in the loop too much, then it’s not actually doing that much useful thing. So, there’s kind of like a weird balance there.”
My AI Agent Prompting Template to Edit (With Examples)
Template for you to edit, with two attached examples below: Make sure that you take the time to edit it based on your use case
P.S. It took me hours to come up with this prompt formula, and you can have it for free ;)
My Goal: Your goal with the context
===
I want you to:
- First step of the task e.g., do a Google search
- Second step of the task e.g., extract information
Information to extract (With examples):
- Output 1 (Example 1, Example 2)
- Output 2 (Example 1, Example 2)
- Output 3 (Example 1, Example 2)
How to handle data inconsistencies:
- Anticipate errors and inconsistencies in the outputs or on the site itself and tell it how to deal with it
Mistakes to avoid:
- Here you should mention some mistakes that an intern would make on his first day
Name your input here: and use a / to refer to a column in your collection
Here’s my framework applied to an e-commerce scraping example:
My Goal: I have a list of e-commerce sites from which I need specific product information.
===
I want you to:
- Visit each site that I am going to give you the link for
- Extract information for each product which I'll tell you more about in a moment
Here's the information I am looking for (With examples):
- Name of the Product (Traveler XP 300)
- Original Price of the product in the displayed currency ($30; €10)
- Product category (Travel backpack; Business bag)
- Product specification 1 (30L Capacity; 17.4×14.3×8)
- Product specification 2 (Black; 11 Compartments)
- Special Tags (Last chance; Limited offer)
How to handle data inconsistencies:
- Return only one piece of information for each type
- Return "N/A" if the data isn't available
Mistakes to avoid:
- Don't return anything that doesn't fall in the mentioned data types e.g. call to action, reviews, etc.
- Not all pages are structured in the same way but the products are all labeled well enough that you should be able to recognize the distinctions between the data points.
Here are the sites to scrape the products from: /Category Pages
Here’s my framework applied to a media research example:
My Goal: I want to find recent press mentions of specific people based on their name and company name
===
Task Instructions:
- Search Google for news/press articles for mentions of each person
- Visit sites that mention the person of interest
- Extract relevant information about their mentions as detailed below
Information to Extract (With Examples):
- Article Title ("John Smith Launches New Tech Startup")
- Publication Name (The Tech Times)
- Publication Date (2025-05-20)
- Type of Mention (Interview, News Coverage, Press Release)
Data Handling Rules:
- Extract only the most recent mention within the last 3 months
- Return "No recent mentions" if no press coverage is found in the last 3 months
Mistakes to Avoid:
- Don't include social media mentions ignore them instead
- Don't confuse similarly named individuals that's why you should always match "full name + company name"
- Skip sponsored content or advertisements
- Verify the publication date is within the specified timeframe
Full name of the person: /People to Research
===Company name of the person: /company name
Frequently Asked Questions About Writing AI Prompts
1. Do I really need to spend time writing prompts? Can't I just ask the AI what I want?
While it might seem time-consuming, investing time in writing clear prompts is crucial for getting accurate results. Vague instructions lead to vague outputs, costing you more time in the long run with iterations and corrections.
2. What makes a prompt "good" versus "bad"?
A good prompt has clear structure, specific instructions, examples of desired outputs, and error handling guidelines. Bad prompts are vague, unstructured, lack examples, and don't account for potential errors or inconsistencies.
3. Is there a simple template I can follow for writing prompts?
Yes, the article provides a template with five main sections: Goal, Task Instructions, Information to Extract, Data Handling Rules, and Mistakes to Avoid. Each section should be clearly separated with "===
"
4. How detailed should my examples be in the prompt?
Examples should be specific and cover different scenarios you might encounter. For instance, when extracting product information, include examples of different formats, currencies, or measurements you expect to see.
5. What's the most important part of a prompt that I shouldn't skip?
Error handling and "Mistakes to Avoid" sections are crucial. They prevent common issues and ensure consistent outputs even when dealing with unexpected data or scenarios.