From Idea to Automation: Building a Serverless AI Lead Generation Agent (And the Bugs I Fixed Along the Way)

We’ve all been there. You have a great service, but finding the right people to tell about it is a constant, time-consuming grind. Lead generation is the lifeblood of any business, but it’s often a manual process of searching, filtering, and writing endless emails. I recently asked myself a question: “What if I could build an AI agent to do it for me?”

I set out with a few simple but firm rules: it had to be completely autonomous, run on a schedule, and use only free, browser-based tools. I’m not a DevOps expert, so I wanted to avoid complex local setups, command lines, or anything that couldn’t be managed from a web browser. I want to share my journey—the wins, the (many) bugs, and the final result. I hope my experience can help you build your own amazing automation tools.

The Grand Idea: What We Built

The goal was to create a “Set it and Forget It” AI agent that acts as a junior business development representative. Every day, it would:

Wake up on a schedule.
Read a profile of my company to understand what we do.
Pick a skill to focus on for the day (e.g., “AI and Machine Learning Solutions”).
Generate a smart Google search query to find companies that might need that skill.
Scrape the search results to find a contact email.
Draft a personalized, professional cold email.
Save the lead and the drafted email to a database, ensuring it never contacts the same company twice.

The Blueprint: Our “Serverless” and Free-Tier Architecture

The magic of this project is its architecture. It’s completely serverless and costs nothing to run, relying on the generous free tiers of some amazing platforms.

Code & Automation (GitHub & GitHub Actions): The entire project lives in a private GitHub repository. GitHub Actions provides the “engine,” a free virtual machine that runs our Python script on a schedule. This is the cron job that wakes the agent up.
Database (Supabase): A fantastic and easy-to-use PostgreSQL database for storing our leads and preventing duplicates.
The “Brain” (Google Gemini API): Google’s AI model handles the “smart” tasks: generating creative search queries and drafting personalized emails.
The “Hunter” (SerpApi): A simple and reliable API for performing the actual Google searches and returning a clean list of results.

The entire automation is controlled by a single workflow file in the repository. Here’s the snippet that tells the agent to run at 9 AM Dhaka time every day:

# .github/workflows/run_agent.yml
on:
  schedule:
    - cron: '0 3 * * *' # Runs at 3:00 AM UTC (9:00 AM GMT+6)

The Real Story: Challenges and Breakthroughs

Building something like this is never a straight line. The real learning happens when things break. Here are the biggest roadblocks I hit and how we solved them.

Challenge 1: The “404 Not Found” Mystery

My first attempt failed immediately. The agent couldn’t read the `company_profile.docx` file from the private GitHub repo. The log showed a `404 Not Found` error.

The Lesson: I was overthinking it! I was trying to download the file from the web, but the GitHub Actions workflow had already cloned the entire repository onto its virtual machine. The solution was much simpler: just read the file directly from the local disk. It was a great reminder that sometimes the easiest solution is the right one.

Challenge 2: The API Identity Crisis

Once the agent could read the profile, it failed at the search step with a `403 Forbidden` error. I was sure my API key was correct, but the server kept rejecting it. This was a frustrating one! After a lot of debugging, the breakthrough came from a simple code snippet. It turned out I was using a key from SerpApi (`serpapi.com`) but my code was written for a different service called Serper (`serper.dev`). They sound similar, but they’re completely different!

The Lesson: Always, always, always double-check that your code library matches the API documentation you’re reading. It’s a simple mistake that can cost hours of debugging.

Challenge 3: The Over-Eager AI Filter

My first successful run found leads, but some were low-quality (blogs, news articles, etc.). So, I tried to get clever. I built a “Qualifier” AI that would read the title of each search result and decide if it was a good lead. The result? It filtered out everything.

The Lesson: It’s easy to over-engineer an AI solution. My “smart” filter was too aggressive and had no nuance. I replaced it with a much simpler (and free) keyword filter that just skipped URLs with words like “blog,” “news,” or “jobs.” It was less “intelligent,” but far more effective.

The Final Engine: How It All Comes Together

After all the refinements, the core logic of the agent became a clean, efficient loop. Here’s a simplified look at the main function in `main.py` that orchestrates the whole process:

# A simplified view of the main loop

def main():
    # ... initialization ...
    
    # Generate a smart search query
    search_query = generate_search_query(company_profile, skill_of_the_day)
    
    # Get leads from SerpApi
    leads = search_leads_serpapi(search_query)
    
    leads_found_count = 0
    for lead in leads:
        # 1. Simple, reliable keyword filter
        if any(keyword in lead.get('url', '') for keyword in config.BAD_LEAD_KEYWORDS):
            continue

        # 2. Scrape the website for an email
        email = scrape_website(lead.get('url'))
        if not email:
            continue
            
        # 3. Check for duplicates in Supabase
        if is_duplicate(email):
            continue
            
        # 4. Clean the company name
        lead['company_name'] = clean_company_name(lead.get('company_name', ''), email)

        # 5. Draft the email with Gemini
        drafted_email_body = draft_email(company_profile, skill_of_the_day, lead)
        
        # 6. Save everything to the database!
        if drafted_email_body:
            save_lead(lead, email, drafted_email_body, lead.get('url'))
            leads_found_count += 1

Conclusion

This project was an incredible learning experience. It proved that you don’t need massive budgets or complex infrastructure to build powerful AI-powered automations. The entire agent runs reliably, finds quality leads, and hasn’t cost me a single penny.

The journey from a simple idea to a fully functional AI agent was all about iterative improvement: building, testing, seeing it fail, understanding why, and making it a little bit smarter each time. I hope sharing my process and my mistakes helps you on your own development journey.

What would you automate with an agent like this? Let me know in the comments!

From Idea to Automation: Building a Serverless AI Lead Generation Agent Plug & Play Product