🎯 Google Maps B2B Lead Generation Scraper

A high-performance, production-ready Google Maps scraper and lead enrichment pipeline engineered for B2B sales teams, agencies, and marketers. It extracts comprehensive business profiles from Google Maps and automatically visits business websites to discover direct email addresses, social media links (LinkedIn, Instagram, Facebook, Twitter/X, TikTok), and contact pages.

🌟 Key Features

• Google Maps Data Extraction:
  • Business Name & Primary Category
  • Star Rating & Review Counts
  • Verified Phone Numbers
  • Full Addresses (Street, City, State, Postal Code, Country)
  • Official Website URLs
  • Operating Hours & Current Status (e.g. Open ⋅ Closes 9 PM)
  • Geographic Coordinates (Latitude & Longitude)
  • Claimed / Verified Business Status
• Automatic Website Lead Enrichment:
  • Asynchronously crawls discovered websites to find direct business emails (info@, contact@, sales@, etc.).
  • Extracts active social media profiles (LinkedIn, Instagram, Facebook, Twitter/X, YouTube, TikTok).
  • Automatically identifies dedicated /contact, /contact-us, and /about pages.
• Deduplication Engine:
  • Automatically removes duplicates across multiple search queries and regional keywords.
• Multiple Export Formats:
  • CSV: Standard UTF-8 BOM encoding ready for cold outreach tools (Instantly, Apollo, Lemlist, Smartlead, HubSpot).
  • Excel (.xlsx): Styled headers, auto-adjusted column widths, and clean formatting.
  • JSON / JSONL: Structured data for backend developer pipelines.
• Two User Interfaces:
  • CLI Tool: Real-time terminal progress bars, colored output tables, and customizable parameters.
  • Web UI Dashboard: Interactive Streamlit application to search, preview tables, and download leads with 1 click.

🚀 Quick Start

1. Installation

1. Clone or navigate to the repository:

   cd Scrapper
   

2. Create and activate a virtual environment:

   python3 -m venv .venv
   source .venv/bin/activate   # On Windows: .venv\Scripts\activate
   

3. Install dependencies and Playwright browser:

   pip install -r requirements.txt
   playwright install chromium
   

💻 Usage

Option A: Command Line Interface (CLI)

1. Single Query Search

Scrape the top 20 dentists in Austin, TX and enrich with emails/socials:

python main.py -q "Dentists in Austin, TX" -l 20 --enrich -o outputs/austin_dentists.csv

2. Batch Search Queries from a File

Scrape multiple queries specified in a file (e.g. sample_queries.txt):

python main.py -f sample_queries.txt --limit 50 --format xlsx -o outputs/b2b_leads.xlsx

3. CLI Command Options:

4. Examples with Proxies & Bit Solver

# Scrape with single proxy and Bit Solver (CaptchaSonic)
python main.py -q "Real Estate in Miami, FL" -l 30 --proxy "http://user:pass@proxy.example.com:8000" --captcha-solver

# Scrape with rotating proxy file and NopeCHA solver
python main.py -f sample_queries.txt --proxy-file proxies.txt --captcha-solver --solver-ext nopecha

Option B: Interactive Web UI Dashboard (Streamlit)

Launch the interactive web dashboard:

streamlit run app.py

Then open your browser at http://localhost:8501. You can:

1. Enter search queries or paste a list of queries.
2. Toggle Proxy and Bit Solver (Captcha) settings in the sidebar.
3. Configure limit sliders and enrichment settings.
4. Watch the real-time scraping progress.
5. Preview the interactive lead data table.
6. Download leads directly as CSV, Excel (.xlsx), or JSON.

📁 Project Structure

Scrapper/
├── main.py                     # CLI entry point
├── app.py                      # Streamlit interactive web dashboard
├── sample_queries.txt          # Ready-to-use sample queries
├── requirements.txt            # Python dependencies
├── src/
│   ├── __init__.py
│   ├── config.py               # Constants, User-Agents, contact regex patterns
│   ├── models.py               # Pydantic schemas (BusinessLead, ScrapeConfig)
│   ├── maps_scraper.py         # Async Playwright Google Maps scraper
│   ├── enricher.py             # Async website crawler for emails & socials
│   ├── exporter.py             # CSV, XLSX, JSON exporters
│   └── utils.py                # URL cleaning, coordinates parsing, deduplication
└── tests/
    ├── __init__.py
    ├── test_enricher.py        # Email/social regex & utility tests
    └── test_models_exporter.py # Data models & file export tests

🧪 Running Tests

Run the test suite with pytest:

pytest -v tests/

📄 Output Data Schema

The generated files include the following fields: