5-Minute Quick Start: Get Your First Structured Dataset with the AntsData API

What Is AntsData?

AntsData is a managed web data collection platform designed for developers and data engineering teams that need reliable access to structured data from public websites at scale. The platform handles the full infrastructure stack -- proxy rotation, anti-bot bypass, browser rendering, and concurrency management -- so that your team can focus on downstream data pipelines rather than scraping maintenance.

The platform exposes a unified REST API with multiple specialized endpoints. The core Web Unlocker API handles arbitrary web pages: it can scrape a single page, capture a screenshot, or crawl an entire site. On top of that, AntsData provides purpose-built scrapers for Google Search, Google Shopping, Amazon, Google Maps, and major social platforms including TikTok, Instagram, Facebook, YouTube, X/Twitter, and LinkedIn. Every endpoint returns structured JSON and supports both synchronous and asynchronous execution modes.

The pricing model is pay-as-you-go with no monthly commitment. Costs range from 1.2 to 2 CNY per 1,000 records depending on the endpoint, making it viable for both exploratory one-off pulls and production-grade data pipelines.

---

Prerequisites

Before you begin, make sure you have the following in place:

1. An AntsData account and API key. You can sign up at the AntsData platform. The process takes under two minutes and requires only an email address. Once registered, generate an API key from your account dashboard.

2. curl available in your terminal, or any HTTP client. The examples below use curl, which is available on macOS, Linux, and Windows (via PowerShell or WSL). You can also use any programming language's HTTP library -- Python requests, Node.js fetch, Go net/http, or similar.

3. A target URL to scrape. For the walkthrough, we will use a public web page. You can substitute any URL you need data from.

---

Step 1: Get Your API Key

Navigate to the AntsData platform and create an account. After confirming your email, log in and go to the API console in your dashboard. Click "Generate API Key" to create a new bearer token.

Store the key as an environment variable so that it is available to all subsequent commands without being hardcoded:

export ANTSDATA_API_KEY="your_api_key_here"

This key authenticates every API request you make. Keep it secure and do not commit it to source control. The examples below all reference the $ANTSDATA_API_KEY environment variable.

AntsData operates on a prepaid credit system. Top up your account balance before making requests. The minimum top-up is low enough for testing purposes, and you are only charged for successful requests.

---

Step 2: Make Your First API Call

The simplest way to get data from AntsData is the Web Unlocker scrape endpoint. It accepts a URL and returns the page content in your choice of formats -- HTML, plain text, markdown, or extracted JSON.

Here is a single curl command that scrapes a public page and returns the content as markdown:

curl -X POST "https://api.antsdata.com/v1/scrape" \
  -H "Authorization: Bearer $ANTSDATA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "format": "markdown",
    "render": true
  }'

The render parameter tells AntsData to use a headless browser to execute JavaScript before extracting content. This is necessary for any modern web application that loads data dynamically. Set it to false for simple static pages to reduce latency.

The format parameter controls the output format. Use html for the raw DOM, text for stripped plain text, markdown for clean markdown output, or json when you need structured key-value extraction.

Run this command and you will receive a JSON response within a few seconds containing the scraped page content. That is your first structured dataset from AntsData.

---

Step 3: Parse the Structured Response

The response from the scrape endpoint is a JSON object with a predictable schema. Understanding this schema is important because it determines how you build your downstream parsing logic.

A typical scrape response includes the following fields:

• url — The final URL after any redirects were followed.
• statusCode — The HTTP status code of the target page (200, 404, etc.).
• title — The page title extracted from the HTML <title> tag.
• content — The primary page content in your requested format.
• html — The full rendered HTML of the page, always available regardless of your chosen format.
• markdown — The page content converted to clean markdown.
• text — Plain text with all markup stripped.
• headers — An object containing the HTTP response headers from the target server.
• fetchMs — The time taken to fetch and render the page, in milliseconds.
• scrapedAt — An ISO 8601 timestamp of when the scrape was executed.

The key design principle is that the response always includes the same top-level fields regardless of the target website. Your parsing logic does not need to adapt to different page structures at the metadata level -- statusCode, title, fetchMs, and scrapedAt are always present. The content field contains your requested format, while html is always available if you need to run your own extraction on the raw DOM.

For production use, always check statusCode before processing content. A non-200 status indicates that the target page returned an error, was blocked, or is temporarily unavailable.

---

Step 4: Try a Specialized Endpoint

The Web Unlocker scrape endpoint works for any arbitrary URL, but AntsData also provides specialized endpoints that return pre-structured data for specific platforms. These endpoints save you from writing custom parsers for each site.

Google Search Example

The google-search endpoint returns structured search results including organic listings, AI Overviews, image results, and pagination metadata. To call it, send a POST request to /v1/google-search with a JSON body containing your search query, the country code (gl, e.g. us, uk, de, jp), the interface language (hl), the number of results per page (num, up to 100), and the device type (desktop or mobile). You can also set a location parameter with a free-text location string for geo-specific results and use the start parameter for pagination offset.

The response includes totalResults (the estimated total match count), organicResults (an array where each entry contains position, title, url, snippet, and domain), aiOverview (with AI-generated summary content), imageResults (an array of image search results), and pagination (with nextStart for fetching the next page).

Amazon Product Example

The amazon-product endpoint takes a standard Amazon product URL and returns structured product data. Send a POST request to /v1/amazon-product with the product url and optionally the mode parameter (sync or async).

The response returns fields including asin, title, brand, price, currency, bsr (Best Sellers Rank), bsrCategory, rating, reviewCount, availability, sellerName, features (bullet-point product descriptions), and images (an array of product image URLs). The full field reference is available in the AntsData API documentation.

Other Available Endpoints

Beyond Google and Amazon, AntsData provides scrapers for social platforms (TikTok, Instagram, Facebook, YouTube, X/Twitter, LinkedIn), Google Shopping, and Google Maps. Each returns structured JSON specific to the platform. The calling convention is identical across all endpoints -- a POST request with a JSON body and bearer token authentication.

---

Step 5: Scale Up with Async Mode

For production workloads, you will typically need to scrape hundreds or thousands of pages. Sending sequential synchronous requests is slow and does not take advantage of AntsData's concurrency infrastructure. The async mode is designed for this scenario.

In async mode, you submit a batch of tasks and receive a task ID immediately. You then poll for results once processing is complete. This allows you to submit large batches without holding open HTTP connections.

The workflow is straightforward. For each URL in your batch, send a POST request to any endpoint with "mode": "async" added to the JSON body. Each submission returns a response containing a taskId field. Store these task IDs for polling.

To retrieve results, send a GET request to /v1/task/{taskId} with the same authorization header. The response includes a status field that will be pending, completed, or failed. When the status is completed, the response also contains a data field wrapping the standard endpoint output, along with taskId, status, and completedAt fields. When the status is failed, an error field describes the failure reason.

For polling, check each task at regular intervals (for example, every 2 to 5 seconds) until all tasks have reached a terminal state (completed or failed). In a production implementation, you would use a job queue or task scheduler to manage submissions and polling efficiently, with exponential backoff to avoid excessive polling calls.

The async mode is also available for specialized endpoints. For example, you can submit a batch of Amazon product URLs or Google Search queries in async mode using the same pattern. The task lifecycle is identical: submit, receive a taskId, poll until the status is completed or failed.

For very large batches, AntsData processes requests with high concurrency across distributed infrastructure. A typical throughput for the scrape endpoint is several hundred pages per minute depending on target complexity and rendering requirements. Specialized endpoints like Amazon and Google Search have their own throughput profiles based on the anti-bot complexity of each target platform.

---

Understanding the Response Format

Every AntsData endpoint returns structured JSON. The Web Unlocker scrape endpoint uses a consistent response schema. The table below lists all standard output fields.

Field	Type	Description
url	string	The final URL after redirects
statusCode	number	HTTP status code of the target page (200, 404, etc.)
title	string	Page title extracted from the <title> tag
content	string	Primary content in the requested format
html	string	Full rendered HTML of the page
markdown	string	Page content converted to clean markdown
text	string	Plain text with all markup stripped
headers	object	HTTP response headers from the target server
fetchMs	number	Time taken to fetch and render the page in milliseconds
scrapedAt	string	ISO 8601 timestamp of when the scrape was executed

For specialized endpoints, the response schema varies. The google-search endpoint returns totalResults, organicResults, imageResults, aiOverview, and pagination. The amazon-product endpoint returns product-specific fields like asin, title, price, bsr, and reviewCount. Each endpoint's response schema is documented in the AntsData API reference.

In async mode, the submission response contains a taskId and status field. When you poll the task endpoint after completion, the response wraps the standard endpoint output inside a data field alongside taskId, status, and completedAt.

---

Endpoint Overview

The table below lists all major endpoint categories available on AntsData with their descriptions and typical use cases.

Category	Endpoints	Description	Typical Use Case
Web Unlocker	scrape	Single-page scraping with optional JS rendering	General web data extraction, content monitoring
Web Unlocker	screenshot	Full-page or viewport screenshot capture	Visual QA, archived page evidence, design benchmarking
Web Unlocker	crawl	Multi-page site crawl following internal links	Full-site content migration, SEO audits, documentation extraction
Search	google-search	Google organic search results with AI Overview	SERP tracking, competitive SEO analysis, market research
Search	google-shopping	Google Shopping product listings	Product price comparison, marketplace monitoring
E-commerce	amazon-product	Amazon product detail page data	Price tracking, BSR monitoring, competitive intelligence
E-commerce	amazon-sellers	Amazon seller profile data	Seller verification, channel integrity checks
Maps	google-maps	Google Maps business listing data	Local SEO monitoring, competitor location analysis
Social	tiktok-*	TikTok profile, video, and product data	Social commerce tracking, influencer analysis
Social	instagram-*	Instagram profile and post data	Brand monitoring, engagement tracking
Social	facebook-*	Facebook page and ad data	Ad intelligence, competitor ad tracking
Social	youtube-*	YouTube video and channel data	Content performance analysis, keyword research
Social	x-twitter-*	X/Twitter post and profile data	Sentiment monitoring, trend tracking
Social	linkedin-*	LinkedIn profile and company data	B2B lead research, competitive hiring analysis

All endpoints support both synchronous (sync) and asynchronous (async) execution modes. Pricing ranges from 1.2 CNY per 1,000 records for the Web Unlocker scrape endpoint to 2 CNY per 1,000 records for specialized endpoints like amazon-product and social scrapers. Check the AntsData pricing page for the current rate for each endpoint.

---

Common Issues and How to Fix Them

401 Unauthorized on every request. Your API key is missing, expired, or incorrectly formatted. Verify that the Authorization header uses the format Bearer YOUR_KEY with a space between Bearer and the key. If you are using the environment variable, confirm it is set correctly by running echo $ANTSDATA_API_KEY in your terminal.

Empty or incomplete content in the response. The target page may rely heavily on JavaScript to render its content. Set the render parameter to true to enable headless browser rendering. You can also use the waitFor parameter to specify a CSS selector or a time delay (in milliseconds) to wait for specific elements to load before content extraction.

High fetchMs values or timeouts. Pages with heavy JavaScript, large images, or multiple third-party scripts take longer to render. Increase the timeout parameter (default is 30 seconds) for complex pages. Alternatively, set render to false if the page content you need is available in the initial HTML response without JavaScript execution.

Rate limiting or throttling on your side. If you are sending synchronous requests in a tight loop, you may hit your own client-side connection limits. Switch to async mode for batch workloads. Async mode lets AntsData handle concurrency internally and returns results when they are ready, without requiring you to manage connection pools or rate limiting logic.

Unexpected statusCode in the response (403, 429, 503). The target website may be experiencing issues or enforcing geographic restrictions. Try setting the country parameter to route the request through a different geographic location. If the issue persists, the target page may genuinely be down or blocking -- check the URL directly in a browser from the same target region.

Async task stuck in pending status. Tasks are processed in order and may take longer during high-traffic periods. If a task remains in pending for more than five minutes, contact AntsData support. For time-sensitive workloads, consider using sync mode with a reasonable timeout value.

---

FAQ

How long does it take to get data from the AntsData API?

For the Web Unlocker scrape endpoint in synchronous mode, most pages return within 2 to 10 seconds depending on rendering complexity and target server response time. Simple static pages without JavaScript rendering can return in under 1 second. Specialized endpoints like Google Search and Amazon Product typically complete within 3 to 8 seconds. In async mode, batch tasks are processed in parallel and individual results are available once processing completes -- throughput scales to hundreds of pages per minute depending on the endpoint and target complexity.

Can I use AntsData for real-time price monitoring?

Yes. The amazon-product endpoint returns real-time price data including the current selling price, list price, currency, discount percentage, and seller information. To build a price monitoring pipeline, schedule periodic sync calls for your target ASINs or use async mode for batch updates across large product sets. The BSR field updates in near real-time, allowing you to track ranking changes throughout the day. The pay-as-you-go pricing model at 2 CNY per 1,000 records makes frequent monitoring economically viable.

Does AntsData handle CAPTCHAs and anti-bot detection?

Yes. Anti-bot bypass is a core capability of the AntsData platform. The infrastructure includes proxy rotation across residential and datacenter IP ranges, browser fingerprint randomization, CAPTCHA solving, and JavaScript challenge resolution. You do not need to configure any of this -- it is handled automatically on every request. The Web Unlocker endpoints are designed to work against sites with aggressive anti-bot measures, including Cloudflare, Akamai, and custom bot detection systems.

What is the difference between sync and async modes?

In synchronous mode, you send a single request and wait for the response. The HTTP connection stays open until the scrape completes and the result is returned. This is the simplest approach and works well for individual page scrapes or low-volume requests. In asynchronous mode, you submit a task and receive a taskId immediately. The task is processed in the background, and you poll a separate endpoint to retrieve the result once it is ready. Async mode is designed for batch workloads where you need to process hundreds or thousands of URLs. It eliminates connection timeout issues and lets AntsData optimize concurrency across your entire batch.

How does AntsData pricing work?

AntsData uses a prepaid credit system measured in CNY. You top up your account balance and are charged per successful record returned. The Web Unlocker scrape endpoint costs 1.2 CNY per 1,000 records. Specialized endpoints such as Google Search, Google Shopping, Amazon Product, and social scrapers cost between 1.2 and 2 CNY per 1,000 records depending on the endpoint. There is no monthly subscription, no minimum commitment, and no charge for failed requests. You can monitor your usage and remaining balance in the AntsData dashboard.