Callback / Webhook

The callback feature lets you send scraping requests asynchronously. Instead of waiting for the response, ScrAPI immediately returns a reference ID and delivers the complete result to your specified URL via a POST request once processing is finished.

Set the callbackUrl parameter to the URL where you want to receive the results.

When To Use This

Long-running scrapes - Pages that take a long time to load (captcha solving, complex browser commands) benefit from asynchronous processing so your application doesn’t block.
Batch processing - Submit many scraping requests at once and process the results as they arrive at your webhook endpoint.
Decoupled architectures - Separate the system that initiates scrapes from the system that processes the results (e.g., a job scheduler triggering scrapes that feed into a data pipeline).
Avoiding timeouts - If your HTTP client has short timeout limits, callbacks let ScrAPI take as long as needed without your request timing out.
Queue-based workflows - Use the callback as a trigger for downstream processes (database writes, notifications, further scraping).

How It Works

You submit a scrape request with a callbackUrl.
ScrAPI responds immediately with HTTP 202 (Accepted) and a reference ID.
The scrape executes in the background.
Once complete, ScrAPI POSTs the full JSON response data to your callbackUrl.

Example Request

GET https://api.scrapi.tech/v1/scrape?url=https://deventerprise.com&callbackUrl=https://webhook-test.com/f661e0e4c08dc1cf2449bc9d507b2d8d

POST https://api.scrapi.tech/v1/scrape

{
  "url": "https://deventerprise.com",
  "callbackUrl": "https://webhook-test.com/f661e0e4c08dc1cf2449bc9d507b2d8d"
}

Immediate Response

When using callbackUrl, the API returns a 202 status with a reference for status tracking:

{
  "reference": "2c1e5c6d-d00d-47f1-bab0-de92ed2c81d5"
}

Checking Callback Status

You can poll the status of a callback request using the reference ID. The status is cached for 10 minutes after completion, after which it returns “Unknown”.

Status Endpoint

GET https://api.scrapi.tech/v1/scrape/status/{reference}?apiKey=00000000-0000-0000-0000-000000000000

Status Response

{
  "status": "Complete",
  "code": 4
}

Possible Statuses

Status	Code	Description
Unknown	0	Reference not found or expired (>10 minutes).
Scheduled	1	Request is queued and waiting to be processed.
Busy	2	Scrape is currently in progress.
Error	3	Scrape failed - check the callback payload for details.
Complete	4	Scrape finished and results have been delivered to your callback URL.

Tip: Status polling is optional. In most architectures, simply listen for the POST to your callback URL. Use status polling when the system initiating the scrape is separate from the webhook receiver, or to check for failures.

Testing Webhooks

Use these free services to test your webhook integration and inspect the POST payload:

Sessions - Combine callbacks with sessions for multi-step async scraping workflows.
Browser Commands - Browser commands work fully with async callbacks.
Captcha Solving - Callbacks are especially useful for captcha-heavy pages that take longer to process.

Testing

Test this request in the Playground or view the OpenAPI specification.

Cookies and Headers Sessions