Custom Cookies and Headers

Pass custom cookies and HTTP headers alongside your scraping requests to access authenticated, personalized, or session-dependent content. This is essential when scraping pages that require login sessions, specific site configurations, or custom API tokens.

Cookies and headers are provided as key/value pairs - either as a dictionary object (POST) or a semicolon-separated string (GET).

When To Use This

Authenticated pages - Pass session cookies or authentication tokens to access content behind login walls.
Captcha clearance reuse - Forward captcha clearance cookies from a previous successful request to skip re-verification. Alternatively, use Sessions to handle this automatically.
Site configuration - Some websites use cookies to control language, currency, or content preferences (e.g., locale=en-US).
API key pass-through - Include API keys or authorization headers when scraping third-party APIs through ScrAPI.
User-agent override - Set a specific User-Agent header if the target site requires a particular browser signature.

Anti-fingerprinting note: When scraping without a browser, ScrAPI automatically sends realistic headers to pass fingerprinting checks. Your custom headers are merged with these defaults - they won’t replace the anti-detection headers unless you explicitly set the same header key.

The JSON response includes cookies and headers returned by the target website. You can capture these from a first request and forward them in subsequent requests - useful for maintaining authentication state, following redirects, or continuing multi-step workflows.

For automatic cookie and IP persistence, consider using Sessions instead of manually managing cookies.

Example Request

GET https://api.scrapi.tech/v1/scrape?url=http://httpbin.org/cookies&cookies=cookie1=value1;cookie2=value2

GET https://api.scrapi.tech/v1/scrape?url=http://httpbin.org/headers&headers=header1=value1;header2=value2

POST https://api.scrapi.tech/v1/scrape

With Cookies

{
  "url": "http://httpbin.org/cookies",
  "cookies": {
    "My-Cookie": "Test 1",
    "Session-ID": "Test 2"
  }
}

With Headers

{
  "url": "http://httpbin.org/headers",
  "headers": {
    "My-Header": "Test Header 1",
    "X-SITE-KEY": "Test Header 2"
  }
}

Combined Cookies and Headers

{
  "url": "https://example.com/dashboard",
  "cookies": {
    "session_token": "abc123",
    "preferences": "lang=en"
  },
  "headers": {
    "Authorization": "Bearer eyJhbGciOiJIUzI1NiIs...",
    "X-Requested-With": "XMLHttpRequest"
  }
}

Sessions - Automatically persist cookies and IP across multiple requests without manual management.
Change Request Method - Combine custom headers with POST, PUT, or other HTTP methods for API scraping.
Captcha Solving - After solving a captcha, clearance cookies are included in the response for reuse.
Real Browser - Custom cookies and headers work with both browser and HTTP client modes.

Testing

Test this request in the Playground or view the OpenAPI specification.

Record Video Callback / Webhook