ScrapeUp User Reference Guide

1. Welcome to ScrapeUp

ScrapeUp turns any public web page or PDF into data your application can use. You send a URL and a few options; ScrapeUp fetches the page through the right combination of proxy network, headless browser and unlocking tier, and returns raw HTML, clean Markdown or text, or structured JSON extracted by AI. One endpoint, one API key, metered in credits.

This guide is for three readers: developers integrating the API, analysts and operators using the Request Builder without writing code, and account owners managing keys, credits and billing. Each chapter stands on its own, so jump to what you need.

How a request flows

Your app sends one call; ScrapeUp authenticates it, takes the fetch path your options ask for, fetches the target, shapes the result into the format you asked for, and returns it.

Failed fetches are retried automatically and never billed; you pay only when ScrapeUp delivers content (a 2xx response, or a 404 that confirms the page does not exist).

2. Getting started in five minutes

Your first successful request needs an account, a verified email, and one API key. Everything below happens at scrapeup.com.

1. Start the free trial from scrapeup.com/pricing: tick the Acceptable Use Policy box and choose Start Free Trial. The trial runs 30 days on the Lite Basic plan (25,000 credits) and needs a card on file; you are not billed until the trial ends, and you can cancel from the dashboard at any time.
2. Confirm your email. Click the link in the welcome email. Until you do, every API request is refused with 403.
3. Generate an API key. Open the Dashboard, then the Usage tab, and click Generate Key. Copy it with the copy icon; you can create more keys later to separate projects or environments.
4. Send a test request. Replace YOURAPIKEY and run:

curl "https://api.scrapeup.com/?api_key=YOURAPIKEY&url=https%3A%2F%2Fhttpbin.org%2Fip"

You should get back the page body (here, a small JSON object showing the proxy IP the request exited from). That call costs 1 credit.

5. Try the same call in Python or Node. The target URL goes in url, URL-encoded if you build the query string yourself; client libraries encode it for you.

import requests
      
      response = requests.get(
          "https://api.scrapeup.com/",
          params={"api_key": "YOURAPIKEY", "url": "https://example.com"},
          timeout=70,
      )
      print(response.text)

const axios = require('axios');
      
      const { data } = await axios.get('https://api.scrapeup.com/', {
        params: { api_key: 'YOURAPIKEY', url: 'https://example.com' },
        timeout: 70000,
      });
      console.log(data);

Three things to set before you scale

• A client timeout that matches the path. A standard request can take up to 60 seconds, a rendered one up to 90 (plus up to 90 more with lazy loading), and a Web Unlocker request up to about three minutes. Set at least 70 seconds for standard requests, 100 for rendered ones and 200 for Web Unlocker; a shorter timeout abandons requests that would have succeeded.
• Keep your key out of client-side code. Anyone with the key can spend your credits. Store it as an environment variable and call the API from your server.
• Use HTTPS to our endpoint.https://api.scrapeup.com keeps your key and the returned content encrypted in transit.

No code? Use the Request Builder

If you would rather not write a request by hand, open scrapeup.com/try. Paste a URL, switch on the options the site needs, and copy a working cURL, Python or Node snippet with its credit cost shown live. No account is needed to build the request; you add your key when you run it. The same builder lives in the Dashboard with your keys pre-filled and a Run button that fires the request for real (chapter 8).

3. Core concepts: credits, plans and what a request costs

Every plan buys a monthly allowance of credits and a concurrent-request allowance. A request's cost is set by the most expensive fetch capability it used, plus a small add-on if you asked for Markdown, text or AI extraction. You are only charged when ScrapeUp delivers content.

Base cost: the highest tier you used wins

Add-ons stack on top of the base

Worked examples: a plain page with extract costs 1 + 5 = 6 credits; a rendered page with extract_model=precision costs 10 + 10 = 20; a residential, rendered page with output=markdown costs 40 + 5 = 45. When both an output conversion and extraction are requested, only the extraction add-on is charged.

When you are charged, and when you are not

• Charged: the target answered with a 2xx status, or a 404. A 404 is a real answer (the page does not exist) that ScrapeUp still paid a proxy to confirm.
• Not charged: any other outcome. Timeouts, 403 blocks, 5xx errors from the target, responses over the size limit, and requests refused for bad parameters are free, and most are retried once automatically.
• Partial: if the fetch succeeds but AI extraction fails, you pay the base fetch only and receive the raw HTML so nothing is lost.

Plans

Current prices are always on scrapeup.com/pricing. Credits reset on your billing date. Plan changes take effect immediately and are prorated; the Dashboard shows the exact proration before you confirm. Accounts without an active subscription (including trials that ended without converting) cannot scrape until a plan is chosen. Larger volumes, custom concurrency and custom terms are available through scrapeup.com/enterprise.

Two limits you will meet first

• Monthly credit cap. When your allowance is used up, requests return 403 REQUEST_LIMIT_EXCEEDED until the cycle resets or you upgrade.
• Concurrency cap. Requests above your plan's concurrent limit return 429 CONCURRENCY_LIMIT_EXCEEDED. Queue work on your side so that no more than your limit is in flight at once; a request counts as in flight until its response is delivered, which for rendered pages can be tens of seconds.

4. The Scrape API

The whole product is one endpoint, https://api.scrapeup.com/, plus two helpers for stored results and account status. Pass parameters in the query string for GET, or as a JSON body for POST and PUT.

Authentication

Send your key as api_key, in the query string or in the JSON body. All keys on one account share the same credit pool and concurrent-request allowance.

Parameter reference

All parameters work on GET (query string) and POST/PUT (body fields) unless noted.

Response shapes

A plain scrape returns the target's body as-is with its status code; the response headers carry the target's headers and cookies. Any request that asks for extraction or conversion returns JSON:

{
        "url": "https://example.com/company/acme",
        "status": 200,
        "extracted": { "company_name": "Acme Corp", "industry": "Industrial equipment" },
        "extraction_model": "fast",
        "credits_used": 6
      }

Errors return { "error": "CODE", "message": "…", "status": 4xx }; the full list is in chapter 10.

Sending data to the target

curl -X POST https://api.scrapeup.com/ \
        -H "Content-Type: application/json" \
        -d '{
          "api_key": "YOURAPIKEY",
          "url": "https://httpbin.org/anything",
          "method": "POST",
          "json": {"query": "industrial sensors", "page": 2}
        }'

Use form instead of json for URL-encoded forms. For any other content type (XML, SOAP, multipart), set your own Content-Type header and keep_headers: true; the body you place in json is forwarded verbatim.

Sessions and custom headers

Some flows need several requests to look like one visitor: a login followed by a protected page, a multi-step form, a paginated search. Give every request in the flow the same session_number and they exit through the same IP. Pair it with keep_headers=true to carry the cookies you received back to the next call.

Stored results and account status

GET /query?id=… returns a stored result later (its status, URL, headers and HTML), which is how you collect a PUT job or re-read a page without paying for a second fetch. GET /account?api_key=… returns requestCount, failedRequestCount, requestLimit and concurrencyLimit for the current cycle, plus lifetime totals; use it for your own alerting before you hit a cap.

5. AI extraction: fields in, JSON out

AI extraction replaces CSS selectors, XPath and regex with a description of what you want. ScrapeUp fetches the page, strips navigation, scripts and hidden elements, and hands the main content to a language model that returns JSON in the shape you asked for. It works on any page, any layout, and keeps working when the site redesigns.

Two modes

Schema mode (extract) gives you control of the output structure. Keys become the JSON field names; values tell the model what to put there.

curl -X POST https://api.scrapeup.com/ \
        -H "Content-Type: application/json" \
        -d '{
          "api_key": "YOURAPIKEY",
          "url": "https://example.com/company/acme",
          "extract": {
            "company_name": "The legal or trading name of the company",
            "industry": "The industry the company operates in",
            "headquarters": "City and country of the headquarters",
            "employee_count": "Number of employees as an integer, null if not shown",
            "founded_year": "Year founded as a four-digit integer",
            "website": "The company website URL",
            "products": "Main products or services, as an array of short strings"
          }
        }'

Prompt mode (extract_prompt) lets the model choose the structure. Use it for exploratory pulls or when the page's own organisation is the right one.

{
        "api_key": "YOURAPIKEY",
        "url": "https://example.com/careers",
        "extract_prompt": "Extract every open position with its title, team, location and posting date",
        "extract_model": "balanced"
      }

The two are mutually exclusive; sending both returns 400. Send both in a POST body.

Choosing a model tier

A practical approach: prototype on precision to learn what a correct answer looks like, then step down to balanced or fast and compare on a sample before committing a large batch.

Writing descriptions that extract cleanly

• State the type. "as a number", "as an integer", "as an ISO 8601 date", "as an array of strings". Untyped descriptions return whatever is on the page, including symbols and thousands separators.
• Say what to do when it is missing. "null if not shown" prevents the model from guessing.
• One concept per field.city and country separately beat a single location.
• Use the page's own vocabulary when a site has a specific label ("Effective date", "Registration number", "510(k) number").
• For lists, describe the item. "Array of objects with title, location and url for every job on the page" returns a table; "jobs" alone may return a count or a sentence.
• Bound the scope. "Only the first filing", "only items in the Specifications table".

Reading the response

{
        "url": "https://example.com/company/acme",
        "status": 200,
        "extracted": {
          "company_name": "Acme Corp",
          "industry": "Industrial equipment",
          "headquarters": "Rotterdam, Netherlands",
          "employee_count": 1250,
          "founded_year": 1987,
          "website": "https://acme.example",
          "products": ["Pressure sensors", "Flow meters", "Calibration services"]
        },
        "extraction_model": "fast",
        "credits_used": 6
      }

If extraction fails, extracted is null, extraction_error says why, the raw html is included, and you are charged the fetch only. The usual causes are a page whose real content is built by JavaScript (add render=true), a block page instead of the real one (see chapter 6), or a page whose main content runs past the roughly 150,000 characters the model reads (narrow the URL to the specific page you need).

Combining extraction with other options

Extraction runs after the fetch, so it combines with everything: render=true for JavaScript pages, premium=true and country_code for localised content, session_number for pages behind a login flow. Add include_html: true while you are tuning a schema so you can see what the model saw. Extraction also applies to PDFs automatically (chapter 7).

When not to use AI extraction

If you only need readable content for your own model or search index, output=markdown (+5 credits) is deterministic and cheaper than extraction on long pages. And many sites publish structured schema.org JSON-LD inside the HTML; parsing it yourself from a 1-credit fetch is free and identical every time. A quick check: search the returned HTML for application/ld+json.

6. JavaScript rendering, proxies and reliable access

Start with the cheapest call that works and step up only when the site pushes back. Failed requests are never charged, so trying the 1-credit call first costs nothing but a few seconds. Three options cover almost every hard site, and they stack.

Match the symptom to the fix

Browser rendering

render=true loads the page in a headless Chrome with a coherent device persona, waits for network activity to settle, and returns the resulting DOM. Images, fonts and stylesheets are not downloaded, which keeps renders fast; scripts always run. Use it when the data is built client-side. A rendered request can take up to 90 seconds, so keep your client timeout at 100 seconds or more.

• lazyload=true (lazy_load in a POST body) scrolls to the bottom before capturing, for infinite-scroll pages. It can add up to 90 seconds.
• hash=%23/route navigates inside a single-page app after the shell loads.

Proxies and geotargeting

Every request exits through a datacenter proxy by default. country_code picks the exit country with a two-letter ISO code (us, gb, de, br, jp). If you omit it, ScrapeUp exits from the country implied by the target's domain when it has one (.co.uk, .com.co, .de); for generic domains, set it explicitly. Many sites block or redirect unexpected geographies, so pinning the exit country to the site's home market often clears a failure on its own.

premium=true switches to a residential IP, the single most effective fix when a site refuses datacenter traffic. Residential proxies support a different set of countries from datacenter ones. session_number keeps the same IP for a sequence of calls on either pool.

Web Unlocker

unlock=true routes the request through a managed unlocking service for pages that challenge automated traffic. It is the most expensive path, so use it only after premium has failed, and only for the pages that need it rather than for a whole crawl. It supports country_code and combines with render=true for JavaScript-built pages (75 credits). Web Unlocker requests can take up to about three minutes.

Advanced: in-page fetch and XHR capture

Some sites load their data from an internal JSON API that only answers a real browser session. The Dashboard's Request Builder exposes two options for this, under Advanced:

• In-page fetch (browser_fetch, with context_url) loads a context page in a real browser to establish the session, then runs your request as a fetch() from inside that page. Best-effort for JSON endpoints that need a browser session.
• Capture XHR (capture_url) loads the URL as a page and returns the body of the first in-page request whose URL contains the text you specify, instead of the page HTML. You get the site's own JSON, already structured, without AI extraction.

Both bill at the browser tier. They are experimental: validate on a sample before you depend on them.

7. Document and PDF extraction

Point extract or extract_prompt at a PDF URL and ScrapeUp returns structured JSON from the document; no extra parameter is needed. ScrapeUp detects the PDF content type, reads the document with native document understanding (tables, forms and layout preserved, OCR built in for scanned pages), and applies the same schema or prompt you would use on a web page.

curl -X POST https://api.scrapeup.com/ \
        -H "Content-Type: application/json" \
        -d '{
          "api_key": "YOURAPIKEY",
          "url": "https://example.com/filings/510k-summary.pdf",
          "extract": {
            "applicant": "Company that submitted the filing",
            "device_name": "Name of the medical device",
            "product_code": "The three-letter FDA product code",
            "decision": "The decision, e.g. Substantially Equivalent",
            "decision_date": "Decision date as YYYY-MM-DD",
            "predicate_devices": "510(k) numbers of the predicate devices, as an array"
          },
          "extract_model": "balanced"
        }'

{
        "url": "https://example.com/filings/510k-summary.pdf",
        "status": 200,
        "extracted": {
          "applicant": "Acme Medical Devices, Inc.",
          "device_name": "FlowSense Infusion Monitor",
          "product_code": "FRN",
          "decision": "Substantially Equivalent",
          "decision_date": "2026-03-14",
          "predicate_devices": ["K201122", "K193456"]
        },
        "extraction_model": "balanced",
        "credits_used": 9
      }

What to know before a large PDF batch

• PDFs are read with at least the balanced model. The fast model cannot read documents, so ScrapeUp reads a PDF with balanced even when you ask for fast. Ask for precision or ultra explicitly for dense filings and multi-column layouts.
• The size limit is about 8 MB, the same as any response. Larger documents are refused with 400 and are not charged; split them or link to a per-section PDF where the publisher offers one.
• Typical uses: regulatory filings and public records, financial and annual reports, invoices and purchase orders, research papers, procurement notices, technical spec sheets.
• Describe table fields as arrays. For a document with a table, ask for "an array of rows, each with line_item, quantity and amount" rather than a single table field.

8. The Dashboard

The Dashboard at scrapeup.com/dashboard is where you manage keys, watch usage, build requests and handle billing. Sign in with the account you created.

Usage: keys and the request log

The top of the page shows credits used against your monthly allowance and failed requests, which are never charged. Below it, Generate Key creates a new API key; each key tracks its own request and failure counts so you can tell projects apart, while all keys draw from the same credit pool.

The request log keeps your most recent 500 requests with the URL, response time, credit cost, status and timestamp. Use it to confirm what a request cost before you run a batch, to spot a target that has started failing, and to find the request id you need for /query.

Request Builder

Paste a URL and switch on what the site needs: render, lazy load, residential proxy, country, Web Unlocker, output format, extraction schema or prompt, and model tier. The cost updates as you change settings, and the generated cURL, Python or Node snippet is always valid against the API. Two buttons matter: Copy is free; Run sends a live request with the selected key and charges your account, then shows the response as a rendered preview or raw HTML. AI Assist suggests settings for the URL you pasted; apply them with one click and adjust from there.

The Advanced panel holds in-page fetch, XHR capture, a delivery location (city, latitude, longitude) for location-gated sites, and a session warm-up URL that establishes a session before the real request. These are described in chapter 6.

Plans & Billing

Upgrades and downgrades apply immediately, and the page shows the prorated amount before you confirm. Invoices are listed with a Pay action for any that failed, the card on file can be replaced or removed, and a cancellation takes effect at the end of the current cycle. If a payment fails, your access continues while the card is retried; update the card to avoid interruption.

9. Best practices and cost optimisation

Most accounts can cut their credit spend by half or more with four habits: test cheap first, pin the country, extract only where you need to, and reuse what you already fetched.

Spend fewer credits per page

1. Prove the 1-credit call fails before you step up. Run a handful of URLs from a new site through the plain request and inspect the HTML. Move to render only if the content is missing, to premium only if you are refused, to unlock only if premium is refused too.
2. Set country_code to the site's home market. It is free and removes a large share of refusals and redirects on its own.
3. Do not render what you can fetch. Many sites ship the data you want in the initial HTML, often as schema.org JSON-LD or an inline JSON state object, even when the visible page is built with JavaScript. Search the 1-credit HTML for your target values before paying 10.
4. Capture the site's own API instead of rendering repeatedly. When a page loads a JSON endpoint, XHR capture (chapter 6) returns that JSON directly; one browser-tier request can replace a render plus an extraction.
5. Choose the extraction tier per site, not per account. Prototype on precision, then compare fast on a sample; keep the cheapest tier that matches.
6. Convert, do not extract, when you only need text.output=markdown is +5 credits and deterministic; use AI extraction for fields, not for whole-page content.
7. Reuse results with /query. A stored result is free to read again. Keep the request id and avoid re-fetching the same page for a second extraction pass.
8. Use unlock surgically. Reserve Web Unlocker for the specific pages that need it; the listing pages around them often do not.

Throughput and reliability

• Run at your concurrency limit, not above it. Keep a queue on your side sized to your plan's concurrent requests, and treat 429 as back-pressure: wait, then retry, rather than retrying immediately.
• Let ScrapeUp do the first retry. A failed fetch is retried once automatically when there is time left in the request. Add your own retry with backoff only for 500 REQUEST_TIME_OUT and 429, and step up the fetch path (chapter 6) if the same URL keeps failing.
• Keep sessions short. Use session_number for the handful of requests that must share an IP, then drop it; long-lived sessions concentrate traffic on one IP and get refused sooner.
• Narrow the URL. Fetch the specific record, listing or document page rather than a hub page and extracting from it; the fetch is faster and the model reads less.
• Check /account before large batches and alert yourself at, say, 80% of your allowance so a job does not stop mid-run on a 403.

Keep your integration maintainable

• Keep API keys in environment variables, one key per environment, and rotate a key from the Usage tab if it leaks.
• Log credits_used and the request id with every response; both make billing questions and support requests quick to resolve.
• Store the raw HTML (or set include_html) during schema development so you can re-run extraction prompts offline against the same page.
• Pin your extraction schemas in version control. A schema is your contract with the page; when a site redesigns, you change descriptions, not selectors.

10. Error reference and troubleshooting

Every error is JSON with an error code, a message and the HTTP status:

{ "error": "INVALID_URL", "message": "Invalid url", "status": 400 }

Codes in the 400 range mean the request itself needs changing; 401 is the key; 402, 403 and 429 are your account or plan; 500 means the fetch did not succeed, even after the automatic retry. No error response is ever charged.

Symptoms that are not error codes

Before you contact support

Have the request id (from the Usage log), the exact parameters, the timestamp, and the response body. Support can trace any request by its id and tell you which fetch path ran and what the target returned. Write to contact@scrapeup.com or use the contact form at scrapeup.com/contact.

11. Limits, acceptable use and support

Service limits at a glance

Acceptable use

ScrapeUp is built for lawful, high-volume collection of publicly accessible data: business listings and public records, search results, regulatory and financial documents, research, and dataset building from public sources. You are responsible for making sure your use complies with the law and with any terms you have agreed to with the sites you target.

The Service may not be used to infringe copyright, break the law, access illegal or exploitative content, defeat authentication, paywalls or access controls, harvest personal data in breach of privacy law, support spam or attacks, or resell raw access and share credentials across unrelated parties. ScrapeUp may block specific targets or patterns that create legal, infrastructure or reputational risk, and violations can lead to throttling, suspension or termination. The full policy is at scrapeup.com/acceptable-use; you accept it when you sign up.

Data handling

Request results are stored so that /query can return them and the Dashboard can show your history. Your API keys are shown only to you; rotate one from the Usage tab if it is exposed. To delete your account, write to support: deletion removes your profile, keys and stored results, and we confirm it in writing. Security and data-handling details for a procurement or security review are available through scrapeup.com/enterprise.

Getting help

When you write in, include the request id and the parameters you sent; every request can be traced by its id.