Spicrawl
Web scraping MCP server: any URL to Markdown or JSON, with JavaScript rendering and batch jobs.
| Endpoint | https://mcp.spicrawl.com/mcp (streamable-http) |
|---|---|
| Registry version | 0.2.0 |
| Now | 27 tools · grade D (56) · pin ts_31dc640ca5a751c64ec5bfbbf37987f6 · protocol 2025-06-18 |
| First / last listed | 2026-11-15 01:28 UTC / 2026-11-15 01:28 UTC |
| Grade over time | 2026-11-15: D (56), 27 tools |
Is this your server? Own a server? ToolFit hosted watch re-checks it every 24 h ($0.25 per 30 days) or 6 h ($0.60) and sends you an HMAC-signed webhook when its tools or grade change: POST https://api.youraitools.co/toolfit/v1/watch. To opt out, see About & opt-out.
Tools: spicrawl_scrape spicrawl_batch_submit spicrawl_batch_list spicrawl_batch_status spicrawl_batch_results spicrawl_batch_task_content spicrawl_batch_cancel spicrawl_batch_retry spicrawl_batch_add_items spicrawl_batch_close spicrawl_session_create spicrawl_session_list spicrawl_session_get spicrawl_session_context spicrawl_session_release spicrawl_session_delete spicrawl_usage spicrawl_usage_summary spicrawl_usage_reconciliation spicrawl_requests_list spicrawl_request_get spicrawl_browser_connect_url spicrawl_browser_live_view spicrawl_browser_control spicrawl_docs_search spicrawl_docs_read spicrawl_docs_index
spicrawl_scrape | Retrieve a web page and return its content as markdown (default), text, HTML, a JSON envelope, or a printed PDF. Handles the fetch vs. browser decision, charset decoding, and PDF-to-text for you. Optionally extract structured data with CSS selectors (`extract`) or `autoparse`, return discovered `links`, take a `screenshot`, or scope the DOM with `include_tags`/`exclude_tags`. Each successful call params: url*, render, format, main_content_only, include_tags, exclude_tags, links, extract, screenshot, impersonate, proxy, cache, cache_ttl, wait, mode, method, custom_headers, session_id, proxy_verify, wait_for, wait_for_timeout, block_resources, actions, screenshot_fullpage, screenshot_selector, screenshot_format, screenshot_quality, autoparse, network_capture, parse_pdf, max_cost, original_status, allowed_status_codes |
spicrawl_batch_submit | Submit many URLs to be scraped as one asynchronous job, with shared settings applied to every URL. Give EITHER `urls` or `items`, not both (at least one URL, unless `open` is true). Returns the job object (id, status, estimated_credits, progress). Poll with `spicrawl_batch_status`, read with `spicrawl_batch_results` or `spicrawl_batch_task_content`. Use this instead of many `spicrawl_scrape` calls params: urls, items, render, format, main_content_only, wait, wait_for, block_resources, custom_headers, max_cost, name, open, concurrency, priority, max_attempts, failure_threshold, credit_budget, webhook_endpoint_id |
spicrawl_batch_list | List this API key's project's batch jobs, newest first, with status and progress. Returns `{ batches: [...], next_cursor? }`; pass `next_cursor` back as `cursor` for the next page. params: status, limit, cursor |
spicrawl_batch_status | Return one batch job with its status and progress (total/completed/succeeded/failed/remaining, credits charged). Poll this after `spicrawl_batch_submit`; once status is `completed`, `failed` or `cancelled`, read `spicrawl_batch_results`. params: job_id* |
spicrawl_batch_results | Return a page of FINISHED items of a batch job (one record per item: seq, url, external_id, status, error, and content, or a `result_url` API path (GET /v1/batch/{id}/tasks/{seq}/content, same key) for large bodies). Unfinished items are not listed yet. Returns `{ results: [...], next_cursor? }`; pass `next_cursor` back as `cursor` for the next page. Results expire after the job's retention window params: job_id*, status, limit, cursor |
spicrawl_batch_task_content | Return the full payload one item of a batch job produced, by its `seq` (items are numbered from 0 in submission order). Cheaper than paging results when you need one URL's output. A CONFLICT error means either the item has not finished yet (poll `spicrawl_batch_status` and ask again) or it finished as failed (it will never produce content; the message says which). NOT_FOUND means no such seq. params: job_id*, seq* |
spicrawl_batch_cancel | Cancel a batch job: items not yet started are cancelled and will never run; items already finished keep their results; items already running finish first, so the job comes back `cancelling` and moves to `cancelled` when they drain. Cannot be undone — resubmit to run the URLs again. Returns the job object. params: job_id* |
spicrawl_batch_retry | Re-queue every FAILED item of a batch job so it runs again (succeeded items are untouched; a job with no failures is returned unchanged). Returns `{ job, items_reset, items_dispatched }`. params: job_id* |
spicrawl_batch_add_items | Append URLs to a batch job that was submitted with `open: true`. Same item vocabulary as `spicrawl_batch_submit` (EITHER `urls` or `items`, not both, plus shared settings for these new items). Fails with CONFLICT if the job is closed or finished. Not idempotent: calling twice adds the URLs twice. Returns `{ job, items_added, items_dispatched }`. params: job_id*, urls, items, render, format, main_content_only, wait, wait_for, block_resources, custom_headers, max_cost |
spicrawl_batch_close | Stop an open batch job accepting items, so it completes once its queued work drains. Idempotent (closing a closed job succeeds). Returns the job object. params: job_id* |
spicrawl_session_create | Create a session: a persistent browser identity (cookie jar, localStorage, a fixed rendering tier and, by default, a sticky exit IP) that survives across scrapes. Use it for multi-step flows — log in, then fetch pages behind the login — by passing the returned `id` as `session_id` to every `spicrawl_scrape` call that should share the state. The rendering tier is fixed for the session's lifetime an params: engine, ttl_seconds, domain_scores, session_context |
spicrawl_session_list | List this API key's project's sessions, newest first. Returns `{ sessions: [...], next_cursor? }`; pass `next_cursor` back as `cursor` for the next page. Never includes the stored credentials. params: status, engine, limit, cursor |
spicrawl_session_get | Return one session's metadata: status, `engine` (its rendering tier), proxy/exit, usage count, expiry, lease holder, and a summary of its stored context. Never returns the credentials themselves (use spicrawl_session_context for that). params: session_id* |
spicrawl_session_context | Return the full stored state of a LIVE session — cookies, localStorage and other credentials — plus its fingerprint and domain scores. The `session_context` object can be passed to spicrawl_session_create to clone the session. The result contains secrets (login cookies): do not echo it to the user or logs unless asked. params: session_id* |
spicrawl_session_release | End a session when you are done with it: frees its exit IP and browser state; it can no longer be used with spicrawl_scrape. Its stored cookies and storage are purged and cannot be recovered; start a new session and log in again to continue. The record stays visible (status `released`) so its history can still be inspected. Refused while a scrape holds the session's lease unless `force` is true. R params: session_id*, force |
spicrawl_session_delete | Permanently delete a session and its stored state, including the record itself (prefer spicrawl_session_release if you only want to stop using it). Idempotent: deleting an already-deleted session succeeds. Refused while a scrape holds the session's lease unless `force` is true. params: session_id*, force |
spicrawl_usage | Billing-grade usage for your org over a date window, from the daily rollup (usage_daily), grouped by day, project, engine, feature or key. group_by=key returns per API key (key_id, name, prefix, project_id) counters per metric per day, from usage_key_daily; keyless (dashboard) usage is not in it, and a project-scoped key sees only its own project's keys. Use it to answer 'how many requests/credits params: from, to, group_by, project_id, metrics |
spicrawl_usage_summary | A compact current-period summary of your org's usage (totals as of now, from the daily rollup). No parameters. Use for a quick 'how much have I used so far'; use spicrawl_usage for a windowed breakdown. Requires the `read` scope. params: none |
spicrawl_usage_reconciliation | Drift report for ONE day: re-derives usage from that day's request log and compares it with the billed daily rollup, so you can see whether the rollup missed or double-counted anything. Defaults to yesterday (UTC); today is still accumulating and would always show drift. Requires the `read` scope. params: day |
spicrawl_requests_list | Recent API calls in your key's project, newest first, each with project_id, status, engine, http_status, error_code/error_detail, attempts, URL, proxy used, latency spans (queue/engine/upstream/total ms), block detection and credits. Use it to find failing calls (only_errors=true) or to spot patterns across a target. all_projects=true lists every project in your organization instead (needs a key w params: all_projects, limit, status, only_errors, before, before_id |
spicrawl_request_get | The full log record of one past API call by its request id. Every response carries its id in the X-Request-Id header (and error bodies carry it too), so this is the tool for debugging a failed or slow scrape: it shows status, error_code/error_detail, attempts, the engine and proxy (tier, country, provider, sticky) actually used, whether and how the target blocked it (vendor/signal/rule), per-stage params: id* |
spicrawl_browser_connect_url | Mints a single-use connect URL for Spicrawl's remote browser (GET /v1/browser, a DevTools Protocol (CDP) WebSocket) so YOUR code can drive it with any CDP client, using the URL as the browser WebSocket endpoint. It calls POST /v1/browser/token and returns a ws:// or wss:// URL carrying a short-lived token, never the API key: the URL works ONCE, within 60 seconds, and only with the options given he params: engine, proxy_country, proxy_region, sticky_key, session_ttl, proxy |
spicrawl_browser_live_view | Gets a link a person can open to watch the remote browser your CDP connection is driving, and take control of it: the same tabs, cookies and page. Use it when a page needs a human: a login, a one-time code, a captcha. Send them `live_view_url`; they press "Take control", do the step, and give control back. While they have it, your connection's input and navigation commands fail with error -32001 ( params: session_id* |
spicrawl_browser_control | Reads or changes who drives a connected remote browser: `agent` (your CDP connection) or `human` (a person in its live view, see spicrawl_browser_live_view). `status` reads it: poll it after sending a live-view link, and continue once `controller` is `agent` again. `takeover` hands control to the person before they ask (your input and navigation then fail with error -32001); `release` takes it bac params: session_id*, action |
spicrawl_docs_search | Full-text search over the Spicrawl documentation (guides, API reference, errors, limits). Use it to look up a parameter name, value or combination for a spicrawl_* tool, or to explain an error a spicrawl_* tool returned: search the error code itself (e.g. `ERR::UPSTREAM::CHALLENGE`) and the result also carries a direct link to that code's entry on the errors page. Returns the best-matching pages, params: query*, limit |
spicrawl_docs_read | Fetches one Spicrawl documentation page as Markdown. `path` may be a page path (`guides/anti-bot`), a docs URL from spicrawl_docs_search (url or md_url), or either with a `#anchor` (the whole page is returned; the anchor names the section to look at). Only pages on the Spicrawl docs (https://docs.spicrawl.com/docs) can be read. Pages over 60000 characters are truncated, and the text says so. Don't params: path* |
spicrawl_docs_index | The docs' llms.txt: every Spicrawl documentation page with its title, a one-line description and its Markdown URL. Use it to find the right page when a search does not, or to get an overview of what is documented; read a page with spicrawl_docs_read. No parameters. params: none |
Entries describe changes in tool definitions. They are not statements about a server, its quality or its author. The grade measures how the tool definitions fit model providers (token cost, schema compatibility, confusable names) as scored by the ToolFit engine at the time of the listing. JSON: append .json. Opt-out: how or hello@youraitools.co.