Apps & Software

• 5 min read

Draco brings stealth web scraping to a Rust binary

Draco is a Rust single-binary scraper with stealth TLS fingerprints, SPA hydration, JSON extraction, and a Firecrawl-compatible daemon.

Image: Hacker News

Draco is a self-hostable, single-binary web scraper written in Rust that targets the same general use case as Firecrawl and Browserbase: send it a URL and receive clean Markdown, metadata, or structured data. Its key distinction is a browser-faithful TLS/JA4 fingerprint, designed to reach sites that block ordinary HTTP clients without requiring a fleet of headless browsers.

The project is available at github.com/0xchasercat/draco. The fastest installation path on Linux or macOS is:

''sh curl -fsSL https://raw.githubusercontent.com/0xchasercat/draco/main/install.sh | sh ''

The script detects the operating system and architecture, downloads the latest binary, and adds Draco to ~/.zshrc, ~/.bashrc, or ~/.config/fish/config.fish. A basic scrape writes Markdown to standard output:

Recommended reading

Apple Upgrade launches as iOS 26.6 arrives

''sh draco scrape https://example.com ''

Rust-based extraction without a browser fleet

For a standard HTML page, Draco performs a single fingerprinted fetch and parse in typically about 300 ms, without starting a browser. Its Markdown pipeline mirrors Firecrawl’s approach: deterministic main-content extraction followed by a Turndown/GFM-equivalent conversion, implemented natively in Rust.

The output can include:

  • Clean Markdown with headings, absolutized links, lists, blockquotes, fenced code blocks with language labels, and GFM tables
  • Removal of navigation, headers, sidebars, footers, advertisements, scripts, styles, and base64 images
  • Metadata including the title, description, language, canonical URL, favicon, all og:, twitter:, and article:* tags, plus sourceURL, statusCode, and contentType
  • Trace and timing information showing which processing steps ran and how long they took

The command-line interface also supports HTML, raw HTML, links, JSON, endpoints, and combined Markdown-plus-JSON output. Options include proxies, delays, timeouts, JSONPath extraction, render waiting, robot exclusions, and runtime logging.

''sh draco scrape https://example.com --json --pretty draco scrape https://example.com --proxy socks5://127.0.0.1:9050 --delay 500 ''

Exit codes are 0 for success, 1 for an error, 2 for an unsupported target, and 3 when a browser is needed.

How Draco handles client-rendered SPAs

Draco does not treat JavaScript-heavy sites as a separate browser-scraping workflow. If the initial HTML contains little meaningful content, it can hydrate the page in an in-process V8 isolate, serialize the live DOM, combine the original <head> with the hydrated <body>, and run the same content-extraction pipeline again.

The isolate is restored from a build-time DOM-engine snapshot in single-digit milliseconds. Draco describes it as a discovery mechanism rather than a renderer: page JavaScript has no host bindings and cannot perform I/O. Network activity is brokered by the engine instead.

For pure client-rendered SPAs with no embedded state, Draco can observe the requests made during hydration and identify the API supplying the page’s data. In its Render mode, it fetches safe data requests live through the same stealth client and shared cookie jar. That includes GET and HEAD, as well as read-style POST and PUT requests shaped like GraphQL or JSON-RPC calls.

State-changing requests remain stubbed unless --allow-unsafe-replay is supplied. Streaming endpoints and analytics beacons are never fetched live. The capture window closes once content activity settles, with a hard ceiling as a backstop.

External scripts and ES modules are supported, including import() and dynamic imports. Script resources are fetched concurrently through a pooled stealth client and stored in a process-global immutable cache limited to 512 MiB of RAM and 2 GiB of disk. If hydration cannot improve a page, Draco falls back to the static shell rather than failing.

The project also detects skeleton screens whose content is still marked “Loading…”, even when the page contains substantial interface chrome. Those placeholder lines are stripped from the result.

Tiered JSON extraction and Firecrawl-compatible API

Draco’s optional --format json mode escalates through progressively more expensive extraction methods:

  • Static embedded state such as __NEXT_DATA__, JSON-LD, and window.__NUXT__
  • Direct replay of a Next.js build-ID JSON endpoint at /_next/data/<buildId>/…​.json
  • Runtime interception in V8, followed by ranking and replaying the most likely data endpoint

The same mechanism powers endpoint discovery. --format endpoints returns a ranked catalog of intercepted APIs with their method, URL, score, replayability, and headers. Analytics beacons and static assets are ranked below likely data APIs.

Draco can also run as a persistent daemon:

''sh draco serve ''

By default, it listens on http://127.0.0.1:3002, Firecrawl’s default port. The daemon exposes a Firecrawl-compatible REST API, including POST /v1/scrape, so existing Firecrawl clients can target it without changes. It supports Markdown and JSON formats; requests for HTML, raw HTML, links, or screenshots are rejected with a clear 400 response because Draco does not produce those formats yet.

The daemon additionally provides:

  • POST /v1/map for same-host URL discovery using sitemaps and page links
  • POST /v1/crawl for bounded, asynchronous same-host BFS crawls
  • POST /v1/batch/scrape for parallel URL batches
  • Webhooks for crawl and batch lifecycle events, with retries after 1, 5, and 15 minutes
  • POST /v1/discover for ranked API discovery and replay
  • POST /v1/search and draco search for multi-engine web search

Crawls default to a limit of 10 pages and a maximum depth of 2, with a hard limit of 100 pages. Concurrency is bounded by --max-concurrency, whose default is 8. Tier 2 jobs use fresh isolates, while --isolate-pool-size controls concurrent isolate execution and defaults to automatic sizing at approximately the CPU count.

Build requirements and open details

Users can install a prebuilt binary or compile Draco with Cargo:

''sh git clone https://github.com/0xchasercat/draco cd draco cargo build --release ''

Build prerequisites include CMake, a C/C++ compiler, Clang/libclang, Perl, and pkg-config; V8 is required only for the optional JSON mode. Debian and Ubuntu users need build-essential cmake clang libclang-dev perl pkg-config, while Fedora users need gcc gcc-c++ cmake clang clang-devel llvm-devel perl pkgconf. macOS requires Xcode Command Line Tools and brew install cmake.

The repository does not specify pricing or a separate release date. Its local benchmark reports warm-process SPA hydration at about 150 ms end-to-end, covering fetch, hydration, serialization, and Markdown conversion.

Tomas Berg

Computing Editor

Tomas lives in the terminal. He covers chips, laptops, and operating systems with a focus on performance and efficiency. He reads kernel changelogs the way other people read fiction, and he's always on the hunt for the perfect mechanical keyboard switch. If it processes data, Tomas has an opinion on it.

via Hacker News

/ Keep reading