Introduction
Ray9 is an AI-assisted web data platform. It turns a described outcome into a versioned Flow, runs it on a route that can actually reach the page, stores the records with provenance, watches them for change, and delivers them to the rest of your business.
The problem Ray9 solves
Reliable web data collection is normally assembled rather than bought. A working setup ends up spread across a browser automation library, a proxy vendor, a hosted scraping API, custom parsing code, a scheduler, a database, and a webhook system. Each piece is somebody's job to maintain, and when a page changes, the failure surfaces somewhere in the middle with no clear owner.
Ray9 takes the whole lifecycle as one product. You describe the records you want; Ray9 proves the extraction on the real page, publishes an immutable version of that definition, runs it on the cheapest route that can actually do the job, validates the output, stores it with its evidence, watches for meaningful changes, and delivers it onward.
It is not a URL-to-markdown endpoint. That is a single step inside a longer loop.
The loop
Everything in Ray9 is an instance of the same six-stage lifecycle.
Build
Give a starting URL and describe the records you want. Ray9 detects repeated items, proposes a schema with field names and types, and shows the exact page region behind every field. You correct what it got wrong by clicking on the page — no selectors required, though you can write them if you prefer.
Read more in Flows.
Run
A run is one requested execution of a Flow version. Ray9 evaluates target and organization policy, estimates a credit range, reserves a maximum, and then dispatches the work to an eligible execution route: direct HTTP, a managed provider, or an isolated cloud browser. Runs are durable — they survive process restarts, they can be cancelled, and retries are bounded and classified.
Read more in Execution routes.
Validate
HTTP 200 is not success. A challenge page, a consent wall, the wrong locale, or plausible-but-wrong content all return 200 and all are extraction failures. Ray9 assesses transport, page correctness, extraction, schema conformance, and evidence separately, and reports a run as complete, partial, blocked, action-required, failed, or cancelled — never as a single success boolean.
Store
Valid records land in a dataset: a current table of logical records, plus an append-only history of every observation behind them. Each observation carries its source URL, the Flow version that produced it, the run, the schema version that validated it, and when it was observed.
Read more in Datasets.
Monitor
A monitoring policy compares normalized values between observations rather than diffing raw HTML. It distinguishes added, removed, and changed records, suppresses known noise, and can require repeated confirmation before it tells you anything.
Read more in Monitoring.
Who Ray9 is for
Operators and small teams who need price, catalog, competitor, lead, or market data and have no interest in owning scraping infrastructure. The common path is visual: describe the outcome, check the sample, schedule it.
Analysts and researchers who need a reproducible dataset rather than a one-off copy-paste — with source evidence retained, values keyed to a stable record identity, and a history of how each one changed.
Developers and AI builders who want the same Flows from code. The dashboard, the public API, the CLI, and the MCP server call the same application services and observe the same authorization, idempotency, policy, and credit rules. There is no second-class programmatic path.
What Ray9 is not
Being clear about the edges saves time later.
- It is not a general-purpose automation platform for arbitrary destructive actions on websites. Ray9 collects data; it does not click "buy" for you.
- It does not bypass logins, paywalls, or access controls. Authenticated collection is possible where you have the right to it and supply your own credentials, but Ray9 will never treat an
auth_requiredoutcome as an invitation to work around the wall. - It does not guarantee access to every website. Some targets block cloud infrastructure; some should be reached through an official or licensed API instead. Ray9 tells you which, and why, rather than retrying quietly until your credits run out.
- It is not a warehouse. Datasets are queryable, exportable, and delivered onward; they are not a SQL engine over unlimited scraped history.
Responsible use
Ray9's defaults favour public, logged-out, non-sensitive data and conservative rate limits. Target and organization policy is evaluated before credits are spent and before any work is dispatched, so a run that should not happen is denied rather than billed.
Authenticated jobs require your explicit action and come with isolated credential scopes, encryption, short retention by default, and an audit trail. Requirements you set — a geography, a logged-in session, a headful browser — are hard constraints; Ray9 will not silently relax one to make a run succeed on a cheaper route.
You remain responsible for the data you collect and how you use it. Nothing in these docs is legal advice.
Where to start
Quickstart
Build, test, publish, and run your first Flow, then look at the dataset it produced.
Concepts
The vocabulary — Flow, version, run, attempt, dataset, record, credit — used identically across every interface.
Execution routes
How Ray9 decides how to reach a page, and what it does when a target says no.
Credits
What a credit is, why heavier routes cost more, and how estimates and budgets keep spend bounded.
Public API
The stable /v1 contract: scoped keys, idempotency, structured errors, pagination.
MCP server
Give an agent the same domain model, under the same scoped key and the same credit rules.