Execution routes

How Ray9 decides how to reach a page — official APIs, direct HTTP, managed providers, cloud browsers, and egress classes — and what it does when a target says no.

Access is a routing problem

No single technique reaches the whole web. A headless browser is overkill for a static page and useless against some targets. Direct HTTP is cheap and fails the moment JavaScript matters. A residential exit is not a universal solvent; it is an expensive tool that helps with one specific class of problem.

So Ray9 does not have "a scraper". It has a route: a versioned combination of independent dimensions, chosen per attempt, recorded on the result.

Dimension	What it decides
Acquisition method	How the page is reached — official API, direct fetch, managed provider, cloud browser
Browser profile	Engine, headless or headful, behaviour posture — where a browser is used at all
Egress class	Datacenter, residential, or mobile — the IP the target actually sees
Geography and session	Country, and whether the session must stay sticky across pages
Extraction path	How the capture becomes records

These stay separate because they fail separately. A target blocking your IP reputation is a different problem from a target detecting your browser fingerprint, which is different again from a page whose content is fine but whose extraction is wrong. Collapsing them into one "provider" setting means never being able to tell which one broke.

The routes

Official APIs

Where a platform offers an official or licensed API with the fidelity the job needs, that is the right way to get the data — not page scraping. It is more reliable, it is usually cheaper, and it is on the correct side of the platform's terms.

So the routing model treats an official connector as a first-class route, preferred over page scraping regardless of where it would sit on a generic cost ladder — rather than as an afterthought below the proxy escalation ladder.

Connectors are roadmap

Official and licensed connectors are a route the model reserves a place for, not a catalog you can pick from today. They arrive per platform, as demand and licensing justify each one. Where a target is genuinely better served by an official API, Ray9's answer today is to tell you so rather than to scrape it badly.

Direct HTTP

An isolated HTTP fetch, with robots and policy handling, redirects, content limits, and sitemap or feed discovery. For a public, stable page that does not need a browser, this is the fastest and cheapest route, and it is the one Ray9 will reach for first when the job allows it.

Managed providers

Third-party extraction and rendering services, used where they are the reliable and economical way to reach a particular target. They sit behind a normalized adapter, so their results arrive in the same shape as everything else and their metadata is preserved rather than reinterpreted.

Where a provider's own contract is opaque — a mode labelled "enhanced", an automatic proxy behaviour it does not describe — Ray9 records exactly what the provider reported and does not invent a route tier it cannot verify. A provider mode is not relabelled as "residential" because it sounds like it might be.

Cloud browsers

Isolated browsers on ephemeral infrastructure, for JavaScript, interaction, and anti-bot-sensitive targets. They get a fresh, attempt-scoped context, deterministic steps, resource limits, and full capture — HTML, screenshots, traces.

Headless is the default. Headful is a peer, not a debug mode. Research on real targets showed that headful, behaviour-aware execution changes outcomes on some anti-bot systems, so headful is an escalation profile you can require — and, when you require it, a hard constraint that will not be quietly dropped.

Browser workers are isolated by design: they hold no database credentials, no private network path, and no long-lived secrets. They receive a signed, expiring job envelope and return a signed, normalized result.

Local execution

Not available yet

Running a Flow in your own browser, on your own device, with your own session — under an explicit trust boundary with enrollment, per-job consent, and revocation — is on the roadmap. It is designed for, but it is not shipped. Everything running today runs in the cloud.

Egress classes

Egress is chosen independently of the browser, because IP reputation and browser fingerprint are separate signals that fail for separate reasons.

Class	What it is
Datacenter	The default. Cloud infrastructure egress: cheapest, fastest, and fine for most public pages.
Residential	Consumer ISP exits, with requested geography and sticky-session support where the provider offers it.
Mobile	Mobile carrier exits, for the small number of targets where nothing else works.

More expensive egress is not automatically better. It costs more, it is slower, and on many targets it changes nothing. Ray9 escalates egress for a classified block, challenge, or IP-reputation failure — not because an extraction came back empty, and not as a speculative upgrade you did not ask for.

Residential and mobile routes are available on plans that include them. See pricing.

Hard constraints are never silently relaxed

This is the rule that matters most, and it is the one most easily broken by a system that optimizes for "make the run succeed".

If a job requires a specific geography, an authenticated session, headful interaction, or a sticky session, those are hard eligibility constraints. They are applied before routes are ranked by cost, and a route that cannot satisfy them is removed from consideration entirely.

Ray9 will not fall back to a cheaper route that quietly drops your requirement. A run that asked for German residential egress and got a US datacenter fetch is not a successful run — it is a wrong answer with a green tick on it, and it is worse than a failure, because you will believe it.

"Cheapest first" therefore means cheapest among the routes that can actually satisfy the request.

How a route is chosen

Policy is evaluated first, before any credits are spent or any work is dispatched. A denied run costs nothing.
Hard eligibility is compiled — capability, geography, authentication, sensitivity, plan entitlements. Ineligible routes are removed.
Unhealthy routes are removed, using circuit breakers that track provider health, egress health, and target-specific route health separately, so one blocked domain does not disable a provider globally.
The maximum credit budget is reserved before dispatch.
The remaining candidates are ordered and the first is executed.
The result is validated — transport, page correctness, extraction, schema, evidence — independently.
Ray9 stops on a validated result, or escalates only if the classified failure maps to an allowed next route and attempts, time, and reserved budget all remain.

The route plan is snapshotted before dispatch, so a configuration change mid-run never mutates work already in flight. Every decision and every attempt is persisted.

Today, route ordering is deterministic: a defined, classified waterfall rather than a model deciding on the fly. Adaptive ranking — learning from validated outcomes which route works best for a target class — is enabled only for cohorts with enough validated evidence to justify it, behind circuit breakers and rollback. A router that learns from too little evidence is just a router that is confidently wrong.

Failure categories

Ray9 does not reduce outcomes to a boolean. HTTP 200 is not success — a challenge page, a consent wall, a login screen, and the wrong locale all return 200.

Category	What it means	What to do
`policy_denied`	Target or organization policy refused the run	Review the policy explanation. No dispatch, no charge.
`target_blocked`	The target actively blocked the attempt	Try an eligible route, or reconsider the target. Evidence is preserved.
`auth_required`	The content is behind a login	Supply an approved credential. Ray9 will not attempt a bypass.
`rate_limited`	The target asked you to slow down	Wait for the stated cooldown. Ray9 respects it rather than rotating IPs around it.
`navigation_failed`	The browser could not get where it needed to go	Check the navigation steps against the current page.
`extraction_failed`	The page was reached, but the data was not there	Repair the field mapping — usually against the retained capture, without refetching.
`timeout`	The work exceeded its deadline	Narrow the job, or raise the limit.
`cancelled`	Somebody or something stopped it	—
`platform_error`	Ray9's fault	Retry safely. Idempotency prevents duplicate records or charges.

Each failure tells you what Ray9 already tried and what is available next. A run that half-worked is partial, and its valid records are kept — a partial result is not silently promoted to complete, and it is not thrown away either.

Execution routes

On this page