Monitoring

Monitoring is a Flow capability

A monitor is not a separate product with its own execution engine. It is a policy attached to a Flow.

Each check is an ordinary run: the same policy evaluation, the same credit reservation, the same execution routes, the same evidence, the same dataset. What monitoring adds is a comparison step at the end — take the new observation, compare it with the baseline, and decide whether anything meaningful happened.

That means everything you already know about Flows, schedules, and datasets applies unchanged. A monitor is a Flow on a schedule that also asks "what changed?".

Normalized comparison, not raw HTML diff

Diffing raw HTML is the reason most page-watching tools cry wolf. A rotating advert, a timestamp in the footer, a randomized element ID, a re-ordered script tag — every one of those is a diff, and none of them is news.

Ray9 compares normalized values by default: the fields in your extraction schema, canonicalized. Whitespace, URL form, dates, and currencies are normalized first, so £1,299.00 and £1299 are not a price change.

For raw page monitoring, Ray9 normalizes the selected region first and removes known noise — timestamps, randomized identifiers, advertising — before it compares anything. A genuine raw HTML diff is available as an advanced option, but it is not the default, and it is not what most jobs want.

Deciding what counts as meaningful

A monitoring policy defines four things.

What to compare. The record identity, and which fields participate. You rarely want all of them: a product's price and availability are worth watching, its marketing blurb usually is not.

What counts as a change. Any value change, a numeric threshold (a price moving more than 5%), a text include or exclude rule, an added record, a removed record, or a semantic judgement about whether the change is meaningful.

What to ignore. Include and exclude patterns, plus explicit noise suppression for the fields you already know are jittery.

How sure you need to be. A confirmation count — how many consecutive checks must agree before Ray9 tells you. Quiet hours, and where the notification goes.

Added, removed, and changed records are distinguished. "Three products disappeared from the category" and "three products changed price" are different events and should never arrive looking the same.

Confirmation for jittery fields

Some fields flicker. A stock counter, a dynamic price, a field that briefly reads zero while the page renders — these produce a change, then produce it back again a minute later.

For those, set a confirmation count. Ray9 requires the same change to be observed on that many consecutive checks before it emits a change event. A one-check flicker does not wake anyone up; a real move does.

This is a policy decision, not a guess Ray9 makes for you. A price flapping between two values is the news on some targets and pure noise on others, and only you know which.

The change timeline

Change events land on a timeline. Each one links to:

• the before and after record versions, field by field,
• the run that observed it and the Flow version that produced it,
• the source evidence — the capture that backs the claim,
• the comparison policy version that decided it was meaningful.

That last one matters more than it sounds. When you mark an event as noise, Ray9 updates future comparison behaviour by creating a new monitor version. It does not reach back and rewrite events that already happened. Your history stays a record of what you actually knew at the time, rather than a document that quietly re-edits itself.

When the judgement is uncertain

Where a semantic judgement is used to decide whether a change is meaningful, its outcome — including its uncertainty — is recorded on the change event alongside the deterministic comparison.

If that judgement fails or returns something invalid, the configured behaviour applies and is visible on the event. Ray9 does not silently convert an uncertain result into "changed" or "unchanged"; a monitor that quietly guesses in either direction is worse than one that tells you it could not decide.

Delivery

Change events are delivered like any other event: signed webhooks with idempotent event IDs, bounded retries, and a dead-letter state, or exported and read through the API. See Webhooks.