Watermarks, Made Physical

Events do not always arrive in the order they happened. A watermark is a progress estimate used to decide when to close older windows. Here, a late arrival drops at its own event time, hits an angled lid, and rolls into the left gutter. Events accepted earlier stay in their own windows when those windows close.

A remake of the mechanic fromflink-watermarks.wtf, using Semiotic's EventDropChart so you can inspect both the accepted count and the late-data correction. This example uses an simple clock-based estimate: watermark = arrival frontier − lag. The lag is how far behind the arrival clock we place the event-time completeness estimate.

Admission → closure → late arrival

Watch a window close

Both events happened at 6s. The first arrives at 7s; the second at 32s. The 0–12s window closes after 30s. Advance one moment at a time.

Arrival frontier: 7s. An event arrives while its window is open. Let it reach the bottom before closing the lid.

Skip to data table

0 in event-time bins + 0 in the far-left bin + 0 in flight = 0arrivals

The numbers count balls inside each container. The lid blocks every ball. Source time pauses while a ball travels, so the next deadline cannot overtake its illustrated arrival.

A decision about waiting

Now compare a whole stream

A delayed batch belongs to an earlier window. Closing sooner gives you an answer sooner; waiting longer admits more of that batch. Compare snapshots of the same 10events under two policies. Previously accepted history starts beneath any lids that have since closed.

Waiting 36 seconds longer per window admits 3 additional eventsfrom this tape.

Follow the event at 11s.It arrives at 62s. Its window ends at 12s; this policy closes it after 30s. It arrives behind the watermark of a closed window and goes to the far-left bin.At 70s, with a 18s lag:7 accepted · 3 late.

Read the apparatus: one ball is one event, horizontal position is event time, a lid is a closed window, and the far-left bin collects late arrivals for a separate correction. Numbers over event-time bins count accepted events; the far-left number counts late events. These are totals for the selected snapshot. The sequence above shows the moments of admission and closure.

Here the watermark trails the arrival clock by the chosen lag: it estimates how far event time is complete. A larger lag leaves more windows open. Late arrivals after closure go to a separate correction; this example has no extra grace period after closure.

accepted on arrivallate eventBackfill burst: a batch replay injects old event times after newer windows have already moved on.

18s watermark lagnow 70s − lag 18s = watermark 52s

The watermark is the left edge of the shaded waiting span. A window closes when that edge passes its end.

Skip to data table

Explore other streams and settings

Scenario

Arrival frontier70sWatermark lag18sReplay speedWindow size

Developer diagnostics

Session log

0changes

0edits

0redraws

offnothing yet

0 live bodies · 0 queued · 0ssimulation time · queued

10/10arrived / total

7accepted by the policy

3late arrivals to correct

18swatermark lag

The count you can check

WindowAcceptedLateReceived

0-12s112

12-24s112

24-36s112

36-48s202

48-60s101

60-72s101

72-84s000

Selected event

id: backfill-10
event time: 28s
arrival: 70s
source: backfill
timeliness: late event
arrival watermark: 52s

backfill-10 arrived at 70s. Its window ends at 36s; the watermark at arrival was 52s. The window was already closed, so this event was late.

Arrival order

backfill-016s7sapiarrived

backfill-0212s13sapiarrived

backfill-0324s25sfrontendarrived

backfill-0439s40ssensorarrived

backfill-0545s46ssensorarrived

backfill-0658s59sapiarrived

backfill-0811s62sbackfillarrived

backfill-0763s64sfrontendarrived

backfill-0917s66sbackfillarrived

backfill-1028s70sbackfillarrived

Inspect time states and correction records

One declared time model

The replay, these explanations, and the JSON payload use the same Artifact Contract time and claim records. Event time says when a row happened; the arrival frontier says what the pipeline has seen; the watermark determines which windows can settle.

Live / open

backfill-07 occurred at 63s. At the 70s arrival frontier, the watermark is 52s and its 60–72s window remains open; completeness is provisional.

Window: open
Completeness: provisional
Time audit: 0 failing · 0 unknown
Claim audit: 0 failing · 0warning

Claim state

provisionalThe 60–72s event-time window remains open with 1 event visible at the declared arrival frontier.

Settled

backfill-05 occurred at 45s. By the 70s arrival frontier, the 52s watermark had passed its 36–48s window; the window and completeness are both settled.

Window: settled
Completeness: settled
Time audit: 0 failing · 0 unknown
Claim audit: 0 failing · 1warning

Claim state

supportedThe 36–48s event-time window settled with 2 events.

Late arrival / corrected

backfill-10 belongs to the 24–36s event-time window. It was visible by the 70s arrival frontier after that window settled, so the revision is backfilled, the window is corrected, and completeness is settled.

Window: corrected
Completeness: settled
Time audit: 0 failing · 0 unknown
Claim audit: 0 failing · 1warning

Claim state

supersededBefore backfill-10 arrived, the 24–36s event-time window was reported settled with 1 event.
supportedAfter backfill-10 arrived, the corrected 24–36s event-time window contains 2 events.

Correction watermarks:backfill:late-arrival:backfill-10 preserveswatermarks:backfill:late-corrected.claim.before-late-arrival and links it towatermarks:backfill:late-corrected.claim.after-late-arrival.

Inspect the machine-readable time payload

{ "schema": "semiotic.time-state/0.1", "scenarioId": "backfill", "referenceTime": "2026-01-01T00:01:10.000Z", "eventCount": 10, "arrivedEventCount": 10, "states": [ { "id": "live-open", "label": "Live / open", "eventId": "backfill-07", "time": { "eventTime": { "field": "eventTime", "value": "2026-01-01T00:01:03.000Z", "timezone": "UTC", "granularity": "second" }, "observedAt": "2026-01-01T00:01:03.000Z", "ingestedAt": "2026-01-01T00:01:10.000Z", "presentation": { "state": "live", "label": "Live processing state as of the 70s arrival frontier" }, "freshness": { "status": "fresh", "checkedAt": "2026-01-01T00:01:10.000Z", "heartbeatAt": "2026-01-01T00:01:10.000Z", "expiresAt": "2026-01-01T00:01:11.000Z", "basis": "The deterministic replay cursor is the declared arrival frontier." }, "watermark": { "value": "2026-01-01T00:00:52.000Z", "policy": "Arrival frontier minus 18s watermark lag", "allowedLateness": "PT0S" }, "window": { "start": "2026-01-01T00:01:00.000Z", "end": "2026-01-01T00:01:12.000Z", "status": "open" }, "completeness": { "status": "provisional", "basis": "The watermark has not passed this event-time window." }, "revision": { "status": "original" }, "sources": [ { "id": "watermarks:backfill", "kind": "stream", "label": "Deterministic event replay", "observedAt": "2026-01-01T00:01:03.000Z", "timezone": "UTC", "granularity": "second", "freshness": "fresh", "completeness": "provisional" } ] }, "claimState": { "contractVersion": "0.1", "artifact": { "id": "watermarks:backfill:live-open", "kind": "chart", "component": "EventDropChart", "title": "Live / open", "createdAt": "2026-01-01T00:01:10.000Z", "revision": "1", "configFingerprint": "sha256:d3920b8724bc40c1ab9fdffcf514db60207dcfa86ace0b2f6efe8556e125983e", "dataFingerprint": "sha256:6a8c04c9fac5999184cbc3228c6e7a55647bf4ebb160af83419c44c89ed18831" }, "claims": [ { "id": "watermarks:backfill:live-open.claim.current-window", "text": "The 60–72s event-time window remains open with 1 event visible at the declared arrival frontier.", "kind": "observation", "status": "provisional", "evidenceIds": [ "watermarks:backfill:live-open.evidence.current-window" ], "asOf": "2026-01-01T00:01:10.000Z", "authoredBy": { "kind": "system", "name": "Deterministic watermark replay" }, "scope": { "scenarioId": "backfill", "eventId": "backfill-07", "windowStartSeconds": 60, "windowEndSeconds": 72, "eventCount": 1 } } ], "evidence": [ { "id": "watermarks:backfill:live-open.evidence.current-window", "role": "source-data", "label": "Rows in the declared event-time window", "fingerprint": "sha256:6a8c04c9fac5999184cbc3228c6e7a55647bf4ebb160af83419c44c89ed18831", "dataVersion": "watermarks:backfill:live-open:current-window", "observedAt": "2026-01-01T00:01:10.000Z", "scope": { "scenarioId": "backfill", "eventId": "backfill-07", "windowStartSeconds": 60, "windowEndSeconds": 72, "eventCount": 1 }, "relationship": "descriptive" } ], "corrections": [], "audit": { "ok": true, "fail": 0, "warn": 0 } }, "audit": { "ok": true, "fail": 0, "warn": 0, "manual": 0, "unknown": 0 } }, { "id": "settled", "label": "Settled", "eventId": "backfill-05", "time": { "eventTime": { "field": "eventTime", "value": "2026-01-01T00:00:45.000Z", "timezone": "UTC", "granularity": "second" }, "observedAt": "2026-01-01T00:00:45.000Z", "ingestedAt": "2026-01-01T00:01:10.000Z", "presentation": { "state": "live", "label": "Settled event-time window through 48s" }, "freshness": { "status": "fresh", "checkedAt": "2026-01-01T00:01:10.000Z", "heartbeatAt": "2026-01-01T00:01:10.000Z", "expiresAt": "2026-01-01T00:01:11.000Z", "basis": "The deterministic replay cursor is the declared arrival frontier." }, "watermark": { "value": "2026-01-01T00:00:52.000Z", "policy": "Arrival frontier minus 18s watermark lag", "allowedLateness": "PT0S" }, "window": { "start": "2026-01-01T00:00:36.000Z", "end": "2026-01-01T00:00:48.000Z", "status": "settled" }, "completeness": { "status": "settled", "basis": "The watermark passed this event-time window." }, "revision": { "status": "original" }, "sources": [ { "id": "watermarks:backfill", "kind": "stream", "label": "Deterministic event replay", "observedAt": "2026-01-01T00:00:45.000Z", "timezone": "UTC", "granularity": "second", "freshness": "fresh", "completeness": "settled" } ] }, "claimState": { "contractVersion": "0.1", "artifact": { "id": "watermarks:backfill:settled", "kind": "chart", "component": "EventDropChart", "title": "Settled", "createdAt": "2026-01-01T00:01:10.000Z", "revision": "1", "configFingerprint": "sha256:8621b1cc93a4f86b294db154c8f1e2b8ce61b258cbc38d007305978518e8b2b1", "dataFingerprint": "sha256:c217cad72438e71e93d0c7fb5f9c75fdc1609a1d1c20b9220ec6b76371064b1e" }, "claims": [ { "id": "watermarks:backfill:settled.claim.current-window", "text": "The 36–48s event-time window settled with 2 events.", "kind": "observation", "status": "supported", "evidenceIds": [ "watermarks:backfill:settled.evidence.current-window" ], "asOf": "2026-01-01T00:01:10.000Z", "authoredBy": { "kind": "system", "name": "Deterministic watermark replay" }, "scope": { "scenarioId": "backfill", "eventId": "backfill-05", "windowStartSeconds": 36, "windowEndSeconds": 48, "eventCount": 2 } } ], "evidence": [ { "id": "watermarks:backfill:settled.evidence.current-window", "role": "source-data", "label": "Rows in the declared event-time window", "fingerprint": "sha256:c217cad72438e71e93d0c7fb5f9c75fdc1609a1d1c20b9220ec6b76371064b1e", "dataVersion": "watermarks:backfill:settled:current-window", "observedAt": "2026-01-01T00:01:10.000Z", "scope": { "scenarioId": "backfill", "eventId": "backfill-05", "windowStartSeconds": 36, "windowEndSeconds": 48, "eventCount": 2 }, "relationship": "descriptive" } ], "corrections": [], "audit": { "ok": true, "fail": 0, "warn": 1 } }, "audit": { "ok": true, "fail": 0, "warn": 0, "manual": 0, "unknown": 0 } }, { "id": "late-corrected", "label": "Late arrival / corrected", "eventId": "backfill-10", "time": { "eventTime": { "field": "eventTime", "value": "2026-01-01T00:00:28.000Z", "timezone": "UTC", "granularity": "second" }, "observedAt": "2026-01-01T00:00:28.000Z", "ingestedAt": "2026-01-01T00:01:10.000Z", "presentation": { "state": "live", "label": "Corrected after late arrival backfill-10 at 70s" }, "freshness": { "status": "fresh", "checkedAt": "2026-01-01T00:01:10.000Z", "heartbeatAt": "2026-01-01T00:01:10.000Z", "expiresAt": "2026-01-01T00:01:11.000Z", "basis": "The deterministic replay cursor is the declared arrival frontier." }, "watermark": { "value": "2026-01-01T00:00:52.000Z", "policy": "Arrival frontier minus 18s watermark lag", "allowedLateness": "PT0S" }, "window": { "start": "2026-01-01T00:00:24.000Z", "end": "2026-01-01T00:00:36.000Z", "status": "corrected" }, "completeness": { "status": "settled", "basis": "The watermark passed this event-time window." }, "revision": { "status": "backfilled", "previousArtifactId": "watermarks:backfill:before-backfill-10", "correctionId": "watermarks:backfill:late-arrival:backfill-10", "reason": "A late or replayed event changed a window that had already settled." }, "sources": [ { "id": "watermarks:backfill", "kind": "stream", "label": "Deterministic event replay", "observedAt": "2026-01-01T00:00:28.000Z", "timezone": "UTC", "granularity": "second", "freshness": "fresh", "completeness": "settled" } ] }, "claimState": { "contractVersion": "0.1", "artifact": { "id": "watermarks:backfill:late-corrected", "kind": "chart", "component": "EventDropChart", "title": "Late arrival / corrected", "createdAt": "2026-01-01T00:01:10.000Z", "revision": "2", "configFingerprint": "sha256:632d123162ce64839cf6811e622fe872e02368b57650eb79e98411bc0a492da1", "dataFingerprint": "sha256:aca72212d890d8d870c0f95ce78446634c3633379c7e3e17ba5f20a47bb5cabe" }, "claims": [ { "id": "watermarks:backfill:late-corrected.claim.before-late-arrival", "text": "Before backfill-10 arrived, the 24–36s event-time window was reported settled with 1 event.", "kind": "observation", "status": "superseded", "evidenceIds": [ "watermarks:backfill:late-corrected.evidence.before-late-arrival" ], "asOf": "2026-01-01T00:01:10.000Z", "authoredBy": { "kind": "system", "name": "Deterministic watermark replay" }, "scope": { "scenarioId": "backfill", "eventId": "backfill-10", "windowStartSeconds": 24, "windowEndSeconds": 36, "eventCount": 1 } }, { "id": "watermarks:backfill:late-corrected.claim.after-late-arrival", "text": "After backfill-10 arrived, the corrected 24–36s event-time window contains 2 events.", "kind": "observation", "status": "supported", "evidenceIds": [ "watermarks:backfill:late-corrected.evidence.current-window" ], "asOf": "2026-01-01T00:01:10.000Z", "authoredBy": { "kind": "system", "name": "Deterministic watermark replay" }, "scope": { "scenarioId": "backfill", "eventId": "backfill-10", "windowStartSeconds": 24, "windowEndSeconds": 36, "eventCount": 2 }, "supersedes": [ "watermarks:backfill:late-corrected.claim.before-late-arrival" ] } ], "evidence": [ { "id": "watermarks:backfill:late-corrected.evidence.before-late-arrival", "role": "source-data", "label": "Rows in the declared event-time window", "fingerprint": "sha256:1c0a36d032cd305b949443411182adc4fe411bd9e8d7d2e7d9d258d58b213ca5", "dataVersion": "watermarks:backfill:late-corrected:before-late-arrival", "observedAt": "2026-01-01T00:00:52.000Z", "scope": { "scenarioId": "backfill", "eventId": "backfill-10", "windowStartSeconds": 24, "windowEndSeconds": 36, "eventCount": 1 }, "relationship": "descriptive" }, { "id": "watermarks:backfill:late-corrected.evidence.current-window", "role": "source-data", "label": "Rows in the declared event-time window", "fingerprint": "sha256:aca72212d890d8d870c0f95ce78446634c3633379c7e3e17ba5f20a47bb5cabe", "dataVersion": "watermarks:backfill:late-corrected:current-window", "observedAt": "2026-01-01T00:01:10.000Z", "scope": { "scenarioId": "backfill", "eventId": "backfill-10", "windowStartSeconds": 24, "windowEndSeconds": 36, "eventCount": 2 }, "relationship": "descriptive" } ], "corrections": [ { "id": "watermarks:backfill:late-arrival:backfill-10", "reason": "A late or replayed event changed a window that had already settled.", "createdAt": "2026-01-01T00:01:10.000Z", "createdBy": { "kind": "system", "name": "Deterministic watermark replay" }, "affectedClaimIds": [ "watermarks:backfill:late-corrected.claim.before-late-arrival" ], "replacementClaimIds": [ "watermarks:backfill:late-corrected.claim.after-late-arrival" ] } ], "audit": { "ok": true, "fail": 0, "warn": 1 } }, "audit": { "ok": true, "fail": 0, "warn": 0, "manual": 0, "unknown": 0 } } ] }

How it maps to Semiotic

Data rows carry an event time and an arrival time. The Artifact Contract adapter turns the replay controls into one deterministic time record; its audited window, watermark, completeness, claim, and correction fields drive the labels and inspectable payload. Bodies still drop over their event-time x-position, and closed windows add angled physics colliders. The frame animates the replay; onTick observes its progress.

JSX

import { adaptStreamTopicMetadata, auditTemporalContext, updateTemporalContext, } from "semiotic/artifact" import { EventDropChart } from "semiotic/physics" // One portable time record drives the visible labels and inspectable JSON. const liveTime = adaptStreamTopicMetadata({ id: "orders", eventTime: { field: "eventTime", value: latestEventTime, timezone: "UTC" }, ingestedAt: arrivalFrontier, watermark: { value: watermark, policy: "Arrival frontier minus an 18s lag", allowedLateness: "PT0S" }, window: { start: windowStart, end: windowEnd, status: "open" }, completeness: { status: "provisional" }, }) const correctedTime = updateTemporalContext(liveTime, { window: { start: windowStart, end: windowEnd, status: "corrected" }, completeness: { status: "settled" }, revision: { status: "backfilled", previousArtifactId: "orders-before-late-rows", reason: "Late rows changed the window", }, sources: [{ id: "orders", kind: "stream", freshness: "fresh", completeness: "settled", }], }) const timeAudit = auditTemporalContext(correctedTime, { referenceTime: arrivalFrontier, }) <EventDropChart ref={chartRef} data={arrivedEvents.map(event => ({ ...event, watermarkAtArrival: event.arrivalTime - watermarkLag - 1e-6, }))} timeAccessor="eventTime" arrivalAccessor="arrivalTime" watermarkAtArrivalAccessor="watermarkAtArrival" windows={{ size: windowSize }} watermark={{ value: arrivalFrontier - watermarkLag - 1e-6 }} timeExtent={eventTimeExtent} colorBy={classify} timeScale={8} // playback speed — higher is faster ballRadius={7.5} showProjection={false} frameProps={{ onTick, foregroundGraphics }} />