Every Semiotic chart accepts an onObservation callback that emits structured events when users interact with the chart. These observations can feed into an AI agent's context window, enabling real-time insight generation based on what users are exploring.
Quick Start
Add onObservation to any chart to start receiving events. Each event includes the type, the datum being interacted with, pixel coordinates, a timestamp, and the chart type.
JSX
import { Scatterplot } from "semiotic" <Scatterplot data={data} xAccessor="x" yAccessor="y" onObservation={(obs) => { // obs.type: "hover" | "hover-end" | "focus" | "click" | "click-end" | "activate" // | "annotation-activate" | "brush" | "brush-end" | "selection" | "selection-end" // | "control-start" | "control-change" | "control-end" // obs.datum: the data point being hovered // obs.timestamp: Date.now() // obs.chartType: "Scatterplot" console.log(obs.type, obs.datum) }} chartId="my-scatter" />
Live Demo
Hover over points in the scatterplot to see observation events appear in the log panel in real time. Each hover emits a structured event with the full datum and position.
Observation Log
Hover over the chart...
JSX
function ObservationLog() { const [log, setLog] = useState([]) return ( <> <Scatterplot data={data} xAccessor="x" yAccessor="y" colorBy="region" onObservation={(obs) => { setLog(prev => [...prev.slice(-7), obs]) }} chartId="demo-scatter" /> <pre>{JSON.stringify(log, null, 2)}</pre> </> ) }
Event Types
Chart events cover hover, semantic focus and activation, brush, selection, and direct-control interactions. Semantic events are additive: a keyboard activation emits the existing clickevent and an activate event with its input channel.
| Type | When | Key Fields |
|---|
hover | User hovers over a data point | datum, x, y |
hover-end | Mouse leaves chart or data | — |
focus | Keyboard or structured navigation focuses a datum | datum, inputType |
activate | A datum is activated by pointer, touch, keyboard, or navigation tree | datum, inputType |
annotation-activate | A stable widget annotation is activated | annotationId, inputType |
brush | Brush selection changes | extent.x, extent.y |
brush-end | Brush cleared | — |
selection | Cross-chart selection changes | selection.name, selection.fields |
selection-end | Selection cleared | selection.name |
control-start | A direct control begins a pointer interaction | controlType, controlId, value |
control-change | A control changes through pointer or keyboard input | controlType, value, source |
control-end | A direct-control pointer interaction ends | controlType, controlId, value |
Interactive widget annotations should carry id,stableId, or provenance.stableId. Semiotic never falls back to an annotation's array index for activation identity. Stream Frames accept onAnnotationActivate; chart HOCs expose the same callback throughframeProps.onAnnotationActivate.
All events include timestamp (Date.now()),chartType (e.g. "Scatterplot"), and optionallychartId (your custom identifier).
JSX
import { DirectManipulationControl } from "semiotic/controls" <DirectManipulationControl controlId="priority-threshold" controlType="threshold" chartType="XYCustomChart" chartId="priority-chart" value={threshold} min={0} max={100} label="Priority threshold" onChange={setThreshold} onObservation={onObservation} pointerToValue={pointerToThreshold} />
useChartObserver
Inside a LinkedCharts provider, theuseChartObserver hook aggregates observations from all charts in the coordinated view. This enables "insight panels" that react to what users are exploring across multiple charts.
AI Insight Panel
Hover over either chart to generate observations...
JSX
import { CategoryColorProvider, LinkedCharts, Scatterplot, BarChart, useChartObserver } from "semiotic" function InsightPanel() { const { observations, latest, clear } = useChartObserver({ limit: 20, // keep last 20 observations types: ["hover"], // only hover events chartId: undefined // from all charts (or filter by chartId) }) // Example: count hovers per region for an AI prompt const regionCounts = {} for (const obs of observations) { const region = obs.datum?.region if (region) regionCounts[region] = (regionCounts[region] || 0) + 1 } return <div>Most explored: {Object.entries(regionCounts).sort((a,b) => b[1]-a[1])[0]?.[0]}</div> } function Dashboard() { // CategoryColorProvider is what makes LinkedCharts render a single // unified legend (and gives every chart the same color for each // category). Without it, each chart renders its own legend. return ( <CategoryColorProvider categories={["North", "South", "East", "West"]}> <LinkedCharts> <Scatterplot data={d} xAccessor="x" yAccessor="y" colorBy="region" onObservation={() => {}} chartId="scatter" linkedHover={{ name: "hl", fields: ["region"] }} selection={{ name: "hl" }} /> <BarChart data={agg} categoryAccessor="region" valueAccessor="total" onObservation={() => {}} chartId="bar" linkedHover={{ name: "hl", fields: ["region"] }} selection={{ name: "hl" }} /> <InsightPanel /> </LinkedCharts> </CategoryColorProvider> ) }
Options
| Option | Type | Default | Description |
|---|
limit | number | 50 | Maximum observations returned |
types | string[] | all | Filter by event type |
chartId | string | all | Filter by chart instance |
AI Agent Integration
The observation system completes Semiotic's AI loop: theMCP server lets agents rendercharts, while observation hooks let agents watch what users do. An AI agent can consume observations and respond with annotations, suggested views, or natural language insights.
JSX
// Example: feed observations to an AI agent function AIAssistant() { const { observations } = useChartObserver({ limit: 50 }) useEffect(() => { if (observations.length < 5) return // Build a prompt from recent observations const summary = observations.map(o => o.type === "hover" ? `User examined ${JSON.stringify(o.datum)} in ${o.chartType}` : `User stopped examining (${o.chartType})` ).join("\n") // Send to your AI backend fetchInsight(summary).then(insight => { // Add insight as an annotation on the chart setAnnotations([{ type: "callout", label: insight }]) }) }, [observations.length]) }
- Interaction — hover, click, and brush interaction configuration
- Linked Charts — cross-highlighting and coordinated views that observations compose with
- Tooltips — tooltip content that works alongside observation hooks
- Annotations — add AI-generated annotations in response to observations