What the Machine Sees

Give Semiotic a table and a question. It compares chart forms, explains its recommendation, describes the result and checks accessibility. Follow each step below, then choose a different chart to compare. These calculations run in your browser without a language model.

Ask the data a question
01

What it sees

South Korea's life expectancy at birth, every year from 1990 to 2023.

34Rows
2Fields
noTime axis
xyearylifeExpectancy
02

What it weighed

Built-in charts scored against this table and the question typetrend. Click any candidate to overrule the ranking.

03

What it chose

engine's pick · LineChart · Smooth trend
Intent Mark · trend

time-series: trend. Strong fit for trend (5/5)

{ "ididVersion": "0.1", "chartId": "machine-sees-chart", "title": "How long do people live, over time?", "intent": { "primary": "trend", "communicativeAct": "This is a trend chart; read it for the trajectory of lifeExpectancy, which rises from 71.6 (1,990) to 83.4 (2,023)." }, "reception": { "channels": [ "visual", "interactive", "screen-reader", "agent" ], "strengths": [ "familiar", "deterministic", "agent-readable" ], "scaffolds": [ "description", "accessible table", "navigation tree" ] }, "designContract": { "chartFamily": "time-series", "whyThisForm": "Strong fit for trend (5/5)", "risks": [ "smoothing hides individual outliers" ] }, "accessibility": { "description": "A line chart of lifeExpectancy by year. lifeExpectancy ranges from 71.6 (1,990) to 83.5 (2,021), with a mean of 78.39 across 34 points. Overall lifeExpectancy rises from 71.6 (1,990) to 83.4 (2,023), after peaking at 83.5 (2,021). This is a trend chart; read it for the trajectory of lifeExpectancy, which rises from 71.6 (1,990) to 83.4 (2,023).", "navigation": true, "dataFallback": true, "manualChecks": [ "screen-reader behavior", "keyboard order quality" ] } }
Skip to data table
South Korea's life expectancy at birth, every year from 1990 to 2023.
04

What it says

The generated description moves from the chart’s form and values to patterns and its intended message. The last level uses the same task scores that informed the chart recommendation.

L1EncodingA line chart of lifeExpectancy by year.
L2StatisticslifeExpectancy ranges from 71.6 (1,990) to 83.5 (2,021), with a mean of 78.39 across 34 points.
L3TrendOverall lifeExpectancy rises from 71.6 (1,990) to 83.4 (2,023), after peaking at 83.5 (2,021).
L4IntentThis is a trend chart; read it for the trajectory of lifeExpectancy, which rises from 71.6 (1,990) to 83.4 (2,023).
05

What it checked

The accessibility audit uses Chartability checks. Adding a title and summary resolves the missing-text findings; the remaining warnings point to decisions that need review.

✓ No blocking failures11/14 critical heuristics pass
18pass2warn6verify0fail
  • understandable.axis-labels — Missing axis label: xLabel, yLabel. Ticks alone may not name the variable.
  • compromising.table-static — The data table is read-only — not downloadable, sortable, or filterable.
  • perceivable.low-contrast critical — Mark colors come from the theme/CSS variables — contrast can't be verified statically.
  • perceivable.seizure-risk critical — No flashing detected statically. Confirm no element flashes more than 3×/sec.
  • operable.interaction-cues critical — Chart is interactive and has descriptive text. Confirm that text explains how to interact (keyboard + pointer).
06

How it reads aloud

The structure a screen reader walks — chart → axes/series → values. Move through it with the arrow keys; the matching mark highlights on the chart, and hovering the chart moves this cursor.

  • ▾ A line chart of lifeExpectancy by year. lifeExpectancy ranges from 71.6 (1,990) to 83.5 (2,021), with a mean of 78.39 across 34 points. Overall lifeExpectancy rises from 71.6 (1,990) to 83.4 (2,023), after peaking at 83.5 (2,021).
    • X axis: year, 1,990 to 2,023.
    • Value axis: lifeExpectancy, 71.6 to 83.5.
    • 1,990: 71.6
    • 1,991: 72
    • 1,992: 72.5
    • 1,993: 73
    • 1,994: 73.4
    • 1,995: 73.7
    • 1,996: 74.2
    • 1,997: 74.6
    • 1,998: 75
    • 1,999: 75.4
    • 2,000: 75.9
    • 2,001: 76.4
    • 2,002: 76.8
    • 2,003: 77.2
    • 2,004: 77.7
    • 2,005: 78.2
    • 2,006: 78.7
    • 2,007: 79.1
    • 2,008: 79.5
    • 2,009: 80
    • 2,010: 80.1
    • 2,011: 80.6
    • 2,012: 80.8
    • 2,013: 81.3
    • 2,014: 81.7
    • 2,015: 82
    • 2,016: 82.3
    • 2,017: 82.6
    • 2,018: 82.6
    • 2,019: 83.2
    • 2,020: 83.4
    • 2,021: 83.5
    • 2,022: 82.7
    • 2,023: 83.4
Progressive disclosure

What travels with the chart

The chart still renders normally. An optional, versioned sidecar records what it claims, what supports those claims, when the evidence applies, and what must survive export.

Claims
5
Evidence records
2
Corrections
1
Current check
conditional
When a custom chart is the point

The chart can say why it looks the way it does.

A default bar chart can count shares. A waffle chart can also say that each cell is a unit of population, that the job is part-to-whole, and that a screen-reader path should hit the income groups before the hundred individual cells.

A

Waffle chart

clean proof
Skip to data table
One hundred cells normalize the total population represented in the sample; navigation and text aggregate those cells back into meaningful income groups.
High income12 cellsUpper middle income43 cellsLower middle income42 cellsLow income3 cells
01

Recipe manifest

The recipe records what the marks represent and how readers should navigate them.

{ "id": "semiotic.recipe.waffle.v0", "name": "Waffle chart", "frameFamily": "XYCustomChart", "portability": "portable", "dataRoles": [ { "role": "category", "field": "income", "accessor": "categoryAccessor", "required": true, "semanticType": "nominal", "description": "The group represented by a set of unit cells.", "source": "data" }, { "role": "value", "field": "population", "accessor": "valueAccessor", "required": true, "semanticType": "quantitative", "description": "The magnitude represented by the number of cells.", "source": "data" } ], "encodings": [ { "channel": "position", "role": "category", "meaning": "Contiguous cells are grouped spatially by category.", "redundantWith": [ "color", "category summary" ] }, { "channel": "count", "role": "value", "meaning": "Each visible unit represents one normalized portion of the total.", "redundantWith": [ "category share", "accessible table" ] }, { "channel": "color", "role": "category", "meaning": "Color distinguishes categories but is not the sole accessible cue.", "redundantWith": [ "position", "legend", "category label" ] } ], "intents": [ { "id": "part-to-whole", "strength": "primary", "score": 5, "rationale": "Repeated units directly expose how categories compose a total." }, { "id": "compare-categories", "strength": "secondary", "score": 3, "rationale": "Unit counts support broad comparisons, though bars remain more precise." }, { "id": "explanation", "strength": "secondary", "score": 5, "rationale": "Visible units make normalization and share easy to teach." } ], "reception": { "channels": [ "visual", "interactive", "screen-reader", "agent" ], "strengths": [ "memorable", "glanceable", "explainable", "teaching-friendly" ], "risks": [ "precise comparison may be harder than a bar chart", "individual cells can become navigation noise" ], "scaffolds": [ "legend", "category summary", "accessible table", "description" ], "memorableForm": true }, "designContract": { "whyCustom": "A unit grid makes composition more concrete and memorable than an abstract arc or line.", "whyThisForm": "A unit grid makes composition concrete through repeated, inspectable units.", "whyNotDefault": "A pie chart would show share but provide less unit-level evidence and less interaction surface.", "defaultAlternative": "BarChart", "tradeoff": "A bar chart supports more precise comparison; the waffle favors composition and memorability.", "misuse": [ "too many categories", "too many units", "false precision", "treating each cell as a separate semantic datum" ] }, "accessibility": { "keyboardNavigation": "required", "accessibleTable": "required", "description": "required", "navigationGranularity": "category", "dataBearingSceneNodes": "required", "fallbackTable": true, "requiresTitle": true, "requiresSummary": true, "requiresAccessibleTable": true, "minimumHitTarget": 24, "tableRoles": [ "category", "value" ], "tableFields": [ { "role": "category", "label": "Category" }, { "role": "value", "label": "Value", "format": "number" }, { "field": "share", "label": "Share", "format": "percent" } ], "redundantEncodings": [ "category position", "category summary", "legend" ], "requirements": [ "Announce category value, share, and allocated unit count.", "Aggregate navigation by category so repeated cells do not become meaningless noise." ] } }
02

Generated L1–L4 description

L1EncodingA 100-cell waffle chart showing population composition by income.
L2Statistics4 categories total 4.9B; each cell represents approximately 1% of the whole.
L3TrendUpper middle income is the largest category at 43%, represented by 43 of 100 cells.
L4IntentThis is an apportioning and explanatory chart: repeated units make the composition tangible while category summaries preserve the meaningful reading unit.
03

Navigation tree

Four category nodes summarize the 100-cell visual surface.

  • ▾ A 100-cell waffle chart showing population composition by income. 4 categories total 4.9B; each cell represents approximately 1% of the whole. Upper middle income is the largest category at 43%, represented by 43 of 100 cells. This is an apportioning and explanatory chart: repeated units make the composition tangible while category summaries preserve the meaningful reading unit.
    • High income: 601.8M, 12.3%, 12 of 100 cells.
    • Upper middle income: 2.1B, 43%, 43 of 100 cells.
    • Lower middle income: 2.1B, 42.1%, 42 of 100 cells.
    • Low income: 128.7M, 2.6%, 3 of 100 cells.
04

Observed-scene audit

The actual Waffle layout is run at this viewport and checked against its manifest.

16 observed checks pass100 scene nodes inspected

Declared semantics

Roles: category, value · intents:part-to-whole, compare-categories, explanation · fallback:declared

Observed evidence

  • geometry.finiteAll emitted geometry is finite.
  • geometry.nonzero-areaAll bounded marks have positive area.
  • geometry.boundsAll bounded marks stay inside the plot.
  • interaction.target-size100 targets are smaller than 24px in at least one dimension.
  • identity.stable-idsEvery data-bearing mark exposes a stable id.
  • identity.unique-idsStable mark ids are unique.
  • coverage.input-dataEvery input datum is represented by the observed scene.
  • coverage.scene-dataEvery data-bearing scene mark maps back to input data.
  • interaction.hit-targetsEvery observed data-bearing scene node participates in hit testing.
  • coverage.annotation-anchorsNo pointId annotations require scene resolution.
  • accessibility.table-fieldsThe accessible table preserves all recipe-declared fields.
  • accessibility.color-onlyEvery declared color encoding names a redundant cue.
  • accessibility.titleA chart title is present.
  • accessibility.summaryA chart summary is present.
  • accessibility.descriptionA chart description is present.
  • accessibility.fallback-tableThe accessible fallback table is enabled.
  • accessibility.navigation-depthThe navigation tree exposes structure below the root.
  • visual.contrast43 marks have fill contrast below 3:1.

Manual AT and reception checks

  • manual-at.screen-readerVerify real screen-reader behavior with the target browser/AT combinations.
  • manual-at.cognitive-loadAssess cognitive load and whether the custom metaphor is understood.
  • manual-at.keyboard-orderVerify that keyboard order is meaningful, not merely mechanically available.
  • manual-at.animationVerify that animation is not distracting and reduced-motion behavior is sufficient.
  • manual-at.receptionTest whether the memorable form is received as intended by the target audience.
  • manual-at.overlay-occlusionInspect labels, overlays, annotations, and important marks for clipping or occlusion.
  • manual-at.observationExercise hover, focus, selection, and observation emission in the rendered chart.
Why determinism is the point

Inspect the recommendation, then make your choice.

The recommendation engine makes its reasoning available for review. You can see the rules, compare candidates and select another chart when it better serves your purpose. Repeating the same input gives the same result, so changes are easier to investigate.

It knows its own limits

Use the tradeoffs to compare candidates.

One chart may show a small sample clearly; another may summarize it too aggressively. Bubble area may be harder to compare than length, and smoothing may hide outliers. The notes beside each recommendation help you decide which tradeoff matters for your question.

Core implementation

Profile, suggest, describe, audit, navigate

Five pure functions from semiotic/ai, composed. The engine reads capability descriptors that ship next to every chart; no network, no API key, no model.

JSX
import { useChartSuggestions, describeChart, auditAccessibility, buildNavigationTree, AccessibleNavTree, } from "semiotic/ai" // 1 · The engine profiles the data and ranks every chart's capability. const { suggestions, profile } = useChartSuggestions(data, { intent }) const top = suggestions[0] const Chart = COMPONENT_MAP[top.component] // top.props spreads straight in // 2 · It describes what the chart shows (Lundgard L1–L4). const description = describeChart(top.component, top.props, { capability: { family: top.family, intentScores: top.intentScores }, }) // 3 · Its own description, attached, is what clears the accessibility audit. const enriched = { ...top.props, title, description: description.text } const audit = auditAccessibility(top.component, enriched, { describe: true }) // 4 · And it lays out a path a screen reader can walk. const tree = buildNavigationTree(top.component, top.props) <Chart {...enriched} /> <AccessibleNavTree tree={tree} /> // Not one model call. Every line is a deterministic function of the data.

Real data: World Bank, World Development Indicators, retrieved 2026-06-30 — 16 countries spanning every World Bank region and income group, 544 country-year observations, 1990–2023. The chart recommendations, descriptions, accessibility audit, and navigation tree are computed entirely in your browser with no language model.