Recipes — Kstreams — Semiotic

Home · Getting Started · Charts · Features · Playground

This page requires JavaScript for interactive chart demos.

AI / Machine-readable docs: llms.txt · llms-full.txt · CLAUDE.md · schema.json · API Reference · Examples

View on GitHub · npx semiotic-ai for CLI access

Machine-readable page content

Canonical: https://semiotic.nteract.io/recipes/kstreams

Kafka Streams Topology

Recipe

semioticreact

A left-to-right layered DAG of a Kafka Streams application's topology — topics and processors as composite node glyphs, repartition bridges, a state-store-bearing aggregate, and a cycle rendered as a back-edge. It's built onNetworkCustomChart + the lineageDagLayout recipe (thecustomNetworkLayout escape hatch), which reads pre-computed logical layer/row coordinates and maps them into the plot — no force simulation, fully deterministic.

Preview

Root topic:Display:Snapshot v1

Downstream lineage from orders· hover to preview reach · click to select · scroll for wide graphs

Topology overview

Whole app · click a topic to re-root · shares hover/selection with the main view

Click a node to inspect its partition, semantic, and attached state stores.

Composite glyphs as one hit-testable unit

Each node is more than a shape + a label: a partition-colored container, a semantic icon, a truncated name, a type label, and one chip per attached state store. The recipe emitsexactly one rect scene node per node— that single mark owns the canvas hit area and carries the datum — and draws all the chrome in the layout's overlays layer, which is pointer-events: none. So the rich glyph never intercepts a hover: hover and click always resolve to the node as a unit. As the graph gets denser the recipe drops to a compact glyph, then an icon, then a 5px dot (the minimap) — a level-of-detail ramp driven by the fitted size.

Controlled interaction, owned by the host

Selection and hover state live in the page, not the chart — exactly the controlled model a lineage viewer needs:

Hover → reach preview. Hovering a node hands the host a downstream-reachable id set; it passes that set as layoutConfig.reachableIds, and the recipe dims every node and edge outside it. The set is computed once and fed toboth the main view and the minimap, so hovering in either dims both.
Click → selection. A click fires onClick; the host records the id and pushes it into a shared selection store. Both charts consume that store through their selection prop, so the node rings in both views. The frame never owns selection — it only reflects it.
Re-root. Clicking a topic in the minimap sets it as the new expansion root; the main view re-renders the new downstream subgraph with no stale positions.

Two synchronized views via LinkedCharts

The main view and the minimap are two independent NetworkCustomChart instances wrapped in LinkedCharts. The selection a click produces is keyed on the nodeid and flows through the shared store toNetworkLayoutContext.selection — the recipe reads that predicate to highlight the matching node. This is the proof that the coordinated-views layer works in aStreamNetworkFrame custom-layout context, not just for the built-in chart types.

Snapshot morph

The Morph to snapshot control tweens the lineage between two versions of the topology — nodes that persist slide to their new layered positions, a newly-addedKTABLE-SUPPRESS stage enters, and a retired processor leaves. Because the layout is a pure function of the data, a topology diff is just two layouts to interpolate. This gestures at the temporal/animated-topology direction that motivates the migration.

Repartition bridges & cycles

Re-keying before a stateful operator inserts a repartition bridge — a sink to an internal …-repartition topic in one sub-topology, re-read by a source in the next. A reconciliation path that feeds back upstream closes a cycle; the layout flags that edge isBackEdge and the recipe renders it as a distinct dashed loop bowing below the forward flow.

Customization

What	Where	How
Node glyph / icon	`layoutConfig.renderIcon`	Return any SVG; receives `{ semantic, partition, size, color }`
Partition colors	`layoutConfig.partitionColors`	Map a partition to a fill
Reach dimming	`layoutConfig.reachableIds`	A host-computed id set; everything outside dims
Level of detail	`layoutConfig.lod`	`"auto"` (default), or force `full`/`compact`/`icon`/`dot`
Store chips	`layoutConfig.showStoreChips`	One chip per attached state store, in `full` LOD

Accessibility & performance

The canvas pipeline provides the chart-level affordances today: a role="img"container with an aria-label, an optional accessibleTable summary of the scene, keyboard spatial navigation, and a focus ring. Per-node accessible names, roles, and pressed/selected state reaching assistive tech is the production gap — the overlay-glyph approach is the path there, since the overlay layer can host real focusable DOM. The layout is a pure deterministic function and the catalog graph (~95 nodes) lays out and renders well within a frame; a synthetic ~1,000-node graph is the right next stress test for the canvas + quadtree substrate.

Custom Charts — the customLayout escape hatch
Network Explorer — force-directed interaction patterns
StreamNetworkFrame — the underlying frame

Source Code

Abbreviated for readability — data arrays and steps are truncated. See the full runnable source in docs/src/examples/recipes/.

JSX

1 import { NetworkCustomChart } from "semiotic/network"
2 import { LinkedCharts, useSelection } from "semiotic"
3 import { lineageDagLayout } from "semiotic/recipes"
4 // Your domain pipeline owns layout; Semiotic only *reads* the result:
5 import { dagLayoutFromGraph, reachableFrom, subgraphFrom } from "./kstreamsPipeline"
6 
7 // The recipe stays domain-agnostic — the app supplies the glyph vocabulary.
8 const PARTITION_COLORS = {
9  "topic-source": "#1f8a70", "topic-sink": "#c0552d",
10  "topic-bridge": "#3f5e8c", processor: "#3a3a52",
11 }
12 function renderKstreamsIcon({ semantic, partition, size, color }) {
13  // topic → a log/database glyph; processor → a colored chip + semantic symbol.
14  return /* svg */ null
15 }
16 
17 function toChartData(layout) {
18  return {
19  // logical x = layer, y = row — the recipe maps these into the plot.
20  nodes: layout.nodes.map((dn) => ({
21  id: dn.id, x: dn.x, y: dn.y,
22  partition: dn.node.partition, semantic: dn.node.semantic,
23  label: dn.node.label, stores: dn.node.stores,
24  })),
25  edges: layout.edges.map((de) => ({
26  id: de.id, source: de.source, target: de.target,
27  edgeType: de.edgeType, isBackEdge: de.isBackEdge,
28  })),
29  }
30 }
31 
32 function LineageViews() {
33  const [rootId, setRootId] = useState("orders")
34  const [selectedId, setSelectedId] = useState(null)
35  const [hoveredId, setHoveredId] = useState(null)
36 
37  // Shared selection store — click emits here; both charts consume it.
38  const sel = useSelection({ name: "kstreams", fields: ["id"] })
39 
40  const main = useMemo(() => toChartData(dagLayoutFromGraph(subgraphFrom(full, rootId))), [rootId])
41  const mini = useMemo(() => toChartData(dagLayoutFromGraph(full)), [])
42  // Hover → downstream reach set → dim everything else, in BOTH views.
43  const reachableIds = hoveredId ? [...reachableFrom(full, hoveredId)] : null
44 
45  const onHover = (obs) => setHoveredId(obs.type === "hover" ? obs.datum?.id : null)
46  const onPick = (d) => { setSelectedId(d.id); sel.selectPoints({ id: [d.id] }) }
47 
48  return (
49  <LinkedCharts>
50  <NetworkCustomChart
51  nodes={main.nodes} edges={main.edges} layout={lineageDagLayout}
52  layoutConfig={{
53  layerCount: ..., maxLayerSize: ..., reachableIds, selectedId,
54  renderIcon: renderKstreamsIcon, partitionColors: PARTITION_COLORS,
55  }}
56  selection={{ name: "kstreams" }} // consume the shared store → ring
57  onObservation={onHover} // hover → reach preview
58  onClick={onPick} // click → select (+ emit to store)
59  width={680} height={460}
60  />
61  {/* A second NetworkCustomChart with lod:"dot" is the minimap, sharing the
62  same selection store — proving LinkedCharts works for custom layouts. */}
63  </LinkedCharts>
64  )
65 }