The Latent Crucible — Semiotic

Home · Getting Started · Charts · Features · Playground

This page requires JavaScript for interactive chart demos.

AI / Machine-readable docs: llms.txt · llms-full.txt · CLAUDE.md · schema.json · API Reference · Examples

View on GitHub · npx semiotic-ai for CLI access

Machine-readable page content

Canonical: https://semiotic.nteract.io/examples/latent-crucible

The Latent Crucible

LDA topic modeling × Word Trails · model notebook 02.1

The LatentCrucible

We observe the words. We do not observe the topics. LDA proposes that documents are mixtures, topics are distributions, and every token is a small act of probabilistic allegiance. Then inference begins the long business of revising its guesses.

“Tools are enshrined methodologies.”
Elijah Meeks and Scott B. Weingart, co-guest editors,Journal of Digital Humanities 2.1

The model’s wager

LDA tells a generative story and inference runs that story backward. The computer sees the words below and repeatedly asks which latent assignment would be plausible given all the other assignments.

φ_k ∼ Dirichlet(β)

Mix each topic

Every topic receives probabilities over the same vocabulary.

θ_d ∼ Dirichlet(α)

Mix a document

Each document receives proportions over the same fixed set of topics.

z_dn ∼ Categorical(θ_d)

Choose a topic

For each token position, choose one latent topic from that document’s mixture.

w_dn ∼ Categorical(φ_z)

Choose a word

Each topic is itself a probability distribution over the vocabulary.

Forward story: proportions → topic → word. Our task:words → plausible topic assignments and proportions. The arrow reverses; the uncertainty does not disappear.

A deliberately tiny corpus

This miniature is small enough to inspect token by token. It is a seeded teaching run, not a result about literature, politics, or any other world outside this notebook.

DOC 01

The map makes an argument

mapselectsplaceborderscalelayermakesspatialargumentroutecrossesbordermapframesregionhidesanotherplace

DOC 02

Projection and scale

projectionbendsdistancemapscalechangesregionbordervisibleroutespatiallayerscompareplaceanotherplacemakedistancedebatable

DOC 03

Railway time

railwaystationjoinsroutedistancetimenetworkordersstationcarriestravelersacrossregionmapturnsmovementlinesplaces

DOC 04

The archive shelf

archivepreservesletterpagecatalogrecordcollectionordersdocumentshelfeditorreadsmarginrestoresmissingpage

DOC 05

A newspaper edition

newspaperprintseditionreaderspagecarriesarticledaterecordarchivebindsprintcollectionreaderssearch

DOC 06

The editorial margin

editorassemblesissueessayarticlereviewscholarsannotatewordsmarginjournalbecomesarchiveargumentspageschangingeditions

DOC 07

A network of letters

lettermovessenderrecipientalongpostalrouterepeatedcorrespondencemakesnetworkarchiverecordsaddressdateconnectionjourney

DOC 08

A citation path

citationconnectsarticlearticlescholarscholarnetworkrevealsclustersbridgespathsjournaleditorfollowlinkacrossissues

DOC 09

Documents as mixtures

topicmodeltreatsdocumentmixturetopicgivesprobabilitywordcountsconnectdocumenttopicworddeclaringsinglepermanentmeaning

DOC 10

Inside a Gibbs sweep

samplerremovestokenweighstopicdrawsnewassignmentiterationchangescountsrepeatedsamplingletstopicdocumentmixturescongealnaming

DOC 11

Modeling an archive

scholarturnsarchivepagescorpusmodelcountsworddocumenttopicsofferpatternsreadingeditorreturnscollectiontestspattern

DOC 12

A spatial model

modelcomparesmaplayersdocumentsplacewordsgathertopicsroutescrosstopicboundariesprobabilitylinksscaleregiondocumentspatialpatternerasingambiguity

Run the sampler

One R is one complete Gibbs sweep: visit every retained token, temporarily remove its assignment, calculate its conditional topic probabilities, and draw a replacement. The recorded R values are front-loaded—0, 1, 2, 4—so early turbulence remains inspectable before the trace takes longer jumps.

ACTIVE CHECKPOINTR = 0

Pace

Model iteration checkpoint

Sampling · R=0

THE WEIGHING RULEp(z_i=k | z_-i,w) ∝ (n_dk^-i + α)(n_kw^-i + β)/(n_k^-i + Vβ)

In plain language: the document’s affinity for a topic × the word’s affinity for that topic ÷ the topic’s size, with α and β smoothing the counts; then normalize across K and sample.

assignments changed0seeded initial charge

moves before sampling0no Gibbs visits yet

training reconstruction proxy110.21in-sample description, not a truth meter

Watch the topics congeal

Each Word Trails column is a topic. The vertical axis is not historical time: it is R, the sampler’s iteration count. A word’s size is its estimated probability at that checkpoint. Hue identifies the topic; tint measures topic exclusivity independently of time. Divide that word’s probability in this topic by its summed probability across all four topics, subtract the uniform 25% baseline, and rescale the remainder from zero to one. The result is binned into neutral ink, 25%, 50%, or 100% of the column color. Neutral is white on the dark plot and dark ink on the light plot, so shared words remain legible in either theme. Opacity carries recency: the active checkpoint is fully present, fading linearly to 25% for the oldest visible R. Recorded checkpoints are evenly spaced for legibility, while their labels retain the actual R; no unrecorded intermediate states are implied.

Topic ITopic IITopic IIITopic IV

EXCLUSIVITYneutral/shared → topic-exclusive

RECENCYoldest visible R → current R

Scroll sideways to inspect all four topic columns.

RECORDED SAMPLER TRACE / TOP-WORD SLICE

A word type may appear in several columns because topics are distributions over a shared vocabulary. Repetition is not a bug to tidy away; it is the model. Unreached checkpoints reserve layout space but emit no glyph or hit target, so earlier words do not jump when the next R appears. Tint compares the same word across topics; size remains its within-topic probability.

Read both distributions

LDA’s output has two complementary faces. The cards rank words within each topic (φ, “phi”); the bars divide each document across topics (θ, “theta”). Neither face is adequately summarized by assigning one label to one document.

TOPIC 1

Topic I

topic4.2%
changes2.9%
map2.9%
orders2.9%
scale2.9%
scholar2.9%
across1.5%

Anonymous topic; its terminal high-probability words are document, topic, counts.

TOPIC 2

Topic II

article3.0%
network3.0%
route3.0%
spatial3.0%
ambiguity1.6%
annotate1.6%
another1.6%

Anonymous topic; its terminal high-probability words are article, scholar, journal.

TOPIC 3

Topic III

archive4.3%
place4.3%
editor2.9%
topic2.9%
across1.6%
along1.6%
article1.6%

Anonymous topic; its terminal high-probability words are archive, editor, collection.

TOPIC 4

Topic IV

document5.4%
date2.8%
editor2.8%
model2.8%
region2.8%
repeated2.8%
argument1.5%

Anonymous topic; its terminal high-probability words are place, map, border.

Topic ITopic IITopic IIITopic IV

The map makes an argumentTopic I

Projection and scaleTopic II

Railway timeTopic I

The archive shelfTopic IV

A newspaper editionTopic IV

The editorial marginTopic II

A network of lettersTopic II

A citation pathTopic III

Documents as mixturesTopic IV

Inside a Gibbs sweepTopic III

Modeling an archiveTopic III

A spatial modelTopic IV

Quench the final assignment

The Crucible below does not run Gibbs sampling. It receives the recorded word-topic counts from R=64 and pours those allocations into final topic products. Before the final checkpoint it stays charged but unpoured, making the boundary between inference and presentation explicit. Its replay is a separate three-beat presentation—recorded charge, allocate fixed counts, quench: restarting the sampler is neither required nor implied.

Available when the sampler reaches R=64.

FINAL POUR HELDContinue to R=64

147 source components; 0 products; 0 token occurrences in products. Outcome: in_process.

COMPUTATION collapsed Gibbs samplerPROJECTION bounded Crucible event tape

Six notes against enchantment

Coherent-looking columns arrive quickly; warranted interpretation does not. Keep these six penciled objections beside every polished topic display.

We chose the number of topics

The model did not discover that this corpus contains exactly 4subjects. K was fixed before inference, and another K would make another account.

Word order is absent

Standard LDA treats a document as an exchangeable bag of tokens. Syntax, negation, voice, irony, and sequence do not enter this likelihood.

Priors exert pressure

Dirichlet hyperparameters reward some kinds of sparsity and mixture. Tokenization, stopword removal, vocabulary limits, and the random seed also shape the result.

Stability is not truth

Fewer moving assignments suggest this chain is settling. They do not certify that its topics are unique, meaningful, or adequate to a scholarly question.

Labels are annotations

These topic numbers are anonymous exchangeable handles. Their terse interpretations were added after inspecting high-weight words; LDA does not name concepts.

The display is a top slice

Each topic assigns probability to the whole retained vocabulary. Word Trails shows a legible leading subset, while the complete count state remains in the model.