The Latent Crucible
LDA topic modeling × Word Trails · model notebook 02.1
The LatentCrucible
A topic model starts with words and guesses which tend to belong together. In this example, documents can mix topics, and each topic gives different weights to words. Step through the sampler to watch those guesses change. The colors identify learned groups; their meaning comes from inspecting the words, not from a label supplied in advance.
“Tools are enshrined methodologies.”
Elijah Meeks and Scott B. Weingart, co-guest editors,Journal of Digital Humanities 2.1
The model’s wager
LDA tells a generative story and inference runs that story backward. The computer sees the words below and repeatedly asks which latent assignment would be plausible given all the other assignments.
Mix each topic
Every topic receives probabilities over the same vocabulary.
Mix a document
Each document receives proportions over the same fixed set of topics.
Choose a topic
For each token position, choose one latent topic from that document’s mixture.
Choose a word
Each topic is itself a probability distribution over the vocabulary.
A deliberately tiny corpus
This miniature is small enough to inspect token by token. It is a seeded teaching run, not a result about literature, politics, or any other world outside this notebook.
The map makes an argument
mapselectsplaceborderscalelayermakesspatialargumentroutecrossesbordermapframesregionhidesanotherplace
Projection and scale
projectionbendsdistancemapscalechangesregionbordervisibleroutespatiallayerscompareplaceanotherplacemakedistancedebatable
Railway time
railwaystationjoinsroutedistancetimenetworkordersstationcarriestravelersacrossregionmapturnsmovementlinesplaces
The archive shelf
archivepreservesletterpagecatalogrecordcollectionordersdocumentshelfeditorreadsmarginrestoresmissingpage
A newspaper edition
newspaperprintseditionreaderspagecarriesarticledaterecordarchivebindsprintcollectionreaderssearch
The editorial margin
editorassemblesissueessayarticlereviewscholarsannotatewordsmarginjournalbecomesarchiveargumentspageschangingeditions
A network of letters
lettermovessenderrecipientalongpostalrouterepeatedcorrespondencemakesnetworkarchiverecordsaddressdateconnectionjourney
A citation path
citationconnectsarticlearticlescholarscholarnetworkrevealsclustersbridgespathsjournaleditorfollowlinkacrossissues
Documents as mixtures
topicmodeltreatsdocumentmixturetopicgivesprobabilitywordcountsconnectdocumenttopicworddeclaringsinglepermanentmeaning
Inside a Gibbs sweep
samplerremovestokenweighstopicdrawsnewassignmentiterationchangescountsrepeatedsamplingletstopicdocumentmixturescongealnaming
Modeling an archive
scholarturnsarchivepagescorpusmodelcountsworddocumenttopicsofferpatternsreadingeditorreturnscollectiontestspattern
A spatial model
modelcomparesmaplayersdocumentsplacewordsgathertopicsroutescrosstopicboundariesprobabilitylinksscaleregiondocumentspatialpatternerasingambiguity
Run the sampler
One R is one complete Gibbs sweep: visit every retained token, temporarily remove its assignment, calculate its conditional topic probabilities, and draw a replacement. The recorded R values are front-loaded—0, 1, 2, 4—so early turbulence remains inspectable before the trace takes longer jumps.
Sampling · R=0
p(z_i=k | z_-i,w) ∝ (n_dk^-i + α)(n_kw^-i + β)/(n_k^-i + Vβ)In plain language: the document’s affinity for a topic × the word’s affinity for that topic ÷ the topic’s size, with α and β smoothing the counts; then normalize across K and sample.
Watch the topics congeal
Each Word Trails column is a topic. The vertical axis is not historical time: it is R, the sampler’s iteration count. A word’s size is its estimated probability at that checkpoint. Hue identifies the topic; tint measures topic exclusivity independently of time. Divide that word’s probability in this topic by its summed probability across all four topics, subtract the uniform 25% baseline, and rescale the remainder from zero to one. The result is binned into neutral ink, 25%, 50%, or 100% of the column color. Neutral is white on the dark plot and dark ink on the light plot, so shared words remain legible in either theme. Opacity carries recency: the active checkpoint is fully present, fading linearly to 25% for the oldest visible R. Recorded checkpoints are evenly spaced for legibility, while their labels retain the actual R; no unrecorded intermediate states are implied.
Scroll sideways to inspect all four topic columns.
A word can appear in several topics because each topic assigns probabilities to the same vocabulary. Font size shows its probability within a topic, and tint helps compare that word across topics. New checkpoints fill the reserved rows as the sampler advances.
Read both distributions
LDA’s output has two complementary faces. The cards rank words within each topic (φ, “phi”); the bars divide each document across topics (θ, “theta”). Neither face is adequately summarized by assigning one label to one document.
Topic I
- topic4.2%
- changes2.9%
- map2.9%
- orders2.9%
- scale2.9%
- scholar2.9%
- across1.5%
Anonymous topic; its terminal high-probability words are document, topic, counts.
Topic II
- article3.0%
- network3.0%
- route3.0%
- spatial3.0%
- ambiguity1.6%
- annotate1.6%
- another1.6%
Anonymous topic; its terminal high-probability words are article, scholar, journal.
Topic III
- archive4.3%
- place4.3%
- editor2.9%
- topic2.9%
- across1.6%
- along1.6%
- article1.6%
Anonymous topic; its terminal high-probability words are archive, editor, collection.
Topic IV
- document5.4%
- date2.8%
- editor2.8%
- model2.8%
- region2.8%
- repeated2.8%
- argument1.5%
Anonymous topic; its terminal high-probability words are place, map, border.
Quench the final assignment
The Crucible below does not run Gibbs sampling. It receives the recorded word-topic counts from R=64 and pours those allocations into final topic products. Before the final checkpoint it stays charged but unpoured, making the boundary between inference and presentation explicit. Its replay is a separate three-beat presentation—recorded charge, allocate fixed counts, quench: restarting the sampler is neither required nor implied.
Six notes against enchantment
Coherent-looking columns arrive quickly; warranted interpretation does not. Keep these six penciled objections beside every polished topic display.
We chose the number of topics
The model did not discover that this corpus contains exactly 4subjects. K was fixed before inference, and another K would make another account.
Word order is absent
Standard LDA treats a document as an exchangeable bag of tokens. Syntax, negation, voice, irony, and sequence do not enter this likelihood.
Priors exert pressure
Dirichlet hyperparameters reward some kinds of sparsity and mixture. Tokenization, stopword removal, vocabulary limits, and the random seed also shape the result.
Stability is not truth
Fewer moving assignments suggest this chain is settling. They do not certify that its topics are unique, meaningful, or adequate to a scholarly question.
Labels are annotations
These topic numbers are anonymous exchangeable handles. Their terse interpretations were added after inspecting high-weight words; LDA does not name concepts.
The display is a top slice
Each topic assigns probability to the whole retained vocabulary. Word Trails shows a legible leading subset, while the complete count state remains in the model.