Diagnostics

Model-agnostic quality, interpretation, and validation tools. They take any fitted model's topic_word / doc_topic (or raw arrays), so they work the same across every model family. They live in the topica.evaluate namespace (topica.evaluate.<name>, the documented path below); every name is also reachable bare at the top level (topica.<name>) as a compatibility alias.

One-call table

topica.evaluate.diagnostics

diagnostics(model, texts=None, *, n=10, coherence_type=None, stability=False, n_boot=20, model_factory=None, seed=0)

One per-topic diagnostics table for a fitted model.

Consolidates the quality numbers people otherwise gather one function at a time — coherence, exclusivity, FREX words, size, prevalence, top words, and (optionally) bootstrap stability — into a single row-per-topic table. It reads a model's analysis surface, so it works for every model and you never pass a raw matrix where a model is wanted, or vice versa.

Parameters:

Name	Type	Description	Default
model	a fitted topica model.		required
texts	the reference corpus for windowed coherence (a ``Corpus``, raw	strings, or token lists). Without it, coherence falls back to the model's own UMass score. Required when stability=True.	None
n	top-word count used for coherence, exclusivity, FREX, and the word lists.		10
coherence_type	override the coherence metric (``"c_v"`` default when	texts is given, "u_mass" otherwise).	None
stability	also report per-topic bootstrap stability (mean top-word Jaccard	over n_boot refits, matched back to this model). Off by default since it refits the model; needs texts (the documents) to resample.	False
model_factory	``callable(seed) -> unfitted model`` for the stability refits;	defaults to rebuilding the model's own type as type(model)(num_topics=K, seed=seed). Pass your own for models whose constructor needs more.	None

Returns:

Type	Description
A pandas ``DataFrame`` indexed by topic (columns: ``label``, ``size``,
``prevalence``, ``coherence``, ``exclusivity``, ``stability``, ``top_words``,
``frex``), or a list of row dicts when pandas is not installed.

topica.evaluate.perplexity

perplexity(model, held_out, *, seed=0)

Document-completion held-out perplexity for a generative model.

For each held-out document, half its tokens (even positions) estimate the document's topic mixture through the model's transform, and the other half (odd positions) are scored under that mixture, p(w) = sum_k theta_k * topic_word[k, w]. Returns exp(-sum log p / N_eval); lower is better.

Because the scored tokens are held out from the mixture estimate, this does not trivially fall as K grows the way in-sample likelihood does, so it is a fair quantity to compare across K when justifying a topic count. It works for any model with a generative transform(documents) and a topic_word distribution (LDA, DMR, CTM, STM, HDP, keyATM, ...). The embedding-cluster models have no document likelihood; compare those with coherence or diversity.

(LDA additionally offers the more rigorous Wallach et al. left-to-right estimator as LDA.perplexity / LDA.evaluate.)

Parameters:

Name	Type	Description	Default
model	a fitted generative model.		required
held_out	documents the model was not trained on (token lists or a ``Corpus``).		required
seed	RNG seed for the Gibbs ``transform`` (ignored by the variational models).		0

topica.evaluate.reply_completion

reply_completion(docs, parents, *, num_topics, covariates=None, covariate_names=None, heldout_frac=0.5, min_eval_tokens=2, eval_frac=1.0, baselines=('no_tree', 'permuted'), contrasts=None, em_iters=100, min_count=1, seed=13, n_boot=1000, predictive_samples=400, keyatm_keywords=None, keyatm_weights='information-theory', rtm_links='thread', rtm_max_links=20000)

Held-out leaf-token comparison of ThreadTM against matched baselines.

This is the turnkey preference test for ThreadTM: does the reply tree add predictive information on real data, and does the gain come from the observed edge? It fits the matched models named in baselines (the tree plus up to eight comparators) on the SAME reduced corpus (identical vocabulary, num_topics, min_count, and seed) and scores held-out tokens of short leaf comments under each.

For the tree-attributable gain read delta["no_tree"]: it is the same model with the reply tree switched off, so it isolates the tree from the rest of the modeling stack. delta["lda"] / delta["stm"] instead answer the "why not just an off-the-shelf tool" question and fold in every difference (Dirichlet vs logistic-normal, covariate anchors, the estimator), not the tree alone; delta["keyatm"] and delta["rtm"] (issue #860) answer it for the two tools a reviewer reaches for next — a keyword-assisted model on STM's supervision axis, and the nearest structural neighbor, a document-link model.

Protocol. We select non-root leaf comments (a reply with no replies of its own) that have at least min_eval_tokens tokens, and for each we hold out a heldout_frac share of its tokens. Each model is then fit on the corpus with those tokens removed, so the held-out tokens never influence any fit. For each held-out token w in leaf d we score log(sum_k theta[d, k] * topic_word[k, w]) under that model's fitted theta and topic_word, and average per token. To keep the estimator fair across models (issue #838), a logistic-normal model (ThreadTM and the STM baseline) is scored with the posterior-predictive E[softmax(η)] (a Monte-Carlo average of predictive_samples draws from its own η posterior), not the plug-in softmax(mean η) that doc_topic returns. The plug-in is an overconfident point estimate that ignores the posterior variance ν, so it is sharpest on exactly the thin leaves that are the eval targets, whereas LDA's doc_topic is an already-averaged (hedged) posterior mean; matching the two estimators keeps delta["lda"] a model comparison rather than an estimator artifact (on the real corpora of issue #838 this closed most of the apparent LDA gap). Because a leaf's theta is inferred from its (few) seen tokens plus its prior, a tree that couples the prior to the parent should predict thin leaves better.

The models:

• tree: ThreadTM with the true reply tree.
• no_tree: ThreadTM with every document a root (parents = -1), a logistic-normal baseline with the same covariate anchors but no tree coupling. This is the model-versus-model comparator.
• permuted: ThreadTM with a depth-stratified within-thread parent permutation (the placebo). If the gain is real it should shrink here.
• root (issue #831): ThreadTM whose prior shrinks each node toward its THREAD ROOT instead of its immediate parent (a broadcast / topic-around-the- root structure). delta["root"] is parent-coupling minus root-coupling, so it is positive where the reply edge matters more than the thread topic and negative where the thread root is the operative structure (sports, fandom).
• blend (issue #831): ThreadTM that couples each node to BOTH its parent and its thread root (alpha*parent + beta*root + (1-alpha-beta)*anchor), with the mix estimated. delta["blend"] is parent-coupling minus blend-coupling; a negative value means the blend of edge and thread structure predicts better than the reply edge alone.
• lda / stm (issue #828): off-the-shelf comparators — a plain LDA(K), and an STM(K) with the covariates one-hot encoded as prevalence — fit on the same reduced corpus (pinned to the tree model's vocabulary) and scored through the identical leaf mask and fit-time-theta protocol, so delta["lda"]/delta["stm"] are the tree-minus-tool difference with the same thread-clustered interval. stm requires a covariate with at least two groups.
• keyatm (issue #860): keyATM, the keyword-assisted model. With no keyatm_keywords it is keyATM's own weightedLDA — keyword-free, but with keyATM's token weighting and estimated asymmetric alpha, so it is a distinct model from the plain lda baseline rather than a second copy of it. Pass keyatm_keywords for the seeded model. When covariates has at least two groups it is fit as the COVARIATE keyATM on the same one-hot design stm gets (a DMR document-topic prior), which is what puts it on STM's supervision/covariate axis; without a usable covariate it is the base model. Read delta["keyatm"] with keyatm_weights in mind: under keyATM's own information-theory weighting its theta is a weighted-count plug-in that is near one-hot on a short leaf, so part of the gap is that sharpness rather than the reply tree.
• rtm (issue #860): the Relational Topic Model, the nearest structural neighbor to ThreadTM — it models document LINKS, but generic undirected ones, with no directed parent-conditional prior. rtm_links chooses the graph it sees; the default is reply-BLIND intra-thread co-membership (which comments share a conversation, without saying which pairs are replies), so delta["rtm"] reads as the reply edge's gain over a generic link model given thread structure. Use rtm_links="reply" for the complementary read: the same edges undirected, so the only thing left between the two models is the directed, parent-conditional prior.

Aggregation clusters on the thread root, not the comment, because comments within a thread are correlated. The paired difference (tree minus baseline) is bootstrapped over threads, so the interval reflects the number of independent conversations, not the number of tokens.

Parameters:

Name	Type	Description	Default
docs	list of token lists, or a ``topica.Corpus``.		required
parents	list of int. ``parents[d]`` is ``d``'s parent document index, or	-1 for a thread root, in the same order as docs.	required
num_topics	int. K, shared by all three models.		required
covariates	optional per-document categorical group ids	(dense 0..num_groups) and their names, passed through to every fit so the anchors match.	None
covariate_names	optional per-document categorical group ids	(dense 0..num_groups) and their names, passed through to every fit so the anchors match.	None
heldout_frac	float. Share of each eval leaf's tokens held out (the rest	are seen by the fit). At least one seen and one held-out token are kept.	0.5
min_eval_tokens	int. Minimum tokens for a leaf to be eligible (``>= 2``).		2
eval_frac	float. Share of eligible leaves to evaluate (sampled with	seed); 1.0 uses them all.	1.0
baselines	which comparators to fit, any of ``"no_tree"``, ``"permuted"``,	"root", "blend", "lda", "stm", "keyatm", "rtm".	('no_tree', 'permuted')
contrasts	optional paired *baseline-vs-baseline* contrasts to report with a	thread-clustered CI (issue #852), in addition to the tree-minus-baseline delta. Each entry is either a (first, second) pair of scored-model names (giving a CI on first - second) or the string alias "edge" (the edge-attribution quantity permuted - no_tree: how much of a reply's held-out predictability comes from the observed parent, above a within-thread parent permutation). "edge" is computed automatically whenever both permuted and no_tree are scored, so most callers do not need to pass it. The intervals reuse the same root-resampling bootstrap as delta; the raw paired draws are also on the result (result.paired) and :meth:ReplyCompletionResult.contrast_ci forms an arbitrary pair on demand.	None
em_iters	EM iterations for the ThreadTM fits (tree, no_tree, permuted, root, blend). Match	this to the analysis fit. The off-the-shelf lda / stm / keyatm / rtm comparators run at their own default iteration counts, not em_iters.	100
min_count	words rarer than this are dropped (shared across models).		1
seed	RNG seed for leaf sampling, the token split, the permutation, the	bootstrap, and the posterior-predictive theta draws; also the model seed.	13
n_boot	thread-clustered bootstrap resamples for the interval.		1000
predictive_samples	int. Monte-Carlo draws used for the posterior-predictive	E[softmax(η)] that scores each logistic-normal model (see Notes). The default (400) matches the estimate that validated the fix in issue #838; a much smaller value makes the scored likelihoods noisy.	400
keyatm_keywords	optional ``{topic_name: [keyword, ...]}`` dictionary for the	keyatm baseline (issue #860). The default (None) fits keyATM's keyword-free weightedLDA, so the baseline runs turnkey like lda; pass a dictionary for the seeded model (keywords absent from the reduced vocabulary are dropped by keyATM). It is an error to pass this without "keyatm" in baselines.	None
keyatm_weights	keyATM's token weighting for the ``keyatm`` baseline:	"information-theory" (the default, keyATM's own), "inv-freq", or "none". This is not a cosmetic knob here. keyATM's theta is (weighted counts + alpha) / total, and the information-theory weights multiply a token's count by its surprisal (~10x on a small vocabulary), which swamps the prior and leaves a near one-hot theta on a short leaf — the same overconfidence issue #838 fixed for the logistic-normal models, except that here it is baked into the model's weighting rather than into the estimator, so no posterior-predictive average undoes it. Keep the default for keyATM as published (and expect part of delta["keyatm"] to be that sharpness on thin leaves); pass "none" for an unweighted, estimator-matched keyATM whose theta hedges like LDA's. Reporting both is the honest read.	'information-theory'
rtm_links	the document graph the ``rtm`` baseline is fit on (issue #860).	"thread" (default) is reply-blind intra-thread co-membership: every pair of comments in the same thread, so RTM sees conversation membership without being told which pairs are replies (a reply pair is in the graph, but as one unlabeled, undirected edge among the thread's others; settings["rtm_reply_share"] reports what fraction of the fitted links are reply pairs, and the fit warns when shallow threads push that past a half). "none" fits RTM with an empty link set (its link model is then inert, which makes it close to a second lda). "reply" hands it the parent-child pairs with their direction dropped — NOT reply-blind, but the sharpest isolation of ThreadTM's contribution: the same edges under a generic symmetric link model instead of a directed parent-conditional prior. An explicit sequence of (i, j) document-index pairs is also accepted. The resolved choice and the fitted link count are reported in settings (rtm_links, rtm_n_links).	'thread'
rtm_max_links	int or None. Cap on the ``"thread"`` co-membership graph, whose	size grows with the square of thread length. Above it the pairs are thinned uniformly (each thread keeping its share) under seed, with a warning; None disables the cap. Ignored for the other rtm_links modes.	20000

Returns:

Type	Description
ReplyCompletionResult

Notes

Requires topica.enable_experimental() (ThreadTM is experimental). Held-out tokens whose word never appears in the reduced training corpus are out of vocabulary and are dropped from scoring (counted in oov_dropped), as in :func:perplexity.

topica.evaluate.thread_stability

thread_stability(docs, parents, *, num_topics, covariates=None, covariate_names=None, n_boot=20, seed=13, em_iters=100, min_count=1, coupling='parent', seed_words=None, metric='cosine', stable_threshold=0.7, ci=0.95)

Thread-bootstrap robustness for :class:~topica.ThreadTM.

ThreadTM's fit is deterministic given its inputs (the variational EM starts from a fixed spectral init), so refitting across seed values does NOT perturb it — a multi-seed "stability" check is a silent no-op (issue #856). The right question for a threaded corpus is instead: are my topics and the per-group prevalence stable to which conversations I happened to sample? This resamples whole reply trees (thread roots) with replacement, refits ThreadTM on each resampled corpus, aligns its topics back to the reference fit, and reports how intact each topic and each group-prevalence cell stays. Threads are the resampling unit because comments within a thread are correlated (the same unit ThreadTM clusters its standard errors on).

Protocol. Fit a reference model on the full corpus. For each of n_boot draws, sample n_threads thread roots with replacement, rebuild a corpus from those threads (each copy re-indexed with its reply edges intact), refit ThreadTM with the SAME num_topics/coupling/seed_words/seed, and Hungarian-align its topics to the reference by word distribution (:func:align_topics). similarity[t] aggregates reference topic t's best match across the refits; prevalence[(g, t)] aggregates that group and topic's probability-scale prevalence.

Parameters:

Name	Type	Description	Default
docs	the corpus and reply forest, exactly as :meth:`ThreadTM.fit`	takes them (parents[d] is d's parent index, -1 for a root).	required
parents	the corpus and reply forest, exactly as :meth:`ThreadTM.fit`	takes them (parents[d] is d's parent index, -1 for a root).	required
num_topics	int. K, shared by the reference and every refit.		required
covariates	optional per-document categorical group id and	names, passed through to every fit (and needed for the prevalence CIs).	None
covariate_names	optional per-document categorical group id and	names, passed through to every fit (and needed for the prevalence CIs).	None
n_boot	int. Number of thread-resampled refits.		20
seed	RNG seed for the resampling; also the (fixed) model seed for every fit.		13
em_iters	passed through to every	:meth:ThreadTM.fit, so the robustness check matches your analysis fit (seed a topic and it stays pinned across the refits, which also makes the alignment trivial for the seeded slots).	100
min_count	passed through to every	:meth:ThreadTM.fit, so the robustness check matches your analysis fit (seed a topic and it stays pinned across the refits, which also makes the alignment trivial for the seeded slots).	100
coupling	passed through to every	:meth:ThreadTM.fit, so the robustness check matches your analysis fit (seed a topic and it stays pinned across the refits, which also makes the alignment trivial for the seeded slots).	100
seed_words	passed through to every	:meth:ThreadTM.fit, so the robustness check matches your analysis fit (seed a topic and it stays pinned across the refits, which also makes the alignment trivial for the seeded slots).	100
metric	word-distribution distance for :func:`align_topics` (default cosine).		'cosine'
stable_threshold	float. A reference topic is ``stable`` when its mean	matched similarity across refits is at least this.	0.7
ci	float. Central interval mass for the reported CIs (default 0.95).		0.95

Returns:

Type	Description
ThreadStabilityResult

Notes

Requires topica.enable_experimental() (ThreadTM is experimental). This runs n_boot + 1 full ThreadTM fits, so it is the heaviest diagnostic here; lower n_boot or em_iters for a quick look. This is the conditional (fixed-K, fixed-vocabulary-rule) resampling; it does not add model-selection error.

Quality

topica.evaluate.coherence

coherence(topics, texts, *, coherence_type='c_v', metric=None, n=10, topn=None, window_size=None, epsilon=1e-12)

Per-topic coherence against a reference corpus.

Parameters:

Name	Type	Description	Default
topics	a fitted model, or a list of topics (each a list of words, or of	(word, prob) pairs).	required
texts	the reference corpus, as a :class:`Corpus`, a list of raw-string	documents (split on whitespace), or already-tokenized documents (list[list[str]]). Pass your training documents, or an external corpus. (Raw strings are tokenized for you — passing them is not silently scored character-by-character; see issue #648.)	required
coherence_type	one of ``"u_mass"``, ``"c_uci"``, ``"c_npmi"``, ``"c_v"``	(default "c_v"). metric= is accepted as an alias (issue #732).	'c_v'
n	number of top words per topic to score (default 10). ``n`` is topica's	canonical top-words name, shared with model.coherence(n=...) and top_words(n); topn is accepted as an alias.	10
window_size	sliding-window width for the windowed measures; ``None`` uses	the per-measure default (110 for c_v, 10 for c_uci/c_npmi). Ignored by u_mass.	None

Returns:

Type	Description
numpy.ndarray of shape ``(num_topics,)`` — the coherence of each topic, aligned
to topic index; higher is more coherent. Take ``.mean()`` for an overall score.

Notes

This is the same computation a fitted model exposes as model.coherence(n=..., coherence_type=...): model.coherence(coherence_type=ct) equals coherence(model, training_texts, coherence_type=ct). c_v lies in [0, 1].

Default note (issue #742): this function defaults to c_v while the model method model.coherence() defaults to u_mass. The difference is deliberate: u_mass is intrinsic (it needs only the model's own corpus), so it is the one measure the method can compute without an external reference, whereas this function always receives texts and so can default to the stronger sliding-window c_v. Pass coherence_type= explicitly on both sides if you need to compare scores across the two entry points — the measures are on different scales.

u_mass is the sum of the pairwise log-ratios, the original Mimno et al. (2011) definition, matching the model method. gensim's u_mass reports the mean of the same per-pair scores, so topica's values differ from gensim's by the pairwise-count factor n·(n-1)/2 at a fixed n — but because that is a constant divisor, the two rank topics identically; only the absolute scale differs.

topica.evaluate.coherence_ci

coherence_ci(topics, texts, *, coherence_type='c_v', topn=10, window_size=None, n_boot=200, ci=0.9, seed=0, epsilon=1e-12)

Bootstrap standard errors and a credible interval for topic coherence.

Coherence is a corpus statistic with no model likelihood or posterior behind it, so its uncertainty is obtained by bootstrap: hold each topic's top words fixed, resample the reference documents with replacement n_boot times, recompute coherence on each resample, and report the per-topic standard error and percentile interval. The topics never change, so there is no refit and no topic-alignment step — the interval reflects how much a topic's coherence score would wobble under a different sample of the reference corpus, the right answer to "is topic A's coherence reliably higher than topic B's?".

estimate is the coherence on the full corpus (the conventional point summary); because resampling documents estimates the sampling distribution of that same statistic, the percentile interval is centered on it (unlike the posterior-draw intervals elsewhere).

Parameters:

Name	Type	Description	Default
topics	a fitted model, or a list of topics (each a list of words / ``(word,	prob)`` pairs). The top words are extracted once and held fixed.	required
texts	the reference corpus to resample — a :class:`Corpus`, raw-string	documents, or tokenized documents (as in :func:coherence). A fitted model passed here (the topics/texts arguments swapped) is rejected with a clear error.	required
coherence_type	as in :func:`coherence`.		'c_v'
topn	as in :func:`coherence`.		'c_v'
window_size	as in :func:`coherence`.		'c_v'
epsilon	as in :func:`coherence`.		'c_v'
n_boot	number of bootstrap resamples (each recomputes co-occurrence, so this	is O(n_boot x corpus size); the windowed measures (c_v etc.) are the costliest).	200
ci	central interval mass (default 0.9 for a 90% interval).		0.9
seed	seed for the document resampling.		0

Returns:

Type	Description
CoherenceCI	(estimate, se, ci_low, ci_high), each (num_topics,).

topica.evaluate.semantic_coherence

semantic_coherence(model_or_phi, texts, vocabulary=None, *, n=10)

Per-topic semantic coherence, shape (num_topics,) — stm's semCoh1beta.

The UMass document-co-occurrence coherence over each topic's top-n words, with stm's 0.01 smoothing (higher = better). This is stm's exact semantic coherence, from topica's Rust core (topica-core's inspect), shared with faSTM and the Stata plugin. For the broader, gensim-aligned coherence measures (c_v, c_npmi, u_mass) use :func:coherence instead.

model_or_phi is a fitted model (uses its topic_word / vocabulary) or a (K, V) array (then pass vocabulary). texts is the reference corpus: a :class:topica.Corpus, raw-string documents, or a list of token lists (raw strings are tokenized, not scored character-by-character; see issue #648).

topica.evaluate.embedding_coherence

embedding_coherence(topics, word_embeddings, vocabulary=None, *, topn=10, method='pairwise')

Per-topic coherence measured as top-word proximity in a word-embedding space. For each topic's top-topn words, either the mean pairwise cosine similarity (method="pairwise", OCTIS we_pairwise; Belford & Greene 2019) or the mean cosine to the topic's word centroid (method="centroid", OCTIS we_centroid; Ding, Nallapati & Xiang 2018). Higher = more coherent for both methods. Unlike :func:coherence it needs no reference corpus, and unlike :func:topica.llm.coherence it needs no LLM.

Both variants reproduce OCTIS at full top-word coverage. OCTIS reports we_centroid as a cosine distance (lower = better); we report the cosine similarity 1 - distance so the two methods share one direction — the centroid itself is built from the raw (un-normalized) top-word vectors, exactly as OCTIS builds it, so embedding_coherence(..., method="centroid") equals 1 - OCTIS_we_centroid.

topics is a fitted model, a (K, V) topic_word (with vocabulary), or a list of word lists. word_embeddings is either a dict {word: vector} (matched by word, robust to any topics form) or a (V, E) matrix aligned to vocabulary; vectors need not be unit length. No embedder of your own? :func:~topica.embeddings.llm_embed builds one::

emb = topica.embeddings.llm_embed(model.vocabulary)          # (V, E) matrix
topica.evaluate.embedding_coherence(model, emb, model.vocabulary)

Words with no embedding — including any whose vector is NaN/inf or all-zero — are dropped from a topic; a topic left with fewer than two embedded words scores nan and raises a warning naming its coverage. This is a small divergence from OCTIS, which instead divides by the fixed topn count so missing words pull the score toward 0; here a partly-covered topic is scored on the words it does have.

Returns a (num_topics,) array. Aggregate with np.nanmean (plain .mean() propagates a single nan topic to the whole corpus score).

Interpretation: the number is only comparable across models scored on the same embedding — there is no absolute "good" threshold, and it is not centered at 0 (random all-positive vectors already score high). For models that learn their own word embeddings (ETM, DETM, IdealPointTM), scoring against those vectors is circular — the model is graded in the space it optimized, so pass an external embedding for an honest number.

topica.evaluate.topic_diversity

topic_diversity(topics, topn=25, *, rank='prob', w=0.5)

Fraction of unique words across all topics' top-topn words (Dieng, Ruiz & Blei 2020). 1.0 means every top word is unique to its topic; low values indicate topics that recycle the same words.

topics is a fitted model or a list of word lists.

rank selects each topic's top-topn words: "prob" (raw P(w | topic), the default) or "frex" (FREX = frequency-exclusivity, STM's word ranking). Ranking by probability floats the corpus's shared high-frequency words into every topic's list, so it penalizes shrinkage models (ReplyTM/CTM/STM) for a measurement artifact rather than a topic-quality defect; FREX ranking measures diversity over each topic's distinctive vocabulary instead (issue #844, and STM's own coherence/exclusivity-frontier framing, Roberts et al. 2014). w is the FREX frequency weight in [0, 1] (used only when rank="frex"). rank="frex" needs a fitted model or a (K, V) topic_word matrix (its ranking is model-derived), not a pre-extracted list of word lists.

topica.evaluate.topic_semantic_diversity

topic_semantic_diversity(topics, topn=25)

Fraction of unique top-word pairs across all topics (Wu, Nguyen & Luu 2024, "A Survey on Neural Topic Models", Eq. 18). Where topic_diversity counts unique single words, this counts unique pairs drawn from each topic's top-topn words: a pair occurrence is "unique" when that unordered pair appears in exactly one topic's top words. 1.0 means every top-word pair is unique to its topic; higher = more diverse. A pair disambiguates word sense, so this is "semantic-aware" — no embeddings are needed.

topics is a fitted model or a list of word lists. topn must be an integer >= 2 (pairs require at least two words).

topica.evaluate.inverted_rbo

inverted_rbo(topics, *, topn=10, p=0.9)

Rank-biased-overlap diversity across topics (Bianchi, Terragni & Hovy 2021, used in OCTIS as InvertedRBO). 1 - mean pairwise RBO over every topic pair's top-topn word rankings. Where :func:topic_diversity treats top words as an unordered set, RBO weights agreement by rank: two topics sharing their #1-#2 words are penalized more than two sharing their

9-#10 words. p is the RBO persistence (0 < p < 1; higher weights

deeper ranks more). 1.0 means maximally diverse (no rank-weighted overlap); lower means topics recycle high-rank words.

topics is a fitted model or a list of word lists. Returns a float, or nan when there are fewer than two topics.

topica.evaluate.exclusivity

exclusivity(model_or_phi, *, n=10, w=0.7, rank='prob')

Per-topic exclusivity, shape (num_topics,) — stm's exclusivity.

For each topic, the FREX summary over its top-n words: the sum of each word's frequency–exclusivity score (the rank harmonic mean of probability and exclusivity φ_{t,v} / Σ_k φ_{k,v}, weighted by w, stm's default 0.7). Higher means the topic's top words are more distinctive. Pair with per-topic coherence to make stm's coherence-vs-exclusivity quality plot: good topics sit toward the upper-right (coherent and distinctive).

rank chooses which top-n words are summed: "prob" (top-n by raw P(w | topic), stm's exact exclusivity, the default) or "frex" (top-n by FREX score). Selecting by probability lets the corpus's shared high-frequency words — which every topic's top-probability list carries under a shrinkage prior — drag the summary down, penalizing shrinkage models (ReplyTM/CTM/STM) for a measurement artifact; rank="frex" sums each topic's genuinely most distinctive words instead (issue #844). Both use the identical FREX scores, so they are on the same scale (a sum over n words, roughly [0, n]) and directly comparable.

The scores come from the single stm-faithful implementation in topica's Rust core (topica-core's inspect), shared with faSTM and the Stata plugin.

model_or_phi is a fitted model (uses its topic_word) or a (K, V) array.

topica.evaluate.topic_significance

topic_significance(model, *, kind='vacuous', per_topic=False)

How far each topic sits from a null distribution, so background and junk topics score low (Aletras and Stevenson 2013, following OCTIS).

kind selects the null:

• "uniform": KL(topic-word || uniform over the vocabulary). A topic spread evenly over all words carries little information and scores near zero.
• "vacuous": KL(topic-word || the corpus-average word distribution sum_k phi_k * p(k)). A topic that looks like the corpus as a whole scores low.
• "background": KL(topic-document || uniform over documents). A topic present in every document (a background topic) scores low.

Returns the mean over topics, or the per-topic scores when per_topic=True. Higher is more distinctive. The scores are KL divergences in nats, so they are comparable across topics of one model but not across corpora of different vocabulary size; read them as a ranking, and pass per_topic=True to find the weakest topics (compare with evaluate.flag_topics for a fuller diagnosis). Needs only the fitted model's topic-word and, for vacuous/background, its document-topic matrix.

topica.evaluate.coherence_over_time

coherence_over_time(model, texts, timestamps, *, n=10, coherence_type='c_npmi', per_slice=False)

Coherence of a dynamic topic model, scored one time slice at a time against that slice's own documents and averaged (TopMost's dynamic_coherence).

texts is the reference corpus and timestamps maps each reference document to its integer time slice 0..T-1 (the same slicing used to fit the model). For each slice, the model's slice-specific topics are scored only against the documents in that slice, which is the point: a dynamic model should be coherent within each period, not just on average. Returns the mean over slices, or the per-slice list when per_slice=True. Applies to DTM and DETM, whose topics change over time; a static model has no per-slice topics to score.

timestamps must use the same integer slice codes 0..T-1 the model was fit with, not raw dates; a slice with no reference documents scores NaN and is dropped from the mean. Words that survived corpus pruning but never occur in a slice's documents contribute nothing to that slice's score, as in any windowed coherence.

topica.evaluate.diversity_over_time

diversity_over_time(model, *, n=25, per_slice=False)

Topic diversity of a dynamic model, computed one time slice at a time and averaged: the fraction of distinct top-n words among a slice's topics. Low diversity in a slice means its topics repeat the same words. Applies to DTM and DETM. Returns the mean over slices, or the per-slice list when per_slice=True.

This is the per-slice analogue of :func:topic_diversity, in the spirit of TopMost's dynamic_diversity but not identical to it: TopMost additionally counts only words unique to a single topic in the slice and present in that slice's own documents, so its numbers run lower when topics share words.

topica.select.quality_frontier

quality_frontier(model, *, n=10, texts=None, coherence_type='u_mass', plot=False)

Per-topic coherence, exclusivity, and prevalence — the data behind stm's classic coherence-vs-exclusivity quality plot.

Returns a :class:~topica._results.QualityFrontier (a dict of equal-length arrays — topic, coherence, exclusivity, prevalence (mean θ) — that also offers .to_frame() for a tidy one-row-per-topic DataFrame). By default coherence is the fast per-topic UMass score; pass texts and a windowed coherence_type (e.g. "c_v") for the human-aligned measure. With plot=True (and matplotlib installed) a labeled scatter Figure is returned alongside the data as (data, fig).

External validation

When you have gold (or partially gold) labels for your documents, agreement scores how well the discovered topics recover them — the check that actually tracks recovery, where coherence can mislead.

topica.agreement

External validation: score a topic assignment against gold labels.

:func:agreement answers the most basic validation question a topic model can be asked — given documents I have hand-labeled, how well do the discovered topics recover those labels? It reports the standard partition-comparison metrics (ARI, NMI, homogeneity, completeness, V-measure, and cluster purity), computed from the two label vectors alone.

This complements :func:topica.evaluate.coherence. Coherence rates the interpretability of a topic's top words; it does not tell you whether documents were assigned to the right topic, and for embedding-based cluster models it can be actively misleading (a model can keep tight, coherent top-words while the document partition drifts). When you have labels to check against, agreement is the number that tracks recovery.

The metrics are label-agnostic (invariant to how the cluster/class ids are named), so they work whether pred is 0..k cluster ids and gold is category codes, or any other integer labeling. The formulas match scikit-learn's adjusted_rand_score, normalized_mutual_info_score (arithmetic averaging), and homogeneity_completeness_v_measure; agreement needs only numpy.

__cached__ module-attribute

__cached__ = '/home/runner/work/topica/topica/python/topica/__pycache__/agreement.cpython-311.pyc'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__doc__ module-attribute

__doc__ = "External validation: score a topic assignment against gold labels.\n\n:func:`agreement` answers the most basic validation question a topic model can be\nasked — *given documents I have hand-labeled, how well do the discovered topics\nrecover those labels?* It reports the standard partition-comparison metrics (ARI,\nNMI, homogeneity, completeness, V-measure, and cluster purity), computed from the\ntwo label vectors alone.\n\nThis complements :func:`topica.evaluate.coherence`. Coherence rates the *interpretability*\nof a topic's top words; it does not tell you whether documents were assigned to the\nright topic, and for embedding-based cluster models it can be actively misleading\n(a model can keep tight, coherent top-words while the document partition drifts).\nWhen you have labels to check against, ``agreement`` is the number that tracks\nrecovery.\n\nThe metrics are label-agnostic (invariant to how the cluster/class ids are named),\nso they work whether ``pred`` is ``0..k`` cluster ids and ``gold`` is category\ncodes, or any other integer labeling. The formulas match ``scikit-learn``'s\n``adjusted_rand_score``, ``normalized_mutual_info_score`` (arithmetic averaging),\nand ``homogeneity_completeness_v_measure``; ``agreement`` needs only numpy.\n"

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__file__ module-attribute

__file__ = '/home/runner/work/topica/topica/python/topica/agreement.py'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__name__ module-attribute

__name__ = 'topica.agreement'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__package__ module-attribute

__package__ = 'topica'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

agreement

agreement(pred, gold, *, noise='keep')

Score a topic/cluster assignment against gold labels.

Parameters:

Name	Type	Description	Default
pred	array-like of int	Predicted topic/cluster per document — e.g. model.labels for a cluster model, or model.doc_topic.argmax(1) for a mixture model.	required
gold	array-like of int	Reference (hand-coded) label per document, aligned one-to-one with pred. To score a partially-labeled corpus, pass only the labeled subset: agreement(pred[mask], gold[mask]).	required
noise	(keep, drop)	How to treat documents that pred left unassigned (label -1, the HDBSCAN/BERTopic noise bucket). "keep" scores -1 as its own topic (so a large noise bucket is penalized honestly). "drop" excludes those documents before scoring (scores only the assigned ones). Documents with a gold label of -1 are dropped under "drop" as well.	"keep"

Returns:

Type	Description
dict	{"ari", "nmi", "homogeneity", "completeness", "v_measure", "purity"}. ARI is adjusted for chance (0 = random, 1 = identical partitions, and it can go slightly negative); the others lie in [0, 1]. purity is asymmetric — it measures how class-pure the predicted clusters are — and, unlike the others, is not penalized for splitting one class across many clusters.

Notes

All metrics are invariant to how the labels are named. Values match scikit-learn (normalized_mutual_info_score with arithmetic averaging). Pair with :func:topica.evaluate.coherence: coherence for whether the top words read as a theme, agreement for whether the document partition is right.

topica.evaluate.classification_quality

classification_quality(model, labels, *, test_size=0.3, C=1.0, kernel='linear', seed=13)

Downstream utility of the topics as document features. Train a linear support vector machine on the document-topic matrix and report how well it predicts labels on a held-out split, as accuracy and macro-averaged F1 (higher is better). This is the extrinsic check both OCTIS and TopMost use for whether the learned topics carry label-relevant signal; the topic model is used as a feature extractor, not a classifier. The exact number depends on the classifier and split, so read it as a relative score between models on one dataset, not an absolute.

labels is one label per document, in corpus order. Returns a dict with keys accuracy and macro_f1. Requires scikit-learn (pip install scikit-learn); it is not a core dependency.

Interpretation

topica.inspect.label_topics

label_topics(topic_word, vocabulary=None, *, n=10, word_counts=None, corpus=None)

stm-style topic labels: prob, FREX, lift, and score word lists per topic.

Returns a list with one :class:TopicLabels per topic. Each is a dict with keys prob, frex, lift, score, and each value is a list of (word, value) pairs — so select a labeling before reading words::

labels = topica.inspect.label_topics(model)     # one TopicLabels per topic
frex_words = [w for w, _ in labels[0]["frex"]]   # top FREX words of topic 0
prob_score = labels[0]["prob"]                   # [(word, prob), ...]

Iterating a topic directly (for w in labels[0]) yields the dict keys ('prob', 'frex', ...), not words — a common first-timer trap, so an integer index on a topic raises a directive error. (For a table of bare word strings instead of pairs, use :func:topic_table.) FREX, lift, and score all come from the single stm-faithful implementation in topica's Rust core (topica-core's inspect), so they cannot drift from faSTM / the Stata plugin.

lift is stm's lift, log P(w|topic) − log P(w), where P(w) is the empirical word frequency. Pass word_counts (a length-V array) or corpus (a :class:topica.Corpus, whose word counts are read for you) for the exact value; without either, P(w) is estimated from the topic-word matrix's column marginal (lift depends only on relative word frequency, so the ranking matches). word_counts / corpus also enable stm's James-Stein FREX shrinkage (see :func:frex).

topic_word is a fitted model (uses its topic_word and vocabulary) or a (K, V) array, in which case pass vocabulary.

topica.inspect.topics_for_term

topics_for_term(topic_word, terms, vocabulary=None, *, top_n=5, per_term=False, normalize=False, with_labels=False, label_n=5)

The inverse of "top words for a topic": the top topics for a term.

Given the topic-word matrix φ, rank topics by the weight they place on a queried term (or terms) — "which topics is 'immigration' important in, and how strongly?". Where :func:label_topics / :func:frex go topic → words, this goes word → topics, which is handy for corpus exploration and for checking where a seed/anchor word actually landed after fitting.

Parameters:

Name	Type	Description	Default
topic_word	a fitted model (uses its ``topic_word`` and ``vocabulary``) or a	(K, V) array, in which case pass vocabulary.	required
terms	str or sequence of str	A single term, or several. Terms are matched against vocabulary exactly — a topic model's vocabulary is its processed tokens (usually lowercased, often stemmed), so query the stored form ("immigr"), not the surface word ("Immigration"). A term absent from the vocabulary is dropped with a warning; if every term is absent a ValueError is raised. Duplicate terms are collapsed to their first occurrence.	required
vocabulary	sequence of str	Required only when topic_word is a bare array with no vocabulary.	None
top_n	int or None	How many topics to return, highest-weighted first. None returns all topics ranked. Ties are broken by ascending topic id.	5
per_term	bool	Ignored for a single term. For several terms, False (the default) pools the terms — topics are ranked by the summed weight across the queried terms — and returns one ranked list. True instead returns a dict mapping each (found) term to its own ranked list, in the order the terms were given.	False
normalize	bool	How each term's per-topic weight is defined. False uses the raw φ entry, P(word | topic) — directly "what share of this topic's mass is this word". Because φ columns are not comparable across words (a frequent word carries more mass everywhere), raw weights and the raw pooled sum are dominated by the most frequent term. True instead column-normalizes each term to P(topic | word) = φ[:, w] / Σ_t φ[t, w] — "what share of this word lives in each topic" — which is comparable across terms and frequency-robust, so pooling weights terms equally. This is a distribution only for a nonnegative φ; on a signed-axis model (S³, an ideal-point model) normalize=True raises rather than return meaningless ratios, so leave it off there.	False
with_labels	bool	When True, each ranked entry gains a third element: the topic's label_n highest-probability words (from φ), so the result reads on its own without a manual join back to :func:label_topics. Entries become (topic_id, weight, top_words) where top_words is a list[str].	False
label_n	int	How many words to attach per topic when with_labels=True.	5

Returns:

Type	Description
For a single term, or several terms with ``per_term=False``: a list of
``(topic_id, weight)`` pairs sorted by descending weight. With several terms
and ``per_term=True``: a ``dict`` ``{term: [(topic_id, weight), ...]}``. When
``with_labels=True`` every pair is instead a ``(topic_id, weight, top_words)``
triple.

Examples:

>>> topics_for_term(model, "immigr", top_n=5)
[(12, 0.031), (4, 0.018), ...]
>>> topics_for_term(model, ["immigr", "border"], per_term=True)
{"immigr": [...], "border": [...]}
>>> topics_for_term(model, "immigr", top_n=2, with_labels=True)
[(12, 0.031, ["immigr", "border", "illeg", ...]), (4, 0.018, [...])]

topica.llm_topic_labels

llm_topic_labels(model, texts=None, *, backend=None, llm_model='gpt-4o-mini', n_words=12, n_docs=3, max_chars=300, instructions=None, set_labels=False)

A short, human-readable label for each topic, generated by an LLM.

For each topic, assembles a prompt from its top words and representative documents (see :func:topic_label_prompts) and asks a model for a concise label. Returns a list of labels, one per topic.

Supply the model one of two ways:

• backend: any callable str(prompt) -> str(label) — your own client, ollama, whatever, or :func:topica.llm_backend / :func:topica.llm.backend. Zero extra dependencies; you own determinism.
• otherwise llm_model names a model used through :func:llm_backend (the topica[llm] extra). backend takes precedence when both are given.

With set_labels=True the labels are stored via :func:topica.set_topic_labels, so they flow into :func:topica.topic_info, :func:topica.topic_labels, and :func:topica.plot_report.

LLM labels are a convenience, not a reproducible measurement: pin the model and set temperature to 0, and keep :func:topica.inspect.label_topics (FREX / probability / lift) for the defensible descriptors.

topica.llm_backend

llm_backend(model='gpt-4o-mini', *, key=None, system=None, **options)

A str -> str callable backed by the llm library, for the backend= argument of :func:llm_topic_labels.

model names any model llm can reach — OpenAI, Anthropic, or local models through plugins such as llm-ollama. By default the API key is resolved by llm itself: a stored llm keys value, else the provider's environment variable (OPENAI_API_KEY for OpenAI). Pass key to override that with an explicit key. options pass through to llm (e.g. temperature=0 for reproducible labels where the provider supports it). Requires the optional llm package (pip install llm or pip install "topica[llm]").

topica.topic_label_prompts

topic_label_prompts(model, texts=None, *, n_words=12, n_docs=3, max_chars=300, instructions=None)

One labeling prompt per topic — exactly the text a model is asked to label.

Each prompt lists the topic's top n_words words and, when texts is given, up to n_docs representative documents (each whitespace-collapsed and truncated to max_chars). instructions overrides the default task framing. Returns a list of prompt strings, one per topic.

This is the plumbing behind :func:llm_topic_labels; build it yourself to see or adjust what the model sees, or to drive a model topica does not know about.

topica.inspect.frex

frex(topic_word, vocabulary=None, *, w=0.5, n=10, word_counts=None, corpus=None)

FREX (FRequency–EXclusivity) top words per topic.

For each topic, words are scored by the weighted harmonic mean of the rank of their probability (frequency) and the rank of their exclusivity φ_{t,v} / Σ_k φ_{k,v} — stm's calcfrex. w weights frequency vs exclusivity. Returns a list (per topic) of (word, frex).

The scores come from the single, stm-faithful implementation in topica's Rust core (topica-core's inspect module — the same one faSTM and the Stata plugin use), so the FREX definition can never drift between languages.

Pass word_counts (a length-V array of corpus word frequencies) or corpus (a :class:topica.Corpus, whose word counts are read for you) to apply stm's James-Stein exclusivity shrinkage, which is stm's default; it damps the exclusivity of rare words that appear in only one topic by chance. Without either (the default here) no shrinkage is applied.

topic_word is a fitted model (uses its topic_word and vocabulary) or a (K, V) array, in which case pass vocabulary.

topica.inspect.mmr

mmr(topic_word, word_embeddings, vocabulary=None, *, n=10, diversity=0.3, n_candidates=None)

Maximal-marginal-relevance top words, to cut redundant near-synonyms.

For each topic, take the top n_candidates words by topic_word weight and greedily reselect n of them, each pick maximizing

``(1 - diversity) * relevance(word) - diversity * max_cos(word, picked)``

where relevance is the (per-topic, max-normalized) topic_word weight and the redundancy term is the cosine between word embeddings. diversity=0 returns the plain top words; higher trades relevance for variety, like BERTopic's MaximalMarginalRelevance(diversity=...).

Parameters:

Name	Type	Description	Default
topic_word	a fitted model (uses its ``topic_word`` and ``vocabulary``) or a	(K, V) array, in which case pass vocabulary.	required
word_embeddings	a ``(V, E)`` matrix aligned to the vocabulary — the word	vectors (for Top2Vec, the ones you fit with; otherwise embed the vocabulary with your embedding model, as BERTopic's MMR does internally).	required
n	words returned per topic.		10
diversity	in ``[0, 1]``; 0 is the plain top words, higher is more diverse.		0.3
n_candidates	how many top words to rerank (default ``max(5 * n, n)``).		None

Returns:

Type	Description
A list per topic of ``(word, topic_word_weight)`` pairs, like ``top_words``.

topica.inspect.relevance

relevance(topic_word, vocabulary=None, *, topic=None, lam=0.6, n=10, term_frequency=None)

LDAvis relevance of words to topics (Sievert & Shirley 2014):

relevance(w | t) = λ·log p(w|t) + (1-λ)·log[p(w|t) / p(w)]

λ=1 ranks by probability; λ=0 by lift (exclusivity); the LDAvis default 0.6 balances them. p(w) is the corpus word marginal — pass term_frequency (word counts in vocabulary order) for the empirical marginal, else the topic-averaged φ is used. Returns (word, relevance) lists per topic, or for one topic.

topic_word is a fitted model (uses its topic_word and vocabulary) or a (K, V) array, in which case pass vocabulary.

topica.inspect.find_thoughts

find_thoughts(doc_topic, texts=None, *, topic, n=3)

The n documents most associated with topic (≈ stm's findThoughts).

Returns a list of (doc_index, proportion, text) sorted by descending topic proportion; text is None when texts is not supplied.

doc_topic is a fitted model (uses its doc_topic) or a (D, K) array. texts is a sequence of the documents' texts, or a :class:~topica.Corpus. A Corpus only retains tokens, so its text field comes back as the space-joined processed tokens (lowercased, stopword-stripped), not the original prose; to read the raw documents, pass your original text sequence indexed by corpus.kept_indices (pruning may have dropped some rows).

topica.inspect.find_thoughts_html

find_thoughts_html(model, texts, *, topics=None, n_docs=3, n_words=8, max_chars=400, markdown=False)

Render each topic's most representative documents for close reading, with the topic's top words highlighted in the document text.

Distant reading (top words) is only half of topic validation; the other half is reading the actual documents a topic loads on. This builds a self-contained HTML snippet (or Markdown) you can display in a notebook: per topic, its top words followed by its n_docs highest-θ documents, each truncated to max_chars with the topic's words marked.

model is any fitted model exposing topic_word, doc_topic and vocabulary; texts are the original document strings, aligned to the rows of doc_topic. A :class:~topica.Corpus is also accepted, in which case its tokenized documents are joined back into text for display. Returns a string (HTML unless markdown=True).

topica.inspect.topic_correlation

topic_correlation(doc_topic, *, threshold=0.05)

Topic-correlation network (≈ stm's topicCorr "simple" method).

Correlates topic proportions across documents; topic pairs whose correlation exceeds threshold become network edges. Returns a :class:TopicCorrelation with the correlation matrix, a 0/1 adjacency matrix (zero diagonal), and the edge list.

This is the raw across-document theta correlation, matching stm's topicCorr default ("simple") method. Raw theta correlation is compositionally biased (the simplex constraint induces spurious negative correlation); for the closure-corrected alternatives use viz.topic_correlation(model, method="clr") (the viz layer's default) or method="partial"/"eta".

doc_topic is a fitted model (uses its doc_topic) or a (D, K) array.

topica.inspect.prepare_pyldavis

prepare_pyldavis(model, docs, **kwargs)

Build the LDAvis intertopic-distance visualization for a fitted model.

docs are the tokenized training documents (list[list[str]]), used for document lengths and term frequencies. If pyLDAvis is installed this returns its PreparedData (pass to pyLDAvis.display / save_html); otherwise it returns a :class:PyLDAvisInputs you can feed to pyLDAvis.prepare later. Extra kwargs go to pyLDAvis.prepare (e.g. sort_topics=False).

Validation

topica.evaluate.word_intrusion

word_intrusion(model_or_phi, vocabulary=None, *, n_words=5, seed=0)

Build a word intrusion test for human topic validation.

For each topic, take its top n_words words and splice in one intruder — a word that ranks highly in some other topic but has low probability in this one. A coherent topic is one where a human can reliably spot the intruder (Chang et al. 2009, "Reading Tea Leaves"). Returns a list (per topic) of dicts with:

• topic — the topic index,
• words — the n_words + 1 words in shuffled, presentation order,
• intruder — the intruder word,
• intruder_index — its position in words (the answer key).

model_or_phi is a fitted model (uses its topic_word / vocabulary) or a (K, V) array (then pass vocabulary). Deterministic for a fixed seed.

topica.evaluate.document_intrusion

document_intrusion(model_or_theta, texts=None, *, n_docs=3, seed=0)

Build a document intrusion test for human topic validation.

For each topic, take the n_docs documents with the highest proportion of that topic and splice in one intruder — a document where the topic is nearly absent (and another topic dominates). A topic that captures real document similarity is one where a human can spot the intruder. Returns a list (per topic) of dicts with:

• topic — the topic index,
• doc_indices — the n_docs + 1 document indices in shuffled order,
• intruder_index — the intruder's position in doc_indices,
• texts — the corresponding text previews (only if texts is given).

model_or_theta is a (D, K) θ array (or a fitted model, whose doc_topic is used). Deterministic for a fixed seed.

LLM-based evaluation (topica.llm)

topica.llm.coherence

coherence(model, *, backend, n_words=10, scale=(1, 3), dataset_description=None, seed=0, n_samples=1, shuffle=True, prompts=None)

LLM-rated topic coherence (Stammbach et al. 2023): the headline LLM metric.

For each topic, the top n_words words are shuffled and an LLM rates how related they are on a scale (default 1-3). Returns a per-topic numpy array of mean ratings (higher = more coherent). This is the metric that beats NPMI / c_v at tracking human judgment in the paper; it sits beside :func:coherence, :func:topic_diversity, and :func:topic_semantic_diversity, but is llm-bounded -- it calls an external model and is not bit-deterministic.

Parameters:

Name	Type	Description	Default
model	fitted model or list of word lists	Anything :func:_extract_topics accepts.	required
backend	callable ``str -> str`` or model-name str	The LLM. Pass topica.llm.backend(name, temperature=0) or a model name.	required
n_words	the number of top words shown and the rating range.		10
scale	the number of top words shown and the rating range.		10
dataset_description	optional str	A one-line corpus description added to the prompt (small reported gains).	None
seed	int	Seeds the per-topic word shuffles (reproducible task; the LLM is not).	0
n_samples	int	Calls the LLM this many times per topic and averages (tames non-determinism; the paper uses temperature=1 to mimic annotator variation).	1
prompts	optional dict	Override the editable templates (key "rating"); defaults to :data:LLM_EVAL_PROMPTS.	None

topica.llm.intrusion

intrusion(model, vocabulary=None, *, backend, n_words=5, dataset_description=None, seed=0, n_samples=1, prompts=None)

LLM word-intrusion accuracy (Stammbach et al. 2023).

Builds the intrusion task with :func:word_intrusion (top n_words words plus one intruder, shuffled), asks the LLM to pick the intruder, and scores it against the answer key. Returns {"accuracy": float, "per_topic": [...]} where each per-topic dict has topic, intruder, picked, and correct.

The paper finds an LLM matches human accuracy on this task (~72%), but rating (:func:llm_coherence) tracks human topic rankings better -- lead with llm_coherence and report this alongside. llm-bounded; see :func:llm_coherence for the shared backend / n_samples semantics.

topica.llm.select_k

select_k(models, docs, *, backend, n_docs=10, granularity='broad', example_labels=None, research_question=None, criterion='knee', tol=0.03, seed=0, n_samples=1, max_chars=1500, prompts=None)

Choose the number of topics by LLM document-label purity (Stammbach et al. 2023). For each candidate fitted model, take each topic's top n_docs documents, have an LLM assign each a theme label, and score the topic by label purity — the fraction of its documents sharing the majority label. The model's score is the mean per-topic purity.

This is the paper's working number-of-topics signal: doc-label purity tracks ground-truth cluster quality (ARI), whereas rating the top words across K does not (their negative result). Complements :func:search_k (coherence / exclusivity / perplexity) with a human-aligned, llm-bounded criterion.

.. note:: Purity rises then plateaus as K grows — over-splitting one theme into two topics yields two same-labelled, still-pure topics — so the raw maximum tends to over-split (the mirror of coherence's bias toward small K; cf. :func:search_k's frontier). The default criterion="knee" therefore returns the smallest K whose purity is within tol of the best (the plateau onset), not the bare argmax. Always read the full scores curve; criterion="max" restores the literal highest-purity pick.

Parameters:

Name	Type	Description	Default
models	sequence of fitted models	Candidates, typically the same corpus fit at different num_topics.	required
docs	Corpus | list of str | list of token lists	The documents, in the order the models were fit on (their doc_topic rows).	required
backend	callable ``str -> str`` or model-name str	The LLM (see :func:llm_coherence).	required
n_docs	int	Top documents per topic to label.	10
granularity	(broad, narrow)	Whether to ask for a broad or a narrow theme label.	"broad"
example_labels	optional sequence of str	Example label vocabulary shown to the model (steers granularity/format).	None
research_question	optional str	A one-line framing ("label by the policy area discussed", ...).	None
criterion	(knee, max)	How best is chosen from the purity curve. "knee" (default) returns the smallest K within tol of the best purity (the plateau onset); "max" returns the highest-purity model (which tends to over-split).	"knee"
tol	float	Purity tolerance for the knee (default 0.03).	0.03
n_samples	int	Majority-vote the label over this many calls per document.	1
max_chars	int	Truncate each document to this many characters in the prompt.	1500

Returns:

Type	Description
dict with ``best`` (the chosen model's ``num_topics``), ``best_index``, and
``scores`` (a list of ``{"num_topics", "purity", "per_topic_purity"}`` per model).

topica.llm.outlier

outlier(model, *, backend, n_words=10, n_samples=5, threshold=3, dataset_description=None, seed=0, prompts=None)

Unsupervised semantic-outlier detection (Tan & D'Souza 2025, C_outlier).

For each topic, asks the LLM to list the words that do not fit the topic, over n_samples runs, and keeps a word flagged in at least threshold runs (the paper's 3-of-5 vote). Returns a per-topic list of dicts with topic, outliers (the flagged words), and count. Unlike :func:llm_intrusion there is no planted answer — this surfaces which words make a topic incoherent. llm-bounded; see :func:llm_coherence for backend/n_samples semantics.

topica.llm.repetitiveness

repetitiveness(model, *, backend, n_words=10, n_samples=1, dataset_description=None, seed=0, prompts=None)

LLM repetitiveness (Tan & D'Souza 2025): is apparent coherence just redundancy?

Returns a per-topic list of dicts with rate (R_rate: 1 = highly repetitive, 3 = diverse/distinctive; averaged over n_samples), duplicate_pairs (R_duplicate: word pairs the LLM judges the same concept), and duplicate_count. A robust coherent topic has a high rate and a low duplicate count. Complements :func:topic_semantic_diversity on the LLM side. llm-bounded.

topica.llm.diversity

diversity(model, *, backend, n_words=10, n_samples=1, max_pairs=None, dataset_description=None, seed=0, prompts=None)

Cross-topic LLM diversity (Tan & D'Souza 2025, D_rate).

Rates the thematic distinctiveness of every pair of topics 1-3 (1 = overlapping, 3 = distinctive) and averages. Returns {"mean": float, "pairwise": [...]} with one {"topics": (i, j), "rate": r} per scored pair. O(K²) calls; pass max_pairs to score a deterministic random subset. The LLM analog of :func:topic_diversity / :func:topic_semantic_diversity. llm-bounded.

topica.llm.alignment

alignment(model, docs, *, backend, n_words=10, n_docs=5, dataset_description=None, seed=0, prompts=None, max_chars=1500)

Topic-document alignment (Tan & D'Souza 2025, A_ir-topic / A_missing-theme).

For each topic, takes its top n_docs documents and asks the LLM, per document, (1) how many topic words are irrelevant to it (overrepresentation) and (2) how many document themes are missing from the topic words (underrepresentation), averaging over the documents. Returns a per-topic list of dicts with topic, irrelevant (mean count) and missing (mean count); lower is better on both. Needs the documents and O(K·n_docs) calls. llm-bounded.

topica.llm.adversarial

adversarial(model, *, backend, intruder='shakespeare', n_words=10, n_samples=5, threshold=3, dataset_description=None, seed=0, prompts=None)

Gold-free adversarial self-check (Tan & D'Souza 2025, AdvT_outlier).

Plants a known-unrelated word (default "shakespeare") into each topic's top words and measures how often the LLM's :func:llm_outlier detection flags it. This validates the metric and the model's capability without human-gold data, on any corpus — a low detection rate means the model is too weak for these tasks. Returns {"detection_rate": float, "intruder": str, "per_topic": [...]}.

topica.evaluate.bootstrap_stability

bootstrap_stability(docs, *args, k=None, num_topics=None, n_boot=20, topn=10, seed=0, model_factory=None, reference=None, fit_kwargs=None, **extra_fit_kwargs)

Flag fragile topics by refitting on bootstrap resamples of the corpus.

The standard defense against "topic modeling is a fishing expedition": fit a reference model on the full corpus, then refit on n_boot resamples of the documents (drawn with replacement). Each bootstrap model's topics are matched to the reference's by top-word overlap, and a reference topic's stability is the mean Jaccard overlap of its top-topn words with its matched bootstrap topic. Topics that dissolve under resampling score low.

Matching is on the top words as strings, so it is correct even though each resample is fit as a fresh corpus with its own vocabulary indexing.

Parameters:

Name	Type	Description	Default
docs	the corpus (``list[list[str]]`` or a ``Corpus``).		required
k	number of topics. Required unless ``reference`` is given (then taken from	it). num_topics= is accepted as an alias (the library-wide name).	None
n_boot	number of bootstrap resamples.		20
model_factory	``callable(seed) -> unfitted model``. Defaults to	LDA(num_topics=k, seed=seed). Use it to bootstrap any model.	None
reference	an already-fitted model to measure the stability *of*. When given,	the resample topics are matched back to it (rather than to a fresh full-corpus fit), so the per-topic stability lines up with that model's topic indices. model_factory should rebuild the same model type. The factory's single argument is the seed (not k), e.g. model_factory=lambda seed: topica.LDA(8, seed=seed); the factory sets the topic count, so k= is ignored when a factory is given.	None
fit_kwargs	dict of keyword arguments forwarded to each model's ``fit`` (e.g.	fit_kwargs={"iters": 500}), matching :func:cross_validate. For convenience the same arguments may also be passed inline as keywords (bootstrap_stability(docs, k=5, iters=500)); the two are merged, with inline keywords taking precedence on a clash. Covariate models. A per-document design passed here (STM prevalence / covariates, a content label vector, dynamic timestamps) is resampled with the same bootstrap draw as the documents, so the covariate rows stay aligned to the resampled corpus and a covariate model can be bootstrapped as itself — no silent downgrade to a covariate-free model (issue #751). Any fit-kwarg whose first-axis length equals the number of documents is treated as per-document and resampled; per-feature arrays (e.g. prevalence_names, length p) and scalars are passed through unchanged. So a full STM stability check reads:: X, names = topica.design.one_hot(df["party"]) stm = topica.STM(num_topics=10, seed=13).fit( docs, prevalence=X, prevalence_names=names) bs = topica.evaluate.bootstrap_stability( docs, reference=stm, model_factory=lambda s: topica.STM(num_topics=10, seed=s), prevalence=X, prevalence_names=names)	None

Returns:

Type	Description
class:`~topica._results.BootstrapStability` (a ``dict``) with ``topic``
(indices), ``stability`` (per-topic mean Jaccard in ``[0, 1]``), ``mean``
(overall), and ``reference`` (the reference model). Call ``.to_frame()`` for a
per-topic ``(topic, stability)`` DataFrame.

topica.select.search_k

search_k(docs, ks, *, model='lda', fit=None, prevalence=None, prevalence_names=None, content=None, held_out=None, iters=500, num_samples=3, sample_interval=10, seed=13, coherence_n=10, coherence_type='u_mass', n_jobs=1, num_seeds=1, criteria=())

Fit a model for each K and report quality metrics (stm's searchK).

model= selects a built-in: "lda" (default), "stm", "nmf", or "lsa". For "stm" pass prevalence (a covariate design matrix) and optional content (group labels) to scan K for the model you'll actually report. "nmf" additionally reports a reconstruction_error column.

For any other model, pass fit= — a callable (k, seed) -> fitted model that builds and fits the model, closing over docs and any covariates or embeddings it needs. search_k then scores whatever it returns with the same generic metrics, so it works for every model without knowing its fit signature (the same escape hatch bootstrap_stability / diagnostics / standard_errors offer via model_factory=)::

search_k(docs, [10, 20, 30], fit=lambda k, s: topica.NMF(k, seed=s).fit(docs))

A fitted model only needs topic_word and top_words for the coherence and exclusivity columns. The dispersion column and the held_out columns are generative-count diagnostics, so they are reported only for models that expose a generative transform (LDA/STM/DMR/CTM/HDP); they are omitted for matrix-factorization models (NMF factors a tf-idf matrix; LSA has signed SVD factors), which are not generative count models. The opt-in criteria (deveaud/cao_juan) treat each topic as a word distribution, so they are omitted for a signed topic_word (LSA). The stm semantic-coherence metric is used only for the built-in model="stm"; a fit= closure returning an STM is scored with plain UMass coherence.

Returns a :class:SearchKResult (a list of per-K dicts) with k, coherence (mean of the selected coherence type; for model="stm" with the default u_mass this is stm's semantic coherence, labelled coherence_metric="semcoh"), exclusivity (mean top-word exclusivity), dispersion (residual dispersion, Taddy 2012 — >> 1 means K is too small) with its dispersion_pvalue, and — when held_out is supplied — a held-out quality metric. The result also carries .directions (whether higher or lower is better per metric) and a .best_k(metric=...) selector. best_k defaults to the held-out metric when one is supplied, otherwise to a coherence/exclusivity frontier (a knee), because bare UMass coherence is roughly monotone in K and would just return the smallest K scanned. Duplicate ks are dropped; ties in best_k break toward the smaller (simpler) K.

Two held-out paths are supported, determined by the type of held_out:

• Heldout object (from :func:make_heldout): scored with :func:eval_heldout; results stored under "heldout_loglik" (mean_per_doc_loglik, higher / less negative is better). Use this path for the standard within-corpus word-heldout diagnostic.
• Corpus or token lists (legacy): scored with :func:perplexity; results stored under "perplexity" (lower is better). This is the document-completion perplexity on a separate held-out set.

Parameters:

Name	Type	Description	Default
docs	training documents (``list[list[str]]`` or a ``Corpus``).		required
ks	sequence of topic counts to scan.		required
model	``"lda"`` (default), ``"stm"``, ``"nmf"``, or ``"lsa"``. Ignored when	fit= is given.	'lda'
fit	optional ``callable(k, seed) -> fitted model``. When given it takes	precedence over model and lets search_k scan any model type; the callable owns the fit (it closes over docs and any covariates).	None
prevalence	covariate design matrix for ``model="stm"``; ignored otherwise.		None
prevalence_names	accepted for signature-parity with ``STM.fit`` (so the same	kwargs drop straight in) and ignored — search_k scans K on the design matrix and never labels the covariates, so their names do not affect any metric it reports.	None
content	optional content group labels (sequence of str/int) for ``model="stm"``.		None
held_out	optional held-out set. Pass a :class:`Heldout` (from	:func:make_heldout) or a separate corpus / token lists.	None
iters	training iterations per fit.		500
num_samples	Gibbs samples per fit (LDA only).		3
sample_interval	iterations between Gibbs samples (LDA only).		10
seed	RNG seed for every fit and transform call.		13
coherence_n	top-word count used for coherence and exclusivity.		10
coherence_type	one of ``"u_mass"``, ``"c_uci"``, ``"c_npmi"``, ``"c_v"`` (default ``"u_mass"``).		'u_mass'
n_jobs	number of worker threads for the per-fit work (default ``1``, serial).	The fits are independent and each keeps its own fixed seed, so the results are identical to the serial run; only the wall-clock changes (the Rust fits release the GIL). n_jobs<=0 (or None) uses all cores, capped at the number of fits. Note it multiplies with any intra-fit threading (num_threads= on the model), so n_jobs above the core count can oversubscribe.	1
num_seeds	number of seeds fit per K (default ``1``). With ``num_seeds>1``,	each K is refit over seeds seed, seed+1, ...; every metric column then holds the across-seed mean and gains a <metric>_se standard-error column, and best_k(rule="1se") becomes available (the simplest K within one SE of the optimum). The per-K work parallelizes over (K, seed) via n_jobs. A single seed carries no standard errors (backward-compatible).	1
criteria	optional extra K-selection criteria to report as columns (default	none). "deveaud" (Deveaud et al. 2014; mean pairwise Jensen-Shannon divergence between topics, higher = more distinct) and "cao_juan" (Cao Juan et al. 2009; mean pairwise topic cosine, lower = less redundant). Opt-in and out of the frontier, but selectable via best_k("deveaud") / best_k("cao_juan") and carry standard errors under num_seeds>1 like any other metric.	()

topica.evaluate.check_residuals

check_residuals(model, docs, *, tol=0.01)

Residual-dispersion test for whether K is too small (Taddy 2012), a faithful port of R stm's checkResiduals.

Under a correctly specified model the multinomial residuals have dispersion σ² = 1. A dispersion well above 1 (small p-value) is evidence the latent topics cannot absorb the overdispersion — i.e. K is too low. Run it alongside :func:search_k. docs are the tokenized training documents aligned to model.doc_topic's rows.

Returns a :class:ResidualCheck with dispersion (σ²), pvalue (χ² test of σ²=1 vs σ²>1), and df.

The dispersion, df, and χ² statistic come from the shared topica-core inspect::residual_dispersion implementation (the same port faSTM and the Stata plugin consume), so every host reports one stm-faithful number. Only the upper-tail χ² p-value is formed here.

topica.evaluate.document_residuals

document_residuals(model, docs, *, floor=1e-12)

How poorly the fitted model explains each document, for outlier hunting.

Reconstructs each document's expected word distribution as theta_d @ beta and compares it to the document's actual word counts. A high residual marks a document the current topics cannot account for: an off-topic intruder, an anomaly, or a sign the model is missing a theme. This is the per-document complement to :func:check_residuals, which collapses the whole corpus into one "is K too small?" dispersion statistic.

docs are the tokenized documents aligned row-for-row to model.doc_topic (the corpus the model was fit on). To score new documents, get their theta with model.transform first.

Returns a list of per-document dicts sorted by descending novelty (most anomalous first). Each has doc (row index), novelty (the headline score: OOV-aware per-word cross-entropy), cross_entropy (the length-robust in-vocabulary-only per-word log-loss; nan if the document has no in-vocab tokens), kl (KL(actual || recon); length-confounded, use with care), cosine_dist (1 - cosine), oov (out-of-vocabulary token fraction), n_tokens and n_invocab.

A pure cross-entropy residual can only see in-vocabulary tokens, so a document written entirely in unknown words would otherwise look perfectly explained; novelty folds the OOV mass back in, which is what makes off-topic-vocabulary intruders rank at the top.

topica.evaluate.flag_topics

flag_topics(model, texts, *, n=10, coherence_type='c_v')

Score every topic on cheap quality features and flag likely junk.

A quick "are these topics real, or did I forget to clean my corpus?" check. For each topic it gathers :func:topica.evaluate.coherence, :func:topica.evaluate.exclusivity, the normalized topic-word entropy (1.0 = a perfectly flat, uninformative topic), corpus prevalence, and the fraction of its top words that are stopwords, then flags a topic as junk when any of:

• stopword-soup — at least 40% of the top words are stopwords;
• dead/tiny — prevalence below half its uniform share (0.5 / K);
• incoherent+flat — coherence in the run's bottom quartile and topic-word entropy in its top quartile.

The thresholds are relative to the run, so the flag reads as "junk for this model". texts are the tokenized documents (used only for coherence; they need not align to doc_topic).

Returns a list of per-topic dicts (in topic order) with topic, coherence, exclusivity, beta_entropy, prevalence, stopword_frac, junk (bool), reasons (list of str), and top_words.

topica.evaluate.topic_dendrogram

topic_dendrogram(model, *, metric='js', method='average', n_topwords=20)

Agglomeratively merge a fitted model's topics into a multi-resolution tree.

A post-hoc, no-refit answer to "are these K topics really a handful of super-themes, and are any of them near-duplicates?". It builds a K x K topic distance and runs hierarchical clustering, returning a :class:TopicDendrogram you can :meth:~TopicDendrogram.cut at any resolution or query for :meth:~TopicDendrogram.merge_candidates. This is the flat-model counterpart to :class:~topica.HLDA (which fits a topic tree directly) and to :func:topica.ensemble (which merges across runs).

Works on any fitted model exposing topic_word and vocabulary.

Parameters:

Name	Type	Description	Default
model	a fitted topica model.		required
metric	(js, hellinger, cosine, doctopic)	Topic distance. js (Jensen-Shannon) and hellinger compare the full topic-word distributions; cosine compares top-n_topwords indicator sets; doctopic uses 1 - correlation of the doc_topic columns (how often topics co-occur in documents).	"js"
method	str	SciPy linkage method ("average", "ward", "complete", ...).	"average"
n_topwords	int	Words per topic for the cosine metric and for leaf labels.	20

Returns:

Type	Description
class:`TopicDendrogram`.

Notes

Requires SciPy (pip install 'topica[viz]' or scipy).

topica.evaluate.align_topics

align_topics(a, b, *, by='words', metric='cosine', threshold=0.3, depth=50, p=0.9, word_embeddings=None) -> AlignmentResult

Match the topics of two fits one-to-one by minimal total distance (Hungarian on the cross-fit distance matrix). Use it to compare runs across seeds, across K, or train vs. resample.

by chooses the space the topics are matched in:

• "words" (default): match by each topic's word distribution, so two topics align when they use the same vocabulary. a, b are fitted models or K×V topic-word arrays (same vocabulary order, or automatically intersected if .vocabulary is available), and metric selects the word-space distance.
• "documents": match by each topic's document loading, so two topics align when the same documents load on them, even if their top words differ. This requires both fits to have been trained on the same documents in the same order; the similarity is the correlation between the two topics' columns of the document-topic matrix (each column centered across documents, so a shared baseline prevalence does not inflate the match; metric is ignored). Complements :func:topica.agreement, which scores the two fits' hard document partitions as a whole (ARI/NMI) rather than topic-by-topic.

metric (word space only) is "cosine", "js" (Jensen-Shannon), "rbo" (Rank-biased overlap), or "emd"/"ot" (Earth Mover's Distance). In every space the tuple carries a distance (1 - similarity), while similarity_matrix holds the similarity; lower tuple values are closer. Returns an AlignmentResult object which behaves as a list of (topic_a, topic_b, distance) tuples sorted by topic_a, but exposes additional attributes: matches, splits, merges, unaligned_a, unaligned_b, and similarity_matrix.

matches is the Hungarian assignment restricted to pairs above threshold; splits/merges are a background-relative overlay calibrated to the fit's own cross-topic similarity, so align_topics(tw, tw) returns K matches and no splits/merges for any model with distinct topics — including correlated-topic families (STM/CTM) whose off-diagonal cosines are high (issue #642); exact-duplicate topics are the honest exception (they report as a split/merge, being interchangeable). threshold sets the one-to-one match cut; the split/merge overlay self-calibrates and does not depend on it.

topica.evaluate.topic_stability

topic_stability(runs, *, num_topics=None, k=None, seeds=None, model_factory=None, topn=10, metric='cosine', per_topic=False, fit_kwargs=None, **extra_fit_kwargs)

Term-centric stability of topics across multiple fits (Greene, O'Callaghan & Cunningham 2014): a "how robust is this K?" score.

Two call shapes:

• From fitted runs (the base form): runs is a list of fitted models or topic-word arrays over the same vocabulary (e.g. fits at different seeds, or on bootstrap resamples). You control exactly what is compared.
• From a corpus (the convenience overload, matching :func:bootstrap_stability's docs + num_topics convention): pass the corpus as runs together with num_topics= (k= is accepted as an alias, matching :func:bootstrap_stability) and/or seeds=, and it fits one model per seed for you (LDA(num_topics=k, seed=s) by default, or your model_factory) before scoring. seeds defaults to range(5); extra keywords / fit_kwargs= forward to each fit.

Either way, each later run's topics are matched to the first run's, and stability is the mean Jaccard overlap of their top-topn words. Returns a float in [0, 1]; higher means more reproducible topics. Pass per_topic=True for the per-topic vector instead — one matched-Jaccard mean per reference topic (index-aligned to runs[0]), so you can see which topics are fragile rather than only the corpus-wide average (#775 T3.3).

If every run is bit-identical to the first, a stability of 1.0 is meaningless — the runs never varied. This most often bites when the runs are the same deterministic fit repeated: models with a deterministic initialization (e.g. NMF/LSA with the default init="nndsvd") ignore seed, so [NMF(seed=s).fit(docs) for s in range(5)] is five copies of one fit. A UserWarning is emitted in that case; refit with init="random" (which does respond to seed) or measure stability on bootstrap resamples of the documents instead.

topica.ensemble

Ensemble topic modeling: combine several independent fits into one consensus.

A single topic-model fit is a draw from a noisy procedure — change the seed or a hyperparameter and the topics shift, sometimes a lot (Hoyle et al. 2022, "Are Neural Topic Models Broken?"). Combining several independent runs is more reliable than any one run: across Hoyle et al.'s experiments the ensemble improves on the median run in 97% of contexts and never loses to the worst. This module builds that consensus.

It is the natural follow-on to :func:~topica.select.select_model, which fits N runs at a fixed K. Instead of picking the best run with plot_models, ensemble combines all of them.

Three methods are available:

method="cluster" (default) reproduces Hoyle et al. §6. Pool the topics from every run (m runs of K topics each give m·K topics), measure the pairwise distance between them — a blend lambda_·D(topic-word) + (1-lambda_)·D(doc-topic) using a top-weighted rank distance (Rank-Biased Overlap, or average Jaccard) — cluster the pooled topics into K groups, and take the element-wise mean within each cluster. Clustering does not force a one-to-one match, so a topic that splits or merges across runs is handled naturally, and a cluster only a few runs contributed to is flagged as low-support.

method="align" is a lighter, fully deterministic alternative (the Miller & McCoy 2017 / Mäntylä et al. 2018 lineage): align every run's topics one-to-one to a single reference run (Hungarian matching on the topic-word distributions) and average the aligned topics. No clustering, no Θ, no λ.

method="stable" derives from gensim's EnsembleLda (Brigl 2019). It does not fix K: it pools the topics, measures an asymmetric masked-cosine distance between them, runs Checkback DBSCAN (CBDBSCAN) to find dense, reproducible "cores", and keeps only the clusters with enough cores as stable topics (averaging their members). Unstable topics — those that do not recur densely across runs — are discarded as noise rather than averaged in, so the number of consensus topics is discovered from the data. On well-separated inputs it agrees with gensim bit-for-bit (parity/ensemblelda_gensim_compare.py); it diverges deliberately on two edge cases where gensim degenerates — small-vocabulary rank masking and scan-order-dependent core validation — documented at _rank_mask and _cbdbscan.

The result duck-types as a fitted model for the model-neutral analysis surface (it exposes topic_word, doc_topic, and vocabulary), so the consensus flows straight into :func:~topica.evaluate.coherence, the diagnostics, and the rest. Each ensemble topic carries a stability score and a reliable flag, so a consensus topic the individual runs do not actually agree on is marked, not silently trusted.

__cached__ module-attribute

__cached__ = '/home/runner/work/topica/topica/python/topica/__pycache__/ensemble.cpython-311.pyc'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__doc__ module-attribute

__doc__ = 'Ensemble topic modeling: combine several independent fits into one consensus.\n\nA single topic-model fit is a draw from a noisy procedure — change the seed or a\nhyperparameter and the topics shift, sometimes a lot (Hoyle et al. 2022, "Are\nNeural Topic Models Broken?"). Combining several independent runs is more reliable\nthan any one run: across Hoyle et al.\'s experiments the ensemble improves on the\nmedian run in 97% of contexts and never loses to the worst. This module builds\nthat consensus.\n\nIt is the natural follow-on to :func:`~topica.select.select_model`, which fits N runs at a\nfixed K. Instead of *picking* the best run with ``plot_models``, ``ensemble``\n*combines* all of them.\n\nThree methods are available:\n\n``method="cluster"`` (default) reproduces Hoyle et al. §6. Pool the topics from\nevery run (m runs of K topics each give m·K topics), measure the pairwise distance\nbetween them — a blend ``lambda_·D(topic-word) + (1-lambda_)·D(doc-topic)`` using a\ntop-weighted rank distance (Rank-Biased Overlap, or average Jaccard) — cluster the\npooled topics into K groups, and take the element-wise mean within each cluster.\nClustering does not force a one-to-one match, so a topic that splits or merges\nacross runs is handled naturally, and a cluster only a few runs contributed to is\nflagged as low-support.\n\n``method="align"`` is a lighter, fully deterministic alternative (the Miller &\nMcCoy 2017 / Mäntylä et al. 2018 lineage): align every run\'s topics one-to-one to\na single reference run (Hungarian matching on the topic-word distributions) and\naverage the aligned topics. No clustering, no Θ, no λ.\n\n``method="stable"`` derives from gensim\'s ``EnsembleLda`` (Brigl 2019). It does not\nfix K: it pools the topics, measures an asymmetric masked-cosine distance between\nthem, runs Checkback DBSCAN (CBDBSCAN) to find dense, reproducible "cores", and\nkeeps only the clusters with enough cores as *stable topics* (averaging their\nmembers). Unstable topics — those that do not recur densely across runs — are\ndiscarded as noise rather than averaged in, so the number of consensus topics is\ndiscovered from the data. On well-separated inputs it agrees with gensim\nbit-for-bit (``parity/ensemblelda_gensim_compare.py``); it diverges deliberately on\ntwo edge cases where gensim degenerates — small-vocabulary rank masking and\nscan-order-dependent core validation — documented at ``_rank_mask`` and\n``_cbdbscan``.\n\nThe result duck-types as a fitted model for the model-neutral analysis surface (it\nexposes ``topic_word``, ``doc_topic``, and ``vocabulary``), so the consensus flows\nstraight into :func:`~topica.evaluate.coherence`, the diagnostics, and the rest. Each\nensemble topic carries a ``stability`` score and a ``reliable`` flag, so a\nconsensus topic the individual runs do not actually agree on is marked, not\nsilently trusted.\n'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__file__ module-attribute

__file__ = '/home/runner/work/topica/topica/python/topica/ensemble.py'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__name__ module-attribute

__name__ = 'topica.ensemble'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__package__ module-attribute

__package__ = 'topica'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

EnsembleResult

Consensus of several topic-model fits, returned by :func:ensemble.

Exposes topic_word, doc_topic, and vocabulary so it can be passed wherever a fitted model is accepted by the model-neutral analysis functions (:func:~topica.evaluate.coherence, the diagnostics surface, :func:~topica.evaluate.align_topics).

Attributes:

Name	Type	Description
topic_word	``(K, V)`` averaged, row-normalized topic-word matrix.
doc_topic	``(D, K)`` averaged document-topic matrix, or ``None`` when the	runs were not fit on the same documents in the same order.
vocabulary	the shared vocabulary, or ``None`` when raw arrays were passed.
stability	``(K,)`` per-topic consistency in ``[0, 1]``. For ``"cluster"`` and	"stable" it is one minus the mean pairwise distance among the run topics that formed the cluster; for "align" it is the mean top-word Jaccard with the matched run topics. 1.0 means the contributing topics are identical -- how many distinct runs contributed is recorded separately by support. For "cluster" and "stable", a topic only one run contributed, with nothing to corroborate it, scores 0.0 (a lone run is not agreement), so it never looks like a trustworthy consensus.
support	``(K,)`` how well-backed each topic is. For ``"cluster"`` and	"stable" it is the fraction of runs that contributed a topic to the cluster (1.0 = all runs found it); for "align" it is the match margin over the next-best run topic. A small value means few runs really support the topic.
reliable	``(K,)`` bool — ``stability >= 0.5`` *and* well-supported. An	unreliable topic is a consensus the individual runs do not agree on; treat it with suspicion. ("stable" topics are reproducible by construction, so this is usually all True -- except a single-run core, reachable when min_cores == 1, which scores 0.0 and is not reliable.)
agreement	scalar mean of ``stability`` — an overall "how reproducible is this	K?" number (nan if "stable" found no topics).
method	``"cluster"``, ``"align"``, or ``"stable"``.
cluster_sizes	``(K,)`` number of run topics in each cluster (``"cluster"``	and "stable"; None for "align").
reference	index of the reference run (``"align"`` only; ``None`` for	"cluster").
n_runs	number of fits combined.
runs	the input fits, in the order given.
agreement_ci	``(lo, hi)`` 95% bootstrap CI for ``agreement`` (a normal	interval centered on the estimate with a half-width from the bootstrap standard error, clipped to [0, 1]), or None unless ensemble(..., n_boot>0) was requested. Resamples the runs with replacement and recomputes the consensus, so it says whether a difference in agreement across K (or model families) is real or just noise from which runs were combined.
agreement_se	bootstrap standard error of ``agreement`` (``None`` unless	bootstrapped).
stability_ci	``(K, 2)`` per-topic bootstrap CI for ``stability`` (matched	back to the base topics), or None. Only produced for "cluster" and "align" (fixed K); "stable" gives the scalar agreement_ci only.

__annotations__ class-attribute

__annotations__ = {'topic_word': 'np.ndarray', 'doc_topic': 'np.ndarray | None', 'vocabulary': 'list | None', 'stability': 'np.ndarray', 'support': 'np.ndarray', 'reliable': 'np.ndarray', 'agreement': 'float', 'method': 'str', 'cluster_sizes': 'np.ndarray | None', 'reference': 'int | None', 'n_runs': 'int', 'runs': 'list', 'agreement_ci': 'tuple | None', 'agreement_se': 'float | None', 'stability_ci': 'np.ndarray | None'}

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)

__dataclass_fields__ class-attribute

__dataclass_fields__ = {'topic_word': Field(name='topic_word',type='np.ndarray',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'doc_topic': Field(name='doc_topic',type='np.ndarray | None',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'vocabulary': Field(name='vocabulary',type='list | None',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'stability': Field(name='stability',type='np.ndarray',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'support': Field(name='support',type='np.ndarray',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'reliable': Field(name='reliable',type='np.ndarray',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'agreement': Field(name='agreement',type='float',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'method': Field(name='method',type='str',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'cluster_sizes': Field(name='cluster_sizes',type='np.ndarray | None',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'reference': Field(name='reference',type='int | None',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'n_runs': Field(name='n_runs',type='int',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'runs': Field(name='runs',type='list',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=False,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'agreement_ci': Field(name='agreement_ci',type='tuple | None',default=None,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'agreement_se': Field(name='agreement_se',type='float | None',default=None,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'stability_ci': Field(name='stability_ci',type='np.ndarray | None',default=None,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=False,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD)}

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)

__doc__ class-attribute

__doc__ = 'Consensus of several topic-model fits, returned by :func:`ensemble`.\n\n    Exposes ``topic_word``, ``doc_topic``, and ``vocabulary`` so it can be passed\n    wherever a fitted model is accepted by the model-neutral analysis functions\n    (:func:`~topica.evaluate.coherence`, the diagnostics surface, :func:`~topica.evaluate.align_topics`).\n\n    Attributes\n    ----------\n    topic_word : ``(K, V)`` averaged, row-normalized topic-word matrix.\n    doc_topic : ``(D, K)`` averaged document-topic matrix, or ``None`` when the\n        runs were not fit on the same documents in the same order.\n    vocabulary : the shared vocabulary, or ``None`` when raw arrays were passed.\n    stability : ``(K,)`` per-topic consistency in ``[0, 1]``. For ``"cluster"`` and\n        ``"stable"`` it is one minus the mean pairwise distance among the run\n        topics that formed the cluster; for ``"align"`` it is the mean top-word\n        Jaccard with the matched run topics. 1.0 means the contributing topics are\n        identical -- how many *distinct* runs contributed is recorded separately by\n        ``support``. For ``"cluster"`` and ``"stable"``, a topic only one run\n        contributed, with nothing to corroborate it, scores 0.0 (a lone run is not\n        agreement), so it never looks like a trustworthy consensus.\n    support : ``(K,)`` how well-backed each topic is. For ``"cluster"`` and\n        ``"stable"`` it is the fraction of runs that contributed a topic to the\n        cluster (1.0 = all runs found it); for ``"align"`` it is the match margin\n        over the next-best run topic. A small value means few runs really support\n        the topic.\n    reliable : ``(K,)`` bool — ``stability >= 0.5`` *and* well-supported. An\n        unreliable topic is a consensus the individual runs do not agree on; treat\n        it with suspicion. (``"stable"`` topics are reproducible by construction,\n        so this is usually all ``True`` -- except a single-run core, reachable when\n        ``min_cores == 1``, which scores 0.0 and is not reliable.)\n    agreement : scalar mean of ``stability`` — an overall "how reproducible is this\n        K?" number (``nan`` if ``"stable"`` found no topics).\n    method : ``"cluster"``, ``"align"``, or ``"stable"``.\n    cluster_sizes : ``(K,)`` number of run topics in each cluster (``"cluster"``\n        and ``"stable"``; ``None`` for ``"align"``).\n    reference : index of the reference run (``"align"`` only; ``None`` for\n        ``"cluster"``).\n    n_runs : number of fits combined.\n    runs : the input fits, in the order given.\n    agreement_ci : ``(lo, hi)`` 95% bootstrap CI for ``agreement`` (a normal\n        interval centered on the estimate with a half-width from the bootstrap\n        standard error, clipped to ``[0, 1]``), or ``None`` unless\n        ``ensemble(..., n_boot>0)`` was requested. Resamples the runs with\n        replacement and recomputes the consensus, so it says whether a difference\n        in ``agreement`` across K (or model families) is real or just noise from\n        which runs were combined.\n    agreement_se : bootstrap standard error of ``agreement`` (``None`` unless\n        bootstrapped).\n    stability_ci : ``(K, 2)`` per-topic bootstrap CI for ``stability`` (matched\n        back to the base topics), or ``None``. Only produced for ``"cluster"`` and\n        ``"align"`` (fixed K); ``"stable"`` gives the scalar ``agreement_ci`` only.\n    '

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__match_args__ class-attribute

__match_args__ = ('topic_word', 'doc_topic', 'vocabulary', 'stability', 'support', 'reliable', 'agreement', 'method', 'cluster_sizes', 'reference', 'n_runs', 'runs', 'agreement_ci', 'agreement_se', 'stability_ci')

Built-in immutable sequence.

If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.

If the argument is a tuple, the return value is the same object.

__module__ class-attribute

__module__ = 'topica.ensemble'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__weakref__ property

__weakref__

list of weak references to the object

top_words

top_words(n=10, *, weights=False)

Top-n terms per ensemble topic, matching the fitted-model contract so the result drops into the analysis surface. Returns bare terms; pass weights=True for (term, probability) pairs. The term is a word when a vocabulary is known, else the integer term index.

ensemble

ensemble(runs, *, method='cluster', num_topics=None, lambda_=0.5, distance='rbo', topn=10, reference='medoid', metric='cosine', weights=None, eps=0.1, min_samples=None, min_cores=None, masking='mass', masking_threshold=None, n_boot=0, boot_seed=0)

Combine several topic-model fits into one consensus model.

The consensus is more reliable than any single run — it beats the median run and rarely loses to the best (Hoyle et al. 2022). This is the natural follow-on to :func:~topica.select.select_model: fit N runs, then combine them here instead of picking one.

Parameters:

Name	Type	Description	Default
runs	a list of fitted models (or ``(K, V)`` topic-word arrays sharing a	vocabulary), or a :class:~topica.validation.SelectModelResult. All runs must share the same K and vocabulary.	required
method	``"cluster"`` (default) reproduces Hoyle et al. §6 — pool the topics	from all runs, cluster them, and average within each cluster. "align" matches every run's topics to one reference run and averages the aligned topics (simpler, deterministic, no document-topic distance).	'cluster'
num_topics	number of consensus topics for ``"cluster"`` (default: the runs'	K). Ignored by "align" (which always returns K).	None
lambda_	``"cluster"`` only — weight on the topic-word distance when pooling	topics; 1 - lambda_ weights the document-topic distance. Falls back to 1.0 when the runs were not fit on the same documents.	0.5
distance	``"cluster"`` only — the top-weighted rank distance between topics:	"rbo" (Rank-Biased Overlap, default) or "jaccard" (average Jaccard).	'rbo'
topn	top-word (and top-document) count for the distances and diagnostics.		10
reference	``"align"`` only — which run anchors the matching. ``"medoid"``	(default) picks the run that aligns most cheaply to all others; "first" uses run 0; an int uses that run.	'medoid'
metric	``"align"`` only — topic-word distance for the matching, ``"cosine"``	(default) or "js".	'cosine'
weights	optional per-run weights (length ``n_runs``) for a weighted average —	e.g. down-weight low-coherence runs. None (default) weights equally. Used by "cluster" and "align".	None
eps	``"stable"`` only —	the gensim EnsembleLda knobs. eps (default 0.1) is the CBDBSCAN neighbor radius; min_samples (default int(n_runs/2)) the neighbors needed to be a core; min_cores (default min(3, n_runs//4 + 1)) the cores a cluster needs to count as a stable topic; masking is "mass" (default) or "rank" and masking_threshold its cutoff (gensim defaults 0.95 / 0.11). A larger eps or smaller min_cores yields more (looser) stable topics.	0.1
min_samples	``"stable"`` only —	the gensim EnsembleLda knobs. eps (default 0.1) is the CBDBSCAN neighbor radius; min_samples (default int(n_runs/2)) the neighbors needed to be a core; min_cores (default min(3, n_runs//4 + 1)) the cores a cluster needs to count as a stable topic; masking is "mass" (default) or "rank" and masking_threshold its cutoff (gensim defaults 0.95 / 0.11). A larger eps or smaller min_cores yields more (looser) stable topics.	0.1
min_cores	``"stable"`` only —	the gensim EnsembleLda knobs. eps (default 0.1) is the CBDBSCAN neighbor radius; min_samples (default int(n_runs/2)) the neighbors needed to be a core; min_cores (default min(3, n_runs//4 + 1)) the cores a cluster needs to count as a stable topic; masking is "mass" (default) or "rank" and masking_threshold its cutoff (gensim defaults 0.95 / 0.11). A larger eps or smaller min_cores yields more (looser) stable topics.	0.1
masking	``"stable"`` only —	the gensim EnsembleLda knobs. eps (default 0.1) is the CBDBSCAN neighbor radius; min_samples (default int(n_runs/2)) the neighbors needed to be a core; min_cores (default min(3, n_runs//4 + 1)) the cores a cluster needs to count as a stable topic; masking is "mass" (default) or "rank" and masking_threshold its cutoff (gensim defaults 0.95 / 0.11). A larger eps or smaller min_cores yields more (looser) stable topics.	0.1
masking_threshold	``"stable"`` only —	the gensim EnsembleLda knobs. eps (default 0.1) is the CBDBSCAN neighbor radius; min_samples (default int(n_runs/2)) the neighbors needed to be a core; min_cores (default min(3, n_runs//4 + 1)) the cores a cluster needs to count as a stable topic; masking is "mass" (default) or "rank" and masking_threshold its cutoff (gensim defaults 0.95 / 0.11). A larger eps or smaller min_cores yields more (looser) stable topics.	0.1
n_boot	if ``> 0``, bootstrap the ``agreement`` (and per-topic ``stability``	for "cluster"/"align") by resampling the runs with replacement n_boot times and recomputing the consensus, filling EnsembleResult.agreement_ci / agreement_se / stability_ci. Off by default; a few hundred resamples is typical. Tells you whether a difference in agreement is real or just noise from the run sample.	0
boot_seed	RNG seed for the bootstrap resampling (default ``0``), so the CI is	reproducible.	0

Returns:

Name	Type	Description
An	class:`EnsembleResult`. It exposes ``topic_word``, ``doc_topic``, and
	``vocabulary``, so it passes straight into :func:`~topica.evaluate.coherence`, the
	diagnostics, and other model-neutral analyses. Per-topic ``stability`` and
	``reliable`` flags mark consensus topics the individual runs do not agree on.

cross_ensemble

cross_ensemble(models, texts=None, *, method='cluster', num_topics=None, lambda_=0.5, distance='rbo', topn=10, weights=None) -> EnsembleResult

Combine several topic-model fits from different architectures into one consensus.

Unlike ensemble, cross_ensemble allows combining models with different architectures (e.g. classical parametric models like LDA/STM and neural embedding models like BERTopic) and automatically intersects/aligns their vocabularies if they expose a .vocabulary attribute.

Parameters:

Name	Type	Description	Default
models	list of fitted model instances.		required
texts	optional text corpus / tokenized documents, used for validating document counts.		None
method	``"cluster"`` (default) - pools and clusters the topics.		'cluster'
num_topics	number of consensus topics (default: median K of the input models).		None
lambda_	weight on topic-word distance vs document-topic distance.		0.5
distance	distance metric for clustering (``"rbo"`` or ``"jaccard"``).		'rbo'
topn	number of top words to use for distance calculation.		10
weights	optional per-model weights.		None

Returns:

Type	Description
An ``EnsembleResult``.

topica.EnsembleResult

Consensus of several topic-model fits, returned by :func:ensemble.

Exposes topic_word, doc_topic, and vocabulary so it can be passed wherever a fitted model is accepted by the model-neutral analysis functions (:func:~topica.evaluate.coherence, the diagnostics surface, :func:~topica.evaluate.align_topics).

Attributes:

Name	Type	Description
topic_word	``(K, V)`` averaged, row-normalized topic-word matrix.
doc_topic	``(D, K)`` averaged document-topic matrix, or ``None`` when the	runs were not fit on the same documents in the same order.
vocabulary	the shared vocabulary, or ``None`` when raw arrays were passed.
stability	``(K,)`` per-topic consistency in ``[0, 1]``. For ``"cluster"`` and	"stable" it is one minus the mean pairwise distance among the run topics that formed the cluster; for "align" it is the mean top-word Jaccard with the matched run topics. 1.0 means the contributing topics are identical -- how many distinct runs contributed is recorded separately by support. For "cluster" and "stable", a topic only one run contributed, with nothing to corroborate it, scores 0.0 (a lone run is not agreement), so it never looks like a trustworthy consensus.
support	``(K,)`` how well-backed each topic is. For ``"cluster"`` and	"stable" it is the fraction of runs that contributed a topic to the cluster (1.0 = all runs found it); for "align" it is the match margin over the next-best run topic. A small value means few runs really support the topic.
reliable	``(K,)`` bool — ``stability >= 0.5`` *and* well-supported. An	unreliable topic is a consensus the individual runs do not agree on; treat it with suspicion. ("stable" topics are reproducible by construction, so this is usually all True -- except a single-run core, reachable when min_cores == 1, which scores 0.0 and is not reliable.)
agreement	scalar mean of ``stability`` — an overall "how reproducible is this	K?" number (nan if "stable" found no topics).
method	``"cluster"``, ``"align"``, or ``"stable"``.
cluster_sizes	``(K,)`` number of run topics in each cluster (``"cluster"``	and "stable"; None for "align").
reference	index of the reference run (``"align"`` only; ``None`` for	"cluster").
n_runs	number of fits combined.
runs	the input fits, in the order given.
agreement_ci	``(lo, hi)`` 95% bootstrap CI for ``agreement`` (a normal	interval centered on the estimate with a half-width from the bootstrap standard error, clipped to [0, 1]), or None unless ensemble(..., n_boot>0) was requested. Resamples the runs with replacement and recomputes the consensus, so it says whether a difference in agreement across K (or model families) is real or just noise from which runs were combined.
agreement_se	bootstrap standard error of ``agreement`` (``None`` unless	bootstrapped).
stability_ci	``(K, 2)`` per-topic bootstrap CI for ``stability`` (matched	back to the base topics), or None. Only produced for "cluster" and "align" (fixed K); "stable" gives the scalar agreement_ci only.

__annotations__ class-attribute

__annotations__ = {'topic_word': 'np.ndarray', 'doc_topic': 'np.ndarray | None', 'vocabulary': 'list | None', 'stability': 'np.ndarray', 'support': 'np.ndarray', 'reliable': 'np.ndarray', 'agreement': 'float', 'method': 'str', 'cluster_sizes': 'np.ndarray | None', 'reference': 'int | None', 'n_runs': 'int', 'runs': 'list', 'agreement_ci': 'tuple | None', 'agreement_se': 'float | None', 'stability_ci': 'np.ndarray | None'}

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)

__dataclass_fields__ class-attribute

__dataclass_fields__ = {'topic_word': Field(name='topic_word',type='np.ndarray',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'doc_topic': Field(name='doc_topic',type='np.ndarray | None',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'vocabulary': Field(name='vocabulary',type='list | None',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'stability': Field(name='stability',type='np.ndarray',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'support': Field(name='support',type='np.ndarray',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'reliable': Field(name='reliable',type='np.ndarray',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'agreement': Field(name='agreement',type='float',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'method': Field(name='method',type='str',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'cluster_sizes': Field(name='cluster_sizes',type='np.ndarray | None',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'reference': Field(name='reference',type='int | None',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'n_runs': Field(name='n_runs',type='int',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'runs': Field(name='runs',type='list',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=False,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'agreement_ci': Field(name='agreement_ci',type='tuple | None',default=None,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'agreement_se': Field(name='agreement_se',type='float | None',default=None,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'stability_ci': Field(name='stability_ci',type='np.ndarray | None',default=None,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=False,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD)}

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)

__doc__ class-attribute

__doc__ = 'Consensus of several topic-model fits, returned by :func:`ensemble`.\n\n    Exposes ``topic_word``, ``doc_topic``, and ``vocabulary`` so it can be passed\n    wherever a fitted model is accepted by the model-neutral analysis functions\n    (:func:`~topica.evaluate.coherence`, the diagnostics surface, :func:`~topica.evaluate.align_topics`).\n\n    Attributes\n    ----------\n    topic_word : ``(K, V)`` averaged, row-normalized topic-word matrix.\n    doc_topic : ``(D, K)`` averaged document-topic matrix, or ``None`` when the\n        runs were not fit on the same documents in the same order.\n    vocabulary : the shared vocabulary, or ``None`` when raw arrays were passed.\n    stability : ``(K,)`` per-topic consistency in ``[0, 1]``. For ``"cluster"`` and\n        ``"stable"`` it is one minus the mean pairwise distance among the run\n        topics that formed the cluster; for ``"align"`` it is the mean top-word\n        Jaccard with the matched run topics. 1.0 means the contributing topics are\n        identical -- how many *distinct* runs contributed is recorded separately by\n        ``support``. For ``"cluster"`` and ``"stable"``, a topic only one run\n        contributed, with nothing to corroborate it, scores 0.0 (a lone run is not\n        agreement), so it never looks like a trustworthy consensus.\n    support : ``(K,)`` how well-backed each topic is. For ``"cluster"`` and\n        ``"stable"`` it is the fraction of runs that contributed a topic to the\n        cluster (1.0 = all runs found it); for ``"align"`` it is the match margin\n        over the next-best run topic. A small value means few runs really support\n        the topic.\n    reliable : ``(K,)`` bool — ``stability >= 0.5`` *and* well-supported. An\n        unreliable topic is a consensus the individual runs do not agree on; treat\n        it with suspicion. (``"stable"`` topics are reproducible by construction,\n        so this is usually all ``True`` -- except a single-run core, reachable when\n        ``min_cores == 1``, which scores 0.0 and is not reliable.)\n    agreement : scalar mean of ``stability`` — an overall "how reproducible is this\n        K?" number (``nan`` if ``"stable"`` found no topics).\n    method : ``"cluster"``, ``"align"``, or ``"stable"``.\n    cluster_sizes : ``(K,)`` number of run topics in each cluster (``"cluster"``\n        and ``"stable"``; ``None`` for ``"align"``).\n    reference : index of the reference run (``"align"`` only; ``None`` for\n        ``"cluster"``).\n    n_runs : number of fits combined.\n    runs : the input fits, in the order given.\n    agreement_ci : ``(lo, hi)`` 95% bootstrap CI for ``agreement`` (a normal\n        interval centered on the estimate with a half-width from the bootstrap\n        standard error, clipped to ``[0, 1]``), or ``None`` unless\n        ``ensemble(..., n_boot>0)`` was requested. Resamples the runs with\n        replacement and recomputes the consensus, so it says whether a difference\n        in ``agreement`` across K (or model families) is real or just noise from\n        which runs were combined.\n    agreement_se : bootstrap standard error of ``agreement`` (``None`` unless\n        bootstrapped).\n    stability_ci : ``(K, 2)`` per-topic bootstrap CI for ``stability`` (matched\n        back to the base topics), or ``None``. Only produced for ``"cluster"`` and\n        ``"align"`` (fixed K); ``"stable"`` gives the scalar ``agreement_ci`` only.\n    '

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__match_args__ class-attribute

__match_args__ = ('topic_word', 'doc_topic', 'vocabulary', 'stability', 'support', 'reliable', 'agreement', 'method', 'cluster_sizes', 'reference', 'n_runs', 'runs', 'agreement_ci', 'agreement_se', 'stability_ci')

Built-in immutable sequence.

If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.

If the argument is a tuple, the return value is the same object.

__module__ class-attribute

__module__ = 'topica.ensemble'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__weakref__ property

__weakref__

list of weak references to the object

top_words

top_words(n=10, *, weights=False)

Top-n terms per ensemble topic, matching the fitted-model contract so the result drops into the analysis surface. Returns bare terms; pass weights=True for (term, probability) pairs. The term is a word when a vocabulary is known, else the integer term index.

topica.cross_ensemble

cross_ensemble(models, texts=None, *, method='cluster', num_topics=None, lambda_=0.5, distance='rbo', topn=10, weights=None) -> EnsembleResult

Combine several topic-model fits from different architectures into one consensus.

Unlike ensemble, cross_ensemble allows combining models with different architectures (e.g. classical parametric models like LDA/STM and neural embedding models like BERTopic) and automatically intersects/aligns their vocabularies if they expose a .vocabulary attribute.

Parameters:

Name	Type	Description	Default
models	list of fitted model instances.		required
texts	optional text corpus / tokenized documents, used for validating document counts.		None
method	``"cluster"`` (default) - pools and clusters the topics.		'cluster'
num_topics	number of consensus topics (default: median K of the input models).		None
lambda_	weight on topic-word distance vs document-topic distance.		0.5
distance	distance metric for clustering (``"rbo"`` or ``"jaccard"``).		'rbo'
topn	number of top words to use for distance calculation.		10
weights	optional per-model weights.		None

Returns:

Type	Description
An ``EnsembleResult``.

MCMC convergence (topica.mcmc)

Single-chain autocorrelation and effective sample size for the collapsed-Gibbs models, computed from the retained log-likelihood trace and theta_draws. See the convergence section of the diagnostics guide.

topica.mcmc_diagnostics

mcmc_diagnostics(model, *, warn: bool = True) -> McmcDiagnostics

Single-chain MCMC diagnostics from a fitted Gibbs model's retained traces.

Reads the model's log-likelihood history and thinned theta_draws and reports the autocorrelation and effective sample size of each -- the honest "has the chain mixed?" companion to the convergence_tol plateau check.

Parameters:

Name	Type	Description	Default
model	a fitted topica model	Must expose theta_draws (fit with keep_theta_draws=True, the default). The log-likelihood diagnostics also need a non-empty log_likelihood_history / fit_history.	required
warn	bool	Warn when the model is not a Gibbs sampler. The variational models (STM, CTM, ...) converge a bound and have no MCMC chain; these diagnostics do not apply to them.	True

Returns:

Type	Description
McmcDiagnostics

Raises:

Type	Description
ValueError	If the model retained no theta_draws.

topica.McmcDiagnostics

Single-chain MCMC diagnostics for a fitted Gibbs model.

Attributes:

Name	Type	Description
model	str	The model class name.
inference	str or None	The model's inference engine from the registry ("gibbs" for the samplers these diagnostics are meant for).
n_draws	int	Number of retained theta_draws.
loglik_autocorr	ndarray or None	Autocorrelation of the log-likelihood trace, or None when the model recorded no trace (e.g. the WarpLDA / CVB0 sampler paths).
loglik_tau	float or None	Integrated autocorrelation time of the log-likelihood trace.
loglik_ess	float or None	Effective sample size of the log-likelihood trace (len(trace) / tau).
theta_ess	ndarray	Per-element effective sample size of theta_draws, shaped (num_docs, num_topics).

__annotations__ class-attribute

__annotations__ = {'model': 'str', 'inference': 'str | None', 'n_draws': 'int', 'loglik_autocorr': 'np.ndarray | None', 'loglik_tau': 'float | None', 'loglik_ess': 'float | None', 'theta_ess': 'np.ndarray'}

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)

__dataclass_fields__ class-attribute

__dataclass_fields__ = {'model': Field(name='model',type='str',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'inference': Field(name='inference',type='str | None',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'n_draws': Field(name='n_draws',type='int',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'loglik_autocorr': Field(name='loglik_autocorr',type='np.ndarray | None',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'loglik_tau': Field(name='loglik_tau',type='float | None',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'loglik_ess': Field(name='loglik_ess',type='float | None',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'theta_ess': Field(name='theta_ess',type='np.ndarray',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD)}

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)

__doc__ class-attribute

__doc__ = 'Single-chain MCMC diagnostics for a fitted Gibbs model.\n\n    Attributes\n    ----------\n    model : str\n        The model class name.\n    inference : str or None\n        The model\'s inference engine from the registry (``"gibbs"`` for the\n        samplers these diagnostics are meant for).\n    n_draws : int\n        Number of retained ``theta_draws``.\n    loglik_autocorr : numpy.ndarray or None\n        Autocorrelation of the log-likelihood trace, or ``None`` when the model\n        recorded no trace (e.g. the WarpLDA / CVB0 sampler paths).\n    loglik_tau : float or None\n        Integrated autocorrelation time of the log-likelihood trace.\n    loglik_ess : float or None\n        Effective sample size of the log-likelihood trace (``len(trace) / tau``).\n    theta_ess : numpy.ndarray\n        Per-element effective sample size of ``theta_draws``, shaped\n        ``(num_docs, num_topics)``.\n    '

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__match_args__ class-attribute

__match_args__ = ('model', 'inference', 'n_draws', 'loglik_autocorr', 'loglik_tau', 'loglik_ess', 'theta_ess')

Built-in immutable sequence.

If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.

If the argument is a tuple, the return value is the same object.

__module__ class-attribute

__module__ = 'topica.mcmc'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__weakref__ property

__weakref__

list of weak references to the object

summary

summary() -> str

A short human-readable table of the diagnostics.

topica.effective_sample_size

effective_sample_size(chain)

Effective sample size ESS = N / tau of a chain.

Parameters:

Name	Type	Description	Default
chain	1-D or 2-D array	A single scalar chain of length N, or an (N, params) matrix of params independent chains sharing the N draws (e.g. theta_draws flattened over docs and topics).	required

Returns:

Type	Description
float or ndarray	A scalar for a 1-D chain, or a (params,) array of per-column ESS for a 2-D chain. A degenerate (constant) column yields 0.0.

topica.integrated_autocorr_time

integrated_autocorr_time(x) -> float

Integrated autocorrelation time tau = 1 + 2 * sum_{t>=1} rho_t.

Uses Geyer's (1992) initial-positive-sequence estimator: the pair sums Gamma_m = rho_{2m} + rho_{2m+1} are summed until the first non-positive pair, which truncates the noisy tail of the empirical autocorrelation. Floored at 1 (an independent chain), so ESS never exceeds N.

A degenerate (constant) chain returns inf -- there is no information to resample from, so its effective sample size is zero.

topica.autocorrelation

autocorrelation(x, max_lag: int | None = None) -> np.ndarray

Autocorrelation function of a 1-D trace at lags 0..max_lag.

Parameters:

Name	Type	Description	Default
x	1-D sequence	The trace (e.g. a log-likelihood history or a single scalar chain).	required
max_lag	int	Highest lag to return. Defaults to len(x) - 1. Lag 0 is always 1.	None

Returns:

Type	Description
ndarray	rho[0..max_lag] with rho[0] == 1. A constant trace (zero variance) returns all zeros except rho[0].

Multi-chain Gelman-Rubin R-hat and cross-chain ESS, from several fits of the same model at different seeds. See the R-hat section of the diagnostics guide.

topica.multichain_diagnostics

multichain_diagnostics(chains, *, warmup: float = 0.5, metric: str = 'cosine', reference: int = 0, warn: bool = True) -> MultiChainDiagnostics

Gelman-Rubin diagnostics across several fitted Gibbs models.

Fit the same model at several seeds on the same corpus, pass the fitted models here, and this reports whether the chains agree. Two views are computed: R-hat and cross-chain ESS on the permutation-invariant log-likelihood trace, and per-topic R-hat on each topic's prevalence after the topics are aligned across chains (topic indices are label-switched across seeds, so alignment comes first).

Parameters:

Name	Type	Description	Default
chains	sequence of fitted topica models	At least two fits of the same model class on the same corpus at different seeds. The topic-level statistics need theta_draws on every chain (fit with keep_theta_draws=True, the default).	required
warmup	float	Fraction of each log-likelihood trace to discard from the front as burn-in before computing R-hat. The theta_draws are already the last post-warmup thinned samples, so the warmup fraction applies to the log-likelihood trace only.	0.5
metric	str	Topic-word distance metric for the cross-chain alignment (passed to :func:topica.evaluate.align_topics).	"cosine"
reference	int	Index of the chain whose topic order the others are aligned to.	0
warn	bool	Warn when the chains are not Gibbs samplers, disagree on class, or lack the traces a statistic needs.	True

Returns:

Type	Description
MultiChainDiagnostics

Raises:

Type	Description
ValueError	If fewer than two chains are supplied.

topica.MultiChainDiagnostics

Multi-chain (Gelman-Rubin) diagnostics for a set of fitted Gibbs models.

Attributes:

Name	Type	Description
model	str	The model class name (all chains must share it).
inference	str or None	The model's inference engine from the registry.
n_chains	int	Number of chains compared.
n_draws	int	Retained theta_draws per chain used for the topic-level statistics.
loglik_rhat	float or None	R-hat of the (post-warmup) log-likelihood trace across chains -- the permutation-invariant "did the chains agree?" headline. None when the chains recorded no usable log-likelihood trace.
loglik_ess	float or None	Cross-chain effective sample size of the log-likelihood trace.
loglik_n	int or None	Post-warmup trace length per chain used for the log-likelihood statistics.
topic_rhat	ndarray or None	Per-topic R-hat of each aligned topic's per-draw prevalence, shape (num_topics,). None when the chains retained no theta_draws.
topic_ess	ndarray or None	Per-topic cross-chain ESS of the aligned topic prevalence.
topic_alignment	ndarray or None	Per-topic alignment quality: the minimum top-word Jaccard of the topic to its reference-chain match across the other chains. Low values flag topics whose R-hat compares topics that did not line up.
reference	int or None	Index of the chain used as the alignment reference.

__annotations__ class-attribute

__annotations__ = {'model': 'str', 'inference': 'str | None', 'n_chains': 'int', 'n_draws': 'int', 'loglik_rhat': 'float | None', 'loglik_ess': 'float | None', 'loglik_n': 'int | None', 'topic_rhat': 'np.ndarray | None', 'topic_ess': 'np.ndarray | None', 'topic_alignment': 'np.ndarray | None', 'reference': 'int | None'}

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)

__dataclass_fields__ class-attribute

__dataclass_fields__ = {'model': Field(name='model',type='str',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'inference': Field(name='inference',type='str | None',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'n_chains': Field(name='n_chains',type='int',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'n_draws': Field(name='n_draws',type='int',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'loglik_rhat': Field(name='loglik_rhat',type='float | None',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'loglik_ess': Field(name='loglik_ess',type='float | None',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'loglik_n': Field(name='loglik_n',type='int | None',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'topic_rhat': Field(name='topic_rhat',type='np.ndarray | None',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'topic_ess': Field(name='topic_ess',type='np.ndarray | None',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'topic_alignment': Field(name='topic_alignment',type='np.ndarray | None',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'reference': Field(name='reference',type='int | None',default=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,default_factory=<dataclasses._MISSING_TYPE object at 0x7fa9ffd1ab90>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD)}

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)

__doc__ class-attribute

__doc__ = 'Multi-chain (Gelman-Rubin) diagnostics for a set of fitted Gibbs models.\n\n    Attributes\n    ----------\n    model : str\n        The model class name (all chains must share it).\n    inference : str or None\n        The model\'s inference engine from the registry.\n    n_chains : int\n        Number of chains compared.\n    n_draws : int\n        Retained ``theta_draws`` per chain used for the topic-level statistics.\n    loglik_rhat : float or None\n        R-hat of the (post-warmup) log-likelihood trace across chains -- the\n        permutation-invariant "did the chains agree?" headline. ``None`` when the\n        chains recorded no usable log-likelihood trace.\n    loglik_ess : float or None\n        Cross-chain effective sample size of the log-likelihood trace.\n    loglik_n : int or None\n        Post-warmup trace length per chain used for the log-likelihood statistics.\n    topic_rhat : numpy.ndarray or None\n        Per-topic R-hat of each aligned topic\'s per-draw prevalence, shape\n        ``(num_topics,)``. ``None`` when the chains retained no ``theta_draws``.\n    topic_ess : numpy.ndarray or None\n        Per-topic cross-chain ESS of the aligned topic prevalence.\n    topic_alignment : numpy.ndarray or None\n        Per-topic alignment quality: the minimum top-word Jaccard of the topic to\n        its reference-chain match across the other chains. Low values flag topics\n        whose R-hat compares topics that did not line up.\n    reference : int or None\n        Index of the chain used as the alignment reference.\n    '

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__match_args__ class-attribute

__match_args__ = ('model', 'inference', 'n_chains', 'n_draws', 'loglik_rhat', 'loglik_ess', 'loglik_n', 'topic_rhat', 'topic_ess', 'topic_alignment', 'reference')

Built-in immutable sequence.

If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.

If the argument is a tuple, the return value is the same object.

__module__ class-attribute

__module__ = 'topica.mcmc'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__weakref__ property

__weakref__

list of weak references to the object

converged property

converged

Whether every reported R-hat clears the conventional 1.01 threshold.

summary

summary() -> str

A short human-readable table of the multi-chain diagnostics.

topica.rhat

rhat(chains, *, split: bool = True, rank_normalize: bool = True) -> float

Gelman-Rubin R-hat (potential scale reduction) across MCMC chains.

Compares the variance between chains to the variance within each chain. At convergence the chains are draws from one distribution and R-hat -> 1; a value above roughly 1.01 means the chains have not mixed to a common target and the run needs more sweeps (Vehtari et al. 2021).

Parameters:

Name	Type	Description	Default
chains	2-D array or sequence of 1-D chains	(n_chains, n_draws), or a list of per-chain traces. Unequal-length chains are truncated to the shortest. At least two chains are required.	required
split	bool	Split each chain in half before comparing (split-R-hat), so a single chain that has not stopped drifting is caught as two disagreeing halves.	True
rank_normalize	bool	Rank-normalize the pooled draws first (the improved, tail-robust R-hat). Set False for the classical Gelman-Rubin statistic on the raw scale.	True

Returns:

Type	Description
float	The R-hat statistic. inf if the chains are constant but disagree; 1.0 if they share a single constant value.

Held-out likelihood

Build a within-corpus word-heldout set — the analogue of R stm's make.heldout — and score it under a fitted model to get document-completion log-likelihood.

topica.evaluate.make_heldout

make_heldout(corpus, *, prop_docs=0.5, prop_words=0.5, seed=0)

Build a within-corpus word-heldout set (R stm's make.heldout).

We sample floor(prop_docs * D) documents and remove floor(prop_words * len(doc)) randomly chosen token positions from each. The remaining tokens stay in the corpus; the removed tokens form the heldout set. Fit a model on .documents and score it with :func:eval_heldout.

Documents too short to split (fewer than 2 tokens, or those for which the split would leave 0 retained or 0 held-out tokens) are silently skipped rather than raising an error; the sampled set may therefore be slightly smaller than floor(prop_docs * D).

Parameters:

Name	Type	Description	Default
corpus	a ``Corpus`` (its ``.documents()`` method is called), a list of	raw strings (split on whitespace), or a list of token lists.	required
prop_docs	fraction of documents to sample; default 0.5.		0.5
prop_words	fraction of tokens to hold out per sampled document; default 0.5.		0.5
seed	numpy Generator seed for reproducibility.		0

Returns:

Name	Type	Description
A	class:`Heldout` dataclass. Pass ``.documents`` to ``model.fit`` and
	the whole object to :func:`eval_heldout`.

topica.evaluate.eval_heldout

eval_heldout(model, heldout, *, seed=0)

Score held-out words from :func:make_heldout under a fitted model (R stm's eval.heldout).

We infer each sampled document's topic mixture from its retained tokens (heldout.documents[doc_index]) via the model's transform, then score the withheld tokens under p(w) = sum_k theta_k * phi[k, w].

Requires that model was fit on heldout.documents (the training corpus returned by :func:make_heldout). Works for any generative model that exposes transform and topic_word: LDA, DMR, CTM, STM, HDP, LabeledLDA, SupervisedLDA, and ThreadTM. Note ThreadTM is scored tree-blind here (transform is called without parents, so every held-out document is treated as a root and the reply tree contributes nothing); for the tree-aware held-out comparison use :func:reply_completion instead. The keyword/anchored Gibbs models (keyATM, SeededLDA, SAGE, PA, PT) do not expose transform and so fall outside this diagnostic, and the embedding-cluster models (BERTopic, Top2Vec) define no document likelihood; both raise a clear error.

Parameters:

Name	Type	Description	Default
model	a fitted generative model (must have been fit on ``heldout.documents``).		required
heldout	a :class:`Heldout` returned by :func:`make_heldout`.		required
seed	RNG seed for the Gibbs ``transform`` (variational models ignore it).		0

Returns:

Name	Type	Description
A	class:`HeldoutResult` dataclass. The headline metric is
	``.mean_per_doc_loglik``; higher (less negative) is better.

Estimator conformance

Check any fitted model or model class against the topica estimator contract; returns a list of violation strings (empty means fully conformant).

topica.provenance.check_conformance

check_conformance(model_or_class) -> list[str]

Check model_or_class against the topica estimator contract.

Returns a list of violation strings. An empty list means the model satisfies every applicable tier requirement (or has a valid exemption). Does NOT look up KNOWN_GAPS or EXEMPT -- it reports all raw violations so the conformance test can categorize them. Call this from the test or from your own CI after adding a new estimator.

Parameters:

Name	Type	Description	Default
model_or_class	an estimator instance or class.		required

Returns:

Type	Description
list of str	Each entry is a human-readable description of the violation, e.g. "missing class attribute: topic_names".

Reporting

Model-neutral summaries that work on any fitted model.

topica.plot_report

plot_report(model, *, texts=None, timestamps=None, groups=None, n=8, coherence_type='c_v', title=None, figsize=None)

A one-figure overview of a fitted model, composed from topica's diagnostics.

Panels are adaptive: each is drawn only when its inputs and the model support it, so the report works across every model. Always included is the topic prevalence bar (mean doc_topic per topic, labelled with each topic's top words). Added when available:

• topic quality — coherence vs exclusivity (the stm quality frontier); a windowed coherence_type is used when texts is given (raw strings or token lists are both accepted), else UMass;
• topic correlation — the doc_topic correlation heatmap (K in 2..40);
• topics over time — mean prevalence per distinct timestamps value;
• topics per class — mean prevalence within each level of groups.

Returns a matplotlib Figure; save it with fig.savefig("report.png") or .pdf. Requires matplotlib (the only added dependency).

topica.topic_info

topic_info(model, texts=None, *, n=8, labels=None) -> list

One summary row per topic — the headline table for a fitted model.

Each row is a dict with topic (id), label, size (hard assignments), prevalence (mean of the topic's doc_topic column), and top_words (the top-n words, via model.top_words when available else the raw topic-word row). When texts is given each row also carries representative_docs, its n highest-loading documents. On a clustering model with outliers a final topic=-1 row reports the outlier count and carries no words. Rows are sorted by topic id.

labels overrides the labels for this table only; otherwise :func:topic_labels (custom labels over topic_names) is used.

topica.topics_over_time

topics_over_time(model, timestamps, *, normalize=True) -> dict

Mean topic prevalence at each distinct timestamp value.

timestamps is one value per document. For each distinct timestamp we average doc_topic over the documents stamped with it, giving a topic prevalence trajectory you can plot directly. With normalize=True each row is rescaled to sum to one (so it reads as a topic share at that time).

Returns {"labels": [sorted distinct timestamps], "prevalence": (T, K) array}.

topica.topics_per_class

topics_per_class(model, groups, *, ci=0.95)

Mean topic prevalence within each level of a grouping variable.

A thin wrapper over :func:topica.effects.by_strata on model.doc_topic: groups is one label per document, and the result is a list of per-stratum prevalence records (mean and confidence interval per topic).

topica.contrastive_topics

contrastive_topics(model, texts, groups, *, prior=0.01, informative=False, min_count=5, n_words=10, group_order=None)

Which topics most separate two groups, and the words that shift inside each.

This is the topic-conditional extension of :func:topica.fighting_words. A plain Fighting Words contrast pools the whole corpus into two bags of words; here we hold the topic fixed and ask, within topic t, how the two groups word it differently. Each document's word counts are weighted by its responsibility for the topic (model.doc_topic[d, t]), the weighted counts are split by group, and the Monroe-Colaresi-Quinn z-score is computed per topic. We report two complementary signals, since a topic both groups use equally can still split sharply on how they word it:

• usage_diff — mean doc_topic for group A minus group B: which topics one side simply talks about more.
• vocab_shift — the root-mean-square topic-conditional z over the words it keeps: how much the two groups diverge in their wording of the topic.

Works on any fitted model that exposes doc_topic and vocabulary (LDA, STM, DMR, CTM, keyATM, ...). texts must be the same documents, in the same order, that produced model.doc_topic.

Parameters:

Name	Type	Description	Default
model	a fitted topica model with ``doc_topic`` (D x K) and ``vocabulary``.		required
texts	sequence of token lists (``list[list[str]]``), one per document,	aligned row-for-row with model.doc_topic.	required
groups	sequence of one label per document. Must take exactly two distinct	values (a binary contrast).	required
prior	float	Dirichlet pseudocount passed to the z-score; see :func:fighting_words.	0.01
informative	bool	Use Monroe et al.'s informative (frequency-scaled) prior.	False
min_count	int	Within a topic, ignore words whose responsibility-weighted count across both groups is below this. Keeps the per-topic word lists and vocab_shift from being dominated by near-zero-mass words.	5
n_words	int	How many distinctive words to return per side, per topic.	10
group_order	(a, b)	Fix which group is A (positive z, positive usage_diff). Defaults to the two labels sorted, so the result is deterministic.	None

Returns:

Type	Description
list[dict], one row per topic sorted by descending ``abs(usage_diff)``. Each
row has ``topic`` (id), ``name`` (effective label), ``a_label``/``b_label``
(the two groups), ``usage_diff``, ``leans`` (the label that uses the topic
more), ``vocab_shift``, and ``a_words``/``b_words`` (lists of ``(word, z)``,
each side's most distinctive within-topic words).