keyATM toolkit¶
keyATM-specific workflow helpers live in topica.keyatm. The general post-hoc
diagnostics (top words, representative documents, coherence, pyLDAvis, covariate
effects, …) are model-agnostic and work on a fitted KeyATM directly: see the
Diagnostics and STM toolkit pages.
The fitted model also exposes the convergence trace keyATM's plot_modelfit
reports, as KeyATM.log_likelihood_history — a list of (iteration, per-token
log-likelihood) pairs (perplexity is exp(-log_likelihood)). The same trace is
available in the cross-model form as KeyATM.fit_history. Passing
convergence_tol to fit (default 0.0, disabled) opts into early stopping on
that trace: the Gibbs run halts once the relative change in the recorded
log-likelihood drops below the tolerance, and KeyATM.converged reports whether
it did. The base, covariate, and dynamic Gibbs backends support it; the CVB0
backend (sampler="cvb0") keeps no trace and never early-stops.
topica.keyatm.top_topics ¶
The n most prevalent topics in each document (≈ keyATM::top_topics).
Returns a list (one per document) of (topic_name, proportion) pairs,
sorted by descending document-topic proportion. Pass a fitted
:class:~topica.KeyATM (topic names are taken from it) or a raw theta
array.
topica.keyatm.by_strata ¶
Mean topic prevalence within each level of a document covariate
(≈ keyATM::by_strata_DocTopic).
Splits documents by their value in strata (one label per document) and,
for each level, reports the mean of each topic's proportion with a
normal-approximation confidence interval on that mean. This is keyATM's
descriptive answer to "how does topic prevalence differ across groups".
With nsims (and a fitted model as the first argument), the interval is
widened by the method of composition: the model's θ posterior is drawn for
you (logistic-normal for STM/CTM, Dirichlet for the Gibbs models — pass
corpus= so document lengths are available) and the per-stratum means are
pooled by Rubin's rules, so the topic-estimation uncertainty is propagated, not
just the across-document spread. For a regression with the same propagation use
:func:topica.stm.estimate_effect.
Returns a list of :class:StrataPrevalence, one per unique stratum (sorted).
[s.as_dict() for s in result] builds a table.
topica.keyatm.visualize_keywords ¶
Corpus frequency of each keyword (≈ keyATM::visualize_keywords).
For every keyword in every set, reports how common it is in docs so you
can catch keywords that are too rare to anchor a topic or so frequent they
dominate it — the diagnostic keyATM asks you to run before fitting.
Pass the built :class:~topica.Corpus you will fit, not the raw token
lists: frequency pruning (rm_top, min_doc_freq, max_doc_fraction)
can drop a substantive seed word from the vocabulary, and fit then
silently ignores it. Scoring against the corpus reflects what fit sees, so
a pruned seed shows count == 0 and is flagged; scoring against raw docs
reports the pre-pruning counts and can disagree with the fit (issue #743).
Returns a :class:~topica._results.KeywordDiagnostics (a dict) mapping
each keyword-set name to a list of dicts
{"keyword", "count", "proportion", "doc_freq", "in_vocab"} sorted by
descending proportion, where proportion is the keyword's share of all
corpus tokens, doc_freq is the number of documents containing it, and
in_vocab is False for a keyword fit will not see (count 0). Call
.to_frame() for one long DataFrame across all sets (a leading set
column names each row's keyword set). When a Corpus is passed, keywords absent
from its vocabulary raise a warning.
topica.keyatm.refine_keywords ¶
Drop keywords too rare to anchor a topic (≈ keyATM::refine_keywords).
Removes any keyword whose corpus count is below min_count or whose
document frequency is below min_doc_freq (so out-of-vocabulary keywords,
with count 0, always go). Keyword sets that end up empty are dropped, since
a keyword topic needs at least one surviving keyword.
As with :func:visualize_keywords, pass the built :class:~topica.Corpus
you will fit rather than the raw token lists, so a seed word removed by the
corpus's frequency pruning (rm_top / min_doc_freq / max_doc_fraction)
is refined out here too, instead of surviving on its raw pre-pruning count and
then being silently dropped by fit (issue #743).
Returns (refined, dropped) where refined is the cleaned keyword dict
and dropped maps each set name to the list of removed keywords. Set
verbose=True to print a short report.
Dynamic model: time-trend credible intervals¶
Per-period topic prevalence with credible bands from the dynamic keyATM posterior's retained MCMC theta draws.
topica.time_prevalence_ci ¶
Per-period topic prevalence with credible intervals from the dynamic keyATM posterior.
.. note::
This is dynamic-keyATM only — it reads that model's retained MCMC draws.
For prevalence over a time covariate on any other model (LDA, STM, …), use
:func:topica.effects.predicted_prevalence with a continuous year (optionally a
:func:topica.design.spline), which gives the same over-time curve with intervals
via the method of composition.
For a dynamic :class:~topica.KeyATM (fit with timestamps= and
keep_theta_draws=True), this computes per-period prevalence uncertainty
directly from the retained MCMC theta_draws. For each posterior draw and
each time period, the per-draw average of theta over the documents in that
period is computed, giving a (S, T, K) array of per-draw period-level
prevalences. The point estimate is the posterior mean over draws; ci_low
and ci_high are the empirical (1-ci)/2 and (1+ci)/2 quantiles; sd is
the posterior standard deviation.
The periods are ordered to match model.time_labels exactly, so the result
aligns with model.time_prevalence.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
A fitted dynamic :class: |
required | |
timestamps
|
One value per document — the same array passed to |
required | |
ci
|
Credible interval coverage (default 0.95 gives a 95 percent interval). |
0.95
|
|
normalize
|
When |
True
|
Returns:
| Type | Description |
|---|---|
class:`~topica._results.TimePrevalenceCI` (a ``dict``) with keys:
|
|
Call ``.to_frame()`` for a long tidy DataFrame, one row per (period, topic).
|
|