Content-covariate diagnostics¶

Diagnostics for the content-covariate models (STM, STS, SAGE), which learn a group-specific topic-word tensor \(\beta_{k,g,v}\) — how each topic is worded by each group. The global topic-word average hides that variation; these read it back out.

They answer a question that recurs with content models: does a group's distinctive language land within a topic (one topic, worded differently by group) or fragment into parallel, group-skewed topics? topic_polarization measures the first; split_topics detects the second. The main lever that moves a fit between the two is the STM content prior content_prior_var (looser prior → more within-topic variation).

All read the group tensor through one adapter, so they work across model families — STM via topic_word_by_group, SAGE via its 3-D topic_word. For an STM fit with an ordered content_time covariate the group axis is the base-by-period cross (labels "base@period"), which content_trajectory / content_divergence read over ordered time. STS has a continuous sentiment axis rather than discrete groups, so the adapter discretizes it — evaluating topic_word_at(level) at the sentiment poles -1/0/+1 (negative/neutral/positive) by default; pass levels= to choose your own.

topica.content.topic_polarization ¶

topic_polarization(model, *, weights=None, levels=None) -> np.ndarray

Per-topic Jensen-Shannon divergence across groups, normalized to [0, 1].

For each topic k, JSD(beta_{k,1}, ..., beta_{k,G}) -- the spread of its group-specific wordings. 0 = every group words the topic identically; 1 = groups use disjoint vocabularies for it (maximal framing divergence).

Parameters:

Name	Type	Description	Default
`weights`	`array(num_groups)`	Group weights `p(g)` (e.g. group prevalence). Defaults to uniform.	`None`
`levels`	`mapping or sequence`	STS only: sentiment levels to discretize into groups (see :func:`group_topic_word`).	`None`

Returns:

Type	Description
`(ndarray, shape(num_topics))`

topica.content.group_exclusivity ¶

group_exclusivity(model, *, n=10, summary='min', levels=None) -> np.ndarray

Group-adjusted exclusivity per topic (a FREX-tensor summary), in [0, 1].

Extends the usual exclusivity beta_{k,v} / sum_j beta_{j,v} to the group tensor: excl_{k,g,v} = beta_{k,g,v} / sum_j sum_g' beta_{j,g',v}. For each (k, g) it averages the exclusivity of that group's top-n words, then reduces across groups by summary ("min" = worst-case group, the default; "mean" = average). High = the topic stays distinctive in every group's sub-vocabulary; low = at least one group's wording overlaps other topics.

levels (STS only) chooses the sentiment discretization (see :func:group_topic_word).

Returns:

Type	Description
`(ndarray, shape(num_topics))`

topica.content.split_topics ¶

split_topics(model, content, *, cos_thresh=0.6, skew_thresh=0.65)

Near-duplicate topics pulled apart by group prevalence (fragmentation).

When a single discourse fragments into parallel group-topics instead of living within one topic, you get two topics that (a) share vocabulary -- high cosine between their group-averaged topic-word rows -- yet (b) are each dominated by a different group in the document-topic loadings. This flags those pairs.

Parameters:

Name	Type	Description	Default
`content`	`sequence`	The per-document content-group labels passed to `fit` (one per row of `doc_topic`), used to compute each topic's group prevalence.	required
`cos_thresh`	`float`	Minimum cosine similarity of the two topics' word rows.	`0.6`
`skew_thresh`	`float`	Minimum single-group share of a topic's mass for it to count as group-skewed.	`0.65`

Returns:

Type	Description
list of dict, each ``{"pair": (k, j), "cosine": float, "skew": (float, float),
"groups": (label_k, label_j)}``.

topica.content.stratified_coherence ¶

stratified_coherence(model, texts, content, *, coherence_type='c_v', n=10, weights=None) -> np.ndarray

Group-stratified topic coherence for a content-covariate model.

Global coherence scores a topic's group-averaged words against the whole corpus, which hides that a topic can be coherent for one group and gibberish for another. This instead scores each group's own top words (TopWords(beta_{k,g})) against that group's own subcorpus (:math:\mathcal{D}_g, the documents with content == g), and averages the per-group coherences weighted by group prevalence:

.. math:: \text{Coherence}(k) = \sum_g p(g)\, \text{Coherence}(\text{TopWords}(\beta_{k,g}), \mathcal{D}_g)

Parameters:

Name	Type	Description	Default
`texts`	`Corpus, list of strings, or list of token lists`	The training corpus, aligned row-for-row with `content`.	required
`content`	`sequence`	The per-document content-group labels passed to `fit`.	required
`coherence_type`	one of ``"u_mass"``, ``"c_uci"``, ``"c_npmi"``, ``"c_v"``.		`'c_v'`
`weights`	`array(num_groups)`	Group weights `p(g)`. Defaults to each group's document share.	`None`

Returns:

Type	Description
`(ndarray, shape(num_topics))`

topica.content.diagnostics ¶

diagnostics(model, texts=None, content=None, *, n=10, coherence_type='c_v')

One per-topic content-diagnostics table for a content-covariate model.

Consolidates the content metrics into a single row-per-topic table: polarization (JSD across groups), group_exclusivity (worst-case group), and -- when texts and content are supplied -- stratified_coherence. Returns a pandas DataFrame if pandas is available, else a list of row dicts.

topica.content.group_topic_word ¶

group_topic_word(model, *, levels=None)

Normalize any content-covariate model to (beta_KGV, group_labels).

beta_KGV is (num_topics, num_groups, num_words) with each row a probability distribution over words. Raises ValueError for a model with no group-specific topic-word structure (not a content-covariate model, or fit without a content covariate).

Parameters:

Name	Type	Description	Default
`levels`	`mapping or sequence`	STS only: the sentiment levels to discretize its continuous content axis into groups. A `{value: label}` mapping or a sequence of numeric levels; defaults to the poles `-1`/`0`/`+1` (negative/neutral/positive). Ignored by the discrete-group models.	`None`

Wording over ordered time (STM `content_time`)¶

Fit STM with an ordered content_time= covariate and these readers trace how two groups word a topic across time: the per-period, per-word contrast p(w | g1) - p(w | g2), and the per-period whole-vocabulary distance between the groups. Both take ci=True for a design-preserving bootstrap that resamples documents (or whole cluster=s), refits, realigns the topic by anchor_words, and returns percentile bands — intervals widen where the random walk is least constrained (the first and last period).

from topica import content

stm = topica.STM(num_topics=20, seed=1)
stm.fit(docs, content=party, content_time=year, content_prior="l1")

tr = content.content_trajectory(stm, ["tax", "climate"],
                                groups=("Democrat", "Republican"), anchor_words=econ)
tr.to_frame()                       # word, period, estimate

dv = content.content_divergence(stm, groups=("Democrat", "Republican"),
                                anchor_words=econ, measure="hellinger",
                                ci=True, corpus=docs, fit_kwargs=fk)  # + bootstrap CI

topica.content.content_trajectory ¶

content_trajectory(model, words, *, groups=None, topic=None, anchor_words=None, ci=False, corpus=None, fit_kwargs=None, cluster=None, B=50, level=0.95, seed=0)

How two content groups word a topic differently, across ordered time.

Reads the STM content_time surface (fit with STM.fit(..., content_time=)) for the per-period, per-word contrast p(w | group1, period) - p(w | group2, period) on the target topic -- how the between-group wording contrast moves over ordered time, read off the smoothed STM surface rather than independent (group, period) cells.

Parameters:

Name	Type	Description	Default
`model`	`fitted STM (content_time)`	Fit with an ordered `content_time=` covariate.	required
`words`	`sequence[str]`	Words to trace (silently dropped if out of vocabulary).	required
`groups`	`(str, str)`	The two base content groups to contrast. Defaults to the first two.	`None`
`topic`	`int`	Target topic. Give this or `anchor_words`.	`None`
`anchor_words`	`sequence[str]`	Identify the target topic by top-word overlap. Required when `ci=True` so the topic can be realigned across bootstrap refits.	`None`
`ci`	`bool`	Attach a design-preserving bootstrap CI (needs `corpus` + `fit_kwargs`).	`False`
`corpus`		Bootstrap controls. `corpus` is the documents (a Corpus or list of token lists); `fit_kwargs` the STM refit arguments (`num_topics`, `content`, `content_time`, `content_prior`, ...); `cluster` a length-`num_docs` label to resample whole clusters (e.g. speaker) instead of documents.	`None`
`fit_kwargs`		Bootstrap controls. `corpus` is the documents (a Corpus or list of token lists); `fit_kwargs` the STM refit arguments (`num_topics`, `content`, `content_time`, `content_prior`, ...); `cluster` a length-`num_docs` label to resample whole clusters (e.g. speaker) instead of documents.	`None`
`cluster`		Bootstrap controls. `corpus` is the documents (a Corpus or list of token lists); `fit_kwargs` the STM refit arguments (`num_topics`, `content`, `content_time`, `content_prior`, ...); `cluster` a length-`num_docs` label to resample whole clusters (e.g. speaker) instead of documents.	`None`
`B`		Bootstrap controls. `corpus` is the documents (a Corpus or list of token lists); `fit_kwargs` the STM refit arguments (`num_topics`, `content`, `content_time`, `content_prior`, ...); `cluster` a length-`num_docs` label to resample whole clusters (e.g. speaker) instead of documents.	`None`
`level`		Bootstrap controls. `corpus` is the documents (a Corpus or list of token lists); `fit_kwargs` the STM refit arguments (`num_topics`, `content`, `content_time`, `content_prior`, ...); `cluster` a length-`num_docs` label to resample whole clusters (e.g. speaker) instead of documents.	`None`
`seed`		Bootstrap controls. `corpus` is the documents (a Corpus or list of token lists); `fit_kwargs` the STM refit arguments (`num_topics`, `content`, `content_time`, `content_prior`, ...); `cluster` a length-`num_docs` label to resample whole clusters (e.g. speaker) instead of documents.	`None`

Returns:

Type	Description
`ContentTrajectory`

topica.content.content_divergence ¶

content_divergence(model, *, groups=None, topic=None, anchor_words=None, measure='hellinger', ci=False, corpus=None, fit_kwargs=None, cluster=None, B=50, level=0.95, seed=0)

Whole-vocabulary distance between two groups' wording of a topic, per period.

Reads the STM content_time surface for the per-period Hellinger (default) or total-variation (measure="tv") distance between the two groups' topic-word distributions -- the whole-distribution group distance per period, pooled over the vocabulary so the aggregate has a usable bootstrap CI where single-word contrasts do not. Parameters mirror :func:content_trajectory.

Returns:

Type	Description
`ContentDivergence`

Choosing K with group-stratified coherence¶

topica.search_k accepts a "stratified_<type>" coherence metric for content models (model="stm" with a content covariate): it scores each group's own top words against that group's subcorpus, and reports group-adjusted exclusivity and mean polarization alongside.

res = topica.search_k(
    corpus, ks=[5, 10, 15], model="stm",
    prevalence=X, content=source,
    coherence_type="stratified_c_npmi",
)
# each row: k, coherence (stratified), exclusivity (group-adjusted), polarization

Content-covariate diagnostics¶

topica.content.topic_polarization ¶

topica.content.group_exclusivity ¶

topica.content.split_topics ¶

topica.content.stratified_coherence ¶

topica.content.diagnostics ¶

topica.content.group_topic_word ¶

Wording over ordered time (STM content_time)¶

topica.content.content_trajectory ¶

topica.content.content_divergence ¶

Choosing K with group-stratified coherence¶

Wording over ordered time (STM `content_time`)¶