Analysis manifest¶
A portable, privacy-aware record of one fit: the inputs and decisions needed to
inspect or replay it, plus a verifier that reports what actually holds rather than
a single pass/fail. The manifest is not a second model format. It composes with
Corpus.save and model.save, and records fingerprints and settings, not raw
content.
We write the manifest with an explicit record_fit call. The explicit boundary
is deliberate: a call cannot honestly see preprocessing performed before it, so it
records only what it observes and marks the rest.
import topica
corpus = topica.Corpus.from_documents(tokens)
model = topica.STM(num_topics=20, seed=42)
model.fit(corpus, prevalence=X, iters=1000)
record = topica.record_fit(model, corpus, prevalence=X, prevalence_names=names, iters=1000)
record.add_decision("K", "Chose 20 after inspecting 15/20/25.")
record.save("analysis.topica.json")
# On another machine or session:
record = topica.AnalysisManifest.load("analysis.topica.json")
print(record.verify(corpus, model).summary())
Privacy¶
The default privacy="minimal" records the model, the environment, and coarse
corpus counts (document count, total tokens) only. It carries no length
distribution, vocabulary size, preprocessing detail, or raw content.
privacy="aggregate" opts those descriptive fields in. A content fingerprint,
which lets verify prove corpus identity, is a separate opt-in
(content_fingerprint=True) and is sensitive: a hash is not anonymisation,
and a small or public corpus can be brute-forced. privacy="full" (raw values)
is intentionally not part of this version.
What verify reports¶
verify re-derives fingerprints from the supplied corpus and model and reports a
status per field, never collapsing them into one flag: exact, input_changed,
artifact_changed, environment_changed (differs, but the model's determinism
class says it may still replay), or unverifiable (nothing recorded to check
against, a bounded component, or a fingerprint from a spec this build does not
recognise).
The determinism the manifest records is config-aware (issue #401), not the
coarse per-class registry tag: a cvb0 sampler or an init="random" fit is
recorded as seed-reproducible even when the model class is nominally bit-exact,
and determinism_detail carries the machine-readable replay_requires (the seed,
plus num_threads for the collapsed-Gibbs approximate parallel sampler) and any
caveats. Compute it directly for any model with topica.effective_determinism.
Recording evidence¶
record_fit(..., diagnostics=["coherence", "exclusivity"]) computes those
built-in topic-quality metrics and records each as computed evidence (its mean
over topics), or value=None with a note when a metric is not defined for the
model. You can also attach your own with record.add_diagnostic(name, value) and
interpretive notes with record.add_decision(key, note); both are stored as
visibly researcher-authored, never as tool-verified conclusions.
Comparing two fits¶
a.compare(b) diffs two manifests directly, with no corpus or model needed, and
reports per field same / changed / only_in_a / only_in_b / incomparable.
It answers "did these two runs use the same corpus, model, and inputs, or did
something change, and what?" — for example, whether a collaborator changed the
corpus or only the seed.
a = topica.AnalysisManifest.load("run-a.json")
b = topica.AnalysisManifest.load("run-b.json")
print(a.compare(b).summary())
Bundling the analysis¶
record.bundle(path, model=model, corpus=corpus) writes a self-contained,
content-addressed .zip: the manifest plus the saved artifacts, each file
named by its own hash, as one shareable, self-verifying unit.
record.bundle("analysis.zip", model=model) # manifest + model
record.bundle("analysis.zip", model=model, corpus=corpus, include_corpus=True)
bundle refuses to package a model or corpus whose fingerprints do not match the
manifest (guarding against bundling the wrong fit). AnalysisManifest.load_bundle(path)
reloads the manifest and checks every artifact's bytes against its
content-addressed name and the manifest reference, raising on a corrupt bundle or
artifacts that no longer match. This is an integrity / content-addressing check,
not authenticity — detecting a fully rewritten bundle (artifact, digest, and
reference all changed together) needs a signature, which is tracked separately.
AnalysisManifest.extract_bundle(path, dest) writes the artifacts out so you can
reload them (topica.LDA.load(...)). Bundling the corpus is opt-in and
sensitive — it embeds the raw tokens.
The analysis card¶
record.render(path) writes a self-contained HTML analysis card, and
record.to_markdown() returns a Markdown version for a notebook or Quarto. The
card shows only what the manifest recorded, so it cannot over-claim: researcher
decisions are labelled as authored (not tool-verified), diagnostics as computed
evidence, and fingerprints as verifiable identity rather than content. Pass a
VerifyResult to include the graded verification table.
Reference¶
topica.record_fit ¶
record_fit(model, corpus, *, prevalence=None, prevalence_names=None, privacy: str = 'minimal', content_fingerprint: bool = False, fingerprint_key: bytes | None = None, thread_count: int | None = None, diagnostics=None, diagnostics_n: int = 10, **fit_settings) -> AnalysisManifest
Record a fitted model on corpus as an :class:AnalysisManifest.
The explicit call is deliberate: it cannot honestly see preprocessing done before it, so it records only what it can observe and marks the rest.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
privacy
|
``"minimal"`` (default) or ``"aggregate"``. ``"full"`` is not in V1.
|
|
'minimal'
|
content_fingerprint
|
opt-in, **sensitive**. Adds an order-sensitive hash of
|
the corpus tokens so |
False
|
fingerprint_key
|
optional key for keyed/salted fingerprints (private
|
verification only; keyed digests are marked non-portable). |
None
|
diagnostics
|
optional list of built-in diagnostics to compute and record as
|
evidence: any of :data: |
None
|
diagnostics_n
|
the top-N words each diagnostic uses (default 10).
|
|
10
|
fit_settings
|
the fit arguments you passed (``iters=``, …), recorded as
|
provenance. Only JSON-serialisable values are kept; others are dropped with a note. |
required |
topica.effective_determinism ¶
The determinism class this instance actually has, given its configuration
(issue #401), refining the per-class :attr:ModelInfo.determinism tag.
Determinism is often per-configuration, not per-class: the same model can be
bit-exact under one setting and seed-reproducible under another. This
reads the model's :attr:~topica settings (constructor config, issue #400)
and the fit_settings you record, and returns::
{"effective": "seed-reproducible", # the config-aware class
"registry_class": "bit-exact", # the coarse per-class tag
"replay_requires": {"seed": 42}, # machine-readable replay conditions
"notes": ["..."]} # human-readable caveats
replay_requires carries the conditions an exact replay needs — always the
seed for a seed-reproducible fit, plus num_threads for the collapsed-Gibbs
approximate parallel sampler. bit-exact fits carry no replay requirements.
Scope (minimal, honest): claims the configuration determines are made exactly.
Where the deciding factor is a runtime outcome the config cannot know — did a
spectral initialization succeed or silently fall back to a seeded random one? did
anchor selection hit its degenerate-basis fallback? — the report reads the route
the fit actually took, recorded on the fitted model (model.initialization for
STM/CTM/STS/DTM, model.anchor_fallback_used for AnchorLDA, issue #410), and
makes the exact call. For an unfitted model, or one saved before that route was
recorded, it falls back to the common-case class plus a caveat in notes rather
than over- or under-claiming.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
a topica model (fitted or not; reads its construction config).
|
|
required |
fit_settings
|
the keyword arguments passed to ``fit`` (e.g. ``num_threads``,
|
|
None
|
topica.manifest.AnalysisManifest ¶
A versioned, privacy-aware record of one fit. See the module docstring.
__annotations__
class-attribute
¶
__annotations__ = {'topica_version': 'str', 'environment': 'dict[str, Any]', 'model': 'dict[str, Any]', 'corpus': 'dict[str, Any]', 'inputs': 'dict[str, Any]', 'diagnostics': 'list[dict[str, Any]]', 'decisions': 'list[dict[str, Any]]', 'schema_version': 'int', 'created': 'str | None'}
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__dataclass_fields__
class-attribute
¶
__dataclass_fields__ = {'topica_version': Field(name='topica_version',type='str',default=<dataclasses._MISSING_TYPE object at 0x7f4a64521850>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f4a64521850>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'environment': Field(name='environment',type='dict[str, Any]',default=<dataclasses._MISSING_TYPE object at 0x7f4a64521850>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f4a64521850>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'model': Field(name='model',type='dict[str, Any]',default=<dataclasses._MISSING_TYPE object at 0x7f4a64521850>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f4a64521850>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'corpus': Field(name='corpus',type='dict[str, Any]',default=<dataclasses._MISSING_TYPE object at 0x7f4a64521850>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f4a64521850>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'inputs': Field(name='inputs',type='dict[str, Any]',default=<dataclasses._MISSING_TYPE object at 0x7f4a64521850>,default_factory=<class 'dict'>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'diagnostics': Field(name='diagnostics',type='list[dict[str, Any]]',default=<dataclasses._MISSING_TYPE object at 0x7f4a64521850>,default_factory=<class 'list'>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'decisions': Field(name='decisions',type='list[dict[str, Any]]',default=<dataclasses._MISSING_TYPE object at 0x7f4a64521850>,default_factory=<class 'list'>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'schema_version': Field(name='schema_version',type='int',default=1,default_factory=<dataclasses._MISSING_TYPE object at 0x7f4a64521850>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'created': Field(name='created',type='str | None',default=None,default_factory=<dataclasses._MISSING_TYPE object at 0x7f4a64521850>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD)}
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__doc__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__match_args__
class-attribute
¶
__match_args__ = ('topica_version', 'environment', 'model', 'corpus', 'inputs', 'diagnostics', 'decisions', 'schema_version', 'created')
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
schema_version
class-attribute
¶
int([x]) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by '+' or '-' and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal.
int('0b100', base=0) 4
add_decision ¶
add_decision(key: str, note: str, *, author: str | None = None, time: str | None = None) -> AnalysisManifest
Record a researcher-authored decision (K choice, labels, …).
Always stored as visibly researcher-authored, never as a substantive conclusion the tool is vouching for.
add_diagnostic ¶
Record a diagnostic as computed evidence (not a conclusion).
verify ¶
Re-derive fingerprints from the supplied corpus/model and report, per field, whether the fit's identity and replay conditions still hold.
Never collapses the fields into one boolean. Uses the recorded determinism class to decide whether an environment difference is potentially replayable rather than a mismatch.
compare ¶
Compare this manifest with other field by field.
Uses only what each manifest recorded (fingerprints, counts, settings), so it needs neither corpus nor model. Answers "did these two runs use the same corpus / model / inputs, or did something change, and what?".
render ¶
render(path: str | None = None, *, title: str | None = None, verification: VerifyResult | None = None) -> str
Render the manifest as a self-contained HTML analysis card.
Returns the HTML string, and writes it to path if given. The card
shows only what the manifest actually recorded, so it cannot over-claim:
researcher decisions are labelled as authored (not tool-verified),
diagnostics as computed evidence, and fingerprints as verifiable identity
rather than content. Pass a :class:VerifyResult (from :meth:verify) to
include the graded verification table.
to_markdown ¶
Render the manifest as a Markdown analysis card (Quarto/notebook).
bundle ¶
bundle(path: str, *, model=None, corpus=None, include_model: bool = True, include_corpus: bool = False) -> str
Write a self-contained, content-addressed bundle (a .zip).
The bundle packages this manifest with the saved artifacts as one shareable, self-verifying unit::
manifest.json this manifest, plus artifact references
artifacts/<digest>.tt model.save() / corpus.save(), named by hash
Each artifact's file name is its BLAKE2b digest, and the manifest's
model / corpus blocks gain an artifact reference, so
:meth:load_bundle can detect a corrupt bundle or one whose artifacts no
longer match the manifest. This is integrity / content-addressing, not
authenticity: it does not detect a fully rewritten bundle (artifact +
digest + reference all changed together) — that needs a signature (#404).
Before writing, the supplied model / corpus are checked against
this manifest's recorded fingerprints; a mismatch (the wrong fit) raises,
so a bundle's artifacts always match the analysis it documents.
include_corpus is opt-in and sensitive: a bundled corpus embeds
the raw tokens (a hash is not anonymisation). Pass the fitted model /
corpus objects to bundle them (the manifest itself holds only
fingerprints, not the objects). The file digest here proves "this exact
saved file"; it is distinct from the corpus content fingerprint, which
proves corpus identity across save-format versions.
load_bundle
classmethod
¶
Load a bundle written by :meth:bundle, checking artifact integrity.
Recomputes each bundled artifact's digest and checks it against both the
content-addressed file name and the manifest reference; a corrupt bundle
(or one whose artifacts no longer match the manifest) raises ValueError.
This is an integrity check, not authenticity -- see :meth:bundle. Returns
the manifest; use :meth:extract_bundle to recover the artifacts.
extract_bundle
staticmethod
¶
Extract a bundle's artifacts to dest_dir (checking integrity).
Returns {"model": <path or None>, "corpus": <path or None>}; reload
each with the matching class, e.g. topica.LDA.load(paths["model"]).
Validates the bundle version and artifact digests, like :meth:load_bundle.
topica.manifest.VerifyResult ¶
The outcome of :meth:AnalysisManifest.verify, per field.
fields maps each checked field to one of: exact (recomputed value
matches), input_changed, artifact_changed, environment_changed
(differs but the model's determinism class says it may still replay),
unverifiable (nothing to check against, or a bounded/unknown component).
ok is a convenience for "every field exact"; the summary always shows the
full breakdown -- the statuses are never collapsed into one flag.
__annotations__
class-attribute
¶
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__dataclass_fields__
class-attribute
¶
__dataclass_fields__ = {'fields': Field(name='fields',type='dict[str, str]',default=<dataclasses._MISSING_TYPE object at 0x7f4a64521850>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f4a64521850>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD)}
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__doc__
class-attribute
¶
__doc__ = 'The outcome of :meth:`AnalysisManifest.verify`, per field.\n\n ``fields`` maps each checked field to one of: ``exact`` (recomputed value\n matches), ``input_changed``, ``artifact_changed``, ``environment_changed``\n (differs but the model\'s determinism class says it may still replay),\n ``unverifiable`` (nothing to check against, or a bounded/unknown component).\n ``ok`` is a convenience for "every field exact"; the summary always shows the\n full breakdown -- the statuses are never collapsed into one flag.\n '
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__match_args__
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
topica.manifest.ManifestDiff ¶
The outcome of :meth:AnalysisManifest.compare, per field.
Compares two manifests directly (no corpus or model needed), so it answers
"did these two runs differ, and where?". fields maps each field to
same, changed, only_in_a / only_in_b (recorded in one but not
the other), or incomparable (e.g. fingerprints from different specs, or a
keyed digest). same is a convenience for "every field same"; the summary
always shows the breakdown.
__annotations__
class-attribute
¶
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__dataclass_fields__
class-attribute
¶
__dataclass_fields__ = {'fields': Field(name='fields',type='dict[str, str]',default=<dataclasses._MISSING_TYPE object at 0x7f4a64521850>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f4a64521850>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD)}
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__doc__
class-attribute
¶
__doc__ = 'The outcome of :meth:`AnalysisManifest.compare`, per field.\n\n Compares two manifests directly (no corpus or model needed), so it answers\n "did these two runs differ, and where?". ``fields`` maps each field to\n ``same``, ``changed``, ``only_in_a`` / ``only_in_b`` (recorded in one but not\n the other), or ``incomparable`` (e.g. fingerprints from different specs, or a\n keyed digest). ``same`` is a convenience for "every field same"; the summary\n always shows the breakdown.\n '
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__match_args__
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
topica.manifest.fingerprint_corpus ¶
Sensitive, order-sensitive fingerprint of a corpus's token content.
Hashes token strings in document order, so it survives a vocabulary re-indexing but changes if any document's tokens or the document order change. A hash is not anonymisation; treat this as sensitive.
topica.manifest.fingerprint_array ¶
Order-sensitive fingerprint of a numeric array (embeddings, vectors).
topica.manifest.fingerprint_design ¶
Fingerprint of a numeric design matrix, binding column names to values.