The data classification checklist

1. Summary
2. Detail
3. See also

1. Summary

This page is the form a modeller fills in before writing a model, a message or a table. It is short on purpose. The classification is the decision; the pattern, the table and the wire follow from it.

A modeller classifies first, then uses the sanctioned pattern for the classification. A pattern never chooses the classification. This order is the whole point of the page: almost every bad shape in this codebase was a pattern applied to data whose classification was never asked for.

A set that fits no classification is a finding to report. It is not a shape to invent.

2. Detail

2.1. How to use it

Fill in the checklist before writing the model. Step 2 sets the classification; step 7 then reads the sanctioned pattern from Data classifications. The classification selects the pattern, so a step 7 answer that contradicts a step 2 answer means step 2 was wrong.

Fill it in before a shape is committed to a file. The checklist is also the review artefact: a reviewer checks the classification and the invariant, and does not audit the field count.

2.2. The checklist

Name the data and its operation. What is the set, which caller receives it, and what operation is it? One sentence. If the set has no named caller, it has no classification yet.
Classify the data. Pick one: Entity, List, Family, Bundle, Package, Artefact. One classification per message. If two answers both look right, write two messages rather than one shape that does both. The definitions are in Data classifications.
Name the invariant, or record that there is none. Write one sentence of the form "X must hold at instant T". If the sentence needs a file, a screen, a workflow step or a boundary format to be true, there is no invariant: write "none" and say why.
Count the owners. How many services write rows in this set? Name the owner of each kind. The answer is "one" or "several", and it is never "the coordinator".
Answer bundle or package? This is the relatedness question, and it is the discriminator between the two. Do the families share either (a) a data invariant that must hold at one instant, or (b) a single operation whose intent is all-or-nothing?
- Related, and one owner holds every family. A Bundle, applied in one unit_of_work. It is the largest thing that can be atomic.
- Related, but several owners must write. No bundle spans them. A Package carrying one bundle per owner, and the package is never atomic.
- Not related. The set is gathered by provenance — one file, one export, one report run — and provenance is not relatedness. If the bytes must move as one artefact, it is a Package; each family in it is its own bundle. If the bytes need not move, the set is Entity, List or Family per member, and the gathering is a read.
- A Package has no relatedness constraint. State that its bundles need not be related, may belong to different owners, and are never atomic as a group.
Choose the classification from the rule. No relatedness: Entity, List, or split the set. Related and one owner: Bundle. Related and several owners, or gathered by provenance and moved as bytes: Package. A file at the boundary: Artefact, and classify separately what it converts into. A file's contents are never an invariant.
Name the sanctioned pattern and the generated artefact that carries it. Entity: the entity or composite pattern, and its generated type and verbs. List: the list verbs and the paged reply. Family: ores.codegen.family, <family>_item and <family>_write. Bundle: the single unit_of_work over its families, carried in a package's manifest when it crosses a boundary. Package: the manifest and the per-bundle message. Artefact: the mapping layer and the object-storage key. If no pattern carries it, stop and report the gap. Do not compose a flag combination that no facet consumes.
State the atomicity scope. The row; per element; one owner and one transaction; one owner and one unit_of_work for the bundle; or none for the package as a whole. Name what commits in which transaction, and what a failure halfway leaves behind.
State the consistency point if the set claims to be a view of the world. Give the as_of instant or snapshot the set was taken at, and the rule that rejects a stale one. A set that claims to be one view of the world without one instant is a set of rows that may disagree.
State what is forbidden here. Name the anti-pattern the classification replaces, plus any local rule a reviewer must enforce. An Entity forbids the nested aggregate. A List forbids a cross-element invariant. A Family forbids the hand-written restatement and the field group that fakes it. A Bundle forbids the archive field and the union of typed arrays. A Package forbids all-or-nothing across its bundles or owners, and forbids being treated as a data model. An Artefact forbids the document tree as the model.
Record the finding if no classification fits. Name the set, the invariant you could not find, the owners you could not line up, and the classification you believe is missing. Stop there. Do not invent a shape; an invented classification is a second type system, and the generator cannot check it.

2.3. The filled-in form

Copy this table into the task and fill every row. An empty row is a step that was skipped.

Field	Answer
Data and operation
Classification	Entity / List / Family / Bundle / Package / Artefact
Invariant, or none
Owners
Bundle or package, and why
Sanctioned pattern
Generated artefact
Atomicity scope
Consistency point
Forbidden here

2.4. Worked examples

2.4.1. A list of currencies

Classification: List.
Invariant: none. The caller wants every active currency, and no currency constrains another.
Owners: one, refdata.
Pattern: the list verbs, paged, with the total.
Atomicity: per element.
Forbidden: a wrapper aggregate so the list can be saved whole; a client-side join.

2.4.2. A tenor basis swap convention

Classification: Family.
Invariant: the central convention row and its parts must describe the same convention at one instant, and a convention fills exactly one of the two forms.
Owners: one, refdata.
Pattern: ores.codegen.family, tenor_basis_swap_convention_item and tenor_basis_swap_convention_write, and the four derived verbs.
Atomicity: one owner, one transaction over the central row and the part that the write names.
Forbidden: a hand-written family struct; a field group that nests the parts.

2.4.3. A structure with its trades and legs

Classification: Bundle.
Invariant: the trades and legs must describe the same deal at one instant, and one operation writes the set or none of it.
Bundle or package: a bundle. The families are related, and one owner, trading, holds every one of them.
Owners: one, trading.
Pattern: ores.codegen.family for each family, and one unit_of_work over the families the operation names.
Atomicity: one owner, one transaction. A failure halfway writes nothing.
Forbidden: a struct with one typed array per kind, built because a screen lists the kinds together.

2.4.4. An environment export

Classification: Package, carrying one bundle per owner.
Bundle or package: a package. Its bundles are not related to one another; they are together because one export produced them. No transaction joins them, and the package is never atomic as a whole.
Invariant: none across the bundles. Within each bundle, the bundle's own invariant holds at one instant.
Owners: several. Each component writes its own bundle.
Pattern: a content-addressed artefact with a manifest that names the bundles and their parts, and one bundle-reference message per owner.
Atomicity: per bundle, which is per owner. Global atomicity is not available.
Consistency point: the as_of instant in the manifest.
Forbidden: a struct with one typed array per kind; a tar or zip field in a message; a promise of all-or-nothing across the bundles.
Transport: Transporting a package.

2.4.5. An ORE conventions file

Classification: the file is an Artefact. What it converts into is one Family per convention kind, with its Lists.
Invariant: none across the families. A zero convention and a CDS convention never constrain each other; the file lists them because the format lists them.
Owners: one, refdata, for every family.
Pattern: a mapping layer from the XML to per-family family messages. The file, if kept, is an object-storage artefact with a key.
Atomicity: per family write. A conversion that must be all-or-nothing is one bundle, because the families are related by that one operation and refdata owns them all; the file that carried it is a package.
Forbidden: one struct with a typed array per kind. The format's element list is not a type. The test that refuses it is in Data classifications, section "The anti-type".

2.5. Signs the checklist was skipped

A struct named after a file, a screen or a workflow step. The name records where the list of kinds came from, not what constrains them.
One struct with several arrays of unrelated concrete types.
A member list that grows whenever the format gains an element.
"The document must be stored in one transaction", said without naming an owner or an instant.
A compile error used to justify a shape. A compile error is a technical problem; see Data classifications, section "Compilation errors do not reclassify data".

3. See also

Data classification — the structure note that orders this cluster: this page is the second step in its reading order, and it applies the definitions the first step states.
Data classifications: entity, list, family, bundle, package and artefact — the definitions, the sanctions, the relatedness test, the anti-type and the decision rule this checklist applies.
Transporting a package — the transport a Package or Bundle classification selects.
Codegen patterns: choosing a model shape — the pattern library step 7 selects from.
Data-Oriented Design in ORE Studio — the regimes and criteria behind the rule that classification precedes pattern choice.
NATS entity protocol specification — the verbs and envelopes a classification ultimately maps onto.
Knowledge — the index this page belongs to.