Story: Bring every DQ artefact table onto the canonical model
Table of Contents
This page documents a story in Sprint 27. It captures the goal, current status, acceptance criteria, and the tasks that compose it.
1. Goal
Move DQ artefact tables onto the canonical model: the duplicated ores.dq models and the hand-written artefact tables are generated from the entities they stage, and a guide states the one way to add a DQ artefact.
2. Status
| Field | Value |
|---|---|
| State | DONE |
| Parent sprint | Sprint 27 |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-10-13 |
3. Acceptance
- A knowledge page states the one way to add a DQ artefact, the full pipeline from model to published row, a checklist and the legacy patterns named as patterns not to copy.
- No ores.dq model duplicates a domain model; the five duplicated models and their generated DQ C++, shell and TypeScript stacks are deleted.
- Five hand-written artefact tables are generated from their owning models with an Artefact columns section, and the artefact template renders check constraints.
- The datasets publish as before: pgTAP, ctest and an Acme provision pass.
4. Tasks
| Task | State | Start | End | Description |
|---|---|---|---|---|
| Write the DQ artefact guide | DONE | 2026-10-12 | 2026-10-12 | A knowledge page that states the one way to add a DQ artefact: the owning model's domain_entity_artefact_create flag, an Artefact columns section where the staging shape differs, and the pipeline from model to published row (artefact table, dataset, artefact type, bundle member, publish function, subject, handler allow-list, dataset dependencies), with a checklist and the legacy patterns named as patterns not to copy. |
| Move the duplicated DQ models onto their owners | DONE | 2026-10-12 | 2026-10-13 | Six artefact tables come from a separate ores.dq model that duplicates a domain model elsewhere: csas, netting_agreements and netting_sets (refdata), report_definitions and risk_report_configs (reporting), and synthetic_market_data_providers (synthetic). Set the flag on each owning model, delete the ores.dq model and its generated DQ C++, shell and TypeScript stack, and follow the table rename through the populates, publish functions and artefact types. |
| Model the hand-written artefact tables | DONE | 2026-10-13 | 2026-10-13 | Five artefact tables are hand-written SQL with no model: crm_topology_bundles, market_data_observations, synthetic_gmm_components, synthetic_ir_curve_configs and synthetic_ir_curve_template_entries. Generate each from its owning model, with an Artefact columns section where the staging shape differs from the entity. Two more were in scope and moved: lei_bic to the DQ-only task, and ip2country to its own task, because its geo store has no model. |
5. Decisions
The remainder is carried in Move the lookup artefact tables and settle the remaining DQ artefact models.
- A moved table keeps its shape. The owning model states the old staging
columns in an
* Artefact columnssection, so a move changes no populate or publish function. A table takes the name its owning model gives it, as the provider table did. - The staging shim gate stays. No dq shim copies another component's entity now, but codegen still drops the shell unit of one that would, and a fixture tests it.
- A staging table states its own checks. The
* Artefact columnssection takes an expression table, and the artefact template renders it, so a hand-written table with constraints moves without losing them. - A table whose owner has no model waits for the model.
ip2countrymoves once its geo store is modelled, andlei_bicmoves with the other GLEIF tables.