Task: Fix the artefact index emission and the tags ON CONFLICT
Table of Contents
This page documents a task in the Clean ores.dq to the component clean standard story. It captures the goal, current status, acceptance, and any notes or results.
1. Goal
(Describe what user-visible-or-internal change this task produces.)
2. Status
| Field | Value |
|---|---|
| State | DONE |
| Parent story | Clean ores.dq to the component clean standard |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-09-27 |
3. Acceptance
- A staging table emits its key index once, however many artefact columns it declares.
ores_dq_tags_artefact_tblcarries a unique index on(tenant_id, dataset_id, name), so the seed'son conflictclause has an arbiter.- A declared artefact index is unique only when it says
:unique: true. - A check fails on an unterminated block comment and on a repeated index in one file, and it runs on the pull request as well as in Continuous Linux.
4. Plan
- Confirm the two defects. The five population-function comments were already fixed by PR #2227; a PostgreSQL-aware scan proves the current tree is balanced. The tags
on conflictand the repeated indexes are live. - Fix the index emission at source. Move the artefact key index out of the artefact-columns loop in
ores.sql.schema.domain_entity_artefact_create, and teach the* Artefact indexessection a:unique:flag. - Declare the arbiter. Add the unique
tag_identityartefact index toores.assets.tag, which owns the staging table the seed writes. - Regenerate and narrow the diff. Regenerate the SQL facet tree-wide, then keep only the ten
dq_*_artefact_create.sqlfiles this template can write and revert the drift the run materialized elsewhere. - Pin both behaviours in tests. The key index count is asserted exactly, which the old presence assertion did not do, and the unique flag is asserted both ways.
- Encode the classes as a check.
check_sql_hygiene.pyreads the tree for unterminated block comments and repeated indexes;validate_schemas.shand the codegen-drift job both call it.
5. Notes
6. Test Scenarios
Manual QA scenarios (scaffolded via compass add test_scenario)
that verify this task. Link
new ones here as they're created; the scenario doc itself links back
via its "Verifies task" field.
| Scenario | State | Notes |
|---|---|---|
7. PRs
8. Review
| Comment summary | File | Decision | Notes |
|---|---|---|---|
9. Result
Both defects are fixed at source and both classes are gated.
The index emission. The key index was inside the artefact_columns loop, so a table rendered one copy per artefact column: 245 lines of duplicate index across ten staging tables, worst dq_currencies_artefact_iso_code_idx at seventeen copies. It is now emitted once, under artefact_key_column.
The missing arbiter. ores_dq_tags_artefact_tbl had no unique index at all, so the seed's on conflict (tenant_id, dataset_id, name) had nothing to infer and would have failed at run time. The * Artefact indexes section gained a :unique: flag and the tag model declares the index.
The five comments were already fixed. PR #2227 closed them. The scan written here confirms the tree is balanced and would have caught the original five, which is the evidence that the fix holds.
Two findings recorded, not fixed here. Regenerating the SQL facet tree-wide materialized generated SQL that main has never committed – marketdata, refdata, synthetic and a whole workspace table with its trigger and drops. They are reverted here and belong to their own tasks. And regenerate_physical_space_inventories.py --check reports three stale facet inventories (ores.org, ores.cpp.org, ores.cpp.http-route.org); it reproduces on a stashed clean tree, the staleness is a row reordering, and the gate is in no workflow, so it is recorded rather than swept into this change.