publish_from_dq codegen facet: hand-writing artefact tables is a recurring cost
Table of Contents
This page is a capture in the next bucket of the product backlog — a pre-sprint idea, not yet pulled into a sprint as a story.
What
Implement the publish_from_dq codegen facet (designed but never
built — see References) so a DQ-published dataset's artefact table,
publish-from-dq SQL function, NATS handler, and registrar wiring are
generated from the owning entity's existing codegen model, instead of
hand-written per dataset.
Why
Discovered while raising a task to publish a curated FX driver-rate
dataset via DQ/librarian, the same way refdata.currency_pairs is
published: there is no codegen model anywhere in the tree for a DQ
artefact table (confirmed by grep across every modeling/ directory)
— dq_currency_pairs_artefact_tbl, the synthetic FX spot configs
artefact, and now this new dataset's artefact table all had to be
hand-written from scratch, each shape near-identical to the last
(dataset_id/tenant_id/version + domain columns, a publish-from-dq
SECURITY DEFINER function reading the artefact and upserting into
the real domain table). The facet's own design task called for
proving it against report_definitions with zero behavioural diff —
that reference target and design still stand; there are now three
real hand-written instances to generalise from instead of one.
References
- Design the publish_from_dq codegen facet — ABANDONED.
- Implement the publish_from_dq codegen facet — ABANDONED.
- Publish a curated FX driver-rate dataset for 2016-02-05 — the task that hit this gap a third time.
See also
- Synthetic data librarian support: FX foundation — the story both abandoned facet tasks belonged to.