Task: DQ-publish the calendar reference data
Table of Contents
This page documents a task in the Finish integrating calendar reference data: XML export and DQ publish story. It captures the goal, current status, acceptance, and any notes or results.
Goal
Wire the remaining calendar-adjacent reference-data tables —
currency_country, currency_calendar, and
currency_pair_convention_calendar — into the DQ bundle publication
pipeline (see DQ Bundle
Publication Pipeline), so this data reaches target services as
dataset bundle publications, the same way currency_pairs is
published today. calendar and calendar_type already have this
support, delivered early as part of
Model the calendar entity
for codegen (refdata.v1.calendar-types.publish-from-dq,
refdata.v1.calendars.publish-from-dq) — leaving both junction tables
without a DQ publish path was flagged during
the currency-calendar FK
task's review as a gap that shouldn't be left unaddressed: every
other reference-data table in this story is DQ-publishable, and the
junctions inheriting "no DQ publish" from each other rather than from
the fully-supported calendar pattern is not something to let stand.
Since the publish_from_dq codegen facet described in
the open capture has not
been built yet, this follows the existing hand-written pattern:
artefact table(s), publish-from-dq SQL function(s), and NATS handler
wiring, per entity.
Status
| Field | Value |
|---|---|
| State | DONE |
| Parent story | Finish integrating calendar reference data: XML export and DQ publish |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-07-23 |
Acceptance
[X]Artefact tables (and matching drop scripts) exist forcurrency_countryandcurrency_calendar, mirroringdq_calendars_artefact_tbl's shape.[X]publish-from-dqSQL functions exist for both junctions inrefdata_publish_from_dq_create.sql, supporting the same upsert/insert_only/replace_all mode contract as every other publish-from-dq function.[X]Both are registered as NATS subjects (refdata.v1.currency-countries.publish-from-dq,refdata.v1.currency-calendars.publish-from-dq) inregistrar.cpp.[X]Both artefact types are registered indq_artefact_types_populate.sql.[X]currency_pair_convention_calendaris evaluated separately: it does not yet exist as a table (calendars are still attached tocurrency_pair_convention.advance_calendaras free text, per the pending advance_calendar migration task) — DQ publish for it is out of scope here and belongs with that migration once the column becomes a real relationship. Turned out moot: see Notes — it was already fully wired as part of task 1's delivery.[X]Published bundles are visible and consumable via the Librarian UI. Verified via Verify currency_countries DQ publish: PASSED.
Plan
(Implementation strategy. Written when work starts; key decisions
are distilled into the parent story's * Decisions at close, but the
plan itself stays — it is the historical record of what we did.)
Checked what already existed before writing anything: currency_calendar
and, surprisingly, currency_pair_convention_calendar both already had
a full DQ-publish path (artefact table, drop script, publish-from-dq SQL
function, NATS subject registration, artefact-type row) — delivered
alongside the calendar=/=calendar_type pipeline in
0a167d548 [sql/dq] Add DQ publish pipeline for calendars and calendar
types, before this task's acceptance criteria were written. Only
currency_country had a real gap. Followed the existing hand-written
pattern exactly (currency_calendar's artefact table/function as the
template, since it is the closest shape: a simple two-column junction):
dq_currency_countries_artefact_create.sql/dq_currency_countries_artefact_drop.sql, wired intodq_create.sql=/=dq_drop.sql.ores_refdata_publish_currency_countries_from_dq_fninrefdata_publish_from_dq_create.sql.refdata.v1.currency-countries.publish-from-dqadded to the subject list inregistrar.cpp(genericpublish_from_dq_handlerderives the SQL function name from the subject, so no C++ dispatch logic needed).currency_countriesartefact-type row indq_artefact_types_populate.sql(display_order 62).
Registering the artefact type alone isn't enough for the Librarian to
actually offer it — a bundle only publishes datasets it's a member of,
and a dataset needs its own ores_dq_datasets_tbl row plus real rows
in the shadow artefact table. So, mirroring
refdata_currency_calendars_seed_populate.sql:
refdata_currency_countries_seed_populate.sql: registers therefdata.currency_countriesdataset and copiesores_refdata_currency_countries_tbl(already seeded by the foundation layer) intoores_dq_currency_countries_artefact_tblverbatim — no derivation needed here, unlikecurrency_calendars, since currency_countries is the source of truth, not something derived from it.refdata.currency_countriesadded to thebasebundle's members indq_dataset_bundle_member_populate.sqlatdisplay_order 18(aftercurrency_pair_convention_calendarsat 17, to avoid renumbering an already-shipped entry).
compass db recreate confirmed the new SQL applies cleanly end to
end: 39 currency-country rows land in the DQ artefact table, and
calling ores_refdata_publish_currency_countries_from_dq_fn directly
against the real seeded dataset successfully republishes all 39 rows
(same live-table upsert path the Librarian's workflow step would
invoke). Calling it with a bogus dataset id raises the same "Dataset
not found" error as every sibling function. Full rat (build + test)
passed clean on both the pre- and post-seed-script db recreate.
Notes
currency_calendar's and currency_pair_convention_calendar's DQ
publish infrastructure was already complete when this task started —
this task's own acceptance criteria (written 2026-07-13/21) were
already stale by the time work began. Left a note here rather than
silently rewriting history, since the "not yet a table" claim about
currency_pair_convention_calendar was true when written but the
table, and its DQ publish path, landed in the interim (task 1's early
delivery + this story's junction-persistence follow-up, PR #1679).
Test Scenarios
Manual QA scenarios (scaffolded via compass add test_scenario, run
through the QA Validation Runner panel) that verify this task. Link
new ones here as they're created; the scenario doc itself links back
via its "Verifies task" field.
| Scenario | State | Notes |
|---|---|---|
| Verify currency_countries DQ publish | PASSED | Librarian bundle publish + Publication Audit dialog; cross-checked against the database (39 rows, no dupes, EUR maps to all 6 Eurozone countries). |
PRs
| PR | Title |
|---|---|
| #1681 | [refdata,sql] DQ-publish currency_country into the base bundle |
Review
| # | Comment summary | File | Decision | Notes |
|---|---|---|---|---|
| 1 | display_order (62) placed before currency_pair_convention_calendars (61) in file order | dq_artefact_types_populate.sql | Accepted | Reordered the INSERT rows to match numeric display_order |
| 2 | Missing blank line before "* Out of scope" heading | story.org | Accepted | Added blank line for consistency with other heading transitions |
Result
currency_country now has a full DQ-publish path: artefact table
(ores_dq_currency_countries_artefact_tbl), drop script,
ores_refdata_publish_currency_countries_from_dq_fn, a
refdata.v1.currency-countries.publish-from-dq NATS subject, a
currency_countries artefact-type row, a dataset-registration +
seed-populate script mirroring the live table, and base bundle
membership at display_order 18. currency_calendar and
currency_pair_convention_calendar turned out to already be fully
wired (see Notes) — only currency_country had a real gap.
Verified end to end: full rat build+test clean; a real
barclays_system_provision.ores run publishing the base bundle
against the Barclays Plc tenant completed
refdata.currency_countries without error; the Publication Audit
dialog and a direct database check both confirm 39 currency-country
rows landed with zero duplicates and the correct EUR/Eurozone
mapping. Manual QA scenario Verify currency_countries DQ publish:
PASSED.
Filed a follow-up capture,
Add a currency-country assignment widget to CurrencyDetailDialog,
for the UI gap discovered while writing the scenario: currency_calendar
has a CalendarAssignmentWidget in the currency detail dialog;
currency_country has no UI anywhere. Out of scope for this task.