Story: Commission: country

Table of Contents

This page documents a story in Sprint 21. It captures the goal, current status, acceptance criteria, and the tasks that compose it.

Goal

Commission country across all access layers: verify the Qt UI end-to-end post-NATS migration, verify existing shell and CLI commands, fix any regressions found, and add the entity chapter to the user manual. File backlog captures for Wt and HTTP support.

Status

Field Value
State STARTED
Parent sprint Sprint 21
Now SQL, CLI, and shell verification DONE. SQL codegen back-out DONE. C++ core, messaging, and Qt codegen tasks added (BACKLOG).
Waiting on Nothing.
Next C++ core codegen (DFD25563) — create domain entity org model and iterate to zero diff.
Last touched 2026-06-25

Acceptance

  • Qt list window (MDI): loads and displays records correctly.
  • Qt detail window: edit and save round-trips without error.
  • Qt history window: shows change history for the entity.
  • Qt delete command: removes the record; history is preserved.
  • Qt eventing: a change made in one connected session appears in a second session without manual refresh.
  • Shell list command: verified working post-NATS.
  • Shell add command: verified working post-NATS.
  • Shell remove command: verified working post-NATS.
  • CLI list command: verified working post-NATS.
  • CLI add command: verified working post-NATS.
  • CLI remove command: verified working post-NATS.
  • Manual: entity chapter documents all Qt windows, shell, and CLI commands.
  • Backlog captures filed for Wt and HTTP support.
  • All regressions found are fixed inline or filed as captures.
  • Site builds cleanly.

Analysis

Analysis commissioned during the Verify country SQL against the evaluation checklist task, prompted by two items deferred from the PR #1300 review. Goal: determine whether each item is a deliberate design choice or an historical artefact requiring correction, and fix it at the template level so future entities are correct on first generation.

Item 1: security definer absent from validate functions

Observation

All 50 ores_refdata_validate_*_fn functions in the codebase lacked security definer and set search_path = public, pg_temp. Of the 115 refdata create files at the time of analysis, only 5 carried security definer at all. Country's insert trigger had it; currency's and most others did not. The IAM layer (ores_iam_validate_*_fn, ores_iam_*_insert_fn) applied it consistently from the start.

PostgreSQL security model

A BEFORE INSERT trigger function runs under the security context of the calling role (the service role that issued the INSERT), not the function owner — unless the function declares security definer. Without it, a service role with a misconfigured or compromised search_path that includes a schema it controls can shadow any ores_*_tbl reference inside the trigger, causing validation to query attacker-controlled data and silently accept any input.

security definer and set search_path are required together. security definer alone pins the executing user but leaves the search path open; if search_path is not pinned, the function owner's own schemas are searched and an attacker with write access to one of them can still shadow tables. Both are needed.

Crucially, security definer does not propagate from a caller to its callees. When an insert function (security definer) calls a validate function (security invoker), the validate function inherits the effective current user at call time — which happens to be the function owner because the caller is definer-scoped. This gives indirect protection only while called from a correctly annotated insert function. If the insert function lacks security definer (as most existing ones did), the validate function is fully exposed. Validate functions are also candidates for future direct application calls (pre-validation before insert); they must be independently safe.

Verdict

Historical artefact. The IAM layer established the pattern; the refdata layer never adopted it. The discrepancy is a latent security gap — not exploitable under the current role model without prior deep compromise, but semantically incorrect and a risk as the codebase grows.

Correct approach

Apply security definer set search_path = public, pg_temp to:

  • All insert trigger functions (primary protection layer)
  • All validate functions (defence in depth; enables future direct-call use)

Fixed in refdata_countries_create.sql (the commissioning reference) and in the sql_schema_create literate template sql_schema_create.mustache. Documented as a mandatory rule in PostgreSQL: Database Architecture and Conventions (§ Insert trigger patterns → Validation functions).

Entities generated before this fix retain the old form until recommissioned or regenerated; that sweep occurs naturally through the commissioning pipeline, not as a separate remediation story.

Item 2: Bootstrap pass-through checks any rows, not active rows

Observation

All validate functions contained a bootstrap pass-through of this form:

if not exists (
    select 1 from ores_refdata_countries_tbl
    where tenant_id in (p_tenant_id, ores_utility_system_tenant_id_fn())
    limit 1  -- no valid_to filter
) then
    return p_value;
end if;

The intent is: "skip validation if this tenant has never been seeded." The implementation checks for any row, including soft-deleted historical rows.

The bug

If a tenant has had data seeded and all rows have since been soft-deleted (valid_to = current_timestamp), the table contains historical rows but no active ones. The bootstrap check finds those rows (not exists → false), skips the pass-through, then fails the active-row validation:

ERROR: Invalid country: GB. Must be one of: (empty list)

This is both confusing and semantically wrong: the caller cannot distinguish "the value is not in the catalogue" from "there is no usable catalogue at all."

The limit 1 inside EXISTS is a no-op — EXISTS short-circuits on the first matching row regardless — and is omitted alongside the fix.

Practical impact

The scenario requires all active rows for a tenant to be soft-deleted — unusual but not impossible. The more serious concern is that this code is the template for all generated validate functions. Left uncorrected, every future entity generated by codegen would inherit the bug.

For the active-row convention see Time and Timestamps: Architecture and Conventions.

Verdict

Semantic bug, not a deliberate design choice. Present in all 50 validate functions at the time of analysis.

Correct approach

Filter the bootstrap check for active rows:

if not exists (
    select 1 from ores_refdata_countries_tbl
    where tenant_id in (p_tenant_id, ores_utility_system_tenant_id_fn())
      and valid_to = ores_utility_infinity_timestamp_fn()
) then
    return p_value;
end if;

Fixed in refdata_countries_create.sql (the commissioning reference) and in the sql_schema_create.mustache template (all three scope variants: scope_system, scope_both, scope_tenant). Documented in PostgreSQL: Database Architecture and Conventions (§ Insert trigger patterns → Validation functions).

Disposition

Both items are historical artefacts now corrected at the template level. The country commission story (sprint 21) serves as the canonical reference state: entities brought under codegen from this point forward will have both fixes applied automatically. Existing generated entities (currencies, rounding_types, monetary_natures, etc.) retain the old patterns until recommissioned; no separate remediation story is needed.

Tasks

Task State Start End Description
Verify country SQL against the evaluation checklist DONE 2026-06-25 2026-06-25 Thorough DDL verification beyond the appraisal's spot-grep: every checklist criterion against refdata_countries_create.sql and the notify trigger — constraints, indexes, trigger validation functions, soft-delete rule, soft-FK validations — mirroring currency's verify-SQL task.
Verify country CLI commands post-NATS DONE 2026-06-25 2026-06-25 Run the four country CLI recipes (help, list, add, delete) against a live provisioned database; fix stale flags, add .env and explicit-db-args variants, refresh captured results.
Verify country shell commands post-NATS DONE 2026-06-25 2026-06-25 All five commands verified on v0.0.21. Fixed countries get parse regression (do_requestdo_auth_request). Recipes refreshed.
Sync SQL codegen for country DONE 2026-06-25 2026-06-25 Table model (country_table.org) created; SQL regenerated via sql_schema_create.mustache; deviations documented (index rename, security definer placement). SQL layer only.
Sync C++ core codegen for country DONE 2026-06-25 2026-06-26 Run domain, repository, service, generator profiles; iterate until zero diff against hand-written C++ core artefacts.
Sync C++ messaging codegen for country DONE 2026-06-26 2026-06-26 Run protocol, nats-eventing, nats-handler profiles; iterate until zero diff against country_protocol.hpp, country_changed_event.hpp, country_handler.hpp.
Sync Qt codegen for country DONE 2026-06-26 2026-06-27 Run Qt profile; iterate until zero diff against Qt artefacts in ores.qt.refdata.
Verify country Qt UI end-to-end post-NATS DONE 2026-06-27 2026-06-27 List window, detail dialog, history dialog (HistoryDialogBase), delete, and eventing against live services. History likely shares the epoch-timestamp blocker (story 71116F94) — confirm and mark BLOCKED if so.
Write country documentation (manual chapter, NATS reference) BACKLOG     Manual chapter mirroring chapter 5 (currencies): entity description, Qt windows, shell and CLI commands; wire into user_manual.org. Extend the refdata NATS message reference with country subjects and the country_changed event.
File Wt and HTTP gap captures for country BACKLOG     Per the epic: file backlog captures for the missing Wt widgets and HTTP endpoints, referencing the appraisal.
Bring country under codegen and verify zero-diff regeneration ABANDONED     Superseded by task 3E63E209 (Sync SQL codegen for country) and the three layer-specific codegen tasks (DFD25563, 8AF11AB2, 7663EB07).
Switch codegen model type detection to frontmatter ABANDONED     Moved to codegen model unification story (step 1 of 7, D2256981).
Unify SQL template to handle full validation function and insert trigger ABANDONED     Moved to codegen model unification story (step 2 of 7, D2256981).
Merge table and entity org parsers ABANDONED     Moved to codegen model unification story (step 3 of 7, D2256981).
Add sql_only and has_qt variability guards to codegen ABANDONED     Moved to codegen model unification story (step 4 of 7, D2256981).
Migrate country to unified codegen model BACKLOG     Move _table.org sections (validation function, insert trigger, coding scheme, image_id, has_tenant_id) into ores.refdata.country.org; verify generated SQL is byte-identical to the table-pathway output; delete country_table.org. Blocked on steps 1-4 above (D2256981).

Emacs 29.1 (Org mode 9.6.6)