Task: Retarget the ORE samples bundle to the import needs

Table of Contents

This page documents a task in the Make the ACME data realistic and isolate the ORE samples story. It captures the goal, current status, acceptance, and any notes or results.

1. Goal

Keep the ore_samples bundle and shrink it to the reference data the ORE sample import needs. Label every item as sample data. Install it on every party. Land every import in a sandbox portfolio tree.

2. Status

Field Value
State DONE
Parent story Make the ACME data realistic and isolate the ORE samples
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-10-10

3. Acceptance

  • The bundle holds only: the sample counterparty aliases, the sample netting agreements, sets and CSAs the aliases resolve to, the netting set aliases, the sample portfolio names, and the sandbox portfolio.
  • Every item has a code and a description that mark it as ORE sample data.
  • The sample CSAs keep ORE's terms.
  • The bundle installs on every party in acme_demo. Agreement numbers carry a per-party part, so no two parties collide.
  • A sample import creates its portfolio and books inside the sandbox tree. The ACME portfolio tree gets nothing.
  • After provisioning, each party resolves all 19 ORE netting set ids and the portfolio names the samples use.
  • The methodology, catalogue and licence describe sample data, not ACME data.

4. Plan

  1. Take the item list from the current bundle. Remove everything the import does not need.
  2. Rename the codes and descriptions to mark sample data. Give agreement numbers a per-party part.
  3. Add the sandbox portfolio. Find where an import puts its books today, and point it at the sandbox tree.
  4. Move the bundle from one party to every party in acme_demo.
  5. Update pgTAP. Provision ACME. Import a sample into two parties and check that each lands in its own sandbox.

5. Measurements

Measured on a database recreated from the scripts and a fresh ACME provision through the bundle path.

Party Sample agreements Sample sets Sample aliases Sandboxes PF1/PF2 in sandbox Official PF1/PF2
ACMCOR 8 19 19 1 2 0
ACCOUK 8 19 19 1 2 0
ACCOUS 8 19 19 1 2 0
ACCOHK 8 19 19 1 2 0
  • Every sandbox is open, shared, purpose sample, anchored at Global Portfolio and owned by tenant_admin.
  • The ACME base data from the earlier task is intact on every party.
  • pgTAP: 12 of 12 new sandbox tests pass. The sample netting and envelope names tests pass.

6. Notes

Audit and design, 2026-10-10.

  • The sandbox already exists. Sprint 26 added the sandbox entity: an owner, an anchor in the official tree, visibility, and its own portfolio tree. The ORE import does not use it yet. A sample import still creates portfolios and books from the directory tree it reads.
  • Decision (user, 2026-10-10). This task seeds data only. It opens one shared sample sandbox per party, anchored at the party's official top portfolio and owned by the publishing actor. A separate task makes the importer place its books in that sandbox.
  • Decision: the artefact type. The named_portfolios type published sample names as official top-level portfolios. It is replaced by sandbox_portfolios, which writes the same names as roots of the sandbox. The names PF1 and PF2 stay, because an import resolves them by name.
  • Decision: codes. Agreement numbers start ORE-SAMPLE-, netting set codes start ORE-SAMPLE-NS-, and every description starts ORE sample data:. The per-party part the acceptance asked for is not needed: agreement numbers are unique per party since the party-scoping task.
  • Decision: counterparty aliases. They stay a tenant-level opt-in (ore.counterparty_aliases). Counterparties are tenant-level, so one publish serves every party.
  • CSA terms. Unchanged. They keep ORE's terms, including EUR-EONIA.

7. Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
     

8. PRs

PR Title
#3084 [sql,refdata] Retarget the ORE samples bundle

9. Review

# Comment summary File Decision Notes
1 Duplicate-name check and the empty sandbox refdata_publish_from_dq_create.sql Accepted The check stays party-wide, because portfolio names are unique per party. The sandbox is now created only when a name is to be added. The comment says so.
2 Silent skips publish handler Declined Logging needs a handler change. Left for a follow-up; noted under Known gap.
3 Missing party code refdata_publish_from_dq_create.sql Accepted A missing code returns skipped_no_party.
4 Anchor fallback refdata_publish_from_dq_create.sql Declined anchor_portfolio_name is a preference, as documented. Every ACME party has Global Portfolio.
5 Test coverage refdata_publish_sandbox_portfolios_test.sql Partly accepted Added a check that a repeat publish opens no second sandbox. The admin fallback is shown by the clean provision, not by a fixture.
6 Process: run a full provision PR body Done Clean provision run on the final scripts; see Proof.

10. Result

The ORE samples bundle is labelled, installed on every party, and its portfolios are published into a sample sandbox.

  • Agreement numbers start ORE-SAMPLE-, netting set codes start ORE-SAMPLE-NS- and each sample description starts ORE sample data:. The CSA terms are ORE's own.
  • The bundle goes to Plc, UK, US and HK. Each holds 8 sample agreements, 19 sets and 19 aliases.
  • The new publish sandbox_portfolios replaces named_portfolios. It opens a shared sample sandbox per party and writes PF1 and PF2 as its roots. The owner is the publishing actor, else the tenant administrator.

Acceptance. Met: labelling; the CSA terms; installation on every party; the sample portfolios outside the official tree; the methodology and dataset descriptions say sample data.

Not met.

  • A sample import does not yet create its portfolios and books in the sandbox. The importer still builds them from the directory tree it reads. The task "Place the ORE import in the sample sandbox" covers it.
  • No import was run, so it is not shown that each party resolves the portfolio names inside the sandbox. The 19 netting set ids and their aliases do exist on every party.
  • The bundle does not hold the counterparty aliases. They stay a tenant-level opt-in, because counterparties are tenant-level.

Proof. A clean provision through the bundle path, from a database recreated from the final committed scripts and services built from the same tree, gave every party one open, shared sample sandbox owned by tenant_admin, with PF1 and PF2 as its roots. An earlier provision, before the owner fallback, showed the publish ran with no actor and wrote nothing.

Known gap. The publish handler logs only inserted, updated and skipped counts. A skipped_no_owner or skipped_no_anchor result is silent.

Emacs 29.3 (Org mode 9.6.15)