Task: Build the realistic ACME base bundle
Table of Contents
This page documents a task in the Make the ACME data realistic and isolate the ORE samples story. It captures the goal, current status, acceptance, and any notes or results.
1. Goal
Give each ACME party realistic netting agreements, netting sets, CSAs and portfolios, built from the ORE samples and improved, and make the base bundle follow the report definitions each party holds. The result looks like a real group's data and carries ACME's own catalogue, methodology and licence.
2. Status
| Field | Value |
|---|---|
| State | DONE |
| Parent story | Make the ACME data realistic and isolate the ORE samples |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-10-10 |
3. Acceptance
- A new ACME methodology states how the data is built. It names the ORE samples as the starting point and lists every improvement.
- Each party has agreements with banks that fit its jurisdiction, with a matching agreement type and governing law.
- The CSAs cover variation margin, initial margin and none. Currencies, thresholds, minimum transfer amounts, call frequencies and overnight indices are realistic for each party. No CSA uses a retired index.
- Every netting set, counterparty and portfolio that a seeded report definition names exists for its party.
- Nothing in the base bundle reads as test or sample data.
- pgTAP and a fresh ACME provision pass.
4. Plan
- Audit what is published today, per party. Find every reference to the current dataset codes, in profiles, tests, recipes and the manual.
- Read the report-needs investigation. List the netting sets, counterparties
and portfolios each party's report definitions need. The reports that read
netting.xmlandcounterparty.xml(SA-CCR, CVA, exposure) set the demand. - Design the data: banks per legal entity, agreement types, CSA archetypes, the portfolio and book tree.
- Write the ACME methodology, datasets and bundle. Wire them into
acme_demoat the step that follows each party's report definitions. - Verify with pgTAP and a fresh ACME provision.
5. Measurements
Measured on a freshly recreated database and a fresh ACME provision.
| Party | Agreements | Netting sets | CSAs |
|---|---|---|---|
| Acme Corporation Plc | 5 | 5 | 2 |
| ACME Corporation UK plc | 7 | 8 | 5 |
| ACME Corporation US Inc | 5 | 6 | 4 |
| ACME Corporation HK Ltd | 4 | 4 | 3 |
| Total new | 21 | 23 | 14 |
- Acme Corporation Plc also still holds the 8 sample agreements, 19 sample sets and 7 sample CSAs. The retarget task removes them.
- The four intragroup agreements each appear under both parties.
- Four ACME counterparties carry the
9695ACME...LEIs. - No netting set belongs to a different party than its agreement. No CSA belongs to a different party than its set.
- New CSAs: 7 USD-SOFR, 3 GBP-SONIA, 4 EUR-ESTER. The 5 EUR-EONIA CSAs are the sample ones.
- pgTAP: 14 of 14 new tests pass. 6 of them fail when the data is corrupted. The full pgTAP run passes except two known failures that also fail on main.
6. Notes
Audit, 2026-10-10 (plan step 1). Read from a freshly provisioned ACME tenant and from the repository. The database was queried; the report needs are not yet read (plan step 2).
- What ACME publishes today, per party. Four legal entities plus the system
party. Each entity has a bundle (
acme_group,acme_uk,acme_us,acme_hk) of datasets namedacme.<entity>.<thing>: business units, portfolios, books, accounts and contact information. Theload_staffstep of theacme_demoprofile publishes each bundle to its own party. The realistic netting data fits this pattern with no new party machinery: addacme.<entity>.netting_agreements,netting_setsandcsasdatasets to the entity's bundle. - Counts after provisioning. Acme Corporation Plc 1 business unit, 22 portfolios, 22 books. UK 15 business units, HK 6, US 6, each with 20 portfolios and 22 books. The tenant holds 12,959 GLEIF counterparties, 67 market series, 155 report definitions and no trades.
- Mixed portfolios. Every entity has two official top-level portfolios,
Global PortfolioandRegulatory Capital, both aggregating in USD. Acme Corporation Plc also holds the sample portfoliosPF1andPF2at the top of the same tree, because theore_samplesbundle publishes them there. Official and sample portfolios are not separated. - What the sample netting data is. Eight banks, 19 netting sets, 7 CSAs and 19 ORE aliases, published to Acme Corporation Plc only. All 7 CSAs are in EUR and every threshold pair is identical. Five use the overnight index EUR-EONIA, retired in 2022. The set descriptions read as test data ("pricing test book", "wrapped trades").
- References to the current dataset codes. The sample netting and portfolio
dataset scripts, the bundle and bundle member scripts, the
acme_demoprofile,party_provisioning_plan.hpp, the pgTAP sample netting test, the trading envelope names test, the manual chapter on initial setup, and a number of plans. The retarget task owns the renaming; this task adds new datasets and leaves these in place. - Banks are real GLEIF entities. A counterparty is found by LEI, and the GLEIF set holds many affiliates of one bank (BNP Paribas alone has dozens), so each agreement must name the right legal entity by LEI.
Decisions.
- Panel. US Inc faces JPMorgan, Bank of America and Citi under New York law. UK plc faces Barclays, HSBC, Deutsche Bank, BNP Paribas and UBS under English law. HK Ltd faces HSBC Hong Kong, Standard Chartered and Citi under English law. Group Plc faces HSBC and JPMorgan.
- Hong Kong. Bank of China (Hong Kong) is not in the loaded GLEIF set. Citi is the third bank. Standard Chartered Bank is the covering legal entity for its Hong Kong branch.
- Intragroup. Group Plc holds an uncollateralised agreement with each subsidiary. UK plc and US Inc hold a variation margin agreement in USD. Both sides share one reference.
- CSA archetypes. Variation margin only, variation and initial margin, and uncollateralised. Only live overnight indices are used.
- Counterparties. The new dataset
acme.lei_counterpartiesmakes the ACME entities counterparties, so intragroup agreements resolve by LEI. - Assumption. The initial margin regulation codes
ESA,CFTC,PRUDENTIALandHKMAare not yet checked against the codes ORE accepts.
7. Test Scenarios
Manual QA scenarios (scaffolded via compass add test_scenario)
that verify this task. Link
new ones here as they're created; the scenario doc itself links back
via its "Verifies task" field.
| Scenario | State | Notes |
|---|---|---|
8. PRs
| PR | Title |
|---|---|
| #3077 | [seeder,sql] Build the realistic ACME netting base data |
9. Review
| # | Comment summary | File | Decision | Notes |
|---|---|---|---|---|
| 1 | Generator workflow unclear | generate_sql.py | Accepted | Header and docstring name both generators and their order. Regenerated. |
| 2 | IM regulation codes unverified | generate_netting_data.py | Declined for now | No accepted-value list exists in the repo (the xsd types the field as a string). Stays an assumption in Result. |
| 3 | Dangling report references | task page | Noted | The report-needs read stays an open item for the next task. |
| 4 | Two failures on main | PR body | Noted | Both listed in Result. No issue opened. |
| 5 | Gate still running | PR body | Noted | Result checked before merge. |
| 6 | Pinned counts in the test | acme_netting_data_test.sql | Accepted | Comment added. |
| 7 | Generator out of step with five files | task page | Noted | Listed in Result. |
10. Result
Each ACME legal entity now holds realistic netting agreements, netting sets and
CSAs. They come from the new datasets acme.<entity>.netting_agreements,
netting_sets and csas, with a methodology that names the ORE samples as the
starting point and lists every improvement. The data is in the entity bundles
and is verified on a fresh provision.
Acceptance. Met: the methodology; per-jurisdiction agreements with matching type and law; all three CSA regimes with live indices; no sample wording in the new data; pgTAP and a fresh provision.
Not met, and why.
- The report definitions' needs were not read (plan step 2). It is not verified that every netting set, counterparty and portfolio a seeded report definition names exists.
- Acme Corporation Plc still carries the sample netting data and the portfolios PF1 and PF2. The retarget task removes them.
- No trade corpus exists yet.
- The regulation codes are assumptions.
Known issues not caused by this task. The generator is out of step with five
checked-in files (books and photos). The codegen test
test_trade_type_catalogue fails in a full run on main. The pgTAP synthetic
scope test hard-codes a DDL user name.