Task: Round trip the shared reference configuration

Table of Contents

This page documents a task in the Model the ORE report configuration as entities story. It captures the goal, current status, acceptance, and any notes or results.

1. Goal

Every other document references these by key, so proving them removes a source of noise from every later round trip.

2. Status

Field Value
State DONE
Parent story Model the ORE report configuration as entities
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-09-28

3. Acceptance

  • Every reference document in the corpus imports, exports and matches.
  • The reference configuration is owned by the reference component and only referenced by reporting.
  • A reference set that the corpus does not cover is recorded as a measurement.

4. Plan

  1. Register each reference kind with the round-trip harness and measure it before changing anything, because the mappers were written for import and nothing had ever compared their output with their input.
  2. State a normalisation in the kind's comparison only when the two documents carry the same information, and only as narrowly as the normalisation is.
  3. For a kind whose difference is a lost field, fix the model rather than the comparison, and record the fix if it is larger than this task.

5. Measurements

What Value Command Commit
Calendar adjustment files that round trip 5 of 5 ores.ore.core.tests "[ore][xml][roundtrip][reference]" 77c286dcce
Currency files that round trip 12 of 13 ores.ore.core.tests "[.][measurement]" 77c286dcce
Convention files that round trip 0 of 72 ores.ore.core.tests "[.][measurement]" 77c286dcce
The reasoning for each, with the first difference quoted evidence/round_trip_reference_kinds.md the probe above 77c286dcce

6. Notes

  • The harness is hidden for the two kinds that do not pass, because their failures are losses rather than comparisons that need stating, and a test that asserts a loss is green is worse than a measurement a person reads.
  • Currencies. ORE's CurrencyType has no home on the refdata currency. monetary_nature is a different axis and the translation into it is not invertible, since Major and Minor both become fiat. The fix is a column carrying ORE's type, seeded as a lookup, mapped one to one.
  • Conventions. The mapper models nine categories and skips the rest silently, which its own header says. The corpus needs at least the Zero convention's TenorBased and the categories the header names. The mapper must also stop skipping silently, because a silent skip is why 72 files went unmeasured.
  • Calendar adjustments. The one normalisation is that an empty date list may be written as an empty element or omitted. A list with dates is compared date by date, so the rule cannot hide a dropped date.

7. Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
     

8. PRs

PR Title
#2350 [ore] Measure the shared reference kinds

9. Review

Comment summary File Decision Notes
       

10. Result

Measured in PR #2350, which closes this task as far as the corpus allows it today. Two of the three kinds need work that is larger than a comparison, so the task's acceptance is met for one kind and recorded as a measurement for the other two, which is what its third acceptance bullet asks for.

Calendar adjustments round trip, five of five. The kind is registered with the harness and asserts it. Its comparison declares the one normalisation the mapper makes: ORE writes a date list it has no dates for as an empty element and the reverse omits it. An empty list is the same list either way, and a list with dates is compared date by date, so the rule cannot hide a dropped date. The story records where the line is between a normalisation that carries the same information and a comparison widened to hide a loss.

Currencies do not, twelve of thirteen. The only file in the corpus that sets CurrencyType fails on it. The mapper translates ORE's type into the refdata monetary_nature, and Major and Minor both become fiat, so the export cannot recover which one the document held. They are different axes as well: monetary_nature says what a currency is, CurrencyType says how important it is, and market_tier is a foreign key to a lookup whose values are g10 and emerging. The fix is a column carrying ORE's type, seeded as a lookup and mapped one to one. That is a refdata model change with the usual blast radius, and it is the next step rather than part of this measurement.

Conventions do not, none of seventy-two. The mapper models nine categories and skips the rest silently, which its own header states. The corpus needs at least the Zero convention's TenorBased, which the export drops, and the categories the header names: AverageOIS, TenorBasisSwap, CrossCurrencyBasis and InflationSwap. The mapper must also stop skipping silently, because a silent skip is why seventy-two files went unmeasured for as long as they did.

The two failing kinds are measured by a hidden test rather than asserted green. A test that passes while a document loses a field is worse than a measurement a person reads, and the evidence file names the byte and the line of each first difference. The reasoning and the required fixes are in evidence/round_trip_reference_kinds.md.

Emacs 29.3 (Org mode 9.6.15)