Task: ORE export drops required elements from 17 example documents

Table of Contents

This page documents a task in the Close the credit loose ends story. It captures the goal, current status, acceptance, and any notes or results.

1. Goal

The import to export round trip over external/ore/examples/Products/Example_Trades is lossy for 17 of the 132 documents. The walk writes all 132 and reports no skips, but the exported document is missing a child element the schema requires, so it cannot be read back. One trade per affected document is lost.

The failures are not scattered. They cluster in the Commodity, Credit, FX worst-of and Exotic product mappers, and each one drops a different required element:

Document Missing element Line
Commodity_APO_Strip_NYMEX_CSX.xml Quantity 27
Commodity_Basis_Swap_NYMEX_CL.xml Quantity 38
Commodity_Option_Strip_NYMEX_NG.xml Quantity 27
Commodity_Swap_LME_AL_AVG_ALT.xml Quantity 38
Commodity_Swap_NYMEX_A7Q.xml Quantity 38
Commodity_Swaption_NYMEX_NG.xml Quantity 26
Exotic_EquityAccumulator_single_name.xml RangeBound 31
Exotic_EquityTaRF.xml BarrierData 30
Exotic_EquityWorstOfBasketSwap.xml InitialPrice 26
Exotic_FXWorstOfBasketSwap.xml InitialPrice 26
Exotic_FxAccumulator.xml RangeBound 31
Exotic_KnockOutSwap.xml LegData 20
Exotic_PerformanceOption_01_COM.xml StrikePrice 27
Exotic_PerformanceOption_01_FX.xml StrikePrice 35
FX_WorstOfBasketSwap.xml InitialPrice 26

The two risk participation agreement documents were listed here. The credit rework fixed them: the agreement now stores and exports its protection fee, its underlying and its participation rate, and both documents re-read.

Run of 2026-09-26 over the 132 documents:

Measure Value
XML files walked 132
Skipped 0
Written 132
Trades mapped 182
Trades passthrough 0
Outputs that re-read 115
Trades recovered 165

2. Status

Field Value
State DONE
Parent story Close the credit loose ends
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-10-15

3. Acceptance

  • Every document the walk writes under external/ore/examples/Products/Example_Trades whose dropped element has a field in the domain instrument can be read back, so known_export_gaps() in xml_exporter_directory_roundtrip_tests.cpp shrinks to the five documents whose value the domain does not hold.
  • A value the export now writes comes from the domain instrument and is not invented: what the reverse mapper writes, the forward mapper reads.
  • No document that already re-read stops re-reading.

4. Plan

(Implementation strategy. Written when work starts; key decisions are distilled into the parent story's * Decisions at close, but the plan itself stays — it is the historical record of what we did.)

5. Measurements

Every number this task claims about a tree, with the command that produced it and the commit it was measured at. A number without both is not a measurement, because a clean-up record describes a tree and trunk moves under a branch: a consumer count, a gate's component count and a suite's size all change with no code changing, and the reader of a bare number cannot tell which tree it came from. Where the raw output matters, put it in an evidence/ file beside the task and name that file in the Command column.

What Value Command Commit
Documents that re-read 127 of 132, up from 118 compass test run -- -R ores.ore.core.tests (the case "roundtrip mirrors every portfolio in the ORE example corpus") 04d150a501
Documents that still fail to re-read 5, down from 14 the same case; known_export_gaps() names them 04d150a501
ORE core suite 667 cases, 0 failures compass test run -- -R ores.ore.core.tests 04d150a501
Local check gate 18 of 18 checks passed compass check 04d150a501

6. Notes

7. Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
     

8. PRs

PR Title
#3368 [ore] Close nine of the fourteen ORE export fidelity gaps

9. Review

Comment summary File Decision Notes
       

10. Result

The documents that fail to re-read fell from fourteen to five. Nine now round trip: the six commodity documents, the equity TaRF, and the two performance options. Each fix carries a value the domain holds — a commodity leg's quantity, the swaption's fixed price, the TaRF's knock-out level, and the performance option's strike prices.

The five that remain each need a value the domain instrument does not hold, so closing them means changing the domain, not the mapper. That work has its own task, which the story now carries.

Acceptance met: the nine documents whose dropped element has a domain field re-read, every value written comes from the instrument, and no document that already re-read stopped re-reading.

11. Promoted from capture

Captured 2026-09-26 in the product backlog; promoted preserving the UUID.

11.1. What

The import to export round trip over external/ore/examples/Products/Example_Trades is lossy for 17 of the 132 documents. The walk writes all 132 and reports no skips, but the exported document is missing a child element the schema requires, so it cannot be read back. One trade per affected document is lost.

The failures are not scattered. They cluster in the Commodity, Credit, FX worst-of and Exotic product mappers, and each one drops a different required element:

Document Missing element Line
Commodity_APO_Strip_NYMEX_CSX.xml Quantity 27
Commodity_Basis_Swap_NYMEX_CL.xml Quantity 38
Commodity_Option_Strip_NYMEX_NG.xml Quantity 27
Commodity_Swap_LME_AL_AVG_ALT.xml Quantity 38
Commodity_Swap_NYMEX_A7Q.xml Quantity 38
Commodity_Swaption_NYMEX_NG.xml Quantity 26
Exotic_EquityAccumulator_single_name.xml RangeBound 31
Exotic_EquityTaRF.xml BarrierData 30
Exotic_EquityWorstOfBasketSwap.xml InitialPrice 26
Exotic_FXWorstOfBasketSwap.xml InitialPrice 26
Exotic_FxAccumulator.xml RangeBound 31
Exotic_KnockOutSwap.xml LegData 20
Exotic_PerformanceOption_01_COM.xml StrikePrice 27
Exotic_PerformanceOption_01_FX.xml StrikePrice 35
FX_WorstOfBasketSwap.xml InitialPrice 26

The two risk participation agreement documents were listed here. The credit rework fixed them: the agreement now stores and exports its protection fee, its underlying and its participation rate, and both documents re-read.

Run of 2026-09-26 over the 132 documents:

Measure Value
XML files walked 132
Skipped 0
Written 132
Trades mapped 182
Trades passthrough 0
Outputs that re-read 115
Trades recovered 165

11.2. Why

A round trip that silently drops a required element is worse than one that refuses the document: the walk reports success and writes a file that no longer satisfies the schema. Nothing caught this before, because the per-document suites assert field fidelity for the products they name, and the CLI-era checker compared document trees without re-reading them through the domain loader.

The gap is now visible. The test that found it, xml_exporter_directory_roundtrip_tests.cpp, records these 17 names and fails if the set changes in either direction, so a new regression fails and a fix has to shrink the list.

11.3. Reproduce

./build/output/linux-clang-debug-make/publish/bin/ores.ore.core.tests "[exporter][external]"

11.4. References

  • projects/ores.ore/core/tests/xml_exporter_directory_roundtrip_tests.cpp, the test that found it.
  • projects/ores.ore/core/src/xml/exporter.cpp, the directory walk and the export it calls.
  • projects/ores.ore/core/src/domain/, the per-product mappers that build the exported document.
  • external/ore/examples/Products/Example_Trades/, the corpus.

11.5. See also

Emacs 29.3 (Org mode 9.6.15)