How do I import the 2016-02-05 ORE reference vintage market data?

Table of Contents

Reproduce, from the command line, the market data import needed by Import the 2016-02-05 ORE reference vintage — the dominant ORE anchor date, sourced from Legacy/Example_56. The ores-shell block below is tangled into the runnable script library as import_legacy_example_56.ores.

This vintage's market.txt genuinely repeats ~100 (date, key) pairs with a different value later in the file (real ORE example data, not a decomposition bug — see Full market.txt import hits duplicate observation key for the investigation). parse_market_data=/=parse_fixings (projects/ores.ore/core/src/market/market_data_parser.cpp) now de-duplicate such repeats — the later line wins — and report each one via import_market_data_response.warnings, which the shell prints.

Question

How do I import the 2016-02-05 ORE reference vintage market data from the shell?

Answer

The marketdata import command (projects/ores.shell/src/app/commands/marketdata_commands.cpp) sends the file content as a single import_market_data_request over NATS (subject marketdata.v1.import), the same request the Qt ImportTradeDialog sends — reusing import_service as-is, no server-side code specific to the shell. This vintage has no fixings.txt, so only --file is passed.

Expectations: the services are up (compass services start), a tenant/party is provisioned and logged in (e.g. via the Acme Corporation provisioning recipe), and --source tags every imported market_observation row so it can be queried back out separately from other imports. 

marketdata import --file external/ore/examples/Legacy/Example_56/Input/market.txt --source ore.reference
logout

Run it from a connected, logged-in shell, or load the generated script. File paths are relative to the repo root — compass shell -f <script> always runs ores.shell with the repo root as its working directory, so no absolute path or project-root environment variable is needed: 

./ores.shell ${log_args} ${connect_args} ${login_args} << 'EOF'
<<import-legacy-example-56>>
EOF

> marketdata import --file external/ore/examples/Legacy/Example_56/Input/market.txt --source ore.reference
Importing market data...
✓ Imported 176 series, 7777 observation(s), 0 fixing(s):
⚠ 103 warning(s):
  ⚠ market data line 296: duplicate key 'FRA/RATE/EUR/1M/6M' — superseded by line 272
  ⚠ market data line 3294: duplicate key 'CAPFLOOR/RATE_LNVOL/USD/20Y/3M/0/0/0.06' — superseded by line 6564
  ... (103 total — one per repeated key, each superseded by its later occurrence)
> logout
✓ Logged out successfully.

7880 raw lines minus 103 de-duplicated repeats = 7777 observations, confirmed exactly. Pass --duplicates-are-errors semantics via the request's duplicates_are_errors field (not yet exposed as a shell flag) to treat these as hard failures instead — see import_market_data_request in projects/ores.marketdata/api/include/ores.marketdata.api/messaging/import_protocol.hpp.

Script

projects/ores.shell/scripts/library/marketdata/import_legacy_example_56.ores — generated from this recipe's ores-shell block by compass build --direct tangle_shell_scripts (do not edit the .ores by hand).

Tested by

Manual: run against a freshly recreated, Acme-provisioned database (compass db recreate -y -k then the Acme Corporation provisioning recipe), then compass shell -u <user> -p <password> -f import_legacy_example_56.ores.

See also

Emacs 29.3 (Org mode 9.6.15)