ORE Market Data File Types — Phase 1 (ores.ore)

There are 38 distinct quote-key prefixes across the ORE example library, covering interest rates, FX, volatilities, credit, equity, commodity, inflation, and miscellaneous types. Representative examples:

20160205 ZERO/RATE/EUR/BANK_EUR_BORROW/A365/2Y            0.0024
20160205 IR_SWAP/RATE/CHF/2D/1D/1M                        0.0035
20160205 FX/RATE/EUR/CHF                                  1.0947
20160205 SWAPTION/RATE_LNVOL/CHF/25Y/10Y/ATM              0.3820
20160205 CAPFLOOR/RATE_LNVOL/CHF/20Y/6M/0/0/0.025         0.5410
20160205 EQUITY/PRICE/SP5/USD                             2023.81
20160205 CDS/CREDIT_SPREAD/BANK/SR/USD/1Y                 0.0050
20160205 COMMODITY_FWD/PRICE/GOLD/USD/2016-02-29         1207.40

2021-09-30 EUR-EONIA          0.0074600000
2016-01-28 EQ-SP5             2244.2
2020-09-01 UKRPI              293.3

ores.ore needs at most two structs:

struct market_datum {
    std::chrono::year_month_day date;
    std::string key;    // opaque — not decomposed at this layer
    std::string value;  // raw string — exact round-trip, no float precision loss
};

struct fixing {
    std::chrono::year_month_day date;
    std::string index_name;
    std::string value;  // raw string
};

market_datum and fixing are structurally identical but semantically distinct, so they are separate types for clarity at the call site.

Values are stored as std::string rather than double or a decimal type. This is intentional:

• ORE files contain values with varying precision (e.g. 0.0074600000, 2244.2, -0.006119). Any floating-point representation introduces representation error that manifests as noise on serialization.
• ores.ore is a file-format layer, not a computation layer. Callers that need arithmetic work with values through ores.marketdata types (Phase 2), which will use a decimal type.

Both file types parse the same three-column line format. A single market_data_line_parser handles tokenization and date parsing for both, auto-detecting the date format (YYYYMMDD vs YYYY-MM-DD).

Serialization writes the canonical date format per file type:

• market_data_serializer — outputs YYYYMMDD
• fixings_serializer — outputs YYYY-MM-DD

The structs are trivially simple and handwritten. Codegen adds overhead without benefit here. The existing XSD-driven codegen in ores.ore applies only to XML configuration files.

New headers in projects/ores.ore/include/ores.ore/market/:

• market_datum.hpp — market_datum struct
• fixing.hpp — fixing struct

No implementation files needed; structs are value types only.

New files:

• include/ores.ore/market/market_data_parser.hpp
• src/market/market_data_parser.cpp

Interface:

namespace ores::ore::market {

/**
 * @brief Parses an ORE market data quote file (market*.txt).
 *
 * Skips blank lines and comment lines (starting with #).
 * Accepts both YYYYMMDD and YYYY-MM-DD date formats.
 */
std::vector<market_datum>
parse_market_data(std::istream& in);

/**
 * @brief Parses an ORE fixings file (fixings*.txt).
 *
 * Skips blank lines and comment lines (starting with #).
 * Accepts both YYYYMMDD and YYYY-MM-DD date formats.
 */
std::vector<fixing>
parse_fixings(std::istream& in);

} // namespace ores::ore::market

Both functions delegate to a shared internal line tokenizer. Error handling: malformed lines produce a std::invalid_argument with the line number and offending content in the message.

New files:

• include/ores.ore/market/market_data_serializer.hpp
• src/market/market_data_serializer.cpp

Interface:

namespace ores::ore::market {

/**
 * @brief Serializes market data quotes to ORE text format (YYYYMMDD dates).
 */
void serialize_market_data(std::ostream& out,
                           const std::vector<market_datum>& data);

/**
 * @brief Serializes fixings to ORE text format (YYYY-MM-DD dates).
 */
void serialize_fixings(std::ostream& out,
                       const std::vector<fixing>& fixings);

} // namespace ores::ore::market

Output format:

• One entry per line: DATE<TAB>KEY<TAB>VALUE
• Values are written exactly as stored (no reformatting)
• No comment lines are emitted (comments are not preserved through parse/serialize)

New test file: projects/ores.ore/tests/market_roundtrip_tests.cpp

Test strategy: for each fixture file in the table above:

1. Read the file content from external/ore/examples/...
2. Parse into structs via the parser
3. Serialize back to a string via the serializer
4. Compare the entry count and each date / key / value triple against the original parse

The comparison is structural (field-by-field), not byte-identical string diff, because:

• Comment lines are dropped on serialization
• Column spacing may differ

A separate market_data_parser_tests.cpp covers unit-level cases: malformed lines, unsupported date formats, comment-only files, empty files.