Task: Import a pricing engine configuration as ORE XML

Two requests and two responses, in the shape the currencies' pair established:

• preview_pricing_engines_import_request carries the file's text and the target: the reporting configuration to compare against, or nothing for a configuration the file would create. The response states the root element it found, the products and settings the file holds, the names of the root-level settings, the difference in counts and in rows, and three lists of what the system does not know.
• import_pricing_engines_request carries the same file, the same target or a new configuration's name and variant, the keys of the removed rows the person kept, and the reason and commentary. The response states the configuration it wrote, what it added, changed, deleted and kept, and the rows it could not write.

A row of the difference is named by a key: a product by its type and which product of that type it is (Swap#0), a setting by its product's key, its scope and its name (Swap#0/engine/DiscountingCurve). The key is what a choice names, and it is what makes two products of the same type, or a setting repeated twice in one scope, tellable apart.

pricing_engines_import::parse reads the text with pugixml and answers the same analytics pricing_engines_document the exporter writes: the products in their order, each carrying Model, ModelParameters, Engine and EngineParameters, then one GlobalParameters block, with a position per scope so the export comes back in the order the file wrote. A text that is not well formed, or whose root element is another document's, answers a refusal that names the root element it found and writes nothing.

Three things the tables cannot hold are named rather than dropped:

The vocabulary is analytics': the handler reads the pricing engine types and passes the codes to the difference, so the parser decides nothing about what a screen may offer.

A product is matched by its type and by which product of that type it is; a setting by its product, its scope and its name, and then by which repeat of that name it is. Nothing is matched by identity, because the file states no identifiers.

Only removed rows ask a question, and the review starts every one of them at Keep it: a file that omits a row usually omits it by accident. pricing_engines_import::kept_by_default states that default once, so the screen and the command cannot default differently, and the import request carries the keys of the removed rows the person kept.

The import re-reads the target and re-parses the file, so the write acts on the configuration as it stands rather than on the preview's reading. It assembles the document with pricing_engines_import::merge: the file's rows laid over the target's, each row keeping the identity and the version of the row the target already holds, so a write corrects a row rather than replacing it and the history stays one line per setting. A kept removal is carried from the target with the settings it owns; a removal the person dropped is absent from the document, which is how the analytics save removes it.

It then saves through analytics.v1.pricing_engines_documents.put, the same operation the configuration's own screen uses, so the validation, the history and the party rules are the screen's, and the person's reason and commentary travel on the rows, which is where that save keeps them.

The screen's create writes the reporting configuration the record fills and then the document, because analytics finds a document by that reporting configuration. Both the browser and the shell import, so neither may carry that order: the ORE service's handler registers the reporting configuration through reporting's own put_configuration, saves the document, and removes the header again if the save is refused, so a refused import leaves nothing behind.

The configuration screen standard makes a file put into an official configuration a proposal, with a sign-off. Sign-off is not built. The import writes directly, with the person's reason and commentary, and the review states that in as many words before the press. It is the one place this build does not reach the standard, and it is recorded as the known gap rather than dressed as an approval.

Import stands on the list header between Refresh and Workflows (RS-ORE-02), at /reporting/pricing-engines/import, and follows the standard's five steps on the steps UI: the file, what it holds, what the system does not know, the target (new or existing), and the difference with Keep it as the default for a removed row. The preview is read twice: once on leaving the file step, which states what the file holds and what the system does not know without a target, and again on leaving the target step, which states the difference against the configuration the person chose.

The five steps are the shared module's, not the kind's: OreImportPage takes a kind that declares its document, its target and its wording, so the currencies and the pricing engines draw the same dialog. The kind's own file states what a row is called and what the review offers for it, and nothing else.

pricing_engines import <id or --new-name> --file <path> [--dry-run] sends the same two operations, prints what the file holds, what the system does not know and the difference, and then writes, keeping every removed row, because a command line states no per-row choice. --dry-run stops after the preview.

1. The protocol, generated to C++ and TypeScript.
2. The parser and the difference in ores.ore.core, with its tests.
3. The handler: the preview, then the import, with the reporting configuration's registration.
4. The BFF's two routes, with their tests.
5. The shared steps module, generalised for a kind with a target, and the pricing engines' Import page and list action, with their tests.
6. The shell command and its tests.
7. The build, the suites, the local checks, and the live round trip.

The environment was recreated and provisioned with compass db recreate -y -k and compass smoke ticks --leave-running, which left 24 services running, the acme_corporation tenant provisioned and 44214 ticks on 10 routes.

Committed on feature/import-a-pricing-engine-configuration-as-ore-xml:

• ./compass.sh build (bare) passes, and passes again after every commit that followed it.
• The ORE core suite passes (416 cases, 475709 assertions) and holds the 10 import cases; the shell reporting suite passes (577 cases); the analytics suites pass; the BFF suite passes (1100 tests in 81 files); the web suite passes (2232 tests in 179 files).
• The live round trip, on a database recreated with compass db recreate -y -k and provisioned with compass smoke ticks --leave-running (24 services, 44214 ticks on 10 routes):
  • The UK party's XVA Overnight exported to tmp/live/xva-overnight.xml: 8 products, 19 settings, one configuration setting, well formed under xmllint --noout.
  • Previewed through the BFF against the configuration it came from: 0 added, 0 changed, 0 removed, 27 held, no issues — the export and the importer agree.
  • Previewed against Pre-Trade Quick Price: 17 added, 1 changed, 0 removed, 7 held, the changed row stating "Model DiscountedCashflows → CrossAssetModel; Engine DiscountingSwapEngine → AMC".
  • A currency document offered to the preview: success: false, the root element CurrencyConfig named, and the sentence "This file is not a pricing engine configuration: its root element is CurrencyConfig. Choose a PricingEngines file."
  • The configurations in the table around a preview: 26 before, 26 after.
  • The shell import into a new configuration: 27 added, and the configuration 01a17fd0-9cf9-77e2-9aec-e4c5311dd467 exported again is the same document, byte for byte.
  • The API import into a new configuration: 27 added, the configuration 01a17fd2-6bc3-7310-9d13-b33fcf99f76e, and its ore-document read back with content-type: application/xml and content-disposition: attachment; filename"pricingengine.xml"= is the same document, byte for byte.
  • The reason and commentary the person states reach the rows: the configurations made by the shell and the API carry system.external_data_import, and the one made in the browser carries the reason the review started on.
  • Playwright drove the five steps on the running environment (tmp/live/drive_import.mjs, screenshots in tmp/live/shots/): the counts and the product list, "everything in the file is known", the target step, the review, the reason and commentary last, and the press. A second run with a file holding one product against the seeded XVA Overnight drew the two removed rows first, each with four choice buttons of which the two Keep ours were checked, and the sentence "Writes over XVA Overnight, in place. Every row the file omits is kept unless you say otherwise, and the history records the change."
• The live round trip is what found the one defect the unit tests could not: the ordering of the settings within a scope. It is fixed, and the round-trip case now carries three settings of one scope in an order no sort would produce.

The five things the review of #3999 found are taken on fix/import-pricing-engines-review (see * Review). What that branch proved again, on a database recreated with compass db recreate -y -k and provisioned with compass smoke ticks --leave-running:

• The UK party's XVA Overnight exported as 8 products and 19 settings.
• Imported through the shell into a configuration that exists — the case the review's third finding was about — 13 rows were added and the export of that configuration is the file's document, byte for byte. No product and no parameter row in the database points at a configuration that does not exist, which is what the finding would have caused.
• Imported through the API into a new configuration: 27 added, and the export is the file's document, byte for byte.
• A file whose product states no model is refused by the save, the refusal names the row by its key (Swap#0), and no configuration with that name is left behind — the rollback, live.
• A body of four megabytes and one byte is refused over HTTP with "An ORE pricing engines document may be at most 4194304 bytes."
• The suites: ORE core 444 cases (478894 assertions), ORE service 6 cases (the first this target has held), BFF 1139 tests in 83 files, and ./compass.sh build passes bare.

python3 build/scripts/local_checks.py --exclude-heavy runs 41 checks and passes

1. The eight that fail carry main's inputs, not this branch's:

format, bff-list-requests and unity-collisions were red when this task started and are green now: the first two because of this branch's own files, the third because the regeneration of the equity underlyings repaired main's collision.