Task: Make the parameter data model the source of valid values and rules
Table of Contents
This page documents a task in the Make every configuration screen follow one standard story. It captures the goal, current status, acceptance, and any notes or results.
1. Goal
One parameter model states a parameter's control and its rule, and every screen
reads it from there. A parameter's allowed values, its bounds, its pattern and
its catalogue live in the value set tables and nowhere else. The per-pair
kind and options columns go, because the value set now says the same thing
once instead of on every pair that takes the setting. The engine run settings
reach the same contract, so the Setup parameters stop carrying a second copy
of their control in the group settings table. The screens keep one source, the
write path keeps one enforcement point, and a build check fails when a
parameter loses its set.
2. Status
| Field | Value |
|---|---|
| State | DONE |
| Parent story | Make every configuration screen follow one standard |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-10-30 |
3. Acceptance
- The value set tables are the only place that states a parameter's control and its rule. No screen, server or seed restates an allowed value.
- The 15 settings whose set said text although the guide lists their choices state an enumerated or boolean set, from a curation entry that names the guide as its source. The settings that stay text say in the set why they are text.
ores_analytics_pricing_pair_settings_tblholds nokindand nooptionscolumn. The C++, the protocol, the SQL, the TypeScript, the shell recipe and the seed are generated from the model without them.usual_valuestays, because the pricing screen shows the usual setting of a pair, and it states a value and not a rule.- Every
Setupparameter that the engine run settings screen draws has a value set and a link, and that screen draws its control and checks its value from the set through the same fields the pricing screen reads. - A check in
build/scripts/local_checks.pyfails when a parameter a screen draws has no value set, when a set breaks its own rule, or when a per-pair override column returns. - The build, the four test suites, the drift checks and a live environment prove the behaviour: a change to a set changes the screen's choices, a refused value names its set, and the ORE export and import still roundtrip.
4. Plan
4.1. The inventory, taken before any change
The inventory was taken at commit 92fd6990bc85. Counts came from the
curation and the seed files, which are deterministic, and from the restored
database, whose schema is a4924adbc941bfc7 and is therefore a cross-check
only; the seed row count is stated beside each database count.
4.1.1. The two tables
- The per-pair settings table
ores_analytics_pricing_pair_settings_tbl(model ores.analytics.pricing_pair_setting, create analytics_pricing_pair_settings_create.sql) holds one row per (type, model, engine, scope, name): 834 rows and 126 distinct(scope, name)settings. The columns that state a value rule are:column holds kindhow the screen draws the value: select,number,tenor,referenceortextoptionsthe allowed values of a select, as one comma separated stringusual_valuethe value ORE's examples use is_requiredwhether the screens start a new product with the setting The kind counts are
number332,select291,reference140,text67 andtenor4. 291 of the 834 rows carry a non-emptyoptions, which repeats the same list on every pair that takes the setting. - The value set model
The newer source is four reporting tables, read through the view
ores_reporting_ore_parameter_settings_v(create reporting_ore_parameter_settings_view_create.sql):table holds rows ores_reporting_ore_parameter_value_sets_tblthe set: its code, source_kind,catalogue_code,minimum,maximum,pattern218 ores_reporting_ore_parameter_value_set_values_tblthe values of an enumerated set, with a label and an order 202 ores_reporting_ore_parameter_value_set_catalogues_tblthe relations a catalogue set may read 5 ores_reporting_ore_parameter_setting_value_sets_tblthe link from (scope, name)to its set712 The source kinds are
text72,number63,integer42,enumerated29,catalogue7,boolean3 andregex2. The one enforcement point is a trigger on the analytics parameter table, analytics_pricing_model_product_parameters_value_set_trigger_create.sql, which callsores_reporting_validate_parameter_value_fn. It reads the set by(scope, name)and reads no per-pair column.
4.1.2. The consumers
- The BFF vocabulary.
vocabulary()in pricing-engines.ts reads the pair settings and serves, for each setting,kindfrom the row andoptionsfrom the row's comma separated string, beside the value set fieldsvalueSet,sourceKind,choices,minimum,maximumandpattern, which it also serves as thevalueSetsmap keyed(scope, name). The reader is ore-value-sets.ts, which registers 8 catalogue relations. - The web pricing screens.
pricingEngines.ts
parses
kindandoptionsinto the vocabulary type. parts.tsx readssetting.kindto decide one thing: when it isreferenceit draws the sensitivity template's asset class and method. The control itself already comes fromcontrolOfValueSet()in control.ts, which reads the value set alone. - The analytics parameter rows.
ores_analytics_pricing_model_product_parameters_tblholds the values a product carries and is guarded by the trigger above. - The seeder curation and its loaders.
projects/ores.seeder/datasets/ore_pricing_engines/curation/holdsvalue-sets.json(the guide's sets),value-sets-code.json(the sets the type extractor writes),value-sets-review.json(a person's settlements),engine-settings-groups.json(theSetupgroups) andsettings-code.json.curation/__init__.pyloads them and refuses an entry without a reason, a source, a source reference, a curator or a date. - The generators.
generate_vocabulary_sql.pywritesanalytics_pricing_pair_settings_populate.sql, includingkindandoptions.projects/ores.sql/utility/generate_value_sets.pywritesreporting_ore_parameter_value_sets_populate.sqland fails when a contract parameter has no set.generate_engine_settings_groups.pywrites the twoSetuptables fromengine-settings-groups.json.pricing_parameter_rows.pywrites the ore parameter contract, and renders the guide's options into theformatprose asOne of: A, B, C. - The codegen. The analytics entity model renders the pair setting into C++,
SQL, the wire protocol, the TypeScript domain and protocol, the shell
commands and the shell recipe document. The model's
Table display,ColumnsandDetail fieldsstatekindandoptions, so every one of those carries them. - The shell.
pricing_pair_settings listprints the row,kindincluded.ore_parameter_settings list|show|checkalready reads the sets and the links, and is the live gate.
4.1.3. Where kind and options are read today, and what depends on them
| reader | reads | depends on it for |
|---|---|---|
BFF vocabulary() |
kind, options |
the reply's kind and options fields |
web parts.tsx SettingValue |
kind |
the sensitivity template heading when the kind is reference |
web api/pricingEngines.ts |
kind, options |
parsing the reply |
| generated C++, protocol, TypeScript, shell, recipe | kind, options |
the row's fields |
generate_vocabulary_sql.py |
the guide's kind, options |
the seed's two columns |
No validator reads them. The trigger reads the sets. The ORE XML export and
import in projects/ores.ore read neither: the options in that component are
instrument options and have nothing to do with this table, and the import writes
values through the save that the trigger guards. is_required is read by the
screen and is not a value rule, so it stays.
4.1.4. Which parameters still have no value set, and why
- The pricing, convention, curve and global parameters have no gap: 251
parameters (61
ModelParameters, 173EngineParameters, 17GlobalParameters) carry 1 link each, andgenerate_value_sets.pyfails on the first one that does not. The links also cover 461 convention and curve leaves, which is why 712 links stand against 251 pricing parameters. - 72 sets are
text. That is the deliberate statement that a parameter is free text, and it is data, not a gap. It is also under-stated for 15 of them: the guide lists their choices and the code typed them as text, and a code set replaces a guide set. They areCurve,FepCurve,OptionExpiryPosition,Policy,RegressorModel,SensitivityDecomposition,SeparateCallPutVols,SpreadOnIncomeCurve,useIndexCDSPricer,Calibration,CalibrationStrategy,FxCalibration,ReversionType,TreatSecuritySpreadAsCreditSpreadandVolatilityType. - The
Setupscope is the real gap: the engine run settings screen draws 15 settings and none of them has a set or a link. Their control lives inores_reporting_engine_settings_group_settings_tblaskind,options,minimumandmaximum, so the second screen restates what a set would say. - The
Setupblock holds 54 more parameters from the ORE input files, such asoutputPath,csvSeparatorandbaselTrafficLightConfig. No screen draws them and the contract states their domains, so a set would be a rule nothing reads. Three of them do name a catalogue vocabulary (dateCalendar,dateConvention,baseCurrency) and are a candidate for a later task.
4.1.5. What this task changes
- Add the setup sets:
generate_value_sets.pyreads the engine settings groups' curation and writes one set per setting (flagis a boolean set, awholewith bounds is a numeric set withminimumandmaximum, achoiceis an enumerated set), linked at scopeSetupby the ORE name. The numeric kinds gain bound enforcement in the same function the parameter trigger uses, because awholesetting states a floor today and a set must be able to. - Convert the 12 settings the evidence supports to an enumerated or boolean
set through a reviewed curation entry.
Calibrationstays text because the stored population holdsDeal, which the guide's list omits, andSeparateCallPutVolsstays text because ORE also accepts a regex pattern there; both sets say so in their description. - Read the set on the engine settings screen: the BFF serves the
Setupvalue sets, checks the values a person writes against them, and the screen draws its control fromcontrolOfValueSet()and checks the text from the set. The duplicated columns leave the group settings model. - Retire
kindandoptionsfrom the pair setting model, regenerate every address, and delete the seed, BFF and web code that reads them. The web reads the catalogue of the set instead of thereferencekind. - Add
build/scripts/check_parameter_value_sets.pyand register it inlocal_checks.py.
5. Measurements
Every number this task claims about a tree, with the command that produced it
and the commit it was measured at. A number without both is not a measurement,
because a clean-up record describes a tree and trunk moves under a branch: a
consumer count, a gate's component count and a suite's size all change with no
code changing, and the reader of a bare number cannot tell which tree it came
from. Where the raw output matters, put it in an evidence/ file beside the
task and name that file in the Command column.
| What | Value | Command | Commit |
|---|---|---|---|
| Pair settings in the seed | 834 rows, 126 distinct settings | grep -c "^ ('" projects/ores.sql/populate/analytics/analytics_pricing_pair_settings_populate.sql |
92fd6990bc85 |
| Pair settings with options | 291 rows | select count(*) filter (where options <> '') from ores_analytics_pricing_pair_settings_tbl |
database a4924adbc941bfc7 |
| Kinds in the pair settings | number 332, select 291, reference 140, text 67, tenor 4 | select kind, count(*) from ores_analytics_pricing_pair_settings_tbl group by kind |
database a4924adbc941bfc7 |
| Value sets and links | 218 sets, 712 links | python3 projects/ores.sql/utility/generate_value_sets.py --check |
92fd6990bc85 |
| Pricing contract parameters | 251 (61 model, 173 engine, 17 global), 0 without a set | generate_value_sets.contract_pairs() and resolve() |
92fd6990bc85 |
| Setup parameters | 15 engine settings, 0 links | curation/engine-settings-groups.json and the link rows |
92fd6990bc85 |
| Setup block parameters with no screen | 54 | block_parameter_rows.rows() filtered to Setup |
92fd6990bc85 |
| Sets and links after the change | 224 sets, 755 links | python3 projects/ores.sql/utility/generate_value_sets.py --check |
a1aa7f6a00 |
| Drawn parameters the gate checks | 266 | python3 build/scripts/check_parameter_value_sets.py |
a1aa7f6a00 |
| Settings the guide lists and the code typed as text | 15, of which 13 become a set and 2 stay text | compare pricing_vocabulary.json options with value-sets-code.json |
92fd6990bc85 |
| The build | 325 targets, exit 0 | ./compass.sh build |
7676e1e237 |
| The local gate | 54 of 54 checks passed | python3 build/scripts/local_checks.py --exclude-heavy |
f529c48e72 |
| The C++ suites | reporting core 970, reporting service 26, analytics core 882, ore core 478353, ore service 49, refdata core 5723, shell reporting 1642, shell analytics 1368, shell refdata 7980 assertions: all pass | the suite binaries under build/output/linux-clang-release-ninja/publish/bin |
f529c48e72 |
| The pgTAP suite | 65 files, 623 tests, all pass | projects/ores.sql/test/run_tests.sh |
ea97288e0d |
| The BFF suite | 1144 tests passed | npx vitest run in projects/ores.web/packages/bff |
7676e1e237 |
| The web suite | 2328 tests passed | npx vitest run in projects/ores.web/packages/web |
7676e1e237 |
6. Notes
6.1. What changed, file by file
| Area | Change |
|---|---|
curation/engine-settings-groups.json |
States six Setup sets with provenance, and every group setting names its set instead of a kind, words and a range. |
curation/value-sets-guide-enums.json |
New. States the guide's choices for the 13 settings the code could only read as text, and says in the set why two stay text. |
curation/__init__.py |
Loads the engine settings groups and refuses a setting that states a control beside its set, or names a set the file does not state. |
ores.sql/utility/setup_value_sets.py |
New. The Setup sets and links, read from the curation. |
ores.sql/utility/generate_value_sets.py |
Adds the Setup sets after the curve ones, applies a reviewed file only when it carries its provenance, and gains the guide-enums file. |
ores.sql/utility/generate_engine_settings_groups.py |
Writes only what the group settings table still states. |
ores.sql/create/reporting/reporting_ore_parameter_value_validate_fn_create.sql |
Adds ores_reporting_number_in_bounds_fn so a numeric set's floor and ceiling are checked, and uses it for a range. |
ores.reporting/modeling/ores.reporting.engine_settings_group_setting.org |
Loses kind, options, minimum and maximum. |
ores.analytics/modeling/ores.analytics.pricing_pair_setting.org |
Loses kind and options. |
bff/src/engine-settings.ts |
Serves the Setup value sets beside each setting and checks a written value against its set. |
bff/src/ore-value-sets.ts |
Serves the set's catalogue as well as its kind, values and bounds. |
bff/src/pricing-engines.ts |
Stops serving the pair setting's kind and options. |
web/src/parameters/control.ts |
One control mapping for both screens; the engine settings kind mapping is deleted. |
web/src/engineSettings/model.ts |
Reads the value, its bounds and its words from the set. |
web/src/pricingEngines/parts.tsx |
Reads the sensitivity template heading from the set's catalogue, not a reference kind. |
build/scripts/check_parameter_value_sets.py |
New. The gate. |
seeder/generate_vocabulary_sql.py |
Stops writing the two columns, and gains --from-json so a seed change needs no ORE checkout. |
sql/utility/pricing_parameter_rows.py |
The contract's format prose stops listing the choices; the set states them. |
reporting/service/engine_settings_proposal.cpp |
A proposal asks the store's own value set check, and keeps the whole-number width check the apply makes. |
6.2. Where the value set model still cannot state what a per-pair column stated
- A kind that is also a flag.
SeparateCallPutVolsacceptstrue,falseor a regular expression inclusion pattern. A set is one kind, so it cannot say "a switch or a pattern"; it staystextand its description says why. The per-pairselectcolumn could not state it either, and refused a pattern the guide allows, so nothing is lost here. - A vocabulary the evidence states in part. The guide lists
Bootstrap,BestFitandNoneforCalibration, and the documents ORE ships writeDeal. The set staystextwith the reason. Making it enumerated would refuse a value the seeded population holds, so the task needs the ORE enumeration read from the source before it narrows. - A per-pair difference. The per-pair columns could, in principle, state one
list for one pair. They never did: every row of a setting carried the same
list. The set is keyed by
(scope, name)and cannot state a difference by pair. If ORE ever needs one, the link becomes keyed by the pair too; that is a model change and no screen needs it today. - A default per pair.
usual_valuestates the value ORE's examples use and is a value, not a rule, so it stays on the pair row. It is not in the set, and a set cannot state it: two pairs may use different example values for one setting. - Documentation prose. The
formattext of aSetupore parameter still says "A whole number, at least 1." The set is the rule and the prose is documentation a person reads; the pricing contract's prose no longer repeats the choices, and theSetupprose could follow in the documentation task.
6.3. Decisions
- The
Setupsets share a code where the rule is shared. Eleven flags takesetup.flag, and the two fixing lags takesetup.fixing_lag_days, because a set states a rule and the rule is the same. The alternative, one set per setting, would repeat the same rule fifteen times. - The numeric bounds moved into the shared check. A
wholesetting stated a floor in the group settings row. The value set states it now, and the function the parameter trigger calls applies it, so a bound is data and one function reads it. - The proposal service asks the database rather than re-reading the tables.
The engine run settings proposal path had its own copy of the rule. It asks
ores_reporting_value_in_set_fnnow, so a proposal is refused with the sentence the store would use, and the whole-number width check the apply makes stays with it. - The check reads the curation and the seed, not a live database. The gate must run on a pull request, where no environment exists. A stale seed is caught by comparing the curation with the committed script.
- The
Setupblock's 54 input-file parameters get no set. No screen draws them and the contract states their domains.dateCalendar,dateConventionandbaseCurrencydo name a catalogue vocabulary and are a candidate for a later task; they are recorded here rather than decided silently.
6.4. The live proof, as a person repeats it
compass db recreate -y -k, then the ACME provisioning script.npx tsx tmp/prove-value-sets.tsinprojects/ores.web, signed in astenant_admin@acme_corporation(oradrian.vance@acme_corporationfor the two vocabularies alone). It prints each setting's set, kind, choices and bounds, and exports and previews one pricing engines document.- Add a value to
setup.observation_modeland raise the floor ofsetup.n_threadsin the database, then run it again: the choices and the floor follow the set. npx tsx tmp/prove-write.tsshows the group write refused below the floor.update ores_analytics_pricing_model_product_parameters_tbl ...with a value outsidecalibrationstrategyis refused by the trigger, naming the value, the setting and the set.
The two scripts are scratch, beside the task rather than in the tree: they sign in to a live environment and are not part of a test suite.
7. Test Scenarios
Manual QA scenarios (scaffolded via compass add test_scenario)
that verify this task. Link
new ones here as they're created; the scenario doc itself links back
via its "Verifies task" field.
| Scenario | State | Notes |
|---|---|---|
8. PRs
9. Review
| Comment summary | File | Decision | Notes |
|---|---|---|---|
10. Result
The parameter data model is the one source of a parameter's control and rule, on both configuration screens, and a check holds it.
- The value set tables state every parameter's allowed values. 266 parameters a screen draws carry 1 link each, and the 224 sets state an enumerated list, a catalogue, a range, a regular expression, a switch or text.
- The
Setupscope joined the contract. Six sets state the fifteen engine run settings' controls, the group settings table lost its copy, and the screen draws and checks from the set through the same fields the pricing screen uses. - The per-pair
kindandoptionscolumns are gone from the model, the SQL, the C++, the protocol, the TypeScript, the shell recipe and the seed. The compiler found the last reader, a hand-written proposal check, and it asks the store's own function now. - Fifteen settings the guide states a vocabulary for stopped being free text: thirteen carry the guide's list, and two stay text with the reason in the set.
build/scripts/check_parameter_value_sets.pyruns in the local gate. It fails on a parameter a screen draws with no set, on a set whose own kind cannot hold its rule (a floor above its ceiling, an empty or duplicated list, an unregistered catalogue, a patternless expression), on a bound or a list written as a column check on an engine settings part, and on a read of the retired per-pair column. Each was proved by breaking it and reading the failure. The seeded script is compared with its sources exactly bygenerate_value_sets.py --check, which owns that test.- The engine run settings' rule lives only in the set: the part tables lost their column checks and carry a guard that asks the verdict function, and the ORE run document codec lost its own list of observation models.
ore_parameter_settings checkreports the 266 parameters the screens draw, the engine run settings among them, and answers OK when every one names a set of a kind its domain takes. It refuses to judge a database whose ORE parameter contract is not published, and says so.
Proven live on a recreated, provisioned environment: adding a value to
setup.observation_model in the database added it to the screen's choices, and
raising the floor of setup.n_threads moved the bound the server enforced, with
no server change. A group write below the floor was refused. A pricing engines
document exported and previewed with 0 rows added, changed or removed and 0
issues. A parameter write outside its set was refused by the trigger, naming the
value, the setting and the set. The retired columns appear in no reply.
Not done here, and stated rather than hidden: the Setup block's 54 input-file
parameters have no set because no screen draws them; three of them name a
catalogue vocabulary and want a later task. Calibration stays text until the
ORE enumeration can be read from the source, because the guide's list omits a
value the seeded documents write. The pending provenance columns on the row are
the story's own task, not this one.
Commits on the branch, in order: the inventory, the Setup sets, the guide's
choices, the engine settings screen reading the set, the retired per-pair
columns, the gate, and the proposal check.