Story: Migrate remaining ~100 entity models from JSON to org

Table of Contents

This page documents a story in Sprint 25. It captures the goal, current status, acceptance criteria, and the tasks that compose it.

1. Goal

Complete the org-migration story: convert the remaining legacy JSON-format entity models to the literate .org entity-model format.

2. Status

Field Value
State ABANDONED
Parent sprint Sprint 25
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-09-23

3. Acceptance

4. Tasks

Task State Start End Description
Prune orphaned deployed skills and make the skills build delete removed sources ABANDONED 2026-09-18 2026-09-23 Task for: Migrate remaining ~100 entity models from JSON to org
Retire the JSON-era residue the org migration left behind ABANDONED 2026-09-18 2026-09-23 Task for: Migrate remaining ~100 entity models from JSON to org
Decommission the legacy ores.codegen bash scripts DONE 2026-09-18 2026-09-18 Task for: Migrate remaining ~100 entity models from JSON to org
Add the dataset archetypes the four-dataset migration needs DONE 2026-09-18 2026-09-23 Image artefacts, the ip2country artefact plus TSV import, and FPML per-entity refdata, so crypto/flags/ip2country/FPML can move onto the org dataset pipeline.

5. Decisions

  • The ores.sql.populate archetypes are the single dataset-populate pipeline. Crypto, flags, ip2country and FPML move onto it; their bespoke generators are deleted once the new output is byte-identical. The archetypes take every dataset name from the dataset model itself, so no dataset needs a hardcoded template.
  • Slovaris is abandoned, not migrated. It was an experiment that did not work, and it was the only reason several archetypes carried hardcoded names. Its sprint 22 story and task stay in the discarded bucket, marked ABANDONED, so the id-link from sprint_19/sprint.org still resolves.
  • ores.dq must not be regenerated until its two hand-written callers are updated. The current cpp_nats_handler template no longer emits the unpaginated dataset_bundle_members list operation, but registrar.cpp and DataLibrarianWindow.cpp still use it. Five generated ores.dq artefacts therefore still carry slovaris doc comments until that pass happens.

6. Out of scope

7. Reframing — current context (2026-09-18)

The title and the goal above record the state when the capture was written. The migration is complete. This section records what is true today, and it reframes the story.

7.1. The count of "~100 models" is stale

  • Zero JSON entity models remain. The modeling/ directories hold 0 .json files and 450 .org files.
  • projects/ores.codegen/models/ does not exist. The last deletion was 3f03b1c433 on 2026-06-02.
  • Every codegen_org_model_migration task is DONE. Two were abandoned on purpose: the profiles.json conversion to a literate facet_catalogue.org, and the slovaris reference data.
  • 239 org models are live today: 174 entity, 34 component, 13 junction, 11 module, 7 field_group.

The JSON that remains is dataset payload. The seeder datasets and the test corpora are live inputs to the codegen and to the tests. They are not models.

7.2. The migration left residue behind

The migration deleted the models. Nothing deleted the artefacts that point at them. That is one defect class: a rename or a delete with no prune step.

  1. Four one-shot migration scripts still ship, and each one points at the deleted tree: migrate_component_to_org.py, migrate_field_group_to_org.py, migrate_lookup_entity_to_org.py, and migrate_junction_to_org.py.
  2. Two skills tell the reader to author JSON models under projects/ores.codegen/models/{component}/. The directory does not exist, so the reader cannot follow the instruction: compass-codegen-add-entity and compass-codegen-add-sql-schema.
  3. Five template org sources carry text from the generator.py era. All 136 .mustache templates are tangled from an org source, so the fix belongs in the org file and never in the tangling output. Three generated SQL headers name generator.py and models/slovaris/model.json. One field-group header documents a _field_group.json filename convention. One facet summary attributes the dataset pipeline to generator.py.
  4. projects/ores.compass/src/env_init.py and three modeling docs reference the deleted tree.
  5. The skills deployment kept output whose source was gone. 80 orphaned directories survived the compass- namespace rename, because org-publish only writes. The prune task fixed the mechanism.

7.3. Reframed goal

Retire the residue that the JSON-to-org migration left behind. Delete what nothing calls. Correct what still misleads a reader.

8. Result

Abandoned at sprint 25 close. Sprint 26 replaces the codegen approaches tried so far with one method: for each module, sync codegen and clear its drift. Sprint 26 opens new, clean stories for each module. This story is not carried forward.

Emacs 29.3 (Org mode 9.6.15)