ores.codegen

Table of Contents

1. Diagram

ores.codegen component diagram

Figure 1: ores.codegen

2. Summary

ores.codegen is the project's Python-based code generator. It takes JSON models and Mustache templates and produces output files — primarily SQL for the data-quality system, but also C++ scaffolding for domain types and, since the v2 information architecture, org-mode documents for the task / story / sprint hierarchy. The generator is self-contained: a virtualenv, a small set of Python entry points, and a flat library of templates. Generated files carry standardised licence headers and editor modelines.

3. Inputs

  • JSON model files under models/<subject>/.
  • Mustache templates under library/templates/ — generated artefacts, tangled from the literate facet docs that live alongside them; the Codegen template library overview tops the hierarchy (template ← facet doc ← group doc ← groups overview).
  • Static data (licences, modelines) under library/data/.
  • Optional generator profiles (collections of templates) declared in each facet's Archetypes table across the ores physical-space graph.

4. Outputs

  • SQL files for data-quality schemas and populates (default).
  • C++ scaffolding for domain types (via profile all-cpp).
  • Information-architecture org documents — task / story / sprint / version / component — via compass add.

By default, output lands in output/ alongside the wrapper script; each entry point accepts a custom output directory.

5. Facet catalogue

C++ facets are declared as nodes in the ores physical-space graph and invoked with --address <address>. Each facet is a named collection of Mustache templates; every template's literate source — prose on role, model inputs and outputs, plus the tangle block — lives in its facet doc under the template library. The type mappings that apply across all facets are in Entity lifecycle§"Type mappings".

Address Covers Key library
--address ores.cpp.domain Domain class, JSON I/O, table I/O reflectcpp, libfort
--address ores.cpp.generator Test data generator faker-cxx
--address ores.cpp.repository Entity, mapper, repository sqlgen
--address ores.cpp.service Service layer —
--address ores.cpp.protocol NATS messaging protocol header —
--address ores.sql.schema SQL schema, notify triggers, drop scripts —
--address ores.cpp domain + generator + repository + service + protocol all above
--address ores ores.cpp + ores.sql.schema all above

5.1. domain

Generates the domain class (a plain struct), JSON I/O, and table I/O.

  • JSON I/O: uses reflectcpp (<rfl/json.hpp>, rfl::json::write()) with custom reflectors from ores.utility/rfl/reflectors.hpp to handle types not natively supported (boost::uuids::uuid, std::chrono::system_clock::time_point, etc.). Exposes operator<<(std::ostream&, const T&).
  • Table I/O: uses libfort (<fort.hpp>, fort::char_table with FT_BASIC_STYLE) for aligned column output. Type-formatting conventions: UUIDs → boost::uuids::to_string(); booleans → "Y"/"N"; timestamps → std::put_time() as "YYYY-MM-DD HH:MM:SS"; IP addresses → .to_string(). Exposes operator<<(std::ostream&, const std::vector<T>&).

5.2. generator

Generates a test data generator with two static methods: generate() → one random instance; generate_set(n) → n instances. Uses faker-cxx for realistic fake field values and ores.utility/uuid/uuid_v7_generator for UUID fields.

5.3. repository

Generates three file pairs: entity, mapper, and repository.

  • Entity: a flat struct with sqlgen annotations that maps to the database table. One domain class does not always map to one entity — consult the model when they diverge.
  • Mapper: converts entity↔domain via to_domain() and from_domain(). Handles type conversions: UUID strings ↔ boost::uuids::uuid, database timestamps ↔ std::chrono::system_clock::time_point.
  • Repository: uses sqlgen for database operations and ores.database/repository/bitemporal_operations.hpp for all read/write helpers. Write operations use upsert semantics — there is no separate create vs update at the repository layer. Read methods: read_latest(), read_all(), read_latest_since(time_point) (incremental loading), read_at_timepoint(timestamp).

5.4. service

Generates the service layer that wraps the repository. Method names follow the convention in Entity lifecycle§"Service method naming": save_*, remove_*, find_*, list_*, list_*_since, get_*_history.

5.5. protocol

Generates a single NATS messaging protocol header declaring the request/response types for the entity's pub-sub channels.

5.6. sql

Generates SQL table create, notify trigger, and drop scripts. Naming and idempotency conventions are in SQL entity schema patterns.

5.7. Component scaffold profiles

The CMake files run via --address ores.cmake.component and the C++ files via --address ores.cpp; the overview's #+component_kind: frontmatter (api=/=core=/=service=/=flat=/=composite=/=adapter) picks which one. They create a new component skeleton rather than entity code: the CMakeLists and source lists, the export macros, the test harness, and — only where the component opts into ores.cpp.scaffold — a master header, a stub header+impl and a stub test. The scaffold facet is #+default: disabled, so regenerating an existing component never recreates the placeholders it has outgrown. See Component architecture for the split model and CMake dep chain.

#+component_kind: Part Files
api ores.COMPONENT.api 8 always (CMakeLists ×4, src/ and tests/ component file lists, =export.hpp, test main) + 4 with the opt-in (header, stub.hpp, stub.cpp, stub_tests.cpp)
core ores.COMPONENT.core the same set, with core CMake deps
service ores.COMPONENT.service the core set plus the service application, and 2 with the opt-in (header and stub_tests.cpp)
flat (default) standalone (no split) the api set with generic CMake deps
composite composite root 1 (the root CMakeLists that adds the #+parts:)
adapter adapter part 2 (the src/ and tests/component_files.cmake file lists)

A composite holds parts and no code of its own, and an adapter is one of its parts with hand-authored build files. Both exist so that generation never writes over a hand-tuned CMakeLists. An undeclared kind selects flat, which serves neither case.

6. Test fixtures

Component-scaffold integration tests use fixture components under tests/component_scaffold/. Each fixture is a minimal component tree that exercises the scaffold profiles end-to-end.

Fixture Address exercised
ores.sample_composite --address ores.cmake.component (group-level composite: its root CMakeLists only)
ores.sample_composite.api --address ores.cpp (API subcomponent, plus the opt-in scaffold)
ores.sample_composite.core --address ores.cpp (core subcomponent, plus the opt-in scaffold)

7. Entry points

  • codegen.sh — the CLI wrapper (activates venv, calls codegen.py). generate takes a model path and --address (physical-space address); regenerate takes --component or --all.
  • src/codegen/cli.py — the CLI entry point.
  • src/doc_generate.py — the org-document generator.
  • src/fpml_parser.py — FPML Genericode XML → JSON converter.
  • src/images_generate_sql.py, src/lei_extract_subset.py — specialised data importers.
  • external/iso/iso_generate_metadata_sql.py — the ISO importer, kept beside the dataset it generates from.

8. Dependencies

  • Python 3 with pystache>=0.6.0 (in requirements.txt).
  • uuidgen on PATH (system uuid-runtime package on Debian/Ubuntu) for the document generator's ID minting.
  • No runtime dependency on other ores.* components — the generator is upstream of the C++ tree.

9. See also

Emacs 29.3 (Org mode 9.6.15)