Service Architecture Migration

Table of Contents

Overview

This document describes the plan to migrate ORE Studio's service architecture to a clean three-layer model with clear separation of concerns and testability at every layer.

Current state

Services are structured as two components:

ores.iam            ← domain + repository + service + messaging (mixed)
ores.iam.service    ← executable: wires everything together

Clients (ores.qt, ores.shell, ores.cli) link against ores.iam to access protocol types, dragging in database, repository, and service logic they never use. Protocol types have no dedicated tests.

The ores.http.server binary mixes server infrastructure with route handlers for IAM, refdata, assets, variability — tightly coupling the HTTP server to every service domain.

Target architecture

Each service domain becomes three distinct components:

ores.iam.api     ← shared contract: domain types (POCOs + IO) and
                      messaging protocol (request/response structs,
                      nats_subject constants); no DB, no handlers
ores.iam.core      ← implementation: handlers, service logic, repository,
                      DB access; depends on ores.iam.api
ores.iam.service   ← executable; links both, wires NATS and DB

The separation is driven by a key invariant: ores.iam.api contains everything that appears on the wire — domain types that are serialised as request/response fields belong here, not just protocol envelopes. Repository and handler code depends on domain types but is private implementation detail; it lives in ores.iam.core.

Clients link only ores.iam.api. The dependency graph is strict and expresses intent:

ores.qt   ──→ ores.iam.api
ores.qt   ──→ ores.compute.api
...

ores.iam.core    ──→ ores.iam.api   (implements the protocol)
ores.iam.service ──→ ores.iam.core   (runs the service)

The ores.http.server binary becomes a thin orchestrator:

ores.http.routes     ← all HTTP route registrations (new component)
ores.http.server     ← server infrastructure only: main, application,
                        config, parser, NATS setup; links ores.http.routes

Scope

Ten service libraries require splitting:

Library Protocol headers Route file in http.server?
ores.iam 23 yes (iam_routes)
ores.refdata 47 yes (risk_routes)
ores.compute 19 no
ores.dq 22 no
ores.assets 3 yes (assets_routes)
ores.variability 3 yes (variability_routes)
ores.scheduler 3 no
ores.reporting 9 no
ores.trading 3 no
ores.synthetic 3 no

ores.http already has messaging/http_info_protocol.hpp (created in the service discovery feature) and is a working example of the target *.api pattern.

What moves where

Types components (ores.{service}.api)

Move from ores.{service}/include/ores.{service}/:

  • All domain/*.hpp POCOs → new component's include/ores.{service}.api/domain/
  • domain/*_io.hpp (JSON + table serialisation helpers) → new component
  • messaging/*_protocol.hpp files → new component's include/ores.{service}.api/messaging/
  • nats_subject constants → new component

Stay in ores.{service}.core:

  • messaging/*_handler.hpp/.cpp files (implement the protocol using repo/service)
  • messaging/registrar.hpp/.cpp (registers handlers; called by the service executable)
  • repository/ (database access, SQL mappers)
  • service/ (business logic)

The new types component has:

  • include/ores.{service}.api/ — domain types + protocol headers
  • tests/ — unit tests for serialisation round-trips
  • CMakeLists.txt — INTERFACE target if header-only (no .cpp files for IO helpers that are pure templates); static lib otherwise

HTTP routes (ores.http.routes)

Move from ores.http.server/src/routes/ and ores.http.server/include/ores.http.server/routes/:

All *_routes.{hpp,cpp} files → new ores.http.routes component.

ores.http.routes depends on: ores.http.lib, ores.iam.core, ores.refdata.core, ores.assets.core, ores.variability.core (the full core libs — handlers are implemented there).

ores.http.server drops the route dependencies and links ores.http.routes instead. The server itself retains only:

  • src/app/application.{hpp,cpp} — server setup, NATS init
  • src/app/host.{hpp,cpp}
  • src/config/ — options, parser
  • src/messaging/ — HTTP info handler and registrar
  • src/main.cpp

Migration phases

Phase 1 — Split ores.iam and ores.compute (highest value)

These are the largest and most-used components.

Steps:

  1. Create projects/ores.iam.api/ scaffold via codegen (component profile then hand-adjust: no domain stub needed).
  2. Move all domain POCOs and IO helpers from ores.iam into projects/ores.iam.api/include/ores.iam.api/domain/.
  3. Move all ores.iam/include/ores.iam/messaging/*_protocol.hpp into projects/ores.iam.api/include/ores.iam.api/messaging/.
  4. Rename ores.iam project to ores.iam.core; update #include "ores.iam/..."#include "ores.iam.core/..." throughout, and update all dependent CMakeLists.
  5. Update ores.iam.core to #include "ores.iam.api/..." and add ores.iam.api as a PUBLIC dependency.
  6. Update clients (ores.qt, ores.shell, ores.cli) to link ores.iam.api instead of ores.iam (now ores.iam.core).
  7. Write serialisation round-trip tests in ores.iam.api/tests/: for each protocol type, serialize and deserialise via rfl::json::write / rfl::json::read and assert field equality.
  8. Repeat for ores.compute.api.

PR title: [arch] Extract ores.iam.api and ores.compute.api

Phase 2 — Remaining service splits

Repeat Phase 1 steps for:

  • ores.refdata.api / ores.refdata.core (47 headers — largest)
  • ores.dq.api / ores.dq.core
  • ores.reporting.api / ores.reporting.core
  • ores.scheduler.api / ores.scheduler.core
  • ores.assets.api / ores.assets.core
  • ores.variability.api / ores.variability.core
  • ores.trading.api / ores.trading.core
  • ores.synthetic.api / ores.synthetic.core

Do in batches of 2-3 to keep PRs reviewable.

PR title per batch: [arch] Extract {service} types and core components

Phase 3 — ores.http.routes extraction

Steps:

  1. Create projects/ores.http.routes/ component scaffold.
  2. Move all *_routes.{hpp,cpp} from ores.http.server into the new component under the same internal structure.
  3. ores.http.routes/CMakeLists.txt links PUBLIC to ores.http.lib and PRIVATE to the core libs it delegates to (ores.iam.core, ores.refdata.core, etc.).
  4. ores.http.server/CMakeLists.txt drops those core lib deps and links ores.http.routes instead.
  5. Build and verify end-to-end.

PR title: [arch] Extract HTTP routes into ores.http.routes

Phase 4 — ores.http.server cleanup

After Phase 3, ores.http.server links only:

ores.http.lib
ores.http.routes
ores.nats.lib
ores.service.lib
ores.telemetry.lib
ores.utility.lib
ores.database.lib   (for DB setup in application.cpp)

At this point ores.http.server is a pure infrastructure component: server lifecycle, config parsing, NATS service discovery, signal handling. No domain knowledge.

PR title: [arch] Clean up ores.http.server dependencies

Naming conventions

Pattern Contains
ores.{service}.api domain POCOs, IO helpers, *_protocol.hpp, nats_subject constants
ores.{service}.core handlers, service logic, repository, DB access
ores.{service}.service executable: main, app, config
ores.http.routes HTTP route handlers for all services
ores.http.server HTTP server infrastructure only

Key invariant

Clients (ores.qt, ores.shell, ores.cli) MUST NOT link any *.core or service implementation library. They link only *.api components and infrastructure libs (ores.nats, ores.eventing, etc.).

This is enforced structurally: the *.api component has no database, no repository, no service dependencies — so linking it is safe for any client. The *.core component is implementation detail shared only between the service executable and ores.http.routes.

Testing approach

Each ores.{service}.api component includes its own Catch2 test binary. Tests cover:

  1. Round-trip serialisation: rfl::json::write then rfl::json::read produces equal struct.
  2. Subject names: assert nats_subject constants match expected strings (guards against accidental renames breaking the wire protocol).
  3. Required fields: confirm that missing required fields produce a deserialisation error rather than silent defaults.

No database, no NATS connection required — tests run entirely in-process.

Open questions

  • Should ores.{service}.api be a shared library (.lib) or a CMake INTERFACE target? Answer: use INTERFACE if no .cpp files are needed (pure header templates); add a .lib target if IO helpers have non-trivial implementation.
  • HTTP routes for ores.compute (file upload/download) are infrastructure-only and do not use service domain types. They can stay in ores.http.routes or remain in ores.http.server depending on team preference.
  • The ores.cli client also uses repositories directly for bulk operations (import/export). In Phase 1, ores.cli continues to link ores.iam.core for these operations; the repository is not separately exposed.