How do I start the ORE Studio services?

Question
Answer
Script
Tested by
See also

Requires .env and built binaries (see How do I set up a development environment?). Every service is a concrete, per-environment systemd --user unit (one per (service, environment) pair, generated from the service_definition=/=service_dependency DB tables) aggregated under one per-environment target, ores-<env>.target – there is no separate orchestrator process; systemd itself owns dependency-ordered startup and readiness gating (Type=notify=/=sd_notify(READY=1)). ores.controller.service=/=process_supervisor, which used to spawn every other service as its own child, were decommissioned once this landed. Remote/podman hosts use the same DB-driven graph rendered as Quadlet units instead – see compass systemd quadlet.

Two equivalent ways to drive this locally:

compass services (recommended for day-to-day dev) – generates and deploys the units for you, then wraps systemctl --user with log-based readiness waiting and a status table.
compass systemd + raw systemctl --user – the same units, but you drive systemd directly (useful for starting/stopping a single service, watching a live journal, or debugging a unit that won't come up).

Question

How do I start, stop, and check the status of all ORE Studio backend services for a local checkout – with compass services, or directly with systemd?

Answer

Recommended: compass services

Ensure the environment is initialised and the project is built (see How do I initialise the checkout environment? and How do I build the system?).
Start services:
```
./projects/ores.compass/compass.sh services start
```
This runs compass systemd generate + compass systemd deploy (picks up the current DB state), then systemctl --user start ores-<env>.target, then polls every service's own log file for its "Service ready." line (NATS readiness is checked via a TCP LISTEN probe on its port instead – its unit has no log file, only journald). The preset defaults to ORES_PRESET from .env; override with --preset. Change log verbosity with --log-level (default: trace).

Cold starts with 20+ services all connecting to the DB at once can legitimately take several minutes; the command's own progress output streams live to /tmp/ores_<env>_services_start.log (overwritten each run) – tail -f that path from another shell to watch startup without waiting on this command's own output.
Check status:
```
./projects/ores.compass/compass.sh services status
```
Per-unit table: running (active + "Service ready." seen in its log), starting (active, not ready yet), stopped (not active), missing (unit not loaded at all – run services start first).

For a process-tree or live-usage view instead of a status table:
```
./projects/ores.compass/compass.sh services tree
./projects/ores.compass/compass.sh services top -1 -b
```
tree runs systemd-cgls across this environment's Claude session slice (app-claude-<env>.slice, see How do I launch Claude Code inside a systemd scope?) plus every unit this environment's fleet aggregates, printed as one subtree per unit – fleet services don't have a per-environment slice of their own (yet), only Claude sessions and, once builds move into their own cgroup, builds do. top runs systemd-cgtop filtered to that same Claude slice for live CPU/memory usage; both forward extra flags verbatim (-a=/-l= for tree, -1=/-b=/=-m= for top, etc).
Stop services:
```
./projects/ores.compass/compass.sh services stop
```
Runs systemctl --user stop ores-<env>.target; every unit's PartOf=ores-<env>.target means this cascades to nats-server and every service in one command, the same one-shot stop the old controller used to perform itself.

Clear logs (optional):

./projects/ores.compass/compass.sh services clear-logs

Direct: compass systemd + systemctl –user

Use this when you want to start/stop/inspect one unit rather than the whole fleet, or watch a live journal.

Generate and deploy the units (same DB-driven render compass services start runs internally – do this whenever service_definition=/=service_dependency changes, or after a rebuild):
```
./projects/ores.compass/compass.sh systemd generate
./projects/ores.compass/compass.sh systemd deploy
```
generate writes concrete unit files to systemd/units/ (not portable across checkouts – absolute paths and this environment's own .env baked in); deploy syncs them into ~/.config/systemd/user/ and reloads only if something changed.

Start the whole fleet, or one service:

systemctl --user start ores-<env>.target        # everything
systemctl --user start ores.iam.service-<env>    # just IAM

(<env> is ORES_ENV_NAME from .env, e.g. swift_curie.)

Status and logs:
```
systemctl --user status ores-<env>.target
systemctl --user status ores.iam.service-<env>
journalctl --user -u 'ores*-<env>*' -f           # live, whole fleet
journalctl --user -u ores.iam.service-<env> -f   # live, one unit
```
Application-level logs (not the systemd/journal wrapper) still land in build/output/<preset>/publish/log/, one file per <service_name>.<replica_index>.log (e.g. ores.iam.service.0.log) – note this is the binary's own filename convention, not the systemd unit name (which carries a -<env> suffix to stay distinct from a sibling checkout's units on the same user session). NATS logs to journald only, no file.

Stop:

systemctl --user stop ores-<env>.target          # everything, cascades via PartOf=
systemctl --user stop ores.iam.service-<env>      # just IAM

A unit stuck failed after a dependency raced it: Requires means a unit whose first start attempt fails (e.g. it raced nats-server or another dependency) stays failed even once that dependency's own Restart=always later succeeds – systemd does not automatically retry it. Reset and restart by hand:
```
systemctl --user reset-failed ores.iam.service-<env>
systemctl --user start ores.iam.service-<env>
```

Prerequisites checklist

.env exists (run compass env configure if not – see How do I initialise the checkout environment?).
Database initialised (compass db recreate -y).
Project built (./compass.sh build --preset <preset>).
nats-server on PATH or in /usr/sbin.

Script

compass services start/stop/status/tree/top/clear-logs – generate+deploy the systemd units and drive them, with readiness waiting, a status table, and per-environment process-tree/usage views. compass systemd generate/deploy – render and install the units without starting anything. systemctl --user start/stop/status <unit> – drive one unit or the whole ores-<env>.target directly.

Tested by

Manual, each development session. No CI path – CI uses pre-configured runners.