Task: Prove and polish the remote compute-node join

Table of Contents

This page documents a task in the Remote WSL offload and compute nodes story. It captures the goal, current status, acceptance, and any notes or results.

Goal

Prove the compute role of compass env deploy (PR #1937) live, end-to-end, against a real serving environment:

  • Deploy ores.compute.wrapper to a remote box with compass env deploy <host> --role compute. The wrapper joins the serving environment's compute core over NATS: it registers as a compute node of that environment, receives work, and reports results.
  • Register the remote host in the serving environment's compute.hosts table. The host IDs come from the ORES_GRID_NODE_*_HOST_ID ids in the environment's generated .env; keep them in sync with compass db recreate (env_init.py).
  • Fix any ergonomics gaps the live run exposes: registration flow, docs, or the deploy command itself.

Status

Field Value
State DONE
Parent story Remote WSL offload and compute nodes
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-08-10

Acceptance

  • A compute node deployed with compass env deploy <host> --role compute joins the serving environment end-to-end: it appears in the environment's compute topology, receives and executes work, and its results are visible in the serving environment.
  • The remote host is registered in the serving environment's compute.hosts table with IDs matching the environment's generated ORES_GRID_NODE_*_HOST_ID values.
  • Any ergonomics gaps found during the live proof are fixed or tracked as follow-up work in this story.

Plan

(Implementation strategy. Written when work starts; key decisions are distilled into the parent story's * Decisions at close, but the plan itself stays — it is the historical record of what we did.)

Notes

Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario, run through the QA Validation Runner panel) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
     

PRs

PR Title
#1946 [ores.compass,ores.cli,ores.nats,ores.sql] Prove and polish the remote compute-node join

Review

# Comment summary File Decision Notes
1 --package-sha256 accepts any value after the =; no 64-lowercase-hex validation (two reviewers) projects/ores.cli/src/config/entity_parsers/compute_app_versions_parser.cpp Accepted Fixed: parse-time shape check (64 chars, all lowercase hex); typo surfaces at parse time, not at dispatch
2 No test coverage for the new --package-sha256 parsing branches projects/ores.cli/tests/entity_parser_tests.cpp Accepted Fixed: 6 cases covering happy path attach-by-code, missing required fields, malformed pair, unknown platform code, short hex, uppercase hex
3 _local_ipv4_addresses() runs twice per generation (SAN list + log line) (two reviewers) projects/ores.compass/src/nats_certs.py Accepted Fixed: resolved once per generate() into a local, reused for both; also removes a race where the printed SAN set could differ from the cert's

Result

Proved live, end-to-end, on 2026-08-10 against the brave_hopper serving environment (this host, 192.168.1.172) with the Newton box (192.168.1.22) as the remote compute node:

  • Deployed ores.compute.wrapper to Newton with compass env deploy newton --role compute. The wrapper joined over NATS mTLS and registered as compute host 1cefaf85-e89a-4126-b5af-3548c4e5bef9 in ores_compute_hosts_tbl (system tenant), heartbeating every 30 s (telemetry_handler → host_service "Saved host" entries in the compute service log).
  • Dispatched a workunit end-to-end: IAM login (system admin super_admin, RS256 JWT) → compute.v1.workunits.save with Authorization: Bearer → the service stamped the workunit, created result 46ba8af7-8a81-484a-a613-5a07acd7b945 (server_state=2 Unsent) and published the assignment to the JetStream stream ores_dev_brave_hopper_compute_assignments on compute.v1.work.assignments.ffffffff-ffff-ffff-ffff-ffffffffffff.x64-linux.
  • The Newton wrapper's durable (compute_wrapper_ffffffff-ffff-ffff-ffff-ffffffffffff_x64-linux) delivered the assignment, downloaded the package from http://192.168.1.172:20400, verified its SHA-256, extracted the manifest, ran the hello engine with the input bundle, uploaded Output/result.txt to the serving environment's compute storage (/api/v1/storage/compute/output/46ba8af7-…tar.gz), and submitted the result.
  • Verified in ores_compute_results_tbl: result row now has server_state=5 and outcome=1 (Success), received 2026-08-10 17:56:15 — one second after dispatch.

Ergonomics gaps found and fixed in this task:

  • nats_certs.py: NATS broker cert now carries the serving host's non-loopback IPv4 addresses as IP SANs (RFC 6125: remote clients connect by IP, which must appear as an "IP Address" SAN, never a DNS entry).
  • env_deploy.py: nested .env values written without quoting (remote shell then re-parsed PEM lines as multiple values) and ORES_COMPUTE_WRAPPER_HTTP_BASE_URL not threaded through on compute-role deploys; both fixed. (Initial deployment worked around the quoting by hand-editing the generated compute.env.)
  • http_client.cpp: the wrapper's package download used a non-streaming body read (unbounded copy); switched to the streaming file_body download used elsewhere.
  • ores.nats client: NATS async errors and connect failures now include nats.c's fine-grained last-error detail (e.g. the OpenSSL certificate-verify reason) instead of only the coarse status text — this is what made the mTLS failures on Newton diagnosable.
  • ores.cli app-versions add: new repeatable --package-sha256 '<platform-code>=<hex>' option; the wrapper hard-fails dispatch when the checksum is empty, so the CLI had no way to register a dispatchable version. Backfilled the proof row via SQL; CLI verified via ores.cli.tests.
  • compute_ore_app_seed.sql: ORE package_uri now matches the publish-package storage layout (uppercase ORE/… directory).

Known follow-up (out of scope here): the compute service's report_submit_handler still publishes assignment replies with a placeholder platform triplet / app_version_id; normalise when the result-reporting path is revisited.

Emacs 29.3 (Org mode 9.6.15)