Story: Manage the compute grid from the shell
Table of Contents
This page documents a story in Sprint 25. It captures the goal, current status, acceptance criteria, and the tasks that compose it.
1. Goal
ores.shell gains a comprehensive compute command set so the compute
grid can be driven entirely from the shell, with no UI:
- Read the grid: list apps, app versions, platforms, hosts and batches.
- Drive jobs: add a batch, dispatch N consecutive jobs from it, list its workunits and results.
- Fetch artifacts: download a job's input bundle and a result's output bundle from compute storage.
- Observe the grid:
grid-statsrenders the telemetry summary as a per-node table (host, heartbeat age, tasks completed, bytes moved);delete-hostremoves a host row. - Prove the grid: a library script dispatches 10-20 consecutive jobs and asserts every online node was exercised and every result succeeded — the "did all engines kick in" test, runnable without a UI.
2. Status
| Field | Value |
|---|---|
| State | DONE |
| Parent sprint | Sprint 25 |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-09-23 |
3. Acceptance
- Every verb above exists in the shell's compute menu and works against a live serving environment.
- A script can add a batch, dispatch 10-20 consecutive jobs, poll
grid-statsuntil the batch drains, and assert every online node was exercised and every result has outcome Success. - The smoke-test script is proven live against brave_hopper with the Newton compute node online; per-node task counts show jobs spread across the grid, not all on one host.
4. Tasks
| Task | State | Start | End | Description |
|---|---|---|---|---|
| Scaffold story: Manage the compute grid from the shell | DONE | 2026-08-10 | 2026-08-10 | Story scaffolding rides this task: documents, sprint wiring, and the scaffold PR. Close it before merging that PR. |
| Add compute read verbs to the shell: list apps, app versions, platforms, hosts and batches | DONE | 2026-08-10 | 2026-08-25 | Read-only verbs over the compute domain: list apps, app versions, platforms, hosts and batches, with the fields a script needs. |
| Add job lifecycle verbs to the shell: add-batch, dispatch-batch with a job-count loop, list-workunits and list-results | DONE | 2026-08-11 | 2026-08-25 | Shell verbs to drive the compute job lifecycle end-to-end without a UI: create a batch, dispatch N consecutive jobs from it (pack and upload the input bundle once, then loop N saves), list the workunits of a batch, and list the results with their outcome and host. |
| Add grid monitoring verbs to the shell: grid-stats with per-node detail and delete-host | DONE | 2026-08-11 | 2026-08-25 | Shell verbs to observe the compute grid: grid-stats fetches the telemetry grid summary and renders a per-node table (host, heartbeat age, tasks completed, input/output bytes), so a test can verify every online node was exercised; delete-host removes a host row by id. |
| Add artifact download verbs to the shell: download job inputs and outputs | DONE | 2026-08-11 | 2026-08-25 | Shell verbs to fetch a job's input bundle and a result's output bundle from compute storage (ores.storage.net.storage_transfer is already linked into the shell via package_publisher): download-input and download-output, taking the workunit id or result id and unpacking the tar.gz into a target directory. |
| Add a compute grid smoke-test script and prove it live | DONE | 2026-08-11 | 2026-08-25 | Library .ores script that exercises the whole grid from the shell: add a batch, dispatch 10-20 consecutive jobs, poll grid-stats until the batch drains, then assert every online node was exercised and every result has outcome Success. Prove it live against brave_hopper with the Newton node online, and record the proof in the task result. |
| Reconcile provisional shell outputs with the codegenerated tables | ABANDONED | 2026-08-10 | 2026-09-23 | Swap the provisional raw-row outputs for the codegen table renderers once the ores.compute drift story lands: list-platforms streams the generated platform table (out streams the platforms vector) and grid-stats renders the node-sample tables, matching the other verbs. Blocked on Bind ores.compute entities to profiles; verify zero-diff regen (platform renderer). |
| Fix the compute grid dispatch and wrapper result-submit channels dropped by the ores.compute bind | DONE | 2026-09-05 | 2026-09-05 | Restore the two side effects bind PR #2012 dropped: the workunit dispatch seam in the service app layer and the trusted wrapper result-submit channel (compute.v1.results.submit) with its Validator and Assimilator. |
| Write up the post-merge system-test report for the compute and iam binds | DONE | 2026-09-06 | 2026-09-06 | Post-merge system test of the compute and iam bind PRs (#2012/#2014), driven from ores.shell: findings F4/F5 and the iam refdata grant, smoke evidence, root-cause narrative, UI limitations and follow-ups. |
5. Decisions
6. Out of scope
- Backend changes to the compute service: it already exposes every required API subject; this story is the shell verb layer only.
- Desktop client UI: the shell is the delivery vehicle here.
- Process-workflow verbs: the shell
workflowmenu already covers that domain; this story is compute jobs.
7. Result
Closed at sprint 25 close. Every verb and the smoke test shipped and ran live. The reconcile task goes to the sprint 26 codegen pass.