Task: Add job lifecycle verbs to the shell: add-batch, dispatch-batch with a job-count loop, list-workunits and list-results
Table of Contents
This page documents a task in the Manage the compute grid from the shell story. It captures the goal, current status, acceptance, and any notes or results.
1. Goal
The shell's compute menu gains four job lifecycle verbs:
add-batch, dispatch-batch, list-workunits and list-results.
Together they create a batch, dispatch a run of consecutive jobs from
it, and inspect the resulting workunits and results as tables — the
whole lifecycle driven from a script with no UI.
2. Status
| Field | Value |
|---|---|
| State | DONE |
| Parent story | Manage the compute grid from the shell |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-08-25 |
3. Acceptance
compute add-batch <external_ref> <job_count>creates a batch and prints its id;--smokebounds the count to the grid smoke range (10 to 20 jobs).compute dispatch-batch <external_ref> <job_count> <app_version_id> <input_dir>packs the input directory, uploads it once as the batch input bundle (input/{batch_id}.tar.gz), marks the batch dispatched, and saves one workunit per job; each save dispatches to the grid via JetStream.compute list-workunits [--batch <external_ref>]renders the workunits table (id, batch id, app version id, input uri, priority, target redundancy);--batchfilters to one batch.compute list-results [--batch <external_ref>]renders the results table (id, workunit id, host id, server state, outcome, output uri, error message);--batchfilters to one batch.- Every verb works against a live serving environment; empty domains render a header-only table, not an error.
4. Plan
The four verbs reuse existing NATS subjects (batch add/dispatch,
workunit save/list, result list) and the codegen table renderers in
ores.compute.api; no backend or codegen changes. Implementation
lives in compute_commands.{hpp,cpp} next to publish-package,
following its shape: require a login, send the request authenticated,
log the row count, print the table. dispatch-batch packs and uploads
the input bundle once, then loops the job count calling
save_workunit; the batch external reference is the script-friendly
handle, so the verbs look a batch up by external ref where the wire
protocol needs its id.
5. Notes
- Live proof (2026-08-11): scripted session against brave_hopper
(all 23 services up).
compute add-batch smoke-20260810 10 --smokecreated batch6b1efd4c-…;compute dispatch-batch smoke-20260810 10 840f6194-2f11-4142-996a-2c90efabe053 <input_dir>(hello 1.0.0, x64-linux) packed and uploaded the input bundle once and saved 10 workunits; each save dispatched via JetStream. - The batch drained in ~2 seconds: all 10 results have outcome 1 (Success) and server state 5 (Done), spread across the five real wrappers (amber-capybara 3, brave-caracal 2, eager-tapir 2, bumbling-dhole 2, upbeat-iguana 1).
list-workunits/list-resultsrender the generated tables;--batchfilters to the 10 rows of the smoke batch (11 unfiltered, one workunit/result pair from the earlier join-proof batch).- The workspace tables are temporal (valid_from/valid_to): every heartbeat inserts a new host version; the list verbs read the current versions only, so a running grid shows no duplicate rows.
- Recipes live in
doc/recipes/shell/compute/(one per verb), tangled intoprojects/ores.shell/scripts/library/compute/, indexed under theComputesection ofdoc/recipes/shell/shell.org.
6. Test Scenarios
Manual QA scenarios (scaffolded via compass add test_scenario, run
through the QA Validation Runner panel) that verify this task. Link
new ones here as they're created; the scenario doc itself links back
via its "Verifies task" field.
| Scenario | State | Notes |
|---|---|---|
7. PRs
| PR | Title |
|---|---|
| #1992 | [ores.shell] Add compute grid management verbs to the shell |
8. Review
| Comment summary | File | Decision | Notes |
|---|---|---|---|
9. Result
- Four lifecycle verbs shipped in
compute_commands.{hpp,cpp}:add-batch,dispatch-batch,list-workunitsandlist-results. All reuse existing NATS subjects and codegen table renderers; no backend or codegen changes. - Acceptance met, verified live against brave_hopper (2026-08-11):
add-batch smoke-20260810 10 --smokecreated the batch;dispatch-batchpacked and uploaded the input bundle once and saved 10 workunits, each dispatched via JetStream; the batch drained in2 seconds with all 10 results outcome Success (server state Done), spread across the five real wrappers; ~list-workunitsandlist-resultsrender the generated tables and--batchfilters to the batch's rows. - Hardened in the PR #1992 review round: the batch is marked
"dispatched" only after every workunit save succeeds, so a partial
failure leaves it open and re-dispatchable; each new workunit sets
the nil
canonical_result_idsentinel. - Recipes live in
doc/recipes/shell/compute/(one per verb), tangled intoprojects/ores.shell/scripts/library/compute/, indexed under theComputesection ofdoc/recipes/shell/shell.org.