Story: Close the object storage interface leftovers

Table of Contents

This page documents a story in Sprint 26. It captures the goal, current status, acceptance criteria, and the tasks that compose it.

1. Goal

Two leftovers of the interface story are closed.

The first is the raw shell surface. The Object Storage target state separates raw use from wrapped use, and promises that a caller which only wants bytes somewhere can name a bucket, a key and a local path and be done. The generated objects menu covers the NATS protocol, so the work is the HTTP half: verbs that move files, and the client operations they need.

The second is the schema setup. A second setup_schema.sql run stopped, which made the seed helpers' idempotency a matter of review rather than a thing observed. The story closes that gap and gates the class, so the next statement of the same kind cannot arrive unnoticed.

2. Status

Field Value
State DONE
Parent sprint Sprint 26
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-09-27
End 2026-09-27

3. Acceptance

  • storage put <bucket> <key> <path> uploads a local file, and storage get <bucket> <key> <path> writes it back byte-identical.
  • storage delete and storage list complete the raw verb set.
  • A second setup_schema.sql run over the database the first run built completes without error.
  • The consumer verbs that wrap storage keep working on the protocol keys.

4. Tasks

Task State Start End Description
Scaffold story: Close the object storage interface leftovers DONE 2026-09-27 2026-09-27 Story scaffolding rides this task: documents, sprint wiring, and the scaffold PR. Close it before merging that PR.
Give the shell a raw storage surface over a local path DONE 2026-09-27 2026-09-27 A shell menu whose verbs are a bucket, a key and a local path, over the storage HTTP interface.
Make the schema setup re-runnable DONE 2026-09-27 2026-09-27 Make the create and populate phases re-runnable, gate the class, and prove a clean second run.

5. Decisions

  • The raw surface speaks HTTP, not NATS. A message bounds the value it carries, and the target state already says the bulk path is HTTP. So the new menu moves files and the generated objects menu keeps the command surface. Both interfaces stay exercisable by hand, which is what the interface story asked for.
  • The populate phase declares itself a reseed. Its scripts state version zero, and the version gate refuses a create that meets a live row: 53 files stopped on the second run. Opening ores_utility_allow_version_replace_fn() for the populate transaction is the mechanism the gate documents for a reseeding generator, so one change closes all 53 without touching any of them. The cost is that a re-run re-asserts each seeded row at a new version instead of skipping it.
  • The class is gated, not the instance. check_sql_hygiene.py now reports DDL a second run cannot repeat, so the next create trigger without or replace fails a check rather than a database recreation.
  • The two transports carry one listing document. The HTTP listing and delete replies now carry the fields the protocol declares, so the shell parses either interface with the protocol's own type.

6. Notes

  • The base URL had two byte-identical copies in the shell before this. ores.shell.api holds the one definition now.
  • The route patterns compose from storage_paths::prefix, closing the last duplicate declaration of the storage path.
  • The story began as two leftovers recorded in the interface task's Result. The census that sized the second one is in that task's Plan.

7. Out of scope

  • Fixing the reporting write path. The research this story commissioned found that ores.reporting writes its gathered blobs to the retired bucket report-data while ores.ore reads them from ores, so the writer and the reader of one object disagree. Reporting is another environment's live work, so the defect is captured rather than fixed here: Reporting writes the report blobs to a bucket ore does not read.
  • HEAD on the client. Existence stays a read, which is what the protocol declares.

8. Result

Both leftovers are closed on PR #2250, with all nine checks green.

The raw surface. The shell has a storage menu whose verbs are put, get, delete and list, over HTTP, beside the generated objects menu that speaks NATS. Both interfaces are therefore exercisable by hand, which is what the interface story set out to do. The client grew the operations the surface needed – a body-returning GET and DELETE – and the transfer helper and the URL contract grew with it. The shell's two identical copies of the storage base URL collapsed into one, and the routes now compose their patterns from the declared prefix.

The schema setup. The create scripts needed one word: create or replace trigger in a single hand-written file. The populate phase needed the reseed declaration its own version gate documents, which closes 53 files at once. check_sql_hygiene.py now reports DDL a second run cannot repeat, so the class is gated rather than the instance.

Cost paid, recorded. A re-run re-asserts every seeded row at a new version instead of skipping it, so the version history grows with each setup run. That is the price of declaring the run a reseed, and it is the behaviour the gate's own documentation asks for.

Found, not fixed. The research the story commissioned established what the interface task's note about the trading and marketdata export handlers meant: those handlers own no bucket, ores.reporting names one for them, and it names report-data while ores.ore reads ores. The capture carries the finding and the lines that prove it.

Emacs 29.3 (Org mode 9.6.15)