The compute wrapper's HTTP download opens its destination before it checks the status
Table of Contents
This page is a capture in the inbox bucket of the product backlog — a pre-sprint idea, not yet pulled into a sprint as a story.
1. What
ores.compute.wrapper carries its own Beast HTTP client, and its download
opens the destination file before it looks at the response status:
std::filesystem::create_directories(dest.parent_path()); // :78 parser.get().body().open(dest.string().c_str(), beast::file_mode::write, open_ec); // :82 http::read(stream, buf, parser); if (parser.get().result_int() < 200 || parser.get().result_int() >= 300) // :89 throw std::runtime_error(...);
A 404 therefore truncates a file that was already at that path, leaves the error
body in a file that was not, and only then throws. ores.storage had the
identical shape in http_client::get, and
Bring ores.storage to the clean standard
fixed it by reading the header first and opening the destination only for a
2xx:
http::read_header(stream, buf, parser); if (parser.get().result_int() < 200 || parser.get().result_int() >= 300) throw std::runtime_error(...); // only now create the directory and open the body sink
The wrapper's upload is not affected: it reads from disk and checks the status
before anything is written.
2. Why
The wrapper's download is how a compute node receives its engine package and its input tarballs, so its destination is usually the artefact the node already has. Destroying the previous artefact on a failed download is worse than failing and leaving it in place, and nothing reports the difference: the caller sees the same exception either way.
The two clients are one behaviour written twice, which is the capture Unify the codebase's three separate Beast HTTP client implementations exists to collapse. That consolidation is the lasting fix and would retire this defect with the file. Until it lands, this is a standalone correctness fix of three moved lines.
3. References
projects/ores.compute/wrapper/src/net/http_client.cpp:56-97—download, opening at :82 and checking at :89.projects/ores.storage/src/net/http_client.cpp— the same method after the fix.projects/ores.storage/tests/http_client_tests.cpp:107-137— two cases that fail against the old order, one for a destination that did not exist and one for a file that did.
4. See also
- Unify the codebase's three separate Beast HTTP client implementations — the consolidation that retires both copies.
- Component Clean Standard — V07, the rule that a fix carries a test that fails without it.