Stream large HTTP upload bodies to disk instead of buffering
Table of Contents
This page is a capture in the inbox bucket of the product backlog — a pre-sprint idea, not yet pulled into a sprint as a story.
What
ores.http.server's generic per-connection loop
(http_session::run(), projects/ores.http/api/src/net/http_session.cpp)
reads every incoming request via a plain
http::request_parser<http::string_body>, shared by every JSON API
route and by storage_routes's PUT/GET/DELETE storage endpoints alike.
That means a large binary upload (a compute engine package, tens of
MB) gets fully buffered into a std::string before storage_routes::handle_put
writes it to disk – correct, but not memory-efficient. The idiomatic
fix is route-specific streaming: give the storage PUT/GET endpoints
their own http::file_body-based read/write path that streams
straight to/from disk without ever holding the whole object in RAM,
mirroring what ores.storage::net::http_client::get already does
client-side for downloads (streams into an http::file_body response).
This needs the router/=http_request= abstraction (currently
content-agnostic, body typed as std::string) to grow a way for a
specific route to opt into a different body type, which is a real
design change, not a one-line fix.
Why
Discovered while fixing the ACME compute package publishing flow (task
0F768B80-184A-44C7-B725-A594A873AB4A): uploading the ~57MB vendored
ORE engine package failed with a "Broken pipe" error because Beast's
default body_limit for a plain parser is 1MB. The immediate fix
raised http_server_options::max_body_size to 256MB so the upload is
merely accepted, but that's a bound on how much can be buffered in
memory per upload, not a fix for the buffering itself. At compute
publishing's actual scale (a handful of admin-triggered engine
publishes, not concurrent high-throughput traffic) a bounded in-memory
buffer is an acceptable interim trade-off, but the buffering approach
won't scale gracefully if uploads grow larger or more frequent, and
streaming to disk is the more correct long-term shape for this
endpoint regardless.
References
projects/ores.http/api/src/net/http_session.cpp(http_session::run())projects/ores.http/api/include/ores.http.api/net/http_server_options.hpp(max_body_size)projects/ores.http/core/src/routes/storage_routes.cpp(handle_put=/=handle_get)projects/ores.storage/src/net/http_client.cpp(client-sidefile_bodystreaming already in place for GET)