Story: Fix image-batch NATS payload overflow, add SVG compression
Table of Contents
This page documents a story in Sprint 24. It captures the goal, current status, acceptance criteria, and the tasks that compose it.
Goal
Discovered while testing the vintage IR dataset story: country flags (and any other batch-loaded image set) were partially missing in the Qt client. Root-caused to two distinct issues:
- A now-fixed stale-binary bug:
ores.assets.servicequeried a renamed column (svg_data->mime_type=/=data) and had not been rebuilt since. (Fixed by rebuild; no code change needed for this part.) ImageCache's batch-fetch caps each NATS request atMAX_IMAGES_PER_REQUESTimages by count, not byte size. SVG flags vary widely in size (a few KB for a simple flag, considerably more for a complex one, e.g. Norfolk Island or CEFTA), so a batch that happens to cluster several large images together can still exceed NATS's default 1MB max payload – and the entire batch silently fails when that happens, dropping every image in it. As an immediate mitigation,MAX_IMAGES_PER_REQUESTwas lowered from 50 to 15 (this story's prerequisite commit) – reduces the odds without eliminating the pathological case.
Additionally: image data (SVG) is sent completely uncompressed – base64-encoded raw XML inside a JSON payload, no gzip/zstd anywhere in the pipeline. SVGs compress very well (typically 70-80% size reduction), so this is both a payload-size contributor and a bandwidth/latency cost independent of the batching bug.
This story's goal: make batching genuinely byte-size aware (track cumulative payload size while building each batch, not just count) so a batch is cut short whenever adding the next image would risk exceeding the NATS payload limit, and add compression to the image transfer pipeline (client and server) so payloads shrink regardless of batch size.
Alternative worth weighing: HTTP download instead of NATS request-reply
ores.compute.wrapper already downloads binaries via a plain HTTP GET
against the generic S3-like object storage endpoint
(ores.http.core/routes/storage_routes.hpp,
GET /api/v1/storage/{bucket}/{key}) rather than a NATS request-reply
round trip. Bulk binary transfer (many images, each independently
addressable by key) is arguably a better fit for that same pattern than
for NATS request-reply: no payload-size ceiling to chunk around at all,
standard HTTP caching/range-request semantics apply for free, and it
sidesteps the batching problem entirely rather than tuning around it.
Leaning towards staying on NATS unless the decision task turns up a good
reason not to: NATS itself has no issue with binary payloads (a NATS
message is just an opaque byte array), so there's no inherent transport
ceiling being fought here. The actual cost stack today is self-inflicted
by this app's own wire format, not by NATS or by using request-reply:
every message (including image responses) goes through
rfl::json::write=/=read (ores.service/messaging/handler_helpers.hpp),
so binary SVG data is base64-encoded into a JSON string field – ~33%
inflation – on top of being sent fully uncompressed. Sending raw
(optionally gzip'd) bytes directly as the NATS payload for image-
carrying messages, bypassing JSON/base64 for that field, plausibly
removes most of the overflow risk on its own. The decision task should
verify this against the synthetic worst case before treating the HTTP-
transport migration as necessary – it's a materially bigger change
(new storage-addressing scheme for images) that should only be taken on
if NATS-with-compression genuinely can't be made to fit.
Status
| Field | Value |
|---|---|
| State | DONE |
| Parent sprint | Sprint 24 |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-07-29 |
Acceptance
ImageCache's client-side batching tracks cumulative estimated payload size (accounting for base64 + JSON overhead, not just raw SVG bytes) and closes a batch early, before adding an image that would push it over a safe threshold below NATS's max payload.- The fix is verified against a synthetic worst case: many large SVGs (e.g. duplicate the largest known flag N times) requested in one fetch, confirming no batch ever exceeds the payload limit regardless of MAX_IMAGES_PER_REQUEST.
- Image data is compressed on the wire (server compresses on write or send, client decompresses on receive) – gzip is sufficient, no need for a more exotic codec for text/SVG payloads this size.
MAX_IMAGES_PER_REQUESTcan be raised back towards its original value (or removed as a meaningful constraint) once byte-size-aware batching and compression are both in place, since count alone is no longer the binding constraint.
Tasks
| Task | State | Start | End | Description |
|---|---|---|---|---|
| Decide: NATS binary+compression vs. HTTP storage-endpoint transport for image delivery | DONE | 2026-07-26 | 2026-07-29 | Confirm that sending images as raw (non-base64) compressed bytes over NATS resolves the payload-overflow problem on its own, before considering a move to the existing HTTP storage-endpoint transport. Record the decision and rationale. |
| Add gzip compression to the image NATS pipeline, send raw bytes instead of base64 | DONE | 2026-07-27 | 2026-07-27 | Server compresses SVG image data before sending; client decompresses on receive. Stop routing binary image data through the JSON/base64 encoding used for the rest of the messaging protocol – send it as raw bytes on the wire for image-carrying messages, eliminating both the base64 inflation and the missing compression in one pass. |
| Make ImageCache batching byte-size aware | DONE | 2026-07-27 | 2026-07-28 | Track cumulative estimated payload size (post-compression) while building each image-fetch batch, not just image count, so a batch is cut short before it would risk exceeding the NATS payload limit. Backstop for whatever pathological cases remain after compression; revisit MAX_IMAGES_PER_REQUEST once this and the compression task both land. |
| Make ImageCache batching byte-size aware, add SVG compression (ABANDONED – split above) | ABANDONED | Superseded: bundled a design decision with two independent implementation concerns under one task. | ||
| Send raw bytes instead of base64 for image-carrying NATS messages (ABANDONED – superseded) | ABANDONED | 2026-07-28 | 2026-07-28 | Superseded by the generic NATS wire format story: MessagePack solves the base64 problem for every message type, not just images, via a single .env-driven format choice. |
Decisions
Stay on NATS request-reply; do not migrate to HTTP storage-endpoint transport
Decided in the decision task. Measured (not estimated) the actual
worst-case batch – the 15 largest SVGs in
external/flags/flag-icons/ (the real worst offenders turned out to
be Serbia and Ascension Island, not the story's original examples of
Norfolk Island/CEFTA, which are both small) – against NATS's actual
1 MiB default max payload (no override configured anywhere in this
repo):
| Encoding | Bytes | vs. 1 MiB limit |
|---|---|---|
| Raw, base64'd (today's wire format) | 1,348,864 | 132% – overflows |
| Raw bytes, no base64, uncompressed | 1,011,648 | 96% – no margin |
| Gzip, still base64'd | 415,601 | 40% |
| Gzip, raw bytes (no base64) | 311,701 | 30% |
The worst case already overflows today, confirming the bug. Gzip compression alone (independent of the base64 question) fixes it with a ~2.5x margin; adding raw-byte encoding on top gets ~3.4x. Neither number comes close to justifying the bigger, riskier HTTP-transport migration (new storage-addressing scheme for images). Both remaining tasks proceed as scoped: compression + raw-byte wire encoding (the load-bearing fix), and byte-size-aware batching (defense-in-depth backstop, not load-bearing given the margin above).
Compression implemented generically at the NATS transport layer, not image-specifically
Decided in the compression task. Every NATS message (client and
server, request-reply and pub/sub) already funnels through two choke
points in ores.nats (make_msg() on send, extract_message() on
receive). Added a size-thresholded (4096 bytes), header-flagged
(X-Content-Encoding: gzip) compress/decompress pair there instead of
in image-specific code – every message above the threshold benefits
automatically, with zero changes to ImageCache, ClientManager, or
any protocol/domain type, and the mechanism is unit-testable in
isolation. This effectively delivers the "compress all NATS messages"
capture below as infrastructure, without deciding to spend effort
auditing every other message type's traffic patterns – it simply
applies wherever a payload happens to be large enough.
Raw-bytes-instead-of-base64 (the compression task's other originally- named mechanism) was descoped as not load-bearing given the 2.5x margin gzip-with-base64 already provides; split into a separate follow-up task.
Out of scope
- Auditing non-image NATS traffic to decide whether any of it should be deliberately restructured to benefit from compression (as opposed to benefiting automatically, for free, whenever it happens to cross the size threshold above) – separate, broader question, filed as a backlog capture: Consider compressing all NATS JSON messages, not just images.
- Decompressing
jetstream_adminpeek payloads – unrelated to this story's own goal (it's an admin-tooling gap exposed as a side effect of the compression this story introduced, not something blocking the fix itself) – split into its own story at close: Fix jetstream_admin missing gzip decompression.