One HTTP Server: designing ores.http to replace the web BFF

A gateway: terminate HTTP, authenticate, forward to the owning service over NATS. It holds no service, repository or database context, so it cannot decide anything the service decides. Twenty-four generated route units hold 108 route handlers over assets, core, iam and refdata, under /api/v1/<component>/<plural>. It is deployed as ores.http.server on ORES_HTTP_PORT (21601 in this checkout).

The parts that matter for this design:

Two corrections to what one might assume:

The gateway is not asynchronous in the way that matters. Handlers are asio::awaitable, but the NATS call inside them is request_sync, which blocks the single io_context thread for up to 30 s. The coroutine is a wrapper around a blocking call. One slow service stalls every connection, including the ones only serving static files.

TLS is a dead option. The fields exist and the parser does not even read them. Everything is plaintext HTTP/1.1 today. That is not acceptable for a server that faces the internet, so 6.12 designs both listeners rather than leaving the options as decoration.

Every row above is correct for a stateless gateway at low connection counts and wrong for the target. They are the work.

Node 22 and Fastify, 292 /api routes across 36 modules, 26.9k lines of non-test source and 34.2k lines of tests. It serves the React bundle, which is 124.8k lines and 214 routes: the browser is the thing that defines the surface.

What it does that the gateway does not:

• Session custody. An opaque identifier in an HttpOnly cookie (ores_web_session_<environment>, per environment so two ports on one host cannot overwrite each other), hashed in memory, with the bearer token and the party scope held server side (packages/bff/src/sessions.ts).
• One SSE stream per session, GET /api/events, plus POST /api/events/watch to declare what a screen shows (packages/bff/src/server.ts:2926 and :2968).
• A shared subscription registry keyed by (component, entity), refcounted, filtered by tenant and by the parties a session works in, coalesced into 150 ms windows with at most 20 record keys per message.
• Request shaping: pagination, search, sort, joins and journey composites that the canonical protocol does not express.
• Static serving of the built bundle, an image cache, rate limiting, client-log ingest, structured logs.
• Two file-backed domains. The agile board, sprint charts and work trees, and the QA scenario documents, read the checkout's doc/ tree and shell out to git (packages/bff/src/agile/file-agile-store.ts, scenarios/file-scenario-store.ts, agile/work-trees.ts). Twelve routes touch no service at all.

That last one is the exception that proves the rule. None of the rest is business logic; all of it is transport and custody. That is exactly the set of jobs a gateway should grow into, and the file-backed twelve are the set it should not.

The two sides are not foreign to each other. Both speak the same org models through ores.codegen, the same msgpack protocol, the same subject names, the same error envelope. Both already delegate the caller's token: ores.http through nats_client::with_delegation, the BFF through a per-session NATS connection. ores.service already provides the host runner, the rate limiter, the request context, the heartbeat publisher and systemd notification.

The important distinction is between the contract layer and the transport layer. The contract layer is shared and in good shape. The transport layer is not: the C++ server is single-threaded, blocking inside its handlers, capped at 100 connections, and has no way to write a response in pieces. That gap, and not the contract, is the whole of the work in Section 6.

The arithmetic behind N1 and N2. Per idle stream connection:

TLS changes that arithmetic, and the change is large. An OpenSSL 3 SSL object plus its read and write BIOs costs roughly 20-40 KiB per idle connection, and that dwarfs everything else in the table:

So the 1.5 GiB budget holds for plaintext at 100,000, holds for TLS at tens of thousands, and does not hold for both at once. Section 6.12 says what to do about it, and the answer is not to wish the number smaller: it is to measure it in connection_bench and to choose where TLS terminates.

File descriptors: one per connection plus a working margin. The unit needs LimitNOFILE=131072 at minimum and 1,048,576 to leave headroom. Confirm with sysctl fs.file-max and net.core.somaxconn.

This is why the write ring is small and bounded: a slow client is a client we conflate for, not a client we buffer for (Section 6.6).

The 100,000 target needs an honest basis, because the repository does not support the number as a forecast. It is worth stating what it does support.

Three facts settle the question:

1. Grid engines are not an HTTP control-plane load. A wrapper holds no user JWT and speaks compute.v1.ops.claim_work, heartbeat, node_sample and a work_available doorbell over NATS. It reaches HTTP only to move bytes, with a capability token minted for one assignment. Nothing in the tree suggests thousands of them, let alone hundreds of thousands.
2. The 100,000 figure in this project is feeds, not connections. doc/agile/product_backlog/next/run_the_market_data_path_at_one_hundred_thousand_feeds.org asks for about 29,000 source ticks a second at 100,000 feeds and four times that in an active market. That load travels on NATS, not on HTTP.
3. No HTTP target exists anywhere. No document states a connection count, a latency budget or a throughput target for ores.http, and benchmarks/ measures XML parsing only.

So the requirement stands as a design ceiling: build for 100,000 so that the architecture has no cliff at 1,000, and so that the number is a config value and not a rewrite. Design to it; do not staff to it. The sizing arithmetic in 4.3 is what makes the decision cheap either way.

One consequence of that evidence is worth stating early, because it cuts against the framing that started this work: the grid is not an argument for HTTP streaming. It is an argument for large-file transfer with flow control, on the same server and the same port, which this design also provides (Section 6.7).

                 ┌──────────────────────────────────────┐
kernel           │  N cores, each with its own          │
┌────────┐       │  ┌────────────────────────────────┐  │
│ accept │──SO_REUSEPORT──▶ core 0: io_context,   │  │
│ queue  │       │  │        acceptor, slots, arena │  │
└────────┘       │  ├────────────────────────────────┤  │
                 │  │ core 1: io_context, ...        │  │
                 │  └────────────────────────────────┘  │
                 │  one NATS connection (shared)        │
                 │  one NATS dispatch thread            │
                 │  one timer thread for the 150 ms tick│
                 └──────────────────────────────────────┘

• N io_contexts, one per core. N defaults to min(hardware_concurrency, 8) and is configurable. A connection is bound to the core that accepted it for its whole life.
• SO_REUSEPORT on every acceptor. The kernel hashes the 4-tuple to a core, so accepts are spread without a shared accept mutex or a cross-thread handoff.
• No strands. Strands exist to serialise access from many threads. With one thread per connection there is nothing to serialise, and a strand is a per-handler allocation and an indirect call in the hot path.
• Cross-core delivery is a queue, not a lock. The event hub runs on the NATS dispatch thread; it pushes a batch handle to each owning core's boost::asio::post queue. The core appends to its connections' rings.
• One NATS connection with per-request delegation. with_delegation shares the underlying connection and injects X-Delegated-Authorization per request, so 100,000 sessions need one NATS connection, not 100,000. The BFF opens one per signed-in browser; that decision does not survive the port.
• A real asynchronous request path. The existing nats_client request methods are synchronous, and client::request spawns a detached thread per call. So the gateway gets a request path of its own: either cnats' asynchronous request API driven by one completion thread, or a fixed pool of NATS worker threads with futures handed back to the owning core. Either way the I/O thread never blocks, and the request timeout is enforced by a timer on the core, not by the worker.
• No database work on an I/O thread. The session byte counters are aggregate per core and flushed by one writer thread on a timer, the way the market-data ingest loop already batches. This removes the synchronous PostgreSQL write that runs after every authenticated request today.

accept ─▶ slot acquire ─▶ read headers ─▶ route match ─▶ auth profile
   │                                                        │
   │                            ┌───────────────────────────┴─────────┐
   │                            ▼                                     ▼
   │                     request handler (coroutine)            stream handler
   │                       await NATS round trip                (state machine)
   │                            │                                     │
   └────────────────────────────┴──────▶ write pump ◀─────────── append ─┘
                                          │
                                     async_write ─▶ slot release (on close)

Per connection the server keeps three arrays, indexed by the same slot. The hot state is contiguous so a poll event touches one cache line.

// Array 1: hot. Touched on every event.
struct connection_hot {
    std::uint32_t generation;
    std::uint32_t socket;      // native handle, or an index into the socket array
    std::uint16_t write_head;  // ring offsets, masked to the ring size
    std::uint16_t write_tail;
    std::uint8_t  state;       // reading, dispatching, writing, streaming, closing
    std::uint8_t  profile;     // anonymous, session, bearer
    std::uint16_t core;
    session_handle session;    // invalid for anonymous and bearer
};

// Array 2: buffers. Fixed and inline, so a connection allocates nothing.
struct connection_buffers {
    beast::http::request_parser<beast::http::empty_body> parser;
    std::array<std::byte, 512>  read;
    std::array<std::byte, 4096> write;   // the ring
};

// Array 3: cold. Touched on transition only.
struct connection_cold {
    timer_handle idle_timer;
    std::uint32_t remote_address;
    std::uint16_t remote_port;
    std::uint8_t  stream_kind;   // none, sse, websocket, file
};

The three arrays are chunked in blocks of 4096 slots, so a handle stays valid while a block is reused, and a slot is three index lookups rather than one pointer chase.

The ring is inline and not a pointer on purpose. A =std::unique_ptr<std::byte[]> per connection would be one allocation per connection and one more cache miss per event, which is exactly what this layout exists to avoid.

struct connection_handle {
    std::uint32_t index : 24;      // up to 16.7 M slots
    std::uint32_t generation : 8;  // wraps; a stale handle is rejected, not followed
};

Write pump. There is exactly one in-flight async_write per connection. An append either starts it or does nothing, because the ring already holds the bytes. This is what keeps a slow client from creating a queue of futures, and it is what makes conflation possible: the ring is small, so being behind is visible immediately.

The rules, applied concretely:

1. Separate hot from cold. Section 6.2. The state machine touches one line; the buffer and the ring live behind a pointer that is only read when work starts.
2. Batch by access pattern. The 150 ms tick walks a dirty list of entities, not every subscription. Serialisation happens once per batch, not once per event.
3. Prefer indices to pointers. Connections, sessions and subscriptions are all handles into chunked arrays. A handle survives a rehash and a reconnect.
4. Small contiguous beats a hash map. A route's watchers are usually one to four; the watch list is an intrusive list in an arena, iterated linearly.
5. Keep the working set per core small. Nothing shared. Counters are per core, padded to a cache line, summed only when exported.
6. No std::function in the dispatcher. Handlers are a function pointer plus a void* context, or a template instantiation. std::function is an allocation and an indirect call that the optimiser cannot see through.
7. Timers are a wheel, not n asio timers. One hierarchical timing wheel per core handles idle timeouts, session expiry and stream keep-alives. 100,000 asio timers is 100,000 heap entries and 100,000 cancellations at shutdown.

// One wheel per core, ticked by one timerfd. O(1) insert, O(1) cancel.
class timer_wheel {
public:
    timer_handle schedule(std::chrono::milliseconds delay, timer_callback cb,
                          void* ctx) noexcept;
    void cancel(timer_handle h) noexcept;
    void expire(std::chrono::steady_clock::time_point now) noexcept;
private:
    static constexpr std::size_t slots_per_level = 512;
    // level 0: 1 ms, level 1: 512 ms, level 2: ~4 min, level 3: hours
    std::array<std::vector<wheel_entry>, slots_per_level> level0_;
};

// One open-addressed map for every watcher in the process, keyed by the batch key.
struct batch_key {
    std::uint64_t tenant_id;
    std::uint32_t component_hash;
    std::uint32_t entity_hash;
    bool operator==(const batch_key&) const = default;
};

Today the router scans every route and runs a regular expression per candidate (router.cpp:230-251), then copies the winner. At the 300-odd routes a merged table would hold, and 10,000 requests a second, that is millions of pattern comparisons a second for nothing.

Replace it with a trie built once at startup from a generated manifest:

struct route_entry {
    std::string_view method;          // interned, compared as an enum
    std::string_view path_template;   // "/api/v1/reference-data/currencies/{iso_code}"
    std::uint16_t segment_count;
    auth_profile profile;             // anonymous | session | bearer
    body_limit body;
    handler_fn handler;               // free function, not std::function
};

// Generated. Sorted, so the trie is a static table and the build is a scan.
inline constexpr std::array<route_entry, n_routes> route_table = { ... };

Matching walks the path segments through a compiled trie in O(segments), and a literal segment beats a parameter in the same node, so /api/session never loses to /api/{resource}.

The manifest is the single contract (Section 8). It produces:

• the C++ route table,
• openapi.json,
• the TypeScript client and its validators,
• a route list for the tests and the benchmark.

A route declares its profile. The pipeline is the same for both; only the proof of identity differs.

Session store.

• The cookie carries a 256-bit random identifier. The store keys on its SHA-256, so a memory dump is not a set of usable credentials.
• The record holds the bearer token, the party scope, the tenant, the account, the creation time and the last use.
• Sliding expiry, in the timing wheel. A session that expires is evicted and its subscriptions are refcount-decremented in the same step.
• Sign in mints a new identifier (no fixation). Privilege change rotates it.
• Cookie flags: HttpOnly, SameSite=Lax, Secure when the deployment has TLS. The name takes the __Host- prefix when it is Secure, because the prefix requires it; over plain HTTP the prefix-less name of today is used. Keep the per-environment suffix the BFF uses (ores_web_session_<environment>), so two deployments on one host cannot overwrite each other.

Delegation that cannot be forgotten. The reason the BFF opens one NATS connection per session is that a shared connection with a forgotten token would silently act as the service identity. Make that unrepresentable:

// A user-scoped call cannot be made without a token. The type enforces it.
class delegated_call {
public:
    delegated_call(nats_client& nats, std::string token);   // token required
    boost::asio::awaitable<message> request(std::string_view subject,
                                            std::span<const std::byte> body);
private:
    nats_client client_;   // built with with_delegation(token)
};

// Session-bearing routes receive this, never the raw client. A function
// pointer and a context, not a std::function, as 6.3 requires.
using route_handler = awaitable<http_response> (*)(const http_request&,
                                                   delegated_call&);

Bearer verification. RS256 only, algorithm pinned, iss and aud checked, the JWKS cached and refreshed on an unknown kid. A refused token is 401 with no detail. The verification cost (tens of microseconds) is paid once per request for machine clients, and never for browser sessions, which is why the session path is not merely a cache: it is a different proof.

CSRF. A cookie-only request is not proof of intent. Every unsafe method (POST, PUT, PATCH, DELETE) on a session route must carry an Origin matching the deployment's own, and a custom X-Ores-Request header that a cross-site form cannot set. SameSite=Lax is the second line, not the first.

Authorisation. Mostly unchanged: the service checks the permission, against the token the gateway delegated. But "the gateway authorises nobody" is not quite true today, and the port must not pretend otherwise. The BFF has four narrow boundaries, and each has to be reproduced deliberately or dropped deliberately:

Everything else stays the service's. The gateway reads no permission table of its own and caches none.

The hardest part, and the reason the BFF exists.

Shape. One shared NATS connection and one subscription for the whole process, over the events root: *.v1.events.>. The hub routes each message in process by (component, collection).

One subscription and not one per entity. cnats delivers a callback subscription on a library thread of its own — the client's own comment says the handler runs "on a NATS library thread" (client.cpp:351) — so a subscription per (tenant, entity) would be hundreds of threads doing nothing but waiting. A wildcard costs one thread and one decode per event, which is the work the BFF already does per subscription.

The wildcard has a second benefit the per-entity design cannot have: the hub sees every event in one place, so the dirty list and the tick are one structure rather than one timer per subscription. If a thread count under load shows cnats multiplexes after all, the wildcard is still the right shape for that reason.

Watchers are refcounted per batch key, so the process holds a batch only while somebody watches it, but the subscription itself stays open for the life of the process. That is the trade: one always-open wildcard, no subscription churn, no thread per tenant.

The event arrives on the NATS dispatch thread. The hub does not serialise per session; it serialises once per batch and copies a finished frame into each watching connection's ring.

NATS event  ──▶ mark dirty (entity, tenant, at)  ──▶ 150 ms tick
                                                        │
                             per dirty entity:  build the batch
                             ├─ keys, capped at 20
                             ├─ group watchers by party bucket
                             └─ serialise one SSE frame per bucket
                                                        │
                             per watching session: memcpy the frame into its
                             ring, post to the owner core, start the write pump

Why 150 ms is cheap. It is one timer for the whole process, and the batch is built from a dirty list, so the cost per tick is proportional to what changed, not to what is watched.

What a frame names, measured against the stores. MAX_KEYS assumes a subscription can read a row's key out of an event payload. The first ported version cannot: the subject names the collection and the action, the headers name the tenant and the party, and the key inside the payload is that entity's own key record, encoded by that entity's codec. One wildcard subscription therefore cannot decode the key of every entity in the system, and the frame names the component and the collection and nothing finer:

event: iam.tenant_types
data: {"tenant":"…","party":"…","component":"iam","entity":"tenant_types"}

A client that hears a frame refetches the collection it draws. That is what a poll does, once per burst instead of once per screen per second, and it is correct without knowing what the event was about. Keys return to the frame as an optimisation when a per-entity key decoder exists; the event names do not change when they do. The window is armed by the first change and closes one interval later, so a burst that begins between ticks is one frame per collection rather than two, and a window is never extended by what arrives inside it: a long import is a series of windows. Between windows the clock sleeps, because nothing arms it.

Filtering. The subject is the same for every tenant, so every event reaches the hub and each carries its tenant in the envelope. A session hears its own tenant and the system tenant; a member hears only the parties it works in; an administrator hears every party of its tenant. Party filtering changes the payload (the key list), which is why serialisation is per party bucket and not per entity.

Why an event carries an events namespace. An event is published on <component>.v1.events.<collection>.<action>. The namespace is a token of its own because a NATS wildcard is a whole token: the first version of this design published on <component>.v1.<collection>_events.<action>, and a subscriber to *.v1.*_events.> heard nothing at all, because *_events is a literal token that matches no subject. The server accepted the subscription, delivered nothing, and the stream was silent with no error anywhere; only a live run found it. The feed had then to subscribe *.v1.>, which copied every versioned request in the system into the gateway. The namespace removes that copy, one subscription hears the events of every component, and the cost of the correction was a new subject_prefix in every entity's event header.

The constants. Two of the BFF's four survive the port and are pinned by tests; two do not and are recorded here so the difference is not mistaken for an oversight. BATCH_WINDOW_MS = 150 is the window, armed by its first change and not extended. The watch declaration and its ^[a-z][a-z0-9_]*$ validation go with the watch list: the server's stream is per session and per party, not per arbitrary client-chosen collection. MAX_KEYS = 20 goes with the key list, for the reason in the frame-shape paragraph above. Changing the window is a client-facing behaviour change and the tests pin it.

Conflation, and why it is safe. The payload is a time, not a list. A client that misses three frames needs one fact: that something changed after its load began. So a connection that is behind does not grow a queue; the pending batch holds one frame per collection per tenancy, and a newer frame replaces the one still pending instead of queueing behind it. The replaced frame moves to the back of the batch, so the ids a client reads keep ascending and its Last-Event-ID stays a statement it can rely on. The ring is 4 KiB and holds one frame per collection at most.

Two things the BFF does not have, and the port adds. It sends no keep-alive comment on an idle stream, and it has no resume. Both are cheap and both matter once the stream is the interface's only push channel:

• Keep-alive. One comment line every 15 s, from the timing wheel. It is what keeps an idle stream distinguishable from a dead one behind a proxy.
• Resume. Core NATS has no replay. A reconnecting client sends Last-Event-ID. If the gap is inside the hub's short retention the hub replays; otherwise it sends resync once, and the client marks every open screen stale. That is exactly the semantics the client already implements, so no per-connection history is stored.

Catch-up on watch. buffered_subscription retains the last n messages per subject with a thread-safe snapshot. A session that opens a screen after a change gets a correct "changed at" without a full reload, and a new watcher does not need a subscription of its own to obtain it. Because the raw snapshot copies under a mutex, the hub takes it once per (component, entity) and shares the result, not once per session.

Backpressure policy, stated. A frame that cannot be appended because the ring is full is dropped for that entity, and the entity is marked dirty so the next tick rebuilds it. A client that is persistently behind is closed after a stated number of consecutive drops, and the client reconnects and resyncs. Dropping is safe here and would not be for an ordered command stream.

One bug to fix rather than port. Destroying a session does not remove its watches. The hub is cleared only when the stream socket closes, so a destroyed session with an open stream keeps receiving events. In C++ the session and the watch entry share a handle, and destroying one invalidates the other by construction (Section 6.3), which is the better fix than a second cleanup path.

SSE first. It is HTTP/1.1, it rides the same pipeline, it reconnects by itself, and it is one direction, which is all the design needs.

• A stream handler never returns a http_response. It hijacks the connection: the write pump stops treating the ring as one response and starts treating it as a frame queue.
• Response headers: Content-Type: text/event-stream, Cache-Control: no-cache, =X-Accel-Buffering: no.
• Idle keep-alive: one comment line — a single colon followed by a blank line — every 15 s, from the timing wheel. This is what keeps a NATS-to-browser path truthful in the presence of proxies.
• No request deadline on a stream. The keep-alive resets the idle timer, so a healthy stream is never idle for more than 15 s; a stream that cannot be written to for the stall deadline is closed, because that is a dead client and not a quiet market.

The stream above needs a body source that can wait. The synchronous body_source serves a file, which always has its next piece; an event stream does not, and reading it synchronously would hold the connection's thread. awaitable_body_source is the same contract with a co_await in it, and the session takes the loop that suits whichever source the response names.

File transfer. The grid already moves every package, input, output and log through this server over /api/v1/storage/{bucket}/{key}, with a capability token minted for one assignment. Today that path buffers: store_.put(bucket, key, req.body) takes the whole body, so a 256 MiB upload costs 256 MiB resident per concurrent connection, and the BFF by contrast never holds a file.

The port streams in both directions:

• Upload consumes a beast::http::buffer_body through a file writer with a bounded window, so memory is bounded by the window and not by the object.
• Download serves with boost::asio::stream_file and a fixed buffer, so a large object costs a buffer and not an object.
• Both enforce a per-route byte limit and a stall deadline, and both report progress to the timing wheel so an abandoned transfer is reclaimed.

This is the one place where "100,000 connections" and "one very large file" meet, and it is why the write ring is bounded rather than elastic.

WebSocket. Beast 1.92 ships it, boost-beast is already a dependency, and no new port is needed. Do not adopt it yet. It buys bidirectional traffic the design does not need, and it costs a second protocol to test and to secure. The place it will earn its keep is a future interactive grid console, and the connection slot already carries a protocol byte for it.

The HTTP/1.1 limit. Beast has no HTTP/2. A browser allows six connections per origin, and a tab holds one for its stream. One tab is fine; ten tabs are ten streams across ten connection pools and still fine; a shared kiosk is not. Two mitigations, in order: (a) one stream per session, shared by tabs through a BroadcastChannel in the SPA; (b) an HTTP/2 or HTTP/3 proxy in front, which is a deployment option and not a code change. State (a) as the default and (b) as the escape hatch. Section 6.12 makes the same proxy serve as the answer to the TLS memory question, so the two decisions reinforce each other.

• One immutable cache header for hashed filenames, no-cache for index.html.
• Sent from an open file descriptor with boost::asio::stream_file, not read into a buffer, so a 5 MiB asset does not cost 5 MiB of connection memory.
• Path resolution is a table lookup against the manifest of built files, not a string join, so .. cannot escape the root.
• Compression at build time, not per request. Serve the .br file when the client offers it.

Two allocation problems, both on the hot path:

1. wire_codec::encode returns a fresh std::vector<std::byte> per call. Add an encode-into-buffer overload driven by a per-core scratch buffer that is reset per request.
2. rfl writes into a container it owns. Wrap the writer so the destination is the connection's ring or the core's scratch, and keep the rfl types unchanged.

The codec takes a std::span<const std::byte> and does not copy the encoded bytes. The copy has already happened by then: cnats hands over a buffer and extract_message copies the subject, the reply, the data and every header into owned containers. That is one allocation and one memcpy per message, and it is worth attacking, but it belongs to ores.nats and to a later phase. Keep the decoded message alive for the request's lifetime and copy nothing again unless a route reshapes it.

• Counters, per core, cache-line padded: accepted, closed, requests by profile, requests by status class, drops, current connections and streams.
• Histograms: request duration (with and without the service time), fan-out batch size, tick duration, ring occupancy at drop.
• One log line per request, with session id, correlation id, route, status and duration; a stream logs start, end and reason.
• The existing ores.telemetry and Boost.Log conventions carry over, so the journal keeps one shape. Publish the counters as a fire-and-forget sample on a timer (Section 11); the message does not exist yet and has to be added.

Today stop() closes the acceptor, logs the live connection count, and leaves every session running until the process exits. For a server holding long-lived streams that is a mass disconnect with no explanation and no chance to say why.

The drain sequence, bounded by a deadline:

1. Stop accepting: close every acceptor, so new connections are refused by the kernel rather than by us.
2. Tell streams: write one final event: shutdown frame to every stream, then close it with a normal close. A client that understands it reconnects once, with backoff.
3. Drain requests: stop admitting new requests, let in-flight handlers finish, and close idle keep-alive connections with Connection: close.
4. Flush the counter writer and the session store, publish one last telemetry sample, then stop the io_contexts.
5. If the deadline passes, close the rest and log how many were cut off. A drain that can hang is a deployment that cannot roll.

This server faces the internet, so TLS is not an option to consider later. It is part of the design. Both schemes are supported, in one process, on two listeners, behind one route table and one session store.

The port block is allocated per worktree, one number at a time from ORES_BASE_PORT. Adding a scheme adds a number to that block; it does not add a service. The block already has overlapping names for one number (ORES_HTTP_PORT, ORES_HTTP_SERVER_PORT and ORES_CONTROLLER_SERVICE_HTTP_PORT all name 21601), so the new one is added as a distinct ORES_HTTP_TLS_PORT and the overlap is left to the backlogged port-consistency task rather than made worse here.

One SSL_CTX for the process. Session tickets and the server-side session cache live in the context, so a resumed handshake works whichever core accepts the connection. Certificate reload swaps the context, not the listeners.

Two stream types, one pipeline. The hot array from Section 6.2 is identical for both schemes. The stream lives in the cold array as a tagged union, so there is no virtual call per event and no allocation per connection:

using plain_stream = beast::tcp_stream;
using tls_stream   = beast::ssl_stream<beast::tcp_stream>;
using stream_variant = std::variant<plain_stream, tls_stream>;

// Cold storage. Two alternatives, so a variant beats a vtable: no allocation,
// no indirect call, and the compiler sees both paths.
struct connection_cold {
    stream_variant stream;
    timer_handle handshake_deadline;
    timer_handle idle_timer;
    std::uint32_t remote_address;
    std::uint16_t remote_port;
    std::uint8_t  stream_kind;
    bool tls;
};

Handshake, bounded. async_handshake(server) runs on the owning core with its own deadline from the timing wheel (default 10 s) and a per-core cap on concurrent handshakes. A connection above the cap waits in the accept backlog or is refused; it never occupies an unbounded slot. This is the TLS-era equivalent of slowloris and it is why the handshake deadline is separate from the idle one.

Protocol policy. TLS 1.2 minimum, TLS 1.3 preferred; TLS 1.0 and 1.1 refused, renegotiation refused, compression off, NULL and export suites impossible. ALPN advertises http/1.1 and nothing else: Beast has no HTTP/2, and advertising h2 would make a browser speak a protocol this server cannot answer.

Resumption is not a nicety at scale. TLS 1.3 tickets plus a session cache sized to the expected connection count. A resumed handshake is one round trip and skips the asymmetric operation, which is the difference between a few hundred and several thousand handshakes a second on one core.

Keys and certificates. ECDSA P-256 preferred, because signing is cheap; RSA-2048 accepted for compatibility. Certificate and key from files with mode 0600, or from the environment for the container path. The key never appears in a log line and never in an error body. The existing service mains already call OPENSSL_cleanup() at exit, which is the right habit to keep.

Reload without a restart. SIGHUP, or a control message on the bus, builds a new SSL_CTX and swaps it; existing connections keep their session and finish it, and the old context is released when its last connection closes. Renewing a certificate then costs zero dropped streams. Issuing the certificate is out of scope; renewing it without an outage is not.

What the plain listener is for. The grid wrapper reaches this server at --http-base-url, which is http://localhost:${ORES_HTTP_PORT} in every wrapper unit, and systemd and the container probe reach /health the same way. Three modes keep that working without exposing a plaintext surface:

The default for a deployment that names a certificate is redirect.

Session cookies are TLS-only. A session route never sets a cookie on a plain connection. On TLS the cookie is Secure and therefore carries the __Host- prefix (6.5). A plaintext response may carry a redirect and nothing else.

Where TLS terminates at 100,000 connections. Both answers are supported, and the choice is a deployment decision, not a code change:

Recommend in process as the default, and a proxy once the count passes roughly 20,000 or when HTTP/2 is wanted. That is also the answer to the HTTP/1.1 six-connection limit in 6.7: the same proxy solves both.

Client certificates. Browsers do not need them. The machine surface already carries a bearer token or a capability token, and mTLS would push certificate lifecycle onto every caller. Do not add mTLS without a caller that requires it.

Tests. A self-signed certificate generated per test run, a TLS listener in the integration suite, a test that a plain request to the TLS port fails closed, a test that a certificate reload leaves a live stream connected, and a handshake flood test that asserts the per-core cap holds.

ores.cpp.http-route projects one entity onto one route unit: header, source and tests, one route per derived verb, opt-in and disabled by default. ores.ts.protocol and ores.ts.domain emit the TypeScript wire types and entity shapes. The two sides are generated from the same org models, but the route list is not.

> Generate the shape. Hand-write the decision.

1. ores.http.manifest. Reads every opted-in operation model, plus every hand-written unit's declaration, and emits:
   • http_route_table.generated.cpp — the static table the trie is built from,
   • openapi.generated.json — replacing the runtime registry,
   • routes.generated.ts — the route constants the client imports,
   • route_manifest.generated.org — the document the recipes are written against.
2. ores.ts.client. From the same models, emit typed client functions and runtime validators. This closes the split the web README calls unfinished: the browser would validate with generated validators, not hand-written Zod, and the BFF's @ores/contracts package retires with it.
3. ores.http.resource. For a hand-written resource — a composed view, or a session that mints a credential — emit the skeleton: method, the published area rather than the component name, profile, request type, response type, and the manifest entry. The body is a hand-written function the template leaves a stub for, so the route table stays complete and the TypeScript client stays in step. The template is about the shape of a resource, and knows nothing about callers.

Authorisation, session handling, the event hub, stream framing, and error mapping. These are decisions, and a generated decision is a decision nobody can find six months later. They live in hand-written units — session and cookie in auth, the hub and framing in stream, error mapping once in domain — and the manifest records their routes so the table stays complete.

There is a real efficiency finding, and it is not in the transport. The BFF amplifies reads:

Note what this table does not say. It does not say the views are wrong. The estate summary and the blotter are exactly the composed views 11.1 keeps, and their questions are good ones. What is wrong is the computation: N+8 round trips per page load instead of one answer from the service that owns the question.

So the port keeps the view and fixes the computation. A step-progress read per run becomes one call. A read-then-write becomes one operation the service owns. Where the service cannot change yet, the gateway batches its own reads behind one request and the view carries a cache policy, so the cost is paid once and not on every poll. Porting the shapes unchanged would make the new server as inefficient as the old one, only in a faster language; deleting the shapes would throw away the only real account of how the data has to be consumed.

benchmarks/ today holds one suite, XML parsing: a hand-rolled best-of-five loop, cachegrind for the cache axis, results appended to history.tsv, and a CI gate that fails when a watched input moved without a recorded row. The HTTP benchmarks follow that shape rather than inventing a framework, because the reporting and the gate already work.

1. connection_bench: open 1, 10 and 100 thousand idle streams; record RSS, file descriptors and CPU per core. Asserts N1, N2, N6.
2. request_bench: a loopback route with no NATS call; record p50/p99/p999 and allocation counts. Asserts N3 and N7.
3. fanout_bench: one synthetic event to 10,000 watchers; record tick to last byte written. Asserts N4.
4. stream_bench: 10,000 streams receiving one frame a second for an hour; record RSS at start and end. Asserts that conflation holds and memory is flat.

Each records into history.tsv with the same long-form columns, so compare.py works unchanged, and the nightly run fails when a target regresses by more than the noise floor stated in the suite's README.

The question that decides where a route lives is not who calls it. It is what it is. There are four answers, and only three of them are an API.

1. A resource. It names a domain object or a collection of them, and it does to that object what HTTP says it may: read, list, create, replace, remove, read a version. Its shape comes from the protocol model. A machine client and a browser reach it the same way; only the credential and the representation differ.

2. A composed view. A read model. It answers a question a person actually asks — what is the state of my estate, what is on my blotter, what is waiting for my approval, how is the grid doing — by weaving several entities into one answer. The composition is domain design, not presentation. The screens found the questions; the API names them and documents them, and a CLI or a report can call the same view.

3. A transport capability. Not a domain object at all; something a client may ask of the connection. Four exist: a cookie credential instead of a bearer header, a change stream instead of a poll, the built bundle and its assets, and the liveness and metrics endpoints. None is browser-specific in its contract.

4. A rendering concern, which is not an API. Display order, colour, label text, preformatted strings, translated copy, and booleans whose only purpose is to hide a widget. This is the line the old BFF crossed, and it is the only line that matters. It belongs in the client, and a route that exists to serve it does not belong in the contract.

Three paths in the row above are targets and not what answers today. The views behind them are GET /api/overview, GET /api/blotter and GET /api/requests, which a screen calls now; the inventory at doc/knowledge/architecture/http_route_inventory.org names every view, the route it answers on, and the client module that calls it. So the port moves a working view behind the same request, and the published namespace arrives as a versioned change after the cutover rather than as part of it.

A composed view is legitimate only when it satisfies all four of these. Without them it is the old BFF again, whatever it is called.

1. A name in domain language. "Estate summary", "blotter", "approval inbox". Not "overview", not a page title, not a route that exists because a component above it renders a grid.
2. A published schema. In openapi.json, generated from a model rather than assembled by hand in the handler, so the TypeScript client and the tests read the same shape.
3. An owner. The component that answers the question, in the same place its resources live. A view that spans components still has one owner — the one that cannot give the answer alone does not own it.
4. A stated computation and a cache policy. One service call where possible; a documented batch where not; an ETag and a freshness rule so a screen that polls does not pay for the composition every time. N+8 round trips per page load, which is what /api/overview does today, is not a design, it is an unmetered cost.

Three tests for the review. Apply them to every route:

1. Is the caller in the URL or the shape? If a route exists only because a browser calls it, the difference belongs in a credential or an Accept value. The exception is a documented me alias, which is an ordinary REST idiom.
2. Would a second client use it? A composed view survives this test, because a CLI wants the blotter as much as a screen does. A payload of widget flags does not survive it.
3. Does it contain a word only a widget understands? Colour, layout, order for display, translated text, "show". If yes, the field does not ship.

What the two earlier drafts got wrong, in opposite directions. The first made the API's shape depend on the caller, which produced a browser component and a browser-only route set. The second over-corrected and called every composed view "residue", which throws away the most valuable thing the web work produced: a tested account of how the data has to be consumed. The correct position is in the middle, and it is the one above. The screen told us the question. The API answers it in domain language, and the delivery mechanism is negotiated separately.

Should a client see /api/v1/iam/sessions? No.

iam is the name of an internal component and of an architecture pattern. Neither is a promise to a client. The URL is the most expensive thing in the system to change: clients cache it, documentation quotes it, scripts and bookmarks embed it. A refactor that moves a handler between components must not move a URL, and today it would.

So the published namespace is its own artefact: a short list of domain areas, declared in the manifest, each mapped to an owning component. Today that mapping is close to one to one, and that is a consequence of a sensible decomposition, not a rule to be enforced in reverse.

Four rules follow, and the first is a codegen change:

• A component may own several areas: ores.iam publishes both auth and tenancy. Areas are not derived from components, and the two lists are allowed to diverge.
• Moving a handler between components never changes a URL.
• Renaming an area is a versioned change with a redirect for a stated period, because the old URL is somebody's bookmark.
• The route template takes the area from the model's declaration, not from {component}. Today ores.cpp.http-route writes /api/v1/<component>/<plural> from the component name, which is exactly the leak this section removes.

Shape belongs in the contract; delivery is negotiated. Two lists, and a field in the wrong list is a design bug:

The test: if changing it changes the meaning of the resource, it is shape and it is in the contract. If a different client could ask for something different of the same resource, it is delivery and it is negotiated.

Four rules, in priority order.

1. Fix what is broken before building what is missing. Two defects in today's gateway hurt now: a handler blocks the single I/O thread on every NATS call, and a large upload is held whole in memory. Neither needs any of the new design to fix, and each is a small, provable change. They come first.
2. Prove the shape before optimising for scale. The riskiest question is not "can it hold 100,000 connections" — that is understood engineering. It is "can a browser sign in, list and receive changes from this server, on the routes the API says it should". A thin skeleton answers that early and cheaply.
3. Design the contract in parallel, not in sequence. The classification and the area map need no code and gate the porting work. They start on day one.
4. Harden with something working to measure. The per-core net layer, TLS and the route table come after the skeleton, when a real client and a real benchmark exist to judge them.

Each row is a story. S is a few days, M is one to two weeks, L is more.

Three of these carry more risk than their size suggests, and should be split before they are started if the estimate holds: B3 (the hub is concurrent state), N1 (the net layer touches every connection), and V1..Vn (each view needs a service-side read model, which is not HTTP work).

Every increment, without exception:

• The full local gate is green, and the increment's own test fails if the increment is reverted.
• A benchmark exists for anything the increment claims about cost, and its number is recorded in benchmarks/history.tsv.
• The section of this document that the increment changes is amended in the same change, so the document never describes a system that no longer exists.
• The increment is revertible on its own. If it needs a flag to be safe, the flag is added and its removal is a named follow-up.

The cutover waits for parity and is one switch. The browser calls the BFF until the C++ server answers every route ores.web needs, so no request depends on both servers at once. When the work is done, we move the switch and test ores.web. If ores.web breaks, we move the switch back and use the BFF until the fault is fixed.

• One cookie name, two servers. Both run against the same NATS, the same subject prefix and the same session cookie, so a rollback does not sign anyone out.
• One reverse proxy in front. The proxy sends the browser's requests to the BFF or to the C++ server. The cutover is one line, and a rollback is the same line removed.
• The route inventory says when to try. The switch moves when the inventory's coverage count shows that the C++ server answers every route ores.web calls. The ores.web test decides whether the switch stays.
• The BFF is deleted last, and by evidence. Not by plan date: when its logs carry no traffic for a full working week.