Compute Engine

A Run Configuration (in generic grid systems called a Job Set Definition) is a template describing a logically related collection of work — for example, the full EOD P&L run for a branch. It groups one or more Computation Definitions and captures the dependencies between them. It is a definition, not an execution: it carries no runtime state.

ORE equivalent: An ORE ore.xml analytics configuration file, which lists which analytics to run and in what order.

A Computation Definition (generic: Job Definition) is a template for a single unit of computation. It specifies:

• Inputs: What data the computation needs. Inputs may be:
  • Direct values (XML blobs, JSON payloads)
  • Tokens: symbolic references resolved at runtime into concrete values (e.g. a reference to "today's EOD market data cut" resolves to a specific versioned snapshot when the computation is dispatched)
• Executor: The service responsible for running the computation (e.g. the ORE analytics service).
• Expected outputs: The result schema.

ORE equivalent: An individual analytics block in ore.xml, such as <Analytic type"NPV">= or <Analytic type"SENSITIVITY">=.

A Computation Run (generic: Job) is a runtime instance of a Computation Definition. It is created by supplying concrete values for all inputs (including resolved tokens). A Computation Run is the atomic unit of scheduled and tracked work. Runs have:

• A status (pending, running, succeeded, failed)
• A priority
• A reference to the Computation Definition it instantiates
• Zero or more dependencies on other Runs

A Batch (generic: Job Set) is a runtime instance of a Run Configuration — a collection of Computation Runs tracked and managed together. The Batch is the unit of orchestrated work submitted by a user or a scheduler.

A Run Dependency links two Computation Runs within a Batch, specifying that one must complete successfully before another may begin. Dependencies form a directed acyclic graph (DAG) within a Batch.

The Run Queue is the ordered list of ready-to-run Computation Runs — those whose dependencies have been satisfied and which are waiting for a free executor. The queue is ordered by priority. Persistent storage (e.g. an RDBMS) backs the queue to survive process restarts and node failures.

The Orchestrator (generic: Director) is the central coordinator of the compute engine. Its responsibilities:

• Monitoring all active Executors and handling their failures
• Dispatching Computation Runs from the queue to available Executors
• Receiving status and progress reports from Executors
• Managing Batch lifecycle (creation, completion, cancellation)
• Applying retry logic for failed Runs

The Orchestrator is the single authoritative source of engine state. It ensures that at most one Executor is running any given Computation Run at any time.

An Executor (also called a Worker) is a service that runs on a Compute Node and is capable of executing Computation Runs of one or more types. Executors:

• Register with the Orchestrator and report capacity
• Accept dispatches from the Orchestrator
• Invoke the appropriate Engine for the computation type
• Report progress and completion back to the Orchestrator
• Return results (or error details) on completion

The Engine is the code running inside an Executor that performs the actual computation. It is the domain-logic component:

• Quant Library: Low-level numerical primitives (e.g. QuantLib, ORE). Not suitable for direct invocation; provides building blocks for the Risk Engine.
• Risk Engine: Higher-level orchestration layer built on the Quant Library. Handles trade population loading, market data assembly, pricer dispatch, and result collection. Single-process. In ORE Studio this is ORE.

The Risk Engine is versioned independently of the execution infrastructure. Users deploy a specific version of the Risk Engine to a cluster; Computation Runs reference the engine and version they require.

The Chunker is responsible for partitioning a large input set into sub-units — chunks or partitions — of roughly equal computational cost, distributed across available Executors. Chunking happens up front before dispatch.

Different chunking strategies are required depending on the nature of the input:

• By book: partition the trade population by trading book
• By trade: partition individual trades (for fine-grained parallelism)
• By scenario: partition Monte Carlo scenarios
• By tenor: partition curve bootstrapping by maturity segment

The Chunker's goal is a fair distribution of work: each Executor should finish at approximately the same time to avoid stragglers holding up the Batch.

A Compute Node is a machine (physical or virtual) that hosts one or more Executors. It has the ability to run the Risk Engine. A Compute Node manages:

• Receiving computation inputs from the Orchestrator / message bus
• Invoking the Risk Engine
• Retrieving and returning results

A Cluster is a named group of Compute Nodes. Each cluster has:

• A defined set of authorised users
• Specific technology interfaces and protocols (message queue, REST API, etc.)
• An associated Environment label

Traders can submit ad-hoc reports at any time. These are instantiated as single-Run Batches with high priority and execute against a snapshot of current market data. Results are returned to the requesting UI session.

Runs can be scheduled to run at predefined times (e.g. AM Flash P&L at 08:00, PM Flash at 16:00, EOD batch at 18:30). The schedule is configured per Run Configuration and per environment.

Runs may be triggered automatically when a monitored condition is breached:

• Throughput Panic: Triggered when the number of transactions in a currency pair over a rolling window exceeds a configured threshold.
• Amount Breach: Triggered when a notional amount exceeds a limit.
• Market Data Move: Triggered when spot or vol moves by more than a configured threshold.

Triggered runs are recorded with a timestamp and the triggering event, creating an audit trail.