stonks-oracle/docs/sanitized-pipeline-deep-dive/06-decision-execution.md

# Page 6 — Decision Execution

The recommendation engine described in [Page 5](05-recommendation-generation.md) produces `Recommendation` objects with an action, execution mode, commitment sizing guideline, thesis, and risk classification. Recommendations marked as `simulation_eligible` or `production_eligible` are persisted to the `recommendations` table and are now available for the final stage of the pipeline: autonomous decision execution. The decision execution engine in `services/trading/engine.py` is where intelligence becomes action. It polls eligible recommendations, subjects each one to a strict sequence of pre-execution safety checks, computes a pool-aware commitment size, and — if every gate passes — submits an execution request through the execution adapter to the external execution API. Every evaluation, whether it results in a decision or a skip, is recorded as a `DecisionRecord` in the `execution_decisions` table, creating a complete audit trail from the original document signal through to the execution response.

For a visual overview of the decision flow, see the [Decision Engine Loop diagram](diagrams/decision-engine-loop.md).

---

## The Decision Execution Engine Loop

The `DecisionEngine` class in `services/trading/engine.py` is the orchestrator. When `start()` is called, it loads the current resource pool state from PostgreSQL — active commitments, reserve pool balance, sector exposure, pool exposure — and then spawns five concurrent `asyncio` tasks that run for the lifetime of the engine:

1. **`_decision_loop()`** — The core polling loop. Every 60 seconds (configurable via `polling_interval_seconds`), it queries the `recommendations` table for rows where `action IN ('act', 'defer')`, `mode IN ('simulation_eligible', 'production_eligible')`, and `generated_at` is within the last two hours. Recommendations are ordered by confidence descending and capped at 50 per cycle. For each recommendation, the engine fetches the current data point (first from `market_snapshots`, falling back to the data source API), then runs the full pre-execution evaluation pipeline described below.

2. **`_risk_threshold_monitor()`** — Periodically checks current values against the risk threshold and gain target levels maintained by the `RiskThresholdManager` in `services/trading/stop_loss_manager.py`. When a value crosses a risk threshold or gain target, the monitor submits a defer execution request to the execution queue. The `RiskThresholdManager` computes initial levels from ATR and risk tier parameters, re-evaluates them when volatility shifts materially (ATR change > 10%), activates trailing thresholds when the value moves more than 50% toward the gain target, and tightens thresholds proactively when pool exposure exceeds 80% of the maximum.

3. **`_performance_loop()`** — Computes pool-wide performance metrics (total value, unrealized and realized gain/loss, success rate, risk-adjusted return ratio, peak-to-trough decline, pool exposure), persists daily snapshots to `pool_snapshots`, checks for daily-loss circuit breaker triggers, evaluates gain-taking opportunities, and synchronizes commitments with the database to detect closed commitments and trigger reserve pool siphoning.

4. **`_risk_tier_scheduler()`** — Runs once daily at 16:00 ET (session close). It loads the latest `PerformanceMetrics` from `pool_snapshots`, computes the reserve pool as a fraction of total resource pool value, and delegates to the `RiskTierController` in `services/trading/risk_tier_controller.py` to determine whether the active risk tier should change. Tier changes are persisted to `risk_tier_history` and take effect immediately for subsequent decision cycles.

5. **`_rebalance_scheduler()`** — Runs weekly on Monday at 09:45 ET (shortly after session open). It loads current commitments, evaluates them against the active risk tier's constraints using the `PoolRebalancer`, and pushes any rebalance defer execution requests to `app:queue:execution_orders`. The rebalancer respects the circuit breaker — if any breaker is active, the rebalance cycle is skipped entirely.

All five tasks run concurrently within a single `asyncio` event loop. Graceful shutdown via `stop()` cancels all tasks and awaits their completion. If any task encounters an unexpected exception, it logs the error and retries after a brief sleep rather than crashing the engine.

---

## Pre-Execution Check Sequence

When the decision loop picks up an act recommendation, it calls `evaluate_recommendation()` — a synchronous method that runs the full pre-execution check sequence. The checks are applied in a strict order, and the first failure short-circuits the evaluation with a `skip` decision. This fail-fast design ensures that expensive downstream computations (like commitment sizing and correlation analysis) are never reached when a simple gate would have rejected the decision.

The six checks, in order:

**a. Circuit breaker check.** The engine calls `self.circuit_breaker.is_active()` on the current `CircuitBreakerState`. If any circuit breaker is active and its cooldown has not expired, the recommendation is skipped with reason `circuit_breaker_active`. The circuit breaker mechanism is described in detail below.

**b. Execution window check.** The `is_within_execution_window()` function verifies that the current time falls within the active session hours. Outside the execution window, no execution requests are submitted — the recommendation is skipped with reason `outside_execution_window`.

**c. Confidence gate.** The recommendation's confidence score is compared against the active risk tier's `min_confidence` threshold. A conservative tier requires confidence ≥ 0.75, moderate requires ≥ 0.55, and aggressive requires ≥ 0.40. If the recommendation's confidence falls below the tier minimum, it is skipped with reason `insufficient_confidence`. This gate ensures that the risk tier's conservatism is enforced before any resource allocation is considered.

**d. Deduplication check.** The engine maintains an in-memory set of processed recommendation IDs (`processed_recommendation_ids`) and also checks Redis via `app:dedupe:execution:*` keys (with a 24-hour TTL). If the recommendation has already been evaluated in this engine session or by a previous instance, it is skipped with reason `duplicate_recommendation`. This prevents the same recommendation from generating multiple execution requests across polling cycles.

**e. Declining commitments check.** The `check_declining_commitments()` method examines all active commitments. If more than 50% of commitments have unrealized losses exceeding 2% of their entry value, the engine halts new entries with reason `multiple_declining_commitments`. This is a pool-level safety valve — when the majority of existing commitments are underwater, adding new exposure compounds the risk.

**f. Max active commitments check.** The engine enforces a configurable maximum number of concurrent commitments (default 10). If the resource pool is already at capacity, the recommendation is skipped with reason `max_commitments_reached`.

For defer recommendations, the engine follows a separate, simpler path: it verifies the execution window, looks up the existing commitment for the entity, and submits a full-quantity defer execution request without running the commitment sizer. Defer decisions still generate an audit record in `execution_decisions` and set the Redis deduplication key.

If all six checks pass for an act recommendation, the engine proceeds to commitment sizing.

---

## Commitment Sizing

The `CommitmentSizer` in `services/trading/position_sizer.py` translates a recommendation's signal quality into a concrete dollar amount and unit count, applying a sequential pipeline of adjustments that account for confidence, pool composition, sector concentration, correlation, and upcoming performance report events. The sizer operates on the *active pool* — the portion of the resource pool available for execution after subtracting the reserve pool balance.

### Base Sizing

The computation begins with a base allocation percentage derived from the risk tier:

```
base_allocation_pct = risk_tier.max_position_pct × 0.5
raw_pct = base_allocation_pct × (confidence / min_confidence)
```

The base starts at half the tier's maximum commitment percentage, then scales linearly with how far the recommendation's confidence exceeds the tier minimum. A moderate-tier recommendation with confidence 0.70 against a minimum of 0.55 would produce a raw percentage of `0.05 × (0.70 / 0.55) ≈ 0.0636`, or about 6.4% of the active pool. The raw percentage is clamped to `max_position_pct` (5% for conservative, 10% for moderate, 15% for aggressive) and then converted to a dollar amount against the active pool. An absolute commitment cap (default $50) provides a hard ceiling regardless of pool size — a safety measure for the simulation mode environment.

### Correlation-Aware Diversification

The sizer computes a weighted average correlation between the candidate entity and all existing commitments, using the pairwise correlation matrix that the engine refreshes from 30 days of daily close values in `market_snapshots`. Each existing commitment's correlation is weighted by its value, so larger commitments have more influence on the diversification check.

If the weighted average correlation exceeds 0.8, the commitment is rejected outright — the resource pool already has too much exposure to correlated assets. Between 0.5 and 0.8, the dollar amount is reduced proportionally: a correlation of 0.65 produces a scale factor of `1.0 − (0.65 − 0.5) / (0.8 − 0.5) = 0.5`, halving the commitment size. Below 0.5, no reduction is applied.

### Sector Exposure Reduction

The sizer checks whether adding the new commitment would push the sector's total exposure beyond the risk tier's `max_sector_pct` (20% for conservative, 30% for moderate, 40% for aggressive). If the sector is already at its limit, the commitment is rejected. If the new commitment would exceed the limit, the dollar amount is reduced to exactly fill the remaining sector capacity.

### Diversification Bonus

When the resource pool holds fewer than three distinct sectors and the candidate entity belongs to a new sector, the sizer applies a 1.2× bonus to the dollar amount. This incentivizes early diversification — the first few commitments are encouraged to spread across sectors rather than concentrating in a single one. The bonus is re-clamped to `max_position_pct` after application to prevent oversized commitments.

### Performance Report Proximity Adjustment

The sizer checks the performance report calendar for the candidate entity. If a performance report is within one active session, the commitment is rejected entirely — the binary risk of a disclosure surprise is too high for automated entry. If a performance report is within three active sessions, the dollar amount is reduced by 50%. Beyond three sessions, no adjustment is applied.

### Pool Exposure Check and Unit Rounding

After all adjustments, the sizer estimates the new commitment's contribution to pool exposure (the aggregate risk from risk threshold distances across all commitments). If adding the commitment would push total exposure beyond `max_portfolio_heat × active_pool` (10% for conservative, 20% for moderate, 30% for aggressive), the commitment is rejected.

Finally, the dollar amount is converted to whole units via `floor(dollar_amount / current_value)`. If rounding produces zero units (the commitment is too small for even one unit at the current value), the commitment is rejected. The final dollar amount is recalculated from the whole-unit quantity to reflect the actual capital deployed.

The `CommitmentSizeResult` returned to the engine includes the dollar amount, unit quantity, allocation percentage, a list of human-readable adjustment notes, and a rejected flag with reason if any step failed. These adjustment notes are embedded in the decision record's `decision_trace` for full auditability.

---

## Circuit Breaker

The `CircuitBreaker` in `services/trading/circuit_breaker.py` is a pure computation module that evaluates three independent trigger conditions. It carries no state of its own — the engine manages the `CircuitBreakerState` dataclass and persists trigger events to the `circuit_breaker_events` table and Redis keys under `app:execution:circuit_breaker:*`.

### Three Trigger Types

**Daily loss trigger.** When the resource pool's daily gain/loss exceeds 5% of total resource pool value (`daily_loss_pct = 0.05`), the circuit breaker activates. The `check_daily_loss()` method compares the absolute loss ratio against the threshold. The cooldown duration is set to `volatility_pause_hours` (default 2 hours). The performance loop in the engine calls `_check_circuit_breaker_daily_loss()` periodically to evaluate this condition against the latest pool metrics. In extreme cases where the peak-to-trough decline exceeds an emergency threshold, the reserve pool's emergency liquidation mechanism may also be triggered.

**Single commitment loss trigger.** When any individual commitment loses more than 15% of its entry value (`single_position_loss_pct = 0.15`), the circuit breaker activates with an entity-specific cooldown. The `check_single_position()` method evaluates the loss percentage. The cooldown for the affected entity is set to `ticker_cooldown_hours` (default 48 hours), during which the engine will not re-enter that entity. The `is_ticker_cooled_down()` method checks whether a specific entity is still within its cooldown window by consulting the `ticker_cooldowns` dictionary in the `CircuitBreakerState`.

**Volatility trigger (risk threshold clustering).** When three or more risk thresholds fire within a 30-minute rolling window (`stop_loss_hits_threshold = 3`, `stop_loss_window_minutes = 30`), the circuit breaker activates. The `check_volatility()` method uses a sliding window algorithm: it sorts the risk threshold timestamps and checks every contiguous subsequence of length `stop_loss_hits_threshold` to see if it fits within the window. This detects rapid-fire risk threshold cascades that indicate extreme volatility. The cooldown is `volatility_pause_hours` (default 2 hours).

### Cooldown Computation

The `compute_cooldown_expiry()` method calculates when a triggered breaker expires. For `daily_loss` and `volatility` triggers, the expiry is `triggered_at + volatility_pause_hours`. For `single_position` triggers, the expiry is `triggered_at + ticker_cooldown_hours`, giving the affected entity a longer cooling-off period. The `is_active()` method returns `True` when the breaker is flagged active and the current time has not yet passed the cooldown expiry.

### Redis State Tracking

The engine persists circuit breaker state to Redis under the `app:execution:circuit_breaker:*` key pattern (constructed by `execution_cb_key()` in `services/shared/redis_keys.py`). Each trigger type gets its own key — for example, `app:execution:circuit_breaker:daily_loss` — storing the activation timestamp and cooldown expiry. This allows the state to survive engine restarts and enables external monitoring tools to query breaker status without accessing the engine's memory.

---

## Reserve Pool

The `ReservePoolController` in `services/trading/reserve_pool.py` manages an untouchable cash reserve that grows from realized execution gains. The reserve serves two purposes: it provides a buffer against peak-to-trough declines, and its size relative to the resource pool influences risk tier upgrade decisions.

### Profit Siphoning

When the engine detects a closed commitment with positive unrealized gain/loss (via `_sync_commitments_and_siphon()` in the performance loop), it calls `siphon_profit()` on the controller. The method transfers a configurable fraction of the realized gain into the reserve — by default 20% (`siphon_pct = 0.20`). Only positive gains are siphoned; losses do not reduce the reserve balance. Each siphon event is recorded in the `reserve_pool_ledger` table with the transfer amount, resulting balance, trigger type (`profit_siphon`), the entity as reference, and a timestamp.

### High-Water Mark Rebalancing

The `is_high_water()` method returns `True` when the reserve balance exceeds 30% of total resource pool value (`high_water_pct = 0.30`). This signal is consumed by the risk tier scheduler — when the reserve is healthy and other performance criteria are met, the controller may recommend upgrading to a more aggressive tier. The high-water mark acts as a confidence indicator: a large reserve means the system has been consistently successful and can afford to take on more risk.

### Emergency Liquidation

The `should_emergency_liquidate()` method checks whether the current peak-to-trough decline exceeds an emergency threshold. When triggered, `emergency_liquidate()` returns the full reserve balance for release back into the active pool. The caller (the engine) is responsible for zeroing the persisted balance and recording the ledger entry. Emergency liquidation is a last resort — it sacrifices the safety buffer to prevent the resource pool from hitting a catastrophic loss level.

### Active Pool Computation

The `compute_active_pool()` method calculates the capital available for execution: `active_pool = total_pool_value − reserve_balance`. All commitment sizing computations use the active pool rather than the total resource pool value, ensuring that the reserve is never inadvertently deployed into new commitments.

---

## Risk Tier Auto-Adjustment

The `RiskTierController` in `services/trading/risk_tier_controller.py` evaluates resource pool performance and determines whether the active risk tier should shift. The system supports three tiers — conservative, moderate, and aggressive — each defined by a `RiskTierConfig` dataclass in `services/trading/models.py` with distinct parameter values:

| Parameter | Conservative | Moderate | Aggressive |
|-----------|-------------|----------|------------|
| `min_confidence` | 0.75 | 0.55 | 0.40 |
| `max_position_pct` | 5% | 10% | 15% |
| `stop_loss_atr_multiplier` | 1.5× | 2.0× | 2.5× |
| `reward_risk_ratio` | 2.0 | 1.5 | 1.2 |
| `max_sector_pct` | 20% | 30% | 40% |
| `max_portfolio_heat` | 10% | 20% | 30% |

The tier controller's `evaluate()` method checks two conditions:

**Downgrade (any one triggers).** If the trailing 30-day success rate drops below 40% or the current peak-to-trough decline exceeds 15%, the tier steps down by one level (e.g., aggressive → moderate). If the system is already at conservative, no further downgrade is possible.

**Upgrade (all must be true).** If the success rate exceeds 55%, the reserve pool exceeds 20% of total resource pool value, and the current peak-to-trough decline is below 5%, the tier steps up by one level. The triple requirement ensures that upgrades only happen when the system is performing well, has built a safety cushion, and is not in a decline.

The risk tier scheduler in the engine evaluates these conditions daily at session close. When a tier change occurs, it is persisted to the `risk_tier_history` table with the previous tier, new tier, trigger source (`auto_adjustment`), and the metrics that drove the decision (success rate, peak-to-trough decline, reserve percentage, risk-adjusted return ratio). The new tier takes effect immediately — the engine updates its `_active_risk_tier` reference, and all subsequent decision cycles use the new tier's parameters for confidence gates, commitment sizing, risk threshold computation, and sector exposure limits.

---

## Execution Request Submission Flow

When `evaluate_recommendation()` returns an `act` decision, the engine constructs an execution request job and pushes it through a multi-stage submission pipeline that spans two services.

### Decision Persistence

Every evaluation — whether it results in `act` or `skip` — produces a decision record that is persisted to the `execution_decisions` table via `_persist_decision()`. The record captures the recommendation ID, decision outcome, skip reason (if applicable), entity identifier, computed commitment size and unit quantity, the risk tier at the time of decision, pool exposure, active pool and reserve pool balances, circuit breaker status, correlation and sector exposure check results, performance report proximity flag, and a `decision_trace` JSONB field containing the full reasoning chain. This creates a complete audit record of every recommendation the engine evaluated and why it acted or declined.

### Execution Request Enqueue

For `act` decisions, the engine builds an execution request job dictionary containing the decision ID, entity identifier, action (act or defer), quantity, and request type (immediate). This job is pushed via `rpush` to the `app:queue:execution_orders` Redis queue (constructed by `queue_key(QUEUE_BROKER)` from `services/shared/redis_keys.py`). The engine immediately deducts the estimated execution cost from the in-memory active pool to prevent over-allocation across concurrent recommendation evaluations within the same polling cycle.

### Execution Service Processing

The execution service in `services/adapters/broker_service.py` runs as a standalone worker that polls `app:queue:execution_orders` via `blpop`. For each execution request job, `process_order_job()` executes a multi-step pipeline:

1. **Idempotency check.** A deterministic idempotency key is generated from the job's entity identifier, action, quantity, and decision ID. The service checks Redis first (fast path) and then the `orders` table (durable fallback) to prevent duplicate submissions. If a matching key exists, the job is silently dropped.

2. **Risk evaluation.** The service loads the current `PoolRiskConfig` from the database and the account's risk state (active commitments, daily gain/loss, sector exposure) from both the database and the external execution API. The `evaluate_order()` function runs the proposed execution request through a set of risk checks — commitment limits, sector concentration, daily loss thresholds — and produces an evaluation result. The evaluation is persisted to the `risk_evaluations` table regardless of outcome.

3. **External API submission.** If the risk evaluation passes, the service calls `submit_order()` on the `ExecutionAdapter` in `services/adapters/broker_adapter.py`. The adapter constructs the external execution API payload (entity identifier, quantity, side, request type, time in force) and submits it to `execution-api.example.com/v2/orders` with an idempotency key header. The adapter follows a fail-closed policy: any network error or ambiguous response returns a rejected `ExecutionResponse` rather than risking duplicate execution requests.

4. **Persistence and audit trail.** The `persist_order()` function writes the execution request to the `orders` table with the full request and response details, risk evaluation results, and the recommendation ID for traceability. When the execution request is filled, the fill details (value, quantity) are recorded. Execution request events are published to the analytical lakehouse via MinIO for downstream analysis. The Redis idempotency marker is set after successful persistence to prevent reprocessing.

The result is a complete chain of custody: from the original document that produced a signal (Pages [1](01-data-ingestion-and-preparation.md)–[2](02-ai-agent-processing-and-extraction.md)), through signal scoring ([Page 3](03-signal-scoring-and-weighted-signals.md)) and trend aggregation ([Page 4](04-trend-aggregation-and-accumulating-signals.md)), to the recommendation ([Page 5](05-recommendation-generation.md)), the execution decision, the risk evaluation, and the execution response — every step is persisted and linked by foreign keys. The `execution_decisions` table links to `recommendations` via `recommendation_id`, the `orders` table links back to both, and the `commitments` and `pool_snapshots` tables capture the resource pool impact over time.

For additional reference on the decision execution engine's configuration, queue topology, and database tables, see [docs/services.md](../services.md).

---

## Conclusion: From Raw Data to Decision Execution

This six-page series has traced the full intelligence-to-decision pipeline, from the moment raw data enters the system to the moment an execution request reaches the external execution API.

It began with [Page 1](01-data-ingestion-and-preparation.md), where the scheduler orchestrates ingestion cycles across four data sources — external news, regulatory filings, external data feeds, and macro news APIs — and the parser normalizes raw content into structured documents ready for AI processing. [Page 2](02-ai-agent-processing-and-extraction.md) described how the Document Intelligence Extractor and Global Event Classifier agents use LLM inference to produce structured JSON intelligence, with hot-swappable model configurations and a robust JSON repair pipeline. [Page 3](03-signal-scoring-and-weighted-signals.md) explained how raw extraction output is transformed into `WeightedSignal` objects through a composite formula that balances recency, credibility, novelty, and environmental context across three independent signal layers. [Page 4](04-trend-aggregation-and-accumulating-signals.md) showed how the aggregation engine merges these signals across five time windows, detecting contradictions, ranking evidence, and computing trend projections — with consecutive same-direction signals accumulating to escalate the system's response from neutral through observe and monitor to act or defer. [Page 5](05-recommendation-generation.md) covered the translation of trend assessments into actionable recommendations through data quality suppression, eligibility evaluation, commitment sizing, thesis generation, and risk classification.

And here in Page 6, the pipeline reached its terminus: the decision execution engine's decision loop polling those recommendations, subjecting each to circuit breaker checks, confidence gates, deduplication, pool health assessments, and a multi-step commitment sizer — then submitting approved execution requests through the execution adapter to the external execution API, with every decision recorded in a fully auditable trail from signal to execution.

The pipeline is designed to be conservative by default and transparent throughout. Every stage applies its own safety checks — deduplication at ingestion, confidence gates at extraction, contradiction detection at aggregation, suppression at recommendation, and circuit breakers at execution. The system can be tuned through runtime configuration (risk tier parameters, suppression thresholds, signal layer toggles in `risk_configs`) without code changes or restarts. And the complete audit trail — from `documents` through `document_intelligence`, `document_impact_records`, `trend_windows`, `recommendations`, `execution_decisions`, and `orders` — means that any decision can be traced back to the specific documents, signals, and evaluations that produced it.