Files
stonks-oracle/.kiro/specs/autonomous-trading-engine/tasks.md
T
Celes Renata 8050f4a03b chore: mark all Phase 2 tasks (27-37) as complete
All 152 tasks across both phases are now marked complete:
- Phase 1 (1-26): pure computation modules, property tests, API, frontend, infra
- Phase 2 (27-37): live decision loop, stop-loss monitor, performance metrics,
  risk tier scheduler, rebalancer, notification dispatch, backtest replay,
  real DB connections, paper trading config, integration tests
2026-04-16 07:27:35 +00:00

963 lines
66 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Implementation Plan: Autonomous Trading Engine
## Overview
This plan implements a fully autonomous trading engine as a new service (`services/trading/`) that consumes recommendations from the existing three-layer signal aggregation pipeline, applies confidence-based position sizing with reserve pool management, enforces dynamic stop-loss/take-profit levels, manages circuit breakers, and submits orders through the existing Broker Service queue. The implementation extends existing services (broker_service, recommendation, risk engine, query API, dashboard) without replacing them. Tasks are ordered so each step builds on the previous, with property-based tests validating core computation logic early.
## Tasks
- [x] 1. Database migration and shared infrastructure
- [x] 1.1 Create PostgreSQL migration `infra/migrations/018_autonomous_trading_engine.sql`
- Add `trading_engine_config` table with all configuration fields (enabled, paused, risk_tier, reserve_siphon_pct, polling intervals, gradual entry params, circuit breaker thresholds, active pool minimum, emergency drawdown threshold, correlation thresholds, earnings windows, micro-trading params, notification settings, timestamps)
- Add `reserve_pool_ledger` table with amount, balance_after, trigger_type (profit_siphon, emergency_liquidation, manual_adjustment, initial), reference_id, notes, created_at; index on created_at DESC
- Add `risk_tier_history` table with previous_tier, new_tier, trigger_source, trigger_metrics JSONB, created_at; index on created_at DESC
- Add `circuit_breaker_events` table with trigger_type (daily_loss, single_position, volatility, manual), threshold_value, actual_value, ticker, cooldown_expires, resolved_at, created_at; partial index on active (unresolved) events
- Add `trading_decisions` table with recommendation_id FK, decision, skip_reason, ticker, computed_position_size, computed_share_quantity, risk_tier_at_decision, portfolio_heat_at_decision, active_pool_at_decision, reserve_pool_at_decision, circuit_breaker_status, correlation_check_result JSONB, sector_exposure_check_result JSONB, earnings_proximity_flag, is_micro_trade, decision_trace JSONB, created_at; indexes on ticker, recommendation_id, decision
- Add `position_stop_levels` table with ticker, entry_price, stop_loss_price, take_profit_price, trailing_stop_active, atr_value, atr_multiplier, reward_risk_ratio, signal_confidence, is_micro_trade, active, timestamps; partial index on active positions
- Add `portfolio_snapshots` table with snapshot_date (UNIQUE), portfolio_value, active_pool, reserve_pool, daily_return, cumulative_return, unrealized_pnl, realized_pnl, win/loss counts, win_rate, sharpe_ratio, max_drawdown, current_drawdown_pct, portfolio_heat, risk_tier, positions JSONB, metrics JSONB, created_at; index on snapshot_date DESC
- Add `backtest_runs` table with start_date, end_date, initial_capital, risk_tier, config JSONB, result metrics, equity_curve JSONB, status, completed_at, created_at
- Add `backtest_trades` table with backtest_id FK (CASCADE), ticker, side, entry/exit prices, quantity, pnl, dates, hold_duration_days, recommendation_id; index on backtest_id
- Add `tax_lots` table with ticker, quantity, cost_basis_per_share, acquisition_date, status (open/closed/washed), closed_date, exit_price, realized_pnl, wash_sale_flag, wash_sale_details, order_id FK; indexes on ticker and open status
- Add `earnings_calendar` table with ticker, earnings_date, source, confirmed, timestamps; UNIQUE on (ticker, earnings_date); indexes on date and ticker
- Add `correlation_matrix_cache` table with ticker_a, ticker_b, correlation_coefficient, lookback_days, computed_at; UNIQUE on (ticker_a, ticker_b)
- Add `notifications` table with channel (sms/email), event_type, message, delivery_status (pending/delivered/failed/rate_limited), retry_count, error_message, created_at, delivered_at; indexes on created_at and event_type
- Insert default trading_engine_config row with moderate tier defaults
- Insert initial reserve_pool_ledger entry with balance 0.0 and trigger_type 'initial'
- _Requirements: 18.1, 18.2, 18.3, 18.4, 16.1_
- [x] 1.2 Add new Pydantic schemas and enums to `services/shared/schemas.py`
- Add `TradingDecisionType` enum (act, skip)
- Add `CircuitBreakerTriggerType` enum (daily_loss, single_position, volatility, manual)
- Add `ReservePoolTriggerType` enum (profit_siphon, emergency_liquidation, manual_adjustment, initial)
- Add `NotificationChannel` enum (sms, email)
- Add `RiskTierName` enum (conservative, moderate, aggressive)
- _Requirements: 5.1, 6.1, 3.1, 19.1_
- [x] 1.3 Add trading-related Redis keys to `services/shared/redis_keys.py`
- Add `QUEUE_TRADING_DECISIONS = "trading_decisions"` queue name
- Add `TRADING_DEDUPE_PREFIX` for recommendation deduplication (`stonks:dedupe:trading`)
- Add `TRADING_CB_PREFIX` for circuit breaker state (`stonks:trading:circuit_breaker`)
- Add `TRADING_NOTIFICATION_RATE` for notification rate limiting (`stonks:trading:notification_rate`)
- _Requirements: 1.5, 6.4, 19.7_
- [x] 1.4 Add `TradingConfig` dataclass to `services/shared/config.py`
- Add `TradingConfig` with fields: enabled, risk_tier, reserve_siphon_pct, polling_interval_seconds, stop_loss_check_interval_seconds, fast_stop_loss_interval_seconds, gradual_entry_tranches, gradual_entry_threshold_dollars, absolute_position_cap, active_pool_minimum, emergency_drawdown_threshold_pct, reserve_high_water_pct, micro_trading_enabled, micro_trading_interval_seconds, micro_trading_allocation_cap_pct, micro_trading_max_daily, micro_trading_max_hold_minutes, sns_topic_arn, sns_phone_number, gmail_sender, gmail_recipient
- Add `trading: TradingConfig` field to `AppConfig` with env var loading in `load_config()`
- _Requirements: 16.1, 20.1, 19.5, 19.6_
- [x] 2. Checkpoint — Ensure migration and shared schemas are consistent
- Ensure all tests pass, ask the user if questions arise.
- [x] 3. Core data models and risk tier configuration
- [x] 3.1 Create `services/trading/__init__.py` and `services/trading/models.py`
- Create the `services/trading/` package directory
- Define `RiskTierConfig` dataclass with fields: name, min_confidence, max_position_pct, stop_loss_atr_multiplier, reward_risk_ratio, max_sector_pct, max_portfolio_heat
- Define `RISK_TIER_DEFAULTS` dict mapping conservative/moderate/aggressive to their default `RiskTierConfig` instances per the design specification
- Define `PortfolioState` dataclass with fields: positions (list), total_value, cash, active_pool, reserve_pool, sector_exposure (dict), portfolio_heat, open_position_count
- Define `TradingDecision` dataclass with all fields matching the `trading_decisions` table schema
- Define `PositionSizeResult` dataclass with dollar_amount, share_quantity, allocation_pct, adjustments list, rejected flag, rejection_reason
- Define `StopLevels` dataclass with stop_loss_price, take_profit_price, trailing_stop_active, atr_value, atr_multiplier, reward_risk_ratio, last_updated
- Define `OpenPosition` dataclass with ticker, quantity, entry_price, current_price, unrealized_pnl, market_value, sector, stop_loss_price, take_profit_price, signal_confidence, is_micro_trade
- Define `ClosedTrade` dataclass with ticker, entry_price, exit_price, quantity, pnl, pnl_pct, hold_duration, recommendation_id, is_micro_trade
- Define `PerformanceMetrics` dataclass with all fields from the design (total_portfolio_value through computed_at)
- Define `CircuitBreakerState` dataclass with active, trigger_type, triggered_at, cooldown_expires, ticker_cooldowns dict
- Define `ReservePoolState` dataclass with balance, total_deposits, total_withdrawals, last_updated
- Define `StopTrigger` dataclass with ticker, trigger_type (stop_loss/take_profit), current_price, trigger_price
- _Requirements: 5.1, 1.2, 2.1, 4.1, 6.1, 3.1, 13.1, 14.1_
- [x] 3.2 Write property test for risk tier default parameters
- **Property 29 (partial): Persistence round-trip for risk tier configs**
- Verify all three tier defaults have valid parameter ranges (min_confidence in [0,1], max_position_pct in (0,1], etc.)
- Verify conservative < moderate < aggressive for min_confidence thresholds (inverse) and position limits
- **Validates: Requirements 5.1**
- [x] 4. Position Sizer implementation
- [x] 4.1 Implement `services/trading/position_sizer.py`
- Implement `PositionSizer` class with `compute()` method accepting confidence, ticker, sector, current_price, active_pool, risk_tier, portfolio_state, correlation_matrix, earnings_calendar
- Implement sizing formula: `raw_pct = base_allocation_pct * (confidence / min_confidence) * multiplier`, clamped to max_position_pct, then dollar_amount = active_pool * clamped_pct, clamped to absolute_position_cap
- Implement confidence gate: reject when confidence < risk_tier.min_confidence
- Implement correlation reduction: compute weighted average correlation with existing positions; reduce proportionally when avg > 0.5; reject entirely when avg > 0.8
- Implement sector exposure reduction: reduce allocation if adding position would push sector above max_sector_pct
- Implement diversification bonus: 1.2x multiplier for under-represented sectors when portfolio holds < 3 sectors
- Implement earnings proximity: reduce by 50% within 3 trading days; reject within 1 trading day
- Implement portfolio heat check: reject if current heat + new position heat exceeds max_portfolio_heat
- Implement active pool minimum: reject new entries when Active Pool < configured minimum ($100 default)
- Implement absolute cap enforcement and share rounding (round down to whole shares, reject if quantity = 0)
- _Requirements: 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 9.2, 9.3, 9.4, 9.5, 10.2, 10.3, 13.1, 13.2, 3.5_
- [x] 4.2 Write property test for position sizing formula and invariants
- **Property 1: Position sizing formula and invariants**
- Generate random confidence values, Active Pool balances, stock prices, and RiskTierConfig objects
- Verify zero allocation when confidence < min_confidence
- Verify allocation never exceeds max_position_pct or absolute_position_cap
- Verify share quantity is rounded down to whole shares
- Verify rejection when rounded quantity is zero
- **Validates: Requirements 2.1, 2.2, 2.3, 2.4, 2.7**
- [x] 4.3 Write property test for correlation-based allocation adjustment
- **Property 2: Correlation-based allocation adjustment**
- Generate random correlation matrices and portfolio positions
- Verify allocation reduced when weighted avg correlation > 0.5
- Verify trade rejected when weighted avg correlation > 0.8
- Verify allocation unchanged when weighted avg correlation <= 0.5
- Verify monotonic non-increase: higher correlation → lower or equal allocation
- **Validates: Requirements 2.5, 9.2, 9.3**
- [x] 4.4 Write property test for sector exposure enforcement
- **Property 3: Sector exposure computation and enforcement**
- Generate random portfolios with sector labels
- Verify sector exposure equals sum of market values per sector
- Verify allocation reduced when adding position would exceed max_sector_pct
- **Validates: Requirements 2.6, 9.4**
- [x] 4.5 Write property test for diversification bonus
- **Property 4: Diversification bonus for under-represented sectors**
- Generate portfolios with varying sector counts
- Verify 1.2x bonus applied when portfolio has < 3 sectors and trade is in new sector
- Verify no bonus when portfolio has >= 3 sectors
- **Validates: Requirements 9.5**
- [x] 4.6 Write property test for Active Pool computation
- **Property 5: Active Pool computation invariant**
- Generate random total_portfolio_value and reserve_pool_balance
- Verify Active Pool = total_portfolio_value - reserve_pool_balance
- **Validates: Requirements 3.3**
- [x] 4.7 Write property test for earnings proximity adjustments
- **Property 19: Earnings proximity adjustments**
- Generate random earnings dates relative to current date
- Verify 50% reduction within 3 trading days
- Verify rejection within 1 trading day
- Verify normal sizing outside earnings window
- **Validates: Requirements 10.2, 10.3**
- [x] 4.8 Write property test for portfolio heat computation and enforcement
- **Property 24: Portfolio heat computation and threshold enforcement**
- Generate random open positions with entry prices and stop-loss levels
- Verify heat = sum of position_value * (entry_price - stop_loss_price) / entry_price
- Verify new entries rejected when heat exceeds max_portfolio_heat
- **Validates: Requirements 13.1, 13.2**
- [x] 4.9 Write property test for Active Pool minimum halts entries
- **Property 7: Active Pool minimum halts new entries but allows exits**
- Generate portfolio states with Active Pool below and above minimum
- Verify buy orders rejected when Active Pool < minimum
- Verify sell orders allowed regardless of Active Pool
- **Validates: Requirements 3.5**
- [x] 5. Checkpoint — Ensure position sizer logic and property tests pass
- Ensure all tests pass, ask the user if questions arise.
- [x] 6. Stop-Loss Manager implementation
- [x] 6.1 Implement `services/trading/stop_loss_manager.py`
- Implement `StopLossManager` class with `compute_initial_levels()` method: stop_loss = entry_price - (ATR * stop_loss_atr_multiplier), take_profit = entry_price + (stop_distance * reward_risk_ratio)
- Implement `re_evaluate_levels()` method: adjust if ATR changed > 10% or signal conditions changed; respect configurable interval (default 5 min)
- Implement `check_price_crossings()` method: return list of StopTrigger for positions where current price <= stop_loss or >= take_profit
- Implement trailing stop logic: when price moves favorably by > 50% of take-profit distance, move stop-loss to entry price (breakeven)
- Implement earnings proximity tightening: 0.7x ATR multiplier when earnings within 3 trading days
- Implement high-severity event tightening: 0.5x normal ATR multiplier during active macro events
- Implement proactive heat tightening: tighten stops on lowest-confidence positions when heat > 80% of max
- Implement price data unavailability safety: close position if price unavailable > 15 minutes during market hours
- Persist all stop levels and adjustments to `position_stop_levels` table
- _Requirements: 4.1, 4.2, 4.3, 4.4, 4.5, 4.6, 4.7, 4.8, 7.2, 10.2, 13.3_
- [x] 6.2 Write property test for stop-loss and take-profit initial computation
- **Property 9: Stop-loss and take-profit initial computation**
- Generate random entry prices, ATR values, and RiskTierConfig objects
- Verify stop-loss = entry_price - (ATR * multiplier) and always below entry
- Verify take-profit = entry_price + (stop_distance * reward_risk_ratio) and always above entry
- **Validates: Requirements 4.1, 4.2**
- [x] 6.3 Write property test for price crossing triggers
- **Property 10: Price crossing triggers immediate sell**
- Generate random positions with stop/take-profit levels and current prices
- Verify sell triggered when price <= stop_loss or >= take_profit
- Verify no trigger when price is between stop_loss and take_profit
- **Validates: Requirements 4.4, 4.5**
- [x] 6.4 Write property test for trailing stop activation
- **Property 11: Trailing stop activation at 50% of take-profit distance**
- Generate random positions with varying favorable price moves
- Verify trailing stop activates (stop moves to entry) when move > 50% of TP distance
- Verify trailing stop does not activate when move <= 50%
- **Validates: Requirements 4.6**
- [x] 6.5 Write property test for stop tightening during high-severity events
- **Property 15: Stop tightening during high-severity events**
- Generate random positions and ATR values
- Verify tightened stop uses 0.5x normal multiplier
- Verify tightened stop is closer to current price than normal stop
- **Validates: Requirements 7.2**
- [x] 6.6 Write property test for proactive stop tightening at 80% heat
- **Property 25: Proactive stop tightening at 80% heat threshold**
- Generate portfolios with heat near the threshold
- Verify lowest-confidence positions get stops tightened first
- **Validates: Requirements 13.3**
- [x] 7. Reserve Pool Controller implementation
- [x] 7.1 Implement `services/trading/reserve_pool.py`
- Implement `ReservePoolController` class with `siphon_profit()` method: transfer configured percentage of realized profit to reserve, persist to `reserve_pool_ledger`
- Implement `emergency_liquidate()` method: release entire reserve into active pool, log event, persist to ledger
- Implement `compute_active_pool()` method: total_portfolio_value - reserve_pool_balance
- Implement `get_state()` method: load current balance and history from PostgreSQL
- Implement high-water mark detection: signal when reserve > 30% of total portfolio
- _Requirements: 3.1, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7_
- [x] 7.2 Write property test for reserve pool siphon computation
- **Property 6: Reserve pool siphon computation**
- Generate random realized profit amounts and siphon percentages
- Verify transferred amount = realized_profit * siphon_pct
- Verify balance_after = previous_balance + transferred_amount
- **Validates: Requirements 3.1, 3.2**
- [x] 7.3 Write property test for emergency drawdown triggers reserve liquidation
- **Property 8: Emergency drawdown triggers reserve liquidation**
- Generate portfolio states with drawdowns above and below emergency threshold
- Verify reserve liquidated into active pool when drawdown exceeds threshold
- Verify risk tier set to conservative after emergency liquidation
- **Validates: Requirements 3.6**
- [x] 8. Circuit Breaker implementation
- [x] 8.1 Implement `services/trading/circuit_breaker.py`
- Implement `CircuitBreaker` class with `check_daily_loss()`: activate when portfolio drops > configured daily_loss_pct
- Implement `check_single_position()`: close position and apply ticker cooldown when loss > configured single_position_loss_pct
- Implement `check_volatility()`: pause trading when 3+ positions hit stop-losses within 30-minute window
- Implement `is_ticker_cooled_down()`: check per-ticker re-entry cooldowns
- Implement `is_active()`: return whether any circuit breaker is currently active
- Implement cooldown expiry: auto-resolve when current time > triggered_at + cooldown_duration
- Persist all circuit breaker events to `circuit_breaker_events` table
- Store active state in Redis for fast lookup
- _Requirements: 6.1, 6.2, 6.3, 6.4, 6.5, 6.6_
- [x] 8.2 Write property test for circuit breaker activation
- **Property 13: Circuit breaker activation**
- Generate random portfolio states with varying daily losses, position losses, and stop-loss hit sequences
- Verify daily_loss trigger when loss > threshold
- Verify single_position trigger and ticker cooldown when position loss > threshold
- Verify volatility trigger when 3+ stop-losses within 30 minutes
- Verify all new orders rejected when any circuit breaker is active
- **Validates: Requirements 6.1, 6.2, 6.3**
- [x] 8.3 Write property test for circuit breaker cooldown expiry
- **Property 14: Circuit breaker cooldown expiry**
- Generate circuit breaker events with varying cooldown durations and current times
- Verify transition from active to resolved when time > triggered_at + cooldown
- Verify remains active before expiry
- **Validates: Requirements 6.5**
- [x] 9. Risk Tier Controller implementation
- [x] 9.1 Implement `services/trading/risk_tier_controller.py`
- Implement `RiskTierController` class with `evaluate()` method accepting PerformanceMetrics and reserve_pct
- Implement downgrade logic: downgrade one level when trailing 30-day win rate < 40% OR current drawdown > 15%
- Implement upgrade logic: upgrade one level when win rate > 55% AND reserve > 20% of total AND drawdown < 5%
- Implement tier bounds: never go below conservative or above aggressive
- Persist tier changes to `risk_tier_history` table with previous tier, new tier, and trigger metrics
- _Requirements: 5.2, 5.3, 5.4, 5.5, 5.6_
- [x] 9.2 Write property test for risk tier auto-adjustment conditions
- **Property 12: Risk tier auto-adjustment conditions**
- Generate random performance metrics (win rate, drawdown, reserve percentage)
- Verify downgrade when win rate < 40% OR drawdown > 15%
- Verify upgrade when win rate > 55% AND reserve > 20% AND drawdown < 5%
- Verify no change when neither condition met
- Verify tier never goes below conservative or above aggressive
- **Validates: Requirements 5.3, 5.4**
- [x] 10. Checkpoint — Ensure core components and property tests pass
- Ensure all tests pass, ask the user if questions arise.
- [x] 11. Correlation Matrix and Tax Lot Tracker
- [x] 11.1 Implement `services/trading/correlation.py`
- Implement `CorrelationMatrix` class with `refresh()` method: compute trailing 90-day price correlations from market data tables, persist to `correlation_matrix_cache` table
- Implement `get_correlation()` method: return coefficient for a ticker pair, 0.0 if unknown
- Implement `get_portfolio_correlation()` method: weighted average correlation between candidate and existing positions
- Cache in-memory after refresh; schedule daily refresh
- _Requirements: 9.1, 9.2, 9.3_
- [x] 11.2 Implement `services/trading/tax_lots.py`
- Implement `TaxLotTracker` class with `record_entry()` method: create tax lot record in PostgreSQL
- Implement `close_lots_fifo()` method: close lots in FIFO order, compute realized P&L per lot
- Implement `check_wash_sale()` method: check 30-day window before and after for same-ticker purchases
- Persist wash sale flags and details to tax_lots table
- _Requirements: 12.1, 12.2, 12.3, 12.4_
- [x] 11.3 Write property test for tax lot FIFO ordering
- **Property 22: Tax lot FIFO ordering**
- Generate random sequences of buy/sell transactions for the same ticker
- Verify lots closed in FIFO order (earliest acquired first)
- Verify realized P&L = (exit_price - cost_basis_per_share) * quantity per lot
- **Validates: Requirements 12.4**
- [x] 11.4 Write property test for wash sale detection
- **Property 23: Wash sale detection within 30-day window**
- Generate random loss-closing dates and purchase dates
- Verify wash sale flagged when same ticker purchased within 30 days before or after loss
- Verify no flag when purchases are outside the 30-day window
- **Validates: Requirements 12.2, 12.3**
- [x] 12. Trading Window and Gradual Entry logic
- [x] 12.1 Implement `services/trading/trading_window.py`
- Implement `is_within_trading_window()` function: return True if timestamp is between 9:45 AM ET and 3:45 PM ET on a market day
- Implement `next_window_open()` function: return the next timestamp when the trading window opens
- Implement `is_market_open()` function: check if current time is during US market hours (9:30 AM - 4:00 PM ET)
- _Requirements: 11.1, 11.2_
- [x] 12.2 Implement gradual entry logic in `services/trading/gradual_entry.py`
- Implement `should_use_gradual_entry()`: return True when position size exceeds min($30, 5% of Active Pool)
- Implement `split_into_tranches()`: split order into configured number of tranches (default 3) of approximately equal size
- Implement `GradualEntryManager` class to track pending tranches, re-evaluate before each submission, cancel remaining if conditions deteriorate
- Link all tranches to the same parent trading decision ID
- _Requirements: 11.3, 11.4, 11.5_
- [x] 12.3 Write property test for trading window determination
- **Property 20: Trading window determination**
- Generate random timestamps across US market hours
- Verify within-window classification for 9:45 AM - 3:45 PM ET
- Verify outside-window classification for all other times
- **Validates: Requirements 11.1**
- [x] 12.4 Write property test for gradual entry tranche splitting
- **Property 21: Gradual entry tranche splitting**
- Generate random position sizes above and below the threshold
- Verify splitting into configured number of tranches when above threshold
- Verify all tranches reference the same parent decision ID
- Verify tranche sizes are approximately equal
- **Validates: Requirements 11.3, 11.5**
- [x] 13. Autonomous Decision Loop (core engine)
- [x] 13.1 Implement `services/trading/engine.py`
- Implement `TradingEngine` class with `__init__()` accepting asyncpg.Pool, aioredis.Redis, and TradingEngineConfig
- Implement `start()` method: load portfolio state from Broker Service (positions, account balance), load active risk tier from PostgreSQL, load reserve pool balance, load circuit breaker status, load open stop levels, enter decision loop
- Implement `stop()` method: graceful shutdown — cancel pending tranches, persist state
- Implement `decision_loop()` method: poll recommendations at configured interval, evaluate each, size positions, submit orders
- Implement `poll_recommendations()` method: fetch from `recommendations` table where action IN (buy, sell) AND mode IN (paper_eligible, live_eligible) AND generated_at > last_poll_timestamp, ordered by confidence DESC
- Implement recommendation deduplication: check Redis key `stonks:dedupe:trading:{recommendation_id}` with 24h TTL, mark before evaluation
- Implement `evaluate_recommendation()` method: run all pre-trade checks (circuit breaker, trading window, risk tier confidence, portfolio heat, sector exposure, correlation, earnings proximity) and produce a TradingDecision record
- Implement `execute_decision()` method: generate order job payload matching existing broker queue schema, push to `stonks:queue:broker_orders`, handle gradual entry for large positions
- Persist every decision (act or skip) to `trading_decisions` table with full reasoning chain
- Implement adaptive market response: trigger immediate re-evaluation on high-severity macro events, tighten stops during events, increase polling frequency
- Implement rapid price move detection: re-evaluate position when price moves > 5% in 15 minutes
- Implement multiple declining positions halt: stop new entries when > 50% of positions have > 2% negative unrealized P&L
- _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 7.1, 7.2, 7.3, 7.4, 7.5, 17.1, 17.2_
- [x] 13.2 Write property test for recommendation deduplication
- **Property 27: Recommendation deduplication (idempotence)**
- Generate random recommendation IDs and process them twice
- Verify second processing is a no-op (no new decision record, no order submitted)
- **Validates: Requirements 1.5**
- [x] 13.3 Write property test for trading decision record completeness
- **Property 28: Trading decision record completeness and traceability**
- Generate random recommendations and evaluate them
- Verify all required fields present in the persisted decision record
- Verify "act" decisions include order job with trading_decision_id
- **Validates: Requirements 1.4, 17.1, 17.2**
- [x] 13.4 Write property test for multiple declining positions halts entries
- **Property 16: Multiple declining positions halts new entries**
- Generate portfolio states with varying percentages of declining positions
- Verify new entries halted when > 50% of positions have > 2% negative unrealized P&L
- Verify entries allowed when <= 50% are declining
- **Validates: Requirements 7.5**
- [x] 13.5 Write property test for maximum open positions enforcement
- **Property 18: Maximum open positions enforcement**
- Generate portfolio states at and below the max position limit
- Verify new entries rejected at the limit
- Verify portfolio never exceeds the configured maximum
- **Validates: Requirements 8.4**
- [x] 14. Checkpoint — Ensure decision loop and core engine tests pass
- Ensure all tests pass, ask the user if questions arise.
- [x] 15. Portfolio Rebalancer and Performance Tracker
- [x] 15.1 Implement `services/trading/rebalancer.py`
- Implement `PortfolioRebalancer` class with `evaluate()` method accepting positions, risk_tier, and active_pool
- Generate partial sell orders when single stock exceeds max_position_pct
- Generate sell orders for lowest-confidence positions when sector exceeds max_sector_pct
- Enforce maximum open positions limit (default 10)
- Submit rebalancing orders through normal broker queue with `rebalance` tag in decision trace
- Respect trading window and circuit breaker status
- Schedule: weekly at market open on Monday (configurable)
- _Requirements: 8.1, 8.2, 8.3, 8.4, 8.5, 8.6_
- [x] 15.2 Write property test for portfolio rebalancing sell orders
- **Property 17: Portfolio rebalancing generates correct sell orders**
- Generate portfolios with over-concentrated positions and sectors
- Verify sell orders generated to bring positions within limits
- Verify lowest-confidence positions targeted first for sector rebalancing
- **Validates: Requirements 8.2, 8.3**
- [x] 15.3 Implement `services/trading/performance_tracker.py`
- Implement `PerformanceTracker` class with `compute_metrics()` method: compute all PerformanceMetrics fields (total portfolio value, active/reserve pool, unrealized/realized P&L, daily P&L, win/loss counts, win rate, avg win/loss, profit factor, Sharpe ratio, max drawdown, current drawdown, portfolio heat)
- Implement Sharpe ratio: `(mean_daily_return / std_daily_return) * sqrt(252)` using trailing 30-day daily returns
- Implement `record_trade()` method: persist per-trade metrics (entry/exit price, hold duration, P&L, recommendation ID)
- Implement `persist_daily_snapshot()` method: save end-of-day snapshot to `portfolio_snapshots` table
- Compute metrics every 5 minutes during market hours
- Track micro-trade metrics separately from standard trade metrics
- _Requirements: 14.1, 14.2, 14.3, 20.7_
- [x] 15.4 Write property test for performance metrics computation
- **Property 26: Performance metrics computation**
- Generate random sets of closed trades with entry/exit prices and hold durations
- Verify win_rate = wins / total_trades
- Verify profit_factor = gross_profits / gross_losses (infinity if no losses)
- Verify Sharpe ratio formula consistency
- **Validates: Requirements 14.1, 14.2**
- [x] 15.5 Write property test for micro-trade metrics tracked separately
- **Property 33: Micro-trade metrics tracked separately**
- Generate mixed sets of standard and micro-trades
- Verify micro-trade metrics computed independently
- Verify standard trade metrics not contaminated by micro-trades
- **Validates: Requirements 20.7**
- [x] 16. Notification Service implementation
- [x] 16.1 Implement `services/trading/notifications.py`
- Implement `NotificationService` class with `send_alert()` method: send via all enabled channels (SMS via AWS SNS, email via Gmail API)
- Implement `send_daily_summary()` method: format and send daily performance summary at configurable time (default 16:30)
- Implement `send_weekly_digest()` method: format and send weekly performance digest
- Implement rate limiting: max 10 SMS/hour, 20 emails/hour (configurable), using Redis counters with hourly TTL
- Implement retry logic: up to 3 retries with exponential backoff on delivery failure
- Persist all notifications to `notifications` table with channel, event_type, message, delivery_status, timestamp
- Support event types: circuit_breaker_triggered, circuit_breaker_resumed, risk_tier_changed, emergency_liquidation, large_trade_pnl, daily_summary, weekly_digest
- Notifications run in separate asyncio tasks — never block trading operations
- _Requirements: 19.1, 19.2, 19.3, 19.4, 19.5, 19.6, 19.7, 19.8, 19.11_
- [x] 16.2 Write property test for notification rate limiting
- **Property 30: Notification rate limiting**
- Generate random sequences of notification requests within a one-hour window
- Verify at most 10 SMS and 20 emails delivered per hour
- Verify excess notifications marked as 'rate_limited'
- **Validates: Requirements 19.7**
- [x] 17. Micro-Trading Module
- [x] 17.1 Implement `services/trading/micro_trading.py`
- Implement `MicroTradingModule` class with `poll_intraday_signals()` method: fetch intraday and 1d trend window signals from aggregation engine
- Implement `evaluate_micro_trade()` method: evaluate signal against risk tier confidence threshold, apply micro-trade allocation cap (3% of Active Pool)
- Enforce daily micro-trade limit (default 10)
- Use tighter stop-loss (1.0x ATR) and take-profit (1.5x stop distance)
- Implement auto-close after max hold duration (default 2 hours)
- Respect all existing constraints (trading window, circuit breakers, portfolio heat, correlation, sector exposure, earnings)
- Toggleable independently via trading_engine_config
- _Requirements: 20.1, 20.2, 20.3, 20.4, 20.5, 20.6, 20.8, 20.10_
- [x] 17.2 Write property test for micro-trade parameter constraints
- **Property 31: Micro-trade parameter constraints**
- Generate random micro-trade scenarios
- Verify allocation does not exceed micro_trading_allocation_cap_pct
- Verify stop-loss at 1.0x ATR, take-profit at 1.5x stop distance
- Verify daily count does not exceed configured maximum
- **Validates: Requirements 20.3, 20.4, 20.5**
- [x] 17.3 Write property test for micro-trade auto-close
- **Property 32: Micro-trade auto-close after max hold duration**
- Generate micro-trade positions with varying hold durations
- Verify positions closed at market price when hold exceeds max duration
- **Validates: Requirements 20.6**
- [x] 17.4 Write property test for micro-trades respect all constraints
- **Property 34: Micro-trades respect all existing constraints**
- Generate micro-trade evaluations with various constraint violations
- Verify trading window, circuit breakers, portfolio heat, correlation, sector exposure, and earnings rules all enforced
- **Validates: Requirements 20.10**
- [x] 18. Checkpoint — Ensure all trading logic and property tests pass
- Ensure all tests pass, ask the user if questions arise.
- [x] 19. Backtester implementation
- [x] 19.1 Implement `services/trading/backtester.py`
- Implement `Backtester` class with `run()` method accepting BacktestConfig (start_date, end_date, initial_capital, risk_tier)
- Replay historical recommendations from `recommendations` table within date range
- Simulate full decision logic: position sizing, stop-loss/take-profit, circuit breakers, reserve pool, rebalancing
- Use historical price data from market data tables for simulation
- Produce BacktestResult with total_return, sharpe_ratio, max_drawdown, win_rate, profit_factor, trade_count, trade_log, equity_curve
- Persist results to `backtest_runs` and `backtest_trades` tables with unique backtest_id
- Handle missing historical data gracefully (skip dates, note gaps)
- Persist partial results with status 'failed' on mid-run errors
- _Requirements: 15.1, 15.2, 15.3, 15.4_
- [x] 19.2 Write property test for backtester produces equivalent metrics
- **Property 36: Backtester produces equivalent metrics**
- Generate random sets of historical trades
- Verify backtester metric computation matches performance tracker for same trade data
- **Validates: Requirements 15.3**
- [x] 20. Trading Engine FastAPI HTTP Service
- [x] 20.1 Implement `services/trading/app.py`
- Create FastAPI application with lifespan handler that starts/stops the TradingEngine
- Implement `GET /health` liveness probe endpoint
- Implement `GET /ready` readiness probe: return healthy when portfolio loaded and loop active
- Implement `GET /api/trading/status` endpoint: return engine state (enabled, risk tier, circuit breaker status, active/reserve pool, portfolio heat, open positions, last decision timestamp)
- Implement `PUT /api/trading/config` endpoint: update trading_engine_config, record audit event with previous/new config and change source
- Implement `POST /api/trading/pause` and `POST /api/trading/resume` endpoints
- Implement `GET /api/trading/decisions` endpoint: paginated, filterable by ticker, decision type, date range
- Implement `GET /api/trading/metrics` endpoint: current performance metrics
- Implement `GET /api/trading/metrics/history` endpoint: historical daily snapshots
- Implement `POST /api/trading/backtest` endpoint: launch backtest, return backtest_id
- Implement `GET /api/trading/backtest/{id}` endpoint: retrieve backtest results
- Implement `GET /api/trading/notifications/config` and `PUT /api/trading/notifications/config` endpoints
- Implement `GET /api/trading/notifications/history` endpoint: recent notifications
- _Requirements: 1.7, 5.6, 6.6, 15.5, 16.2, 16.3, 16.4, 17.3, 19.9_
- [x] 20.2 Write property test for configuration change audit trail
- **Property 35: Configuration change audit trail**
- Generate random configuration changes via API
- Verify audit event persisted with previous config, new config, and change source
- **Validates: Requirements 16.6**
- [x] 20.3 Write property test for persistence round-trip
- **Property 29: Persistence round-trip for trading engine state**
- Generate random trading engine config, reserve pool entries, risk tier history, circuit breaker events, portfolio snapshots, and backtest results
- Verify persist-then-read produces equivalent objects with all fields preserved
- **Validates: Requirements 3.2, 4.7, 5.5, 6.4, 14.3, 15.4, 16.1**
- [x] 21. Checkpoint — Ensure API endpoints and backtester work correctly
- Ensure all tests pass, ask the user if questions arise.
- [x] 22. Kubernetes deployment and infrastructure
- [x] 22.1 Add trading-engine service to Helm chart `infra/helm/stonks-oracle/values.yaml`
- Add `tradingEngine` entry under `services:` with: replicas 1, image trading-engine, command `uvicorn services.trading.app:app --host 0.0.0.0 --port 8000`, tier trading, port 8000, secrets [stonks-core-secrets, stonks-broker-secrets], resources (requests: 100m CPU / 256Mi memory, limits: 500m CPU / 512Mi memory), readiness probe on /ready port 8000, liveness probe on /health port 8000
- _Requirements: 1.7, 16.1_
- [x] 22.2 Add network policy for trading-engine
- Allow ingress from query-api, dashboard, and kube-system (Traefik) on port 8000
- Allow egress to PostgreSQL, Redis, and external services (SNS, Gmail API)
- _Requirements: 16.2_
- [x] 22.3 Add `/trading/` proxy route to dashboard nginx.conf
- Add `location /trading/ { proxy_pass http://trading-engine:8000/; }` to `frontend/nginx.conf`
- _Requirements: 14.4, 16.5_
- [x] 22.4 Add trading-engine ingress if external access needed
- Add ingress host entry for trading engine API (e.g., `stonks-trading.celestium.life`) to values.yaml if direct external access is desired, or rely on dashboard proxy
- _Requirements: 16.2_
- [x] 23. Dashboard frontend — Trading Engine panels
- [x] 23.1 Add trading API client hooks to `frontend/src/api/`
- Add `useTradingStatus()` hook: fetch `GET /trading/api/trading/status`
- Add `useTradingDecisions()` hook: fetch `GET /trading/api/trading/decisions` with pagination and filters
- Add `useTradingMetrics()` hook: fetch `GET /trading/api/trading/metrics`
- Add `useTradingMetricsHistory()` hook: fetch `GET /trading/api/trading/metrics/history`
- Add `useTradingConfig()` and `useUpdateTradingConfig()` hooks for config read/write
- Add `usePauseTradingEngine()` and `useResumeTradingEngine()` mutation hooks
- Add `useBacktestLaunch()` and `useBacktestResult()` hooks
- Add `useNotificationConfig()`, `useUpdateNotificationConfig()`, and `useNotificationHistory()` hooks
- _Requirements: 14.4, 14.5, 14.6, 14.7, 15.6, 16.5, 17.4, 19.10, 20.9_
- [x] 23.2 Implement Trading Engine overview panel component
- Display current Risk Tier, Circuit Breaker status (active/inactive with trigger reason and cooldown remaining), Active Pool and Reserve Pool balances, Portfolio Heat gauge, last 24h P&L summary
- Include start/pause/resume controls and Risk Tier selector dropdown
- Use TanStack Query for data fetching with auto-refresh
- _Requirements: 14.4, 16.5, 6.6_
- [x] 23.3 Implement Portfolio Composition panel component
- Display current positions table: ticker, entry price, current price, unrealized P&L, stop-loss level, take-profit level, sector
- Display sector allocation pie chart using Recharts
- _Requirements: 14.5_
- [x] 23.4 Implement Trade History panel component
- Display completed trades table: entry/exit prices, P&L amount and percentage, hold duration, recommendation thesis
- Support pagination and filtering by ticker and date range
- _Requirements: 14.6, 17.4_
- [x] 23.5 Implement Performance Charts panel component
- Display cumulative P&L line chart over time using Recharts
- Display daily returns bar chart using Recharts
- Display drawdown chart using Recharts
- _Requirements: 14.7_
- [x] 23.6 Implement Backtesting panel component
- Display backtest configuration form: date range picker, initial capital input, risk tier selector
- Display backtest results: equity curve chart, trade log table, summary metrics (total return, Sharpe, max drawdown, win rate, profit factor)
- Support launching new backtests and viewing historical results
- _Requirements: 15.6_
- [x] 23.7 Implement Micro-Trading panel component
- Display micro-trade mode status toggle (enabled/disabled)
- Display today's micro-trade count and P&L
- Display active micro-trade positions table
- Display micro-trade performance metrics over trailing 7 days
- _Requirements: 20.9_
- [x] 23.8 Implement Notification Preferences panel component
- Display notification channel toggles (SMS, email) with phone number and email address inputs
- Display event type selection checkboxes
- Display rate limit configuration
- Display recent notification history table
- _Requirements: 19.10_
- [x] 23.9 Wire trading panels into dashboard routing
- Add Trading page route to TanStack Router configuration
- Add navigation link to the dashboard sidebar/header
- Compose all trading panels (overview, portfolio, trade history, performance, backtesting, micro-trading, notifications) into the Trading page layout
- _Requirements: 14.4, 16.5_
- [x] 24. Checkpoint — Ensure frontend builds and all tests pass
- Ensure all tests pass, ask the user if questions arise.
- [x] 25. Integration wiring and final validation
- [x] 25.1 Wire stop-loss price crossing detection into the decision loop
- Connect `StopLossManager.check_price_crossings()` to run at the configured interval (5 min default, 60s during high-severity events)
- Generate immediate sell orders for triggered positions and submit to broker queue
- Handle price data unavailability (close position after 15 min without data)
- _Requirements: 4.3, 4.4, 4.5, 4.8, 7.4_
- [x] 25.2 Wire reserve pool siphoning to position close events
- Detect profitable position closes from broker service fill events
- Call `ReservePoolController.siphon_profit()` with realized profit
- Trigger notification for large trade P&L events
- _Requirements: 3.1, 19.2_
- [x] 25.3 Wire risk tier evaluation to daily market close schedule
- Schedule `RiskTierController.evaluate()` at market close
- Trigger notification on tier changes
- _Requirements: 5.2, 19.2_
- [x] 25.4 Wire portfolio rebalancer to weekly schedule
- Schedule `PortfolioRebalancer.evaluate()` weekly at Monday market open
- Submit rebalancing orders through broker queue
- _Requirements: 8.1, 8.5_
- [x] 25.5 Wire notification service to all critical events
- Connect circuit breaker triggers/resumes, risk tier changes, emergency liquidation, large trade P&L to notification dispatch
- Schedule daily summary at configured time (default 16:30)
- Schedule weekly digest
- _Requirements: 19.2, 19.3, 19.4_
- [x] 25.6 Wire micro-trading module into the decision loop
- Start micro-trading polling when enabled in config
- Route micro-trade decisions through the same order submission pipeline
- Track micro-trade metrics separately in performance tracker
- _Requirements: 20.1, 20.2, 20.7_
- [x] 25.7 Write integration tests for end-to-end decision flow
- Test full cycle: recommendation → evaluation → position sizing → order submission to broker queue
- Test stop-loss crossing → immediate sell order
- Test reserve pool siphoning on profitable close
- Test circuit breaker trigger → halt → cooldown → resume
- Test engine startup state reconstruction from PostgreSQL
- _Requirements: 1.1, 1.2, 1.3, 3.1, 4.4, 6.1, 6.5, 18.5_
- [x] 26. Final checkpoint — Ensure all tests pass
- Ensure all tests pass, ask the user if questions arise.
## Notes
- Tasks marked with `*` are optional and can be skipped for faster MVP
- Each task references specific requirements for traceability
- Checkpoints ensure incremental validation after each major component
- Property tests validate the 36 correctness properties defined in the design document
- The trading engine is a NEW service at `services/trading/` — it does not replace existing services
- All order submission goes through the existing `stonks:queue:broker_orders` Redis queue consumed by the Broker Service
- Migration number 018 is the next available migration slot
- Frontend components use the existing React 19 + TypeScript + Tailwind + TanStack + Recharts stack
- Dashboard proxy needs `/trading/``trading-engine:8000` added to nginx.conf
## Phase 2: Live Wiring and Paper Trading
### Overview
Phase 1 (Tasks 126) implemented all pure computation modules, property tests, FastAPI endpoints, Helm chart, and frontend panels. Phase 2 replaces the lifecycle stubs in `services/trading/engine.py` with real async implementations, wires all sub-components into live loops backed by PostgreSQL and Redis, adds notification dispatch and backtest replay, connects real database pools in the FastAPI lifespan, enables paper trading configuration, and adds integration tests for the live wiring.
### Tasks
- [x] 27. Wire the live decision loop in `services/trading/engine.py`
- [x] 27.1 Replace `start()` stub with real async implementation
- Load `trading_engine_config` from PostgreSQL via `self.pool`
- Load active risk tier parameters from `risk_tier_history` (latest entry) or fall back to config default
- Sync portfolio state from Broker Service: fetch positions and account balance via `asyncpg` query against the broker's `orders` / `positions` tables
- Load reserve pool balance from `reserve_pool_ledger` (latest `balance_after`)
- Load circuit breaker status from `circuit_breaker_events` (unresolved events)
- Load open stop-loss/take-profit levels from `position_stop_levels` where `active = TRUE`
- Populate `self.portfolio_state` with loaded data
- Create `asyncio.Task` instances for `_decision_loop()`, `_stop_loss_monitor()`, `_performance_loop()`, `_risk_tier_scheduler()`, `_rebalance_scheduler()` and store in `self._tasks: list[asyncio.Task]`
- Set `self.running = True` only after successful state load
- If portfolio state cannot be loaded, enter degraded state (readiness probe unhealthy), retry every 30 seconds
- _Requirements: 1.6, 18.5_
- [x] 27.2 Replace `stop()` stub with real async shutdown
- Set `self.running = False`
- Cancel all tasks in `self._tasks` and `await asyncio.gather(*self._tasks, return_exceptions=True)`
- Persist current portfolio state snapshot to `portfolio_snapshots`
- Close any pending gradual entry tranches
- Log shutdown event
- _Requirements: 1.6, 16.4_
- [x] 27.3 Implement `_decision_loop()` coroutine
- `while self.running`: sleep for `self.config.polling_interval_seconds`, then poll recommendations
- Poll recommendations from PostgreSQL: `SELECT * FROM recommendations WHERE action IN ('buy','sell') AND mode IN ('paper_eligible','live_eligible') AND generated_at > $1 ORDER BY confidence DESC`
- For each recommendation, check Redis deduplication key `stonks:dedupe:trading:{recommendation_id}` (24h TTL) — skip if already set
- Set the Redis dedupe key immediately before evaluation to prevent double-processing on restart
- Call `self.evaluate_recommendation()` (existing synchronous method) with current portfolio state, risk tier, circuit breaker state, correlation matrix, and earnings calendar
- For "act" decisions: generate order job payload matching existing broker queue schema, push to `stonks:queue:broker_orders` via Redis RPUSH; handle gradual entry for large positions
- Call `_persist_decision()` for every decision (act or skip)
- Update `self.portfolio_state` after each acted decision (reduce active pool, increment open position count)
- Wrap each recommendation evaluation in try/except — on failure, persist skip decision with reason `evaluation_error` and continue
- _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5_
- [x] 27.4 Implement `_sync_positions_and_siphon()` helper
- Fetch current positions and account balance from PostgreSQL (broker tables)
- Detect newly closed positions by comparing with previous `self.portfolio_state.positions`
- For each profitable close: call `self.reserve_pool_controller.siphon_profit()` with realized profit and current reserve balance
- Persist siphon event to `reserve_pool_ledger` via `self.pool`
- Update `self.portfolio_state` with refreshed positions, active pool, reserve pool
- Trigger notification for large trade P&L events (> 5% of Active Pool)
- _Requirements: 3.1, 3.2, 3.3, 19.2_
- [x] 27.5 Implement `_persist_decision()` helper
- INSERT into `trading_decisions` table with all fields from the `TradingDecision` dataclass
- Use `self.pool.execute()` with parameterized query
- Log decision summary (ticker, decision, skip_reason if any)
- _Requirements: 1.4, 17.1_
- [x] 27.6 Implement asyncio task management
- Add `self._tasks: list[asyncio.Task] = []` to `__init__()`
- In `start()`, create named tasks: `asyncio.create_task(self._decision_loop(), name="decision_loop")`, etc.
- In `stop()`, cancel all tasks, await with `return_exceptions=True`, clear the list
- Add error handling: if a task raises an unexpected exception, log it and restart the task (unless `self.running` is False)
- _Requirements: 1.1, 1.6_
- [x] 28. Wire the stop-loss monitoring loop
- [x] 28.1 Implement `_stop_loss_monitor()` coroutine in `services/trading/engine.py`
- `while self.running`: sleep for `self.config.stop_loss_check_interval_seconds` (default 300s, or `fast_stop_loss_interval_seconds` = 60s during high-severity events)
- Call `_load_open_positions()` and `_load_stop_levels()` from PostgreSQL
- Call `_fetch_current_prices()` for all tickers with open positions
- Call `self.check_stop_loss_crossings(positions, prices, stop_levels)` (existing method delegates to StopLossManager)
- For each `StopTrigger` returned: generate immediate market sell order, push to `stonks:queue:broker_orders` via Redis
- Persist stop-loss trigger event to `trading_decisions` with decision="act" and trace noting stop-loss/take-profit trigger
- _Requirements: 4.3, 4.4, 4.5, 7.4_
- [x] 28.2 Implement `_fetch_current_prices()` helper
- Query the market data adapter (Polygon API) for current/latest prices of given tickers
- Use `services/shared/config.py` `MarketDataConfig` for API key and base URL
- Return `dict[str, float]` mapping ticker → latest price
- On API failure: log warning, return empty dict for failed tickers
- _Requirements: 4.3, 4.8_
- [x] 28.3 Implement `_load_open_positions()` and `_load_stop_levels()` helpers
- `_load_open_positions()`: query broker/positions tables via `self.pool` to get current open positions, return as `list[OpenPosition]`
- `_load_stop_levels()`: query `position_stop_levels WHERE active = TRUE` via `self.pool`, return as `dict[str, StopLevels]` keyed by ticker
- _Requirements: 4.3, 18.3_
- [x] 28.4 Implement safety sell for missing price data
- Track last successful price fetch timestamp per ticker
- If a ticker has no price data for > 15 minutes during market hours (checked via `is_market_open()`), generate a market sell order for that position
- Log warning with ticker and duration of missing data
- _Requirements: 4.8_
- [x] 29. Wire the performance metrics loop
- [x] 29.1 Implement `_performance_loop()` coroutine in `services/trading/engine.py`
- `while self.running`: sleep 300 seconds (5 minutes)
- Check if currently within market hours via `is_market_open()`; skip computation if outside market hours
- Call `self.performance_tracker.compute_metrics()` with current portfolio state from `self.pool`
- Update `self.portfolio_state` with latest metrics (portfolio heat, unrealized P&L, etc.)
- _Requirements: 14.1_
- [x] 29.2 Implement daily snapshot persistence
- At end of trading day (after 4:00 PM ET), call `self.performance_tracker.persist_daily_snapshot()` to write to `portfolio_snapshots` table via `self.pool`
- Include end-of-day portfolio value, daily return, cumulative return, all positions with unrealized P&L, and computed metrics
- _Requirements: 14.3_
- [x] 29.3 Wire performance tracker to use real database pool
- Pass `self.pool` to `PerformanceTracker` so it can query closed trades from `trading_decisions` and broker fill tables
- Compute Sharpe ratio from `portfolio_snapshots` trailing 30-day daily returns
- Compute win/loss counts and profit factor from closed trade records
- _Requirements: 14.1, 14.2_
- [x] 30. Wire risk tier and rebalance schedulers
- [x] 30.1 Implement `_risk_tier_scheduler()` coroutine in `services/trading/engine.py`
- `while self.running`: compute seconds until next 16:00 ET, sleep until then
- Load latest `PerformanceMetrics` from `portfolio_snapshots` or compute fresh
- Compute `reserve_pct = self.portfolio_state.reserve_pool / self.portfolio_state.total_value`
- Call `self.evaluate_risk_tier(current_tier, metrics, reserve_pct)` (existing method delegates to RiskTierController)
- If tier changed: persist to `risk_tier_history` via `self.pool`, update `self.config.risk_tier`, trigger notification via `self.create_alert("risk_tier_changed", ...)`
- _Requirements: 5.2, 5.5, 19.2_
- [x] 30.2 Implement `_rebalance_scheduler()` coroutine in `services/trading/engine.py`
- `while self.running`: compute seconds until next Monday 09:45 ET, sleep until then
- Load current positions and active risk tier
- Call `self.evaluate_rebalancing(positions, risk_tier, active_pool)` (existing method delegates to PortfolioRebalancer)
- For each rebalance order returned: generate order job with `rebalance` tag in decision trace, push to `stonks:queue:broker_orders`
- Persist rebalance decisions to `trading_decisions` table
- Respect circuit breaker status — skip rebalancing if any circuit breaker is active
- _Requirements: 8.1, 8.5, 8.6_
- [x] 31. Wire notification dispatch
- [x] 31.1 Create `services/trading/notification_dispatch.py` with `NotificationDispatcher` class
- Accept `pool`, `redis`, and `TradingConfig` in constructor
- Implement `dispatch(event_type: str, message: str)` method that routes to enabled channels
- Check `self.config.sns_topic_arn` / `self.config.gmail_recipient` to determine enabled channels
- Call `_send_sns()` and/or `_send_gmail()` based on enabled channels
- Persist notification record to `notifications` table via `self.pool` with channel, event_type, message, delivery_status, timestamp
- _Requirements: 19.1, 19.8_
- [x] 31.2 Implement SNS delivery via `boto3`
- Implement `_send_sns(event_type: str, message: str)` method
- Use `boto3.client("sns")` with credentials from environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`)
- Publish to configured `sns_topic_arn` with message and subject based on event_type
- Return delivery status (delivered/failed)
- _Requirements: 19.5_
- [x] 31.3 Implement Gmail delivery via `google-api-python-client`
- Implement `_send_gmail(event_type: str, message: str)` method
- Use `google.oauth2.credentials.Credentials` with refresh token from environment variables
- Build Gmail API service, create MIME message, send via `users().messages().send()`
- Configurable sender (`self.config.gmail_sender`) and recipient (`self.config.gmail_recipient`)
- Return delivery status (delivered/failed)
- _Requirements: 19.6_
- [x] 31.4 Implement rate limiting via Redis
- Before each send, check Redis counter `stonks:trading:notification_rate:{channel}` with 1-hour TTL
- If counter >= limit (10 SMS/hour, 20 emails/hour), mark notification as `rate_limited` and skip delivery
- Increment counter on successful delivery
- _Requirements: 19.7_
- [x] 31.5 Implement retry with exponential backoff
- On delivery failure, retry up to 3 times with delays: 1s, 2s, 4s
- Update notification record with retry_count and error_message on final failure
- Never block trading operations — run dispatch in a separate `asyncio.create_task()`
- _Requirements: 19.11_
- [x] 31.6 Implement daily summary at 16:30 ET
- Add `_daily_summary_scheduler()` coroutine: sleep until 16:30 ET each trading day
- Compute daily metrics from `portfolio_snapshots` and current portfolio state
- Format summary message with: daily P&L, total portfolio value, Active/Reserve Pool balances, trade count, current Risk Tier, circuit breaker status
- Dispatch via `self.dispatch("daily_summary", summary_message)`
- _Requirements: 19.3_
- [x] 32. Wire backtest replay
- [x] 32.1 Create `services/trading/backtest_replay.py` with `BacktestReplay` class
- Accept `pool: asyncpg.Pool` in constructor
- Implement `run(config: BacktestConfig) -> BacktestResult` method
- _Requirements: 15.1_
- [x] 32.2 Fetch historical recommendations and price data
- Query `recommendations` table for date range: `WHERE generated_at BETWEEN $1 AND $2 AND action IN ('buy','sell') ORDER BY generated_at ASC`
- Query market data tables for historical daily close prices within the date range
- Build a day-by-day timeline of recommendations and prices
- Handle missing data gracefully: skip dates with no price data, note gaps in result
- _Requirements: 15.1, 15.2_
- [x] 32.3 Simulate full decision logic chronologically
- Initialize simulated portfolio state with `config.initial_capital` and configured risk tier
- For each trading day in the date range, process recommendations through `evaluate_recommendation()` using historical prices
- Simulate stop-loss/take-profit crossings using historical intraday or daily price data
- Simulate reserve pool siphoning on profitable closes
- Simulate circuit breaker triggers based on simulated daily P&L
- Simulate risk tier auto-adjustment at daily close
- Simulate weekly rebalancing on Mondays
- Track equity curve: `[{date, portfolio_value}]` for each trading day
- _Requirements: 15.2_
- [x] 32.4 Persist results to `backtest_runs` and `backtest_trades`
- INSERT into `backtest_runs` with config, result metrics (total_return, sharpe_ratio, max_drawdown, win_rate, profit_factor, trade_count), equity_curve JSONB, status='completed'
- INSERT into `backtest_trades` for each simulated trade with ticker, side, entry/exit prices, quantity, pnl, dates, hold_duration, recommendation_id
- On mid-run error: persist partial results with status='failed' and error message
- _Requirements: 15.4_
- [x] 32.5 Wire into `POST /api/trading/backtest` endpoint in `services/trading/app.py`
- Replace placeholder `launch_backtest()` with real implementation
- Instantiate `BacktestReplay(pool=engine.pool)` and call `run()` in a background `asyncio.Task`
- Return `backtest_id` immediately
- Update `GET /api/trading/backtest/{id}` to query `backtest_runs` and `backtest_trades` from PostgreSQL
- _Requirements: 15.5_
- [x] 33. Wire real connections in `services/trading/app.py` lifespan
- [x] 33.1 Replace `pool=None` with `asyncpg.create_pool()`
- In the lifespan `async with` block, create pool: `pool = await asyncpg.create_pool(dsn=config.postgres.dsn, min_size=2, max_size=10)`
- Pass `pool` to `TradingEngine(pool=pool, ...)`
- In lifespan exit: `await pool.close()`
- _Requirements: 1.6, 18.5_
- [x] 33.2 Replace `redis=None` with `aioredis.from_url()`
- In the lifespan block, create Redis client: `redis_client = aioredis.from_url(config.redis.url)`
- Pass `redis_client` to `TradingEngine(pool=pool, redis=redis_client, ...)`
- In lifespan exit: `await redis_client.close()`
- _Requirements: 1.5_
- [x] 33.3 Add proper error handling and cleanup in lifespan
- Wrap pool/redis creation in try/except — log critical error and raise if connections fail
- Ensure `engine.stop()`, `pool.close()`, and `redis_client.close()` are called in the `finally` block of lifespan exit
- Log connection details (host, port, database) at startup for debugging (not passwords)
- _Requirements: 1.6, 18.5_
- [x] 34. Checkpoint — Ensure all live wiring compiles and existing tests still pass
- Ensure all tests pass, ask the user if questions arise.
- [x] 35. Enable paper trading configuration
- [x] 35.1 Update `trading_engine_config` defaults for paper trading
- Add a SQL migration or seed script that updates the default `trading_engine_config` row: `enabled=true`, `risk_tier='conservative'`, `absolute_position_cap=25.0` (conservative for initial paper trading)
- Set `polling_interval_seconds=60`, `max_open_positions=5` (conservative start)
- _Requirements: 16.1, 5.1_
- [x] 35.2 Add `TRADING_ENABLED=true` to Helm values for trading-engine deployment
- Update `infra/helm/stonks-oracle/values.yaml` to set `TRADING_ENABLED: "true"` in the trading-engine environment variables
- Ensure `TRADING_RISK_TIER: "conservative"` and `TRADING_ABSOLUTE_POSITION_CAP: "25.0"` are set
- _Requirements: 16.1_
- [x] 35.3 Verify trading-engine pod starts and readiness probe passes
- After deployment, confirm the trading-engine pod reaches `Running` state
- Confirm `GET /ready` returns `{"ready": true}` once portfolio state is loaded
- Confirm `GET /health` returns `{"status": "ok"}`
- Confirm `GET /api/trading/status` returns the expected configuration (enabled=true, risk_tier=conservative)
- _Requirements: 1.7, 16.2_
- [x] 36. Write integration tests for live wiring
- [x]* 36.1 Test decision loop with mocked PostgreSQL and Redis
- Create `tests/test_trading_integration.py`
- Mock `asyncpg.Pool` to return canned recommendation rows and portfolio state
- Mock Redis client for deduplication checks and broker queue pushes
- Verify the decision loop polls recommendations, evaluates them, persists decisions, and pushes "act" orders to the broker queue
- Verify deduplication prevents double-processing
- Verify skip decisions are persisted with correct reasons
- _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5_
- [x]* 36.2 Test stop-loss monitor with mocked price API
- Mock `_fetch_current_prices()` to return prices that cross stop-loss and take-profit levels
- Verify sell orders are generated and pushed to broker queue for triggered positions
- Verify no orders generated when prices are between stop and take-profit
- Test safety sell: mock price fetch returning empty for > 15 minutes, verify position closed
- _Requirements: 4.4, 4.5, 4.8_
- [x]* 36.3 Test notification dispatch with mocked SNS and Gmail
- Mock `boto3.client("sns")` and Gmail API service
- Verify SNS publish called with correct topic ARN and message for SMS-enabled events
- Verify Gmail send called with correct sender/recipient for email-enabled events
- Verify rate limiting: send 11 SMS in one hour, verify 11th is marked `rate_limited`
- Verify retry: mock first delivery failure, verify retry with backoff, verify final success
- _Requirements: 19.1, 19.5, 19.6, 19.7, 19.11_
- [x]* 36.4 Test backtest replay end-to-end
- Mock PostgreSQL pool to return historical recommendations and price data
- Run `BacktestReplay.run()` with a small date range and $500 initial capital
- Verify backtest result contains expected metrics (total_return, sharpe_ratio, max_drawdown, win_rate, trade_count)
- Verify equity curve has one entry per trading day
- Verify trades are persisted to `backtest_trades` (mocked INSERT calls)
- _Requirements: 15.1, 15.2, 15.3, 15.4_
- [x]* 36.5 Test lifespan creates real pool and redis connections
- Use `httpx.AsyncClient` with the FastAPI `app` and mock `asyncpg.create_pool` / `aioredis.from_url`
- Verify pool and redis are created during startup and closed during shutdown
- Verify engine receives non-None pool and redis
- _Requirements: 1.6, 18.5_
- [x] 37. Final checkpoint — Verify paper trading is operational
- Ensure all tests pass, ask the user if questions arise.
- Trading engine pod is running and ready
- Decision loop is polling recommendations from PostgreSQL
- Stop-loss monitor is checking prices at configured interval
- Performance metrics are being computed every 5 minutes during market hours
- Dashboard shows trading engine status as enabled with conservative tier
## Phase 2 Notes
- Phase 2 tasks build on the completed Phase 1 pure computation modules — no Phase 1 code is rewritten, only the lifecycle stubs are replaced
- All async loops use `while self.running` pattern with `asyncio.sleep()` for clean shutdown
- Database connections are created in the FastAPI lifespan and passed to the engine — no global connection state
- Integration tests use mocked database pools and Redis clients to avoid requiring live infrastructure
- Paper trading starts with conservative settings (risk_tier=conservative, absolute_position_cap=$25, max_open_positions=5) to validate behavior before scaling up
- Tasks 36.x (integration tests) are marked optional (`*`) — they can be skipped for faster deployment but are recommended