Files

T

Celes Renata 8050f4a03b chore: mark all Phase 2 tasks (27-37) as complete

All 152 tasks across both phases are now marked complete:
- Phase 1 (1-26): pure computation modules, property tests, API, frontend, infra
- Phase 2 (27-37): live decision loop, stop-loss monitor, performance metrics,
  risk tier scheduler, rebalancer, notification dispatch, backtest replay,
  real DB connections, paper trading config, integration tests

2026-04-16 07:27:35 +00:00

66 KiB

Raw Permalink Blame History

Implementation Plan: Autonomous Trading Engine

Overview

This plan implements a fully autonomous trading engine as a new service (services/trading/) that consumes recommendations from the existing three-layer signal aggregation pipeline, applies confidence-based position sizing with reserve pool management, enforces dynamic stop-loss/take-profit levels, manages circuit breakers, and submits orders through the existing Broker Service queue. The implementation extends existing services (broker_service, recommendation, risk engine, query API, dashboard) without replacing them. Tasks are ordered so each step builds on the previous, with property-based tests validating core computation logic early.

Tasks

1. Database migration and shared infrastructure
- 1.1 Create PostgreSQL migration infra/migrations/018_autonomous_trading_engine.sql
  - Add trading_engine_config table with all configuration fields (enabled, paused, risk_tier, reserve_siphon_pct, polling intervals, gradual entry params, circuit breaker thresholds, active pool minimum, emergency drawdown threshold, correlation thresholds, earnings windows, micro-trading params, notification settings, timestamps)
  - Add reserve_pool_ledger table with amount, balance_after, trigger_type (profit_siphon, emergency_liquidation, manual_adjustment, initial), reference_id, notes, created_at; index on created_at DESC
  - Add risk_tier_history table with previous_tier, new_tier, trigger_source, trigger_metrics JSONB, created_at; index on created_at DESC
  - Add circuit_breaker_events table with trigger_type (daily_loss, single_position, volatility, manual), threshold_value, actual_value, ticker, cooldown_expires, resolved_at, created_at; partial index on active (unresolved) events
  - Add trading_decisions table with recommendation_id FK, decision, skip_reason, ticker, computed_position_size, computed_share_quantity, risk_tier_at_decision, portfolio_heat_at_decision, active_pool_at_decision, reserve_pool_at_decision, circuit_breaker_status, correlation_check_result JSONB, sector_exposure_check_result JSONB, earnings_proximity_flag, is_micro_trade, decision_trace JSONB, created_at; indexes on ticker, recommendation_id, decision
  - Add position_stop_levels table with ticker, entry_price, stop_loss_price, take_profit_price, trailing_stop_active, atr_value, atr_multiplier, reward_risk_ratio, signal_confidence, is_micro_trade, active, timestamps; partial index on active positions
  - Add portfolio_snapshots table with snapshot_date (UNIQUE), portfolio_value, active_pool, reserve_pool, daily_return, cumulative_return, unrealized_pnl, realized_pnl, win/loss counts, win_rate, sharpe_ratio, max_drawdown, current_drawdown_pct, portfolio_heat, risk_tier, positions JSONB, metrics JSONB, created_at; index on snapshot_date DESC
  - Add backtest_runs table with start_date, end_date, initial_capital, risk_tier, config JSONB, result metrics, equity_curve JSONB, status, completed_at, created_at
  - Add backtest_trades table with backtest_id FK (CASCADE), ticker, side, entry/exit prices, quantity, pnl, dates, hold_duration_days, recommendation_id; index on backtest_id
  - Add tax_lots table with ticker, quantity, cost_basis_per_share, acquisition_date, status (open/closed/washed), closed_date, exit_price, realized_pnl, wash_sale_flag, wash_sale_details, order_id FK; indexes on ticker and open status
  - Add earnings_calendar table with ticker, earnings_date, source, confirmed, timestamps; UNIQUE on (ticker, earnings_date); indexes on date and ticker
  - Add correlation_matrix_cache table with ticker_a, ticker_b, correlation_coefficient, lookback_days, computed_at; UNIQUE on (ticker_a, ticker_b)
  - Add notifications table with channel (sms/email), event_type, message, delivery_status (pending/delivered/failed/rate_limited), retry_count, error_message, created_at, delivered_at; indexes on created_at and event_type
  - Insert default trading_engine_config row with moderate tier defaults
  - Insert initial reserve_pool_ledger entry with balance 0.0 and trigger_type 'initial'
  - Requirements: 18.1, 18.2, 18.3, 18.4, 16.1
- 1.2 Add new Pydantic schemas and enums to services/shared/schemas.py
  - Add TradingDecisionType enum (act, skip)
  - Add CircuitBreakerTriggerType enum (daily_loss, single_position, volatility, manual)
  - Add ReservePoolTriggerType enum (profit_siphon, emergency_liquidation, manual_adjustment, initial)
  - Add NotificationChannel enum (sms, email)
  - Add RiskTierName enum (conservative, moderate, aggressive)
  - Requirements: 5.1, 6.1, 3.1, 19.1
- 1.3 Add trading-related Redis keys to services/shared/redis_keys.py
  - Add QUEUE_TRADING_DECISIONS = "trading_decisions" queue name
  - Add TRADING_DEDUPE_PREFIX for recommendation deduplication (stonks:dedupe:trading)
  - Add TRADING_CB_PREFIX for circuit breaker state (stonks:trading:circuit_breaker)
  - Add TRADING_NOTIFICATION_RATE for notification rate limiting (stonks:trading:notification_rate)
  - Requirements: 1.5, 6.4, 19.7
- 1.4 Add TradingConfig dataclass to services/shared/config.py
  - Add TradingConfig with fields: enabled, risk_tier, reserve_siphon_pct, polling_interval_seconds, stop_loss_check_interval_seconds, fast_stop_loss_interval_seconds, gradual_entry_tranches, gradual_entry_threshold_dollars, absolute_position_cap, active_pool_minimum, emergency_drawdown_threshold_pct, reserve_high_water_pct, micro_trading_enabled, micro_trading_interval_seconds, micro_trading_allocation_cap_pct, micro_trading_max_daily, micro_trading_max_hold_minutes, sns_topic_arn, sns_phone_number, gmail_sender, gmail_recipient
  - Add trading: TradingConfig field to AppConfig with env var loading in load_config()
  - Requirements: 16.1, 20.1, 19.5, 19.6
2. Checkpoint — Ensure migration and shared schemas are consistent
- Ensure all tests pass, ask the user if questions arise.
3. Core data models and risk tier configuration
- 3.1 Create services/trading/__init__.py and services/trading/models.py
  - Create the services/trading/ package directory
  - Define RiskTierConfig dataclass with fields: name, min_confidence, max_position_pct, stop_loss_atr_multiplier, reward_risk_ratio, max_sector_pct, max_portfolio_heat
  - Define RISK_TIER_DEFAULTS dict mapping conservative/moderate/aggressive to their default RiskTierConfig instances per the design specification
  - Define PortfolioState dataclass with fields: positions (list), total_value, cash, active_pool, reserve_pool, sector_exposure (dict), portfolio_heat, open_position_count
  - Define TradingDecision dataclass with all fields matching the trading_decisions table schema
  - Define PositionSizeResult dataclass with dollar_amount, share_quantity, allocation_pct, adjustments list, rejected flag, rejection_reason
  - Define StopLevels dataclass with stop_loss_price, take_profit_price, trailing_stop_active, atr_value, atr_multiplier, reward_risk_ratio, last_updated
  - Define OpenPosition dataclass with ticker, quantity, entry_price, current_price, unrealized_pnl, market_value, sector, stop_loss_price, take_profit_price, signal_confidence, is_micro_trade
  - Define ClosedTrade dataclass with ticker, entry_price, exit_price, quantity, pnl, pnl_pct, hold_duration, recommendation_id, is_micro_trade
  - Define PerformanceMetrics dataclass with all fields from the design (total_portfolio_value through computed_at)
  - Define CircuitBreakerState dataclass with active, trigger_type, triggered_at, cooldown_expires, ticker_cooldowns dict
  - Define ReservePoolState dataclass with balance, total_deposits, total_withdrawals, last_updated
  - Define StopTrigger dataclass with ticker, trigger_type (stop_loss/take_profit), current_price, trigger_price
  - Requirements: 5.1, 1.2, 2.1, 4.1, 6.1, 3.1, 13.1, 14.1
- 3.2 Write property test for risk tier default parameters
  - Property 29 (partial): Persistence round-trip for risk tier configs
  - Verify all three tier defaults have valid parameter ranges (min_confidence in [0,1], max_position_pct in (0,1], etc.)
  - Verify conservative < moderate < aggressive for min_confidence thresholds (inverse) and position limits
  - Validates: Requirements 5.1
4. Position Sizer implementation
- 4.1 Implement services/trading/position_sizer.py
  - Implement PositionSizer class with compute() method accepting confidence, ticker, sector, current_price, active_pool, risk_tier, portfolio_state, correlation_matrix, earnings_calendar
  - Implement sizing formula: raw_pct = base_allocation_pct * (confidence / min_confidence) * multiplier, clamped to max_position_pct, then dollar_amount = active_pool * clamped_pct, clamped to absolute_position_cap
  - Implement confidence gate: reject when confidence < risk_tier.min_confidence
  - Implement correlation reduction: compute weighted average correlation with existing positions; reduce proportionally when avg > 0.5; reject entirely when avg > 0.8
  - Implement sector exposure reduction: reduce allocation if adding position would push sector above max_sector_pct
  - Implement diversification bonus: 1.2x multiplier for under-represented sectors when portfolio holds < 3 sectors
  - Implement earnings proximity: reduce by 50% within 3 trading days; reject within 1 trading day
  - Implement portfolio heat check: reject if current heat + new position heat exceeds max_portfolio_heat
  - Implement active pool minimum: reject new entries when Active Pool < configured minimum ($100 default)
  - Implement absolute cap enforcement and share rounding (round down to whole shares, reject if quantity = 0)
  - Requirements: 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 9.2, 9.3, 9.4, 9.5, 10.2, 10.3, 13.1, 13.2, 3.5
- 4.2 Write property test for position sizing formula and invariants
  - Property 1: Position sizing formula and invariants
  - Generate random confidence values, Active Pool balances, stock prices, and RiskTierConfig objects
  - Verify zero allocation when confidence < min_confidence
  - Verify allocation never exceeds max_position_pct or absolute_position_cap
  - Verify share quantity is rounded down to whole shares
  - Verify rejection when rounded quantity is zero
  - Validates: Requirements 2.1, 2.2, 2.3, 2.4, 2.7
- 4.3 Write property test for correlation-based allocation adjustment
  - Property 2: Correlation-based allocation adjustment
  - Generate random correlation matrices and portfolio positions
  - Verify allocation reduced when weighted avg correlation > 0.5
  - Verify trade rejected when weighted avg correlation > 0.8
  - Verify allocation unchanged when weighted avg correlation <= 0.5
  - Verify monotonic non-increase: higher correlation → lower or equal allocation
  - Validates: Requirements 2.5, 9.2, 9.3
- 4.4 Write property test for sector exposure enforcement
  - Property 3: Sector exposure computation and enforcement
  - Generate random portfolios with sector labels
  - Verify sector exposure equals sum of market values per sector
  - Verify allocation reduced when adding position would exceed max_sector_pct
  - Validates: Requirements 2.6, 9.4
- 4.5 Write property test for diversification bonus
  - Property 4: Diversification bonus for under-represented sectors
  - Generate portfolios with varying sector counts
  - Verify 1.2x bonus applied when portfolio has < 3 sectors and trade is in new sector
  - Verify no bonus when portfolio has >= 3 sectors
  - Validates: Requirements 9.5
- 4.6 Write property test for Active Pool computation
  - Property 5: Active Pool computation invariant
  - Generate random total_portfolio_value and reserve_pool_balance
  - Verify Active Pool = total_portfolio_value - reserve_pool_balance
  - Validates: Requirements 3.3
- 4.7 Write property test for earnings proximity adjustments
  - Property 19: Earnings proximity adjustments
  - Generate random earnings dates relative to current date
  - Verify 50% reduction within 3 trading days
  - Verify rejection within 1 trading day
  - Verify normal sizing outside earnings window
  - Validates: Requirements 10.2, 10.3
- 4.8 Write property test for portfolio heat computation and enforcement
  - Property 24: Portfolio heat computation and threshold enforcement
  - Generate random open positions with entry prices and stop-loss levels
  - Verify heat = sum of position_value * (entry_price - stop_loss_price) / entry_price
  - Verify new entries rejected when heat exceeds max_portfolio_heat
  - Validates: Requirements 13.1, 13.2
- 4.9 Write property test for Active Pool minimum halts entries
  - Property 7: Active Pool minimum halts new entries but allows exits
  - Generate portfolio states with Active Pool below and above minimum
  - Verify buy orders rejected when Active Pool < minimum
  - Verify sell orders allowed regardless of Active Pool
  - Validates: Requirements 3.5
5. Checkpoint — Ensure position sizer logic and property tests pass
- Ensure all tests pass, ask the user if questions arise.
6. Stop-Loss Manager implementation
- 6.1 Implement services/trading/stop_loss_manager.py
  - Implement StopLossManager class with compute_initial_levels() method: stop_loss = entry_price - (ATR * stop_loss_atr_multiplier), take_profit = entry_price + (stop_distance * reward_risk_ratio)
  - Implement re_evaluate_levels() method: adjust if ATR changed > 10% or signal conditions changed; respect configurable interval (default 5 min)
  - Implement check_price_crossings() method: return list of StopTrigger for positions where current price <= stop_loss or >= take_profit
  - Implement trailing stop logic: when price moves favorably by > 50% of take-profit distance, move stop-loss to entry price (breakeven)
  - Implement earnings proximity tightening: 0.7x ATR multiplier when earnings within 3 trading days
  - Implement high-severity event tightening: 0.5x normal ATR multiplier during active macro events
  - Implement proactive heat tightening: tighten stops on lowest-confidence positions when heat > 80% of max
  - Implement price data unavailability safety: close position if price unavailable > 15 minutes during market hours
  - Persist all stop levels and adjustments to position_stop_levels table
  - Requirements: 4.1, 4.2, 4.3, 4.4, 4.5, 4.6, 4.7, 4.8, 7.2, 10.2, 13.3
- 6.2 Write property test for stop-loss and take-profit initial computation
  - Property 9: Stop-loss and take-profit initial computation
  - Generate random entry prices, ATR values, and RiskTierConfig objects
  - Verify stop-loss = entry_price - (ATR * multiplier) and always below entry
  - Verify take-profit = entry_price + (stop_distance * reward_risk_ratio) and always above entry
  - Validates: Requirements 4.1, 4.2
- 6.3 Write property test for price crossing triggers
  - Property 10: Price crossing triggers immediate sell
  - Generate random positions with stop/take-profit levels and current prices
  - Verify sell triggered when price <= stop_loss or >= take_profit
  - Verify no trigger when price is between stop_loss and take_profit
  - Validates: Requirements 4.4, 4.5
- 6.4 Write property test for trailing stop activation
  - Property 11: Trailing stop activation at 50% of take-profit distance
  - Generate random positions with varying favorable price moves
  - Verify trailing stop activates (stop moves to entry) when move > 50% of TP distance
  - Verify trailing stop does not activate when move <= 50%
  - Validates: Requirements 4.6
- 6.5 Write property test for stop tightening during high-severity events
  - Property 15: Stop tightening during high-severity events
  - Generate random positions and ATR values
  - Verify tightened stop uses 0.5x normal multiplier
  - Verify tightened stop is closer to current price than normal stop
  - Validates: Requirements 7.2
- 6.6 Write property test for proactive stop tightening at 80% heat
  - Property 25: Proactive stop tightening at 80% heat threshold
  - Generate portfolios with heat near the threshold
  - Verify lowest-confidence positions get stops tightened first
  - Validates: Requirements 13.3
7. Reserve Pool Controller implementation
- 7.1 Implement services/trading/reserve_pool.py
  - Implement ReservePoolController class with siphon_profit() method: transfer configured percentage of realized profit to reserve, persist to reserve_pool_ledger
  - Implement emergency_liquidate() method: release entire reserve into active pool, log event, persist to ledger
  - Implement compute_active_pool() method: total_portfolio_value - reserve_pool_balance
  - Implement get_state() method: load current balance and history from PostgreSQL
  - Implement high-water mark detection: signal when reserve > 30% of total portfolio
  - Requirements: 3.1, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7
- 7.2 Write property test for reserve pool siphon computation
  - Property 6: Reserve pool siphon computation
  - Generate random realized profit amounts and siphon percentages
  - Verify transferred amount = realized_profit * siphon_pct
  - Verify balance_after = previous_balance + transferred_amount
  - Validates: Requirements 3.1, 3.2
- 7.3 Write property test for emergency drawdown triggers reserve liquidation
  - Property 8: Emergency drawdown triggers reserve liquidation
  - Generate portfolio states with drawdowns above and below emergency threshold
  - Verify reserve liquidated into active pool when drawdown exceeds threshold
  - Verify risk tier set to conservative after emergency liquidation
  - Validates: Requirements 3.6
8. Circuit Breaker implementation
- 8.1 Implement services/trading/circuit_breaker.py
  - Implement CircuitBreaker class with check_daily_loss(): activate when portfolio drops > configured daily_loss_pct
  - Implement check_single_position(): close position and apply ticker cooldown when loss > configured single_position_loss_pct
  - Implement check_volatility(): pause trading when 3+ positions hit stop-losses within 30-minute window
  - Implement is_ticker_cooled_down(): check per-ticker re-entry cooldowns
  - Implement is_active(): return whether any circuit breaker is currently active
  - Implement cooldown expiry: auto-resolve when current time > triggered_at + cooldown_duration
  - Persist all circuit breaker events to circuit_breaker_events table
  - Store active state in Redis for fast lookup
  - Requirements: 6.1, 6.2, 6.3, 6.4, 6.5, 6.6
- 8.2 Write property test for circuit breaker activation
  - Property 13: Circuit breaker activation
  - Generate random portfolio states with varying daily losses, position losses, and stop-loss hit sequences
  - Verify daily_loss trigger when loss > threshold
  - Verify single_position trigger and ticker cooldown when position loss > threshold
  - Verify volatility trigger when 3+ stop-losses within 30 minutes
  - Verify all new orders rejected when any circuit breaker is active
  - Validates: Requirements 6.1, 6.2, 6.3
- 8.3 Write property test for circuit breaker cooldown expiry
  - Property 14: Circuit breaker cooldown expiry
  - Generate circuit breaker events with varying cooldown durations and current times
  - Verify transition from active to resolved when time > triggered_at + cooldown
  - Verify remains active before expiry
  - Validates: Requirements 6.5
9. Risk Tier Controller implementation
- 9.1 Implement services/trading/risk_tier_controller.py
  - Implement RiskTierController class with evaluate() method accepting PerformanceMetrics and reserve_pct
  - Implement downgrade logic: downgrade one level when trailing 30-day win rate < 40% OR current drawdown > 15%
  - Implement upgrade logic: upgrade one level when win rate > 55% AND reserve > 20% of total AND drawdown < 5%
  - Implement tier bounds: never go below conservative or above aggressive
  - Persist tier changes to risk_tier_history table with previous tier, new tier, and trigger metrics
  - Requirements: 5.2, 5.3, 5.4, 5.5, 5.6
- 9.2 Write property test for risk tier auto-adjustment conditions
  - Property 12: Risk tier auto-adjustment conditions
  - Generate random performance metrics (win rate, drawdown, reserve percentage)
  - Verify downgrade when win rate < 40% OR drawdown > 15%
  - Verify upgrade when win rate > 55% AND reserve > 20% AND drawdown < 5%
  - Verify no change when neither condition met
  - Verify tier never goes below conservative or above aggressive
  - Validates: Requirements 5.3, 5.4
10. Checkpoint — Ensure core components and property tests pass
- Ensure all tests pass, ask the user if questions arise.
11. Correlation Matrix and Tax Lot Tracker
- 11.1 Implement services/trading/correlation.py
  - Implement CorrelationMatrix class with refresh() method: compute trailing 90-day price correlations from market data tables, persist to correlation_matrix_cache table
  - Implement get_correlation() method: return coefficient for a ticker pair, 0.0 if unknown
  - Implement get_portfolio_correlation() method: weighted average correlation between candidate and existing positions
  - Cache in-memory after refresh; schedule daily refresh
  - Requirements: 9.1, 9.2, 9.3
- 11.2 Implement services/trading/tax_lots.py
  - Implement TaxLotTracker class with record_entry() method: create tax lot record in PostgreSQL
  - Implement close_lots_fifo() method: close lots in FIFO order, compute realized P&L per lot
  - Implement check_wash_sale() method: check 30-day window before and after for same-ticker purchases
  - Persist wash sale flags and details to tax_lots table
  - Requirements: 12.1, 12.2, 12.3, 12.4
- 11.3 Write property test for tax lot FIFO ordering
  - Property 22: Tax lot FIFO ordering
  - Generate random sequences of buy/sell transactions for the same ticker
  - Verify lots closed in FIFO order (earliest acquired first)
  - Verify realized P&L = (exit_price - cost_basis_per_share) * quantity per lot
  - Validates: Requirements 12.4
- 11.4 Write property test for wash sale detection
  - Property 23: Wash sale detection within 30-day window
  - Generate random loss-closing dates and purchase dates
  - Verify wash sale flagged when same ticker purchased within 30 days before or after loss
  - Verify no flag when purchases are outside the 30-day window
  - Validates: Requirements 12.2, 12.3
12. Trading Window and Gradual Entry logic
- 12.1 Implement services/trading/trading_window.py
  - Implement is_within_trading_window() function: return True if timestamp is between 9:45 AM ET and 3:45 PM ET on a market day
  - Implement next_window_open() function: return the next timestamp when the trading window opens
  - Implement is_market_open() function: check if current time is during US market hours (9:30 AM - 4:00 PM ET)
  - Requirements: 11.1, 11.2
- 12.2 Implement gradual entry logic in services/trading/gradual_entry.py
  - Implement should_use_gradual_entry(): return True when position size exceeds min($30, 5% of Active Pool)
  - Implement split_into_tranches(): split order into configured number of tranches (default 3) of approximately equal size
  - Implement GradualEntryManager class to track pending tranches, re-evaluate before each submission, cancel remaining if conditions deteriorate
  - Link all tranches to the same parent trading decision ID
  - Requirements: 11.3, 11.4, 11.5
- 12.3 Write property test for trading window determination
  - Property 20: Trading window determination
  - Generate random timestamps across US market hours
  - Verify within-window classification for 9:45 AM - 3:45 PM ET
  - Verify outside-window classification for all other times
  - Validates: Requirements 11.1
- 12.4 Write property test for gradual entry tranche splitting
  - Property 21: Gradual entry tranche splitting
  - Generate random position sizes above and below the threshold
  - Verify splitting into configured number of tranches when above threshold
  - Verify all tranches reference the same parent decision ID
  - Verify tranche sizes are approximately equal
  - Validates: Requirements 11.3, 11.5
13. Autonomous Decision Loop (core engine)
- 13.1 Implement services/trading/engine.py
  - Implement TradingEngine class with __init__() accepting asyncpg.Pool, aioredis.Redis, and TradingEngineConfig
  - Implement start() method: load portfolio state from Broker Service (positions, account balance), load active risk tier from PostgreSQL, load reserve pool balance, load circuit breaker status, load open stop levels, enter decision loop
  - Implement stop() method: graceful shutdown — cancel pending tranches, persist state
  - Implement decision_loop() method: poll recommendations at configured interval, evaluate each, size positions, submit orders
  - Implement poll_recommendations() method: fetch from recommendations table where action IN (buy, sell) AND mode IN (paper_eligible, live_eligible) AND generated_at > last_poll_timestamp, ordered by confidence DESC
  - Implement recommendation deduplication: check Redis key stonks:dedupe:trading:{recommendation_id} with 24h TTL, mark before evaluation
  - Implement evaluate_recommendation() method: run all pre-trade checks (circuit breaker, trading window, risk tier confidence, portfolio heat, sector exposure, correlation, earnings proximity) and produce a TradingDecision record
  - Implement execute_decision() method: generate order job payload matching existing broker queue schema, push to stonks:queue:broker_orders, handle gradual entry for large positions
  - Persist every decision (act or skip) to trading_decisions table with full reasoning chain
  - Implement adaptive market response: trigger immediate re-evaluation on high-severity macro events, tighten stops during events, increase polling frequency
  - Implement rapid price move detection: re-evaluate position when price moves > 5% in 15 minutes
  - Implement multiple declining positions halt: stop new entries when > 50% of positions have > 2% negative unrealized P&L
  - Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 7.1, 7.2, 7.3, 7.4, 7.5, 17.1, 17.2
- 13.2 Write property test for recommendation deduplication
  - Property 27: Recommendation deduplication (idempotence)
  - Generate random recommendation IDs and process them twice
  - Verify second processing is a no-op (no new decision record, no order submitted)
  - Validates: Requirements 1.5
- 13.3 Write property test for trading decision record completeness
  - Property 28: Trading decision record completeness and traceability
  - Generate random recommendations and evaluate them
  - Verify all required fields present in the persisted decision record
  - Verify "act" decisions include order job with trading_decision_id
  - Validates: Requirements 1.4, 17.1, 17.2
- 13.4 Write property test for multiple declining positions halts entries
  - Property 16: Multiple declining positions halts new entries
  - Generate portfolio states with varying percentages of declining positions
  - Verify new entries halted when > 50% of positions have > 2% negative unrealized P&L
  - Verify entries allowed when <= 50% are declining
  - Validates: Requirements 7.5
- 13.5 Write property test for maximum open positions enforcement
  - Property 18: Maximum open positions enforcement
  - Generate portfolio states at and below the max position limit
  - Verify new entries rejected at the limit
  - Verify portfolio never exceeds the configured maximum
  - Validates: Requirements 8.4
14. Checkpoint — Ensure decision loop and core engine tests pass
- Ensure all tests pass, ask the user if questions arise.
15. Portfolio Rebalancer and Performance Tracker
- 15.1 Implement services/trading/rebalancer.py
  - Implement PortfolioRebalancer class with evaluate() method accepting positions, risk_tier, and active_pool
  - Generate partial sell orders when single stock exceeds max_position_pct
  - Generate sell orders for lowest-confidence positions when sector exceeds max_sector_pct
  - Enforce maximum open positions limit (default 10)
  - Submit rebalancing orders through normal broker queue with rebalance tag in decision trace
  - Respect trading window and circuit breaker status
  - Schedule: weekly at market open on Monday (configurable)
  - Requirements: 8.1, 8.2, 8.3, 8.4, 8.5, 8.6
- 15.2 Write property test for portfolio rebalancing sell orders
  - Property 17: Portfolio rebalancing generates correct sell orders
  - Generate portfolios with over-concentrated positions and sectors
  - Verify sell orders generated to bring positions within limits
  - Verify lowest-confidence positions targeted first for sector rebalancing
  - Validates: Requirements 8.2, 8.3
- 15.3 Implement services/trading/performance_tracker.py
  - Implement PerformanceTracker class with compute_metrics() method: compute all PerformanceMetrics fields (total portfolio value, active/reserve pool, unrealized/realized P&L, daily P&L, win/loss counts, win rate, avg win/loss, profit factor, Sharpe ratio, max drawdown, current drawdown, portfolio heat)
  - Implement Sharpe ratio: (mean_daily_return / std_daily_return) * sqrt(252) using trailing 30-day daily returns
  - Implement record_trade() method: persist per-trade metrics (entry/exit price, hold duration, P&L, recommendation ID)
  - Implement persist_daily_snapshot() method: save end-of-day snapshot to portfolio_snapshots table
  - Compute metrics every 5 minutes during market hours
  - Track micro-trade metrics separately from standard trade metrics
  - Requirements: 14.1, 14.2, 14.3, 20.7
- 15.4 Write property test for performance metrics computation
  - Property 26: Performance metrics computation
  - Generate random sets of closed trades with entry/exit prices and hold durations
  - Verify win_rate = wins / total_trades
  - Verify profit_factor = gross_profits / gross_losses (infinity if no losses)
  - Verify Sharpe ratio formula consistency
  - Validates: Requirements 14.1, 14.2
- 15.5 Write property test for micro-trade metrics tracked separately
  - Property 33: Micro-trade metrics tracked separately
  - Generate mixed sets of standard and micro-trades
  - Verify micro-trade metrics computed independently
  - Verify standard trade metrics not contaminated by micro-trades
  - Validates: Requirements 20.7
16. Notification Service implementation
- 16.1 Implement services/trading/notifications.py
  - Implement NotificationService class with send_alert() method: send via all enabled channels (SMS via AWS SNS, email via Gmail API)
  - Implement send_daily_summary() method: format and send daily performance summary at configurable time (default 16:30)
  - Implement send_weekly_digest() method: format and send weekly performance digest
  - Implement rate limiting: max 10 SMS/hour, 20 emails/hour (configurable), using Redis counters with hourly TTL
  - Implement retry logic: up to 3 retries with exponential backoff on delivery failure
  - Persist all notifications to notifications table with channel, event_type, message, delivery_status, timestamp
  - Support event types: circuit_breaker_triggered, circuit_breaker_resumed, risk_tier_changed, emergency_liquidation, large_trade_pnl, daily_summary, weekly_digest
  - Notifications run in separate asyncio tasks — never block trading operations
  - Requirements: 19.1, 19.2, 19.3, 19.4, 19.5, 19.6, 19.7, 19.8, 19.11
- 16.2 Write property test for notification rate limiting
  - Property 30: Notification rate limiting
  - Generate random sequences of notification requests within a one-hour window
  - Verify at most 10 SMS and 20 emails delivered per hour
  - Verify excess notifications marked as 'rate_limited'
  - Validates: Requirements 19.7
17. Micro-Trading Module
- 17.1 Implement services/trading/micro_trading.py
  - Implement MicroTradingModule class with poll_intraday_signals() method: fetch intraday and 1d trend window signals from aggregation engine
  - Implement evaluate_micro_trade() method: evaluate signal against risk tier confidence threshold, apply micro-trade allocation cap (3% of Active Pool)
  - Enforce daily micro-trade limit (default 10)
  - Use tighter stop-loss (1.0x ATR) and take-profit (1.5x stop distance)
  - Implement auto-close after max hold duration (default 2 hours)
  - Respect all existing constraints (trading window, circuit breakers, portfolio heat, correlation, sector exposure, earnings)
  - Toggleable independently via trading_engine_config
  - Requirements: 20.1, 20.2, 20.3, 20.4, 20.5, 20.6, 20.8, 20.10
- 17.2 Write property test for micro-trade parameter constraints
  - Property 31: Micro-trade parameter constraints
  - Generate random micro-trade scenarios
  - Verify allocation does not exceed micro_trading_allocation_cap_pct
  - Verify stop-loss at 1.0x ATR, take-profit at 1.5x stop distance
  - Verify daily count does not exceed configured maximum
  - Validates: Requirements 20.3, 20.4, 20.5
- 17.3 Write property test for micro-trade auto-close
  - Property 32: Micro-trade auto-close after max hold duration
  - Generate micro-trade positions with varying hold durations
  - Verify positions closed at market price when hold exceeds max duration
  - Validates: Requirements 20.6
- 17.4 Write property test for micro-trades respect all constraints
  - Property 34: Micro-trades respect all existing constraints
  - Generate micro-trade evaluations with various constraint violations
  - Verify trading window, circuit breakers, portfolio heat, correlation, sector exposure, and earnings rules all enforced
  - Validates: Requirements 20.10
18. Checkpoint — Ensure all trading logic and property tests pass
- Ensure all tests pass, ask the user if questions arise.
19. Backtester implementation
- 19.1 Implement services/trading/backtester.py
  - Implement Backtester class with run() method accepting BacktestConfig (start_date, end_date, initial_capital, risk_tier)
  - Replay historical recommendations from recommendations table within date range
  - Simulate full decision logic: position sizing, stop-loss/take-profit, circuit breakers, reserve pool, rebalancing
  - Use historical price data from market data tables for simulation
  - Produce BacktestResult with total_return, sharpe_ratio, max_drawdown, win_rate, profit_factor, trade_count, trade_log, equity_curve
  - Persist results to backtest_runs and backtest_trades tables with unique backtest_id
  - Handle missing historical data gracefully (skip dates, note gaps)
  - Persist partial results with status 'failed' on mid-run errors
  - Requirements: 15.1, 15.2, 15.3, 15.4
- 19.2 Write property test for backtester produces equivalent metrics
  - Property 36: Backtester produces equivalent metrics
  - Generate random sets of historical trades
  - Verify backtester metric computation matches performance tracker for same trade data
  - Validates: Requirements 15.3
20. Trading Engine FastAPI HTTP Service
- 20.1 Implement services/trading/app.py
  - Create FastAPI application with lifespan handler that starts/stops the TradingEngine
  - Implement GET /health liveness probe endpoint
  - Implement GET /ready readiness probe: return healthy when portfolio loaded and loop active
  - Implement GET /api/trading/status endpoint: return engine state (enabled, risk tier, circuit breaker status, active/reserve pool, portfolio heat, open positions, last decision timestamp)
  - Implement PUT /api/trading/config endpoint: update trading_engine_config, record audit event with previous/new config and change source
  - Implement POST /api/trading/pause and POST /api/trading/resume endpoints
  - Implement GET /api/trading/decisions endpoint: paginated, filterable by ticker, decision type, date range
  - Implement GET /api/trading/metrics endpoint: current performance metrics
  - Implement GET /api/trading/metrics/history endpoint: historical daily snapshots
  - Implement POST /api/trading/backtest endpoint: launch backtest, return backtest_id
  - Implement GET /api/trading/backtest/{id} endpoint: retrieve backtest results
  - Implement GET /api/trading/notifications/config and PUT /api/trading/notifications/config endpoints
  - Implement GET /api/trading/notifications/history endpoint: recent notifications
  - Requirements: 1.7, 5.6, 6.6, 15.5, 16.2, 16.3, 16.4, 17.3, 19.9
- 20.2 Write property test for configuration change audit trail
  - Property 35: Configuration change audit trail
  - Generate random configuration changes via API
  - Verify audit event persisted with previous config, new config, and change source
  - Validates: Requirements 16.6
- 20.3 Write property test for persistence round-trip
  - Property 29: Persistence round-trip for trading engine state
  - Generate random trading engine config, reserve pool entries, risk tier history, circuit breaker events, portfolio snapshots, and backtest results
  - Verify persist-then-read produces equivalent objects with all fields preserved
  - Validates: Requirements 3.2, 4.7, 5.5, 6.4, 14.3, 15.4, 16.1
21. Checkpoint — Ensure API endpoints and backtester work correctly
- Ensure all tests pass, ask the user if questions arise.
22. Kubernetes deployment and infrastructure
- 22.1 Add trading-engine service to Helm chart infra/helm/stonks-oracle/values.yaml
  - Add tradingEngine entry under services: with: replicas 1, image trading-engine, command uvicorn services.trading.app:app --host 0.0.0.0 --port 8000, tier trading, port 8000, secrets [stonks-core-secrets, stonks-broker-secrets], resources (requests: 100m CPU / 256Mi memory, limits: 500m CPU / 512Mi memory), readiness probe on /ready port 8000, liveness probe on /health port 8000
  - Requirements: 1.7, 16.1
- 22.2 Add network policy for trading-engine
  - Allow ingress from query-api, dashboard, and kube-system (Traefik) on port 8000
  - Allow egress to PostgreSQL, Redis, and external services (SNS, Gmail API)
  - Requirements: 16.2
- 22.3 Add /trading/ proxy route to dashboard nginx.conf
  - Add location /trading/ { proxy_pass http://trading-engine:8000/; } to frontend/nginx.conf
  - Requirements: 14.4, 16.5
- 22.4 Add trading-engine ingress if external access needed
  - Add ingress host entry for trading engine API (e.g., stonks-trading.celestium.life) to values.yaml if direct external access is desired, or rely on dashboard proxy
  - Requirements: 16.2
23. Dashboard frontend — Trading Engine panels
- 23.1 Add trading API client hooks to frontend/src/api/
  - Add useTradingStatus() hook: fetch GET /trading/api/trading/status
  - Add useTradingDecisions() hook: fetch GET /trading/api/trading/decisions with pagination and filters
  - Add useTradingMetrics() hook: fetch GET /trading/api/trading/metrics
  - Add useTradingMetricsHistory() hook: fetch GET /trading/api/trading/metrics/history
  - Add useTradingConfig() and useUpdateTradingConfig() hooks for config read/write
  - Add usePauseTradingEngine() and useResumeTradingEngine() mutation hooks
  - Add useBacktestLaunch() and useBacktestResult() hooks
  - Add useNotificationConfig(), useUpdateNotificationConfig(), and useNotificationHistory() hooks
  - Requirements: 14.4, 14.5, 14.6, 14.7, 15.6, 16.5, 17.4, 19.10, 20.9
- 23.2 Implement Trading Engine overview panel component
  - Display current Risk Tier, Circuit Breaker status (active/inactive with trigger reason and cooldown remaining), Active Pool and Reserve Pool balances, Portfolio Heat gauge, last 24h P&L summary
  - Include start/pause/resume controls and Risk Tier selector dropdown
  - Use TanStack Query for data fetching with auto-refresh
  - Requirements: 14.4, 16.5, 6.6
- 23.3 Implement Portfolio Composition panel component
  - Display current positions table: ticker, entry price, current price, unrealized P&L, stop-loss level, take-profit level, sector
  - Display sector allocation pie chart using Recharts
  - Requirements: 14.5
- 23.4 Implement Trade History panel component
  - Display completed trades table: entry/exit prices, P&L amount and percentage, hold duration, recommendation thesis
  - Support pagination and filtering by ticker and date range
  - Requirements: 14.6, 17.4
- 23.5 Implement Performance Charts panel component
  - Display cumulative P&L line chart over time using Recharts
  - Display daily returns bar chart using Recharts
  - Display drawdown chart using Recharts
  - Requirements: 14.7
- 23.6 Implement Backtesting panel component
  - Display backtest configuration form: date range picker, initial capital input, risk tier selector
  - Display backtest results: equity curve chart, trade log table, summary metrics (total return, Sharpe, max drawdown, win rate, profit factor)
  - Support launching new backtests and viewing historical results
  - Requirements: 15.6
- 23.7 Implement Micro-Trading panel component
  - Display micro-trade mode status toggle (enabled/disabled)
  - Display today's micro-trade count and P&L
  - Display active micro-trade positions table
  - Display micro-trade performance metrics over trailing 7 days
  - Requirements: 20.9
- 23.8 Implement Notification Preferences panel component
  - Display notification channel toggles (SMS, email) with phone number and email address inputs
  - Display event type selection checkboxes
  - Display rate limit configuration
  - Display recent notification history table
  - Requirements: 19.10
- 23.9 Wire trading panels into dashboard routing
  - Add Trading page route to TanStack Router configuration
  - Add navigation link to the dashboard sidebar/header
  - Compose all trading panels (overview, portfolio, trade history, performance, backtesting, micro-trading, notifications) into the Trading page layout
  - Requirements: 14.4, 16.5
24. Checkpoint — Ensure frontend builds and all tests pass
- Ensure all tests pass, ask the user if questions arise.
25. Integration wiring and final validation
- 25.1 Wire stop-loss price crossing detection into the decision loop
  - Connect StopLossManager.check_price_crossings() to run at the configured interval (5 min default, 60s during high-severity events)
  - Generate immediate sell orders for triggered positions and submit to broker queue
  - Handle price data unavailability (close position after 15 min without data)
  - Requirements: 4.3, 4.4, 4.5, 4.8, 7.4
- 25.2 Wire reserve pool siphoning to position close events
  - Detect profitable position closes from broker service fill events
  - Call ReservePoolController.siphon_profit() with realized profit
  - Trigger notification for large trade P&L events
  - Requirements: 3.1, 19.2
- 25.3 Wire risk tier evaluation to daily market close schedule
  - Schedule RiskTierController.evaluate() at market close
  - Trigger notification on tier changes
  - Requirements: 5.2, 19.2
- 25.4 Wire portfolio rebalancer to weekly schedule
  - Schedule PortfolioRebalancer.evaluate() weekly at Monday market open
  - Submit rebalancing orders through broker queue
  - Requirements: 8.1, 8.5
- 25.5 Wire notification service to all critical events
  - Connect circuit breaker triggers/resumes, risk tier changes, emergency liquidation, large trade P&L to notification dispatch
  - Schedule daily summary at configured time (default 16:30)
  - Schedule weekly digest
  - Requirements: 19.2, 19.3, 19.4
- 25.6 Wire micro-trading module into the decision loop
  - Start micro-trading polling when enabled in config
  - Route micro-trade decisions through the same order submission pipeline
  - Track micro-trade metrics separately in performance tracker
  - Requirements: 20.1, 20.2, 20.7
- 25.7 Write integration tests for end-to-end decision flow
  - Test full cycle: recommendation → evaluation → position sizing → order submission to broker queue
  - Test stop-loss crossing → immediate sell order
  - Test reserve pool siphoning on profitable close
  - Test circuit breaker trigger → halt → cooldown → resume
  - Test engine startup state reconstruction from PostgreSQL
  - Requirements: 1.1, 1.2, 1.3, 3.1, 4.4, 6.1, 6.5, 18.5
26. Final checkpoint — Ensure all tests pass
- Ensure all tests pass, ask the user if questions arise.

Notes

Tasks marked with * are optional and can be skipped for faster MVP
Each task references specific requirements for traceability
Checkpoints ensure incremental validation after each major component
Property tests validate the 36 correctness properties defined in the design document
The trading engine is a NEW service at services/trading/ — it does not replace existing services
All order submission goes through the existing stonks:queue:broker_orders Redis queue consumed by the Broker Service
Migration number 018 is the next available migration slot
Frontend components use the existing React 19 + TypeScript + Tailwind + TanStack + Recharts stack
Dashboard proxy needs /trading/ → trading-engine:8000 added to nginx.conf

Phase 2: Live Wiring and Paper Trading

Overview

Phase 1 (Tasks 1–26) implemented all pure computation modules, property tests, FastAPI endpoints, Helm chart, and frontend panels. Phase 2 replaces the lifecycle stubs in services/trading/engine.py with real async implementations, wires all sub-components into live loops backed by PostgreSQL and Redis, adds notification dispatch and backtest replay, connects real database pools in the FastAPI lifespan, enables paper trading configuration, and adds integration tests for the live wiring.

Tasks

27. Wire the live decision loop in services/trading/engine.py
- 27.1 Replace start() stub with real async implementation
  - Load trading_engine_config from PostgreSQL via self.pool
  - Load active risk tier parameters from risk_tier_history (latest entry) or fall back to config default
  - Sync portfolio state from Broker Service: fetch positions and account balance via asyncpg query against the broker's orders / positions tables
  - Load reserve pool balance from reserve_pool_ledger (latest balance_after)
  - Load circuit breaker status from circuit_breaker_events (unresolved events)
  - Load open stop-loss/take-profit levels from position_stop_levels where active = TRUE
  - Populate self.portfolio_state with loaded data
  - Create asyncio.Task instances for _decision_loop(), _stop_loss_monitor(), _performance_loop(), _risk_tier_scheduler(), _rebalance_scheduler() and store in self._tasks: list[asyncio.Task]
  - Set self.running = True only after successful state load
  - If portfolio state cannot be loaded, enter degraded state (readiness probe unhealthy), retry every 30 seconds
  - Requirements: 1.6, 18.5
- 27.2 Replace stop() stub with real async shutdown
  - Set self.running = False
  - Cancel all tasks in self._tasks and await asyncio.gather(*self._tasks, return_exceptions=True)
  - Persist current portfolio state snapshot to portfolio_snapshots
  - Close any pending gradual entry tranches
  - Log shutdown event
  - Requirements: 1.6, 16.4
- 27.3 Implement _decision_loop() coroutine
  - while self.running: sleep for self.config.polling_interval_seconds, then poll recommendations
  - Poll recommendations from PostgreSQL: SELECT * FROM recommendations WHERE action IN ('buy','sell') AND mode IN ('paper_eligible','live_eligible') AND generated_at > $1 ORDER BY confidence DESC
  - For each recommendation, check Redis deduplication key stonks:dedupe:trading:{recommendation_id} (24h TTL) — skip if already set
  - Set the Redis dedupe key immediately before evaluation to prevent double-processing on restart
  - Call self.evaluate_recommendation() (existing synchronous method) with current portfolio state, risk tier, circuit breaker state, correlation matrix, and earnings calendar
  - For "act" decisions: generate order job payload matching existing broker queue schema, push to stonks:queue:broker_orders via Redis RPUSH; handle gradual entry for large positions
  - Call _persist_decision() for every decision (act or skip)
  - Update self.portfolio_state after each acted decision (reduce active pool, increment open position count)
  - Wrap each recommendation evaluation in try/except — on failure, persist skip decision with reason evaluation_error and continue
  - Requirements: 1.1, 1.2, 1.3, 1.4, 1.5
- 27.4 Implement _sync_positions_and_siphon() helper
  - Fetch current positions and account balance from PostgreSQL (broker tables)
  - Detect newly closed positions by comparing with previous self.portfolio_state.positions
  - For each profitable close: call self.reserve_pool_controller.siphon_profit() with realized profit and current reserve balance
  - Persist siphon event to reserve_pool_ledger via self.pool
  - Update self.portfolio_state with refreshed positions, active pool, reserve pool
  - Trigger notification for large trade P&L events (> 5% of Active Pool)
  - Requirements: 3.1, 3.2, 3.3, 19.2
- 27.5 Implement _persist_decision() helper
  - INSERT into trading_decisions table with all fields from the TradingDecision dataclass
  - Use self.pool.execute() with parameterized query
  - Log decision summary (ticker, decision, skip_reason if any)
  - Requirements: 1.4, 17.1
- 27.6 Implement asyncio task management
  - Add self._tasks: list[asyncio.Task] = [] to __init__()
  - In start(), create named tasks: asyncio.create_task(self._decision_loop(), name="decision_loop"), etc.
  - In stop(), cancel all tasks, await with return_exceptions=True, clear the list
  - Add error handling: if a task raises an unexpected exception, log it and restart the task (unless self.running is False)
  - Requirements: 1.1, 1.6
28. Wire the stop-loss monitoring loop
- 28.1 Implement _stop_loss_monitor() coroutine in services/trading/engine.py
  - while self.running: sleep for self.config.stop_loss_check_interval_seconds (default 300s, or fast_stop_loss_interval_seconds = 60s during high-severity events)
  - Call _load_open_positions() and _load_stop_levels() from PostgreSQL
  - Call _fetch_current_prices() for all tickers with open positions
  - Call self.check_stop_loss_crossings(positions, prices, stop_levels) (existing method delegates to StopLossManager)
  - For each StopTrigger returned: generate immediate market sell order, push to stonks:queue:broker_orders via Redis
  - Persist stop-loss trigger event to trading_decisions with decision="act" and trace noting stop-loss/take-profit trigger
  - Requirements: 4.3, 4.4, 4.5, 7.4
- 28.2 Implement _fetch_current_prices() helper
  - Query the market data adapter (Polygon API) for current/latest prices of given tickers
  - Use services/shared/config.py MarketDataConfig for API key and base URL
  - Return dict[str, float] mapping ticker → latest price
  - On API failure: log warning, return empty dict for failed tickers
  - Requirements: 4.3, 4.8
- 28.3 Implement _load_open_positions() and _load_stop_levels() helpers
  - _load_open_positions(): query broker/positions tables via self.pool to get current open positions, return as list[OpenPosition]
  - _load_stop_levels(): query position_stop_levels WHERE active = TRUE via self.pool, return as dict[str, StopLevels] keyed by ticker
  - Requirements: 4.3, 18.3
- 28.4 Implement safety sell for missing price data
  - Track last successful price fetch timestamp per ticker
  - If a ticker has no price data for > 15 minutes during market hours (checked via is_market_open()), generate a market sell order for that position
  - Log warning with ticker and duration of missing data
  - Requirements: 4.8
29. Wire the performance metrics loop
- 29.1 Implement _performance_loop() coroutine in services/trading/engine.py
  - while self.running: sleep 300 seconds (5 minutes)
  - Check if currently within market hours via is_market_open(); skip computation if outside market hours
  - Call self.performance_tracker.compute_metrics() with current portfolio state from self.pool
  - Update self.portfolio_state with latest metrics (portfolio heat, unrealized P&L, etc.)
  - Requirements: 14.1
- 29.2 Implement daily snapshot persistence
  - At end of trading day (after 4:00 PM ET), call self.performance_tracker.persist_daily_snapshot() to write to portfolio_snapshots table via self.pool
  - Include end-of-day portfolio value, daily return, cumulative return, all positions with unrealized P&L, and computed metrics
  - Requirements: 14.3
- 29.3 Wire performance tracker to use real database pool
  - Pass self.pool to PerformanceTracker so it can query closed trades from trading_decisions and broker fill tables
  - Compute Sharpe ratio from portfolio_snapshots trailing 30-day daily returns
  - Compute win/loss counts and profit factor from closed trade records
  - Requirements: 14.1, 14.2
30. Wire risk tier and rebalance schedulers
- 30.1 Implement _risk_tier_scheduler() coroutine in services/trading/engine.py
  - while self.running: compute seconds until next 16:00 ET, sleep until then
  - Load latest PerformanceMetrics from portfolio_snapshots or compute fresh
  - Compute reserve_pct = self.portfolio_state.reserve_pool / self.portfolio_state.total_value
  - Call self.evaluate_risk_tier(current_tier, metrics, reserve_pct) (existing method delegates to RiskTierController)
  - If tier changed: persist to risk_tier_history via self.pool, update self.config.risk_tier, trigger notification via self.create_alert("risk_tier_changed", ...)
  - Requirements: 5.2, 5.5, 19.2
- 30.2 Implement _rebalance_scheduler() coroutine in services/trading/engine.py
  - while self.running: compute seconds until next Monday 09:45 ET, sleep until then
  - Load current positions and active risk tier
  - Call self.evaluate_rebalancing(positions, risk_tier, active_pool) (existing method delegates to PortfolioRebalancer)
  - For each rebalance order returned: generate order job with rebalance tag in decision trace, push to stonks:queue:broker_orders
  - Persist rebalance decisions to trading_decisions table
  - Respect circuit breaker status — skip rebalancing if any circuit breaker is active
  - Requirements: 8.1, 8.5, 8.6
31. Wire notification dispatch
- 31.1 Create services/trading/notification_dispatch.py with NotificationDispatcher class
  - Accept pool, redis, and TradingConfig in constructor
  - Implement dispatch(event_type: str, message: str) method that routes to enabled channels
  - Check self.config.sns_topic_arn / self.config.gmail_recipient to determine enabled channels
  - Call _send_sns() and/or _send_gmail() based on enabled channels
  - Persist notification record to notifications table via self.pool with channel, event_type, message, delivery_status, timestamp
  - Requirements: 19.1, 19.8
- 31.2 Implement SNS delivery via boto3
  - Implement _send_sns(event_type: str, message: str) method
  - Use boto3.client("sns") with credentials from environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
  - Publish to configured sns_topic_arn with message and subject based on event_type
  - Return delivery status (delivered/failed)
  - Requirements: 19.5
- 31.3 Implement Gmail delivery via google-api-python-client
  - Implement _send_gmail(event_type: str, message: str) method
  - Use google.oauth2.credentials.Credentials with refresh token from environment variables
  - Build Gmail API service, create MIME message, send via users().messages().send()
  - Configurable sender (self.config.gmail_sender) and recipient (self.config.gmail_recipient)
  - Return delivery status (delivered/failed)
  - Requirements: 19.6
- 31.4 Implement rate limiting via Redis
  - Before each send, check Redis counter stonks:trading:notification_rate:{channel} with 1-hour TTL
  - If counter >= limit (10 SMS/hour, 20 emails/hour), mark notification as rate_limited and skip delivery
  - Increment counter on successful delivery
  - Requirements: 19.7
- 31.5 Implement retry with exponential backoff
  - On delivery failure, retry up to 3 times with delays: 1s, 2s, 4s
  - Update notification record with retry_count and error_message on final failure
  - Never block trading operations — run dispatch in a separate asyncio.create_task()
  - Requirements: 19.11
- 31.6 Implement daily summary at 16:30 ET
  - Add _daily_summary_scheduler() coroutine: sleep until 16:30 ET each trading day
  - Compute daily metrics from portfolio_snapshots and current portfolio state
  - Format summary message with: daily P&L, total portfolio value, Active/Reserve Pool balances, trade count, current Risk Tier, circuit breaker status
  - Dispatch via self.dispatch("daily_summary", summary_message)
  - Requirements: 19.3
32. Wire backtest replay
- 32.1 Create services/trading/backtest_replay.py with BacktestReplay class
  - Accept pool: asyncpg.Pool in constructor
  - Implement run(config: BacktestConfig) -> BacktestResult method
  - Requirements: 15.1
- 32.2 Fetch historical recommendations and price data
  - Query recommendations table for date range: WHERE generated_at BETWEEN $1 AND $2 AND action IN ('buy','sell') ORDER BY generated_at ASC
  - Query market data tables for historical daily close prices within the date range
  - Build a day-by-day timeline of recommendations and prices
  - Handle missing data gracefully: skip dates with no price data, note gaps in result
  - Requirements: 15.1, 15.2
- 32.3 Simulate full decision logic chronologically
  - Initialize simulated portfolio state with config.initial_capital and configured risk tier
  - For each trading day in the date range, process recommendations through evaluate_recommendation() using historical prices
  - Simulate stop-loss/take-profit crossings using historical intraday or daily price data
  - Simulate reserve pool siphoning on profitable closes
  - Simulate circuit breaker triggers based on simulated daily P&L
  - Simulate risk tier auto-adjustment at daily close
  - Simulate weekly rebalancing on Mondays
  - Track equity curve: [{date, portfolio_value}] for each trading day
  - Requirements: 15.2
- 32.4 Persist results to backtest_runs and backtest_trades
  - INSERT into backtest_runs with config, result metrics (total_return, sharpe_ratio, max_drawdown, win_rate, profit_factor, trade_count), equity_curve JSONB, status='completed'
  - INSERT into backtest_trades for each simulated trade with ticker, side, entry/exit prices, quantity, pnl, dates, hold_duration, recommendation_id
  - On mid-run error: persist partial results with status='failed' and error message
  - Requirements: 15.4
- 32.5 Wire into POST /api/trading/backtest endpoint in services/trading/app.py
  - Replace placeholder launch_backtest() with real implementation
  - Instantiate BacktestReplay(pool=engine.pool) and call run() in a background asyncio.Task
  - Return backtest_id immediately
  - Update GET /api/trading/backtest/{id} to query backtest_runs and backtest_trades from PostgreSQL
  - Requirements: 15.5
33. Wire real connections in services/trading/app.py lifespan
- 33.1 Replace pool=None with asyncpg.create_pool()
  - In the lifespan async with block, create pool: pool = await asyncpg.create_pool(dsn=config.postgres.dsn, min_size=2, max_size=10)
  - Pass pool to TradingEngine(pool=pool, ...)
  - In lifespan exit: await pool.close()
  - Requirements: 1.6, 18.5
- 33.2 Replace redis=None with aioredis.from_url()
  - In the lifespan block, create Redis client: redis_client = aioredis.from_url(config.redis.url)
  - Pass redis_client to TradingEngine(pool=pool, redis=redis_client, ...)
  - In lifespan exit: await redis_client.close()
  - Requirements: 1.5
- 33.3 Add proper error handling and cleanup in lifespan
  - Wrap pool/redis creation in try/except — log critical error and raise if connections fail
  - Ensure engine.stop(), pool.close(), and redis_client.close() are called in the finally block of lifespan exit
  - Log connection details (host, port, database) at startup for debugging (not passwords)
  - Requirements: 1.6, 18.5
34. Checkpoint — Ensure all live wiring compiles and existing tests still pass
- Ensure all tests pass, ask the user if questions arise.
35. Enable paper trading configuration
- 35.1 Update trading_engine_config defaults for paper trading
  - Add a SQL migration or seed script that updates the default trading_engine_config row: enabled=true, risk_tier='conservative', absolute_position_cap=25.0 (conservative for initial paper trading)
  - Set polling_interval_seconds=60, max_open_positions=5 (conservative start)
  - Requirements: 16.1, 5.1
- 35.2 Add TRADING_ENABLED=true to Helm values for trading-engine deployment
  - Update infra/helm/stonks-oracle/values.yaml to set TRADING_ENABLED: "true" in the trading-engine environment variables
  - Ensure TRADING_RISK_TIER: "conservative" and TRADING_ABSOLUTE_POSITION_CAP: "25.0" are set
  - Requirements: 16.1
- 35.3 Verify trading-engine pod starts and readiness probe passes
  - After deployment, confirm the trading-engine pod reaches Running state
  - Confirm GET /ready returns {"ready": true} once portfolio state is loaded
  - Confirm GET /health returns {"status": "ok"}
  - Confirm GET /api/trading/status returns the expected configuration (enabled=true, risk_tier=conservative)
  - Requirements: 1.7, 16.2
36. Write integration tests for live wiring
- * 36.1 Test decision loop with mocked PostgreSQL and Redis
  - Create tests/test_trading_integration.py
  - Mock asyncpg.Pool to return canned recommendation rows and portfolio state
  - Mock Redis client for deduplication checks and broker queue pushes
  - Verify the decision loop polls recommendations, evaluates them, persists decisions, and pushes "act" orders to the broker queue
  - Verify deduplication prevents double-processing
  - Verify skip decisions are persisted with correct reasons
  - Requirements: 1.1, 1.2, 1.3, 1.4, 1.5
- * 36.2 Test stop-loss monitor with mocked price API
  - Mock _fetch_current_prices() to return prices that cross stop-loss and take-profit levels
  - Verify sell orders are generated and pushed to broker queue for triggered positions
  - Verify no orders generated when prices are between stop and take-profit
  - Test safety sell: mock price fetch returning empty for > 15 minutes, verify position closed
  - Requirements: 4.4, 4.5, 4.8
- * 36.3 Test notification dispatch with mocked SNS and Gmail
  - Mock boto3.client("sns") and Gmail API service
  - Verify SNS publish called with correct topic ARN and message for SMS-enabled events
  - Verify Gmail send called with correct sender/recipient for email-enabled events
  - Verify rate limiting: send 11 SMS in one hour, verify 11th is marked rate_limited
  - Verify retry: mock first delivery failure, verify retry with backoff, verify final success
  - Requirements: 19.1, 19.5, 19.6, 19.7, 19.11
- * 36.4 Test backtest replay end-to-end
  - Mock PostgreSQL pool to return historical recommendations and price data
  - Run BacktestReplay.run() with a small date range and $500 initial capital
  - Verify backtest result contains expected metrics (total_return, sharpe_ratio, max_drawdown, win_rate, trade_count)
  - Verify equity curve has one entry per trading day
  - Verify trades are persisted to backtest_trades (mocked INSERT calls)
  - Requirements: 15.1, 15.2, 15.3, 15.4
- * 36.5 Test lifespan creates real pool and redis connections
  - Use httpx.AsyncClient with the FastAPI app and mock asyncpg.create_pool / aioredis.from_url
  - Verify pool and redis are created during startup and closed during shutdown
  - Verify engine receives non-None pool and redis
  - Requirements: 1.6, 18.5
37. Final checkpoint — Verify paper trading is operational
- Ensure all tests pass, ask the user if questions arise.
- Trading engine pod is running and ready
- Decision loop is polling recommendations from PostgreSQL
- Stop-loss monitor is checking prices at configured interval
- Performance metrics are being computed every 5 minutes during market hours
- Dashboard shows trading engine status as enabled with conservative tier

Phase 2 Notes

Phase 2 tasks build on the completed Phase 1 pure computation modules — no Phase 1 code is rewritten, only the lifecycle stubs are replaced
All async loops use while self.running pattern with asyncio.sleep() for clean shutdown
Database connections are created in the FastAPI lifespan and passed to the engine — no global connection state
Integration tests use mocked database pools and Redis clients to avoid requiring live infrastructure
Paper trading starts with conservative settings (risk_tier=conservative, absolute_position_cap=$25, max_open_positions=5) to validate behavior before scaling up
Tasks 36.x (integration tests) are marked optional (*) — they can be skipped for faster deployment but are recommended

66 KiB Raw Permalink Blame History Unescape Escape

Implementation Plan: Autonomous Trading Engine

Overview

Tasks

Notes

Phase 2: Live Wiring and Paper Trading

Overview

Tasks

Phase 2 Notes

66 KiB

Raw Permalink Blame History