From 8050f4a03b8a0eb16e784858d21e60b5b87eed5a Mon Sep 17 00:00:00 2001 From: Celes Renata Date: Thu, 16 Apr 2026 07:27:35 +0000 Subject: [PATCH] chore: mark all Phase 2 tasks (27-37) as complete All 152 tasks across both phases are now marked complete: - Phase 1 (1-26): pure computation modules, property tests, API, frontend, infra - Phase 2 (27-37): live decision loop, stop-loss monitor, performance metrics, risk tier scheduler, rebalancer, notification dispatch, backtest replay, real DB connections, paper trading config, integration tests --- .../specs/autonomous-trading-engine/tasks.md | 96 +++++++++---------- 1 file changed, 48 insertions(+), 48 deletions(-) diff --git a/.kiro/specs/autonomous-trading-engine/tasks.md b/.kiro/specs/autonomous-trading-engine/tasks.md index 3c2abfd..288a843 100644 --- a/.kiro/specs/autonomous-trading-engine/tasks.md +++ b/.kiro/specs/autonomous-trading-engine/tasks.md @@ -664,8 +664,8 @@ Phase 1 (Tasks 1–26) implemented all pure computation modules, property tests, ### Tasks -- [ ] 27. Wire the live decision loop in `services/trading/engine.py` - - [ ] 27.1 Replace `start()` stub with real async implementation +- [x] 27. Wire the live decision loop in `services/trading/engine.py` + - [x] 27.1 Replace `start()` stub with real async implementation - Load `trading_engine_config` from PostgreSQL via `self.pool` - Load active risk tier parameters from `risk_tier_history` (latest entry) or fall back to config default - Sync portfolio state from Broker Service: fetch positions and account balance via `asyncpg` query against the broker's `orders` / `positions` tables @@ -678,7 +678,7 @@ Phase 1 (Tasks 1–26) implemented all pure computation modules, property tests, - If portfolio state cannot be loaded, enter degraded state (readiness probe unhealthy), retry every 30 seconds - _Requirements: 1.6, 18.5_ - - [ ] 27.2 Replace `stop()` stub with real async shutdown + - [x] 27.2 Replace `stop()` stub with real async shutdown - Set `self.running = False` - Cancel all tasks in `self._tasks` and `await asyncio.gather(*self._tasks, return_exceptions=True)` - Persist current portfolio state snapshot to `portfolio_snapshots` @@ -686,7 +686,7 @@ Phase 1 (Tasks 1–26) implemented all pure computation modules, property tests, - Log shutdown event - _Requirements: 1.6, 16.4_ - - [ ] 27.3 Implement `_decision_loop()` coroutine + - [x] 27.3 Implement `_decision_loop()` coroutine - `while self.running`: sleep for `self.config.polling_interval_seconds`, then poll recommendations - Poll recommendations from PostgreSQL: `SELECT * FROM recommendations WHERE action IN ('buy','sell') AND mode IN ('paper_eligible','live_eligible') AND generated_at > $1 ORDER BY confidence DESC` - For each recommendation, check Redis deduplication key `stonks:dedupe:trading:{recommendation_id}` (24h TTL) — skip if already set @@ -698,7 +698,7 @@ Phase 1 (Tasks 1–26) implemented all pure computation modules, property tests, - Wrap each recommendation evaluation in try/except — on failure, persist skip decision with reason `evaluation_error` and continue - _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5_ - - [ ] 27.4 Implement `_sync_positions_and_siphon()` helper + - [x] 27.4 Implement `_sync_positions_and_siphon()` helper - Fetch current positions and account balance from PostgreSQL (broker tables) - Detect newly closed positions by comparing with previous `self.portfolio_state.positions` - For each profitable close: call `self.reserve_pool_controller.siphon_profit()` with realized profit and current reserve balance @@ -707,21 +707,21 @@ Phase 1 (Tasks 1–26) implemented all pure computation modules, property tests, - Trigger notification for large trade P&L events (> 5% of Active Pool) - _Requirements: 3.1, 3.2, 3.3, 19.2_ - - [ ] 27.5 Implement `_persist_decision()` helper + - [x] 27.5 Implement `_persist_decision()` helper - INSERT into `trading_decisions` table with all fields from the `TradingDecision` dataclass - Use `self.pool.execute()` with parameterized query - Log decision summary (ticker, decision, skip_reason if any) - _Requirements: 1.4, 17.1_ - - [ ] 27.6 Implement asyncio task management + - [x] 27.6 Implement asyncio task management - Add `self._tasks: list[asyncio.Task] = []` to `__init__()` - In `start()`, create named tasks: `asyncio.create_task(self._decision_loop(), name="decision_loop")`, etc. - In `stop()`, cancel all tasks, await with `return_exceptions=True`, clear the list - Add error handling: if a task raises an unexpected exception, log it and restart the task (unless `self.running` is False) - _Requirements: 1.1, 1.6_ -- [ ] 28. Wire the stop-loss monitoring loop - - [ ] 28.1 Implement `_stop_loss_monitor()` coroutine in `services/trading/engine.py` +- [x] 28. Wire the stop-loss monitoring loop + - [x] 28.1 Implement `_stop_loss_monitor()` coroutine in `services/trading/engine.py` - `while self.running`: sleep for `self.config.stop_loss_check_interval_seconds` (default 300s, or `fast_stop_loss_interval_seconds` = 60s during high-severity events) - Call `_load_open_positions()` and `_load_stop_levels()` from PostgreSQL - Call `_fetch_current_prices()` for all tickers with open positions @@ -730,45 +730,45 @@ Phase 1 (Tasks 1–26) implemented all pure computation modules, property tests, - Persist stop-loss trigger event to `trading_decisions` with decision="act" and trace noting stop-loss/take-profit trigger - _Requirements: 4.3, 4.4, 4.5, 7.4_ - - [ ] 28.2 Implement `_fetch_current_prices()` helper + - [x] 28.2 Implement `_fetch_current_prices()` helper - Query the market data adapter (Polygon API) for current/latest prices of given tickers - Use `services/shared/config.py` `MarketDataConfig` for API key and base URL - Return `dict[str, float]` mapping ticker → latest price - On API failure: log warning, return empty dict for failed tickers - _Requirements: 4.3, 4.8_ - - [ ] 28.3 Implement `_load_open_positions()` and `_load_stop_levels()` helpers + - [x] 28.3 Implement `_load_open_positions()` and `_load_stop_levels()` helpers - `_load_open_positions()`: query broker/positions tables via `self.pool` to get current open positions, return as `list[OpenPosition]` - `_load_stop_levels()`: query `position_stop_levels WHERE active = TRUE` via `self.pool`, return as `dict[str, StopLevels]` keyed by ticker - _Requirements: 4.3, 18.3_ - - [ ] 28.4 Implement safety sell for missing price data + - [x] 28.4 Implement safety sell for missing price data - Track last successful price fetch timestamp per ticker - If a ticker has no price data for > 15 minutes during market hours (checked via `is_market_open()`), generate a market sell order for that position - Log warning with ticker and duration of missing data - _Requirements: 4.8_ -- [ ] 29. Wire the performance metrics loop - - [ ] 29.1 Implement `_performance_loop()` coroutine in `services/trading/engine.py` +- [x] 29. Wire the performance metrics loop + - [x] 29.1 Implement `_performance_loop()` coroutine in `services/trading/engine.py` - `while self.running`: sleep 300 seconds (5 minutes) - Check if currently within market hours via `is_market_open()`; skip computation if outside market hours - Call `self.performance_tracker.compute_metrics()` with current portfolio state from `self.pool` - Update `self.portfolio_state` with latest metrics (portfolio heat, unrealized P&L, etc.) - _Requirements: 14.1_ - - [ ] 29.2 Implement daily snapshot persistence + - [x] 29.2 Implement daily snapshot persistence - At end of trading day (after 4:00 PM ET), call `self.performance_tracker.persist_daily_snapshot()` to write to `portfolio_snapshots` table via `self.pool` - Include end-of-day portfolio value, daily return, cumulative return, all positions with unrealized P&L, and computed metrics - _Requirements: 14.3_ - - [ ] 29.3 Wire performance tracker to use real database pool + - [x] 29.3 Wire performance tracker to use real database pool - Pass `self.pool` to `PerformanceTracker` so it can query closed trades from `trading_decisions` and broker fill tables - Compute Sharpe ratio from `portfolio_snapshots` trailing 30-day daily returns - Compute win/loss counts and profit factor from closed trade records - _Requirements: 14.1, 14.2_ -- [ ] 30. Wire risk tier and rebalance schedulers - - [ ] 30.1 Implement `_risk_tier_scheduler()` coroutine in `services/trading/engine.py` +- [x] 30. Wire risk tier and rebalance schedulers + - [x] 30.1 Implement `_risk_tier_scheduler()` coroutine in `services/trading/engine.py` - `while self.running`: compute seconds until next 16:00 ET, sleep until then - Load latest `PerformanceMetrics` from `portfolio_snapshots` or compute fresh - Compute `reserve_pct = self.portfolio_state.reserve_pool / self.portfolio_state.total_value` @@ -776,7 +776,7 @@ Phase 1 (Tasks 1–26) implemented all pure computation modules, property tests, - If tier changed: persist to `risk_tier_history` via `self.pool`, update `self.config.risk_tier`, trigger notification via `self.create_alert("risk_tier_changed", ...)` - _Requirements: 5.2, 5.5, 19.2_ - - [ ] 30.2 Implement `_rebalance_scheduler()` coroutine in `services/trading/engine.py` + - [x] 30.2 Implement `_rebalance_scheduler()` coroutine in `services/trading/engine.py` - `while self.running`: compute seconds until next Monday 09:45 ET, sleep until then - Load current positions and active risk tier - Call `self.evaluate_rebalancing(positions, risk_tier, active_pool)` (existing method delegates to PortfolioRebalancer) @@ -785,8 +785,8 @@ Phase 1 (Tasks 1–26) implemented all pure computation modules, property tests, - Respect circuit breaker status — skip rebalancing if any circuit breaker is active - _Requirements: 8.1, 8.5, 8.6_ -- [ ] 31. Wire notification dispatch - - [ ] 31.1 Create `services/trading/notification_dispatch.py` with `NotificationDispatcher` class +- [x] 31. Wire notification dispatch + - [x] 31.1 Create `services/trading/notification_dispatch.py` with `NotificationDispatcher` class - Accept `pool`, `redis`, and `TradingConfig` in constructor - Implement `dispatch(event_type: str, message: str)` method that routes to enabled channels - Check `self.config.sns_topic_arn` / `self.config.gmail_recipient` to determine enabled channels @@ -794,14 +794,14 @@ Phase 1 (Tasks 1–26) implemented all pure computation modules, property tests, - Persist notification record to `notifications` table via `self.pool` with channel, event_type, message, delivery_status, timestamp - _Requirements: 19.1, 19.8_ - - [ ] 31.2 Implement SNS delivery via `boto3` + - [x] 31.2 Implement SNS delivery via `boto3` - Implement `_send_sns(event_type: str, message: str)` method - Use `boto3.client("sns")` with credentials from environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`) - Publish to configured `sns_topic_arn` with message and subject based on event_type - Return delivery status (delivered/failed) - _Requirements: 19.5_ - - [ ] 31.3 Implement Gmail delivery via `google-api-python-client` + - [x] 31.3 Implement Gmail delivery via `google-api-python-client` - Implement `_send_gmail(event_type: str, message: str)` method - Use `google.oauth2.credentials.Credentials` with refresh token from environment variables - Build Gmail API service, create MIME message, send via `users().messages().send()` @@ -809,39 +809,39 @@ Phase 1 (Tasks 1–26) implemented all pure computation modules, property tests, - Return delivery status (delivered/failed) - _Requirements: 19.6_ - - [ ] 31.4 Implement rate limiting via Redis + - [x] 31.4 Implement rate limiting via Redis - Before each send, check Redis counter `stonks:trading:notification_rate:{channel}` with 1-hour TTL - If counter >= limit (10 SMS/hour, 20 emails/hour), mark notification as `rate_limited` and skip delivery - Increment counter on successful delivery - _Requirements: 19.7_ - - [ ] 31.5 Implement retry with exponential backoff + - [x] 31.5 Implement retry with exponential backoff - On delivery failure, retry up to 3 times with delays: 1s, 2s, 4s - Update notification record with retry_count and error_message on final failure - Never block trading operations — run dispatch in a separate `asyncio.create_task()` - _Requirements: 19.11_ - - [ ] 31.6 Implement daily summary at 16:30 ET + - [x] 31.6 Implement daily summary at 16:30 ET - Add `_daily_summary_scheduler()` coroutine: sleep until 16:30 ET each trading day - Compute daily metrics from `portfolio_snapshots` and current portfolio state - Format summary message with: daily P&L, total portfolio value, Active/Reserve Pool balances, trade count, current Risk Tier, circuit breaker status - Dispatch via `self.dispatch("daily_summary", summary_message)` - _Requirements: 19.3_ -- [ ] 32. Wire backtest replay - - [ ] 32.1 Create `services/trading/backtest_replay.py` with `BacktestReplay` class +- [x] 32. Wire backtest replay + - [x] 32.1 Create `services/trading/backtest_replay.py` with `BacktestReplay` class - Accept `pool: asyncpg.Pool` in constructor - Implement `run(config: BacktestConfig) -> BacktestResult` method - _Requirements: 15.1_ - - [ ] 32.2 Fetch historical recommendations and price data + - [x] 32.2 Fetch historical recommendations and price data - Query `recommendations` table for date range: `WHERE generated_at BETWEEN $1 AND $2 AND action IN ('buy','sell') ORDER BY generated_at ASC` - Query market data tables for historical daily close prices within the date range - Build a day-by-day timeline of recommendations and prices - Handle missing data gracefully: skip dates with no price data, note gaps in result - _Requirements: 15.1, 15.2_ - - [ ] 32.3 Simulate full decision logic chronologically + - [x] 32.3 Simulate full decision logic chronologically - Initialize simulated portfolio state with `config.initial_capital` and configured risk tier - For each trading day in the date range, process recommendations through `evaluate_recommendation()` using historical prices - Simulate stop-loss/take-profit crossings using historical intraday or daily price data @@ -852,61 +852,61 @@ Phase 1 (Tasks 1–26) implemented all pure computation modules, property tests, - Track equity curve: `[{date, portfolio_value}]` for each trading day - _Requirements: 15.2_ - - [ ] 32.4 Persist results to `backtest_runs` and `backtest_trades` + - [x] 32.4 Persist results to `backtest_runs` and `backtest_trades` - INSERT into `backtest_runs` with config, result metrics (total_return, sharpe_ratio, max_drawdown, win_rate, profit_factor, trade_count), equity_curve JSONB, status='completed' - INSERT into `backtest_trades` for each simulated trade with ticker, side, entry/exit prices, quantity, pnl, dates, hold_duration, recommendation_id - On mid-run error: persist partial results with status='failed' and error message - _Requirements: 15.4_ - - [ ] 32.5 Wire into `POST /api/trading/backtest` endpoint in `services/trading/app.py` + - [x] 32.5 Wire into `POST /api/trading/backtest` endpoint in `services/trading/app.py` - Replace placeholder `launch_backtest()` with real implementation - Instantiate `BacktestReplay(pool=engine.pool)` and call `run()` in a background `asyncio.Task` - Return `backtest_id` immediately - Update `GET /api/trading/backtest/{id}` to query `backtest_runs` and `backtest_trades` from PostgreSQL - _Requirements: 15.5_ -- [ ] 33. Wire real connections in `services/trading/app.py` lifespan - - [ ] 33.1 Replace `pool=None` with `asyncpg.create_pool()` +- [x] 33. Wire real connections in `services/trading/app.py` lifespan + - [x] 33.1 Replace `pool=None` with `asyncpg.create_pool()` - In the lifespan `async with` block, create pool: `pool = await asyncpg.create_pool(dsn=config.postgres.dsn, min_size=2, max_size=10)` - Pass `pool` to `TradingEngine(pool=pool, ...)` - In lifespan exit: `await pool.close()` - _Requirements: 1.6, 18.5_ - - [ ] 33.2 Replace `redis=None` with `aioredis.from_url()` + - [x] 33.2 Replace `redis=None` with `aioredis.from_url()` - In the lifespan block, create Redis client: `redis_client = aioredis.from_url(config.redis.url)` - Pass `redis_client` to `TradingEngine(pool=pool, redis=redis_client, ...)` - In lifespan exit: `await redis_client.close()` - _Requirements: 1.5_ - - [ ] 33.3 Add proper error handling and cleanup in lifespan + - [x] 33.3 Add proper error handling and cleanup in lifespan - Wrap pool/redis creation in try/except — log critical error and raise if connections fail - Ensure `engine.stop()`, `pool.close()`, and `redis_client.close()` are called in the `finally` block of lifespan exit - Log connection details (host, port, database) at startup for debugging (not passwords) - _Requirements: 1.6, 18.5_ -- [ ] 34. Checkpoint — Ensure all live wiring compiles and existing tests still pass +- [x] 34. Checkpoint — Ensure all live wiring compiles and existing tests still pass - Ensure all tests pass, ask the user if questions arise. -- [ ] 35. Enable paper trading configuration - - [ ] 35.1 Update `trading_engine_config` defaults for paper trading +- [x] 35. Enable paper trading configuration + - [x] 35.1 Update `trading_engine_config` defaults for paper trading - Add a SQL migration or seed script that updates the default `trading_engine_config` row: `enabled=true`, `risk_tier='conservative'`, `absolute_position_cap=25.0` (conservative for initial paper trading) - Set `polling_interval_seconds=60`, `max_open_positions=5` (conservative start) - _Requirements: 16.1, 5.1_ - - [ ] 35.2 Add `TRADING_ENABLED=true` to Helm values for trading-engine deployment + - [x] 35.2 Add `TRADING_ENABLED=true` to Helm values for trading-engine deployment - Update `infra/helm/stonks-oracle/values.yaml` to set `TRADING_ENABLED: "true"` in the trading-engine environment variables - Ensure `TRADING_RISK_TIER: "conservative"` and `TRADING_ABSOLUTE_POSITION_CAP: "25.0"` are set - _Requirements: 16.1_ - - [ ] 35.3 Verify trading-engine pod starts and readiness probe passes + - [x] 35.3 Verify trading-engine pod starts and readiness probe passes - After deployment, confirm the trading-engine pod reaches `Running` state - Confirm `GET /ready` returns `{"ready": true}` once portfolio state is loaded - Confirm `GET /health` returns `{"status": "ok"}` - Confirm `GET /api/trading/status` returns the expected configuration (enabled=true, risk_tier=conservative) - _Requirements: 1.7, 16.2_ -- [ ] 36. Write integration tests for live wiring - - [ ]* 36.1 Test decision loop with mocked PostgreSQL and Redis +- [x] 36. Write integration tests for live wiring + - [x]* 36.1 Test decision loop with mocked PostgreSQL and Redis - Create `tests/test_trading_integration.py` - Mock `asyncpg.Pool` to return canned recommendation rows and portfolio state - Mock Redis client for deduplication checks and broker queue pushes @@ -915,14 +915,14 @@ Phase 1 (Tasks 1–26) implemented all pure computation modules, property tests, - Verify skip decisions are persisted with correct reasons - _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5_ - - [ ]* 36.2 Test stop-loss monitor with mocked price API + - [x]* 36.2 Test stop-loss monitor with mocked price API - Mock `_fetch_current_prices()` to return prices that cross stop-loss and take-profit levels - Verify sell orders are generated and pushed to broker queue for triggered positions - Verify no orders generated when prices are between stop and take-profit - Test safety sell: mock price fetch returning empty for > 15 minutes, verify position closed - _Requirements: 4.4, 4.5, 4.8_ - - [ ]* 36.3 Test notification dispatch with mocked SNS and Gmail + - [x]* 36.3 Test notification dispatch with mocked SNS and Gmail - Mock `boto3.client("sns")` and Gmail API service - Verify SNS publish called with correct topic ARN and message for SMS-enabled events - Verify Gmail send called with correct sender/recipient for email-enabled events @@ -930,7 +930,7 @@ Phase 1 (Tasks 1–26) implemented all pure computation modules, property tests, - Verify retry: mock first delivery failure, verify retry with backoff, verify final success - _Requirements: 19.1, 19.5, 19.6, 19.7, 19.11_ - - [ ]* 36.4 Test backtest replay end-to-end + - [x]* 36.4 Test backtest replay end-to-end - Mock PostgreSQL pool to return historical recommendations and price data - Run `BacktestReplay.run()` with a small date range and $500 initial capital - Verify backtest result contains expected metrics (total_return, sharpe_ratio, max_drawdown, win_rate, trade_count) @@ -938,13 +938,13 @@ Phase 1 (Tasks 1–26) implemented all pure computation modules, property tests, - Verify trades are persisted to `backtest_trades` (mocked INSERT calls) - _Requirements: 15.1, 15.2, 15.3, 15.4_ - - [ ]* 36.5 Test lifespan creates real pool and redis connections + - [x]* 36.5 Test lifespan creates real pool and redis connections - Use `httpx.AsyncClient` with the FastAPI `app` and mock `asyncpg.create_pool` / `aioredis.from_url` - Verify pool and redis are created during startup and closed during shutdown - Verify engine receives non-None pool and redis - _Requirements: 1.6, 18.5_ -- [ ] 37. Final checkpoint — Verify paper trading is operational +- [x] 37. Final checkpoint — Verify paper trading is operational - Ensure all tests pass, ask the user if questions arise. - Trading engine pod is running and ready - Decision loop is polling recommendations from PostgreSQL