# Bugfix Requirements Document

## Introduction

Multiple operational bugs discovered in the stonks-beta namespace prevent the validation/calibration feedback loop from functioning and degrade ingestion throughput. The core issue is that the outcome evaluation → metrics computation → quality gate pipeline is completely disconnected from the production scheduler, making the platform unable to self-calibrate or validate predictions. Additionally, Polygon API rate limiting causes ~40% request failures per cycle, a broken config query prevents the v3 engine from being toggled, several periodic snapshot tasks are missing from the scheduler, the lake-publisher deployment is idle/redundant, and order rejection reasons are lost.

## Bug Analysis

### Current Behavior (Defect)

1.1 WHEN the scheduler enqueues ingestion jobs for all 50 tickers' news_api and market_api sources simultaneously THEN the system exhausts the Polygon free-tier rate limit (5 req/min) resulting in ~40% of sources receiving HTTP 429 Too Many Requests every cycle

1.2 WHEN the aggregation worker reads the v3_engine_enabled flag via `_V3_ENGINE_FLAG_QUERY` THEN the system queries non-existent columns `key` and `value` on the `risk_configs` table (actual schema: `name` varchar, `config` JSONB) causing a PostgreSQL error every aggregation cycle

1.3 WHEN a scheduler cycle completes THEN the system never calls `evaluate_matured_predictions()` because it is not wired into the scheduler's main loop — only imported in `backtest_replay.py`

1.4 WHEN a scheduler cycle completes THEN the system never calls `compute_and_store_metric_snapshots()` because it is not wired into the scheduler's main loop — only called from backtest replay

1.5 WHEN the model quality gate evaluates trading eligibility THEN the system always fails with "no model metric snapshot available — defaulting to paper-only" because `model_metric_snapshots` table is permanently empty (consequence of bug 1.4)

1.6 WHEN the trading engine runs daily THEN the system never captures portfolio state snapshots to the `portfolio_snapshots` table because no periodic scheduler task invokes this capture

1.7 WHEN the trading engine runs daily THEN the system never captures risk state snapshots to the `daily_risk_snapshots` table because no periodic scheduler task invokes this capture

1.8 WHEN a prediction snapshot is created while Polygon rate-limiting has prevented the market data fetch THEN the system stores NULL in `price_at_prediction` (affecting 21% of snapshots), degrading downstream outcome evaluation accuracy

1.9 WHEN the standalone `lake-publisher` deployment polls `stonks:beta:queue:lake_publish` THEN the queue is always empty (0 items) because all lake publishing happens inline in broker-adapter and recommendation services — the deployment consumes zero work and wastes resources

1.10 WHEN Alpaca returns HTTP 401 for an order submission THEN the system sets order status to "rejected" but leaves the `rejection_reason` column NULL, capturing the error message only in the `decision_trace` JSONB field

### Expected Behavior (Correct)

2.1 WHEN the scheduler enqueues ingestion jobs for Polygon-backed sources (news_api, market_api) THEN the system SHALL pace/stagger requests across the polling interval to stay within the Polygon rate limit, achieving near-zero 429 responses per cycle

2.2 WHEN the aggregation worker reads the v3_engine_enabled flag THEN the system SHALL query `SELECT config FROM risk_configs WHERE name = 'v3_engine_enabled'` and parse the JSONB value to determine the boolean toggle state

2.3 WHEN a scheduler cycle completes and sufficient time has elapsed since the last evaluation THEN the system SHALL call `evaluate_matured_predictions()` to evaluate prediction snapshots whose horizon has elapsed, populating the `prediction_outcomes` table

2.4 WHEN a scheduler cycle completes and sufficient time has elapsed since the last computation THEN the system SHALL call `compute_and_store_metric_snapshots()` to compute aggregate model metrics across all lookback/horizon combinations, populating `model_metric_snapshots`

2.5 WHEN the model quality gate evaluates trading eligibility THEN the system SHALL have recent metric snapshots available and evaluate thresholds against actual model performance data

2.6 WHEN market hours close (or on a daily schedule) THEN the system SHALL capture and persist the current portfolio state to `portfolio_snapshots` including value, returns, positions, and risk metrics

2.7 WHEN market hours close (or on a daily schedule) THEN the system SHALL capture and persist the current risk state to `daily_risk_snapshots` including portfolio value, daily P&L, trade count, and sector positions

2.8 WHEN a prediction snapshot is created and market price is unavailable due to rate limiting THEN the system SHALL retry the price fetch or defer the snapshot until price data is available, reducing NULL `price_at_prediction` occurrences to near zero

2.9 WHEN the lake-publisher deployment architecture is reviewed THEN the system SHALL either route lake publish jobs through the Redis queue to the standalone deployment, or remove the redundant deployment — eliminating the idle pod

2.10 WHEN Alpaca returns an HTTP error (401, 403, or any rejection) for an order submission THEN the system SHALL populate the `rejection_reason` column with the HTTP error message/status in addition to recording it in `decision_trace`

### Unchanged Behavior (Regression Prevention)

3.1 WHEN sources with valid rate-limit headroom are enqueued THEN the system SHALL CONTINUE TO enqueue and process them without artificial delay

3.2 WHEN risk_configs is queried for other configuration keys (e.g., `model_quality_gate_config`, `macro_enabled`) THEN the system SHALL CONTINUE TO read them correctly using the existing `name`/`config` column pattern

3.3 WHEN the backtest replay module calls `evaluate_matured_predictions()` and `compute_and_store_metric_snapshots()` THEN the system SHALL CONTINUE TO execute them as part of backtest validation

3.4 WHEN prediction snapshots are created with available market prices THEN the system SHALL CONTINUE TO store the correct `price_at_prediction` value immediately

3.5 WHEN the existing inline lake publishing in broker-adapter and recommendation services writes facts THEN the system SHALL CONTINUE TO produce correct Parquet partitions in MinIO

3.6 WHEN orders succeed (HTTP 200 from Alpaca) THEN the system SHALL CONTINUE TO process them normally without modifying the `rejection_reason` column

3.7 WHEN the scheduler runs ingestion, extraction, aggregation, recommendation, and trading tasks THEN the system SHALL CONTINUE TO execute them on the existing cadence without disruption

3.8 WHEN the trading engine makes decisions and submits orders THEN the system SHALL CONTINUE TO record full decision context in `decision_trace` JSONB as before

3.9 WHEN the model quality gate passes (once metrics are populated) THEN the system SHALL CONTINUE TO allow promotion to live trading mode per existing threshold logic

3.10 WHEN the reporting collector fetches portfolio_snapshots and daily_risk_snapshots for report generation THEN the system SHALL CONTINUE TO query and render them using the existing schema