# Bugfix Requirements Document

## Introduction

Five operational bugs in the stonks-beta deployment degrade pipeline health: 1,809 documents stuck in `parsed` status due to recovery batch limits, 26.5% of prediction snapshots missing prices due to incomplete fallback chains, 64% sell bias from uncalibrated NuExtract3 sentiment outputs, idle signal-engine consuming resources while doing nothing, and a quality gate stuck in paper-only mode due to an overly strict staleness threshold interacting with the NULL price problem.

## Bug Analysis

### Current Behavior (Defect)

1.1 WHEN the `recover_stale_documents` task runs with 1,809+ documents stuck in `parsed` status THEN the system only processes 100 per cycle (every ~5 minutes), requiring 90+ cycles (~7.5 hours) to clear the backlog while new documents may continue accumulating

1.2 WHEN a prediction snapshot is created for a ticker without an open position AND without recent market_snapshots data THEN the system stores NULL in `price_at_prediction` because the fallback chain stops at the positions table (26.5% of snapshots affected — 33,324 of 125,590)

1.3 WHEN the outcome evaluator encounters a prediction snapshot with NULL `price_at_prediction` THEN the system skips the snapshot entirely, creating a validation blind spot where 26.5% of predictions are never evaluated

1.4 WHEN the aggregation pipeline processes NuExtract3 extraction outputs THEN the system passes raw `impact_score` and `sentiment` values directly into signal weighting without any distribution normalization, resulting in systematic negative bias producing 64% sell / 23% watch / 12% buy recommendations

1.5 WHEN the signal-engine pod starts with `dual_pipeline_enabled=False` THEN the system enters an infinite sleep loop consuming CPU (100m request / 500m limit) and memory (128Mi request / 256Mi limit) while producing zero signal evaluations

1.6 WHEN the quality gate checks `model_metric_snapshots` freshness with a 24-hour staleness threshold AND the validation cycle skips all predictions due to NULL prices (Bug 1.2/1.3) THEN the system permanently defaults to paper-only mode because no fresh metric snapshots are ever generated

### Expected Behavior (Correct)

2.1 WHEN the scheduler detects more than 100 documents stuck in `parsed` status older than the threshold THEN the system SHALL increase the batch limit for recovery processing (up to 500 per cycle) and provide a one-time management command to bulk-recover the existing backlog without waiting for periodic sweeps

2.2 WHEN a prediction snapshot is created and no price is available from market_snapshots (exact time) or positions table THEN the system SHALL query `market_snapshots` with a wider time window (last 24 hours of bar data for the ticker) as an additional fallback before accepting NULL

2.3 WHEN backfilling existing prediction snapshots with NULL `price_at_prediction` THEN the system SHALL use the extended fallback chain (market_snapshots within 24h of `generated_at`, then positions) to populate prices retroactively via a migration script

2.4 WHEN the aggregation pipeline computes signal weights from impact records THEN the system SHALL apply z-score normalization to `impact_score` values relative to the rolling 7-day distribution of impact records for the same ticker, preventing systematic model bias from dominating the directional signal

2.5 WHEN the signal-engine deployment is not ready for production use (`dual_pipeline_enabled=False`) THEN the system SHALL be scaled to 0 replicas in the Helm values files (beta, paper, live) to eliminate wasted CPU, memory, and any GPU time-slice allocations

2.6 WHEN the quality gate evaluates metric snapshot freshness during the bootstrapping period THEN the system SHALL use a 48-hour staleness threshold (instead of 24h) to tolerate gaps while the validation cycle ramps up after Bug 1.2/1.3 are fixed

### Unchanged Behavior (Regression Prevention)

3.1 WHEN documents enter `parsed` status and are processed within the normal threshold window (< 240 minutes) THEN the system SHALL CONTINUE TO leave them for the extraction queue consumer without interference from the recovery task

3.2 WHEN a prediction snapshot is created and market_snapshots contains a recent bar for the ticker THEN the system SHALL CONTINUE TO use the primary `market_snapshots` close price without invoking any fallback

3.3 WHEN the aggregation pipeline processes tickers with balanced sentiment distributions (equal bullish/bearish evidence) THEN the system SHALL CONTINUE TO produce neutral/mixed recommendations without artificial skew from the normalization step

3.4 WHEN the signal-engine is re-enabled in the future (dual_pipeline_enabled=True with replicas > 0) THEN the system SHALL CONTINUE TO function correctly with its existing queue-based architecture and configuration loading

3.5 WHEN the quality gate evaluates a metric snapshot that is less than 48 hours old and meets all threshold criteria THEN the system SHALL CONTINUE TO promote recommendations to live_eligible mode per existing threshold logic

3.6 WHEN the outcome evaluator processes prediction snapshots with valid (non-NULL) prices THEN the system SHALL CONTINUE TO evaluate them normally and produce prediction_outcomes records

3.7 WHEN the `retry_failed_extractions` task handles documents in `extraction_failed` status THEN the system SHALL CONTINUE TO process them on the existing cadence and logic without interference from the parsed-document recovery changes