fix: pipeline health — stuck docs, price fallback, sentiment normalization, signal-engine scale, quality gate

- Scheduler: lower stale threshold 240→30 min, batch limit 100→500, TTL 14400→3600
- Prediction snapshot: add 24h market_snapshots time-window fallback
- Aggregation: add normalize_impact_scores() z-score normalization
- Helm: signal-engine replicas → 0 (idle when dual pipeline disabled)
- Quality gate: max_snapshot_age_hours 24→48
- Add backfill script for NULL price_at_prediction snapshots
- Add PBT bug condition and preservation tests (14 tests)
This commit is contained in:
Celes Renata
2026-07-10 20:16:01 +00:00
parent a4f51c00e1
commit ca712ad4a0
15 changed files with 1815 additions and 9 deletions
+8
View File
@@ -51,6 +51,7 @@ from services.aggregation.scoring import (
ScoringConfig,
WeightedSignal,
compute_signal_weight,
normalize_impact_scores,
sentiment_to_numeric,
weighted_sentiment_average,
)
@@ -2312,6 +2313,13 @@ async def aggregate_company_window(
# 2. Fetch market context
market_ctx = await fetch_market_context(pool, ticker, window, reference_time)
# 2a. Normalize impact scores using 7-day z-score per ticker
if impacts:
raw_scores = [imp.impact_score for imp in impacts]
normalized_scores = await normalize_impact_scores(pool, ticker, raw_scores)
for imp, norm_score in zip(impacts, normalized_scores):
imp.impact_score = norm_score
# 3. Build weighted signals — pass source accuracy and market data
# when in probabilistic mode (Req 4.14.3, 6.16.5)
signals = build_weighted_signals(