fix: pipeline health — stuck docs, price fallback, sentiment normalization, signal-engine scale, quality gate

- Scheduler: lower stale threshold 240→30 min, batch limit 100→500, TTL 14400→3600
- Prediction snapshot: add 24h market_snapshots time-window fallback
- Aggregation: add normalize_impact_scores() z-score normalization
- Helm: signal-engine replicas → 0 (idle when dual pipeline disabled)
- Quality gate: max_snapshot_age_hours 24→48
- Add backfill script for NULL price_at_prediction snapshots
- Add PBT bug condition and preservation tests (14 tests)
This commit is contained in:
Celes Renata
2026-07-10 20:16:01 +00:00
parent a4f51c00e1
commit ca712ad4a0
15 changed files with 1815 additions and 9 deletions
+23 -1
View File
@@ -323,7 +323,29 @@ async def create_prediction_snapshot(
"Used positions fallback price for %s: %s", ticker, ticker_price
)
else:
logger.warning("No market price available for %s at snapshot time", ticker)
# Extended fallback: query market_snapshots within 24h time window
extended_row = await pool.fetchrow(
"""SELECT (data->>'c')::float AS close
FROM market_snapshots
WHERE ticker = $1
AND snapshot_type = 'bar'
AND data->>'c' IS NOT NULL
AND captured_at >= NOW() - INTERVAL '24 hours'
ORDER BY captured_at DESC
LIMIT 1""",
ticker,
)
if extended_row:
ticker_price = float(extended_row["close"])
logger.info(
"Used extended 24h market_snapshots fallback price for %s: %s",
ticker,
ticker_price,
)
else:
logger.warning(
"No market price available for %s at snapshot time", ticker
)
spy_price = await fetch_latest_close_price(pool, "SPY")
if spy_price is None: