feat: signal math upgrade — probabilistic, regime-aware scoring pipeline
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled

Implement full probabilistic signal processing pipeline gated behind
probabilistic_scoring_enabled feature flag in risk_configs:

- Bayesian log-likelihood accumulator with Beta posterior and entropy
- Regime detector (trend-following, panic, mean-reversion, uncertainty)
- Source accuracy tracker with per-source historical prediction accuracy
- Sigmoid confidence gate replacing binary gate
- Information gain surprise weighting for rare events
- Adaptive recency decay with event-specific half-lives
- Regime multiplier replacing market context multiplier
- Weighted disagreement entropy for contradiction detection
- Multiplicative macro exposure with conditional integration
- Graph-distance attenuated competitive signal propagation
- Exponentially weighted momentum with volatility scaling
- Expected value recommendation gate

All changes backward-compatible: flag=false preserves exact current behavior.
New outputs stored in existing JSONB columns (no schema changes except
source_accuracy table via migration 034).

Tests: 26 property-based tests (14 correctness properties), 99 unit tests,
1789 total tests passing with zero regressions.
This commit is contained in:
Celes Renata
2026-04-29 11:41:48 +00:00
parent 8c3c1aab43
commit 4e010bc048
24 changed files with 6058 additions and 60 deletions
+237
View File
@@ -0,0 +1,237 @@
"""Unit tests for regime detector (services/aggregation/regime.py).
Tests specific (R, V_r) → regime classification, threshold adjustments
per regime, and insufficient data fallback to uncertainty.
Requirements: 7.1, 7.2, 7.3, 7.4, 7.5, 7.6, 7.7, 7.9
"""
from __future__ import annotations
import pytest
from services.aggregation.regime import (
MarketRegime,
classify_regime,
compute_ema,
)
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _make_uptrend_prices(n: int = 120) -> list[float]:
"""Generate prices with EMA_20 > EMA_100 (uptrend, R=+1)."""
# Start low, end high — recent prices much higher than old ones
return [100.0 + i * 0.5 for i in range(n)]
def _make_downtrend_prices(n: int = 120) -> list[float]:
"""Generate prices with EMA_20 < EMA_100 (downtrend, R=-1)."""
# Start high, end low — recent prices much lower than old ones
return [200.0 - i * 0.5 for i in range(n)]
def _make_flat_prices(n: int = 120) -> list[float]:
"""Generate flat prices where EMA_20 ≈ EMA_100 (R=0)."""
return [100.0] * n
def _make_low_vol_returns(n: int = 120) -> list[float]:
"""Generate returns with σ_20 / σ_100 < 1.0 (low recent volatility)."""
# First 100 returns have higher variance, last 20 have lower variance
base = [0.02 * ((-1) ** i) for i in range(n - 20)]
recent = [0.005 * ((-1) ** i) for i in range(20)]
return base + recent
def _make_high_vol_returns(n: int = 120) -> list[float]:
"""Generate returns with σ_20 / σ_100 > 1.5 (panic volatility)."""
# First 100 returns have low variance, last 20 have very high variance
base = [0.005 * ((-1) ** i) for i in range(n - 20)]
recent = [0.08 * ((-1) ** i) for i in range(20)]
return base + recent
def _make_moderate_vol_returns(n: int = 120) -> list[float]:
"""Generate returns with V_r between 1.0 and 1.2."""
# Slightly higher recent volatility
base = [0.01 * ((-1) ** i) for i in range(n - 20)]
recent = [0.012 * ((-1) ** i) for i in range(20)]
return base + recent
# ---------------------------------------------------------------------------
# compute_ema
# ---------------------------------------------------------------------------
class TestComputeEma:
"""Test EMA computation."""
def test_single_value(self):
assert compute_ema([100.0], 1) == pytest.approx(100.0)
def test_constant_values(self):
"""EMA of constant values equals that constant."""
assert compute_ema([50.0] * 20, 20) == pytest.approx(50.0)
def test_empty_raises(self):
with pytest.raises(ValueError):
compute_ema([], 10)
def test_zero_period_raises(self):
with pytest.raises(ValueError):
compute_ema([1.0, 2.0], 0)
# ---------------------------------------------------------------------------
# Regime classification: specific (R, V_r) → expected regime
# ---------------------------------------------------------------------------
class TestRegimeClassification:
"""Test specific (R, V_r) → expected regime classification (Req 7.3)."""
def test_trend_following_uptrend(self):
"""R=+1, V_r < 1.2 → trend_following."""
prices = _make_uptrend_prices()
returns = _make_moderate_vol_returns()
result = classify_regime(prices, returns)
assert result.regime == MarketRegime.TREND_FOLLOWING
assert result.trend_indicator == 1.0
def test_trend_following_downtrend(self):
"""R=-1, V_r < 1.2 → trend_following."""
prices = _make_downtrend_prices()
returns = _make_moderate_vol_returns()
result = classify_regime(prices, returns)
assert result.regime == MarketRegime.TREND_FOLLOWING
assert result.trend_indicator == -1.0
def test_panic_regime(self):
"""V_r > 1.5 → panic (regardless of R)."""
prices = _make_uptrend_prices()
returns = _make_high_vol_returns()
result = classify_regime(prices, returns)
assert result.regime == MarketRegime.PANIC
def test_mean_reversion_regime(self):
"""R=0, V_r < 1.0 → mean_reversion."""
prices = _make_flat_prices()
returns = _make_low_vol_returns()
result = classify_regime(prices, returns)
assert result.regime == MarketRegime.MEAN_REVERSION
def test_uncertainty_regime(self):
"""R=0, V_r between 1.0 and 1.5 → uncertainty."""
prices = _make_flat_prices()
# Returns with V_r between 1.0 and 1.5 but not < 1.0
# Use moderate vol that gives V_r ≈ 1.1 with flat prices
returns = _make_moderate_vol_returns()
result = classify_regime(prices, returns)
# With flat prices R=0, and moderate vol V_r ≈ 1.1 (> 1.0)
# This falls into uncertainty (R=0 AND V_r >= 1.0)
assert result.regime == MarketRegime.UNCERTAINTY
# ---------------------------------------------------------------------------
# Threshold adjustments per regime (Req 7.4, 7.5, 7.6, 7.7)
# ---------------------------------------------------------------------------
class TestRegimeThresholds:
"""Test threshold adjustments per regime."""
def test_panic_threshold(self):
"""Panic regime → threshold ±0.10 (Req 7.4)."""
prices = _make_uptrend_prices()
returns = _make_high_vol_returns()
result = classify_regime(prices, returns)
assert result.regime == MarketRegime.PANIC
assert result.bullish_threshold == pytest.approx(0.10)
assert result.bearish_threshold == pytest.approx(-0.10)
def test_mean_reversion_threshold(self):
"""Mean-reversion regime → threshold ±0.20 (Req 7.5)."""
prices = _make_flat_prices()
returns = _make_low_vol_returns()
result = classify_regime(prices, returns)
assert result.regime == MarketRegime.MEAN_REVERSION
assert result.bullish_threshold == pytest.approx(0.20)
assert result.bearish_threshold == pytest.approx(-0.20)
def test_trend_following_threshold(self):
"""Trend-following regime → threshold ±0.15 (Req 7.6)."""
prices = _make_uptrend_prices()
returns = _make_moderate_vol_returns()
result = classify_regime(prices, returns)
assert result.regime == MarketRegime.TREND_FOLLOWING
assert result.bullish_threshold == pytest.approx(0.15)
assert result.bearish_threshold == pytest.approx(-0.15)
def test_uncertainty_contradiction_multiplier(self):
"""Uncertainty regime → contradiction multiplier 0.6 (Req 7.7)."""
prices = _make_flat_prices()
returns = _make_moderate_vol_returns()
result = classify_regime(prices, returns)
assert result.regime == MarketRegime.UNCERTAINTY
assert result.contradiction_penalty_multiplier == pytest.approx(0.6)
def test_non_uncertainty_contradiction_multiplier(self):
"""Non-uncertainty regimes → contradiction multiplier 0.4."""
prices = _make_uptrend_prices()
returns = _make_moderate_vol_returns()
result = classify_regime(prices, returns)
assert result.regime == MarketRegime.TREND_FOLLOWING
assert result.contradiction_penalty_multiplier == pytest.approx(0.4)
# ---------------------------------------------------------------------------
# Insufficient data fallback to uncertainty (Req 7.9)
# ---------------------------------------------------------------------------
class TestInsufficientDataFallback:
"""Test fallback to uncertainty when data is insufficient."""
def test_too_few_prices(self):
"""Fewer than 100 closing prices → uncertainty."""
prices = [100.0] * 50 # only 50 days
returns = [0.01] * 100
result = classify_regime(prices, returns)
assert result.regime == MarketRegime.UNCERTAINTY
def test_too_few_returns(self):
"""Fewer than 100 returns → uncertainty."""
prices = [100.0] * 120
returns = [0.01] * 50 # only 50 returns
result = classify_regime(prices, returns)
assert result.regime == MarketRegime.UNCERTAINTY
def test_empty_prices(self):
"""Empty price list → uncertainty."""
result = classify_regime([], [0.01] * 100)
assert result.regime == MarketRegime.UNCERTAINTY
def test_empty_returns(self):
"""Empty return list → uncertainty."""
result = classify_regime([100.0] * 120, [])
assert result.regime == MarketRegime.UNCERTAINTY
def test_zero_sigma_returns_uncertainty(self):
"""All identical returns (σ=0) → uncertainty."""
prices = _make_uptrend_prices()
returns = [0.0] * 120 # zero standard deviation
result = classify_regime(prices, returns)
assert result.regime == MarketRegime.UNCERTAINTY
def test_default_uncertainty_values(self):
"""Default uncertainty has standard threshold values."""
result = classify_regime([], [])
assert result.regime == MarketRegime.UNCERTAINTY
assert result.bullish_threshold == pytest.approx(0.15)
assert result.bearish_threshold == pytest.approx(-0.15)
assert result.contradiction_penalty_multiplier == pytest.approx(0.6)
assert result.trend_indicator == 0.0
assert result.volatility_ratio == 1.0