feat: signal math upgrade — probabilistic, regime-aware scoring pipeline
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
Implement full probabilistic signal processing pipeline gated behind probabilistic_scoring_enabled feature flag in risk_configs: - Bayesian log-likelihood accumulator with Beta posterior and entropy - Regime detector (trend-following, panic, mean-reversion, uncertainty) - Source accuracy tracker with per-source historical prediction accuracy - Sigmoid confidence gate replacing binary gate - Information gain surprise weighting for rare events - Adaptive recency decay with event-specific half-lives - Regime multiplier replacing market context multiplier - Weighted disagreement entropy for contradiction detection - Multiplicative macro exposure with conditional integration - Graph-distance attenuated competitive signal propagation - Exponentially weighted momentum with volatility scaling - Expected value recommendation gate All changes backward-compatible: flag=false preserves exact current behavior. New outputs stored in existing JSONB columns (no schema changes except source_accuracy table via migration 034). Tests: 26 property-based tests (14 correctness properties), 99 unit tests, 1789 total tests passing with zero regressions.
This commit is contained in:
@@ -0,0 +1,237 @@
|
||||
"""Unit tests for regime detector (services/aggregation/regime.py).
|
||||
|
||||
Tests specific (R, V_r) → regime classification, threshold adjustments
|
||||
per regime, and insufficient data fallback to uncertainty.
|
||||
|
||||
Requirements: 7.1, 7.2, 7.3, 7.4, 7.5, 7.6, 7.7, 7.9
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from services.aggregation.regime import (
|
||||
MarketRegime,
|
||||
classify_regime,
|
||||
compute_ema,
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _make_uptrend_prices(n: int = 120) -> list[float]:
|
||||
"""Generate prices with EMA_20 > EMA_100 (uptrend, R=+1)."""
|
||||
# Start low, end high — recent prices much higher than old ones
|
||||
return [100.0 + i * 0.5 for i in range(n)]
|
||||
|
||||
|
||||
def _make_downtrend_prices(n: int = 120) -> list[float]:
|
||||
"""Generate prices with EMA_20 < EMA_100 (downtrend, R=-1)."""
|
||||
# Start high, end low — recent prices much lower than old ones
|
||||
return [200.0 - i * 0.5 for i in range(n)]
|
||||
|
||||
|
||||
def _make_flat_prices(n: int = 120) -> list[float]:
|
||||
"""Generate flat prices where EMA_20 ≈ EMA_100 (R=0)."""
|
||||
return [100.0] * n
|
||||
|
||||
|
||||
def _make_low_vol_returns(n: int = 120) -> list[float]:
|
||||
"""Generate returns with σ_20 / σ_100 < 1.0 (low recent volatility)."""
|
||||
# First 100 returns have higher variance, last 20 have lower variance
|
||||
base = [0.02 * ((-1) ** i) for i in range(n - 20)]
|
||||
recent = [0.005 * ((-1) ** i) for i in range(20)]
|
||||
return base + recent
|
||||
|
||||
|
||||
def _make_high_vol_returns(n: int = 120) -> list[float]:
|
||||
"""Generate returns with σ_20 / σ_100 > 1.5 (panic volatility)."""
|
||||
# First 100 returns have low variance, last 20 have very high variance
|
||||
base = [0.005 * ((-1) ** i) for i in range(n - 20)]
|
||||
recent = [0.08 * ((-1) ** i) for i in range(20)]
|
||||
return base + recent
|
||||
|
||||
|
||||
def _make_moderate_vol_returns(n: int = 120) -> list[float]:
|
||||
"""Generate returns with V_r between 1.0 and 1.2."""
|
||||
# Slightly higher recent volatility
|
||||
base = [0.01 * ((-1) ** i) for i in range(n - 20)]
|
||||
recent = [0.012 * ((-1) ** i) for i in range(20)]
|
||||
return base + recent
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# compute_ema
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestComputeEma:
|
||||
"""Test EMA computation."""
|
||||
|
||||
def test_single_value(self):
|
||||
assert compute_ema([100.0], 1) == pytest.approx(100.0)
|
||||
|
||||
def test_constant_values(self):
|
||||
"""EMA of constant values equals that constant."""
|
||||
assert compute_ema([50.0] * 20, 20) == pytest.approx(50.0)
|
||||
|
||||
def test_empty_raises(self):
|
||||
with pytest.raises(ValueError):
|
||||
compute_ema([], 10)
|
||||
|
||||
def test_zero_period_raises(self):
|
||||
with pytest.raises(ValueError):
|
||||
compute_ema([1.0, 2.0], 0)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Regime classification: specific (R, V_r) → expected regime
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestRegimeClassification:
|
||||
"""Test specific (R, V_r) → expected regime classification (Req 7.3)."""
|
||||
|
||||
def test_trend_following_uptrend(self):
|
||||
"""R=+1, V_r < 1.2 → trend_following."""
|
||||
prices = _make_uptrend_prices()
|
||||
returns = _make_moderate_vol_returns()
|
||||
result = classify_regime(prices, returns)
|
||||
assert result.regime == MarketRegime.TREND_FOLLOWING
|
||||
assert result.trend_indicator == 1.0
|
||||
|
||||
def test_trend_following_downtrend(self):
|
||||
"""R=-1, V_r < 1.2 → trend_following."""
|
||||
prices = _make_downtrend_prices()
|
||||
returns = _make_moderate_vol_returns()
|
||||
result = classify_regime(prices, returns)
|
||||
assert result.regime == MarketRegime.TREND_FOLLOWING
|
||||
assert result.trend_indicator == -1.0
|
||||
|
||||
def test_panic_regime(self):
|
||||
"""V_r > 1.5 → panic (regardless of R)."""
|
||||
prices = _make_uptrend_prices()
|
||||
returns = _make_high_vol_returns()
|
||||
result = classify_regime(prices, returns)
|
||||
assert result.regime == MarketRegime.PANIC
|
||||
|
||||
def test_mean_reversion_regime(self):
|
||||
"""R=0, V_r < 1.0 → mean_reversion."""
|
||||
prices = _make_flat_prices()
|
||||
returns = _make_low_vol_returns()
|
||||
result = classify_regime(prices, returns)
|
||||
assert result.regime == MarketRegime.MEAN_REVERSION
|
||||
|
||||
def test_uncertainty_regime(self):
|
||||
"""R=0, V_r between 1.0 and 1.5 → uncertainty."""
|
||||
prices = _make_flat_prices()
|
||||
# Returns with V_r between 1.0 and 1.5 but not < 1.0
|
||||
# Use moderate vol that gives V_r ≈ 1.1 with flat prices
|
||||
returns = _make_moderate_vol_returns()
|
||||
result = classify_regime(prices, returns)
|
||||
# With flat prices R=0, and moderate vol V_r ≈ 1.1 (> 1.0)
|
||||
# This falls into uncertainty (R=0 AND V_r >= 1.0)
|
||||
assert result.regime == MarketRegime.UNCERTAINTY
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Threshold adjustments per regime (Req 7.4, 7.5, 7.6, 7.7)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestRegimeThresholds:
|
||||
"""Test threshold adjustments per regime."""
|
||||
|
||||
def test_panic_threshold(self):
|
||||
"""Panic regime → threshold ±0.10 (Req 7.4)."""
|
||||
prices = _make_uptrend_prices()
|
||||
returns = _make_high_vol_returns()
|
||||
result = classify_regime(prices, returns)
|
||||
assert result.regime == MarketRegime.PANIC
|
||||
assert result.bullish_threshold == pytest.approx(0.10)
|
||||
assert result.bearish_threshold == pytest.approx(-0.10)
|
||||
|
||||
def test_mean_reversion_threshold(self):
|
||||
"""Mean-reversion regime → threshold ±0.20 (Req 7.5)."""
|
||||
prices = _make_flat_prices()
|
||||
returns = _make_low_vol_returns()
|
||||
result = classify_regime(prices, returns)
|
||||
assert result.regime == MarketRegime.MEAN_REVERSION
|
||||
assert result.bullish_threshold == pytest.approx(0.20)
|
||||
assert result.bearish_threshold == pytest.approx(-0.20)
|
||||
|
||||
def test_trend_following_threshold(self):
|
||||
"""Trend-following regime → threshold ±0.15 (Req 7.6)."""
|
||||
prices = _make_uptrend_prices()
|
||||
returns = _make_moderate_vol_returns()
|
||||
result = classify_regime(prices, returns)
|
||||
assert result.regime == MarketRegime.TREND_FOLLOWING
|
||||
assert result.bullish_threshold == pytest.approx(0.15)
|
||||
assert result.bearish_threshold == pytest.approx(-0.15)
|
||||
|
||||
def test_uncertainty_contradiction_multiplier(self):
|
||||
"""Uncertainty regime → contradiction multiplier 0.6 (Req 7.7)."""
|
||||
prices = _make_flat_prices()
|
||||
returns = _make_moderate_vol_returns()
|
||||
result = classify_regime(prices, returns)
|
||||
assert result.regime == MarketRegime.UNCERTAINTY
|
||||
assert result.contradiction_penalty_multiplier == pytest.approx(0.6)
|
||||
|
||||
def test_non_uncertainty_contradiction_multiplier(self):
|
||||
"""Non-uncertainty regimes → contradiction multiplier 0.4."""
|
||||
prices = _make_uptrend_prices()
|
||||
returns = _make_moderate_vol_returns()
|
||||
result = classify_regime(prices, returns)
|
||||
assert result.regime == MarketRegime.TREND_FOLLOWING
|
||||
assert result.contradiction_penalty_multiplier == pytest.approx(0.4)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Insufficient data fallback to uncertainty (Req 7.9)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestInsufficientDataFallback:
|
||||
"""Test fallback to uncertainty when data is insufficient."""
|
||||
|
||||
def test_too_few_prices(self):
|
||||
"""Fewer than 100 closing prices → uncertainty."""
|
||||
prices = [100.0] * 50 # only 50 days
|
||||
returns = [0.01] * 100
|
||||
result = classify_regime(prices, returns)
|
||||
assert result.regime == MarketRegime.UNCERTAINTY
|
||||
|
||||
def test_too_few_returns(self):
|
||||
"""Fewer than 100 returns → uncertainty."""
|
||||
prices = [100.0] * 120
|
||||
returns = [0.01] * 50 # only 50 returns
|
||||
result = classify_regime(prices, returns)
|
||||
assert result.regime == MarketRegime.UNCERTAINTY
|
||||
|
||||
def test_empty_prices(self):
|
||||
"""Empty price list → uncertainty."""
|
||||
result = classify_regime([], [0.01] * 100)
|
||||
assert result.regime == MarketRegime.UNCERTAINTY
|
||||
|
||||
def test_empty_returns(self):
|
||||
"""Empty return list → uncertainty."""
|
||||
result = classify_regime([100.0] * 120, [])
|
||||
assert result.regime == MarketRegime.UNCERTAINTY
|
||||
|
||||
def test_zero_sigma_returns_uncertainty(self):
|
||||
"""All identical returns (σ=0) → uncertainty."""
|
||||
prices = _make_uptrend_prices()
|
||||
returns = [0.0] * 120 # zero standard deviation
|
||||
result = classify_regime(prices, returns)
|
||||
assert result.regime == MarketRegime.UNCERTAINTY
|
||||
|
||||
def test_default_uncertainty_values(self):
|
||||
"""Default uncertainty has standard threshold values."""
|
||||
result = classify_regime([], [])
|
||||
assert result.regime == MarketRegime.UNCERTAINTY
|
||||
assert result.bullish_threshold == pytest.approx(0.15)
|
||||
assert result.bearish_threshold == pytest.approx(-0.15)
|
||||
assert result.contradiction_penalty_multiplier == pytest.approx(0.6)
|
||||
assert result.trend_indicator == 0.0
|
||||
assert result.volatility_ratio == 1.0
|
||||
Reference in New Issue
Block a user