feat: signal math upgrade — probabilistic, regime-aware scoring pipeline
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled

Implement full probabilistic signal processing pipeline gated behind
probabilistic_scoring_enabled feature flag in risk_configs:

- Bayesian log-likelihood accumulator with Beta posterior and entropy
- Regime detector (trend-following, panic, mean-reversion, uncertainty)
- Source accuracy tracker with per-source historical prediction accuracy
- Sigmoid confidence gate replacing binary gate
- Information gain surprise weighting for rare events
- Adaptive recency decay with event-specific half-lives
- Regime multiplier replacing market context multiplier
- Weighted disagreement entropy for contradiction detection
- Multiplicative macro exposure with conditional integration
- Graph-distance attenuated competitive signal propagation
- Exponentially weighted momentum with volatility scaling
- Expected value recommendation gate

All changes backward-compatible: flag=false preserves exact current behavior.
New outputs stored in existing JSONB columns (no schema changes except
source_accuracy table via migration 034).

Tests: 26 property-based tests (14 correctness properties), 99 unit tests,
1789 total tests passing with zero regressions.
This commit is contained in:
Celes Renata
2026-04-29 11:41:48 +00:00
parent 8c3c1aab43
commit 4e010bc048
24 changed files with 6058 additions and 60 deletions
+93
View File
@@ -4,6 +4,9 @@ Tests the pure logic functions (no DB required). The async DB functions
are covered by integration tests.
"""
from datetime import datetime, timedelta, timezone
from unittest.mock import AsyncMock
import pytest
from services.aggregation.scoring import (
ScoringConfig,
@@ -21,6 +24,7 @@ from services.aggregation.worker import (
compute_trend_confidence,
derive_trend_direction,
extract_catalysts_and_risks,
fetch_probabilistic_scoring_enabled,
rank_evidence,
)
from services.shared.schemas import MarketContext, TrendDirection, TrendWindow
@@ -392,3 +396,92 @@ def test_assemble_trend_with_evidence_empty_signals():
assert result.supporting_evidence == []
assert result.opposing_evidence == []
assert result.summary.trend_direction == TrendDirection.NEUTRAL
# ---------------------------------------------------------------------------
# AggregationConfig — probabilistic_scoring_enabled field
# ---------------------------------------------------------------------------
def test_aggregation_config_probabilistic_default_false():
"""probabilistic_scoring_enabled defaults to False (heuristic pipeline)."""
cfg = AggregationConfig()
assert cfg.probabilistic_scoring_enabled is False
def test_aggregation_config_probabilistic_explicit_true():
"""probabilistic_scoring_enabled can be set to True."""
cfg = AggregationConfig(probabilistic_scoring_enabled=True)
assert cfg.probabilistic_scoring_enabled is True
# ---------------------------------------------------------------------------
# fetch_probabilistic_scoring_enabled — DB toggle reading
# ---------------------------------------------------------------------------
class _FakeRecord(dict):
"""Minimal dict-like object that mimics an asyncpg Record."""
pass
@pytest.mark.asyncio
async def test_fetch_probabilistic_enabled_true():
"""Returns True when risk_configs has probabilistic_scoring_enabled='true'."""
pool = AsyncMock()
pool.fetchrow = AsyncMock(
return_value=_FakeRecord({"probabilistic_scoring_enabled": "true"}),
)
result = await fetch_probabilistic_scoring_enabled(pool)
assert result is True
@pytest.mark.asyncio
async def test_fetch_probabilistic_enabled_false():
"""Returns False when risk_configs has probabilistic_scoring_enabled='false'."""
pool = AsyncMock()
pool.fetchrow = AsyncMock(
return_value=_FakeRecord({"probabilistic_scoring_enabled": "false"}),
)
result = await fetch_probabilistic_scoring_enabled(pool)
assert result is False
@pytest.mark.asyncio
async def test_fetch_probabilistic_enabled_missing_key():
"""Returns False when the key is missing from config JSONB (value is None)."""
pool = AsyncMock()
pool.fetchrow = AsyncMock(
return_value=_FakeRecord({"probabilistic_scoring_enabled": None}),
)
result = await fetch_probabilistic_scoring_enabled(pool)
assert result is False
@pytest.mark.asyncio
async def test_fetch_probabilistic_enabled_no_config_row():
"""Returns False when no risk_configs row exists."""
pool = AsyncMock()
pool.fetchrow = AsyncMock(return_value=None)
result = await fetch_probabilistic_scoring_enabled(pool)
assert result is False
@pytest.mark.asyncio
async def test_fetch_probabilistic_enabled_invalid_value():
"""Returns False when the value is not a valid boolean string."""
pool = AsyncMock()
pool.fetchrow = AsyncMock(
return_value=_FakeRecord({"probabilistic_scoring_enabled": "yes"}),
)
result = await fetch_probabilistic_scoring_enabled(pool)
assert result is False
@pytest.mark.asyncio
async def test_fetch_probabilistic_enabled_db_unreachable():
"""Returns False (fail-safe) when the database query raises an exception."""
pool = AsyncMock()
pool.fetchrow = AsyncMock(side_effect=Exception("connection refused"))
result = await fetch_probabilistic_scoring_enabled(pool)
assert result is False