feat: signal math upgrade — probabilistic, regime-aware scoring pipeline
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled

Implement full probabilistic signal processing pipeline gated behind
probabilistic_scoring_enabled feature flag in risk_configs:

- Bayesian log-likelihood accumulator with Beta posterior and entropy
- Regime detector (trend-following, panic, mean-reversion, uncertainty)
- Source accuracy tracker with per-source historical prediction accuracy
- Sigmoid confidence gate replacing binary gate
- Information gain surprise weighting for rare events
- Adaptive recency decay with event-specific half-lives
- Regime multiplier replacing market context multiplier
- Weighted disagreement entropy for contradiction detection
- Multiplicative macro exposure with conditional integration
- Graph-distance attenuated competitive signal propagation
- Exponentially weighted momentum with volatility scaling
- Expected value recommendation gate

All changes backward-compatible: flag=false preserves exact current behavior.
New outputs stored in existing JSONB columns (no schema changes except
source_accuracy table via migration 034).

Tests: 26 property-based tests (14 correctness properties), 99 unit tests,
1789 total tests passing with zero regressions.
This commit is contained in:
Celes Renata
2026-04-29 11:41:48 +00:00
parent 8c3c1aab43
commit 4e010bc048
24 changed files with 6058 additions and 60 deletions
+357
View File
@@ -506,3 +506,360 @@ class TestAcceleratedDecay:
def test_standard_decay_positive(self):
result = compute_standard_recency_decay(168.0)
assert 0.0 < result < 1.0
# ---------------------------------------------------------------------------
# Multiplicative macro exposure formula (Task 10.1, Requirements: 10.110.6)
# ---------------------------------------------------------------------------
from services.aggregation.interpolation import (
_compute_linear_exposure,
_compute_multiplicative_exposure,
compute_conditional_macro_modifier,
integrate_macro_signals,
)
class TestMultiplicativeExposure:
"""Tests for the multiplicative compounding exposure formula."""
def test_zero_overlap_returns_zero(self):
"""All overlaps zero → exposure = 0."""
assert _compute_multiplicative_exposure(0.0, 0.0, 0.0, 0.0) == 0.0
def test_max_overlap_approx_0724(self):
"""All overlaps 1.0 → exposure ≈ 0.689 (from the multiplicative formula)."""
result = _compute_multiplicative_exposure(1.0, 1.0, 1.0, 1.0)
expected = 1.0 - (1 - 0.35) * (1 - 0.25) * (1 - 0.25) * (1 - 0.15)
assert math.isclose(result, expected, abs_tol=1e-6)
# Requirement 10.4 states ≈0.724 but the exact formula yields ≈0.689
assert 0.6 < result < 0.8
def test_single_dimension_equals_weight(self):
"""Only geo overlap at 1.0 → exposure = 0.35."""
result = _compute_multiplicative_exposure(1.0, 0.0, 0.0, 0.0)
assert math.isclose(result, 0.35, abs_tol=1e-6)
def test_multiplicative_differs_from_linear_for_multi_overlap(self):
"""Multiplicative and linear produce different results for multi-dimension overlap."""
geo, supply, commodity, sector = 0.8, 0.6, 0.5, 0.4
mult = _compute_multiplicative_exposure(geo, supply, commodity, sector)
lin = _compute_linear_exposure(geo, supply, commodity, sector)
# They should produce different values (multiplicative compounds)
assert mult != lin
# Both should be positive
assert mult > 0.0
assert lin > 0.0
def test_adding_overlap_increases_score(self):
"""Adding a non-zero overlap in any dimension increases the total."""
base = _compute_multiplicative_exposure(0.5, 0.0, 0.0, 0.0)
with_supply = _compute_multiplicative_exposure(0.5, 0.3, 0.0, 0.0)
assert with_supply > base
def test_probabilistic_flag_uses_multiplicative(self):
"""compute_macro_impact with probabilistic=True uses multiplicative formula."""
event = GlobalEvent(
event_id="evt-mult",
event_types=["supply_disruption"],
severity="critical",
affected_regions=["US"],
affected_commodities=["crude_oil"],
confidence=0.9,
)
profile = ExposureProfileSchema(
company_id="comp-mult",
geographic_revenue_mix={"US": 0.8},
supply_chain_regions=["US"],
key_input_commodities=["crude_oil"],
market_position_tier=MarketPositionTier.REGIONAL,
)
heuristic = compute_macro_impact(event, profile, probabilistic=False)
probabilistic_result = compute_macro_impact(event, profile, probabilistic=True)
# Both should produce positive scores
assert heuristic.macro_impact_score > 0.0
assert probabilistic_result.macro_impact_score > 0.0
# They should produce different scores (different formulas)
assert heuristic.macro_impact_score != probabilistic_result.macro_impact_score
def test_probabilistic_false_preserves_linear(self):
"""probabilistic=False produces identical results to original behavior."""
event = GlobalEvent(
event_id="evt-lin",
event_types=["supply_disruption"],
severity="high",
affected_regions=["US"],
affected_commodities=["crude_oil"],
confidence=0.9,
)
profile = ExposureProfileSchema(
company_id="comp-lin",
geographic_revenue_mix={"US": 0.5},
supply_chain_regions=["US"],
key_input_commodities=["crude_oil"],
market_position_tier=MarketPositionTier.REGIONAL,
)
record = compute_macro_impact(event, profile, probabilistic=False)
# Manually compute expected linear score
geo = 0.5 # revenue mix for US
supply = 1.0 # 1/1 supply regions match
commodity = 1.0 # crude_oil matches
severity = 0.75 # high
expected_raw = severity * (0.35 * geo + 0.25 * supply + 0.25 * commodity + 0.15 * 0.0)
# Single region → no resilience modifier
assert math.isclose(record.macro_impact_score, expected_raw, abs_tol=1e-4)
def test_zero_overlap_returns_zero_score_probabilistic(self):
"""Zero overlap still returns zero in probabilistic mode."""
event = GlobalEvent(
event_id="evt-zero",
event_types=["supply_disruption"],
severity="critical",
affected_regions=["JP"],
affected_commodities=["gold"],
confidence=0.9,
)
profile = ExposureProfileSchema(
company_id="comp-zero",
geographic_revenue_mix={"US": 1.0},
supply_chain_regions=["US"],
key_input_commodities=["crude_oil"],
market_position_tier=MarketPositionTier.REGIONAL,
)
record = compute_macro_impact(event, profile, probabilistic=True)
assert record.macro_impact_score == 0.0
def test_with_sector_probabilistic(self):
"""compute_macro_impact_with_sector supports probabilistic flag."""
event = GlobalEvent(
event_id="evt-sec-prob",
event_types=["supply_disruption"],
severity="high",
affected_regions=["US"],
affected_sectors=["Energy"],
confidence=0.9,
)
profile = ExposureProfileSchema(
company_id="comp-sec-prob",
geographic_revenue_mix={"US": 0.5},
market_position_tier=MarketPositionTier.REGIONAL,
)
heuristic = compute_macro_impact_with_sector(
event, profile, "Energy", probabilistic=False,
)
probabilistic = compute_macro_impact_with_sector(
event, profile, "Energy", probabilistic=True,
)
assert heuristic.macro_impact_score > 0.0
assert probabilistic.macro_impact_score > 0.0
def test_severity_preserved_in_probabilistic(self):
"""Severity mapping is preserved in probabilistic mode."""
profile = ExposureProfileSchema(
company_id="comp-sev",
geographic_revenue_mix={"US": 0.5},
supply_chain_regions=["US"],
key_input_commodities=["crude_oil"],
market_position_tier=MarketPositionTier.REGIONAL,
)
event_low = GlobalEvent(
event_id="evt-low-p",
event_types=["supply_disruption"],
severity="low",
affected_regions=["US"],
affected_commodities=["crude_oil"],
confidence=0.9,
)
event_crit = GlobalEvent(
event_id="evt-crit-p",
event_types=["supply_disruption"],
severity="critical",
affected_regions=["US"],
affected_commodities=["crude_oil"],
confidence=0.9,
)
low = compute_macro_impact(event_low, profile, probabilistic=True)
crit = compute_macro_impact(event_crit, profile, probabilistic=True)
assert crit.macro_impact_score >= low.macro_impact_score
# ---------------------------------------------------------------------------
# Conditional macro signal integration (Task 10.2, Requirements: 11.111.5)
# ---------------------------------------------------------------------------
class TestConditionalMacroModifier:
"""Tests for compute_conditional_macro_modifier."""
def test_agreeing_directions_amplify(self):
"""Bullish company + positive macro → modifier > 1.0."""
modifier = compute_conditional_macro_modifier(
company_strength=0.5,
company_direction="bullish",
macro_impact=0.3,
macro_direction="positive",
)
assert modifier > 1.0
assert math.isclose(modifier, 1.3, abs_tol=1e-6)
def test_disagreeing_directions_dampen(self):
"""Bullish company + negative macro → modifier < 1.0."""
modifier = compute_conditional_macro_modifier(
company_strength=0.5,
company_direction="bullish",
macro_impact=0.3,
macro_direction="negative",
)
assert modifier < 1.0
assert math.isclose(modifier, 0.7, abs_tol=1e-6)
def test_neutral_company_no_alignment(self):
"""Neutral company direction → modifier = 1.0."""
modifier = compute_conditional_macro_modifier(
company_strength=0.5,
company_direction="neutral",
macro_impact=0.5,
macro_direction="positive",
)
assert math.isclose(modifier, 1.0, abs_tol=1e-6)
def test_neutral_macro_no_alignment(self):
"""Neutral macro direction → modifier = 1.0."""
modifier = compute_conditional_macro_modifier(
company_strength=0.5,
company_direction="bullish",
macro_impact=0.5,
macro_direction="neutral",
)
assert math.isclose(modifier, 1.0, abs_tol=1e-6)
def test_clamped_to_max_1_5(self):
"""Large agreeing impact clamped to 1.5."""
modifier = compute_conditional_macro_modifier(
company_strength=0.5,
company_direction="bearish",
macro_impact=0.8,
macro_direction="negative",
)
assert modifier <= 1.5
def test_clamped_to_min_0_5(self):
"""Large disagreeing impact clamped to 0.5."""
modifier = compute_conditional_macro_modifier(
company_strength=0.5,
company_direction="bearish",
macro_impact=0.8,
macro_direction="positive",
)
assert modifier >= 0.5
def test_zero_macro_impact_no_change(self):
"""Zero macro impact → modifier = 1.0."""
modifier = compute_conditional_macro_modifier(
company_strength=0.5,
company_direction="bullish",
macro_impact=0.0,
macro_direction="positive",
)
assert math.isclose(modifier, 1.0, abs_tol=1e-6)
def test_bearish_negative_agree(self):
"""Bearish company + negative macro → they agree → modifier > 1.0."""
modifier = compute_conditional_macro_modifier(
company_strength=0.5,
company_direction="bearish",
macro_impact=0.2,
macro_direction="negative",
)
assert modifier > 1.0
class TestIntegrateMacroSignals:
"""Tests for integrate_macro_signals."""
def _make_signal(self, doc_id: str, sentiment: float, impact: float):
"""Helper to create a minimal WeightedSignal-like object."""
from services.aggregation.scoring import SignalWeight, WeightedSignal
weight = SignalWeight(
recency=1.0,
credibility=0.8,
novelty_bonus=0.0,
confidence_gate=1.0,
market_ctx_multiplier=1.0,
combined=0.8,
)
return WeightedSignal(
document_id=doc_id,
weight=weight,
sentiment_value=sentiment,
impact_score=impact,
)
def _make_macro_impact(self, score: float, direction: str):
"""Helper to create a MacroImpactRecord."""
return MacroImpactRecord(
event_id="evt-1",
company_id="comp-1",
macro_impact_score=score,
impact_direction=direction,
)
def test_heuristic_mode_concatenates(self):
"""probabilistic=False → simple concatenation."""
company = [self._make_signal("c1", 0.5, 0.6)]
macro = [self._make_signal("m1", 0.3, 0.4)]
merged, modifier = integrate_macro_signals(
company, macro, "bullish", [], probabilistic=False,
)
assert len(merged) == 2
assert modifier == 1.0
def test_probabilistic_both_exist_applies_modifier(self):
"""Both company and macro → modifier applied to company signals."""
company = [self._make_signal("c1", 0.5, 0.6)]
macro = [self._make_signal("m1", 0.3, 0.4)]
impacts = [self._make_macro_impact(0.3, "positive")]
merged, modifier = integrate_macro_signals(
company, macro, "bullish", impacts,
ticker="AAPL", probabilistic=True,
)
# Modifier should be > 1.0 (agreeing directions)
assert modifier > 1.0
# Only company signals returned (modified), not macro
assert len(merged) == 1
# Impact score should be scaled by modifier
assert merged[0].impact_score > 0.6
def test_probabilistic_macro_only_fallback(self):
"""Only macro signals → additive fallback."""
macro = [self._make_signal("m1", 0.3, 0.4)]
impacts = [self._make_macro_impact(0.3, "positive")]
merged, modifier = integrate_macro_signals(
[], macro, "neutral", impacts,
ticker="AAPL", probabilistic=True,
)
assert len(merged) == 1
assert modifier == 1.0
def test_probabilistic_company_only_no_modifier(self):
"""Only company signals → modifier = 1.0."""
company = [self._make_signal("c1", 0.5, 0.6)]
merged, modifier = integrate_macro_signals(
company, [], "bullish", [],
ticker="AAPL", probabilistic=True,
)
assert len(merged) == 1
assert modifier == 1.0
assert merged[0].impact_score == 0.6
def test_probabilistic_disagreeing_dampens(self):
"""Disagreeing directions → modifier < 1.0, impact reduced."""
company = [self._make_signal("c1", 0.5, 0.6)]
macro = [self._make_signal("m1", -0.3, 0.4)]
impacts = [self._make_macro_impact(0.3, "negative")]
merged, modifier = integrate_macro_signals(
company, macro, "bullish", impacts,
ticker="AAPL", probabilistic=True,
)
assert modifier < 1.0
assert merged[0].impact_score < 0.6