feat: signal math upgrade — probabilistic, regime-aware scoring pipeline
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled

Implement full probabilistic signal processing pipeline gated behind
probabilistic_scoring_enabled feature flag in risk_configs:

- Bayesian log-likelihood accumulator with Beta posterior and entropy
- Regime detector (trend-following, panic, mean-reversion, uncertainty)
- Source accuracy tracker with per-source historical prediction accuracy
- Sigmoid confidence gate replacing binary gate
- Information gain surprise weighting for rare events
- Adaptive recency decay with event-specific half-lives
- Regime multiplier replacing market context multiplier
- Weighted disagreement entropy for contradiction detection
- Multiplicative macro exposure with conditional integration
- Graph-distance attenuated competitive signal propagation
- Exponentially weighted momentum with volatility scaling
- Expected value recommendation gate

All changes backward-compatible: flag=false preserves exact current behavior.
New outputs stored in existing JSONB columns (no schema changes except
source_accuracy table via migration 034).

Tests: 26 property-based tests (14 correctness properties), 99 unit tests,
1789 total tests passing with zero regressions.
This commit is contained in:
Celes Renata
2026-04-29 11:41:48 +00:00
parent 8c3c1aab43
commit 4e010bc048
24 changed files with 6058 additions and 60 deletions
+71 -2
View File
@@ -4,10 +4,11 @@ Analyses weighted signals to detect and represent disagreement explicitly,
rather than collapsing contradictory evidence into a single unsupported
conclusion.
Requirements: 6.4, 6.5
Requirements: 6.4, 6.5, 15.115.7
"""
from __future__ import annotations
import math
from dataclasses import dataclass
from services.aggregation.scoring import WeightedSignal
@@ -35,6 +36,9 @@ class ContradictionResult:
def detect_contradictions(
signals: list[WeightedSignal],
catalyst_entries: list[CatalystEntry] | None = None,
*,
probabilistic: bool = False,
w_threshold: float = 5.0,
) -> ContradictionResult:
"""Run contradiction detection across multiple dimensions.
@@ -42,6 +46,16 @@ def detect_contradictions(
1. Sentiment disagreement — the core positive-vs-negative split
2. Catalyst disagreement — same catalyst type with opposing sentiment
When ``probabilistic`` is True, the overall score uses weighted
disagreement entropy (Req 15.115.7) instead of the minority/majority
ratio. When False, the existing ratio formula is preserved exactly.
Args:
signals: Weighted signals to analyse.
catalyst_entries: Optional catalyst metadata for per-catalyst analysis.
probabilistic: Use entropy-based scoring when True.
w_threshold: Evidence mass threshold for entropy weighting (default 5.0).
Returns a ContradictionResult with an overall score and per-dimension
disagreement details.
"""
@@ -55,7 +69,10 @@ def detect_contradictions(
catalyst_details = _detect_catalyst_disagreement(signals, catalyst_entries)
details.extend(catalyst_details)
score = _compute_overall_score(signals)
if probabilistic:
score = _compute_entropy_score(signals, w_threshold)
else:
score = _compute_overall_score(signals)
return ContradictionResult(score=score, details=details)
@@ -82,6 +99,58 @@ def _compute_overall_score(signals: list[WeightedSignal]) -> float:
return round(minority / total, 4)
def _compute_entropy_score(
signals: list[WeightedSignal],
w_threshold: float = 5.0,
) -> float:
"""Weighted disagreement entropy — probabilistic contradiction score.
Computes Shannon entropy over the positive/negative weight distribution,
weighted by evidence mass relative to a configurable threshold.
Formula:
f_pos = W_pos / (W_pos + W_neg)
f_neg = 1 - f_pos
H = -f_pos·log₂(f_pos) - f_neg·log₂(f_neg) (in [0, 1])
score = H · min(1.0, (W_pos + W_neg) / W_threshold)
Returns 0.0 when only one direction exists (no disagreement).
Requirements: 15.115.7
"""
if not signals:
return 0.0
pos_weight = 0.0
neg_weight = 0.0
for sig in signals:
w = sig.weight.combined * sig.impact_score
if sig.sentiment_value > 0:
pos_weight += w
elif sig.sentiment_value < 0:
neg_weight += w
# No disagreement when only one direction exists (Req 15.5)
if pos_weight <= 0.0 or neg_weight <= 0.0:
return 0.0
total = pos_weight + neg_weight
# Compute weight fractions (Req 15.2)
f_pos = pos_weight / total
f_neg = neg_weight / total # = 1 - f_pos
# Shannon entropy H = -f_pos·log₂(f_pos) - f_neg·log₂(f_neg) (Req 15.3)
# Guard against log₂(0) — already handled by the early return above
h_contradiction = -f_pos * math.log2(f_pos) - f_neg * math.log2(f_neg)
# Weight by evidence mass (Req 15.4)
evidence_factor = min(1.0, total / w_threshold) if w_threshold > 0.0 else 1.0
score = h_contradiction * evidence_factor
return round(score, 4)
def _detect_sentiment_disagreement(
signals: list[WeightedSignal],
) -> DisagreementDetail | None: