feat: signal math upgrade — probabilistic, regime-aware scoring pipeline
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled

Implement full probabilistic signal processing pipeline gated behind
probabilistic_scoring_enabled feature flag in risk_configs:

- Bayesian log-likelihood accumulator with Beta posterior and entropy
- Regime detector (trend-following, panic, mean-reversion, uncertainty)
- Source accuracy tracker with per-source historical prediction accuracy
- Sigmoid confidence gate replacing binary gate
- Information gain surprise weighting for rare events
- Adaptive recency decay with event-specific half-lives
- Regime multiplier replacing market context multiplier
- Weighted disagreement entropy for contradiction detection
- Multiplicative macro exposure with conditional integration
- Graph-distance attenuated competitive signal propagation
- Exponentially weighted momentum with volatility scaling
- Expected value recommendation gate

All changes backward-compatible: flag=false preserves exact current behavior.
New outputs stored in existing JSONB columns (no schema changes except
source_accuracy table via migration 034).

Tests: 26 property-based tests (14 correctness properties), 99 unit tests,
1789 total tests passing with zero regressions.
This commit is contained in:
Celes Renata
2026-04-29 11:41:48 +00:00
parent 8c3c1aab43
commit 4e010bc048
24 changed files with 6058 additions and 60 deletions
+103 -1
View File
@@ -9,10 +9,11 @@ Evaluates trend summaries against configurable thresholds to decide:
All decisions are rule-based with no model involvement. The LLM is only
used downstream for optional thesis wording (a separate task).
Requirements: 7.1, 7.2, 7.3, 7.4
Requirements: 7.1, 7.2, 7.3, 7.4, 14.1, 14.2, 14.3, 14.4, 14.5, 14.6
"""
from __future__ import annotations
import math
from dataclasses import dataclass, field
from enum import Enum
@@ -78,6 +79,10 @@ class EligibilityConfig:
# Contradiction penalty: higher contradiction → smaller position
contradiction_sizing_penalty: float = 0.5
# --- Expected value gate (Requirement 14) ---
# EV threshold: minimum expected value to allow recommendation through
ev_threshold: float = 0.005
DEFAULT_ELIGIBILITY_CONFIG = EligibilityConfig()
@@ -98,6 +103,11 @@ class EligibilityResult:
time_horizon: str = ""
invalidation_conditions: list[str] = field(default_factory=list)
# Probabilistic pipeline fields (Req 14.5, 16.2)
ev_value: float | None = None
p_bull: float | None = None
pipeline_mode: str = "heuristic"
# ---------------------------------------------------------------------------
# Gate checks
@@ -318,6 +328,57 @@ def _derive_invalidation_conditions(
return conditions
# ---------------------------------------------------------------------------
# Expected value computation (Requirements: 14.114.6)
# ---------------------------------------------------------------------------
# Horizon days mapping for EV computation
_EV_HORIZON_DAYS: dict[str, float] = {
"intraday": 1.0,
"1d": 1.0,
"7d": 7.0,
"30d": 30.0,
"90d": 90.0,
}
def compute_expected_value(
p_bull: float,
strength: float,
sigma_20: float,
horizon_days: float,
) -> float:
"""Compute expected value for the recommendation gate.
Formula:
R_up = strength · σ_20 · √(horizon_days)
R_down = (1 - strength) · σ_20 · √(horizon_days)
EV = P_bull · R_up - P_bear · R_down
where P_bear = 1 - P_bull.
Args:
p_bull: Bayesian bullish probability in [0, 1].
strength: Trend strength in [0, 1].
sigma_20: 20-day return standard deviation.
horizon_days: Number of days for the projection horizon.
Returns:
Expected value (can be negative).
Requirements: 14.1, 14.2
"""
p_bear = 1.0 - p_bull
sqrt_horizon = math.sqrt(max(horizon_days, 0.0))
r_up = strength * sigma_20 * sqrt_horizon
r_down = (1.0 - strength) * sigma_20 * sqrt_horizon
ev = p_bull * r_up - p_bear * r_down
# Guard against NaN/infinity from extreme inputs
if math.isnan(ev) or math.isinf(ev):
return 0.0
return ev
# ---------------------------------------------------------------------------
# Main entry point
# ---------------------------------------------------------------------------
@@ -326,6 +387,10 @@ def _derive_invalidation_conditions(
def evaluate_eligibility(
summary: TrendSummary,
config: EligibilityConfig = DEFAULT_ELIGIBILITY_CONFIG,
*,
probabilistic: bool = False,
p_bull: float | None = None,
sigma_20: float = 0.01,
) -> EligibilityResult:
"""Evaluate a trend summary for recommendation eligibility.
@@ -335,8 +400,27 @@ def evaluate_eligibility(
3. Determines the highest allowed execution mode
4. Computes position sizing from portfolio rules
5. Derives invalidation conditions
6. (probabilistic) Applies EV gate: EV > threshold to proceed
When ``probabilistic=True``:
- Computes EV = P_bull · R_up - P_bear · R_down
- When EV > threshold (default 0.005), allows recommendation through
- When EV ≤ threshold, forces recommendation to informational mode
- Populates expected_value, p_bull, pipeline_mode on result
When ``probabilistic=False``:
- Skips EV gate entirely (existing behavior)
Args:
summary: The current trend summary.
config: Eligibility configuration thresholds.
probabilistic: Use EV gate when True.
p_bull: Bayesian bullish probability (required when probabilistic=True).
sigma_20: 20-day return standard deviation for EV computation.
Returns an EligibilityResult with the full decision trace.
Requirements: 14.1, 14.2, 14.3, 14.4, 14.5, 14.6
"""
rejection_reasons = _check_gates(summary, config)
@@ -353,6 +437,21 @@ def evaluate_eligibility(
if not eligible:
mode = RecommendationMode.INFORMATIONAL
# EV gate (Requirement 14.114.6)
ev_value: float | None = None
if probabilistic and p_bull is not None:
horizon_days = _EV_HORIZON_DAYS.get(summary.window.value, 7.0)
ev_value = compute_expected_value(
p_bull=p_bull,
strength=summary.trend_strength,
sigma_20=sigma_20,
horizon_days=horizon_days,
)
if ev_value <= config.ev_threshold:
# Force to informational mode (Req 14.4)
mode = RecommendationMode.INFORMATIONAL
return EligibilityResult(
eligible=eligible,
action=action,
@@ -361,4 +460,7 @@ def evaluate_eligibility(
rejection_reasons=rejection_reasons,
time_horizon=horizon,
invalidation_conditions=invalidation,
ev_value=ev_value,
p_bull=p_bull if probabilistic else None,
pipeline_mode="probabilistic" if probabilistic else "heuristic",
)