feat: signal math upgrade — probabilistic, regime-aware scoring pipeline
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
Implement full probabilistic signal processing pipeline gated behind probabilistic_scoring_enabled feature flag in risk_configs: - Bayesian log-likelihood accumulator with Beta posterior and entropy - Regime detector (trend-following, panic, mean-reversion, uncertainty) - Source accuracy tracker with per-source historical prediction accuracy - Sigmoid confidence gate replacing binary gate - Information gain surprise weighting for rare events - Adaptive recency decay with event-specific half-lives - Regime multiplier replacing market context multiplier - Weighted disagreement entropy for contradiction detection - Multiplicative macro exposure with conditional integration - Graph-distance attenuated competitive signal propagation - Exponentially weighted momentum with volatility scaling - Expected value recommendation gate All changes backward-compatible: flag=false preserves exact current behavior. New outputs stored in existing JSONB columns (no schema changes except source_accuracy table via migration 034). Tests: 26 property-based tests (14 correctness properties), 99 unit tests, 1789 total tests passing with zero regressions.
This commit is contained in:
@@ -9,10 +9,11 @@ Evaluates trend summaries against configurable thresholds to decide:
|
||||
All decisions are rule-based with no model involvement. The LLM is only
|
||||
used downstream for optional thesis wording (a separate task).
|
||||
|
||||
Requirements: 7.1, 7.2, 7.3, 7.4
|
||||
Requirements: 7.1, 7.2, 7.3, 7.4, 14.1, 14.2, 14.3, 14.4, 14.5, 14.6
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from dataclasses import dataclass, field
|
||||
from enum import Enum
|
||||
|
||||
@@ -78,6 +79,10 @@ class EligibilityConfig:
|
||||
# Contradiction penalty: higher contradiction → smaller position
|
||||
contradiction_sizing_penalty: float = 0.5
|
||||
|
||||
# --- Expected value gate (Requirement 14) ---
|
||||
# EV threshold: minimum expected value to allow recommendation through
|
||||
ev_threshold: float = 0.005
|
||||
|
||||
|
||||
DEFAULT_ELIGIBILITY_CONFIG = EligibilityConfig()
|
||||
|
||||
@@ -98,6 +103,11 @@ class EligibilityResult:
|
||||
time_horizon: str = ""
|
||||
invalidation_conditions: list[str] = field(default_factory=list)
|
||||
|
||||
# Probabilistic pipeline fields (Req 14.5, 16.2)
|
||||
ev_value: float | None = None
|
||||
p_bull: float | None = None
|
||||
pipeline_mode: str = "heuristic"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Gate checks
|
||||
@@ -318,6 +328,57 @@ def _derive_invalidation_conditions(
|
||||
return conditions
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Expected value computation (Requirements: 14.1–14.6)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Horizon days mapping for EV computation
|
||||
_EV_HORIZON_DAYS: dict[str, float] = {
|
||||
"intraday": 1.0,
|
||||
"1d": 1.0,
|
||||
"7d": 7.0,
|
||||
"30d": 30.0,
|
||||
"90d": 90.0,
|
||||
}
|
||||
|
||||
|
||||
def compute_expected_value(
|
||||
p_bull: float,
|
||||
strength: float,
|
||||
sigma_20: float,
|
||||
horizon_days: float,
|
||||
) -> float:
|
||||
"""Compute expected value for the recommendation gate.
|
||||
|
||||
Formula:
|
||||
R_up = strength · σ_20 · √(horizon_days)
|
||||
R_down = (1 - strength) · σ_20 · √(horizon_days)
|
||||
EV = P_bull · R_up - P_bear · R_down
|
||||
|
||||
where P_bear = 1 - P_bull.
|
||||
|
||||
Args:
|
||||
p_bull: Bayesian bullish probability in [0, 1].
|
||||
strength: Trend strength in [0, 1].
|
||||
sigma_20: 20-day return standard deviation.
|
||||
horizon_days: Number of days for the projection horizon.
|
||||
|
||||
Returns:
|
||||
Expected value (can be negative).
|
||||
|
||||
Requirements: 14.1, 14.2
|
||||
"""
|
||||
p_bear = 1.0 - p_bull
|
||||
sqrt_horizon = math.sqrt(max(horizon_days, 0.0))
|
||||
r_up = strength * sigma_20 * sqrt_horizon
|
||||
r_down = (1.0 - strength) * sigma_20 * sqrt_horizon
|
||||
ev = p_bull * r_up - p_bear * r_down
|
||||
# Guard against NaN/infinity from extreme inputs
|
||||
if math.isnan(ev) or math.isinf(ev):
|
||||
return 0.0
|
||||
return ev
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Main entry point
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -326,6 +387,10 @@ def _derive_invalidation_conditions(
|
||||
def evaluate_eligibility(
|
||||
summary: TrendSummary,
|
||||
config: EligibilityConfig = DEFAULT_ELIGIBILITY_CONFIG,
|
||||
*,
|
||||
probabilistic: bool = False,
|
||||
p_bull: float | None = None,
|
||||
sigma_20: float = 0.01,
|
||||
) -> EligibilityResult:
|
||||
"""Evaluate a trend summary for recommendation eligibility.
|
||||
|
||||
@@ -335,8 +400,27 @@ def evaluate_eligibility(
|
||||
3. Determines the highest allowed execution mode
|
||||
4. Computes position sizing from portfolio rules
|
||||
5. Derives invalidation conditions
|
||||
6. (probabilistic) Applies EV gate: EV > threshold to proceed
|
||||
|
||||
When ``probabilistic=True``:
|
||||
- Computes EV = P_bull · R_up - P_bear · R_down
|
||||
- When EV > threshold (default 0.005), allows recommendation through
|
||||
- When EV ≤ threshold, forces recommendation to informational mode
|
||||
- Populates expected_value, p_bull, pipeline_mode on result
|
||||
|
||||
When ``probabilistic=False``:
|
||||
- Skips EV gate entirely (existing behavior)
|
||||
|
||||
Args:
|
||||
summary: The current trend summary.
|
||||
config: Eligibility configuration thresholds.
|
||||
probabilistic: Use EV gate when True.
|
||||
p_bull: Bayesian bullish probability (required when probabilistic=True).
|
||||
sigma_20: 20-day return standard deviation for EV computation.
|
||||
|
||||
Returns an EligibilityResult with the full decision trace.
|
||||
|
||||
Requirements: 14.1, 14.2, 14.3, 14.4, 14.5, 14.6
|
||||
"""
|
||||
rejection_reasons = _check_gates(summary, config)
|
||||
|
||||
@@ -353,6 +437,21 @@ def evaluate_eligibility(
|
||||
if not eligible:
|
||||
mode = RecommendationMode.INFORMATIONAL
|
||||
|
||||
# EV gate (Requirement 14.1–14.6)
|
||||
ev_value: float | None = None
|
||||
if probabilistic and p_bull is not None:
|
||||
horizon_days = _EV_HORIZON_DAYS.get(summary.window.value, 7.0)
|
||||
ev_value = compute_expected_value(
|
||||
p_bull=p_bull,
|
||||
strength=summary.trend_strength,
|
||||
sigma_20=sigma_20,
|
||||
horizon_days=horizon_days,
|
||||
)
|
||||
|
||||
if ev_value <= config.ev_threshold:
|
||||
# Force to informational mode (Req 14.4)
|
||||
mode = RecommendationMode.INFORMATIONAL
|
||||
|
||||
return EligibilityResult(
|
||||
eligible=eligible,
|
||||
action=action,
|
||||
@@ -361,4 +460,7 @@ def evaluate_eligibility(
|
||||
rejection_reasons=rejection_reasons,
|
||||
time_horizon=horizon,
|
||||
invalidation_conditions=invalidation,
|
||||
ev_value=ev_value,
|
||||
p_bull=p_bull if probabilistic else None,
|
||||
pipeline_mode="probabilistic" if probabilistic else "heuristic",
|
||||
)
|
||||
|
||||
@@ -606,6 +606,13 @@ async def persist_recommendation(
|
||||
"invalidation_conditions": eligibility_result.invalidation_conditions,
|
||||
"risk_classification": risk_class,
|
||||
}
|
||||
|
||||
# Store probabilistic EV fields in risk_checks JSONB (Req 16.2)
|
||||
if eligibility_result.pipeline_mode == "probabilistic":
|
||||
risk_checks["ev"] = eligibility_result.ev_value
|
||||
risk_checks["p_bull"] = eligibility_result.p_bull
|
||||
risk_checks["pipeline_mode"] = eligibility_result.pipeline_mode
|
||||
risk_checks["ev_threshold"] = 0.005
|
||||
await pool.execute(
|
||||
_INSERT_RISK_EVALUATION,
|
||||
rec_id,
|
||||
|
||||
Reference in New Issue
Block a user