feat: signal math upgrade — probabilistic, regime-aware scoring pipeline
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled

Implement full probabilistic signal processing pipeline gated behind
probabilistic_scoring_enabled feature flag in risk_configs:

- Bayesian log-likelihood accumulator with Beta posterior and entropy
- Regime detector (trend-following, panic, mean-reversion, uncertainty)
- Source accuracy tracker with per-source historical prediction accuracy
- Sigmoid confidence gate replacing binary gate
- Information gain surprise weighting for rare events
- Adaptive recency decay with event-specific half-lives
- Regime multiplier replacing market context multiplier
- Weighted disagreement entropy for contradiction detection
- Multiplicative macro exposure with conditional integration
- Graph-distance attenuated competitive signal propagation
- Exponentially weighted momentum with volatility scaling
- Expected value recommendation gate

All changes backward-compatible: flag=false preserves exact current behavior.
New outputs stored in existing JSONB columns (no schema changes except
source_accuracy table via migration 034).

Tests: 26 property-based tests (14 correctness properties), 99 unit tests,
1789 total tests passing with zero regressions.
This commit is contained in:
Celes Renata
2026-04-29 11:41:48 +00:00
parent 8c3c1aab43
commit 4e010bc048
24 changed files with 6058 additions and 60 deletions
+233 -16
View File
@@ -283,27 +283,82 @@ def _determine_impact_direction(
# ---------------------------------------------------------------------------
def _compute_multiplicative_exposure(
geo_overlap: float,
supply_overlap: float,
commodity_overlap: float,
sector_match: float,
) -> float:
"""Compute multiplicative compounding exposure.
Formula: 1 - Π_k(1 - w_k · O_k)
Multi-dimensional exposure compounds — a company exposed across
multiple dimensions receives higher impact than simple addition.
Returns a value in [0, ~0.724] (max when all overlaps are 1.0).
Requirements: 10.1, 10.4, 10.7
"""
product = (
(1.0 - GEO_WEIGHT * geo_overlap)
* (1.0 - SUPPLY_WEIGHT * supply_overlap)
* (1.0 - COMMODITY_WEIGHT * commodity_overlap)
* (1.0 - SECTOR_WEIGHT * sector_match)
)
return 1.0 - product
def _compute_linear_exposure(
geo_overlap: float,
supply_overlap: float,
commodity_overlap: float,
sector_match: float,
) -> float:
"""Compute linear weighted-sum exposure (original heuristic formula).
Formula: w_geo·O_geo + w_supply·O_supply + w_commodity·O_commodity + w_sector·O_sector
Returns a value in [0, 1].
"""
return (
GEO_WEIGHT * geo_overlap
+ SUPPLY_WEIGHT * supply_overlap
+ COMMODITY_WEIGHT * commodity_overlap
+ SECTOR_WEIGHT * sector_match
)
def compute_macro_impact(
event: GlobalEvent,
profile: ExposureProfileSchema,
*,
probabilistic: bool = False,
) -> MacroImpactRecord:
"""Compute the macro impact of a global event on a company.
Scoring formula:
When ``probabilistic=False`` (default), uses the linear weighted-sum:
raw_score = severity_weight * (
0.35 * geographic_overlap +
0.25 * supply_chain_overlap +
0.25 * commodity_overlap +
0.15 * sector_match
)
final_score = apply_resilience_modifier(raw_score, tier, is_international)
When ``probabilistic=True``, uses multiplicative compounding exposure:
raw_score = severity_weight * (1 - Π_k(1 - w_k · O_k))
In both modes, the resilience modifier is applied after the raw score.
Args:
event: The classified global event.
profile: The company's exposure profile.
probabilistic: Use multiplicative formula when True.
Returns:
A MacroImpactRecord with the computed score and metadata.
Requirements: 10.1, 10.2, 10.3, 10.4, 10.5, 10.6
"""
now = datetime.now(timezone.utc)
@@ -360,13 +415,16 @@ def compute_macro_impact(
# Severity weight
severity_weight = SEVERITY_WEIGHTS.get(event.severity, 0.25)
# Raw score
raw_score = severity_weight * (
GEO_WEIGHT * geo_overlap
+ SUPPLY_WEIGHT * supply_overlap
+ COMMODITY_WEIGHT * commodity_overlap
+ SECTOR_WEIGHT * sector_match
)
# Raw score: multiplicative or linear depending on mode
if probabilistic:
exposure = _compute_multiplicative_exposure(
geo_overlap, supply_overlap, commodity_overlap, sector_match,
)
else:
exposure = _compute_linear_exposure(
geo_overlap, supply_overlap, commodity_overlap, sector_match,
)
raw_score = severity_weight * exposure
# Determine if event is international (affects multiple regions)
is_international = len(event.affected_regions) > 1
@@ -406,19 +464,27 @@ def compute_macro_impact_with_sector(
event: GlobalEvent,
profile: ExposureProfileSchema,
company_sector: str = "",
*,
probabilistic: bool = False,
) -> MacroImpactRecord:
"""Compute macro impact with explicit sector matching.
Like compute_macro_impact but accepts a company_sector parameter
for proper sector_match computation.
When ``probabilistic=True``, uses multiplicative compounding exposure.
When ``probabilistic=False``, uses the original linear weighted sum.
Args:
event: The classified global event.
profile: The company's exposure profile.
company_sector: The company's GICS sector name.
probabilistic: Use multiplicative formula when True.
Returns:
A MacroImpactRecord with the computed score and metadata.
Requirements: 10.1, 10.2, 10.3, 10.4, 10.5, 10.6
"""
now = datetime.now(timezone.utc)
@@ -472,13 +538,16 @@ def compute_macro_impact_with_sector(
# Severity weight
severity_weight = SEVERITY_WEIGHTS.get(event.severity, 0.25)
# Raw score
raw_score = severity_weight * (
GEO_WEIGHT * geo_overlap
+ SUPPLY_WEIGHT * supply_overlap
+ COMMODITY_WEIGHT * commodity_overlap
+ SECTOR_WEIGHT * sector_match
)
# Raw score: multiplicative or linear depending on mode
if probabilistic:
exposure = _compute_multiplicative_exposure(
geo_overlap, supply_overlap, commodity_overlap, sector_match,
)
else:
exposure = _compute_linear_exposure(
geo_overlap, supply_overlap, commodity_overlap, sector_match,
)
raw_score = severity_weight * exposure
# International check
is_international = len(event.affected_regions) > 1
@@ -588,6 +657,154 @@ def _infer_commodities(sector: str, industry: str) -> list[str]:
return sector_commodities.get(sector, [])
# ---------------------------------------------------------------------------
# Conditional macro signal integration (Requirements: 11.111.5)
# ---------------------------------------------------------------------------
def compute_conditional_macro_modifier(
company_strength: float,
company_direction: str,
macro_impact: float,
macro_direction: str,
) -> float:
"""Compute the multiplicative macro modifier for conditional integration.
When both company and macro signals exist, macro acts as a modifier:
S_adjusted = S_company · clamp(1 + M_macro · sign_alignment, 0.5, 1.5)
sign_alignment is +1 when macro and company agree in direction,
-1 when they disagree.
Args:
company_strength: The company-level signal strength (absolute).
company_direction: Company trend direction (bullish/bearish/neutral/mixed).
macro_impact: Normalized macro impact score in [0, 1].
macro_direction: Macro impact direction (positive/negative/mixed/neutral).
Returns:
The multiplicative modifier in [0.5, 1.5].
Requirements: 11.1, 11.2
"""
# Determine sign alignment between company and macro directions
_DIRECTION_SIGN = {
"bullish": 1,
"positive": 1,
"bearish": -1,
"negative": -1,
}
company_sign = _DIRECTION_SIGN.get(company_direction, 0)
macro_sign = _DIRECTION_SIGN.get(macro_direction, 0)
if company_sign == 0 or macro_sign == 0:
# Neutral or mixed directions — no alignment signal
sign_alignment = 0.0
elif company_sign == macro_sign:
sign_alignment = 1.0
else:
sign_alignment = -1.0
raw_modifier = 1.0 + macro_impact * sign_alignment
return max(0.5, min(1.5, raw_modifier))
def integrate_macro_signals(
company_signals: list,
macro_signals: list,
company_direction: str,
macro_impacts: list,
ticker: str = "",
*,
probabilistic: bool = False,
macro_signal_weight: float = 0.3,
) -> tuple[list, float]:
"""Integrate macro signals with company signals.
When ``probabilistic=True``:
- Both exist: apply macro as multiplicative modifier on company signals
- Only macro: fall back to additive behavior with weight 0.3
- Only company: use modifier = 1.0 (no change)
When ``probabilistic=False``:
- Preserve current additive merge behavior (concatenate lists)
Args:
company_signals: WeightedSignal list from company layer.
macro_signals: WeightedSignal list from macro layer.
company_direction: Derived company trend direction string.
macro_impacts: List of MacroImpactRecord or similar with
macro_impact_score and impact_direction attributes.
ticker: Ticker symbol for logging.
probabilistic: Use conditional modifier when True.
macro_signal_weight: Weight for macro-only fallback (default 0.3).
Returns:
Tuple of (merged_signals, macro_modifier_applied).
macro_modifier_applied is 1.0 when no modifier was used.
Requirements: 11.1, 11.2, 11.3, 11.4, 11.5
"""
if not probabilistic:
# Heuristic mode: simple additive merge (current behavior)
merged = list(company_signals) + list(macro_signals)
return merged, 1.0
has_company = len(company_signals) > 0
has_macro = len(macro_signals) > 0
if has_company and has_macro:
# Compute average macro impact and dominant direction
avg_macro_impact = 0.0
direction_counts: dict[str, float] = {}
for mir in macro_impacts:
score = getattr(mir, "macro_impact_score", 0.0)
direction = getattr(mir, "impact_direction", "neutral")
avg_macro_impact += score
direction_counts[direction] = direction_counts.get(direction, 0.0) + score
if macro_impacts:
avg_macro_impact /= len(macro_impacts)
# Dominant macro direction by total impact weight
macro_direction = max(direction_counts, key=direction_counts.get) if direction_counts else "neutral"
modifier = compute_conditional_macro_modifier(
company_strength=0.0, # not used in current formula
company_direction=company_direction,
macro_impact=avg_macro_impact,
macro_direction=macro_direction,
)
logger.info(
"Macro modifier for %s: %.4f (avg_impact=%.4f, macro_dir=%s, company_dir=%s)",
ticker, modifier, avg_macro_impact, macro_direction, company_direction,
)
# Apply modifier to company signals by scaling their impact scores
# We create modified copies rather than mutating originals
from copy import copy
modified_signals = []
for sig in company_signals:
new_sig = copy(sig)
new_sig.impact_score = sig.impact_score * modifier
modified_signals.append(new_sig)
return modified_signals, modifier
if has_macro and not has_company:
# Macro-only fallback: additive behavior with weight 0.3 (Req 11.3)
logger.info(
"Macro-only fallback for %s: using additive merge with weight %.2f",
ticker, macro_signal_weight,
)
return list(macro_signals), 1.0
# Company-only: no modification (Req 11.4)
logger.info("Company-only signals for %s: macro modifier=1.0", ticker)
return list(company_signals), 1.0
# ---------------------------------------------------------------------------
# PostgreSQL persistence
# ---------------------------------------------------------------------------