# Stonks Oracle — Math Core v3 Upgrade Purpose: replace the linear weighted-sentiment core with a calibrated evidence engine while preserving the existing service boundaries: `ingestion -> parser -> extractor -> aggregation -> recommendation -> risk -> trading` This version keeps the public concepts already used by Stonks Oracle: - `WeightedSignal` - company / macro / competitive layers - intraday / 1d / 7d / 30d / 90d windows - contradiction detection - trend projection - recommendation eligibility - risk-tiered position sizing - circuit breakers The internal math changes from “weighted sum of vibes” to: `EvidenceUnit -> calibrated reliability -> likelihood ratio -> de-correlated posterior -> return distribution -> EV/risk decision` --- ## 0. Core Design Change ### Old core ```text W_combined = confidence * recency * credibility * novelty * context S_avg = sum(W_combined * impact * sentiment) / sum(W_combined * impact) ``` This is explainable, but it is not probabilistic. It double-counts correlated articles, treats arbitrary weights as likelihood, and turns noisy agreement into false certainty. ### New core ```text raw signal -> EvidenceUnit -> calibrated reliability p_correct -> signal log-likelihood ratio LLR_i -> correlation-adjusted cluster evidence LLR_c -> posterior probability P_up -> expected return distribution -> EV-gated recommendation -> Kelly/risk-capped position size ``` The deterministic score remains as an explainability layer only. The trading/recommendation decision should use the posterior and EV layers. --- ## 1. Canonical Evidence Unit Every company, macro, or competitive signal is normalized into the same internal shape before aggregation. ```text EvidenceUnit_i = { symbol, layer_i, # company | macro | competitive event_type_i, source_id_i, source_group_i, timestamp_i, horizon_i, # intraday | 1d | 7d | 30d | 90d direction_i, # -1 bearish, 0 neutral, +1 bullish sentiment_strength_i, # [0, 1] impact_i, # [0, 1] extraction_conf_i, # [0, 1] source_cred_i, # [0, 1] novelty_i, # [0, 1] event_base_rate_i, # P(event_type) cluster_id_i } ``` Neutral signals do not vote directionally, but they still count toward quality, coverage, and contradiction context. --- ## 2. Replace Combined Weight with Calibrated Reliability ### 2.1 Extraction reliability Replace the hard confidence gate and raw sigmoid multiplier with a calibrated reliability term. ```text q_ext = sigmoid(k_ext * (extraction_conf_i - m_ext)) ``` Defaults: ```text k_ext = 8.0 m_ext = 0.55 ``` This makes low-confidence extraction fade instead of abruptly disappearing, but still penalizes weak extraction aggressively. --- ### 2.2 Source reliability with Bayesian shrinkage Replace: ```text F_accuracy = 0.5 + accuracy_ratio ``` with a posterior source skill estimate. For each source or source group: ```text theta_s ~ Beta(alpha_0 + hits_s, beta_0 + misses_s) E[theta_s] = (alpha_0 + hits_s) / (alpha_0 + beta_0 + hits_s + misses_s) ``` Defaults: ```text alpha_0 = 3 beta_0 = 3 ``` This starts every source near 50% until it earns trust. Convert to usable reliability: ```text q_source = clamp((E[theta_s] - 0.50) / 0.35, 0.0, 1.0) ``` A source with 50% realized usefulness contributes little extra skill. A source near 85% realized usefulness approaches full source reliability. --- ### 2.3 Recency reliability Keep exponential half-life decay, but do not clamp stale evidence to a nonzero floor for directional voting. ```text q_recency = 2^(-age_hours / half_life_hours) ``` Use a floor only for explainability display, not for posterior voting. Recommended half-lives: | Horizon | Half-life | |---|---:| | intraday | 2h | | 1d | 12h | | 7d | 72h | | 30d | 240h | | 90d | 720h | Adaptive extension: ```text tau_adaptive = tau_base * (1 + 0.75 * impact_i + 0.50 * surprise_i) q_recency = 2^(-age_hours / tau_adaptive) ``` where: ```text surprise_i = clamp(-log2(event_base_rate_i) / 5, 0, 1) ``` This keeps rare/major events alive longer without letting them live forever. --- ### 2.4 Novelty as de-duplication, not hype boost Replace direct novelty multiplication: ```text 1 + 0.25 * novelty ``` with a saturation term inside each event cluster. For signal `i` inside cluster `c`: ```text q_novelty_i = 1 / sqrt(1 + duplicate_count_before_i) ``` Then: ```text q_uniqueness_i = clamp(0.50 + 0.50 * novelty_i, 0.50, 1.00) * q_novelty_i ``` This prevents 12 near-identical articles from becoming 12 units of evidence. --- ### 2.5 Final signal reliability ```text q_i = q_ext * q_source * source_cred_i * q_recency * q_uniqueness_i ``` Clamp only at the very end: ```text q_i = clamp(q_i, 0, 1) ``` --- ## 3. Convert Signals to Log-Likelihood Ratios The posterior engine should not sum arbitrary sentiment weights. It should sum calibrated evidence. ### 3.1 Directional correctness probability ```text p_correct_i = 0.50 + p_edge_max * q_i * impact_i * sentiment_strength_i ``` Defaults: ```text p_edge_max = 0.35 p_correct_i = clamp(p_correct_i, 0.501, 0.85) ``` Why 0.85 max? Because even very good text signals should not become near-certain by themselves. --- ### 3.2 Signal likelihood ratio ```text LLR_i = direction_i * log(p_correct_i / (1 - p_correct_i)) ``` Examples: | p_correct | abs(LLR) | |---:|---:| | 0.55 | 0.20 | | 0.60 | 0.41 | | 0.70 | 0.85 | | 0.80 | 1.39 | | 0.85 | 1.73 | This makes one strong signal meaningful but not magical. --- ## 4. Correlation-Aware Evidence Clustering ### 4.1 Cluster signals Cluster by: ```text (symbol, horizon, event_type, normalized_event_key, source_group, time_bucket) ``` Also include embedding/content similarity when available. Near-duplicates go into the same cluster even if they come from different URLs. --- ### 4.2 Effective evidence count For a cluster `c` with signal weights `w_i = abs(LLR_i)`: ```text n_eff_c = (sum_i w_i)^2 / (sum_i w_i^2 + 2 * sum_{i 1.5 or abs(trend_z) > 2.5 | 0.70 | 0.70 | high | | trend_following | abs(trend_z) >= 0.75 and vol_ratio < 1.3 | 1.10 | 1.00 | normal | | mean_reversion | abs(trend_z) < 0.50 and vol_ratio < 1.0 | 0.90 | 0.95 | normal | | uncertainty | otherwise | 0.80 | 0.85 | high | --- ## 6. Posterior Trend Assembly ### 6.1 Posterior log-odds ```text logit(P_up) = logit(P_prior) + sum_c(gamma_regime * LLR_c) ``` where: ```text gamma_regime = evidence multiplier from regime table ``` Then: ```text P_up = sigmoid(logit(P_up)) P_down = 1 - P_up ``` --- ### 6.2 Direction Use dynamic thresholds by regime. | Regime | Bullish if | Bearish if | |---|---:|---:| | panic | P_up >= 0.68 | P_up <= 0.32 | | trend_following | P_up >= 0.60 | P_up <= 0.40 | | mean_reversion | P_up >= 0.63 | P_up <= 0.37 | | uncertainty | P_up >= 0.65 | P_up <= 0.35 | Otherwise: neutral or mixed. --- ### 6.3 Strength Replace: ```text strength = abs(S_avg) ``` with: ```text strength = abs(2 * P_up - 1) ``` This keeps strength in `[0, 1]`, but now it actually means posterior directional separation. --- ## 7. Contradiction Score v3 Contradiction should measure meaningful opposing evidence, not just minority weight. ```text E_pos = sum_c(max(LLR_c, 0)) E_neg = sum_c(max(-LLR_c, 0)) E_total = E_pos + E_neg ``` If `E_total = 0`: ```text contradiction = 0 ``` Otherwise: ```text f_pos = E_pos / E_total f_neg = E_neg / E_total H_conflict = -f_pos * log2(f_pos) - f_neg * log2(f_neg) volume_factor = 1 - exp(-E_total / E_conflict_scale) contradiction = H_conflict * volume_factor ``` Default: ```text E_conflict_scale = 3.0 ``` This avoids screaming “contradiction” when there are only two tiny weak signals. --- ## 8. Confidence v3 Confidence must not be the same thing as bullishness. A model can be confidently mixed, weakly bullish, or confidently bearish. ```text n_eff_total = sum_c(n_eff_c) C_evidence = 1 - exp(-n_eff_total / k_evidence) C_direction = abs(2 * P_up - 1) C_quality = weighted_mean(q_i, weight=abs(LLR_i)) C_regime = regime confidence multiplier C_contradiction = 1 - contradiction C_data = data_quality_score ``` Default: ```text k_evidence = 5.0 ``` Final confidence: ```text confidence = clamp( C_evidence * sqrt(C_quality) * sqrt(max(C_direction, 0.05)) * C_regime * C_contradiction * C_data, 0, 1 ) ``` Why multiplicative? Because one bad dimension should actually suppress the trade instead of being averaged away. --- ## 9. Macro Layer v3 ### 9.1 Exposure as noisy-OR Keep the multiplicative macro exposure idea, but normalize it so full exposure can actually reach 1. ```text E_macro_raw = 1 - product_k(1 - w_k * O_k) E_macro_max = 1 - product_k(1 - w_k) E_macro = E_macro_raw / E_macro_max ``` Default weights: ```text w_geo = 0.35 w_supply = 0.25 w_commodity = 0.25 w_sector = 0.15 ``` Now full overlap maps to 1.0, not 0.689. --- ### 9.2 Resilience as effect dampener Do not multiply the whole score blindly. Apply resilience to impact. ```text resilience_dampener = { global_leader: 0.70, multinational: 0.85, regional: 1.00, domestic: 1.20 } ``` For international events: ```text macro_impact = clamp(severity_weight * E_macro * resilience_dampener, 0, 1) ``` For domestic events: ```text macro_impact = clamp(severity_weight * E_macro, 0, 1) ``` --- ### 9.3 Macro likelihood ratio ```text p_macro = 0.50 + 0.30 * macro_impact * event_confidence * q_recency p_macro = clamp(p_macro, 0.501, 0.80) LLR_macro = macro_direction * log(p_macro / (1 - p_macro)) ``` Macro enters the same posterior engine as company evidence. No special post-hoc modifier is needed. --- ## 10. Competitive Layer v3 ### 10.1 Correlation shrinkage Replace raw rolling correlation with shrunk correlation. ```text rho_shrunk = (n / (n + k_rho)) * rho_rolling + (k_rho / (n + k_rho)) * rho_prior ``` Defaults: ```text k_rho = 30 rho_prior_same_sector = 0.30 rho_prior_cross_sector = 0.10 ``` Use only positive propagation unless the relationship is explicitly inverse. ```text rho_effective = max(rho_shrunk, 0) ``` --- ### 10.2 Graph attenuation ```text attenuation = rho_effective * exp(-lambda_graph * d_network) ``` Default: ```text lambda_graph = 0.85 max_distance = 3 ``` No propagation when `d_network > 3`. --- ### 10.3 Competitive likelihood ratio ```text LLR_competitive = LLR_source * attenuation * pattern_confidence ``` Clamp: ```text LLR_competitive = clamp(LLR_competitive, -1.25, 1.25) ``` Competitive signals can support a case. They should not dominate a case. --- ## 11. Trend Projection v3 Replace simple momentum difference with a posterior state. ### 11.1 Evidence state ```text A_t = phi_regime * A_{t-1} + sum_c(LLR_c) P_up_t = sigmoid(logit(P_prior_t) + A_t) ``` Regime decay: | Regime | phi_regime | |---|---:| | panic | 0.35 | | trend_following | 0.80 | | mean_reversion | 0.55 | | uncertainty | 0.50 | This makes trend-following evidence persist and panic evidence decay quickly. --- ### 11.2 Projected alpha For horizon `h`: ```text A_projected_h = phi_regime^h * A_t + expected_known_catalyst_LLR_h P_up_projected_h = sigmoid(logit(P_prior_h) + A_projected_h) ``` Projected strength: ```text strength_projected_h = abs(2 * P_up_projected_h - 1) ``` Divergence flag: ```text divergence = sign(P_up_projected_h - 0.5) != sign(P_up_t - 0.5) ``` --- ## 12. Return Distribution and Expected Value Gate The existing EV formula uses strength as if it were payoff size. Replace it with a return distribution. ### 12.1 Horizon volatility ```text sigma_h = realized_vol_20d * sqrt(horizon_days / 252) ``` For intraday, use intraday realized volatility when available. --- ### 12.2 Expected return ```text edge_z = tanh(A_projected_h / z_scale) mu_h = edge_z * confidence * sigma_h ``` Default: ```text z_scale = 3.0 ``` --- ### 12.3 Trading costs ```text cost = spread_cost + slippage_estimate + commission_estimate ``` --- ### 12.4 Long EV Simple approximation: ```text EV_long = mu_h - cost ``` Risk-adjusted approximation: ```text EV_long_risk_adjusted = mu_h - cost - lambda_tail * CVaR_5_loss ``` Default: ```text lambda_tail = 0.10 ``` For a sell/exit decision on an existing long: ```text EV_hold = mu_h - expected_drawdown_penalty - cost_to_exit_later EV_exit = -exit_cost sell_if EV_exit > EV_hold ``` --- ### 12.5 EV gate ```text min_edge = max(0.0025, 0.25 * cost, regime_min_edge) ``` Suggested regime minimum edge: | Regime | min_edge | |---|---:| | panic | 0.0100 | | trend_following | 0.0035 | | mean_reversion | 0.0050 | | uncertainty | 0.0075 | Trade-eligible only when: ```text EV_long > min_edge confidence >= confidence_min contradiction <= contradiction_max n_eff_total >= evidence_min ``` --- ## 13. Recommendation Mapping v3 ### 13.1 Eligibility gates ```text eligible = ( data_quality_score >= 0.50 and confidence >= confidence_min_regime and n_eff_total >= 2.0 and contradiction <= contradiction_max_regime and abs(2 * P_up - 1) >= strength_min_regime ) ``` Defaults: | Regime | confidence_min | contradiction_max | strength_min | |---|---:|---:|---:| | panic | 0.70 | 0.25 | 0.36 | | trend_following | 0.55 | 0.40 | 0.20 | | mean_reversion | 0.60 | 0.35 | 0.26 | | uncertainty | 0.65 | 0.30 | 0.30 | --- ### 13.2 Action mapping ```text if not eligible: action = WATCH or INFORMATIONAL elif P_up >= bullish_threshold and EV_long > min_edge: action = BUY elif existing_position and EV_exit > EV_hold: action = SELL elif existing_position: action = HOLD else: action = WATCH ``` Do not emit SELL as a short recommendation unless shorting is explicitly enabled in config. --- ### 13.3 Mode escalation ```text live_eligible = ( action in {BUY, SELL} and confidence >= 0.75 and contradiction <= 0.20 and n_eff_total >= 5 and EV_long > 2 * min_edge and risk_engine_passed ) paper_eligible = ( action in {BUY, SELL} and confidence >= 0.60 and EV_long > min_edge and risk_engine_passed ) informational = otherwise ``` --- ## 14. Position Sizing v3 Replace the two separate sizing formulas with one core sizing equation plus hard risk caps. ### 14.1 Stop distance ```text stop_distance_pct = max( ATR_pct * ATR_multiplier_regime, sigma_h * z_stop, min_stop_pct ) ``` Defaults: ```text z_stop = 1.25 min_stop_pct = 0.005 ``` Regime ATR multiplier: | Regime | ATR multiplier | |---|---:| | panic | 2.5 | | trend_following | 1.8 | | mean_reversion | 1.4 | | uncertainty | 2.0 | --- ### 14.2 Win probability ```text p_win = P(R_h > 0) ``` Approximation: ```text p_win = P_up ``` Better version when distribution is available: ```text p_win = 1 - CDF_return_distribution(0) ``` --- ### 14.3 Fractional Kelly ```text b = take_profit_distance_pct / stop_distance_pct f_kelly = (p_win * b - (1 - p_win)) / b ``` Conservative final sizing: ```text f_raw = max(0, f_kelly) * kelly_fraction * confidence * data_quality_score * (1 - contradiction) ``` Defaults: ```text kelly_fraction = 0.25 ``` --- ### 14.4 Hard caps ```text portfolio_pct = clamp(f_raw, 0, max_position_pct) portfolio_pct = min(portfolio_pct, available_sector_capacity_pct) portfolio_pct = min(portfolio_pct, available_correlation_capacity_pct) portfolio_pct = min(portfolio_pct, available_heat_capacity_pct) ``` If `portfolio_pct < min_trade_pct`, downgrade to WATCH. --- ## 15. Correlation and Portfolio Heat v3 ### 15.1 Position correlation penalty ```text rho_portfolio = weighted_mean(abs(rho(symbol, held_symbol)), weight=position_weight) ``` ```text correlation_capacity = clamp(1 - ((rho_portfolio - 0.40) / 0.40), 0, 1) ``` Reject if: ```text rho_portfolio > 0.80 ``` --- ### 15.2 Heat Risk dollars, not position dollars: ```text risk_dollars_new = position_value * stop_distance_pct portfolio_heat_new = current_open_risk + risk_dollars_new ``` Reject if: ```text portfolio_heat_new > max_portfolio_heat * portfolio_value ``` This is better than `dollar_amount * atr_multiplier * 0.02`, because it measures actual stop-defined loss exposure. --- ## 16. Stop Loss and Take Profit v3 For a long position: ```text stop_loss = entry_price * (1 - stop_distance_pct) take_profit = entry_price * (1 + b * stop_distance_pct) ``` Dynamic reward ratio: ```text b = clamp(1.2 + 2.0 * confidence + 1.0 * strength - contradiction, 1.2, 3.0) ``` Trailing activation: ```text activate_trailing = unrealized_gain_pct >= 0.50 * take_profit_distance_pct ``` Trailing stop: ```text trailing_stop = max(existing_stop, current_price * (1 - trailing_distance_pct)) ``` ```text trailing_distance_pct = max(ATR_pct * trailing_ATR_mult, sigma_h * 0.75) ``` --- ## 17. Data Quality v3 Replace additive quality with fail-closed multiplicative quality. ```text Q_parse = 1 - extraction_failure_rate Q_conf = weighted_mean(extraction_conf_i, weight=impact_i) Q_fresh = exp(-age_newest_hours / freshness_tau) Q_coverage = 1 - exp(-N_valid / coverage_scale) Q_diversity = min(1, log2(1 + N_source_types) / log2(4)) ``` Defaults: ```text freshness_tau = 168h coverage_scale = 5 ``` Final: ```text data_quality_score = clamp( Q_parse * sqrt(Q_conf) * Q_fresh * Q_coverage * Q_diversity, 0, 1 ) ``` Suppression: ```text if data_quality_score < 0.50: informational if N_valid < 2: informational if Q_parse < 0.50: informational if company_evidence_abs == 0 and macro_evidence_abs > 0: informational unless macro_only_enabled if company_evidence_abs == 0 and competitive_evidence_abs > 0: informational ``` --- ## 18. Risk Tier Auto-Adjustment v3 Replace raw win rate with risk-adjusted performance. Track: ```text win_rate_30d profit_factor_30d = gross_profit / abs(gross_loss) max_drawdown_30d calibration_error = mean(abs(predicted_probability - realized_outcome)) realized_sharpe_30d ``` Downgrade one tier if any: ```text profit_factor_30d < 1.0 max_drawdown_30d > 0.12 calibration_error > 0.20 realized_sharpe_30d < 0 ``` Upgrade one tier only if all: ```text profit_factor_30d > 1.35 max_drawdown_30d < 0.05 calibration_error < 0.12 reserve_pool > 0.20 N_trades_30d >= 20 ``` --- ## 19. Mapping to Existing Services | Existing service | Keep | Replace / add | |---|---|---| | `services/aggregation/scoring.py` | input normalization, recency windows | replace `W_combined` with `q_i`, `p_correct_i`, `LLR_i` | | `services/aggregation/contradiction.py` | contradiction output field | replace minority-weight formula with LLR entropy contradiction | | `services/aggregation/bayesian.py` | feature-flagged probabilistic path | make this the primary posterior engine | | `services/aggregation/regime.py` | EMA/volatility regime concept | stop boosting evidence in panic; use regime multipliers and thresholds | | `services/aggregation/interpolation.py` | macro/company overlap model | normalize noisy-OR exposure; emit macro LLR as normal evidence | | `services/aggregation/signal_propagation.py` | competitor graph | add correlation shrinkage and cap propagated LLR | | `services/aggregation/projection.py` | projected trend API | replace momentum delta with posterior state `A_t` | | `services/recommendation/suppression.py` | suppression layer | replace additive quality with multiplicative fail-closed quality | | `services/recommendation/eligibility.py` | mode/action output | use posterior, confidence, contradiction, and EV gates | | `services/trading/position_sizer.py` | risk caps, sector/correlation checks | replace allocation formula with fractional Kelly under caps | | `services/trading/stop_loss_manager.py` | ATR stops/trailing stops | make stops volatility/regime/distribution aware | | `services/risk/engine.py` | hard limits | calculate heat from stop-defined risk dollars | | `services/trading/risk_tier_controller.py` | tier state machine | use profit factor, drawdown, calibration error, Sharpe | --- ## 20. Implementation Order Recommended order so the system stays testable: 1. Add `EvidenceUnit` and LLR conversion behind a feature flag. 2. Add evidence clustering and `n_eff`. 3. Replace `S_avg` trend assembly with posterior `P_up`. 4. Replace contradiction with LLR entropy contradiction. 5. Replace confidence formula. 6. Convert macro and competitive layers to emit LLR-compatible evidence. 7. Replace EV gate with expected-return distribution. 8. Replace sizing with fractional Kelly under existing risk caps. 9. Replace portfolio heat with stop-defined risk dollars. 10. Retire heuristic scoring to explainability-only mode. --- ## 21. Output Contract Aggregation output should expose both machine decision fields and explainability fields. ```json { "symbol": "string", "horizon": "7d", "regime": "trend_following", "posterior": { "p_up": 0.64, "p_down": 0.36, "log_odds": 0.58, "strength": 0.28, "confidence": 0.61, "contradiction": 0.18, "n_eff": 5.7, "data_quality": 0.82 }, "return_model": { "mu_h": 0.012, "sigma_h": 0.041, "ev_long": 0.007, "min_edge": 0.0035 }, "recommendation": { "action": "BUY", "mode": "paper_eligible", "reason": "Positive posterior, sufficient effective evidence, EV clears threshold, risk engine passed." }, "explainability": { "top_positive_clusters": [], "top_negative_clusters": [], "suppression_reasons": [], "risk_adjustments": [] } } ``` --- ## 22. The New Core in One Block ```text q_i = q_ext * q_source * source_cred_i * q_recency * q_uniqueness_i p_correct_i = clamp( 0.50 + 0.35 * q_i * impact_i * sentiment_strength_i, 0.501, 0.85 ) LLR_i = direction_i * log(p_correct_i / (1 - p_correct_i)) n_eff_c = (sum_i w_i)^2 / (sum_i w_i^2 + 2 * sum_{i= 0.50 and confidence >= regime_confidence_min and contradiction <= regime_contradiction_max and n_eff_total >= 2.0 and EV_long > min_edge f_kelly = (p_win * b - (1 - p_win)) / b portfolio_pct = clamp( max(0, f_kelly) * 0.25 * confidence * data_quality * (1 - contradiction), 0, max_position_pct ) ```