f468e30af0
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
New service at services/signal_engine/ implementing concurrent heuristic (deterministic scoring) and probabilistic (Bayesian inference) pipelines that evaluate technical signals across 6 timeframes (M30-M) and produce independent BUY/WATCH/SKIP verdicts per ticker per evaluation tick. Components: - Input Normalizer: multi-source data assembly with sentinel fallbacks - Signal Library: Fibonacci, MA Stack, RSI, Cup & Handle, Elliott Wave - Multi-Timeframe Confluence Engine: weighted scoring with D/W/M anchors - Hard Filter Engine: macro_bias, valuation, earnings proximity gating - Heuristic Pipeline: S_total scoring with confidence-gated verdicts - Probabilistic Pipeline: Bayesian log-odds with regime priors, entropy gating, EV_R calculation, and signal correlation penalty - Exit Engine: stop-loss, targets, trailing ATR-based stops - Delta Analyzer: pipeline agreement tracking with rolling Redis metrics - Output Formatter: SignalOutput contract + Recommendation schema mapping - Worker orchestrator: concurrent pipelines with failure isolation - Main entry point: queue polling with fail-safe config loading Infrastructure: - Migration 039: signal_engine_outputs table with 3 indexes - Helm chart: signalEngine service entry (processing tier) - Redis key: QUEUE_SIGNAL_ENGINE constant Tests: 390 tests (unit + property-based) covering all components Config: dual_pipeline_enabled=false by default (safe rollout)
179 lines
5.9 KiB
Python
179 lines
5.9 KiB
Python
# Feature: dual-pipeline-signal-engine, Properties: Bayesian log-odds, entropy gate, EV_R
|
|
"""Property-based tests for the probabilistic pipeline math.
|
|
|
|
Feature: dual-pipeline-signal-engine
|
|
|
|
Tests three properties from the design specification:
|
|
1. Bayesian log-odds round-trip (Requirement 17.2)
|
|
2. Shannon entropy gate properties (Requirement 17.3)
|
|
3. EV_R monotonicity with P_up (Requirement 17.8)
|
|
|
|
Requirements: 6.3, 6.4, 6.5, 17.2, 17.3, 17.8
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import math
|
|
|
|
from hypothesis import given, settings
|
|
from hypothesis import strategies as st
|
|
|
|
from services.signal_engine.probabilistic import (
|
|
_logit,
|
|
_shannon_entropy,
|
|
_sigmoid,
|
|
)
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Hypothesis strategies
|
|
# ---------------------------------------------------------------------------
|
|
|
|
# Prior probability in (0.01, 0.99) — avoids extreme clamping at boundaries
|
|
_prior_prob = st.floats(
|
|
min_value=0.01, max_value=0.99, allow_nan=False, allow_infinity=False,
|
|
)
|
|
|
|
# Log-likelihood ratio values — bounded to avoid overflow in sigmoid
|
|
_log_lr = st.floats(
|
|
min_value=-10.0, max_value=10.0, allow_nan=False, allow_infinity=False,
|
|
)
|
|
|
|
# List of log-LR values (1 to 10 signals)
|
|
_log_lr_list = st.lists(_log_lr, min_size=1, max_size=10)
|
|
|
|
# Probability in open interval (0, 1) for entropy tests
|
|
_open_prob = st.floats(
|
|
min_value=1e-6, max_value=1.0 - 1e-6, allow_nan=False, allow_infinity=False,
|
|
)
|
|
|
|
# P_up values for EV_R monotonicity — two ordered values
|
|
_p_up_pair = st.tuples(
|
|
st.floats(min_value=0.0, max_value=1.0, allow_nan=False, allow_infinity=False),
|
|
st.floats(min_value=0.0, max_value=1.0, allow_nan=False, allow_infinity=False),
|
|
).filter(lambda pair: pair[0] < pair[1])
|
|
|
|
# Positive expected win in R-units
|
|
_e_win_r = st.floats(
|
|
min_value=0.01, max_value=100.0, allow_nan=False, allow_infinity=False,
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Property 1: Bayesian log-odds round-trip
|
|
# Validates: Requirements 6.3, 17.2
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@given(p_prior=_prior_prob, log_lrs=_log_lr_list)
|
|
@settings(max_examples=100)
|
|
def test_bayesian_log_odds_round_trip(
|
|
p_prior: float, log_lrs: list[float],
|
|
) -> None:
|
|
"""**Validates: Requirements 6.3, 17.2**
|
|
|
|
Converting P_prior to logit, adding Σ log(LR_i), and converting back
|
|
via sigmoid SHALL produce a valid probability in [0, 1].
|
|
|
|
logit(P_post) = logit(P_prior) + Σ log(LR_i)
|
|
P_post = sigmoid(logit(P_post))
|
|
|
|
The sigmoid implementation clamps extreme values to 0.0 / 1.0, so the
|
|
result is in the closed interval [0, 1].
|
|
"""
|
|
logit_prior = _logit(p_prior)
|
|
sum_log_lr = sum(log_lrs)
|
|
logit_posterior = logit_prior + sum_log_lr
|
|
p_posterior = _sigmoid(logit_posterior)
|
|
|
|
# Posterior must be a valid probability in [0, 1]
|
|
assert 0.0 <= p_posterior <= 1.0, (
|
|
f"Posterior {p_posterior} not in [0, 1]. "
|
|
f"P_prior={p_prior}, logit_prior={logit_prior}, "
|
|
f"Σ log_lr={sum_log_lr}, logit_posterior={logit_posterior}"
|
|
)
|
|
|
|
# For non-saturated posteriors, round-trip should hold:
|
|
# sigmoid(logit(p)) ≈ p. At saturation (0.0 or 1.0) the logit
|
|
# clamps, so we only check the interior.
|
|
if 1e-9 < p_posterior < 1.0 - 1e-9:
|
|
round_trip = _sigmoid(_logit(p_posterior))
|
|
assert math.isclose(round_trip, p_posterior, rel_tol=1e-6), (
|
|
f"Round-trip failed: sigmoid(logit({p_posterior})) = {round_trip}"
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Property 2: Shannon entropy gate properties
|
|
# Validates: Requirements 6.4, 17.3
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@given(p=_open_prob)
|
|
@settings(max_examples=100)
|
|
def test_entropy_maximized_at_half(p: float) -> None:
|
|
"""**Validates: Requirements 6.4, 17.3**
|
|
|
|
Shannon entropy H(p) SHALL be maximized at p = 0.5.
|
|
For all p in (0, 1): H(0.5) >= H(p).
|
|
"""
|
|
h_p = _shannon_entropy(p)
|
|
h_half = _shannon_entropy(0.5)
|
|
|
|
assert h_half >= h_p - 1e-12, (
|
|
f"Entropy at 0.5 ({h_half}) should be >= entropy at {p} ({h_p})"
|
|
)
|
|
|
|
|
|
def test_entropy_zero_at_boundaries() -> None:
|
|
"""**Validates: Requirements 6.4, 17.3**
|
|
|
|
Shannon entropy SHALL equal 0.0 at p = 0.0 and p = 1.0.
|
|
"""
|
|
assert _shannon_entropy(0.0) == 0.0, "H(0.0) should be 0.0"
|
|
assert _shannon_entropy(1.0) == 0.0, "H(1.0) should be 0.0"
|
|
|
|
|
|
@given(p=_open_prob)
|
|
@settings(max_examples=100)
|
|
def test_entropy_symmetric_around_half(p: float) -> None:
|
|
"""**Validates: Requirements 6.4, 17.3**
|
|
|
|
Shannon entropy SHALL be symmetric around 0.5: H(p) == H(1 - p).
|
|
"""
|
|
h_p = _shannon_entropy(p)
|
|
h_complement = _shannon_entropy(1.0 - p)
|
|
|
|
assert math.isclose(h_p, h_complement, rel_tol=1e-9), (
|
|
f"Entropy not symmetric: H({p}) = {h_p}, H({1.0 - p}) = {h_complement}"
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Property 3: EV_R monotonicity with P_up
|
|
# Validates: Requirements 6.5, 17.8
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@given(p_pair=_p_up_pair, e_win=_e_win_r)
|
|
@settings(max_examples=100)
|
|
def test_ev_r_monotonically_increasing_with_p_up(
|
|
p_pair: tuple[float, float], e_win: float,
|
|
) -> None:
|
|
"""**Validates: Requirements 6.5, 17.8**
|
|
|
|
EV_R = P_up · E[win_R] - (1 - P_up) · 1.0 SHALL be monotonically
|
|
increasing with P_up for fixed E[win_R] > 0.
|
|
|
|
For p1 < p2 and fixed E[win_R] > 0: EV_R(p2) >= EV_R(p1).
|
|
"""
|
|
p1, p2 = p_pair
|
|
|
|
# Compute EV_R directly using the formula (not _compute_ev_r which
|
|
# derives E[win_R] from confluence signals)
|
|
ev_r_1 = p1 * e_win - (1.0 - p1) * 1.0
|
|
ev_r_2 = p2 * e_win - (1.0 - p2) * 1.0
|
|
|
|
assert ev_r_2 >= ev_r_1 - 1e-12, (
|
|
f"EV_R not monotonic: EV_R(p2={p2}) = {ev_r_2} < "
|
|
f"EV_R(p1={p1}) = {ev_r_1} with E[win_R]={e_win}"
|
|
)
|