f468e30af0
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
New service at services/signal_engine/ implementing concurrent heuristic (deterministic scoring) and probabilistic (Bayesian inference) pipelines that evaluate technical signals across 6 timeframes (M30-M) and produce independent BUY/WATCH/SKIP verdicts per ticker per evaluation tick. Components: - Input Normalizer: multi-source data assembly with sentinel fallbacks - Signal Library: Fibonacci, MA Stack, RSI, Cup & Handle, Elliott Wave - Multi-Timeframe Confluence Engine: weighted scoring with D/W/M anchors - Hard Filter Engine: macro_bias, valuation, earnings proximity gating - Heuristic Pipeline: S_total scoring with confidence-gated verdicts - Probabilistic Pipeline: Bayesian log-odds with regime priors, entropy gating, EV_R calculation, and signal correlation penalty - Exit Engine: stop-loss, targets, trailing ATR-based stops - Delta Analyzer: pipeline agreement tracking with rolling Redis metrics - Output Formatter: SignalOutput contract + Recommendation schema mapping - Worker orchestrator: concurrent pipelines with failure isolation - Main entry point: queue polling with fail-safe config loading Infrastructure: - Migration 039: signal_engine_outputs table with 3 indexes - Helm chart: signalEngine service entry (processing tier) - Redis key: QUEUE_SIGNAL_ENGINE constant Tests: 390 tests (unit + property-based) covering all components Config: dual_pipeline_enabled=false by default (safe rollout)
300 lines
10 KiB
Python
300 lines
10 KiB
Python
"""Heuristic Pipeline (Pipeline A) — Deterministic scoring and verdict.
|
|
|
|
Computes ``S_total = S_company + S_macro + S_competitive`` from confluence-
|
|
filtered signals and produces a confidence-gated BUY / WATCH / SKIP verdict.
|
|
|
|
The pipeline reuses the existing ``compute_signal_weight`` infrastructure
|
|
from ``services.aggregation.scoring`` for signal weighting and follows the
|
|
three-layer signal aggregation model (company, macro, competitive).
|
|
|
|
Requirements: 5.1, 5.2, 5.3, 5.4, 5.5, 5.6, 5.7
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import logging
|
|
|
|
from services.signal_engine.config import HeuristicConfig
|
|
from services.signal_engine.models import (
|
|
ConfluenceSignal,
|
|
HeuristicResult,
|
|
NormalizedInput,
|
|
SignalDirection,
|
|
Verdict,
|
|
)
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Signal classification — which confluence signals belong to which layer
|
|
# ---------------------------------------------------------------------------
|
|
|
|
# Company-level technical signals (Layer 1)
|
|
COMPANY_SIGNAL_TYPES: frozenset[str] = frozenset({
|
|
"fibonacci",
|
|
"ma_stack",
|
|
"rsi",
|
|
"cup_handle",
|
|
"elliott_wave",
|
|
})
|
|
|
|
# Competitive signals (Layer 3) — future expansion
|
|
COMPETITIVE_SIGNAL_TYPES: frozenset[str] = frozenset()
|
|
|
|
# Macro weight applied to macro_bias to produce S_macro
|
|
_MACRO_WEIGHT: float = 0.5
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Score computation helpers
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _compute_s_company(confluence_signals: list[ConfluenceSignal]) -> tuple[float, list[dict]]:
|
|
"""Sum confluence scores for company-level signals.
|
|
|
|
Returns the total S_company score and a list of per-signal weight
|
|
breakdowns for audit.
|
|
"""
|
|
s_company = 0.0
|
|
weights: list[dict] = []
|
|
|
|
for sig in confluence_signals:
|
|
if sig.signal_type in COMPANY_SIGNAL_TYPES:
|
|
# Direction-aware: bullish contributes positively, bearish negatively
|
|
direction_sign = _direction_sign(sig.direction)
|
|
contribution = sig.confluence_score * direction_sign
|
|
s_company += contribution
|
|
weights.append({
|
|
"signal_type": sig.signal_type,
|
|
"layer": "company",
|
|
"confluence_score": sig.confluence_score,
|
|
"direction": sig.direction.value,
|
|
"contribution": contribution,
|
|
"active_timeframes": sig.active_timeframes,
|
|
})
|
|
|
|
return s_company, weights
|
|
|
|
|
|
def _compute_s_macro(normalized: NormalizedInput) -> float:
|
|
"""Compute macro score from macro_bias.
|
|
|
|
S_macro = macro_bias * weight, where macro_bias is in [-1.0, 1.0].
|
|
A positive macro_bias contributes positively; negative contributes
|
|
negatively.
|
|
"""
|
|
return normalized.macro_bias * _MACRO_WEIGHT
|
|
|
|
|
|
def _compute_s_competitive(confluence_signals: list[ConfluenceSignal]) -> float:
|
|
"""Sum confluence scores for competitive-layer signals.
|
|
|
|
Currently returns 0.0 as no competitive signal types are defined in
|
|
the signal library. This is a placeholder for future expansion.
|
|
"""
|
|
s_competitive = 0.0
|
|
for sig in confluence_signals:
|
|
if sig.signal_type in COMPETITIVE_SIGNAL_TYPES:
|
|
direction_sign = _direction_sign(sig.direction)
|
|
s_competitive += sig.confluence_score * direction_sign
|
|
return s_competitive
|
|
|
|
|
|
def _direction_sign(direction: SignalDirection) -> float:
|
|
"""Map signal direction to a numeric sign."""
|
|
if direction == SignalDirection.BULLISH:
|
|
return 1.0
|
|
if direction == SignalDirection.BEARISH:
|
|
return -1.0
|
|
return 0.0
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Confidence computation
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _compute_confidence(
|
|
confluence_signals: list[ConfluenceSignal],
|
|
) -> float:
|
|
"""Compute pipeline confidence from confluence signals.
|
|
|
|
Confidence is derived from:
|
|
1. **Base confidence** — average signal strength across all confluence
|
|
signals (mean of confluence_score values).
|
|
2. **Source count boost** — more active signals increase confidence
|
|
(diminishing returns, capped contribution).
|
|
3. **Signal agreement boost** — if all signals point in the same
|
|
direction, confidence is boosted.
|
|
4. **Contradiction penalty** — if signals disagree on direction,
|
|
confidence is penalised.
|
|
|
|
Returns a value clamped to [0.0, 1.0].
|
|
"""
|
|
if not confluence_signals:
|
|
return 0.0
|
|
|
|
# 1. Base confidence: average confluence score (already weighted by
|
|
# timeframe importance)
|
|
total_score = sum(s.confluence_score for s in confluence_signals)
|
|
base_confidence = total_score / len(confluence_signals)
|
|
|
|
# 2. Source count factor: more signals → higher confidence, with
|
|
# diminishing returns. 1 signal → 0.6, 2 → 0.75, 3 → 0.85,
|
|
# 4 → 0.90, 5+ → 0.95 (asymptotic).
|
|
n = len(confluence_signals)
|
|
source_factor = 1.0 - (0.4 / n) # approaches 1.0 as n grows
|
|
|
|
# 3. Signal agreement / contradiction
|
|
directions = [s.direction for s in confluence_signals]
|
|
bullish_count = sum(1 for d in directions if d == SignalDirection.BULLISH)
|
|
bearish_count = sum(1 for d in directions if d == SignalDirection.BEARISH)
|
|
|
|
if n == 1:
|
|
agreement_factor = 1.0
|
|
elif bullish_count == n or bearish_count == n:
|
|
# Perfect agreement — boost
|
|
agreement_factor = 1.15
|
|
elif bullish_count > 0 and bearish_count > 0:
|
|
# Contradiction — penalty proportional to minority fraction
|
|
minority = min(bullish_count, bearish_count)
|
|
contradiction_ratio = minority / n
|
|
agreement_factor = 1.0 - (0.3 * contradiction_ratio)
|
|
else:
|
|
# Mix of directional and neutral — mild boost
|
|
agreement_factor = 1.05
|
|
|
|
confidence = base_confidence * source_factor * agreement_factor
|
|
return max(0.0, min(confidence, 1.0))
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Verdict logic
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _determine_verdict(
|
|
confidence: float,
|
|
s_total: float,
|
|
normalized: NormalizedInput,
|
|
config: HeuristicConfig,
|
|
) -> tuple[Verdict, list[str]]:
|
|
"""Apply threshold logic to determine BUY / WATCH / SKIP verdict.
|
|
|
|
Returns the verdict and a list of reasoning strings explaining the
|
|
decision.
|
|
"""
|
|
reasoning: list[str] = []
|
|
|
|
valuation_score = normalized.valuation_score if normalized.valuation_score is not None else 0.0
|
|
earnings_days = normalized.earnings_proximity_days if normalized.earnings_proximity_days is not None else 0
|
|
|
|
# --- Check BUY conditions ---
|
|
buy_conditions = {
|
|
"confidence": confidence >= config.buy_confidence,
|
|
"s_total": s_total >= config.buy_s_total,
|
|
"valuation": valuation_score >= config.buy_valuation_min,
|
|
"macro_bias": normalized.macro_bias > config.macro_bias_threshold,
|
|
"earnings_proximity": earnings_days > config.earnings_days_threshold,
|
|
}
|
|
|
|
all_buy_met = all(buy_conditions.values())
|
|
|
|
if all_buy_met:
|
|
reasoning.append(
|
|
f"BUY: all conditions met — confidence={confidence:.3f} "
|
|
f"(>= {config.buy_confidence}), S_total={s_total:.3f} "
|
|
f"(>= {config.buy_s_total}), valuation={valuation_score:.2f} "
|
|
f"(>= {config.buy_valuation_min}), macro_bias={normalized.macro_bias:.2f} "
|
|
f"(> {config.macro_bias_threshold}), earnings_days={earnings_days} "
|
|
f"(> {config.earnings_days_threshold})"
|
|
)
|
|
return Verdict.BUY, reasoning
|
|
|
|
# --- Check WATCH conditions ---
|
|
if confidence >= config.watch_confidence:
|
|
# WATCH: confidence is sufficient but not all BUY conditions met
|
|
failed_conditions = [k for k, v in buy_conditions.items() if not v]
|
|
reasoning.append(
|
|
f"WATCH: confidence={confidence:.3f} (>= {config.watch_confidence}) "
|
|
f"but BUY conditions not fully met — failed: {', '.join(failed_conditions)}"
|
|
)
|
|
for cond_name, met in buy_conditions.items():
|
|
if not met:
|
|
reasoning.append(f" - {cond_name} not met")
|
|
return Verdict.WATCH, reasoning
|
|
|
|
# --- SKIP ---
|
|
reasoning.append(
|
|
f"SKIP: confidence={confidence:.3f} < {config.watch_confidence} "
|
|
f"(watch threshold)"
|
|
)
|
|
return Verdict.SKIP, reasoning
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Public API
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def run_heuristic_pipeline(
|
|
normalized: NormalizedInput,
|
|
confluence_signals: list[ConfluenceSignal],
|
|
config: HeuristicConfig,
|
|
) -> HeuristicResult:
|
|
"""Run the deterministic heuristic pipeline.
|
|
|
|
Computes ``S_total = S_company + S_macro + S_competitive`` using the
|
|
existing three-layer signal aggregation model and produces a
|
|
confidence-gated BUY / WATCH / SKIP verdict.
|
|
|
|
Args:
|
|
normalized: The unified input structure for this evaluation tick.
|
|
confluence_signals: Signals that passed multi-timeframe confluence
|
|
filtering.
|
|
config: Heuristic pipeline thresholds.
|
|
|
|
Returns:
|
|
A :class:`HeuristicResult` with verdict, scores, weights, and
|
|
reasoning.
|
|
|
|
Requirements: 5.1, 5.2, 5.3, 5.4, 5.5, 5.6, 5.7
|
|
"""
|
|
# 1. Compute three-layer scores
|
|
s_company, signal_weights = _compute_s_company(confluence_signals)
|
|
s_macro = _compute_s_macro(normalized)
|
|
s_competitive = _compute_s_competitive(confluence_signals)
|
|
s_total = s_company + s_macro + s_competitive
|
|
|
|
# 2. Compute confidence
|
|
confidence = _compute_confidence(confluence_signals)
|
|
|
|
# 3. Determine verdict
|
|
verdict, reasoning = _determine_verdict(confidence, s_total, normalized, config)
|
|
|
|
logger.info(
|
|
"Heuristic pipeline [%s]: verdict=%s confidence=%.3f "
|
|
"S_total=%.3f (company=%.3f macro=%.3f competitive=%.3f) "
|
|
"signals=%d",
|
|
normalized.ticker,
|
|
verdict.value,
|
|
confidence,
|
|
s_total,
|
|
s_company,
|
|
s_macro,
|
|
s_competitive,
|
|
len(confluence_signals),
|
|
)
|
|
|
|
return HeuristicResult(
|
|
verdict=verdict,
|
|
confidence=confidence,
|
|
s_total=s_total,
|
|
s_company=s_company,
|
|
s_macro=s_macro,
|
|
s_competitive=s_competitive,
|
|
signal_weights=signal_weights,
|
|
reasoning=reasoning,
|
|
)
|