feat: implement dual-pipeline signal engine service
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
New service at services/signal_engine/ implementing concurrent heuristic (deterministic scoring) and probabilistic (Bayesian inference) pipelines that evaluate technical signals across 6 timeframes (M30-M) and produce independent BUY/WATCH/SKIP verdicts per ticker per evaluation tick. Components: - Input Normalizer: multi-source data assembly with sentinel fallbacks - Signal Library: Fibonacci, MA Stack, RSI, Cup & Handle, Elliott Wave - Multi-Timeframe Confluence Engine: weighted scoring with D/W/M anchors - Hard Filter Engine: macro_bias, valuation, earnings proximity gating - Heuristic Pipeline: S_total scoring with confidence-gated verdicts - Probabilistic Pipeline: Bayesian log-odds with regime priors, entropy gating, EV_R calculation, and signal correlation penalty - Exit Engine: stop-loss, targets, trailing ATR-based stops - Delta Analyzer: pipeline agreement tracking with rolling Redis metrics - Output Formatter: SignalOutput contract + Recommendation schema mapping - Worker orchestrator: concurrent pipelines with failure isolation - Main entry point: queue polling with fail-safe config loading Infrastructure: - Migration 039: signal_engine_outputs table with 3 indexes - Helm chart: signalEngine service entry (processing tier) - Redis key: QUEUE_SIGNAL_ENGINE constant Tests: 390 tests (unit + property-based) covering all components Config: dual_pipeline_enabled=false by default (safe rollout)
This commit is contained in:
@@ -0,0 +1,271 @@
|
||||
"""Pydantic data models for the dual-pipeline signal engine.
|
||||
|
||||
Defines all input, intermediate, and output models consumed by the heuristic
|
||||
pipeline, probabilistic pipeline, delta analyzer, exit engine, and output
|
||||
formatter. Every model is a Pydantic ``BaseModel`` subclass with field-level
|
||||
constraints where applicable.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
from enum import Enum
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Market data
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class OHLCVBar(BaseModel):
|
||||
"""Single OHLCV bar for a timeframe."""
|
||||
|
||||
timestamp: datetime
|
||||
open: float
|
||||
high: float
|
||||
low: float
|
||||
close: float
|
||||
volume: float
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Position state (for exit engine)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class OpenPositionState(BaseModel):
|
||||
"""Snapshot of an open position for exit evaluation."""
|
||||
|
||||
position_id: str
|
||||
ticker: str
|
||||
entry_price: float
|
||||
current_price: float
|
||||
stop_loss: float
|
||||
target_1: float
|
||||
target_2: float
|
||||
trailing_stop: float | None = None
|
||||
partial_exit_done: bool = False
|
||||
atr: float | None = None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Normalized input consumed by both pipelines
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class NormalizedInput(BaseModel):
|
||||
"""Unified input structure consumed by both pipelines."""
|
||||
|
||||
ticker: str
|
||||
evaluated_at: datetime
|
||||
|
||||
# Multi-timeframe OHLCV bars keyed by timeframe label
|
||||
bars: dict[str, list[OHLCVBar]] # {"M30": [...], "H1": [...], ...}
|
||||
|
||||
# Fundamental / macro context
|
||||
valuation_score: float | None = None # [0.0, 1.0]
|
||||
earnings_proximity_days: int | None = None
|
||||
macro_bias: float = 0.0 # [-1.0, 1.0]
|
||||
|
||||
# Open positions for exit evaluation
|
||||
open_positions: list[OpenPositionState] = Field(default_factory=list)
|
||||
|
||||
# Price series helpers (used by probabilistic pipeline)
|
||||
closing_prices: list[float] = Field(default_factory=list)
|
||||
returns: list[float] = Field(default_factory=list)
|
||||
current_price: float | None = None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Signal evaluation primitives
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class SignalDirection(str, Enum):
|
||||
BULLISH = "bullish"
|
||||
BEARISH = "bearish"
|
||||
NEUTRAL = "neutral"
|
||||
|
||||
|
||||
class SignalResult(BaseModel):
|
||||
"""Output from a single signal evaluator on a single timeframe."""
|
||||
|
||||
signal_type: str
|
||||
timeframe: str
|
||||
strength: float = Field(ge=0.0, le=1.0)
|
||||
direction: SignalDirection
|
||||
confidence: float = Field(ge=0.0, le=1.0)
|
||||
metadata: dict = Field(default_factory=dict)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Multi-timeframe confluence
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ConfluenceSignal(BaseModel):
|
||||
"""A signal that passed multi-timeframe confluence filtering."""
|
||||
|
||||
signal_type: str
|
||||
direction: SignalDirection
|
||||
confluence_score: float
|
||||
active_timeframes: list[str]
|
||||
per_timeframe: dict[str, float]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Pipeline verdicts
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class Verdict(str, Enum):
|
||||
BUY = "BUY"
|
||||
WATCH = "WATCH"
|
||||
SKIP = "SKIP"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Heuristic pipeline output
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class HeuristicResult(BaseModel):
|
||||
"""Output from the heuristic (deterministic) pipeline."""
|
||||
|
||||
verdict: Verdict
|
||||
confidence: float = Field(ge=0.0, le=1.0)
|
||||
s_total: float
|
||||
s_company: float
|
||||
s_macro: float
|
||||
s_competitive: float
|
||||
signal_weights: list[dict] = Field(default_factory=list)
|
||||
reasoning: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Probabilistic pipeline output
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class LikelihoodRatio(BaseModel):
|
||||
"""A single signal's likelihood ratio for Bayesian updating."""
|
||||
|
||||
signal_type: str
|
||||
cluster: str
|
||||
lr: float
|
||||
log_lr: float
|
||||
penalized_log_lr: float
|
||||
hit_rate: float
|
||||
strength: float
|
||||
|
||||
|
||||
class ProbabilisticResult(BaseModel):
|
||||
"""Output from the probabilistic (Bayesian) pipeline."""
|
||||
|
||||
verdict: Verdict
|
||||
p_up: float = Field(ge=0.0, le=1.0)
|
||||
entropy: float = Field(ge=0.0, le=1.0)
|
||||
ev_r: float
|
||||
prior: float
|
||||
posterior: float
|
||||
likelihood_ratios: list[LikelihoodRatio] = Field(default_factory=list)
|
||||
regime: str
|
||||
reasoning: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Delta analyzer output
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class DeltaResult(BaseModel):
|
||||
"""Output from the delta analyzer comparing both pipelines."""
|
||||
|
||||
agreement: bool
|
||||
confidence_delta: float
|
||||
heuristic_verdict: str
|
||||
probabilistic_verdict: str
|
||||
disagreement_reasons: list[str] = Field(default_factory=list)
|
||||
rolling_agreement_rate: float | None = None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Exit engine
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ExitType(str, Enum):
|
||||
EXIT_HALF = "EXIT_HALF"
|
||||
EXIT_FULL = "EXIT_FULL"
|
||||
|
||||
|
||||
class ExitSignal(BaseModel):
|
||||
"""An exit signal for an open position."""
|
||||
|
||||
position_id: str
|
||||
ticker: str
|
||||
exit_type: ExitType
|
||||
reason: str
|
||||
price: float
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Trade plan
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TradePlan(BaseModel):
|
||||
"""Optional trade plan attached to a BUY signal."""
|
||||
|
||||
entry_price: float
|
||||
stop_loss: float
|
||||
target_1: float
|
||||
target_2: float
|
||||
position_size_pct: float = Field(ge=0.0, le=1.0)
|
||||
max_loss_pct: float = Field(ge=0.0, le=1.0)
|
||||
dual_confirmed: bool = False
|
||||
probabilistic_only: bool = False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Structured output contract
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class SignalOutput(BaseModel):
|
||||
"""The structured output contract consumed by the trading engine and audit systems."""
|
||||
|
||||
output_id: str = Field(default_factory=lambda: str(uuid.uuid4()))
|
||||
ticker: str
|
||||
timestamp: datetime
|
||||
price: float
|
||||
|
||||
# Heuristic pipeline section
|
||||
heuristic_verdict: str
|
||||
heuristic_confidence: float
|
||||
heuristic_s_total: float
|
||||
|
||||
# Probabilistic pipeline section
|
||||
probabilistic_verdict: str
|
||||
probabilistic_p_up: float
|
||||
probabilistic_entropy: float
|
||||
probabilistic_ev_r: float
|
||||
|
||||
# Delta analysis section
|
||||
delta_agreement: bool
|
||||
delta_confidence_delta: float
|
||||
delta_reasons: list[str] = Field(default_factory=list)
|
||||
|
||||
# Optional trade plan and exit signals
|
||||
trade_plan: TradePlan | None = None
|
||||
exit_signals: list[ExitSignal] = Field(default_factory=list)
|
||||
|
||||
# Detail payloads for audit / dashboard
|
||||
heuristic_detail: dict = Field(default_factory=dict)
|
||||
probabilistic_detail: dict = Field(default_factory=dict)
|
||||
|
||||
# Pipeline mode metadata
|
||||
pipeline_mode: str = "dual_pipeline"
|
||||
shadow_mode: bool = False
|
||||
Reference in New Issue
Block a user