feat: implement dual-pipeline signal engine service
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
New service at services/signal_engine/ implementing concurrent heuristic (deterministic scoring) and probabilistic (Bayesian inference) pipelines that evaluate technical signals across 6 timeframes (M30-M) and produce independent BUY/WATCH/SKIP verdicts per ticker per evaluation tick. Components: - Input Normalizer: multi-source data assembly with sentinel fallbacks - Signal Library: Fibonacci, MA Stack, RSI, Cup & Handle, Elliott Wave - Multi-Timeframe Confluence Engine: weighted scoring with D/W/M anchors - Hard Filter Engine: macro_bias, valuation, earnings proximity gating - Heuristic Pipeline: S_total scoring with confidence-gated verdicts - Probabilistic Pipeline: Bayesian log-odds with regime priors, entropy gating, EV_R calculation, and signal correlation penalty - Exit Engine: stop-loss, targets, trailing ATR-based stops - Delta Analyzer: pipeline agreement tracking with rolling Redis metrics - Output Formatter: SignalOutput contract + Recommendation schema mapping - Worker orchestrator: concurrent pipelines with failure isolation - Main entry point: queue polling with fail-safe config loading Infrastructure: - Migration 039: signal_engine_outputs table with 3 indexes - Helm chart: signalEngine service entry (processing tier) - Redis key: QUEUE_SIGNAL_ENGINE constant Tests: 390 tests (unit + property-based) covering all components Config: dual_pipeline_enabled=false by default (safe rollout)
This commit is contained in:
@@ -0,0 +1,107 @@
|
||||
"""Database persistence for signal engine outputs.
|
||||
|
||||
Persists ``SignalOutput`` instances to the ``signal_engine_outputs`` table.
|
||||
Persistence failures are logged and swallowed — they never block signal
|
||||
emission to the trading queue.
|
||||
|
||||
Requirements: 15.1, 15.4
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
|
||||
import asyncpg
|
||||
|
||||
from services.signal_engine.models import SignalOutput
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# INSERT statement for the signal_engine_outputs table.
|
||||
_INSERT_SQL = """
|
||||
INSERT INTO signal_engine_outputs (
|
||||
id,
|
||||
ticker,
|
||||
evaluated_at,
|
||||
price,
|
||||
heuristic_verdict,
|
||||
heuristic_confidence,
|
||||
heuristic_s_total,
|
||||
probabilistic_verdict,
|
||||
probabilistic_p_up,
|
||||
probabilistic_entropy,
|
||||
probabilistic_ev_r,
|
||||
delta_agreement,
|
||||
delta_confidence_delta,
|
||||
delta_reasons,
|
||||
trade_plan,
|
||||
full_output,
|
||||
exit_signals,
|
||||
pipeline_mode,
|
||||
shadow_mode
|
||||
) VALUES (
|
||||
$1, $2, $3, $4, $5, $6, $7, $8, $9, $10,
|
||||
$11, $12, $13, $14, $15, $16, $17, $18, $19
|
||||
)
|
||||
"""
|
||||
|
||||
|
||||
async def persist_signal_output(
|
||||
pool: asyncpg.Pool,
|
||||
output: SignalOutput,
|
||||
) -> None:
|
||||
"""Persist a SignalOutput to the signal_engine_outputs table.
|
||||
|
||||
Logs and continues on database errors (non-blocking).
|
||||
|
||||
Requirements: 15.1, 15.4
|
||||
"""
|
||||
try:
|
||||
trade_plan_json: str | None = None
|
||||
if output.trade_plan is not None:
|
||||
trade_plan_json = json.dumps(output.trade_plan.model_dump())
|
||||
|
||||
exit_signals_json = json.dumps(
|
||||
[e.model_dump() for e in output.exit_signals]
|
||||
)
|
||||
|
||||
delta_reasons_json = json.dumps(output.delta_reasons)
|
||||
|
||||
full_output_json = output.model_dump_json()
|
||||
|
||||
await pool.execute(
|
||||
_INSERT_SQL,
|
||||
output.output_id, # $1 id
|
||||
output.ticker, # $2 ticker
|
||||
output.timestamp, # $3 evaluated_at
|
||||
output.price, # $4 price
|
||||
output.heuristic_verdict, # $5 heuristic_verdict
|
||||
output.heuristic_confidence, # $6 heuristic_confidence
|
||||
output.heuristic_s_total, # $7 heuristic_s_total
|
||||
output.probabilistic_verdict, # $8 probabilistic_verdict
|
||||
output.probabilistic_p_up, # $9 probabilistic_p_up
|
||||
output.probabilistic_entropy, # $10 probabilistic_entropy
|
||||
output.probabilistic_ev_r, # $11 probabilistic_ev_r
|
||||
output.delta_agreement, # $12 delta_agreement
|
||||
output.delta_confidence_delta, # $13 delta_confidence_delta
|
||||
delta_reasons_json, # $14 delta_reasons (JSONB)
|
||||
trade_plan_json, # $15 trade_plan (JSONB, nullable)
|
||||
full_output_json, # $16 full_output (JSONB)
|
||||
exit_signals_json, # $17 exit_signals (JSONB)
|
||||
output.pipeline_mode, # $18 pipeline_mode
|
||||
output.shadow_mode, # $19 shadow_mode
|
||||
)
|
||||
|
||||
logger.debug(
|
||||
"Persisted signal output %s for %s",
|
||||
output.output_id,
|
||||
output.ticker,
|
||||
)
|
||||
except Exception:
|
||||
logger.error(
|
||||
"Failed to persist signal output %s for %s — continuing",
|
||||
output.output_id,
|
||||
output.ticker,
|
||||
exc_info=True,
|
||||
)
|
||||
Reference in New Issue
Block a user