feat: implement dual-pipeline signal engine service
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
New service at services/signal_engine/ implementing concurrent heuristic (deterministic scoring) and probabilistic (Bayesian inference) pipelines that evaluate technical signals across 6 timeframes (M30-M) and produce independent BUY/WATCH/SKIP verdicts per ticker per evaluation tick. Components: - Input Normalizer: multi-source data assembly with sentinel fallbacks - Signal Library: Fibonacci, MA Stack, RSI, Cup & Handle, Elliott Wave - Multi-Timeframe Confluence Engine: weighted scoring with D/W/M anchors - Hard Filter Engine: macro_bias, valuation, earnings proximity gating - Heuristic Pipeline: S_total scoring with confidence-gated verdicts - Probabilistic Pipeline: Bayesian log-odds with regime priors, entropy gating, EV_R calculation, and signal correlation penalty - Exit Engine: stop-loss, targets, trailing ATR-based stops - Delta Analyzer: pipeline agreement tracking with rolling Redis metrics - Output Formatter: SignalOutput contract + Recommendation schema mapping - Worker orchestrator: concurrent pipelines with failure isolation - Main entry point: queue polling with fail-safe config loading Infrastructure: - Migration 039: signal_engine_outputs table with 3 indexes - Helm chart: signalEngine service entry (processing tier) - Redis key: QUEUE_SIGNAL_ENGINE constant Tests: 390 tests (unit + property-based) covering all components Config: dual_pipeline_enabled=false by default (safe rollout)
This commit is contained in:
@@ -0,0 +1,148 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Minimal MCP server for OpenAI chat completions.
|
||||
* Accepts ANY model string (gpt-5.2, gpt-5.4, etc.) — no hardcoded enum.
|
||||
* Communicates over stdio using JSON-RPC (MCP protocol).
|
||||
*/
|
||||
|
||||
import { createInterface } from "readline";
|
||||
|
||||
const OPENAI_API_KEY = process.env.OPENAI_API_KEY;
|
||||
if (!OPENAI_API_KEY) {
|
||||
process.stderr.write("ERROR: OPENAI_API_KEY environment variable is required\n");
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const SERVER_INFO = {
|
||||
name: "openai-chat",
|
||||
version: "1.0.0",
|
||||
};
|
||||
|
||||
const TOOLS = [
|
||||
{
|
||||
name: "openai_chat",
|
||||
description:
|
||||
"Send messages to OpenAI chat completions API. Supports all OpenAI models including GPT-5.x series.",
|
||||
inputSchema: {
|
||||
type: "object",
|
||||
properties: {
|
||||
model: {
|
||||
type: "string",
|
||||
description:
|
||||
"OpenAI model name (e.g. gpt-5.2, gpt-5.4, gpt-4o, etc.)",
|
||||
default: "gpt-5.2",
|
||||
},
|
||||
messages: {
|
||||
type: "array",
|
||||
description: "Array of chat messages",
|
||||
items: {
|
||||
type: "object",
|
||||
properties: {
|
||||
role: {
|
||||
type: "string",
|
||||
enum: ["system", "user", "assistant"],
|
||||
},
|
||||
content: { type: "string" },
|
||||
},
|
||||
required: ["role", "content"],
|
||||
},
|
||||
},
|
||||
},
|
||||
required: ["messages"],
|
||||
},
|
||||
},
|
||||
];
|
||||
|
||||
async function callOpenAI(model, messages) {
|
||||
const resp = await fetch("https://api.openai.com/v1/chat/completions", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
Authorization: `Bearer ${OPENAI_API_KEY}`,
|
||||
},
|
||||
body: JSON.stringify({ model, messages }),
|
||||
});
|
||||
|
||||
if (!resp.ok) {
|
||||
const errText = await resp.text();
|
||||
throw new Error(`OpenAI API error ${resp.status}: ${errText}`);
|
||||
}
|
||||
|
||||
const data = await resp.json();
|
||||
return data.choices?.[0]?.message?.content ?? "(no response)";
|
||||
}
|
||||
|
||||
function jsonRpcResponse(id, result) {
|
||||
return JSON.stringify({ jsonrpc: "2.0", id, result });
|
||||
}
|
||||
|
||||
function jsonRpcError(id, code, message) {
|
||||
return JSON.stringify({ jsonrpc: "2.0", id, error: { code, message } });
|
||||
}
|
||||
|
||||
async function handleRequest(req) {
|
||||
const { id, method, params } = req;
|
||||
|
||||
switch (method) {
|
||||
case "initialize":
|
||||
return jsonRpcResponse(id, {
|
||||
protocolVersion: "2024-11-05",
|
||||
capabilities: { tools: {} },
|
||||
serverInfo: SERVER_INFO,
|
||||
});
|
||||
|
||||
case "notifications/initialized":
|
||||
return null; // no response needed for notifications
|
||||
|
||||
case "tools/list":
|
||||
return jsonRpcResponse(id, { tools: TOOLS });
|
||||
|
||||
case "tools/call": {
|
||||
const toolName = params?.name;
|
||||
if (toolName !== "openai_chat") {
|
||||
return jsonRpcError(id, -32602, `Unknown tool: ${toolName}`);
|
||||
}
|
||||
const args = params?.arguments ?? {};
|
||||
const model = args.model || "gpt-5.2";
|
||||
const messages = args.messages || [];
|
||||
|
||||
if (!messages.length) {
|
||||
return jsonRpcError(id, -32602, "messages array is required");
|
||||
}
|
||||
|
||||
try {
|
||||
const content = await callOpenAI(model, messages);
|
||||
return jsonRpcResponse(id, {
|
||||
content: [{ type: "text", text: content }],
|
||||
});
|
||||
} catch (err) {
|
||||
return jsonRpcResponse(id, {
|
||||
content: [{ type: "text", text: `Error: ${err.message}` }],
|
||||
isError: true,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
case "ping":
|
||||
return jsonRpcResponse(id, {});
|
||||
|
||||
default:
|
||||
if (method?.startsWith("notifications/")) return null;
|
||||
return jsonRpcError(id, -32601, `Method not found: ${method}`);
|
||||
}
|
||||
}
|
||||
|
||||
// stdio transport
|
||||
const rl = createInterface({ input: process.stdin });
|
||||
|
||||
rl.on("line", async (line) => {
|
||||
try {
|
||||
const req = JSON.parse(line);
|
||||
const resp = await handleRequest(req);
|
||||
if (resp) {
|
||||
process.stdout.write(resp + "\n");
|
||||
}
|
||||
} catch (err) {
|
||||
process.stderr.write(`Parse error: ${err.message}\n`);
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1 @@
|
||||
{"specId": "d76705a8-fb91-4fce-b59e-c4b3b0dbbd83", "workflowType": "requirements-first", "specType": "feature"}
|
||||
@@ -0,0 +1,723 @@
|
||||
# Design Document — Dual-Pipeline Signal Engine
|
||||
|
||||
## Overview
|
||||
|
||||
The dual-pipeline signal engine is a new service at `services/signal_engine/` that runs as an independent Kubernetes deployment alongside the existing aggregation → recommendation pipeline. It implements a concurrent dual-pipeline architecture where both a heuristic (deterministic scoring) and probabilistic (Bayesian inference) pipeline evaluate the same normalized inputs per ticker per evaluation tick, producing independent BUY/WATCH/SKIP verdicts. A delta analyzer compares the two verdicts, and an output formatter assembles a structured `SignalOutput` contract published to the existing `trading_decisions` Redis queue.
|
||||
|
||||
The engine introduces several new components — Input Normalizer, Signal Library (Fibonacci, MA Stack, RSI, Cup & Handle, Elliott Wave), Multi-Timeframe Engine, Hard Filter Engine, Exit Engine, Delta Analyzer, and Output Formatter — while reusing existing infrastructure: `compute_signal_weight`, `compute_bayesian_posterior`, `classify_regime`, `WeightedSignal`, `BayesianPosterior`, and `RegimeClassification` from `services/aggregation/`.
|
||||
|
||||
The service is toggled via `dual_pipeline_enabled` in the `risk_configs` table (default: false, fail-safe). When disabled, the existing pipeline operates unchanged. When enabled, the signal engine runs alongside the existing pipeline with support for shadow mode (dual-pipeline output persisted but not forwarded to trading).
|
||||
|
||||
### Design Rationale
|
||||
|
||||
- **Separate service, not inline extension**: The signal engine has a fundamentally different evaluation cadence (multi-timeframe technical signals) and data flow (OHLCV bars, not document intelligence). Embedding it in the aggregation worker would couple two distinct concerns.
|
||||
- **Reuse existing math**: The Bayesian posterior, regime classification, and signal weighting functions are battle-tested. The probabilistic pipeline wraps them with regime-based priors and likelihood ratio accumulation rather than reimplementing.
|
||||
- **Concurrent pipelines via asyncio.gather**: Both pipelines share the same `NormalizedInput` reference and run concurrently. If one fails, the other completes normally with the failed pipeline producing a SKIP verdict.
|
||||
- **Signal clustering for correlation penalty**: The Bayesian pipeline groups signals into four clusters (momentum, structure, volatility, fundamentals) and applies exponential decay within each cluster to prevent likelihood ratio stacking inflation from correlated signals.
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
### High-Level Flow
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[Evaluation Tick<br/>Redis queue: signal_engine] --> B[Input Normalizer]
|
||||
B --> C[Hard Filter Engine]
|
||||
C -->|filtered out| D[SKIP verdict for both pipelines]
|
||||
C -->|passed| E[Signal Library]
|
||||
E --> F[Multi-Timeframe Engine]
|
||||
F --> G{asyncio.gather}
|
||||
G --> H[Heuristic Pipeline]
|
||||
G --> I[Probabilistic Pipeline]
|
||||
H --> J[Delta Analyzer]
|
||||
I --> J
|
||||
J --> K[Output Formatter]
|
||||
K --> L[SignalOutput]
|
||||
L --> M[Redis: trading_decisions queue]
|
||||
L --> N[PostgreSQL: signal_engine_outputs]
|
||||
|
||||
subgraph Exit Path
|
||||
B --> O[Exit Engine]
|
||||
O --> K
|
||||
end
|
||||
```
|
||||
|
||||
### Trigger Mechanism
|
||||
|
||||
The signal engine polls a new Redis queue `stonks:queue:signal_engine`. Evaluation ticks are enqueued by the scheduler service after aggregation completes for a ticker. The queue message contains `{"ticker": "AAPL", "triggered_at": "2024-01-15T10:00:00Z"}`.
|
||||
|
||||
### Integration Points
|
||||
|
||||
| Component | Integration | Direction |
|
||||
|---|---|---|
|
||||
| Scheduler | Enqueues ticks to `signal_engine` queue | Scheduler → Signal Engine |
|
||||
| Market data tables | OHLCV bars, closing prices, returns | Signal Engine reads |
|
||||
| `macro_impact_records` | Macro bias computation | Signal Engine reads |
|
||||
| `trend_windows` | Fundamental/valuation context | Signal Engine reads |
|
||||
| `risk_configs` | Feature flags, thresholds | Signal Engine reads |
|
||||
| `classify_regime()` | Regime classification for priors | Signal Engine calls |
|
||||
| `compute_signal_weight()` | Heuristic signal weighting | Signal Engine calls |
|
||||
| `compute_bayesian_posterior()` | Bayesian accumulation | Signal Engine calls |
|
||||
| Redis `trading_decisions` | SignalOutput publication | Signal Engine → Trading Engine |
|
||||
| `signal_engine_outputs` table | Persistence for audit | Signal Engine writes |
|
||||
| Redis rolling agreement | Delta analyzer metrics | Signal Engine writes |
|
||||
|
||||
---
|
||||
|
||||
## Components and Interfaces
|
||||
|
||||
### Module Structure
|
||||
|
||||
```
|
||||
services/signal_engine/
|
||||
├── __init__.py
|
||||
├── main.py # Entry point: asyncio event loop, queue polling
|
||||
├── worker.py # Top-level orchestrator per evaluation tick
|
||||
├── config.py # SignalEngineConfig, loaded from risk_configs + env
|
||||
├── models.py # All Pydantic models (NormalizedInput, SignalResult, etc.)
|
||||
├── normalizer.py # Input Normalizer — fetches and assembles NormalizedInput
|
||||
├── signals/
|
||||
│ ├── __init__.py
|
||||
│ ├── base.py # SignalEvaluator protocol, SignalResult model
|
||||
│ ├── fibonacci.py # Fibonacci retracement evaluator
|
||||
│ ├── ma_stack.py # Moving average stack evaluator
|
||||
│ ├── rsi.py # RSI evaluator
|
||||
│ ├── cup_handle.py # Cup & Handle pattern detector
|
||||
│ └── elliott_wave.py # Elliott Wave detector
|
||||
├── confluence.py # Multi-Timeframe Confluence Engine
|
||||
├── hard_filter.py # Hard Filter Engine
|
||||
├── heuristic.py # Heuristic Pipeline (Pipeline A)
|
||||
├── probabilistic.py # Probabilistic Pipeline (Pipeline B)
|
||||
├── correlation.py # Signal cluster classification + correlation penalty
|
||||
├── exit_engine.py # Exit Engine — position-level exit management
|
||||
├── delta.py # Delta Analyzer
|
||||
├── formatter.py # Output Formatter
|
||||
└── persistence.py # Database persistence for signal_engine_outputs
|
||||
```
|
||||
|
||||
### Key Function Signatures
|
||||
|
||||
#### `main.py` — Entry Point
|
||||
|
||||
```python
|
||||
async def main() -> None:
|
||||
"""Start the signal engine worker loop.
|
||||
|
||||
Connects to PostgreSQL and Redis, loads config from risk_configs,
|
||||
and polls the signal_engine queue indefinitely.
|
||||
"""
|
||||
```
|
||||
|
||||
#### `worker.py` — Orchestrator
|
||||
|
||||
```python
|
||||
async def evaluate_tick(
|
||||
pool: asyncpg.Pool,
|
||||
redis: redis.asyncio.Redis,
|
||||
ticker: str,
|
||||
config: SignalEngineConfig,
|
||||
) -> SignalOutput | None:
|
||||
"""Run a full evaluation tick for a single ticker.
|
||||
|
||||
1. Normalize inputs
|
||||
2. Evaluate exit conditions for open positions
|
||||
3. Run hard filters
|
||||
4. Evaluate signals across timeframes
|
||||
5. Run both pipelines concurrently
|
||||
6. Compute delta analysis
|
||||
7. Format and publish output
|
||||
|
||||
Returns None if the ticker is hard-filtered or both pipelines fail.
|
||||
"""
|
||||
```
|
||||
|
||||
#### `normalizer.py` — Input Normalizer
|
||||
|
||||
```python
|
||||
async def normalize_input(
|
||||
pool: asyncpg.Pool,
|
||||
ticker: str,
|
||||
config: SignalEngineConfig,
|
||||
) -> NormalizedInput:
|
||||
"""Fetch and assemble all data needed for a single evaluation tick.
|
||||
|
||||
Sources:
|
||||
- OHLCV bars from market_data_bars (M30, H1, H4, D, W, M)
|
||||
- Fundamental metrics from trend_windows + companies
|
||||
- Macro context from macro_impact_records + global_events
|
||||
- Open position state from the trading engine's portfolio
|
||||
|
||||
Missing data sources produce sentinel values (None/empty list)
|
||||
with a logged warning.
|
||||
"""
|
||||
```
|
||||
|
||||
#### `signals/base.py` — Signal Evaluator Protocol
|
||||
|
||||
```python
|
||||
from typing import Protocol
|
||||
|
||||
class SignalEvaluator(Protocol):
|
||||
"""Protocol for all signal evaluators in the Signal Library."""
|
||||
|
||||
def evaluate(
|
||||
self,
|
||||
bars: list[OHLCVBar],
|
||||
timeframe: str,
|
||||
) -> SignalResult | None:
|
||||
"""Evaluate a signal on a single timeframe's bar data.
|
||||
|
||||
Returns None when insufficient data is available.
|
||||
"""
|
||||
...
|
||||
```
|
||||
|
||||
#### `confluence.py` — Multi-Timeframe Engine
|
||||
|
||||
```python
|
||||
def compute_confluence(
|
||||
signal_results: dict[str, dict[str, SignalResult]],
|
||||
weights: dict[str, float],
|
||||
) -> list[ConfluenceSignal]:
|
||||
"""Compute weighted confluence scores across timeframes.
|
||||
|
||||
Args:
|
||||
signal_results: {signal_type: {timeframe: SignalResult}}
|
||||
weights: {timeframe: weight} e.g. {"M30": 0.03, "D": 0.30, ...}
|
||||
|
||||
Returns:
|
||||
List of ConfluenceSignal objects that pass the minimum
|
||||
confluence threshold (≥2 timeframes, ≥1 of D/W/M).
|
||||
"""
|
||||
```
|
||||
|
||||
#### `hard_filter.py` — Hard Filter Engine
|
||||
|
||||
```python
|
||||
def evaluate_hard_filters(
|
||||
normalized: NormalizedInput,
|
||||
config: HardFilterConfig,
|
||||
) -> HardFilterResult:
|
||||
"""Evaluate pre-pipeline hard filters.
|
||||
|
||||
Checks:
|
||||
- macro_bias == -1.0 → SKIP
|
||||
- valuation_score < threshold → SKIP
|
||||
- earnings_proximity_days <= threshold → SKIP
|
||||
|
||||
Returns HardFilterResult with filtered=True/False and all triggered reasons.
|
||||
"""
|
||||
```
|
||||
|
||||
#### `heuristic.py` — Heuristic Pipeline
|
||||
|
||||
```python
|
||||
def run_heuristic_pipeline(
|
||||
normalized: NormalizedInput,
|
||||
confluence_signals: list[ConfluenceSignal],
|
||||
config: HeuristicConfig,
|
||||
) -> HeuristicResult:
|
||||
"""Run the deterministic heuristic pipeline.
|
||||
|
||||
Computes S_total = S_company + S_macro + S_competitive using
|
||||
existing compute_signal_weight() and weighted sentiment averaging.
|
||||
Produces BUY/WATCH/SKIP verdict based on confidence and score thresholds.
|
||||
"""
|
||||
```
|
||||
|
||||
#### `probabilistic.py` — Probabilistic Pipeline
|
||||
|
||||
```python
|
||||
def run_probabilistic_pipeline(
|
||||
normalized: NormalizedInput,
|
||||
confluence_signals: list[ConfluenceSignal],
|
||||
regime: RegimeClassification,
|
||||
config: ProbabilisticConfig,
|
||||
) -> ProbabilisticResult:
|
||||
"""Run the Bayesian probabilistic pipeline.
|
||||
|
||||
1. Initialize regime-based prior (bull=0.58, range=0.50, bear=0.42)
|
||||
2. Compute likelihood ratios per signal with correlation penalty
|
||||
3. Accumulate via log-odds: logit(P_post) = logit(P_prior) + Σ log(LR_i)
|
||||
4. Apply entropy gating
|
||||
5. Compute EV_R = P_up · E[win_R] - (1 - P_up) · 1.0
|
||||
6. Produce BUY/WATCH/SKIP verdict
|
||||
"""
|
||||
```
|
||||
|
||||
#### `correlation.py` — Signal Correlation Penalty
|
||||
|
||||
```python
|
||||
class SignalCluster(str, Enum):
|
||||
MOMENTUM = "momentum" # MA stack, RSI
|
||||
STRUCTURE = "structure" # Fibonacci, Elliott Wave
|
||||
VOLATILITY = "volatility" # ATR-based, Bollinger-derived
|
||||
FUNDAMENTALS = "fundamentals" # valuation, earnings, macro
|
||||
|
||||
def classify_signal(signal_type: str) -> SignalCluster:
|
||||
"""Map a signal type to its correlation cluster."""
|
||||
|
||||
def apply_correlation_penalty(
|
||||
likelihood_ratios: list[LikelihoodRatio],
|
||||
) -> list[LikelihoodRatio]:
|
||||
"""Apply within-cluster decay penalty to correlated signals.
|
||||
|
||||
Within each cluster, signals are ranked by LR magnitude.
|
||||
The strongest contributes at full weight; subsequent signals
|
||||
contribute at 0.5^(n-1) decay.
|
||||
|
||||
Cross-cluster signals are independent (no penalty).
|
||||
"""
|
||||
```
|
||||
|
||||
#### `exit_engine.py` — Exit Engine
|
||||
|
||||
```python
|
||||
def evaluate_exits(
|
||||
positions: list[OpenPositionState],
|
||||
current_prices: dict[str, float],
|
||||
config: ExitConfig,
|
||||
) -> list[ExitSignal]:
|
||||
"""Evaluate exit conditions for all open positions.
|
||||
|
||||
Checks: stop_loss hit, target_1 hit (EXIT_HALF), target_2 hit (EXIT_FULL),
|
||||
trailing stop hit (EXIT_FULL for remaining).
|
||||
|
||||
Trailing stop activates after EXIT_HALF and ratchets upward only.
|
||||
"""
|
||||
```
|
||||
|
||||
#### `delta.py` — Delta Analyzer
|
||||
|
||||
```python
|
||||
async def analyze_delta(
|
||||
heuristic: HeuristicResult,
|
||||
probabilistic: ProbabilisticResult,
|
||||
redis: redis.asyncio.Redis,
|
||||
ticker: str,
|
||||
) -> DeltaResult:
|
||||
"""Compare pipeline verdicts and track agreement metrics.
|
||||
|
||||
Computes agreement flag, confidence delta, disagreement reasons.
|
||||
Updates rolling 100-evaluation agreement rate in Redis.
|
||||
Logs warning when agreement rate drops below 0.50.
|
||||
"""
|
||||
```
|
||||
|
||||
#### `formatter.py` — Output Formatter
|
||||
|
||||
```python
|
||||
def format_output(
|
||||
ticker: str,
|
||||
price: float,
|
||||
heuristic: HeuristicResult,
|
||||
probabilistic: ProbabilisticResult,
|
||||
delta: DeltaResult,
|
||||
exit_signals: list[ExitSignal],
|
||||
config: SignalEngineConfig,
|
||||
) -> SignalOutput:
|
||||
"""Assemble the structured SignalOutput contract.
|
||||
|
||||
Populates trade_plan based on verdict combination:
|
||||
- Both BUY → dual_confirmed, full position sizing
|
||||
- Probabilistic-only BUY → probabilistic_only, 50% position sizing
|
||||
- Heuristic-only BUY → standard position sizing
|
||||
- No BUY → no trade_plan (WATCH/SKIP persisted for analysis)
|
||||
"""
|
||||
|
||||
def signal_output_to_recommendation(output: SignalOutput) -> Recommendation:
|
||||
"""Map a SignalOutput to the existing Recommendation schema.
|
||||
|
||||
Enables the trading engine to consume dual-pipeline outputs
|
||||
without modification to its core evaluate_recommendation logic.
|
||||
"""
|
||||
```
|
||||
|
||||
#### `persistence.py` — Database Persistence
|
||||
|
||||
```python
|
||||
async def persist_signal_output(
|
||||
pool: asyncpg.Pool,
|
||||
output: SignalOutput,
|
||||
) -> None:
|
||||
"""Persist a SignalOutput to the signal_engine_outputs table.
|
||||
|
||||
Logs and continues on database errors (persistence failure
|
||||
does not block signal emission to the trading queue).
|
||||
"""
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Data Models
|
||||
|
||||
All new data models are Pydantic `BaseModel` subclasses defined in `services/signal_engine/models.py`. Existing models (`WeightedSignal`, `BayesianPosterior`, `RegimeClassification`, `TrendSummary`, `Recommendation`, `PositionSizing`) are imported from `services/aggregation/` and `services/shared/schemas.py`.
|
||||
|
||||
### OHLCVBar
|
||||
|
||||
```python
|
||||
class OHLCVBar(BaseModel):
|
||||
"""Single OHLCV bar for a timeframe."""
|
||||
timestamp: datetime
|
||||
open: float
|
||||
high: float
|
||||
low: float
|
||||
close: float
|
||||
volume: float
|
||||
```
|
||||
|
||||
### NormalizedInput
|
||||
|
||||
```python
|
||||
class NormalizedInput(BaseModel):
|
||||
"""Unified input structure consumed by both pipelines."""
|
||||
ticker: str
|
||||
evaluated_at: datetime
|
||||
|
||||
# Multi-timeframe OHLCV bars
|
||||
bars: dict[str, list[OHLCVBar]] # {"M30": [...], "H1": [...], ...}
|
||||
|
||||
# Fundamental metrics
|
||||
valuation_score: float | None = None # [0.0, 1.0]
|
||||
earnings_proximity_days: int | None = None
|
||||
|
||||
# Macro context
|
||||
macro_bias: float = 0.0 # [-1.0, 1.0]
|
||||
|
||||
# Open position state (for exit engine)
|
||||
open_positions: list[OpenPositionState] = Field(default_factory=list)
|
||||
|
||||
# Market data for regime classification
|
||||
closing_prices: list[float] = Field(default_factory=list)
|
||||
returns: list[float] = Field(default_factory=list)
|
||||
|
||||
# Current price (latest close from shortest available timeframe)
|
||||
current_price: float | None = None
|
||||
```
|
||||
|
||||
### OpenPositionState
|
||||
|
||||
```python
|
||||
class OpenPositionState(BaseModel):
|
||||
"""Snapshot of an open position for exit evaluation."""
|
||||
position_id: str
|
||||
ticker: str
|
||||
entry_price: float
|
||||
current_price: float
|
||||
stop_loss: float
|
||||
target_1: float
|
||||
target_2: float
|
||||
trailing_stop: float | None = None
|
||||
partial_exit_done: bool = False
|
||||
atr: float | None = None
|
||||
```
|
||||
|
||||
### SignalResult
|
||||
|
||||
```python
|
||||
class SignalDirection(str, Enum):
|
||||
BULLISH = "bullish"
|
||||
BEARISH = "bearish"
|
||||
NEUTRAL = "neutral"
|
||||
|
||||
class SignalResult(BaseModel):
|
||||
"""Output from a single signal evaluator on a single timeframe."""
|
||||
signal_type: str # e.g. "fibonacci", "ma_stack", "rsi"
|
||||
timeframe: str # e.g. "D", "H4"
|
||||
strength: float = Field(ge=0.0, le=1.0)
|
||||
direction: SignalDirection
|
||||
confidence: float = Field(ge=0.0, le=1.0)
|
||||
metadata: dict = Field(default_factory=dict) # signal-specific details
|
||||
```
|
||||
|
||||
### ConfluenceSignal
|
||||
|
||||
```python
|
||||
class ConfluenceSignal(BaseModel):
|
||||
"""A signal that passed multi-timeframe confluence filtering."""
|
||||
signal_type: str
|
||||
direction: SignalDirection
|
||||
confluence_score: float # weighted sum across timeframes
|
||||
active_timeframes: list[str] # which timeframes triggered
|
||||
per_timeframe: dict[str, float] # {timeframe: strength}
|
||||
```
|
||||
|
||||
### Verdict
|
||||
|
||||
```python
|
||||
class Verdict(str, Enum):
|
||||
BUY = "BUY"
|
||||
WATCH = "WATCH"
|
||||
SKIP = "SKIP"
|
||||
```
|
||||
|
||||
### HeuristicResult
|
||||
|
||||
```python
|
||||
class HeuristicResult(BaseModel):
|
||||
"""Output from the heuristic (deterministic) pipeline."""
|
||||
verdict: Verdict
|
||||
confidence: float = Field(ge=0.0, le=1.0)
|
||||
s_total: float
|
||||
s_company: float
|
||||
s_macro: float
|
||||
s_competitive: float
|
||||
signal_weights: list[dict] = Field(default_factory=list)
|
||||
reasoning: list[str] = Field(default_factory=list)
|
||||
```
|
||||
|
||||
### LikelihoodRatio
|
||||
|
||||
```python
|
||||
class LikelihoodRatio(BaseModel):
|
||||
"""A single signal's likelihood ratio for Bayesian updating."""
|
||||
signal_type: str
|
||||
cluster: str # SignalCluster value
|
||||
lr: float # P(sig|up) / P(sig|down)
|
||||
log_lr: float # log(lr)
|
||||
penalized_log_lr: float # after correlation penalty
|
||||
hit_rate: float
|
||||
strength: float
|
||||
```
|
||||
|
||||
### ProbabilisticResult
|
||||
|
||||
```python
|
||||
class ProbabilisticResult(BaseModel):
|
||||
"""Output from the probabilistic (Bayesian) pipeline."""
|
||||
verdict: Verdict
|
||||
p_up: float = Field(ge=0.0, le=1.0)
|
||||
entropy: float = Field(ge=0.0, le=1.0)
|
||||
ev_r: float
|
||||
prior: float
|
||||
posterior: float
|
||||
likelihood_ratios: list[LikelihoodRatio] = Field(default_factory=list)
|
||||
regime: str
|
||||
reasoning: list[str] = Field(default_factory=list)
|
||||
```
|
||||
|
||||
### DeltaResult
|
||||
|
||||
```python
|
||||
class DeltaResult(BaseModel):
|
||||
"""Output from the delta analyzer comparing both pipelines."""
|
||||
agreement: bool
|
||||
confidence_delta: float
|
||||
heuristic_verdict: str
|
||||
probabilistic_verdict: str
|
||||
disagreement_reasons: list[str] = Field(default_factory=list)
|
||||
rolling_agreement_rate: float | None = None
|
||||
```
|
||||
|
||||
### ExitSignal
|
||||
|
||||
```python
|
||||
class ExitType(str, Enum):
|
||||
EXIT_HALF = "EXIT_HALF"
|
||||
EXIT_FULL = "EXIT_FULL"
|
||||
|
||||
class ExitSignal(BaseModel):
|
||||
"""An exit signal for an open position."""
|
||||
position_id: str
|
||||
ticker: str
|
||||
exit_type: ExitType
|
||||
reason: str # "stop_hit", "target_1_hit", "target_2_hit", "trailing_stop_hit"
|
||||
price: float
|
||||
```
|
||||
|
||||
### TradePlan
|
||||
|
||||
```python
|
||||
class TradePlan(BaseModel):
|
||||
"""Optional trade plan attached to a BUY signal."""
|
||||
entry_price: float
|
||||
stop_loss: float
|
||||
target_1: float
|
||||
target_2: float
|
||||
position_size_pct: float = Field(ge=0.0, le=1.0)
|
||||
max_loss_pct: float = Field(ge=0.0, le=1.0)
|
||||
dual_confirmed: bool = False
|
||||
probabilistic_only: bool = False
|
||||
```
|
||||
|
||||
### SignalOutput
|
||||
|
||||
```python
|
||||
class SignalOutput(BaseModel):
|
||||
"""The structured output contract consumed by the trading engine and audit systems."""
|
||||
output_id: str = Field(default_factory=lambda: str(uuid.uuid4()))
|
||||
ticker: str
|
||||
timestamp: datetime
|
||||
price: float
|
||||
|
||||
# Heuristic pipeline results
|
||||
heuristic_verdict: str
|
||||
heuristic_confidence: float
|
||||
heuristic_s_total: float
|
||||
|
||||
# Probabilistic pipeline results
|
||||
probabilistic_verdict: str
|
||||
probabilistic_p_up: float
|
||||
probabilistic_entropy: float
|
||||
probabilistic_ev_r: float
|
||||
|
||||
# Delta analysis
|
||||
delta_agreement: bool
|
||||
delta_confidence_delta: float
|
||||
delta_reasons: list[str] = Field(default_factory=list)
|
||||
|
||||
# Optional trade plan (populated when at least one pipeline says BUY)
|
||||
trade_plan: TradePlan | None = None
|
||||
|
||||
# Exit signals for open positions
|
||||
exit_signals: list[ExitSignal] = Field(default_factory=list)
|
||||
|
||||
# Full pipeline results for audit (stored as JSONB)
|
||||
heuristic_detail: dict = Field(default_factory=dict)
|
||||
probabilistic_detail: dict = Field(default_factory=dict)
|
||||
|
||||
# Pipeline mode metadata
|
||||
pipeline_mode: str = "dual_pipeline"
|
||||
shadow_mode: bool = False
|
||||
```
|
||||
|
||||
### SignalEngineConfig
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class SignalEngineConfig:
|
||||
"""Configuration loaded from risk_configs + environment."""
|
||||
dual_pipeline_enabled: bool = False
|
||||
heuristic_pipeline_enabled: bool = True
|
||||
probabilistic_pipeline_enabled: bool = True
|
||||
shadow_mode: bool = False
|
||||
|
||||
# Timeframe weights
|
||||
timeframe_weights: dict[str, float] = field(default_factory=lambda: {
|
||||
"M30": 0.03, "H1": 0.07, "H4": 0.15,
|
||||
"D": 0.30, "W": 0.30, "M": 0.15,
|
||||
})
|
||||
|
||||
# Hard filter thresholds
|
||||
hard_filter_valuation_min: float = 0.3
|
||||
hard_filter_earnings_days: int = 5
|
||||
hard_filter_macro_bias_skip: float = -1.0
|
||||
|
||||
# Heuristic verdict thresholds
|
||||
heuristic_buy_confidence: float = 0.70
|
||||
heuristic_buy_s_total: float = 1.2
|
||||
heuristic_buy_valuation_min: float = 0.5
|
||||
heuristic_watch_confidence: float = 0.55
|
||||
|
||||
# Probabilistic verdict thresholds
|
||||
prob_buy_p_up: float = 0.60
|
||||
prob_buy_entropy_max: float = 0.90
|
||||
prob_buy_ev_r_min: float = 1.5
|
||||
prob_buy_valuation_min: float = 0.5
|
||||
prob_watch_p_up: float = 0.55
|
||||
prob_watch_entropy_max: float = 0.95
|
||||
prob_entropy_skip: float = 0.95
|
||||
|
||||
# Regime priors
|
||||
regime_prior_bull: float = 0.58
|
||||
regime_prior_range: float = 0.50
|
||||
regime_prior_bear: float = 0.42
|
||||
|
||||
# Exit engine
|
||||
trailing_stop_atr_multiplier: float = 2.0
|
||||
|
||||
# Polling
|
||||
polling_interval_seconds: int = 30
|
||||
```
|
||||
|
||||
### HardFilterConfig / HeuristicConfig / ProbabilisticConfig / ExitConfig
|
||||
|
||||
These are derived from `SignalEngineConfig` fields for cleaner function signatures — simple `@dataclass` wrappers over the relevant subset of config values.
|
||||
|
||||
---
|
||||
|
||||
### Database Migration (039)
|
||||
|
||||
```sql
|
||||
-- Migration 039: Signal Engine Outputs
|
||||
-- Creates the signal_engine_outputs table for persisting dual-pipeline evaluations.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS signal_engine_outputs (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
ticker TEXT NOT NULL,
|
||||
evaluated_at TIMESTAMPTZ NOT NULL,
|
||||
price NUMERIC NOT NULL,
|
||||
|
||||
-- Heuristic pipeline
|
||||
heuristic_verdict TEXT NOT NULL,
|
||||
heuristic_confidence NUMERIC NOT NULL,
|
||||
heuristic_s_total NUMERIC NOT NULL,
|
||||
|
||||
-- Probabilistic pipeline
|
||||
probabilistic_verdict TEXT NOT NULL,
|
||||
probabilistic_p_up NUMERIC NOT NULL,
|
||||
probabilistic_entropy NUMERIC NOT NULL,
|
||||
probabilistic_ev_r NUMERIC NOT NULL,
|
||||
|
||||
-- Delta analysis
|
||||
delta_agreement BOOLEAN NOT NULL,
|
||||
delta_confidence_delta NUMERIC NOT NULL,
|
||||
delta_reasons JSONB NOT NULL DEFAULT '[]'::jsonb,
|
||||
|
||||
-- Trade plan (null when no BUY verdict)
|
||||
trade_plan JSONB,
|
||||
|
||||
-- Full output for audit
|
||||
full_output JSONB NOT NULL,
|
||||
|
||||
-- Exit signals
|
||||
exit_signals JSONB NOT NULL DEFAULT '[]'::jsonb,
|
||||
|
||||
-- Metadata
|
||||
pipeline_mode TEXT NOT NULL DEFAULT 'dual_pipeline',
|
||||
shadow_mode BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- Index for per-ticker time-range queries
|
||||
CREATE INDEX IF NOT EXISTS idx_signal_engine_outputs_ticker_time
|
||||
ON signal_engine_outputs (ticker, evaluated_at);
|
||||
|
||||
-- Index for global time-range queries
|
||||
CREATE INDEX IF NOT EXISTS idx_signal_engine_outputs_evaluated
|
||||
ON signal_engine_outputs (evaluated_at);
|
||||
|
||||
-- Index for filtering by verdict
|
||||
CREATE INDEX IF NOT EXISTS idx_signal_engine_outputs_verdicts
|
||||
ON signal_engine_outputs (heuristic_verdict, probabilistic_verdict);
|
||||
```
|
||||
|
||||
### Helm / Deployment Configuration
|
||||
|
||||
Add to `values.yaml` under `services:`:
|
||||
|
||||
```yaml
|
||||
signalEngine:
|
||||
replicas: 1
|
||||
pipeline: true
|
||||
image: signal-engine
|
||||
command: "python -m services.signal_engine.main"
|
||||
tier: processing
|
||||
secrets: [stonks-core-secrets, stonks-market-secrets]
|
||||
resources:
|
||||
requests: { cpu: 100m, memory: 128Mi }
|
||||
limits: { cpu: 500m, memory: 256Mi }
|
||||
```
|
||||
|
||||
Add to `redis_keys.py`:
|
||||
|
||||
```python
|
||||
QUEUE_SIGNAL_ENGINE = "signal_engine"
|
||||
```
|
||||
|
||||
The service uses the existing `stonks-config` ConfigMap and `stonks-core-secrets` for database/Redis credentials. No new ingress or network policy is needed — the signal engine is a queue-polling worker with no HTTP interface.
|
||||
|
||||
---
|
||||
|
||||
@@ -0,0 +1,300 @@
|
||||
# Requirements Document — Dual-Pipeline Signal Engine
|
||||
|
||||
## Introduction
|
||||
|
||||
The Stonks Oracle platform currently operates a single aggregation pipeline that can run in either heuristic or probabilistic mode (toggled via `probabilistic_scoring_enabled`). This feature replaces the single-pipeline toggle with a dual-pipeline architecture where both pipelines run concurrently per evaluation tick, produce independent verdicts (BUY/WATCH/SKIP), and emit a structured output contract for downstream consumers (trading engine, delta analysis, dashboards).
|
||||
|
||||
The dual-pipeline engine introduces:
|
||||
- **Pipeline A (Heuristic)**: Deterministic scoring using the existing `S_total = S_company + S_macro + S_competitive` formula with signal weighting, producing a confidence-gated verdict.
|
||||
- **Pipeline B (Probabilistic)**: Bayesian inference using the existing `bayesian.py` infrastructure with regime-based priors, likelihood ratios, entropy gating, and expected value calculation.
|
||||
- **Hard Filter Engine**: Pre-pipeline filters that short-circuit both pipelines before evaluation.
|
||||
- **Multi-Timeframe Engine**: Signal evaluation across M30, H1, H4, D, W, M timeframes with weighted confluence scoring.
|
||||
- **Exit Engine**: Position-level exit management (stop hit, targets, trailing ATR-based).
|
||||
- **Delta Analyzer**: Compares heuristic vs probabilistic verdicts to generate training signals for future model tuning.
|
||||
- **Output Formatter**: Structured `SignalOutput` contract consumed by the trading engine and delta analysis.
|
||||
|
||||
The design must address the signal independence assumption in the Bayesian pipeline — correlated signals (MA+RSI, Fib+Elliott) require correlation penalty or signal clustering into categories (momentum, structure, volatility, fundamentals) to prevent likelihood ratio stacking inflation.
|
||||
|
||||
## Glossary
|
||||
|
||||
- **Signal_Engine**: The top-level orchestrator in `services/signal_engine/` that coordinates input normalization, hard filters, both pipelines, delta analysis, and output formatting per evaluation tick.
|
||||
- **Heuristic_Pipeline**: Pipeline A — deterministic scoring that computes `S_total = S_company + S_macro + S_competitive` with signal weighting and produces a confidence-gated BUY/WATCH/SKIP verdict.
|
||||
- **Probabilistic_Pipeline**: Pipeline B — Bayesian inference pipeline that computes posterior probability via log-likelihood accumulation with regime-based priors, entropy gating, and expected value calculation.
|
||||
- **Input_Normalizer**: The component that ingests multi-timeframe OHLCV data, fundamentals, macro context, and open positions into a unified `NormalizedInput` structure consumed by both pipelines.
|
||||
- **Signal_Library**: The collection of technical signal evaluators (Fibonacci retracement, MA stack, RSI, Cup & Handle, Elliott Wave) that produce scored signals per timeframe.
|
||||
- **Multi_Timeframe_Engine**: The component that evaluates signals across six timeframes (M30, H1, H4, D, W, M) and computes weighted confluence scores.
|
||||
- **Hard_Filter_Engine**: The pre-pipeline filter stage that evaluates macro bias, valuation score, and earnings proximity to short-circuit evaluation before either pipeline runs.
|
||||
- **Exit_Engine**: The position management component that evaluates stop hits, take-profit targets, and trailing ATR-based stops for open positions.
|
||||
- **Delta_Analyzer**: The component that compares heuristic and probabilistic verdicts, tracks agreement rates, measures confidence deltas, and records disagreement reasons as training signals.
|
||||
- **Output_Formatter**: The component that assembles the structured `SignalOutput` contract from both pipeline results, delta analysis, and optional trade plan.
|
||||
- **SignalOutput**: The structured output contract containing ticker, timestamp, price, heuristic verdict/confidence/S_total, probabilistic verdict/P_up/entropy/EV_R, delta analysis, and optional trade plan.
|
||||
- **Verdict**: A pipeline decision of BUY, WATCH, or SKIP with associated confidence and reasoning.
|
||||
- **Confluence**: The condition where a signal triggers across multiple timeframes; requires activation on at least 2 timeframes including at least one of D, W, or M.
|
||||
- **Entropy_Gate**: Shannon entropy threshold used in the probabilistic pipeline to detect high-uncertainty states and force SKIP verdicts.
|
||||
- **EV_R**: Expected value per unit of risk, computed as `P_up · E[win_R] - (1 - P_up) · 1.0`, used as a quality gate in the probabilistic pipeline.
|
||||
- **Signal_Cluster**: A grouping of correlated signals (momentum, structure, volatility, fundamentals) used to prevent likelihood ratio stacking inflation in the Bayesian pipeline.
|
||||
- **Likelihood_Ratio**: The ratio `P(signal|up) / P(signal|down)` used in Bayesian updating, where `P(sig|up) = h·s + (1-h)·(1-s)·0.5`.
|
||||
- **Regime_Prior**: The initial probability assigned based on market regime classification: bull=0.58, range=0.50, bear=0.42.
|
||||
- **OHLCV**: Open, High, Low, Close, Volume — standard market data bar format.
|
||||
- **ATR**: Average True Range — a volatility measure used for trailing stop calculations.
|
||||
- **Fibonacci_Retracement**: A technical analysis tool computing price levels as `L(r) = SH - r·(SH - SL)` where SH is swing high, SL is swing low, and r is a retracement ratio (0.236, 0.382, 0.5, 0.618, 0.786).
|
||||
|
||||
---
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement 1: Input Normalization
|
||||
|
||||
**User Story:** As a signal engine operator, I want all market data, fundamentals, macro context, and open positions normalized into a single input structure, so that both pipelines consume identical inputs per evaluation tick.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN an evaluation tick is triggered for a ticker, THE Input_Normalizer SHALL construct a `NormalizedInput` containing multi-timeframe OHLCV bars (M30, H1, H4, D, W, M), fundamental metrics (valuation_score, earnings_proximity_days), macro context (macro_bias as float in [-1.0, 1.0]), and open position state (entry_price, current_price, stop_loss, targets).
|
||||
2. THE Input_Normalizer SHALL source OHLCV data from the existing market data tables, fundamental metrics from the existing company and trend data, and macro context from the existing `macro_impact_records` and `global_events` tables.
|
||||
3. IF any required data source is unavailable or returns an error, THEN THE Input_Normalizer SHALL populate the corresponding field with a sentinel value (`None` for optional fields, empty list for OHLCV bars) and log a warning identifying the missing source.
|
||||
4. THE Input_Normalizer SHALL validate that all OHLCV bars have monotonically increasing timestamps within each timeframe series.
|
||||
5. THE Input_Normalizer SHALL produce identical `NormalizedInput` instances for both pipelines within the same evaluation tick (shared reference, no independent fetches).
|
||||
|
||||
---
|
||||
|
||||
### Requirement 2: Signal Library — Technical Signal Evaluation
|
||||
|
||||
**User Story:** As a quantitative analyst, I want a library of technical signal evaluators that produce scored signals per timeframe, so that both pipelines can consume standardized signal assessments.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. THE Signal_Library SHALL implement Fibonacci retracement signal evaluation using the formula `L(r) = SH - r·(SH - SL)` for retracement ratios [0.236, 0.382, 0.5, 0.618, 0.786], where SH is the swing high and SL is the swing low within the evaluation window.
|
||||
2. THE Signal_Library SHALL implement moving average stack evaluation that detects bullish alignment (MA_10 > MA_20 > MA_50 > MA_200) and bearish alignment (MA_10 < MA_20 < MA_50 < MA_200), producing a signal strength proportional to the degree of alignment.
|
||||
3. THE Signal_Library SHALL implement RSI evaluation using the standard 14-period RSI formula, producing overbought signals (RSI > 70) and oversold signals (RSI < 30) with strength scaled by distance from the threshold.
|
||||
4. THE Signal_Library SHALL implement Cup & Handle pattern detection that identifies the cup formation (U-shaped price recovery) and handle (small consolidation), producing a signal with confidence proportional to pattern completeness.
|
||||
5. THE Signal_Library SHALL implement Elliott Wave detection that identifies impulse waves (5-wave structure) and corrective waves (3-wave structure), producing a signal with the current wave position and projected direction.
|
||||
6. WHEN a signal evaluator receives insufficient data for its calculation (fewer bars than the required lookback period), THE Signal_Library SHALL return a null signal with a reason code indicating insufficient data rather than producing a partial evaluation.
|
||||
7. FOR ALL signal evaluators, THE Signal_Library SHALL produce output conforming to a common `SignalResult` structure containing: signal_type, timeframe, strength (float in [0.0, 1.0]), direction (bullish/bearish/neutral), confidence (float in [0.0, 1.0]), and metadata specific to the signal type.
|
||||
|
||||
---
|
||||
|
||||
### Requirement 3: Multi-Timeframe Confluence Engine
|
||||
|
||||
**User Story:** As a quantitative analyst, I want signals evaluated across multiple timeframes with weighted confluence scoring, so that the engine prioritizes signals confirmed across longer timeframes.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. THE Multi_Timeframe_Engine SHALL evaluate each signal type across six timeframes with the following weights: M30=0.03, H1=0.07, H4=0.15, D=0.30, W=0.30, M=0.15.
|
||||
2. THE Multi_Timeframe_Engine SHALL compute a weighted confluence score as `C_confluence = Σ(w_tf · s_tf)` where `w_tf` is the timeframe weight and `s_tf` is the signal strength on that timeframe (0.0 if the signal did not trigger).
|
||||
3. WHEN a signal triggers on fewer than 2 timeframes, THE Multi_Timeframe_Engine SHALL discard the signal from further pipeline processing (minimum confluence threshold).
|
||||
4. WHEN a signal triggers on 2 or more timeframes but none of D, W, or M are included, THE Multi_Timeframe_Engine SHALL discard the signal from further pipeline processing (higher-timeframe anchor requirement).
|
||||
5. THE Multi_Timeframe_Engine SHALL pass the confluence-filtered signals and their weighted scores to both the Heuristic_Pipeline and Probabilistic_Pipeline.
|
||||
6. FOR ALL signal sets where a signal triggers on more timeframes with higher weights, THE Multi_Timeframe_Engine SHALL produce a higher confluence score (monotonicity with respect to timeframe activation count and weight).
|
||||
|
||||
---
|
||||
|
||||
### Requirement 4: Hard Filter Engine — Pre-Pipeline Gating
|
||||
|
||||
**User Story:** As a risk manager, I want hard filters that short-circuit both pipelines before evaluation, so that clearly unfavorable conditions produce immediate SKIP verdicts without wasting computation.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN the macro_bias value from the NormalizedInput equals -1.0, THE Hard_Filter_Engine SHALL produce an immediate SKIP verdict for both pipelines with reason "macro_bias_negative".
|
||||
2. WHEN the valuation_score from the NormalizedInput is below 0.3, THE Hard_Filter_Engine SHALL produce an immediate SKIP verdict for both pipelines with reason "valuation_below_threshold".
|
||||
3. WHEN the earnings_proximity_days from the NormalizedInput is 5 or fewer, THE Hard_Filter_Engine SHALL produce an immediate SKIP verdict for both pipelines with reason "earnings_block".
|
||||
4. WHEN multiple hard filters trigger simultaneously, THE Hard_Filter_Engine SHALL record all triggered filter reasons in the SKIP verdict (not just the first).
|
||||
5. WHEN no hard filters trigger, THE Hard_Filter_Engine SHALL pass the NormalizedInput through to both pipelines without modification.
|
||||
6. THE Hard_Filter_Engine SHALL execute before either pipeline begins evaluation, and both pipelines SHALL receive the same filter decision.
|
||||
|
||||
---
|
||||
|
||||
### Requirement 5: Heuristic Pipeline — Deterministic Scoring and Verdict
|
||||
|
||||
**User Story:** As a quantitative analyst, I want the heuristic pipeline to produce a deterministic BUY/WATCH/SKIP verdict based on composite scoring of company, macro, and competitive signals, so that the system maintains a transparent, auditable scoring path.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. THE Heuristic_Pipeline SHALL compute a total score `S_total = S_company + S_macro + S_competitive` using the existing three-layer signal aggregation with the current `WeightedSignal` abstraction.
|
||||
2. THE Heuristic_Pipeline SHALL compute signal weights using the formula `W_signal = gate · recency · credibility · (1 + novelty) · market_context` consistent with the existing `compute_signal_weight` function in `scoring.py`.
|
||||
3. THE Heuristic_Pipeline SHALL compute a confidence value from the existing trend confidence formula incorporating source count, extraction confidence, signal agreement, and contradiction penalty.
|
||||
4. THE Heuristic_Pipeline SHALL produce a BUY verdict WHEN confidence >= 0.70 AND S_total >= 1.2 AND valuation_score >= 0.5 AND macro_bias > 0 AND earnings_proximity_days > 5.
|
||||
5. THE Heuristic_Pipeline SHALL produce a WATCH verdict WHEN confidence >= 0.55 AND the BUY conditions are not fully met.
|
||||
6. THE Heuristic_Pipeline SHALL produce a SKIP verdict WHEN confidence < 0.55.
|
||||
7. THE Heuristic_Pipeline SHALL emit a `HeuristicResult` containing: verdict (BUY/WATCH/SKIP), confidence (float), S_total (float), S_company (float), S_macro (float), S_competitive (float), signal_weights (list), and reasoning (list of strings explaining the verdict).
|
||||
|
||||
---
|
||||
|
||||
### Requirement 6: Probabilistic Pipeline — Bayesian Inference and Verdict
|
||||
|
||||
**User Story:** As a quantitative analyst, I want the probabilistic pipeline to produce a Bayesian BUY/WATCH/SKIP verdict using regime-based priors, likelihood ratios, entropy gating, and expected value calculation, so that the system captures uncertainty structure and risk-adjusted expected outcomes.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. THE Probabilistic_Pipeline SHALL initialize the prior probability based on the current market regime classification: bull regime → P_prior = 0.58, range regime → P_prior = 0.50, bear regime → P_prior = 0.42.
|
||||
2. THE Probabilistic_Pipeline SHALL compute likelihood ratios for each signal using `P(sig|up) = h·s + (1-h)·(1-s)·0.5` and `LR = P(sig|up) / P(sig|down)`, where h is the signal's historical hit rate and s is the signal strength.
|
||||
3. THE Probabilistic_Pipeline SHALL update the posterior using log-odds accumulation: `logit(P_post) = logit(P_prior) + Σ log(LR_i)`, converting back to probability via the sigmoid function.
|
||||
4. THE Probabilistic_Pipeline SHALL compute Shannon entropy `H = -P_up·log₂(P_up) - (1-P_up)·log₂(1-P_up)` and apply entropy gating: WHEN H > 0.95, THE Probabilistic_Pipeline SHALL force a SKIP verdict with reason "high_entropy".
|
||||
5. THE Probabilistic_Pipeline SHALL compute expected value per unit risk as `EV_R = P_up · E[win_R] - (1 - P_up) · 1.0` where `E[win_R]` is the expected win in risk units derived from signal strength and historical reward-risk ratios.
|
||||
6. THE Probabilistic_Pipeline SHALL produce a BUY verdict WHEN P_up >= 0.60 AND entropy <= 0.90 AND EV_R >= 1.5 AND macro_bias > 0 AND valuation_score >= 0.5.
|
||||
7. THE Probabilistic_Pipeline SHALL produce a WATCH verdict WHEN P_up >= 0.55 AND entropy <= 0.95 AND the BUY conditions are not fully met.
|
||||
8. THE Probabilistic_Pipeline SHALL produce a SKIP verdict in all other cases.
|
||||
9. THE Probabilistic_Pipeline SHALL emit a `ProbabilisticResult` containing: verdict (BUY/WATCH/SKIP), P_up (float), entropy (float), EV_R (float), prior (float), posterior (float), likelihood_ratios (list), regime (string), and reasoning (list of strings).
|
||||
|
||||
---
|
||||
|
||||
### Requirement 7: Signal Correlation Penalty — Preventing LR Stacking Inflation
|
||||
|
||||
**User Story:** As a quantitative analyst, I want correlated signals grouped into clusters with a correlation penalty applied to prevent likelihood ratio stacking inflation, so that the Bayesian pipeline does not overstate confidence from redundant signals.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. THE Probabilistic_Pipeline SHALL classify each signal into one of four clusters: momentum (MA stack, RSI), structure (Fibonacci retracement, Elliott Wave), volatility (ATR-based signals, Bollinger-derived), and fundamentals (valuation, earnings, macro).
|
||||
2. WHEN multiple signals within the same cluster produce likelihood ratios in the same direction, THE Probabilistic_Pipeline SHALL apply a within-cluster penalty: only the strongest LR in the cluster contributes at full weight, and subsequent LRs in the same cluster contribute at a decay factor of 0.5^(n-1) where n is the signal's rank within the cluster by LR magnitude.
|
||||
3. THE Probabilistic_Pipeline SHALL apply no penalty across different clusters (signals from different clusters are treated as independent).
|
||||
4. WHEN a cluster contains only one signal, THE Probabilistic_Pipeline SHALL apply no penalty to that signal.
|
||||
5. FOR ALL signal sets, THE Probabilistic_Pipeline SHALL produce a posterior probability that is less than or equal to the posterior computed without the correlation penalty (the penalty only reduces confidence, never inflates it).
|
||||
|
||||
---
|
||||
|
||||
### Requirement 8: Exit Engine — Position Management
|
||||
|
||||
**User Story:** As a trader, I want the signal engine to evaluate exit conditions for open positions, so that stop hits, take-profit targets, and trailing stops are managed as part of the signal evaluation cycle.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN the current price of an open position hits or crosses below the stop_loss level, THE Exit_Engine SHALL emit an EXIT_FULL signal for that position with reason "stop_hit".
|
||||
2. WHEN the current price of an open position hits or crosses above the first take-profit target (target_1), THE Exit_Engine SHALL emit an EXIT_HALF signal for that position with reason "target_1_hit".
|
||||
3. WHEN the current price of an open position hits or crosses above the second take-profit target (target_2), THE Exit_Engine SHALL emit an EXIT_FULL signal for that position with reason "target_2_hit".
|
||||
4. WHEN a partial exit has been executed (EXIT_HALF), THE Exit_Engine SHALL activate a trailing stop at `current_price - ATR · trailing_multiplier` and update the trailing stop upward as the price advances (the trailing stop moves up but does not move down).
|
||||
5. WHEN the trailing stop is active and the current price crosses below the trailing stop level, THE Exit_Engine SHALL emit an EXIT_FULL signal for the remaining position with reason "trailing_stop_hit".
|
||||
6. THE Exit_Engine SHALL evaluate exit conditions before the signal pipelines run for new entry signals, so that exit signals take priority over new entry signals for the same ticker.
|
||||
7. THE Exit_Engine SHALL emit exit signals as part of the `SignalOutput` contract with the position identifier, exit type (EXIT_HALF/EXIT_FULL), and reason.
|
||||
|
||||
---
|
||||
|
||||
### Requirement 9: Delta Analyzer — Pipeline Agreement Tracking
|
||||
|
||||
**User Story:** As a model developer, I want the delta analyzer to compare heuristic and probabilistic verdicts and record disagreement details, so that I can generate training signals for future model tuning.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN both pipelines produce verdicts for the same ticker and tick, THE Delta_Analyzer SHALL compute an agreement flag (true if both verdicts are identical, false otherwise).
|
||||
2. THE Delta_Analyzer SHALL compute a confidence delta as `|heuristic_confidence - probabilistic_P_up|` representing the magnitude of disagreement between the two pipelines.
|
||||
3. WHEN the pipelines disagree on verdict, THE Delta_Analyzer SHALL record the disagreement reason by identifying which conditions differed (e.g., "heuristic_confidence_below_threshold", "probabilistic_entropy_too_high", "EV_R_below_threshold").
|
||||
4. THE Delta_Analyzer SHALL track a rolling agreement rate over the last 100 evaluations per ticker, stored in Redis for dashboard consumption.
|
||||
5. THE Delta_Analyzer SHALL emit a `DeltaResult` containing: agreement (bool), confidence_delta (float), heuristic_verdict (string), probabilistic_verdict (string), disagreement_reasons (list of strings), and rolling_agreement_rate (float).
|
||||
6. WHEN the rolling agreement rate drops below 0.50 for a ticker, THE Delta_Analyzer SHALL log a warning indicating persistent pipeline disagreement for operator review.
|
||||
|
||||
---
|
||||
|
||||
### Requirement 10: Output Formatter — Structured SignalOutput Contract
|
||||
|
||||
**User Story:** As a downstream system consumer, I want the signal engine to emit a structured `SignalOutput` contract, so that the trading engine, delta analysis dashboard, and audit systems can consume a consistent output format.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. THE Output_Formatter SHALL produce a `SignalOutput` containing: ticker (string), timestamp (datetime), price (float), heuristic section (verdict, confidence, S_total), probabilistic section (verdict, P_up, entropy, EV_R), delta section (agreement, confidence_delta, disagreement_reasons), and optional trade_plan section.
|
||||
2. WHEN the heuristic pipeline produces a BUY verdict, THE Output_Formatter SHALL populate the trade_plan section with entry_price, stop_loss, target_1, target_2, and position_size derived from the heuristic confidence and existing position sizing logic.
|
||||
3. WHEN the probabilistic pipeline produces a BUY verdict but the heuristic pipeline does not, THE Output_Formatter SHALL populate the trade_plan section with a "probabilistic_only" flag and reduced position sizing (50% of standard).
|
||||
4. WHEN both pipelines produce a BUY verdict, THE Output_Formatter SHALL populate the trade_plan section with full position sizing and a "dual_confirmed" flag.
|
||||
5. THE Output_Formatter SHALL serialize the `SignalOutput` as a Pydantic model with JSON serialization support for Redis queue publishing and database persistence.
|
||||
6. FOR ALL valid pipeline results, THE Output_Formatter SHALL produce a `SignalOutput` that round-trips through JSON serialization and deserialization without data loss (parse(format(output)) produces an equivalent object).
|
||||
|
||||
---
|
||||
|
||||
### Requirement 11: Dual Pipeline Orchestration
|
||||
|
||||
**User Story:** As a signal engine operator, I want both pipelines to run concurrently per evaluation tick sharing the same inputs, so that the system produces independent verdicts without redundant data fetching.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN an evaluation tick is triggered, THE Signal_Engine SHALL execute the Input_Normalizer once, then pass the resulting `NormalizedInput` to the Hard_Filter_Engine, then (if not filtered) execute both the Heuristic_Pipeline and Probabilistic_Pipeline concurrently using `asyncio.gather`.
|
||||
2. THE Signal_Engine SHALL enforce that both pipelines receive identical `NormalizedInput` references (no independent data fetches that could produce different snapshots).
|
||||
3. WHEN either pipeline raises an exception during evaluation, THE Signal_Engine SHALL catch the exception, log the error with full traceback, and produce a SKIP verdict for the failed pipeline with reason "pipeline_error" while allowing the other pipeline to complete normally.
|
||||
4. THE Signal_Engine SHALL measure and log the wall-clock execution time of each pipeline per tick for performance monitoring.
|
||||
5. THE Signal_Engine SHALL publish the assembled `SignalOutput` to the existing Redis queue (`stonks:queue:trading_decisions`) for consumption by the trading engine.
|
||||
6. THE Signal_Engine SHALL persist each `SignalOutput` to a database table for historical analysis and audit.
|
||||
|
||||
---
|
||||
|
||||
### Requirement 12: Integration with Existing Trading Engine
|
||||
|
||||
**User Story:** As a platform operator, I want the dual-pipeline signal engine to integrate with the existing trading engine, so that the trading engine can consume `SignalOutput` verdicts and make execution decisions.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. THE Signal_Engine SHALL publish `SignalOutput` to the existing `stonks:queue:trading_decisions` Redis queue in a format compatible with the existing `TradingEngine.evaluate_recommendation` interface.
|
||||
2. THE Signal_Engine SHALL map the `SignalOutput` trade_plan to the existing `Recommendation` schema fields (action, confidence, position_sizing) so that the trading engine can process dual-pipeline outputs without modification to its core evaluation logic.
|
||||
3. WHEN the `SignalOutput` has a "dual_confirmed" flag, THE Signal_Engine SHALL set the recommendation confidence to the maximum of heuristic_confidence and probabilistic_P_up.
|
||||
4. WHEN the `SignalOutput` has a "probabilistic_only" flag, THE Signal_Engine SHALL set the recommendation confidence to `probabilistic_P_up · 0.8` (20% confidence haircut for single-pipeline confirmation).
|
||||
5. WHEN neither pipeline produces a BUY verdict, THE Signal_Engine SHALL not publish a trading recommendation to the queue (WATCH and SKIP verdicts are persisted for analysis but not forwarded to the trading engine).
|
||||
|
||||
---
|
||||
|
||||
### Requirement 13: Configuration and Feature Flags
|
||||
|
||||
**User Story:** As a platform operator, I want the dual-pipeline engine configurable via the existing `risk_configs` table and environment variables, so that I can tune thresholds, enable/disable individual pipelines, and adjust timeframe weights without code changes.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. THE Signal_Engine SHALL support a `dual_pipeline_enabled` feature flag in `risk_configs` that toggles the entire dual-pipeline engine on or off, defaulting to false for safe rollout.
|
||||
2. THE Signal_Engine SHALL support independent enable/disable flags for each pipeline: `heuristic_pipeline_enabled` and `probabilistic_pipeline_enabled`, both defaulting to true when the dual-pipeline engine is enabled.
|
||||
3. THE Signal_Engine SHALL support configurable timeframe weights via a `timeframe_weights` JSON object in `risk_configs`, defaulting to `{"M30": 0.03, "H1": 0.07, "H4": 0.15, "D": 0.30, "W": 0.30, "M": 0.15}`.
|
||||
4. THE Signal_Engine SHALL support configurable hard filter thresholds: `hard_filter_valuation_min` (default 0.3), `hard_filter_earnings_days` (default 5), and `hard_filter_macro_bias_skip` (default -1.0).
|
||||
5. THE Signal_Engine SHALL support configurable verdict thresholds for both pipelines via `risk_configs` JSON, including heuristic confidence thresholds (BUY: 0.70, WATCH: 0.55) and probabilistic thresholds (P_up: 0.60, entropy: 0.90, EV_R: 1.5).
|
||||
6. IF the `dual_pipeline_enabled` flag fails to read from the database, THEN THE Signal_Engine SHALL default to disabled (fail-safe behavior) and log a warning.
|
||||
7. THE Signal_Engine SHALL log the active configuration at startup and on each configuration change for auditability.
|
||||
|
||||
---
|
||||
|
||||
### Requirement 14: Regime-Based Prior Engine
|
||||
|
||||
**User Story:** As a quantitative analyst, I want the probabilistic pipeline's prior probability to adapt based on the current market regime, so that the Bayesian inference starts from a regime-appropriate baseline rather than a fixed 0.50.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. THE Probabilistic_Pipeline SHALL use the existing `classify_regime` function from `services/aggregation/regime.py` to determine the current market regime for each ticker.
|
||||
2. THE Probabilistic_Pipeline SHALL map regime classifications to prior probabilities: trend_following with positive trend_indicator → 0.58 (bull), trend_following with negative trend_indicator → 0.42 (bear), mean_reversion → 0.50 (range), panic → 0.42 (bear), uncertainty → 0.50 (range).
|
||||
3. THE Probabilistic_Pipeline SHALL convert the regime prior to log-odds before accumulating likelihood ratios: `logit(P_prior) = log(P_prior / (1 - P_prior))`.
|
||||
4. WHEN market data is insufficient for regime classification (fewer than 100 days of price history), THE Probabilistic_Pipeline SHALL use the uncertainty prior of 0.50.
|
||||
5. THE Probabilistic_Pipeline SHALL record the regime classification and prior probability in the `ProbabilisticResult` for auditability.
|
||||
|
||||
---
|
||||
|
||||
### Requirement 15: Database Schema for Signal Engine Output
|
||||
|
||||
**User Story:** As a platform operator, I want signal engine outputs persisted to a dedicated database table, so that historical evaluations are available for analysis, backtesting, and audit.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. THE Signal_Engine SHALL persist each `SignalOutput` to a `signal_engine_outputs` table with columns for: id (UUID primary key), ticker (text), evaluated_at (timestamptz), price (numeric), heuristic_verdict (text), heuristic_confidence (numeric), heuristic_s_total (numeric), probabilistic_verdict (text), probabilistic_p_up (numeric), probabilistic_entropy (numeric), probabilistic_ev_r (numeric), delta_agreement (boolean), delta_confidence_delta (numeric), delta_reasons (JSONB), trade_plan (JSONB), full_output (JSONB), created_at (timestamptz).
|
||||
2. THE Signal_Engine SHALL create an index on `(ticker, evaluated_at)` for efficient time-range queries per ticker.
|
||||
3. THE Signal_Engine SHALL create an index on `evaluated_at` for efficient global time-range queries.
|
||||
4. WHEN persisting fails due to a database error, THE Signal_Engine SHALL log the error and continue processing (persistence failure does not block signal emission to the trading queue).
|
||||
|
||||
---
|
||||
|
||||
### Requirement 16: Backward Compatibility and Migration Path
|
||||
|
||||
**User Story:** As a platform operator, I want the dual-pipeline engine to coexist with the existing single-pipeline aggregation, so that the rollout is incremental and reversible.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN `dual_pipeline_enabled` is false, THE Signal_Engine SHALL not run, and the existing aggregation pipeline SHALL continue to operate unchanged.
|
||||
2. WHEN `dual_pipeline_enabled` is true, THE Signal_Engine SHALL run alongside the existing aggregation pipeline, with the trading engine consuming `SignalOutput` from the dual-pipeline engine instead of `Recommendation` from the existing recommendation worker.
|
||||
3. THE Signal_Engine SHALL reuse the existing `WeightedSignal`, `BayesianPosterior`, `RegimeClassification`, and `TrendSummary` data structures from `services/aggregation/` rather than duplicating them.
|
||||
4. THE Signal_Engine SHALL reuse the existing `compute_signal_weight`, `compute_bayesian_posterior`, and `classify_regime` functions rather than reimplementing the underlying math.
|
||||
5. THE Signal_Engine SHALL add the new `signal_engine_outputs` table via a new database migration without modifying existing tables.
|
||||
6. THE Signal_Engine SHALL support running in "shadow mode" where both the existing pipeline and the dual-pipeline engine run, but only the existing pipeline's output is forwarded to the trading engine (dual-pipeline output is persisted for comparison only).
|
||||
|
||||
---
|
||||
|
||||
### Requirement 17: Property-Based Testing for Dual-Pipeline Correctness
|
||||
|
||||
**User Story:** As a developer, I want comprehensive property-based tests validating the mathematical correctness and structural invariants of the dual-pipeline engine, so that edge cases and numerical stability issues are caught before deployment.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. THE test suite SHALL include property-based tests for the Fibonacci retracement formula verifying that `L(r) = SH - r·(SH - SL)` produces values in [SL, SH] for all r in [0, 1] and all SH > SL > 0.
|
||||
2. THE test suite SHALL include property-based tests for the Bayesian log-odds update verifying that `logit(P_post) = logit(P_prior) + Σ log(LR_i)` round-trips correctly: converting P_prior to logit, adding log-LRs, and converting back via sigmoid produces a valid probability in (0, 1).
|
||||
3. THE test suite SHALL include property-based tests for the entropy gate verifying that Shannon entropy is maximized at P_up = 0.5 and equals 0.0 at P_up = 0.0 or P_up = 1.0, and is symmetric around 0.5.
|
||||
4. THE test suite SHALL include property-based tests for the signal correlation penalty verifying that the penalized posterior is always less than or equal to the unpenalized posterior for any signal set with correlated signals.
|
||||
5. THE test suite SHALL include property-based tests for the multi-timeframe confluence score verifying monotonicity: activating a signal on an additional timeframe with non-zero weight always increases or maintains the confluence score.
|
||||
6. THE test suite SHALL include property-based tests for the `SignalOutput` contract verifying round-trip serialization: `SignalOutput.model_validate_json(output.model_dump_json())` produces an equivalent object for all valid outputs.
|
||||
7. THE test suite SHALL include property-based tests for the hard filter engine verifying that macro_bias = -1.0 always produces SKIP, valuation_score < 0.3 always produces SKIP, and earnings_proximity_days <= 5 always produces SKIP, regardless of all other input values.
|
||||
8. THE test suite SHALL include property-based tests for the EV_R calculation verifying that `EV_R = P_up · E[win_R] - (1 - P_up) · 1.0` is monotonically increasing with P_up for fixed E[win_R] > 0.
|
||||
@@ -0,0 +1,345 @@
|
||||
# Implementation Plan: Dual-Pipeline Signal Engine
|
||||
|
||||
## Overview
|
||||
|
||||
Implement the dual-pipeline signal engine as a new service at `services/signal_engine/` that runs as an independent Kubernetes deployment. The engine evaluates both a heuristic (deterministic scoring) and probabilistic (Bayesian inference) pipeline concurrently per ticker per evaluation tick, producing independent BUY/WATCH/SKIP verdicts. Implementation proceeds incrementally: infrastructure first, then core models, signal library, pipelines, orchestration, integration, and deployment.
|
||||
|
||||
## Tasks
|
||||
|
||||
- [x] 1. Project scaffolding, configuration, and data models
|
||||
- [x] 1.1 Create service directory structure and `__init__.py` files
|
||||
- Create `services/signal_engine/` with all subdirectories per the design module structure
|
||||
- Create `services/signal_engine/__init__.py`, `services/signal_engine/signals/__init__.py`
|
||||
- _Requirements: 11.1, 13.1_
|
||||
|
||||
- [x] 1.2 Implement `models.py` — all Pydantic data models
|
||||
- Define `OHLCVBar`, `NormalizedInput`, `OpenPositionState`, `SignalResult`, `SignalDirection`
|
||||
- Define `ConfluenceSignal`, `Verdict`, `HeuristicResult`, `LikelihoodRatio`, `ProbabilisticResult`
|
||||
- Define `DeltaResult`, `ExitSignal`, `ExitType`, `TradePlan`, `SignalOutput`
|
||||
- All models must use Pydantic `BaseModel` with proper field constraints (`ge`, `le`)
|
||||
- _Requirements: 1.1, 2.7, 5.7, 6.9, 9.5, 10.1, 10.5_
|
||||
|
||||
- [x] 1.3 Implement `config.py` — `SignalEngineConfig` and sub-configs
|
||||
- Define `SignalEngineConfig` dataclass with all fields from the design
|
||||
- Define `HardFilterConfig`, `HeuristicConfig`, `ProbabilisticConfig`, `ExitConfig` as derived sub-configs
|
||||
- Implement `load_config()` that reads from `risk_configs` table + environment variables
|
||||
- Default `dual_pipeline_enabled` to `False` (fail-safe)
|
||||
- _Requirements: 13.1, 13.2, 13.3, 13.4, 13.5, 13.6, 13.7_
|
||||
|
||||
- [x] 1.4 Add `QUEUE_SIGNAL_ENGINE` to `services/shared/redis_keys.py`
|
||||
- Add `QUEUE_SIGNAL_ENGINE = "signal_engine"` constant
|
||||
- _Requirements: 11.1_
|
||||
|
||||
- [x] 1.5 Write property test for `SignalOutput` round-trip serialization
|
||||
- **Requirement 17.6: SignalOutput round-trip serialization**
|
||||
- Generate arbitrary valid `SignalOutput` instances with Hypothesis
|
||||
- Verify `SignalOutput.model_validate_json(output.model_dump_json())` produces equivalent object
|
||||
- File: `tests/test_pbt_signal_engine_models.py`
|
||||
- _Requirements: 10.5, 17.6_
|
||||
|
||||
- [x] 2. Input Normalizer and Hard Filter Engine
|
||||
- [x] 2.1 Implement `normalizer.py` — Input Normalizer
|
||||
- Implement `normalize_input(pool, ticker, config) -> NormalizedInput`
|
||||
- Fetch OHLCV bars from `market_data_bars` for M30, H1, H4, D, W, M timeframes
|
||||
- Fetch fundamental metrics (valuation_score, earnings_proximity_days) from company/trend data
|
||||
- Fetch macro context (macro_bias) from `macro_impact_records` and `global_events`
|
||||
- Fetch open position state from trading engine portfolio tables
|
||||
- Populate sentinel values (`None`, empty list) for unavailable data with logged warnings
|
||||
- Validate monotonically increasing timestamps within each timeframe series
|
||||
- _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5_
|
||||
|
||||
- [x] 2.2 Implement `hard_filter.py` — Hard Filter Engine
|
||||
- Implement `evaluate_hard_filters(normalized, config) -> HardFilterResult`
|
||||
- Check `macro_bias == -1.0` → SKIP with reason "macro_bias_negative"
|
||||
- Check `valuation_score < 0.3` → SKIP with reason "valuation_below_threshold"
|
||||
- Check `earnings_proximity_days <= 5` → SKIP with reason "earnings_block"
|
||||
- Record all triggered filter reasons (not just first)
|
||||
- Return `HardFilterResult` with `filtered: bool` and `reasons: list[str]`
|
||||
- _Requirements: 4.1, 4.2, 4.3, 4.4, 4.5, 4.6_
|
||||
|
||||
- [x] 2.3 Write property tests for hard filter engine
|
||||
- **Requirement 17.7: Hard filter determinism**
|
||||
- Generate arbitrary `NormalizedInput` with `macro_bias = -1.0` → always SKIP
|
||||
- Generate arbitrary `NormalizedInput` with `valuation_score < 0.3` → always SKIP
|
||||
- Generate arbitrary `NormalizedInput` with `earnings_proximity_days <= 5` → always SKIP
|
||||
- Verify these hold regardless of all other input values
|
||||
- File: `tests/test_pbt_signal_engine_hard_filter.py`
|
||||
- _Requirements: 4.1, 4.2, 4.3, 17.7_
|
||||
|
||||
- [x] 3. Checkpoint — Ensure all tests pass
|
||||
- Ensure all tests pass, ask the user if questions arise.
|
||||
|
||||
- [x] 4. Signal Library — Technical Signal Evaluators
|
||||
- [x] 4.1 Implement `signals/base.py` — SignalEvaluator protocol
|
||||
- Define `SignalEvaluator` protocol with `evaluate(bars, timeframe) -> SignalResult | None`
|
||||
- Define common helper functions for swing high/low detection, lookback validation
|
||||
- _Requirements: 2.6, 2.7_
|
||||
|
||||
- [x] 4.2 Implement `signals/fibonacci.py` — Fibonacci retracement evaluator
|
||||
- Implement `L(r) = SH - r·(SH - SL)` for ratios [0.236, 0.382, 0.5, 0.618, 0.786]
|
||||
- Detect swing high and swing low within the evaluation window
|
||||
- Produce signal strength based on proximity of current price to retracement levels
|
||||
- Return `None` with reason code when insufficient data
|
||||
- _Requirements: 2.1, 2.6, 2.7_
|
||||
|
||||
- [x] 4.3 Write property test for Fibonacci retracement formula
|
||||
- **Requirement 17.1: Fibonacci retracement bounds**
|
||||
- For all `r` in [0, 1] and all `SH > SL > 0`, verify `L(r)` is in [SL, SH]
|
||||
- File: `tests/test_pbt_signal_engine_fibonacci.py`
|
||||
- _Requirements: 2.1, 17.1_
|
||||
|
||||
- [x] 4.4 Implement `signals/ma_stack.py` — Moving average stack evaluator
|
||||
- Detect bullish alignment (MA_10 > MA_20 > MA_50 > MA_200)
|
||||
- Detect bearish alignment (MA_10 < MA_20 < MA_50 < MA_200)
|
||||
- Produce signal strength proportional to degree of alignment
|
||||
- Return `None` when insufficient bars for MA_200 calculation
|
||||
- _Requirements: 2.2, 2.6, 2.7_
|
||||
|
||||
- [x] 4.5 Implement `signals/rsi.py` — RSI evaluator
|
||||
- Implement standard 14-period RSI formula
|
||||
- Produce overbought signals (RSI > 70) and oversold signals (RSI < 30)
|
||||
- Scale strength by distance from threshold
|
||||
- Return `None` when fewer than 14 bars available
|
||||
- _Requirements: 2.3, 2.6, 2.7_
|
||||
|
||||
- [x] 4.6 Implement `signals/cup_handle.py` — Cup & Handle pattern detector
|
||||
- Identify cup formation (U-shaped price recovery) and handle (small consolidation)
|
||||
- Produce signal with confidence proportional to pattern completeness
|
||||
- Return `None` when insufficient data or no pattern detected
|
||||
- _Requirements: 2.4, 2.6, 2.7_
|
||||
|
||||
- [x] 4.7 Implement `signals/elliott_wave.py` — Elliott Wave detector
|
||||
- Identify impulse waves (5-wave structure) and corrective waves (3-wave structure)
|
||||
- Produce signal with current wave position and projected direction
|
||||
- Return `None` when insufficient data or ambiguous wave count
|
||||
- _Requirements: 2.5, 2.6, 2.7_
|
||||
|
||||
- [x] 5. Multi-Timeframe Confluence Engine
|
||||
- [x] 5.1 Implement `confluence.py` — Multi-Timeframe Engine
|
||||
- Implement `compute_confluence(signal_results, weights) -> list[ConfluenceSignal]`
|
||||
- Compute weighted confluence score: `C_confluence = Σ(w_tf · s_tf)`
|
||||
- Apply minimum confluence threshold: discard signals triggering on < 2 timeframes
|
||||
- Apply higher-timeframe anchor: discard signals without at least one of D, W, or M
|
||||
- Return `ConfluenceSignal` objects with active timeframes and per-timeframe strengths
|
||||
- _Requirements: 3.1, 3.2, 3.3, 3.4, 3.5, 3.6_
|
||||
|
||||
- [x] 5.2 Write property test for confluence score monotonicity
|
||||
- **Requirement 17.5: Confluence score monotonicity**
|
||||
- Verify that activating a signal on an additional timeframe with non-zero weight always increases or maintains the confluence score
|
||||
- File: `tests/test_pbt_signal_engine_confluence.py`
|
||||
- _Requirements: 3.6, 17.5_
|
||||
|
||||
- [x] 6. Checkpoint — Ensure all tests pass
|
||||
- Ensure all tests pass, ask the user if questions arise.
|
||||
|
||||
- [x] 7. Heuristic Pipeline (Pipeline A)
|
||||
- [x] 7.1 Implement `heuristic.py` — Heuristic Pipeline
|
||||
- Implement `run_heuristic_pipeline(normalized, confluence_signals, config) -> HeuristicResult`
|
||||
- Compute `S_total = S_company + S_macro + S_competitive` using existing `compute_signal_weight()`
|
||||
- Compute confidence from source count, extraction confidence, signal agreement, contradiction penalty
|
||||
- BUY verdict: confidence >= 0.70 AND S_total >= 1.2 AND valuation_score >= 0.5 AND macro_bias > 0 AND earnings_proximity_days > 5
|
||||
- WATCH verdict: confidence >= 0.55 AND BUY conditions not fully met
|
||||
- SKIP verdict: confidence < 0.55
|
||||
- Emit `HeuristicResult` with all required fields and reasoning
|
||||
- _Requirements: 5.1, 5.2, 5.3, 5.4, 5.5, 5.6, 5.7_
|
||||
|
||||
- [x] 7.2 Write unit tests for heuristic pipeline verdict logic
|
||||
- Test BUY threshold conditions
|
||||
- Test WATCH threshold conditions
|
||||
- Test SKIP conditions
|
||||
- Test edge cases at threshold boundaries
|
||||
- File: `tests/test_signal_engine_heuristic.py`
|
||||
- _Requirements: 5.4, 5.5, 5.6_
|
||||
|
||||
- [x] 8. Probabilistic Pipeline (Pipeline B) and Correlation Penalty
|
||||
- [x] 8.1 Implement `correlation.py` — Signal cluster classification and penalty
|
||||
- Define `SignalCluster` enum: MOMENTUM, STRUCTURE, VOLATILITY, FUNDAMENTALS
|
||||
- Implement `classify_signal(signal_type) -> SignalCluster`
|
||||
- Implement `apply_correlation_penalty(likelihood_ratios) -> list[LikelihoodRatio]`
|
||||
- Within-cluster decay: strongest LR at full weight, subsequent at 0.5^(n-1)
|
||||
- No penalty across different clusters
|
||||
- Single-signal clusters receive no penalty
|
||||
- _Requirements: 7.1, 7.2, 7.3, 7.4_
|
||||
|
||||
- [x] 8.2 Implement `probabilistic.py` — Probabilistic Pipeline
|
||||
- Implement `run_probabilistic_pipeline(normalized, confluence_signals, regime, config) -> ProbabilisticResult`
|
||||
- Initialize regime-based prior: bull=0.58, range=0.50, bear=0.42
|
||||
- Compute likelihood ratios: `P(sig|up) = h·s + (1-h)·(1-s)·0.5`, `LR = P(sig|up) / P(sig|down)`
|
||||
- Apply correlation penalty via `apply_correlation_penalty()`
|
||||
- Accumulate via log-odds: `logit(P_post) = logit(P_prior) + Σ log(LR_i)`
|
||||
- Compute Shannon entropy and apply entropy gating (H > 0.95 → SKIP)
|
||||
- Compute `EV_R = P_up · E[win_R] - (1 - P_up) · 1.0`
|
||||
- BUY: P_up >= 0.60 AND entropy <= 0.90 AND EV_R >= 1.5 AND macro_bias > 0 AND valuation_score >= 0.5
|
||||
- WATCH: P_up >= 0.55 AND entropy <= 0.95 AND BUY conditions not fully met
|
||||
- SKIP: all other cases
|
||||
- Use existing `classify_regime()` from `services/aggregation/regime.py`
|
||||
- _Requirements: 6.1, 6.2, 6.3, 6.4, 6.5, 6.6, 6.7, 6.8, 6.9, 14.1, 14.2, 14.3, 14.4, 14.5_
|
||||
|
||||
- [x] 8.3 Write property test for Bayesian log-odds round-trip
|
||||
- **Requirement 17.2: Bayesian log-odds update correctness**
|
||||
- Verify `logit(P_post) = logit(P_prior) + Σ log(LR_i)` round-trips correctly
|
||||
- Converting P_prior to logit, adding log-LRs, converting back via sigmoid produces valid probability in (0, 1)
|
||||
- File: `tests/test_pbt_signal_engine_bayesian.py`
|
||||
- _Requirements: 6.3, 17.2_
|
||||
|
||||
- [x] 8.4 Write property test for entropy gate
|
||||
- **Requirement 17.3: Entropy gate properties**
|
||||
- Verify Shannon entropy is maximized at P_up = 0.5
|
||||
- Verify entropy equals 0.0 at P_up = 0.0 or P_up = 1.0
|
||||
- Verify entropy is symmetric around 0.5
|
||||
- File: `tests/test_pbt_signal_engine_bayesian.py`
|
||||
- _Requirements: 6.4, 17.3_
|
||||
|
||||
- [x] 8.5 Write property test for signal correlation penalty
|
||||
- **Requirement 17.4: Correlation penalty reduces confidence**
|
||||
- Verify penalized posterior is always <= unpenalized posterior for any signal set with correlated signals
|
||||
- File: `tests/test_pbt_signal_engine_correlation.py`
|
||||
- _Requirements: 7.5, 17.4_
|
||||
|
||||
- [x] 8.6 Write property test for EV_R monotonicity
|
||||
- **Requirement 17.8: EV_R monotonically increasing with P_up**
|
||||
- Verify `EV_R = P_up · E[win_R] - (1 - P_up) · 1.0` is monotonically increasing with P_up for fixed E[win_R] > 0
|
||||
- File: `tests/test_pbt_signal_engine_bayesian.py`
|
||||
- _Requirements: 6.5, 17.8_
|
||||
|
||||
- [x] 9. Checkpoint — Ensure all tests pass
|
||||
- Ensure all tests pass, ask the user if questions arise.
|
||||
|
||||
- [x] 10. Exit Engine
|
||||
- [x] 10.1 Implement `exit_engine.py` — Exit Engine
|
||||
- Implement `evaluate_exits(positions, current_prices, config) -> list[ExitSignal]`
|
||||
- Check stop_loss hit → EXIT_FULL with reason "stop_hit"
|
||||
- Check target_1 hit → EXIT_HALF with reason "target_1_hit"
|
||||
- Check target_2 hit → EXIT_FULL with reason "target_2_hit"
|
||||
- Trailing stop: activate after EXIT_HALF at `current_price - ATR · trailing_multiplier`
|
||||
- Trailing stop ratchets upward only (never moves down)
|
||||
- Trailing stop hit → EXIT_FULL with reason "trailing_stop_hit"
|
||||
- _Requirements: 8.1, 8.2, 8.3, 8.4, 8.5, 8.6, 8.7_
|
||||
|
||||
- [x] 10.2 Write unit tests for exit engine
|
||||
- Test stop_loss trigger
|
||||
- Test target_1 partial exit
|
||||
- Test target_2 full exit
|
||||
- Test trailing stop activation and ratchet behavior
|
||||
- File: `tests/test_signal_engine_exit.py`
|
||||
- _Requirements: 8.1, 8.2, 8.3, 8.4, 8.5_
|
||||
|
||||
- [x] 11. Delta Analyzer and Output Formatter
|
||||
- [x] 11.1 Implement `delta.py` — Delta Analyzer
|
||||
- Implement `analyze_delta(heuristic, probabilistic, redis, ticker) -> DeltaResult`
|
||||
- Compute agreement flag (both verdicts identical)
|
||||
- Compute confidence delta: `|heuristic_confidence - probabilistic_P_up|`
|
||||
- Record disagreement reasons when verdicts differ
|
||||
- Track rolling 100-evaluation agreement rate in Redis
|
||||
- Log warning when agreement rate drops below 0.50
|
||||
- _Requirements: 9.1, 9.2, 9.3, 9.4, 9.5, 9.6_
|
||||
|
||||
- [x] 11.2 Implement `formatter.py` — Output Formatter
|
||||
- Implement `format_output(ticker, price, heuristic, probabilistic, delta, exit_signals, config) -> SignalOutput`
|
||||
- Both BUY → `dual_confirmed`, full position sizing
|
||||
- Probabilistic-only BUY → `probabilistic_only`, 50% position sizing
|
||||
- Heuristic-only BUY → standard position sizing
|
||||
- No BUY → no trade_plan (WATCH/SKIP persisted for analysis)
|
||||
- Implement `signal_output_to_recommendation(output) -> Recommendation`
|
||||
- Map `SignalOutput` to existing `Recommendation` schema for trading engine compatibility
|
||||
- Dual confirmed: confidence = max(heuristic_confidence, probabilistic_P_up)
|
||||
- Probabilistic only: confidence = probabilistic_P_up · 0.8 (20% haircut)
|
||||
- _Requirements: 10.1, 10.2, 10.3, 10.4, 10.5, 10.6, 12.1, 12.2, 12.3, 12.4, 12.5_
|
||||
|
||||
- [x] 11.3 Write unit tests for output formatter
|
||||
- Test dual_confirmed trade plan generation
|
||||
- Test probabilistic_only trade plan with 50% sizing
|
||||
- Test heuristic-only trade plan
|
||||
- Test no-BUY case (no trade_plan)
|
||||
- Test `signal_output_to_recommendation` mapping
|
||||
- File: `tests/test_signal_engine_formatter.py`
|
||||
- _Requirements: 10.2, 10.3, 10.4, 12.3, 12.4_
|
||||
|
||||
- [x] 12. Orchestrator, Persistence, and Main Entry Point
|
||||
- [x] 12.1 Implement `persistence.py` — Database persistence
|
||||
- Implement `persist_signal_output(pool, output) -> None`
|
||||
- Insert into `signal_engine_outputs` table
|
||||
- Log and continue on database errors (non-blocking)
|
||||
- _Requirements: 15.1, 15.4_
|
||||
|
||||
- [x] 12.2 Implement `worker.py` — Top-level orchestrator
|
||||
- Implement `evaluate_tick(pool, redis, ticker, config) -> SignalOutput | None`
|
||||
- Step 1: Normalize inputs (single fetch, shared reference)
|
||||
- Step 2: Evaluate exit conditions for open positions
|
||||
- Step 3: Run hard filters (short-circuit if filtered)
|
||||
- Step 4: Evaluate signals across timeframes via Signal Library
|
||||
- Step 5: Compute confluence
|
||||
- Step 6: Classify regime via existing `classify_regime()`
|
||||
- Step 7: Run both pipelines concurrently via `asyncio.gather` with exception handling
|
||||
- Step 8: Compute delta analysis
|
||||
- Step 9: Format output
|
||||
- Step 10: Persist to database and publish to Redis queue
|
||||
- Catch pipeline exceptions → SKIP verdict for failed pipeline, other continues
|
||||
- Measure and log wall-clock execution time per pipeline
|
||||
- _Requirements: 11.1, 11.2, 11.3, 11.4, 11.5, 11.6_
|
||||
|
||||
- [x] 12.3 Implement `main.py` — Entry point with asyncio event loop
|
||||
- Connect to PostgreSQL (asyncpg pool) and Redis (redis.asyncio)
|
||||
- Load config from `risk_configs` table
|
||||
- Log active configuration at startup
|
||||
- Poll `stonks:queue:signal_engine` queue indefinitely
|
||||
- Check `dual_pipeline_enabled` flag; if disabled, sleep and retry
|
||||
- On config read failure, default to disabled (fail-safe)
|
||||
- Support shadow mode (persist but don't forward to trading queue)
|
||||
- _Requirements: 13.1, 13.6, 13.7, 16.1, 16.6_
|
||||
|
||||
- [x] 12.4 Write integration tests for worker orchestration
|
||||
- Test full tick evaluation with mocked DB/Redis
|
||||
- Test pipeline failure isolation (one fails, other completes)
|
||||
- Test hard filter short-circuit
|
||||
- Test shadow mode behavior
|
||||
- File: `tests/test_signal_engine_worker.py`
|
||||
- _Requirements: 11.3, 16.6_
|
||||
|
||||
- [x] 13. Checkpoint — Ensure all tests pass
|
||||
- Ensure all tests pass, ask the user if questions arise.
|
||||
|
||||
- [x] 14. Database migration and infrastructure
|
||||
- [x] 14.1 Create database migration `infra/migrations/039_signal_engine_outputs.sql`
|
||||
- Create `signal_engine_outputs` table per the design schema
|
||||
- Create index on `(ticker, evaluated_at)` for per-ticker time-range queries
|
||||
- Create index on `evaluated_at` for global time-range queries
|
||||
- Create index on `(heuristic_verdict, probabilistic_verdict)` for verdict filtering
|
||||
- _Requirements: 15.1, 15.2, 15.3_
|
||||
|
||||
- [x] 14.2 Add signal engine service to Helm chart
|
||||
- Add `signalEngine` entry to `infra/helm/stonks-oracle/values.yaml`
|
||||
- Configure: replicas=1, command=`python -m services.signal_engine.main`, tier=processing
|
||||
- Set resource requests/limits per design (100m/128Mi → 500m/256Mi)
|
||||
- Reference existing secrets: `stonks-core-secrets`, `stonks-market-secrets`
|
||||
- _Requirements: 11.1, 13.1_
|
||||
|
||||
- [x] 15. Trading engine integration and backward compatibility
|
||||
- [x] 15.1 Wire signal engine output to trading engine queue
|
||||
- Publish `SignalOutput` (mapped to `Recommendation`) to `stonks:queue:trading_decisions`
|
||||
- Only publish when at least one pipeline produces BUY verdict
|
||||
- WATCH/SKIP verdicts persisted for analysis but not forwarded
|
||||
- Ensure trading engine can consume without modification via `signal_output_to_recommendation()`
|
||||
- _Requirements: 12.1, 12.2, 12.5, 16.2_
|
||||
|
||||
- [x] 15.2 Ensure backward compatibility with existing pipeline
|
||||
- Verify `dual_pipeline_enabled=false` means signal engine does not run
|
||||
- Verify existing aggregation pipeline operates unchanged when flag is off
|
||||
- Reuse existing `WeightedSignal`, `BayesianPosterior`, `RegimeClassification` (import, don't duplicate)
|
||||
- Reuse existing `compute_signal_weight`, `compute_bayesian_posterior`, `classify_regime` functions
|
||||
- No modifications to existing tables (new migration only adds new table)
|
||||
- _Requirements: 16.1, 16.2, 16.3, 16.4, 16.5_
|
||||
|
||||
- [x] 16. Final checkpoint — Ensure all tests pass
|
||||
- Ensure all tests pass, ask the user if questions arise.
|
||||
|
||||
## Notes
|
||||
|
||||
- Tasks marked with `*` are optional and can be skipped for faster MVP
|
||||
- Each task references specific requirements for traceability
|
||||
- Checkpoints ensure incremental validation between major phases
|
||||
- Property-based tests use Hypothesis with `@settings(max_examples=100)` per project conventions
|
||||
- PBT test files are prefixed `test_pbt_*` per project conventions
|
||||
- The service reuses existing math functions from `services/aggregation/` — no reimplementation
|
||||
- All configuration is loaded from `risk_configs` table with fail-safe defaults
|
||||
- Shadow mode allows running alongside existing pipeline without affecting trading decisions
|
||||
Reference in New Issue
Block a user