feat: implement dual-pipeline signal engine service

New service at services/signal_engine/ implementing concurrent heuristic
(deterministic scoring) and probabilistic (Bayesian inference) pipelines
that evaluate technical signals across 6 timeframes (M30-M) and produce
independent BUY/WATCH/SKIP verdicts per ticker per evaluation tick.

Components:
- Input Normalizer: multi-source data assembly with sentinel fallbacks
- Signal Library: Fibonacci, MA Stack, RSI, Cup & Handle, Elliott Wave
- Multi-Timeframe Confluence Engine: weighted scoring with D/W/M anchors
- Hard Filter Engine: macro_bias, valuation, earnings proximity gating
- Heuristic Pipeline: S_total scoring with confidence-gated verdicts
- Probabilistic Pipeline: Bayesian log-odds with regime priors, entropy
  gating, EV_R calculation, and signal correlation penalty
- Exit Engine: stop-loss, targets, trailing ATR-based stops
- Delta Analyzer: pipeline agreement tracking with rolling Redis metrics
- Output Formatter: SignalOutput contract + Recommendation schema mapping
- Worker orchestrator: concurrent pipelines with failure isolation
- Main entry point: queue polling with fail-safe config loading

Infrastructure:
- Migration 039: signal_engine_outputs table with 3 indexes
- Helm chart: signalEngine service entry (processing tier)
- Redis key: QUEUE_SIGNAL_ENGINE constant

Tests: 390 tests (unit + property-based) covering all components
Config: dual_pipeline_enabled=false by default (safe rollout)

docs/services.md

@@ -41,6 +41,7 @@ All queues use the `stonks:queue:<name>` key pattern (configurable via `DEPLOY_S
 | `recommendation` | `stonks:queue:recommendation` | Aggregation | Recommendation |
 | `broker_orders` | `stonks:queue:broker_orders` | Trading Engine, Trading API | Broker Adapter |
 | `lake_publish` | `stonks:queue:lake_publish` | Various services | Lake Publisher |
+| `report_generation` | `stonks:queue:report_generation` | Scheduler | Scheduler (inline consumer) |
 
 ### Queue Message Schemas
 
@@ -131,11 +132,20 @@ All queues use the `stonks:queue:<name>` key pattern (configurable via `DEPLOY_S
 }
 ```
 
+**Report Generation Job** (`stonks:queue:report_generation`):
+```json
+{
+  "report_type": "daily | weekly",
+  "period_start": "2025-01-01",
+  "period_end": "2025-01-01"
+}
+```
+
 ---
 
 ## 1. Scheduler
 
-**Purpose**: Triggers ingestion cycles for tracked companies and sources on a configurable cadence. Polls the symbol registry for active companies and their configured sources, respects per-source polling intervals and backoff windows, coordinates rate limits across source types, and enqueues ingestion jobs for downstream workers. Also runs periodic maintenance: stale document recovery, failed extraction retries, and data retention cleanup.
+**Purpose**: Triggers ingestion cycles for tracked companies and sources on a configurable cadence. Polls the symbol registry for active companies and their configured sources, respects per-source polling intervals and backoff windows, coordinates rate limits across source types, and enqueues ingestion jobs for downstream workers. Also runs periodic maintenance: stale document recovery, failed extraction retries, data retention cleanup, periodic aggregation re-runs, and automated report generation (daily/weekly).
 
 **Entry Point**: `services.scheduler.app`
 
@@ -176,12 +186,16 @@ All queues use the `stonks:queue:<name>` key pattern (configurable via `DEPLOY_S
 | `recommendations` | Write (delete) | Retention cleanup |
 | `order_events` | Write (delete) | Retention cleanup |
 | `model_performance_metrics` | Write (delete) | Retention cleanup |
+| `ingestion_runs` | Write (delete) | Retention cleanup |
+| `trading_reports` | Write | Report generation storage |
 
 ### Redis Queues
 
 | Direction | Queue | Purpose |
 |---|---|---|
 | Publish | `stonks:queue:ingestion` | Enqueue ingestion jobs for due sources |
+| Publish | `stonks:queue:aggregation` | Periodic aggregation re-runs |
+| Publish/Consume | `stonks:queue:report_generation` | Enqueue and consume report generation jobs |
 | Read | `stonks:pipeline:enabled` | Pipeline toggle (skip cycle if `"0"`) |
 | Read/Write | `stonks:lock:scheduler_cycle` | Distributed lock for single-writer |
 | Read/Write | `stonks:ratelimit:*` | Per-source-type and global Polygon rate limits |
@@ -195,6 +209,8 @@ All queues use the `stonks:queue:<name>` key pattern (configurable via `DEPLOY_S
 - **Stale document recovery**: Every ~5 minutes, re-enqueues documents stuck in `parsed` status for >240 minutes.
 - **Failed extraction retry**: Every ~10 minutes, re-enqueues `extraction_failed` documents older than 60 minutes.
 - **Data retention cleanup**: Every ~25 minutes, deletes old rows from 10 tables with configurable retention windows (14–90 days).
+- **Periodic aggregation**: Re-enqueues aggregation jobs for all active tickers to keep trend summaries fresh.
+- **Report generation**: Enqueues daily and weekly report jobs on schedule; consumes them inline via `process_report_job` with retry logic (3 attempts, exponential backoff 30s/60s/120s).
 
 ---
 
@@ -281,7 +297,7 @@ None — this service is purely HTTP-driven.
 ### MinIO Buckets
 
 - `stonks-raw-market` — Raw market data JSON
-- `stonks-raw-news` — Raw news article JSON
+- `stonks-raw-news` — Raw news article JSON (also used for macro news)
 - `stonks-raw-filings` — Raw SEC filing data
 - `stonks-normalized` — Normalized text (written by parser)
 
@@ -296,6 +312,13 @@ None — this service is purely HTTP-driven.
 | `broker` | `AlpacaBrokerAdapter` | Alpaca |
 | `macro_news` | `MacroNewsAdapter` | Polygon.io |
 
+### Key Behaviors
+
+- Macro news jobs (`source_type=macro_news`) may lack a `company_id` — the worker handles this gracefully
+- Macro news documents are typed as `macro_event` so the parser routes them to the macro classification queue
+- Duplicate documents detected via content hash are linked to the current company (except for `macro_news`)
+- Tracks `last_published_at` per source to fetch only newer articles on subsequent runs
+
 ---
 
 ## 4. Parser
@@ -349,7 +372,7 @@ None — this service is purely HTTP-driven.
 
 ## 5. Extractor
 
-**Purpose**: Performs LLM-based intelligence extraction from documents using Ollama. Handles two pipelines: (1) standard document extraction producing `DocumentIntelligence` with per-company impact records, and (2) macro event classification producing `GlobalEventSchema` with company-level macro impact interpolation. Supports AI agent configuration with variant-based A/B testing.
+**Purpose**: Performs LLM-based intelligence extraction from documents using Ollama or a remote vLLM inference server. Handles two pipelines: (1) standard document extraction producing `DocumentIntelligence` with per-company impact records, and (2) macro event classification producing `GlobalEventSchema` with company-level macro impact interpolation. Supports AI agent configuration with variant-based A/B testing and provider routing (Ollama or vLLM).
 
 **Entry Point**: `services.extractor.main`
 
@@ -363,9 +386,16 @@ None — this service is purely HTTP-driven.
 | `REDIS_*` | _(see shared)_ | Redis connection |
 | `MINIO_*` | _(see shared)_ | MinIO connection |
 | `OLLAMA_BASE_URL` | `http://localhost:11434` | Ollama API endpoint |
-| `OLLAMA_MODEL` | `qwen3.5:9b` | Default LLM model |
+| `OLLAMA_MODEL` | `qwen3.5:9b` | Default Ollama model |
 | `OLLAMA_TIMEOUT` | `120` | Request timeout (seconds) |
 | `OLLAMA_MAX_RETRIES` | `2` | Max retry attempts |
+| `VLLM_BASE_URL` | `http://192.168.42.254:8000` | vLLM inference server endpoint |
+| `VLLM_MODEL` | `RedHatAI/Qwen3.6-35B-A3B-NVFP4` | Default vLLM model |
+| `VLLM_TIMEOUT` | `120` | vLLM request timeout (seconds) |
+| `VLLM_MAX_RETRIES` | `2` | vLLM max retry attempts |
+| `VLLM_MAX_TOKENS` | `4096` | vLLM max output tokens |
+| `VLLM_TEMPERATURE` | `0.7` | vLLM sampling temperature |
+| `VLLM_API_KEY` | _(empty)_ | Optional API key for authenticated vLLM deployments |
 | `MACRO_CONFIDENCE_THRESHOLD` | `0.4` | Minimum confidence for macro event inclusion |
 | `LOG_LEVEL` | `INFO` | Logging level |
 
@@ -395,6 +425,7 @@ None — this service is purely HTTP-driven.
 
 ### Key Behaviors
 
+- **LLM provider routing**: The `AgentConfigResolver` resolves agent configuration from the DB, including a `model_provider` field (`"ollama"` or `"vllm"`). The `build_llm_client` factory returns the appropriate client (`OllamaClient` or `VLLMClient`).
 - Alternates between macro and extraction queues (1 macro per 3 jobs) to prevent starvation
 - Resolves agent configuration from DB with 60-second TTL cache (`AgentConfigResolver`)
 - Supports separate models for document extraction and event classification
@@ -565,7 +596,7 @@ None — this service is purely HTTP-driven.
 | `risk_tier_history` | Read/Write | Risk tier change audit trail |
 | `circuit_breaker_events` | Read/Write | Circuit breaker trigger/reset events |
 | `positions` | Read | Current open positions |
-| `position_stop_levels` | Read/Write | Stop-loss and take-profit levels |
+| `position_stop_levels` | Read/Write | Stop-loss and take-profit levels per position |
 | `orders` | Read | Order history for dedup |
 | `backtest_runs` | Read/Write | Backtest configuration and results |
 | `backtest_trades` | Read/Write | Individual trades within a backtest |
@@ -652,7 +683,7 @@ None — called synchronously by the broker adapter and via HTTP.
 | `positions` | Write (upsert) | Sync positions from Alpaca |
 | `broker_accounts` | Write (upsert) | Register/update broker account |
 | `daily_risk_snapshots` | Read | Daily portfolio state for risk evaluation |
-| `risk_configs` | Read | Active risk configuration |
+| `risk_configs` | Read | Active risk configuration for order evaluation |
 | `approval_requests` | Write | Create approval requests for gated orders |
 | `audit_events` | Write | Full audit trail |
 
@@ -728,7 +759,7 @@ None — called synchronously by the broker adapter and via HTTP.
 
 ## 12. Query API
 
-**Purpose**: Read-only FastAPI service for analytics, evidence drill-down, and admin controls. Serves the React dashboard and external integrations with endpoints for companies, documents, trends, recommendations, orders, positions, portfolio metrics, global events, macro impacts, competitive signals, trend projections, AI agents, dead-letter queues, pipeline control, SQL explorer, saved queries, audit trail, DevOps metrics, and Prometheus metrics.
+**Purpose**: Read-only FastAPI service for analytics, evidence drill-down, and admin controls. Serves the React dashboard and external integrations with endpoints for companies, documents, trends, recommendations, orders, positions, portfolio metrics, global events, macro impacts, competitive signals, trend projections, AI agents, dead-letter queues, pipeline control, SQL explorer, saved queries, audit trail, DevOps metrics, Prometheus metrics, model validation, and trading reports.
 
 **Entry Point**: `services.api.app` (FastAPI)
 
@@ -745,6 +776,7 @@ None — called synchronously by the broker adapter and via HTTP.
 | `TRINO_PORT` | `8080` | Trino port |
 | `TRINO_CATALOG` | `lakehouse` | Trino catalog |
 | `TRINO_SCHEMA` | `stonks` | Trino schema |
+| `TRINO_ICEBERG_CATALOG` | `iceberg` | Trino Iceberg catalog |
 | `LOG_LEVEL` | `INFO` | Logging level |
 
 ### Database Tables
@@ -757,9 +789,9 @@ The Query API reads from nearly all tables in the database, including:
 | `sources` | Source configurations |
 | `documents`, `document_company_mentions` | Document timelines |
 | `document_intelligence`, `document_impact_records` | Intelligence extraction results |
-| `trend_windows`, `trend_history`, `trend_projections` | Trend summaries and projections |
+| `trend_windows`, `trend_history`, `trend_projections`, `trend_evidence` | Trend summaries and projections |
 | `recommendations`, `recommendation_evidence` | Recommendation history with evidence |
-| `risk_evaluations` | Risk evaluation results |
+| `risk_evaluations`, `risk_configs` | Risk evaluation results and configuration |
 | `orders`, `order_events` | Order history and lifecycle |
 | `positions`, `portfolio_snapshots` | Portfolio state |
 | `global_events`, `macro_impact_records` | Macro event data |
@@ -768,6 +800,13 @@ The Query API reads from nearly all tables in the database, including:
 | `audit_events` | Audit trail |
 | `market_snapshots` | Market price data |
 | `watchlists`, `watchlist_members` | Watchlist data |
+| `ingestion_runs` | Ingestion throughput and source health |
+| `model_performance_metrics` | Model quality metrics |
+| `prediction_snapshots`, `prediction_outcomes` | Model validation and calibration |
+| `trading_decisions` | Trading decision history |
+| `trading_reports` | Generated daily/weekly reports |
+| `approval_requests` | Pending approval workflow |
+| `symbol_lockouts` | Active trading lockouts per symbol |
 
 ### Redis Queues
 
@@ -776,15 +815,22 @@ The Query API reads from nearly all tables in the database, including:
 | Read/Write | `stonks:pipeline:enabled` | Pipeline toggle control |
 | Read | `stonks:queue:*` | Queue depth monitoring for DLQ and DevOps metrics |
 | Read | `stonks:dlq:*` | Dead-letter queue inspection and replay |
+| Read | `stonks:ratelimit:*` | Rate limit status monitoring |
 
 ### Key Behaviors
 
 - Exposes `/metrics` endpoint for Prometheus scraping
 - Trace context propagation via `x-trace-id` header middleware
-- SQL explorer endpoint for ad-hoc Trino queries
+- SQL explorer endpoint for ad-hoc Trino queries (`/analytics/query`)
+- PostgreSQL schema explorer (`/pg/schema`, `/pg/query`)
 - Dead-letter queue management (list, inspect, replay)
 - Pipeline control (enable/disable via Redis toggle)
 - Saved queries with CRUD operations
+- Macro and competitive layer toggle endpoints
+- Model validation endpoints (summary, calibration, IC by horizon, gate status, attribution)
+- Trading report listing and retrieval
+- SSE pipeline health stream (`/pipeline/stream`)
+- Market price backfill endpoints
 
 ---
 
@@ -1042,6 +1088,67 @@ All services load configuration from environment variables via `services/shared/
 | `OLLAMA_MODEL` | `qwen3.5:9b` | Default model |
 | `OLLAMA_TIMEOUT` | `120` | Request timeout (seconds) |
 | `OLLAMA_MAX_RETRIES` | `2` | Max retry attempts |
+| `OLLAMA_RETRY_BASE_DELAY` | `1.0` | Base delay between retries (seconds) |
+| `OLLAMA_RETRY_MAX_DELAY` | `10.0` | Maximum delay between retries (seconds) |
+| `OLLAMA_RETRY_BACKOFF_MULTIPLIER` | `2.0` | Backoff multiplier |
+
+### vLLM
+
+| Variable | Default | Description |
+|---|---|---|
+| `VLLM_BASE_URL` | `http://192.168.42.254:8000` | vLLM inference server endpoint |
+| `VLLM_MODEL` | `RedHatAI/Qwen3.6-35B-A3B-NVFP4` | Default vLLM model |
+| `VLLM_TIMEOUT` | `120` | Request timeout (seconds) |
+| `VLLM_MAX_RETRIES` | `2` | Max retry attempts |
+| `VLLM_MAX_TOKENS` | `4096` | Max output tokens |
+| `VLLM_TEMPERATURE` | `0.7` | Sampling temperature |
+| `VLLM_API_KEY` | _(empty)_ | Optional API key for authenticated deployments |
+| `VLLM_RETRY_BASE_DELAY` | `1.0` | Base delay between retries (seconds) |
+| `VLLM_RETRY_MAX_DELAY` | `10.0` | Maximum delay between retries (seconds) |
+| `VLLM_RETRY_BACKOFF_MULTIPLIER` | `2.0` | Backoff multiplier |
+
+### Trino
+
+| Variable | Default | Description |
+|---|---|---|
+| `TRINO_HOST` | `localhost` | Trino host |
+| `TRINO_PORT` | `8080` | Trino port |
+| `TRINO_CATALOG` | `lakehouse` | Trino catalog |
+| `TRINO_SCHEMA` | `stonks` | Trino schema |
+| `TRINO_ICEBERG_CATALOG` | `iceberg` | Trino Iceberg catalog |
+
+### Market Data
+
+| Variable | Default | Description |
+|---|---|---|
+| `MARKET_DATA_API_KEY` | _(empty)_ | Polygon.io API key |
+| `MARKET_DATA_BASE_URL` | `https://api.polygon.io` | Polygon base URL |
+| `MARKET_DATA_PROVIDER` | `polygon` | Market data provider |
+
+### Broker
+
+| Variable | Default | Description |
+|---|---|---|
+| `BROKER_MODE` | `paper` | Trading mode (`paper` or `live`) |
+| `BROKER_PROVIDER` | `alpaca` | Broker provider |
+| `BROKER_API_KEY` | _(none)_ | Alpaca API key |
+| `BROKER_API_SECRET` | _(none)_ | Alpaca API secret |
+| `BROKER_BASE_URL` | _(none)_ | Alpaca base URL |
+
+### Retention
+
+| Variable | Default | Description |
+|---|---|---|
+| `RETENTION_RAW_MARKET_DAYS` | `90` | Raw market data retention (days) |
+| `RETENTION_RAW_NEWS_DAYS` | `180` | Raw news data retention (days) |
+| `RETENTION_RAW_FILINGS_DAYS` | `365` | Raw filings retention (days) |
+| `RETENTION_NORMALIZED_DAYS` | `180` | Normalized text retention (days) |
+| `RETENTION_LLM_PROMPTS_DAYS` | `365` | LLM prompt retention (days) |
+| `RETENTION_LLM_RESULTS_DAYS` | `365` | LLM result retention (days) |
+| `RETENTION_LAKEHOUSE_DAYS` | `730` | Lakehouse data retention (days) |
+| `RETENTION_AUDIT_DAYS` | `730` | Audit log retention (days) |
+| `RETENTION_CLEANUP_INTERVAL_HOURS` | `24` | Cleanup interval (hours) |
+| `RETENTION_BATCH_SIZE` | `1000` | Rows deleted per batch |
 
 ### Observability