feat: Intelligence Pipeline v3 — full implementation

Multi-stage evidence-grounded inference architecture replacing the
monolithic 9B model extraction pipeline. CPU-first specialist services
handle routine extraction while the 9B vLLM model is preserved for
semantic adjudication of ambiguous cases.

Key components:
- Capability-aware inference gateway (OpenAI-compatible + Ollama)
- Endpoint registry with DB migrations and REST API
- Sentence-aware document segmenter (property tests)
- Deterministic financial parsing with offset integrity
- Symbol resolution with ambiguity detection
- Specialist service (GLiNER2, dynamic batching, K8s deployment)
- Company-specific sentiment (FinBERT, calibration)
- Retrieval-based novelty and duplicate detection
- Confidence calibration pipeline
- Deterministic routing engine (property tests)
- 9B adjudication layer with VRAM gating
- Stock-specific impact model (features, labels, baseline, trained)
- Pipeline orchestrator (state machine, queues, leases, feature flags)
- Bounded parallelism (async workers, semaphore, load shedding)
- Observability (tracing, metrics, alerts)
- Compatibility adapter (v3→v2 golden mapping tests)
- Shadow/canary promotion framework
- Active learning and fine-tuning pipeline

Test results: 1,161 tests pass, ruff lint clean.
All 282 spec tasks completed.

.kiro/specs/ops-pipeline-fixes/tasks.md

@@ -0,0 +1,69 @@
+# Implementation Plan: ops-pipeline-fixes
+
+## Overview
+
+Fix 10 operational bugs preventing the validation/calibration feedback loop from functioning and degrading ingestion throughput. Changes span scheduler (rate limiting + periodic tasks), aggregation worker (config query), broker service (rejection reason), prediction snapshot (price fallback), and Helm chart (dead pod removal).
+
+## Tasks
+
+- [x] 1. Fix Polygon global rate limit — In `services/scheduler/app.py`, replace `POLYGON_GLOBAL_RATE_LIMIT: int = 45` with `POLYGON_GLOBAL_RATE_LIMIT: int = int(os.getenv("POLYGON_GLOBAL_RATE_LIMIT", "5"))` to make it env-configurable and default to the free-tier limit
+    - **Validates: Bugfix 2.1; Regression 3.1, 3.7**
+
+- [x] 2. Fix v3_engine_enabled query — In `services/aggregation/worker.py`, replace `_V3_ENGINE_FLAG_QUERY` from `SELECT value FROM risk_configs WHERE key = 'v3_engine_enabled'` to `SELECT config->>'v3_engine_enabled' AS enabled FROM risk_configs WHERE name = 'default' AND active = TRUE LIMIT 1`, and rewrite `_read_v3_flag()` to parse the returned string (checking for "true"/"1"/"yes"), returning False for NULL/missing/error
+    - **Validates: Bugfix 2.2; Regression 3.2**
+
+- [x] 3. Add validation cycle constant and counter — In `services/scheduler/app.py`, add `VALIDATION_CYCLE_INTERVAL = int(os.getenv("VALIDATION_CYCLE_INTERVAL", "240"))` constant and `validation_counter = 0` initialization in `main()`
+    - **Validates: Bugfix 2.3, 2.4**
+
+- [x] 4. Implement run_validation_cycle function — In `services/scheduler/app.py`, implement `run_validation_cycle(pool)` that calls `evaluate_matured_predictions(pool)` followed by `compute_and_store_metric_snapshots(pool)`, with try/except logging for each and skipping metrics if outcomes fail
+    - **Validates: Bugfix 2.3, 2.4, 2.5; Regression 3.3**
+
+- [x] 5. Wire validation cycle into main loop — In `services/scheduler/app.py` main loop, add the counter increment and conditional call to `run_validation_cycle(pool)` after the existing `report_schedule_counter` block
+    - **Validates: Bugfix 2.3, 2.4, 2.5**
+
+- [x] 6. Add snapshot cycle constant and counter — In `services/scheduler/app.py`, add `SNAPSHOT_CYCLE_INTERVAL = int(os.getenv("SNAPSHOT_CYCLE_INTERVAL", "240"))` constant and `snapshot_counter = 0` initialization in `main()`
+    - **Validates: Bugfix 2.6, 2.7**
+
+- [x] 7. Implement maybe_capture_daily_snapshots function — In `services/scheduler/app.py`, implement `maybe_capture_daily_snapshots(pool)` that checks time (after 16:30 ET), checks idempotency (no existing row for today), queries positions table for portfolio value/unrealized PnL, and inserts into `portfolio_snapshots` and `daily_risk_snapshots`
+    - **Validates: Bugfix 2.6, 2.7; Regression 3.10**
+
+- [x] 8. Wire snapshot cycle into main loop — In `services/scheduler/app.py` main loop, add the counter increment and conditional call to `maybe_capture_daily_snapshots(pool)` after the validation counter block
+    - **Validates: Bugfix 2.6, 2.7**
+
+- [x] 9. Add prediction price fallback — In `services/validation/prediction_snapshot.py`, after the primary market_snapshots price lookup returns NULL for `price_at_prediction`, add a fallback query to positions table: `SELECT current_price FROM positions WHERE ticker = $1 AND current_price IS NOT NULL LIMIT 1`
+    - **Validates: Bugfix 2.8; Regression 3.4**
+
+- [x] 10. Scale down lake-publisher — In `infra/helm/stonks-oracle/values.yaml`, change the lake-publisher `replicas` from `1` to `0`
+    - **Validates: Bugfix 2.9; Regression 3.5**
+
+- [x] 11. Extend _INSERT_ORDER SQL — In `services/adapters/broker_service.py`, extend `_INSERT_ORDER` SQL to include `rejection_reason` and `rejected_at` as parameters $18 and $19, with COALESCE in the ON CONFLICT UPDATE clause to preserve existing values
+    - **Validates: Bugfix 2.10; Regression 3.6, 3.8**
+
+- [x] 12. Update persist_order parameters — In `services/adapters/broker_service.py`, update `persist_order()` to compute `rejection_reason = resp.error if resp.status == OrderStatus.REJECTED else None` and `rejected_at = now if resp.status == OrderStatus.REJECTED else None`, passing them as the final two parameters in the execute call
+    - **Validates: Bugfix 2.10; Regression 3.6, 3.8**
+
+- [x] 13. Lint and test — Run `.venv/bin/ruff check services/` and `.venv/bin/python -m pytest tests/ -x --tb=short -q` to verify no regressions
+    - **Validates: Regression 3.1–3.10**
+
+## Task Dependency Graph
+
+```json
+{
+  "waves": [
+    {"tasks": [1, 2, 9, 10]},
+    {"tasks": [3, 6, 11]},
+    {"tasks": [4, 7, 12]},
+    {"tasks": [5, 8]},
+    {"tasks": [13]}
+  ]
+}
+```
+
+Tasks 1, 2, 9, 10 are fully independent. Tasks 3/6/11 set up constants needed by 4/7/12. Tasks 5/8 wire into the main loop after their functions exist. Task 13 validates everything last.
+
+## Notes
+
+- No database migrations required — all tables already exist with correct columns
+- All scheduler changes use the existing counter-based periodic task pattern already established for cleanup, aggregation, and report tasks
+- Lazy imports in `run_validation_cycle` avoid circular imports and keep scheduler startup fast
+- The `maybe_capture_daily_snapshots` idempotency check prevents duplicate rows on scheduler restart