Commit Graph

22 Commits

Author SHA1 Message Date
Celes Renata 4e010bc048 feat: signal math upgrade — probabilistic, regime-aware scoring pipeline
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
Implement full probabilistic signal processing pipeline gated behind
probabilistic_scoring_enabled feature flag in risk_configs:

- Bayesian log-likelihood accumulator with Beta posterior and entropy
- Regime detector (trend-following, panic, mean-reversion, uncertainty)
- Source accuracy tracker with per-source historical prediction accuracy
- Sigmoid confidence gate replacing binary gate
- Information gain surprise weighting for rare events
- Adaptive recency decay with event-specific half-lives
- Regime multiplier replacing market context multiplier
- Weighted disagreement entropy for contradiction detection
- Multiplicative macro exposure with conditional integration
- Graph-distance attenuated competitive signal propagation
- Exponentially weighted momentum with volatility scaling
- Expected value recommendation gate

All changes backward-compatible: flag=false preserves exact current behavior.
New outputs stored in existing JSONB columns (no schema changes except
source_accuracy table via migration 034).

Tests: 26 property-based tests (14 correctness properties), 99 unit tests,
1789 total tests passing with zero regressions.
2026-04-29 11:41:48 +00:00
Celes Renata 8c3c1aab43 fix: pipeline stop now halts all workers and flushes queues
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
Workers (ingestion, parser, extractor, aggregation, recommendation,
broker, lake-publisher) now check the pipeline:enabled Redis flag on
each loop iteration and sleep when disabled.

The toggle endpoint flushes all pipeline queues on disable so queued
jobs don't resume when workers eventually check. Broker/trading queues
are excluded from flush to avoid dropping in-flight orders.
2026-04-29 07:59:35 +00:00
Celes Renata 49bff9de50 fix: delete stale trend_evidence before inserting to prevent duplicate accumulation
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
2026-04-28 19:45:15 +00:00
Celes Renata 82da8af02b fix: dedup recommendation queue at aggregation level — prevent duplicate ticker+window flooding 2026-04-17 22:11:59 +00:00
Celes Renata 5efccb1e03 fix: deduplicate evidence refs in trend summaries
Backend: assemble_trend_with_evidence now deduplicates document IDs
via dict.fromkeys() (the rollup code already did this, but the base
assembly didn't — same doc could appear multiple times from different
intelligence extractions).

Frontend: Trends.tsx deduplicates via Set before rendering as a safety
net for existing data already stored with duplicates.
2026-04-17 07:25:32 +00:00
Celes Renata e21f162e48 fix: dampen agreement factor by sample size in trend confidence to prevent low-evidence inflation
Agreement of 1-2 signals was inflating confidence to paper-eligible
levels (0.575) even with low credibility sources. Added log2-based
dampener that scales agreement contribution by unique source count,
saturating at n=7. Single signals now cap at 0.39 confidence,
2 signals at 0.49 — both correctly below paper threshold (0.50).
2026-04-17 03:41:39 +00:00
Celes Renata 7c589353f8 fix: blank company charts + competitor GUIDs instead of tickers
Trend charts blank:
- trend_windows uses upsert (1 row per ticker/window), so charts had
  at most 1 data point. Added trend_history table (migration 024) that
  appends every snapshot. New /api/trends/history endpoint serves the
  time series. Frontend now uses useTrendHistory for charts and
  useTrends for the latest summary card.

Competitor GUIDs:
- list_competitors query returned raw company_b_id UUIDs without
  joining companies table. Added LEFT JOIN with CASE to resolve the
  other company's ticker and legal_name. Updated Pydantic model to
  include enriched fields. Frontend fallback changed from truncated
  UUID to ticker/legal_name/Unknown.
2026-04-17 00:42:55 +00:00
Celes Renata 18eb150c75 fix: confidence formula now uses unique doc count + signal agreement instead of raw signal count — prevents 99.9% inflation 2026-04-16 23:48:05 +00:00
Celes Renata 6bab199159 fix: trend_windows now upserts instead of accumulating (7.5GB→4MB), add competitive signal retention cleanup 2026-04-16 14:32:24 +00:00
Celes Renata 5cc64498c0 fix: skip already-propagated docs in aggregation, limit to last hour + 10 docs instead of 50 2026-04-16 09:35:11 +00:00
Celes Renata f0887afd9b fix: cap competitive signals to 500 most recent per window to prevent 67k row aggregation bottleneck 2026-04-16 09:24:31 +00:00
Celes Renata e1792e06b9 fix: skip non-UUID pattern signal IDs in evidence persistence 2026-04-14 20:18:50 +00:00
Celes Renata d37a5732d0 fix: use timedelta instead of string for interval query params 2026-04-14 20:10:38 +00:00
Celes Renata c438d0d60a fix: competitor lookup uses ticker join instead of UUID comparison 2026-04-14 20:04:56 +00:00
Celes Renata d8ea58104c fix: lint errors (import sorting, unused vars) 2026-04-14 19:48:19 +00:00
Celes Renata f7a11d14ea feat: competitive intelligence & historical pattern matching layer 2026-04-14 19:42:48 +00:00
Celes Renata 181ed2b6cd phase 17: quote reserved word 'window' in aggregation SQL INSERT 2026-04-12 03:38:45 -07:00
Celes Renata 019eaa40d7 phase 17: fix datetime JSON serialization in aggregation worker market_context 2026-04-12 03:31:32 -07:00
Celes Renata 012b973bb7 phase 17: wire extractor→aggregation→recommendation queue chain, add company_id_map to extractor 2026-04-12 03:16:27 -07:00
Celes Renata 109440c91e phase 15: fix ruff lint errors across services 2026-04-11 12:10:01 -07:00
Celes Renata ce10afa034 phase 14-15: docker build validation and helm deployment 2026-04-11 11:59:45 -07:00
Celes Renata ebea70573b phase 0+1: project scaffold, k8s manifests, CI pipeline, steering, hooks, tests
- Repository structure for all services, infra, lakehouse, dashboards
- K8s manifests targeting stonks-oracle namespace with GHCR images
- Ingress via Traefik with ca-issuer TLS for internal services
- ConfigMap wired to existing cluster services (pg, redis, minio, ollama)
- GitHub Actions workflow for lint, test, multi-service container builds
- Dockerfile with build-arg CMD per service
- Makefile for local build/push/deploy
- Steering rules for TDD workflow, K8s conventions, project context
- Agent hooks for lint-on-save, test-on-save, k8s-validate, phase-commit
- Ruff linter config, all lint issues fixed
- 14 passing tests for schemas, config, redis keys
- PostgreSQL migrations, Trino catalogs, Superset config, MinIO lifecycle
2026-04-11 03:25:08 -07:00