admin/stonks-oracle
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
95 Commits 1 Branch 1 Tag
ff5055ee4e4c513a28e33acc0c2d72e104524464
Commit Graph
95 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Celes Renata ff5055ee4e fix: backtest replay field mapping and logging 
- Map DB 'id' field to 'recommendation_id' for evaluate_recommendation()
- Ensure confidence is cast to float (asyncpg may return Decimal)
- Add per-day logging showing rec count, act/skip, positions, pool balance
- Helps diagnose why backtests produce 0 trades
 2026-04-15 22:55:26 +00:00
Celes Renata 4501bbebd4 feat: add Polygon grouped daily endpoint for broad market data 
Two tiers of market data:
1. Per-ticker prev bars (existing 50 sources, 15-min cadence) for
   watchlist detail — trading decisions, stop-loss, position sizing
2. Grouped daily (new single source, once per day) for broad market
   context — correlation analysis, sector rotation, competitive intel

Changes:
- Add grouped_daily endpoint to PolygonMarketAdapter with auto date
  calculation (previous trading day, skip weekends)
- Add fetch_global_market_sources() to scheduler for sources without
  company_id, scheduled once daily (86400s cadence)
- Update _persist_market_items to use item-level ticker from T field
  and look up company_id dynamically for grouped daily bars
- Migration 020: make company_id nullable on sources and
  market_snapshots tables, add grouped daily source row
- Fix backtest replay to query market_snapshots data->>'c' for prices
 2026-04-15 22:38:18 +00:00
Celes Renata ea6c2b3f54 fix: market data rate limiting and backtest price lookup 
- Increase market_api polling cadence from 60s to 900s (15 min).
  The prev-day bar endpoint returns the same data all day, so polling
  every minute wastes API quota. 50 tickers at 15-min cadence = ~3.3
  req/min, well within the 5/min rate limit.
- Reduce market_api rate limit from 30/min to 5/min to match.
- Fix backtest replay to query market_snapshots with data->>'c' for
  close prices instead of nonexistent market_data.close_price column.
- Enrich backtest recommendations with prices from market_snapshots
  and sectors from companies table.
 2026-04-15 22:19:44 +00:00
Celes Renata 69eb366bf9 fix: backtest simulation uses ET timezone for trading window 
The simulated timestamp was 10:00 UTC (6:00 AM ET) which is outside
the trading window. Changed to 11:00 AM ET so backtested decisions
actually pass the trading window check.
 2026-04-15 21:48:32 +00:00
Celes Renata 70bad7709a feat: wire live decision loop and enable paper trading 
Phase 2 of the autonomous trading engine:

- Replace start()/stop() stubs with real async implementations
- Decision loop: polls recommendations from PostgreSQL, deduplicates
  via Redis, evaluates through the full pipeline, submits orders to
  stonks:queue:broker_orders
- Stop-loss monitor: fetches prices from Polygon API, checks crossings,
  submits immediate sell orders, safety sell after 15 min without data
- Performance loop: computes metrics every 5 min during market hours,
  persists daily snapshots at market close
- Risk tier scheduler: evaluates daily at 16:00 ET, persists tier changes
- Rebalance scheduler: evaluates Monday 09:45 ET, respects circuit breaker
- Notification dispatch: SNS + Gmail with rate limiting and retry
- Backtest replay: fetches historical data, simulates decisions, persists
- Real asyncpg/redis connections in FastAPI lifespan (graceful degradation)
- Migration 019: enable paper trading with conservative tier, 5 cap
- Added max_open_positions to TradingConfig with env var loading
- Phase 2 tasks added to autonomous-trading-engine spec
 2026-04-15 20:52:28 +00:00
Celes Renata c4b90a5224 fix: sidebar nav highlights both Trading Controls and Trading Engine 
When on /trading/engine, the /trading nav item also matched via
startsWith. Now checks if a more specific child route matches
first and uses exact match in that case.
 2026-04-15 20:07:31 +00:00
Celes Renata 78aee4c9e6 fix: add trading-engine to CI build matrix 
The service was added to the Helm chart but not to the GitHub Actions
build matrix, causing ImagePullBackOff on deploy.
 2026-04-15 19:55:34 +00:00
Celes Renata 3ff910433f fix: reject empty LLM classifications for global events 
When the LLM returns empty summary and no key facts, raise ValueError
so the retry logic kicks in instead of persisting an empty event.
Also strip whitespace from summary and filter empty key_facts entries.

Cleaned up 17 empty events from the database.
 2026-04-15 19:46:31 +00:00
Celes Renata 326c409d63 fix: use _global fallback for empty ticker in MinIO storage paths 
Macro news documents have no ticker, causing upload_normalized_text
and upload_parser_output to produce paths like parsed//2026/...
which MinIO rejects as XMinioInvalidObjectName. Use '_global' as
the path segment when ticker is empty, matching the existing
macro prefix pattern in upload_raw_document.
 2026-04-15 19:25:23 +00:00
Celes Renata cdf1cd06bc fix: bump GitHub Actions to Node 24 versions 
- actions/checkout v4 → v5
- actions/setup-python v5 → v6
- actions/setup-node v4 → v5
- docker/login-action v3 → v4
- docker/setup-buildx-action v3 → v4
- docker/build-push-action v6 → v7

Silences Node.js 20 deprecation warnings ahead of the
June 2026 forced migration deadline.
 2026-04-15 16:24:50 +00:00
Celes Renata 516731e69a fix: remove explicit type annotations on Recharts formatter callbacks 
TypeScript strict mode in CI rejects explicit parameter types on
Recharts formatter/tickFormatter callbacks. Use inference with
'as number' casts on the value instead. Also fix unsafe cast in
PortfolioComposition and handle possibly-undefined percent.
 2026-04-15 16:17:00 +00:00
Celes Renata 4ffde8cc06 feat: autonomous trading engine — full implementation 
- Database migration 018 with 13 tables for trading engine state
- Trading engine service (services/trading/) with 12 pure computation modules:
  position sizer, stop-loss manager, reserve pool, circuit breaker,
  risk tier controller, correlation matrix, tax lots, trading window,
  gradual entry, notifications, micro-trading, backtester
- Core TradingEngine with pre-trade evaluation pipeline and integration wiring
- FastAPI HTTP service with 14 endpoints (health, config, decisions, metrics, backtest)
- Performance tracker with Sharpe ratio, drawdown, profit factor computation
- 194 Python tests (165 property-based + 29 integration)
- Frontend: 13 TanStack Query hooks, 7 dashboard panels, tabbed Trading Engine page
- Helm chart entry, network policy, nginx proxy, ingress for trading-engine
- Shared infrastructure: enums, Redis keys, TradingConfig in AppConfig
 2026-04-15 16:12:22 +00:00
Celes Renata da86132f0c fix: add num_predict=16384 to prevent output truncation on large articles 2026-04-15 03:11:13 +00:00
Celes Renata b8a2cdc52a fix: fill default values for missing fields in truncated LLM output 2026-04-15 03:08:10 +00:00
Celes Renata 00044af993 fix: switch to think=false with json-repair — 20x faster extraction 2026-04-15 02:54:39 +00:00
Celes Renata 4f2ae23d42 fix: set num_predict=16384 so model has token budget for thinking + content 2026-04-15 01:47:00 +00:00
Celes Renata 46b069a748 fix: switch to non-streaming Ollama calls — streaming breaks thinking mode 2026-04-15 01:19:17 +00:00
Celes Renata ffe19eb23a fix: handle empty ticker in MinIO storage paths, clean up debug log 2026-04-15 00:39:53 +00:00
Celes Renata 8b5b692d3c fix: update stall timer during thinking phase to prevent premature stream abort 2026-04-15 00:06:49 +00:00
Celes Renata 01726af360 fix: remove think=false (Ollama bug #14645), bump max_tokens to 32k 2026-04-14 23:50:28 +00:00
Celes Renata b0fc207c1a docs: update steering docs with competitive layer, 50 companies, deployment lessons 2026-04-14 23:27:52 +00:00
Celes Renata 8d0526470d feat: expand seed to 50 companies with competitor relationships and macro source 2026-04-14 21:00:19 +00:00
Celes Renata 86808465de fix: remove explicit type annotations on Recharts callbacks for v3 compat 2026-04-14 20:40:31 +00:00
Celes Renata 916aaff9f3 feat: add trend history chart with strength/confidence/contradiction time series 2026-04-14 20:37:25 +00:00
Celes Renata e1792e06b9 fix: skip non-UUID pattern signal IDs in evidence persistence 2026-04-14 20:18:50 +00:00
Celes Renata d37a5732d0 fix: use timedelta instead of string for interval query params 2026-04-14 20:10:38 +00:00
Celes Renata c438d0d60a fix: competitor lookup uses ticker join instead of UUID comparison 2026-04-14 20:04:56 +00:00
Celes Renata 2a96c8338b fix: remove unused StatusBadge import in GlobalEvents.tsx 2026-04-14 19:53:40 +00:00
Celes Renata 4e191957b4 fix: remove duplicate closing tag in CompanyDetail.tsx 2026-04-14 19:50:53 +00:00
Celes Renata d8ea58104c fix: lint errors (import sorting, unused vars) 2026-04-14 19:48:19 +00:00
Celes Renata f7a11d14ea feat: competitive intelligence & historical pattern matching layer 2026-04-14 19:42:48 +00:00
Celes Renata b478022ba3 fix: data quality query and suppression fallback in recommendation worker 
- Fix _DATA_QUALITY_QUERY: remove nonexistent d.source_id/s.source_class,
  use d.source_type directly
- Fix LIMIT 1 applied after jsonb expansion by restructuring as CTE
- Fix fallback build_quality_context_from_summary returning empty
  source_types which always triggered LOW_SOURCE_DIVERSITY suppression
- Update test to reflect corrected fallback behavior
 2026-04-14 06:57:46 +00:00
Celes Renata 4fbddc307a fix(extractor): fallback for any unrecognized impact_horizon value 2026-04-12 16:27:37 -07:00
Celes Renata 6ae8aa779e fix(extractor): add underscore variants to impact_horizon normalizer 
Model returns long_term/short_term/medium_term instead of hyphenated versions
 2026-04-12 16:08:25 -07:00
Celes Renata cd782d1552 fix(extractor): streaming with guardrails + catalyst_type normalization 
- Switch Ollama calls from non-streaming to streaming with early termination
- Add loop detection, max token limit, and stall timeout guards
- Add catalyst_type alias normalizer to handle model hallucinations
- Add explicit enum values in extraction prompt for catalyst_type
- Add streaming config knobs to OllamaConfig
 2026-04-12 15:28:20 -07:00
Celes Renata 527be42f82 phase 17: restore script — per-bucket size reporting, full DB row count verification 2026-04-12 14:53:19 -07:00
Celes Renata 85e0ef7580 phase 17: backup script now reports per-bucket sizes and file counts 2026-04-12 14:46:47 -07:00
Celes Renata dd13045ca6 phase 17: fix backup/restore scripts — use postgres:18-alpine for pg_dump version match 2026-04-12 14:41:08 -07:00
Celes Renata 3f5b4adcec phase 17: add backup/restore scripts — PostgreSQL + MinIO → NFS 2026-04-12 14:21:59 -07:00
Celes Renata 6e2f174b19 phase 17: disable qwen3.5 thinking mode (think:false) to reduce latency and improve structured output 2026-04-12 12:35:24 -07:00
Celes Renata 45f0c03639 phase 17: add request-level URL logging to OllamaClient for proxy debugging 2026-04-12 12:32:44 -07:00
Celes Renata fd35e12d5e phase 17: switch Ollama to external proxy at 10.1.1.12:2701 2026-04-12 11:37:23 -07:00
Celes Renata 80e0f0976f phase 17: switch back to qwen3.5:9b-fast (Ollama restarted, model available again) 2026-04-12 11:05:42 -07:00
Celes Renata a3e8009fa9 phase 17: revert to qwen3.5:9b (9b-fast was removed from Ollama), add retry script 2026-04-12 10:58:53 -07:00
Celes Renata 109a2485cf phase 17: increase Ollama timeout to 300s for qwen3.5:9b-fast 32k context 2026-04-12 10:32:13 -07:00
Celes Renata 7ee1d0f050 phase 17: switch to qwen3.5:9b-fast (32k context), add queue management scripts 2026-04-12 10:19:28 -07:00
Celes Renata 1993bfdf3e phase 17: add extraction output normalization — clamp scores to 0-1, map impact_horizon alternatives 2026-04-12 10:15:38 -07:00
Celes Renata 608ccc8b68 phase 17: revert to qwen3.5:9b, keep improved prompt style 2026-04-12 10:06:13 -07:00
Celes Renata 66ed38bf18 phase 17: switch to gemma4:e4b, rewrite prompts for fill-the-fields style with forced ticker inclusion 2026-04-12 10:05:31 -07:00
Celes Renata 2e42310f07 phase 17: fix SEC EDGAR 403 — use descriptive User-Agent with contact email per fair access policy 2026-04-12 09:50:29 -07:00
Celes Renata 311d76dc0b phase 17: enrich SEC EDGAR filings with URLs, titles, dedupe by accession number, skip XML fragments 2026-04-12 09:42:12 -07:00
Celes Renata 28b3361833 phase 17: remove embedded JSON schema from user prompt (4.7KB saved), Ollama format param handles it 2026-04-12 09:28:28 -07:00
Celes Renata 57d0fc7d33 phase 17: pass all tracked tickers to extractor, soften prompt for macro-to-company relevance 2026-04-12 09:18:08 -07:00
Celes Renata 59f89d03d2 phase 17: enrich short parsed articles with Polygon description/keywords from raw payload 2026-04-12 08:52:46 -07:00
Celes Renata cd32c3e3fe phase 17: increase parser→extractor text limit from 8k to 32k chars 2026-04-12 08:37:29 -07:00
Celes Renata ffcc66ae0b phase 17: sync standalone k8s/trino.yaml with Helm template (native S3, s3.region) 2026-04-12 08:24:04 -07:00
Celes Renata 34787ad825 phase 17: fix Trino hive catalog — use native S3 filesystem, remove defunct hive.s3 props 2026-04-12 08:18:18 -07:00
Celes Renata 999648d90b phase 17: add s3.region to Trino catalog config for MinIO (fixes AWS SDK region error) 2026-04-12 08:16:14 -07:00
Celes Renata 4f2f113cda phase 17: fix text[]/varchar[] type mismatch in coverage-gaps SQL 2026-04-12 04:15:00 -07:00
Celes Renata d16e15c885 phase 17: quote reserved word 'window' in all SQL queries across recommendation worker and query API 2026-04-12 03:45:51 -07:00
Celes Renata 181ed2b6cd phase 17: quote reserved word 'window' in aggregation SQL INSERT 2026-04-12 03:38:45 -07:00
Celes Renata 019eaa40d7 phase 17: fix datetime JSON serialization in aggregation worker market_context 2026-04-12 03:31:32 -07:00
Celes Renata 48bf4f7e7e phase 17: extractor fetches normalized text from MinIO when not in job payload 2026-04-12 03:24:10 -07:00
Celes Renata 012b973bb7 phase 17: wire extractor→aggregation→recommendation queue chain, add company_id_map to extractor 2026-04-12 03:16:27 -07:00
Celes Renata 226cc3ff44 phase 17: switch Ollama model to qwen3.5:9b (available on cluster) 2026-04-12 03:10:49 -07:00
Celes Renata e4a1d2d69a phase 17: fix HTML parser NoneType attrs crash in boilerplate stripping 2026-04-12 03:03:07 -07:00
Celes Renata 264b83ea56 phase 17: fix Polygon article_url and published_utc field mapping in metadata persistence 2026-04-12 02:58:30 -07:00
Celes Renata 0ac4493bd4 phase 17: fix parser URL lookup from DB and extractor text field name mismatch 2026-04-12 02:54:23 -07:00
Celes Renata 67cdb0b8c8 phase 17: fix ruff lint error in scheduler import order 2026-04-12 02:47:47 -07:00
Celes Renata f2b9d6c00a phase 17: fix scheduler config parsing, worker entry points, and seed data for Polygon sources 2026-04-12 02:45:37 -07:00
Celes Renata 1410d324b7 phase 17: add vertical slice tasks for live pipeline activation 2026-04-11 20:43:47 -07:00
Celes Renata 37d5f9b01c update steering docs and hooks for current project state 2026-04-11 20:41:57 -07:00
Celes Renata 99e17be282 phase 16: fix env var fallback - use || instead of ?? for empty string 2026-04-11 20:06:13 -07:00
Celes Renata 4d0c38bba7 phase 16: vitest + MSW frontend tests, CI integration 2026-04-11 19:28:38 -07:00
Celes Renata 5758a704ec phase 16: fix UUID serialization in symbol registry responses 2026-04-11 19:22:13 -07:00
Celes Renata 6f5b2231a2 phase 16: add registry/risk nginx proxies, add company form, network policies 2026-04-11 19:12:07 -07:00
Celes Renata 4cd8961db6 phase 16: add dashboard network policy, allow query-api from dashboard 2026-04-11 18:20:48 -07:00
Celes Renata cc7014e33d phase 16: fix superset - trino driver in venv, psycopg2 metadata db, core secrets 2026-04-11 17:37:39 -07:00
Celes Renata 5f87cbe464 phase 16: custom superset image with trino driver, fix security context 2026-04-11 17:18:17 -07:00
Celes Renata afa627322a phase 16: fix ruff lint - move imports to top of file 2026-04-11 16:48:50 -07:00
Celes Renata 59da3fe89e phase 16: nginx-unprivileged on 8080, helm dashboard deployment 2026-04-11 16:37:59 -07:00
Celes Renata 1fcb79503e phase 16: fix TS strict mode errors, node 24, update steering docs 2026-04-11 16:35:50 -07:00
Celes Renata faccb0b8db phase 16: React dashboard with full platform control and analytics 2026-04-11 16:19:46 -07:00
Celes Renata 25e0e386b7 spec: add frontend dashboard requirements, design, and tasks (phase 16) 2026-04-11 14:46:30 -07:00
Celes Renata e7b2a5e67f phase 15: helm deployment complete, runbook, trino/superset fixes 2026-04-11 14:27:47 -07:00
Celes Renata fe3d6c0cb0 fix: trino iceberg catalog s3 props, superset port env override 2026-04-11 14:21:44 -07:00
Celes Renata 273a86e2bd fix: risk engine command points to services.risk.app, redis password, window quoting 2026-04-11 14:15:47 -07:00
Celes Renata 1c3a5f8285 fix: uncomment ruff in requirements.txt for CI 2026-04-11 12:58:30 -07:00
Celes Renata 5a2827d5ee fix: use python -m for ruff and pytest in CI 2026-04-11 12:56:40 -07:00
Celes Renata 7524aa5f2c phase 15: helm chart for stonks-oracle deployment 2026-04-11 12:21:49 -07:00
Celes Renata 109440c91e phase 15: fix ruff lint errors across services 2026-04-11 12:10:01 -07:00
Celes Renata ce10afa034 phase 14-15: docker build validation and helm deployment 2026-04-11 11:59:45 -07:00
Celes Renata 7394d241c9 phase 2: symbol registry validation, seed data, nix dev shell 
- Enhanced CompanyCreate with ticker format validation (1-10 uppercase letters)
- Enhanced SourceCreate with pydantic validators for source_type, access_policy, config URLs
- Added /health endpoint to symbol registry
- Seed data: 10 companies (AAPL, MSFT, NVDA, AMZN, GOOGL, JPM, JNJ, XOM, TSLA, META)
- Seed sources: Alpha Vantage (market), NewsAPI (news), SEC EDGAR (filings), Alpaca (paper trading)
- Seed watchlist: 'Starter 10' with all companies and aliases
- Added flake.nix dev shell (nixos-25.11) with Python 3.12, ruff, pytest, kubectl, helm
- 30 passing tests, lint clean, Docker build verified
 2026-04-11 03:41:41 -07:00
Celes Renata ebea70573b phase 0+1: project scaffold, k8s manifests, CI pipeline, steering, hooks, tests 
- Repository structure for all services, infra, lakehouse, dashboards
- K8s manifests targeting stonks-oracle namespace with GHCR images
- Ingress via Traefik with ca-issuer TLS for internal services
- ConfigMap wired to existing cluster services (pg, redis, minio, ollama)
- GitHub Actions workflow for lint, test, multi-service container builds
- Dockerfile with build-arg CMD per service
- Makefile for local build/push/deploy
- Steering rules for TDD workflow, K8s conventions, project context
- Agent hooks for lint-on-save, test-on-save, k8s-validate, phase-commit
- Ruff linter config, all lint issues fixed
- 14 passing tests for schemas, config, redis keys
- PostgreSQL migrations, Trino catalogs, Superset config, MinIO lifecycle
 2026-04-11 03:25:08 -07:00
Celes Renata 8cfc4f423b initial commit 2026-04-11 02:15:06 -07:00