Commit Graph

56 Commits

Author SHA1 Message Date
Celes Renata bad7e02e53 fix: add first-skip-reason logging to backtest replay for debugging 2026-04-16 00:16:32 +00:00
Celes Renata 4634f1f3fc fix: remove unused skip_count variable (lint) 2026-04-16 00:07:29 +00:00
Celes Renata ff5055ee4e fix: backtest replay field mapping and logging
- Map DB 'id' field to 'recommendation_id' for evaluate_recommendation()
- Ensure confidence is cast to float (asyncpg may return Decimal)
- Add per-day logging showing rec count, act/skip, positions, pool balance
- Helps diagnose why backtests produce 0 trades
2026-04-15 22:55:26 +00:00
Celes Renata 4501bbebd4 feat: add Polygon grouped daily endpoint for broad market data
Two tiers of market data:
1. Per-ticker prev bars (existing 50 sources, 15-min cadence) for
   watchlist detail — trading decisions, stop-loss, position sizing
2. Grouped daily (new single source, once per day) for broad market
   context — correlation analysis, sector rotation, competitive intel

Changes:
- Add grouped_daily endpoint to PolygonMarketAdapter with auto date
  calculation (previous trading day, skip weekends)
- Add fetch_global_market_sources() to scheduler for sources without
  company_id, scheduled once daily (86400s cadence)
- Update _persist_market_items to use item-level ticker from T field
  and look up company_id dynamically for grouped daily bars
- Migration 020: make company_id nullable on sources and
  market_snapshots tables, add grouped daily source row
- Fix backtest replay to query market_snapshots data->>'c' for prices
2026-04-15 22:38:18 +00:00
Celes Renata ea6c2b3f54 fix: market data rate limiting and backtest price lookup
- Increase market_api polling cadence from 60s to 900s (15 min).
  The prev-day bar endpoint returns the same data all day, so polling
  every minute wastes API quota. 50 tickers at 15-min cadence = ~3.3
  req/min, well within the 5/min rate limit.
- Reduce market_api rate limit from 30/min to 5/min to match.
- Fix backtest replay to query market_snapshots with data->>'c' for
  close prices instead of nonexistent market_data.close_price column.
- Enrich backtest recommendations with prices from market_snapshots
  and sectors from companies table.
2026-04-15 22:19:44 +00:00
Celes Renata 69eb366bf9 fix: backtest simulation uses ET timezone for trading window
The simulated timestamp was 10:00 UTC (6:00 AM ET) which is outside
the trading window. Changed to 11:00 AM ET so backtested decisions
actually pass the trading window check.
2026-04-15 21:48:32 +00:00
Celes Renata 70bad7709a feat: wire live decision loop and enable paper trading
Phase 2 of the autonomous trading engine:

- Replace start()/stop() stubs with real async implementations
- Decision loop: polls recommendations from PostgreSQL, deduplicates
  via Redis, evaluates through the full pipeline, submits orders to
  stonks:queue:broker_orders
- Stop-loss monitor: fetches prices from Polygon API, checks crossings,
  submits immediate sell orders, safety sell after 15 min without data
- Performance loop: computes metrics every 5 min during market hours,
  persists daily snapshots at market close
- Risk tier scheduler: evaluates daily at 16:00 ET, persists tier changes
- Rebalance scheduler: evaluates Monday 09:45 ET, respects circuit breaker
- Notification dispatch: SNS + Gmail with rate limiting and retry
- Backtest replay: fetches historical data, simulates decisions, persists
- Real asyncpg/redis connections in FastAPI lifespan (graceful degradation)
- Migration 019: enable paper trading with conservative tier, 5 cap
- Added max_open_positions to TradingConfig with env var loading
- Phase 2 tasks added to autonomous-trading-engine spec
2026-04-15 20:52:28 +00:00
Celes Renata 3ff910433f fix: reject empty LLM classifications for global events
When the LLM returns empty summary and no key facts, raise ValueError
so the retry logic kicks in instead of persisting an empty event.
Also strip whitespace from summary and filter empty key_facts entries.

Cleaned up 17 empty events from the database.
2026-04-15 19:46:31 +00:00
Celes Renata 326c409d63 fix: use _global fallback for empty ticker in MinIO storage paths
Macro news documents have no ticker, causing upload_normalized_text
and upload_parser_output to produce paths like parsed//2026/...
which MinIO rejects as XMinioInvalidObjectName. Use '_global' as
the path segment when ticker is empty, matching the existing
macro prefix pattern in upload_raw_document.
2026-04-15 19:25:23 +00:00
Celes Renata 4ffde8cc06 feat: autonomous trading engine — full implementation
- Database migration 018 with 13 tables for trading engine state
- Trading engine service (services/trading/) with 12 pure computation modules:
  position sizer, stop-loss manager, reserve pool, circuit breaker,
  risk tier controller, correlation matrix, tax lots, trading window,
  gradual entry, notifications, micro-trading, backtester
- Core TradingEngine with pre-trade evaluation pipeline and integration wiring
- FastAPI HTTP service with 14 endpoints (health, config, decisions, metrics, backtest)
- Performance tracker with Sharpe ratio, drawdown, profit factor computation
- 194 Python tests (165 property-based + 29 integration)
- Frontend: 13 TanStack Query hooks, 7 dashboard panels, tabbed Trading Engine page
- Helm chart entry, network policy, nginx proxy, ingress for trading-engine
- Shared infrastructure: enums, Redis keys, TradingConfig in AppConfig
2026-04-15 16:12:22 +00:00
Celes Renata da86132f0c fix: add num_predict=16384 to prevent output truncation on large articles 2026-04-15 03:11:13 +00:00
Celes Renata b8a2cdc52a fix: fill default values for missing fields in truncated LLM output 2026-04-15 03:08:10 +00:00
Celes Renata 00044af993 fix: switch to think=false with json-repair — 20x faster extraction 2026-04-15 02:54:39 +00:00
Celes Renata 4f2ae23d42 fix: set num_predict=16384 so model has token budget for thinking + content 2026-04-15 01:47:00 +00:00
Celes Renata 46b069a748 fix: switch to non-streaming Ollama calls — streaming breaks thinking mode 2026-04-15 01:19:17 +00:00
Celes Renata ffe19eb23a fix: handle empty ticker in MinIO storage paths, clean up debug log 2026-04-15 00:39:53 +00:00
Celes Renata 8b5b692d3c fix: update stall timer during thinking phase to prevent premature stream abort 2026-04-15 00:06:49 +00:00
Celes Renata 01726af360 fix: remove think=false (Ollama bug #14645), bump max_tokens to 32k 2026-04-14 23:50:28 +00:00
Celes Renata 8d0526470d feat: expand seed to 50 companies with competitor relationships and macro source 2026-04-14 21:00:19 +00:00
Celes Renata e1792e06b9 fix: skip non-UUID pattern signal IDs in evidence persistence 2026-04-14 20:18:50 +00:00
Celes Renata d37a5732d0 fix: use timedelta instead of string for interval query params 2026-04-14 20:10:38 +00:00
Celes Renata c438d0d60a fix: competitor lookup uses ticker join instead of UUID comparison 2026-04-14 20:04:56 +00:00
Celes Renata d8ea58104c fix: lint errors (import sorting, unused vars) 2026-04-14 19:48:19 +00:00
Celes Renata f7a11d14ea feat: competitive intelligence & historical pattern matching layer 2026-04-14 19:42:48 +00:00
Celes Renata b478022ba3 fix: data quality query and suppression fallback in recommendation worker
- Fix _DATA_QUALITY_QUERY: remove nonexistent d.source_id/s.source_class,
  use d.source_type directly
- Fix LIMIT 1 applied after jsonb expansion by restructuring as CTE
- Fix fallback build_quality_context_from_summary returning empty
  source_types which always triggered LOW_SOURCE_DIVERSITY suppression
- Update test to reflect corrected fallback behavior
2026-04-14 06:57:46 +00:00
Celes Renata 4fbddc307a fix(extractor): fallback for any unrecognized impact_horizon value 2026-04-12 16:27:37 -07:00
Celes Renata 6ae8aa779e fix(extractor): add underscore variants to impact_horizon normalizer
Model returns long_term/short_term/medium_term instead of hyphenated versions
2026-04-12 16:08:25 -07:00
Celes Renata cd782d1552 fix(extractor): streaming with guardrails + catalyst_type normalization
- Switch Ollama calls from non-streaming to streaming with early termination
- Add loop detection, max token limit, and stall timeout guards
- Add catalyst_type alias normalizer to handle model hallucinations
- Add explicit enum values in extraction prompt for catalyst_type
- Add streaming config knobs to OllamaConfig
2026-04-12 15:28:20 -07:00
Celes Renata 6e2f174b19 phase 17: disable qwen3.5 thinking mode (think:false) to reduce latency and improve structured output 2026-04-12 12:35:24 -07:00
Celes Renata 45f0c03639 phase 17: add request-level URL logging to OllamaClient for proxy debugging 2026-04-12 12:32:44 -07:00
Celes Renata 1993bfdf3e phase 17: add extraction output normalization — clamp scores to 0-1, map impact_horizon alternatives 2026-04-12 10:15:38 -07:00
Celes Renata 66ed38bf18 phase 17: switch to gemma4:e4b, rewrite prompts for fill-the-fields style with forced ticker inclusion 2026-04-12 10:05:31 -07:00
Celes Renata 2e42310f07 phase 17: fix SEC EDGAR 403 — use descriptive User-Agent with contact email per fair access policy 2026-04-12 09:50:29 -07:00
Celes Renata 311d76dc0b phase 17: enrich SEC EDGAR filings with URLs, titles, dedupe by accession number, skip XML fragments 2026-04-12 09:42:12 -07:00
Celes Renata 28b3361833 phase 17: remove embedded JSON schema from user prompt (4.7KB saved), Ollama format param handles it 2026-04-12 09:28:28 -07:00
Celes Renata 57d0fc7d33 phase 17: pass all tracked tickers to extractor, soften prompt for macro-to-company relevance 2026-04-12 09:18:08 -07:00
Celes Renata 59f89d03d2 phase 17: enrich short parsed articles with Polygon description/keywords from raw payload 2026-04-12 08:52:46 -07:00
Celes Renata cd32c3e3fe phase 17: increase parser→extractor text limit from 8k to 32k chars 2026-04-12 08:37:29 -07:00
Celes Renata 4f2f113cda phase 17: fix text[]/varchar[] type mismatch in coverage-gaps SQL 2026-04-12 04:15:00 -07:00
Celes Renata d16e15c885 phase 17: quote reserved word 'window' in all SQL queries across recommendation worker and query API 2026-04-12 03:45:51 -07:00
Celes Renata 181ed2b6cd phase 17: quote reserved word 'window' in aggregation SQL INSERT 2026-04-12 03:38:45 -07:00
Celes Renata 019eaa40d7 phase 17: fix datetime JSON serialization in aggregation worker market_context 2026-04-12 03:31:32 -07:00
Celes Renata 48bf4f7e7e phase 17: extractor fetches normalized text from MinIO when not in job payload 2026-04-12 03:24:10 -07:00
Celes Renata 012b973bb7 phase 17: wire extractor→aggregation→recommendation queue chain, add company_id_map to extractor 2026-04-12 03:16:27 -07:00
Celes Renata e4a1d2d69a phase 17: fix HTML parser NoneType attrs crash in boilerplate stripping 2026-04-12 03:03:07 -07:00
Celes Renata 264b83ea56 phase 17: fix Polygon article_url and published_utc field mapping in metadata persistence 2026-04-12 02:58:30 -07:00
Celes Renata 0ac4493bd4 phase 17: fix parser URL lookup from DB and extractor text field name mismatch 2026-04-12 02:54:23 -07:00
Celes Renata 67cdb0b8c8 phase 17: fix ruff lint error in scheduler import order 2026-04-12 02:47:47 -07:00
Celes Renata f2b9d6c00a phase 17: fix scheduler config parsing, worker entry points, and seed data for Polygon sources 2026-04-12 02:45:37 -07:00
Celes Renata 5758a704ec phase 16: fix UUID serialization in symbol registry responses 2026-04-11 19:22:13 -07:00