Commit Graph

253 Commits

Author SHA1 Message Date
Celes Renata b5c0c6d7c9 fix: aggregate ingestion throughput chart by time bucket
The throughput API returns one row per source_type per time bucket,
but the chart was mapping each row as a separate bar. With 5 source
types × 24 hours, the bars were tiny and overlapping. Now aggregates
completed/failed/items across source types per time bucket so the
chart shows meaningful totals.
2026-04-16 00:52:29 +00:00
Celes Renata 6eda988e3b fix: strip /v2 suffix from broker base URL to prevent doubled path
The alpaca.url config file contains https://paper-api.alpaca.markets/v2
but the adapter code also prepends /v2/ to all paths, resulting in
/v2/v2/positions which returns 404. Now strips trailing /v2 or /v1
from the configured base URL since the adapter manages API versioning.

This was causing 1,017 consecutive broker sync failures.
2026-04-16 00:45:19 +00:00
Celes Renata 88c2bc84a1 feat: upgrade paper trading to $100k moderate tier
Paper money has no downside — bigger capital exposes more model
behavior: position sizing, diversification, sector exposure,
correlation checks, circuit breakers, reserve pool siphoning,
and risk tier auto-adjustment all become meaningful.

- risk_tier: conservative → moderate (min_confidence 0.55)
- absolute_position_cap: $25 → $10,000
- max_open_positions: 5 → 10
- initial portfolio value: $500 → $100,000
- Updated migration 019, Helm values, and engine default
2026-04-16 00:37:35 +00:00
Celes Renata d21110b3d2 fix: backtest skips duplicate ticker positions to prevent overwrite
When multiple recommendations for the same ticker produce 'act'
decisions, the second one would overwrite the first in
simulated_positions, losing the first position's value and causing
incorrect portfolio value calculations. Now skips if already holding.
2026-04-16 00:28:59 +00:00
Celes Renata 2eaf6dc025 fix: upgrade backtest day summary to WARNING level for visibility 2026-04-16 00:21:46 +00:00
Celes Renata bad7e02e53 fix: add first-skip-reason logging to backtest replay for debugging 2026-04-16 00:16:32 +00:00
Celes Renata 357e68a764 feat: add database and Redis backup/restore scripts
- scripts/backup-db.sh: pg_dump compressed backup with table count
  verification, optional MinIO upload, auto-prune keeping last 7
- scripts/restore-db.sh: restore from backup with safety confirmation,
  scales down services during restore, re-grants permissions, scales
  back up with correct replica counts
- scripts/backup-redis.sh: triggers BGSAVE, copies RDB dump locally,
  shows key stats
2026-04-16 00:10:27 +00:00
Celes Renata 4634f1f3fc fix: remove unused skip_count variable (lint) 2026-04-16 00:07:29 +00:00
Celes Renata ff5055ee4e fix: backtest replay field mapping and logging
- Map DB 'id' field to 'recommendation_id' for evaluate_recommendation()
- Ensure confidence is cast to float (asyncpg may return Decimal)
- Add per-day logging showing rec count, act/skip, positions, pool balance
- Helps diagnose why backtests produce 0 trades
2026-04-15 22:55:26 +00:00
Celes Renata 4501bbebd4 feat: add Polygon grouped daily endpoint for broad market data
Two tiers of market data:
1. Per-ticker prev bars (existing 50 sources, 15-min cadence) for
   watchlist detail — trading decisions, stop-loss, position sizing
2. Grouped daily (new single source, once per day) for broad market
   context — correlation analysis, sector rotation, competitive intel

Changes:
- Add grouped_daily endpoint to PolygonMarketAdapter with auto date
  calculation (previous trading day, skip weekends)
- Add fetch_global_market_sources() to scheduler for sources without
  company_id, scheduled once daily (86400s cadence)
- Update _persist_market_items to use item-level ticker from T field
  and look up company_id dynamically for grouped daily bars
- Migration 020: make company_id nullable on sources and
  market_snapshots tables, add grouped daily source row
- Fix backtest replay to query market_snapshots data->>'c' for prices
2026-04-15 22:38:18 +00:00
Celes Renata ea6c2b3f54 fix: market data rate limiting and backtest price lookup
- Increase market_api polling cadence from 60s to 900s (15 min).
  The prev-day bar endpoint returns the same data all day, so polling
  every minute wastes API quota. 50 tickers at 15-min cadence = ~3.3
  req/min, well within the 5/min rate limit.
- Reduce market_api rate limit from 30/min to 5/min to match.
- Fix backtest replay to query market_snapshots with data->>'c' for
  close prices instead of nonexistent market_data.close_price column.
- Enrich backtest recommendations with prices from market_snapshots
  and sectors from companies table.
2026-04-15 22:19:44 +00:00
Celes Renata 69eb366bf9 fix: backtest simulation uses ET timezone for trading window
The simulated timestamp was 10:00 UTC (6:00 AM ET) which is outside
the trading window. Changed to 11:00 AM ET so backtested decisions
actually pass the trading window check.
2026-04-15 21:48:32 +00:00
Celes Renata 70bad7709a feat: wire live decision loop and enable paper trading
Phase 2 of the autonomous trading engine:

- Replace start()/stop() stubs with real async implementations
- Decision loop: polls recommendations from PostgreSQL, deduplicates
  via Redis, evaluates through the full pipeline, submits orders to
  stonks:queue:broker_orders
- Stop-loss monitor: fetches prices from Polygon API, checks crossings,
  submits immediate sell orders, safety sell after 15 min without data
- Performance loop: computes metrics every 5 min during market hours,
  persists daily snapshots at market close
- Risk tier scheduler: evaluates daily at 16:00 ET, persists tier changes
- Rebalance scheduler: evaluates Monday 09:45 ET, respects circuit breaker
- Notification dispatch: SNS + Gmail with rate limiting and retry
- Backtest replay: fetches historical data, simulates decisions, persists
- Real asyncpg/redis connections in FastAPI lifespan (graceful degradation)
- Migration 019: enable paper trading with conservative tier, 5 cap
- Added max_open_positions to TradingConfig with env var loading
- Phase 2 tasks added to autonomous-trading-engine spec
2026-04-15 20:52:28 +00:00
Celes Renata c4b90a5224 fix: sidebar nav highlights both Trading Controls and Trading Engine
When on /trading/engine, the /trading nav item also matched via
startsWith. Now checks if a more specific child route matches
first and uses exact match in that case.
2026-04-15 20:07:31 +00:00
Celes Renata 78aee4c9e6 fix: add trading-engine to CI build matrix
The service was added to the Helm chart but not to the GitHub Actions
build matrix, causing ImagePullBackOff on deploy.
2026-04-15 19:55:34 +00:00
Celes Renata 3ff910433f fix: reject empty LLM classifications for global events
When the LLM returns empty summary and no key facts, raise ValueError
so the retry logic kicks in instead of persisting an empty event.
Also strip whitespace from summary and filter empty key_facts entries.

Cleaned up 17 empty events from the database.
2026-04-15 19:46:31 +00:00
Celes Renata 326c409d63 fix: use _global fallback for empty ticker in MinIO storage paths
Macro news documents have no ticker, causing upload_normalized_text
and upload_parser_output to produce paths like parsed//2026/...
which MinIO rejects as XMinioInvalidObjectName. Use '_global' as
the path segment when ticker is empty, matching the existing
macro prefix pattern in upload_raw_document.
2026-04-15 19:25:23 +00:00
Celes Renata cdf1cd06bc fix: bump GitHub Actions to Node 24 versions
- actions/checkout v4 → v5
- actions/setup-python v5 → v6
- actions/setup-node v4 → v5
- docker/login-action v3 → v4
- docker/setup-buildx-action v3 → v4
- docker/build-push-action v6 → v7

Silences Node.js 20 deprecation warnings ahead of the
June 2026 forced migration deadline.
2026-04-15 16:24:50 +00:00
Celes Renata 516731e69a fix: remove explicit type annotations on Recharts formatter callbacks
TypeScript strict mode in CI rejects explicit parameter types on
Recharts formatter/tickFormatter callbacks. Use inference with
'as number' casts on the value instead. Also fix unsafe cast in
PortfolioComposition and handle possibly-undefined percent.
2026-04-15 16:17:00 +00:00
Celes Renata 4ffde8cc06 feat: autonomous trading engine — full implementation
- Database migration 018 with 13 tables for trading engine state
- Trading engine service (services/trading/) with 12 pure computation modules:
  position sizer, stop-loss manager, reserve pool, circuit breaker,
  risk tier controller, correlation matrix, tax lots, trading window,
  gradual entry, notifications, micro-trading, backtester
- Core TradingEngine with pre-trade evaluation pipeline and integration wiring
- FastAPI HTTP service with 14 endpoints (health, config, decisions, metrics, backtest)
- Performance tracker with Sharpe ratio, drawdown, profit factor computation
- 194 Python tests (165 property-based + 29 integration)
- Frontend: 13 TanStack Query hooks, 7 dashboard panels, tabbed Trading Engine page
- Helm chart entry, network policy, nginx proxy, ingress for trading-engine
- Shared infrastructure: enums, Redis keys, TradingConfig in AppConfig
2026-04-15 16:12:22 +00:00
Celes Renata da86132f0c fix: add num_predict=16384 to prevent output truncation on large articles 2026-04-15 03:11:13 +00:00
Celes Renata b8a2cdc52a fix: fill default values for missing fields in truncated LLM output 2026-04-15 03:08:10 +00:00
Celes Renata 00044af993 fix: switch to think=false with json-repair — 20x faster extraction 2026-04-15 02:54:39 +00:00
Celes Renata 4f2ae23d42 fix: set num_predict=16384 so model has token budget for thinking + content 2026-04-15 01:47:00 +00:00
Celes Renata 46b069a748 fix: switch to non-streaming Ollama calls — streaming breaks thinking mode 2026-04-15 01:19:17 +00:00
Celes Renata ffe19eb23a fix: handle empty ticker in MinIO storage paths, clean up debug log 2026-04-15 00:39:53 +00:00
Celes Renata 8b5b692d3c fix: update stall timer during thinking phase to prevent premature stream abort 2026-04-15 00:06:49 +00:00
Celes Renata 01726af360 fix: remove think=false (Ollama bug #14645), bump max_tokens to 32k 2026-04-14 23:50:28 +00:00
Celes Renata b0fc207c1a docs: update steering docs with competitive layer, 50 companies, deployment lessons 2026-04-14 23:27:52 +00:00
Celes Renata 8d0526470d feat: expand seed to 50 companies with competitor relationships and macro source 2026-04-14 21:00:19 +00:00
Celes Renata 86808465de fix: remove explicit type annotations on Recharts callbacks for v3 compat 2026-04-14 20:40:31 +00:00
Celes Renata 916aaff9f3 feat: add trend history chart with strength/confidence/contradiction time series 2026-04-14 20:37:25 +00:00
Celes Renata e1792e06b9 fix: skip non-UUID pattern signal IDs in evidence persistence 2026-04-14 20:18:50 +00:00
Celes Renata d37a5732d0 fix: use timedelta instead of string for interval query params 2026-04-14 20:10:38 +00:00
Celes Renata c438d0d60a fix: competitor lookup uses ticker join instead of UUID comparison 2026-04-14 20:04:56 +00:00
Celes Renata 2a96c8338b fix: remove unused StatusBadge import in GlobalEvents.tsx 2026-04-14 19:53:40 +00:00
Celes Renata 4e191957b4 fix: remove duplicate closing tag in CompanyDetail.tsx 2026-04-14 19:50:53 +00:00
Celes Renata d8ea58104c fix: lint errors (import sorting, unused vars) 2026-04-14 19:48:19 +00:00
Celes Renata f7a11d14ea feat: competitive intelligence & historical pattern matching layer 2026-04-14 19:42:48 +00:00
Celes Renata b478022ba3 fix: data quality query and suppression fallback in recommendation worker
- Fix _DATA_QUALITY_QUERY: remove nonexistent d.source_id/s.source_class,
  use d.source_type directly
- Fix LIMIT 1 applied after jsonb expansion by restructuring as CTE
- Fix fallback build_quality_context_from_summary returning empty
  source_types which always triggered LOW_SOURCE_DIVERSITY suppression
- Update test to reflect corrected fallback behavior
2026-04-14 06:57:46 +00:00
Celes Renata 4fbddc307a fix(extractor): fallback for any unrecognized impact_horizon value 2026-04-12 16:27:37 -07:00
Celes Renata 6ae8aa779e fix(extractor): add underscore variants to impact_horizon normalizer
Model returns long_term/short_term/medium_term instead of hyphenated versions
2026-04-12 16:08:25 -07:00
Celes Renata cd782d1552 fix(extractor): streaming with guardrails + catalyst_type normalization
- Switch Ollama calls from non-streaming to streaming with early termination
- Add loop detection, max token limit, and stall timeout guards
- Add catalyst_type alias normalizer to handle model hallucinations
- Add explicit enum values in extraction prompt for catalyst_type
- Add streaming config knobs to OllamaConfig
2026-04-12 15:28:20 -07:00
Celes Renata 527be42f82 phase 17: restore script — per-bucket size reporting, full DB row count verification 2026-04-12 14:53:19 -07:00
Celes Renata 85e0ef7580 phase 17: backup script now reports per-bucket sizes and file counts 2026-04-12 14:46:47 -07:00
Celes Renata dd13045ca6 phase 17: fix backup/restore scripts — use postgres:18-alpine for pg_dump version match 2026-04-12 14:41:08 -07:00
Celes Renata 3f5b4adcec phase 17: add backup/restore scripts — PostgreSQL + MinIO → NFS 2026-04-12 14:21:59 -07:00
Celes Renata 6e2f174b19 phase 17: disable qwen3.5 thinking mode (think:false) to reduce latency and improve structured output 2026-04-12 12:35:24 -07:00
Celes Renata 45f0c03639 phase 17: add request-level URL logging to OllamaClient for proxy debugging 2026-04-12 12:32:44 -07:00
Celes Renata fd35e12d5e phase 17: switch Ollama to external proxy at 10.1.1.12:2701 2026-04-12 11:37:23 -07:00