Commit Graph

33 Commits

Author SHA1 Message Date
Celes Renata 27b84fcd2e fix: return open_position_count in trading status for dashboard display
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
2026-04-28 19:27:40 +00:00
Celes Renata 7e8d518946 feat: add max open positions and position cap controls to trading dashboard
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
2026-04-28 19:16:50 +00:00
Celes Renata f251c53f92 fix: risk engine blocking sell orders on over-concentrated positions
Two bugs: (1) trading engine omitted estimated_value from sell order
jobs, causing risk engine to compute 0 reduction; (2) risk engine
applied position size limits to sells, trapping users in positions
they couldn't exit. Sells now always pass position value/pct checks.
2026-04-22 02:07:24 +00:00
Celes Renata c85c0068a2 fix: clean up utcnow deprecation warnings, fix 12 failing tests, add CI/CD pipeline manifests
- Replace all datetime.utcnow() with datetime.now(tz=timezone.utc) across 8 files
- Fix 12 failing tests to match current implementation behavior
- Fix pytest_plugins in non-top-level conftest (moved to root conftest.py)
- Auto-fix 189 lint issues (import sorting, unused imports)
- Add CI/CD pipeline infrastructure (ARC, ArgoCD, Kargo manifests)
- Add values-beta.yaml and values-paper.yaml for staged deployments
- Update GitHub Actions workflow to use self-hosted-gremlin runners
- Add integration-test job to CI pipeline

Result: 1596 passed, 0 failed, 0 warnings
2026-04-18 03:59:28 +00:00
Celes Renata ee5fd30398 fix: backtest force-closes open positions at end + uses real market prices for exits 2026-04-18 00:03:51 +00:00
Celes Renata 913fe8b0b3 feat: override trade tab — manual order entry with auto-registration
Backend:
- OverrideOrderRequest/Response Pydantic models with ticker, quantity, price validators
- POST /api/trading/override/order endpoint (enqueue to Redis broker queue)
- auto_register_symbol() module for untracked ticker registration via Symbol Registry
- Unit tests (17) and property-based tests (3 x 100 examples)

Frontend:
- OverrideTradePanel component (order form + positions display)
- Override tab in TradingEngine page with URL search param navigation
- Override Trade button on Trading Controls page
- useSubmitOverrideOrder mutation hook
- MSW handler and 13 component/integration tests

Steering:
- Updated steering docs for Ubuntu dev machine with nvm/Node 24
2026-04-17 07:02:30 +00:00
Celes Renata fd862da29e fix: remove broken capital controls, reset now queries broker for real balance
- Removed PUT /api/trading/capital (set capital) — only touched in-memory state
- Removed POST /api/trading/capital/adjust (add/withdraw) — same problem
- Reset endpoint now: liquidates Alpaca positions, cancels orders, clears DB,
  then queries Alpaca for real portfolio_value to set engine capital
- Frontend: replaced CapitalCard with simple ResetCard (one button)
- Removed useSetTradingCapital and useAdjustCapital hooks
2026-04-17 04:24:10 +00:00
Celes Renata 5fb59b379c feat: reset endpoint now liquidates Alpaca positions and cancels orders
- Added cancel_all_orders() and close_all_positions() to AlpacaBrokerAdapter
- Reset endpoint creates a temporary adapter to call Alpaca DELETE /v2/orders
  and DELETE /v2/positions before clearing DB and engine state
- Also clears positions table and processed_recommendation_ids on reset
- Broker reset is best-effort — DB/engine reset proceeds even if Alpaca fails
2026-04-17 04:03:31 +00:00
Celes Renata 90614dd7bb feat: paper trading capital controls — add, withdraw, and full reset
Three distinct capital operations on the Trading Controls page:

- Set Capital: overwrites pool balances to a new amount (existing)
- Add/Withdraw: adjusts active pool by a delta without touching
  positions, orders, or history. Validates sufficient balance for
  withdrawals. Logged to reserve_pool_ledger as manual_adjustment.
- Reset Everything: nuclear option — deletes all positions, orders,
  trading decisions, stop levels, snapshots, backtests, notifications,
  and circuit breaker events, then resets capital fresh. Red button
  with double-confirmation dialog.

Backend: POST /api/trading/capital/adjust and POST /api/trading/reset
Frontend: CapitalCard rebuilt with three sections and confirmation UIs
2026-04-17 02:23:26 +00:00
Celes Renata f2d8744a4f fix: backtest submission shows no results — 4 bugs fixed
- ID mismatch: API generated a throwaway UUID while BacktestReplay
  generated its own internally. Frontend polled with wrong ID and
  never found the DB row. Now pre-generate ID in endpoint and pass
  it to BacktestReplay.
- Field name: API returned 'backtest_id' but frontend read 'data.id'.
  Unified to 'id' everywhere.
- No polling: useBacktestResult fired once and never refreshed.
  Added refetchInterval that polls every 2s while status is running.
- Response shape: GET endpoint nested results under 'result' object
  but frontend expected flat fields. Flattened response to match
  BacktestResult type.
- Added running/failed/completed status indicators in BacktestPanel.
2026-04-17 00:31:17 +00:00
Celes Renata 29f46d387c fix: 6 buy/sell logic bugs — sells check trading window, persist audit trail, dedup after position check, no duplicate buys, fix stop-level insert, profit-taking respects market hours 2026-04-17 00:07:50 +00:00
Celes Renata 1246b3868b fix: use current_price not avg_entry_price for invested calc — prevents margin-inflated numbers showing $0 available 2026-04-17 00:04:12 +00:00
Celes Renata f57167ce4d fix: only poll recommendations from last 2 hours (not 24h), persist snapshots during market hours for performance tab 2026-04-16 23:53:54 +00:00
Celes Renata 63287903d0 feat: wire up stop levels, circuit breaker daily loss, profit-taking, real portfolio/decisions/history endpoints 2026-04-16 15:52:46 +00:00
Celes Renata 1329df0bbf feat: sell execution, correlation matrix from market data, US market holiday awareness
- Sell path: looks up existing position, sells full quantity, returns proceeds to pool
- Correlation matrix: computed from 30-day market_snapshots on startup + every 5min
- Holidays: 10 major US market holidays for 2026 checked in trading window functions
2026-04-16 15:36:49 +00:00
Celes Renata 2e77cf32fd fix: critical — track capital properly: load invested positions on startup, deduct on act, sync every 5min 2026-04-16 15:29:28 +00:00
Celes Renata 354c3d484a fix: fetch current prices from market_snapshots before evaluating recommendations — fixes 'Invalid current price' skip 2026-04-16 15:17:49 +00:00
Celes Renata 2a6aac47a6 fix: add decision logging to trading engine, flushed 103k stale dedup keys 2026-04-16 15:12:58 +00:00
Celes Renata c114e77b1c fix: limit recommendation poll to 50 per cycle to prevent 85k-rec processing stall, add poll logging 2026-04-16 15:05:26 +00:00
Celes Renata 136b149000 fix: add logging config to trading engine, add /api/trading/debug diagnostic endpoint 2026-04-16 14:55:42 +00:00
Celes Renata 58a8726306 feat: add paper trading capital controls — API endpoint + UI with presets, fix status/metrics to read real state, fix migration duplicates 2026-04-16 14:06:30 +00:00
Celes Renata 14e411daf9 fix: trading status and metrics endpoints now read real portfolio state instead of hardcoded zeros 2026-04-16 14:02:38 +00:00
Celes Renata c4666c071b feat: wire Gmail SMTP notifications with app password
Replaced the Gmail API (OAuth2) notification delivery with plain
SMTP using a Gmail app password. Much simpler setup — no Google
Cloud project, no OAuth2 flow, no extra dependencies.

- Rewrote _send_gmail() to use smtplib with smtp.gmail.com:587 TLS
- Added stonks-gmail-secrets to Helm chart (GMAIL_SENDER,
  GMAIL_RECIPIENT, GMAIL_APP_PASSWORD)
- Added gmail secret to trading-engine deployment
- Updated runmefirst.sh to read gmail.app from kube dir
- Sender/recipient: celes@celestium.life
2026-04-16 02:37:40 +00:00
Celes Renata 88c2bc84a1 feat: upgrade paper trading to $100k moderate tier
Paper money has no downside — bigger capital exposes more model
behavior: position sizing, diversification, sector exposure,
correlation checks, circuit breakers, reserve pool siphoning,
and risk tier auto-adjustment all become meaningful.

- risk_tier: conservative → moderate (min_confidence 0.55)
- absolute_position_cap: $25 → $10,000
- max_open_positions: 5 → 10
- initial portfolio value: $500 → $100,000
- Updated migration 019, Helm values, and engine default
2026-04-16 00:37:35 +00:00
Celes Renata d21110b3d2 fix: backtest skips duplicate ticker positions to prevent overwrite
When multiple recommendations for the same ticker produce 'act'
decisions, the second one would overwrite the first in
simulated_positions, losing the first position's value and causing
incorrect portfolio value calculations. Now skips if already holding.
2026-04-16 00:28:59 +00:00
Celes Renata 2eaf6dc025 fix: upgrade backtest day summary to WARNING level for visibility 2026-04-16 00:21:46 +00:00
Celes Renata bad7e02e53 fix: add first-skip-reason logging to backtest replay for debugging 2026-04-16 00:16:32 +00:00
Celes Renata 4634f1f3fc fix: remove unused skip_count variable (lint) 2026-04-16 00:07:29 +00:00
Celes Renata ff5055ee4e fix: backtest replay field mapping and logging
- Map DB 'id' field to 'recommendation_id' for evaluate_recommendation()
- Ensure confidence is cast to float (asyncpg may return Decimal)
- Add per-day logging showing rec count, act/skip, positions, pool balance
- Helps diagnose why backtests produce 0 trades
2026-04-15 22:55:26 +00:00
Celes Renata ea6c2b3f54 fix: market data rate limiting and backtest price lookup
- Increase market_api polling cadence from 60s to 900s (15 min).
  The prev-day bar endpoint returns the same data all day, so polling
  every minute wastes API quota. 50 tickers at 15-min cadence = ~3.3
  req/min, well within the 5/min rate limit.
- Reduce market_api rate limit from 30/min to 5/min to match.
- Fix backtest replay to query market_snapshots with data->>'c' for
  close prices instead of nonexistent market_data.close_price column.
- Enrich backtest recommendations with prices from market_snapshots
  and sectors from companies table.
2026-04-15 22:19:44 +00:00
Celes Renata 69eb366bf9 fix: backtest simulation uses ET timezone for trading window
The simulated timestamp was 10:00 UTC (6:00 AM ET) which is outside
the trading window. Changed to 11:00 AM ET so backtested decisions
actually pass the trading window check.
2026-04-15 21:48:32 +00:00
Celes Renata 70bad7709a feat: wire live decision loop and enable paper trading
Phase 2 of the autonomous trading engine:

- Replace start()/stop() stubs with real async implementations
- Decision loop: polls recommendations from PostgreSQL, deduplicates
  via Redis, evaluates through the full pipeline, submits orders to
  stonks:queue:broker_orders
- Stop-loss monitor: fetches prices from Polygon API, checks crossings,
  submits immediate sell orders, safety sell after 15 min without data
- Performance loop: computes metrics every 5 min during market hours,
  persists daily snapshots at market close
- Risk tier scheduler: evaluates daily at 16:00 ET, persists tier changes
- Rebalance scheduler: evaluates Monday 09:45 ET, respects circuit breaker
- Notification dispatch: SNS + Gmail with rate limiting and retry
- Backtest replay: fetches historical data, simulates decisions, persists
- Real asyncpg/redis connections in FastAPI lifespan (graceful degradation)
- Migration 019: enable paper trading with conservative tier, 5 cap
- Added max_open_positions to TradingConfig with env var loading
- Phase 2 tasks added to autonomous-trading-engine spec
2026-04-15 20:52:28 +00:00
Celes Renata 4ffde8cc06 feat: autonomous trading engine — full implementation
- Database migration 018 with 13 tables for trading engine state
- Trading engine service (services/trading/) with 12 pure computation modules:
  position sizer, stop-loss manager, reserve pool, circuit breaker,
  risk tier controller, correlation matrix, tax lots, trading window,
  gradual entry, notifications, micro-trading, backtester
- Core TradingEngine with pre-trade evaluation pipeline and integration wiring
- FastAPI HTTP service with 14 endpoints (health, config, decisions, metrics, backtest)
- Performance tracker with Sharpe ratio, drawdown, profit factor computation
- 194 Python tests (165 property-based + 29 integration)
- Frontend: 13 TanStack Query hooks, 7 dashboard panels, tabbed Trading Engine page
- Helm chart entry, network policy, nginx proxy, ingress for trading-engine
- Shared infrastructure: enums, Redis keys, TradingConfig in AppConfig
2026-04-15 16:12:22 +00:00