Commit Graph

42 Commits

Author SHA1 Message Date
Celes Renata 20faa8e20d fix: bake secrets into values-paper.yaml and auto-seed on empty DB
- All paper stage credentials now in values-paper.yaml so ArgoCD
  renders them correctly on every sync (no more empty secrets)
- Added seed-if-empty init container to scheduler: runs the seed
  script if the companies table is empty after migrations
2026-04-20 17:40:41 +00:00
Celes Renata 740ddc1c54 fix: revert extractor to 1 replica (single GPU bottleneck) 2026-04-20 12:16:51 +00:00
Celes Renata f1f0b7e34c fix: scale extractor to 3 replicas in paper stage
The extraction queue had 3000+ SEC filings backed up with a single
extractor pod processing them at 10-115s each. Ollama handles
concurrent requests so multiple extractor pods can share the GPU.
2026-04-20 10:59:05 +00:00
Celes Renata f3aac0ac3d fix: superset config uses POSTGRES_DB and REDIS_DB env vars for stage isolation 2026-04-19 23:49:11 +00:00
Celes Renata 0f2f0460a6 fix: dedicated scheduler Dockerfile with psql for migrations, remove Python splitter 2026-04-19 23:35:00 +00:00
Celes Renata 48fed18078 feat: per-stage PostgreSQL users for database isolation (stonks_beta, stonks_paper) 2026-04-19 23:17:22 +00:00
Celes Renata 47f10cd3cf fix: use Python asyncpg migration runner instead of psql, remove postgresql-client from image 2026-04-19 22:54:01 +00:00
Celes Renata 021efba294 feat: auto-run migrations via psql init container on scheduler startup 2026-04-19 22:37:50 +00:00
Celes Renata 5c63264393 feat: stage-isolated infrastructure — separate Postgres DBs, Redis DBs, and MinIO bucket prefixes per stage 2026-04-19 22:20:03 +00:00
Celes Renata 2621b3c5c5 feat: add stage-specific ingress hostnames for beta and paper 2026-04-19 22:00:47 +00:00
Celes Renata 827be709df fix: use Recreate strategy for hive-metastore and superset (RWO PVC) 2026-04-19 20:41:22 +00:00
Celes Renata a9be904afe fix: guard ghcr-secret template against nil ghcrAuth values 2026-04-19 19:51:29 +00:00
Celes Renata 5f6d23888a ci: fix lint errors across project, update ruff.toml per-file ignores 2026-04-18 21:02:28 +00:00
Celes Renata c85c0068a2 fix: clean up utcnow deprecation warnings, fix 12 failing tests, add CI/CD pipeline manifests
- Replace all datetime.utcnow() with datetime.now(tz=timezone.utc) across 8 files
- Fix 12 failing tests to match current implementation behavior
- Fix pytest_plugins in non-top-level conftest (moved to root conftest.py)
- Auto-fix 189 lint issues (import sorting, unused imports)
- Add CI/CD pipeline infrastructure (ARC, ArgoCD, Kargo manifests)
- Add values-beta.yaml and values-paper.yaml for staged deployments
- Update GitHub Actions workflow to use self-hosted-gremlin runners
- Add integration-test job to CI pipeline

Result: 1596 passed, 0 failed, 0 warnings
2026-04-18 03:59:28 +00:00
Celes Renata 3a856cf6ff fix: reduce Ollama timeout from 300s to 240s (4 min) 2026-04-16 18:43:50 +00:00
Celes Renata 540d54c3f7 feat: scale aggregation to 4 replicas across cluster nodes 2026-04-16 09:26:22 +00:00
Celes Renata a3b2e97c2c fix: allow SMTP port 587 egress for Gmail notifications
The trading engine network policy only allowed egress on ports 443
(HTTPS) and 53 (DNS). Gmail SMTP uses port 587 (STARTTLS), causing
'Network is unreachable' when sending notifications.
2026-04-16 05:04:08 +00:00
Celes Renata c4666c071b feat: wire Gmail SMTP notifications with app password
Replaced the Gmail API (OAuth2) notification delivery with plain
SMTP using a Gmail app password. Much simpler setup — no Google
Cloud project, no OAuth2 flow, no extra dependencies.

- Rewrote _send_gmail() to use smtplib with smtp.gmail.com:587 TLS
- Added stonks-gmail-secrets to Helm chart (GMAIL_SENDER,
  GMAIL_RECIPIENT, GMAIL_APP_PASSWORD)
- Added gmail secret to trading-engine deployment
- Updated runmefirst.sh to read gmail.app from kube dir
- Sender/recipient: celes@celestium.life
2026-04-16 02:37:40 +00:00
Celes Renata 00ea917fc0 fix: add broker secrets to ingestion worker deployment
The ingestion worker creates an AlpacaBrokerAdapter but the pod
didn't have BROKER_API_KEY/BROKER_API_SECRET env vars, causing
401 Unauthorized on every broker source fetch. Added
stonks-broker-secrets to the ingestion service's secrets list.
2026-04-16 01:25:54 +00:00
Celes Renata 88c2bc84a1 feat: upgrade paper trading to $100k moderate tier
Paper money has no downside — bigger capital exposes more model
behavior: position sizing, diversification, sector exposure,
correlation checks, circuit breakers, reserve pool siphoning,
and risk tier auto-adjustment all become meaningful.

- risk_tier: conservative → moderate (min_confidence 0.55)
- absolute_position_cap: $25 → $10,000
- max_open_positions: 5 → 10
- initial portfolio value: $500 → $100,000
- Updated migration 019, Helm values, and engine default
2026-04-16 00:37:35 +00:00
Celes Renata 70bad7709a feat: wire live decision loop and enable paper trading
Phase 2 of the autonomous trading engine:

- Replace start()/stop() stubs with real async implementations
- Decision loop: polls recommendations from PostgreSQL, deduplicates
  via Redis, evaluates through the full pipeline, submits orders to
  stonks:queue:broker_orders
- Stop-loss monitor: fetches prices from Polygon API, checks crossings,
  submits immediate sell orders, safety sell after 15 min without data
- Performance loop: computes metrics every 5 min during market hours,
  persists daily snapshots at market close
- Risk tier scheduler: evaluates daily at 16:00 ET, persists tier changes
- Rebalance scheduler: evaluates Monday 09:45 ET, respects circuit breaker
- Notification dispatch: SNS + Gmail with rate limiting and retry
- Backtest replay: fetches historical data, simulates decisions, persists
- Real asyncpg/redis connections in FastAPI lifespan (graceful degradation)
- Migration 019: enable paper trading with conservative tier, 5 cap
- Added max_open_positions to TradingConfig with env var loading
- Phase 2 tasks added to autonomous-trading-engine spec
2026-04-15 20:52:28 +00:00
Celes Renata 4ffde8cc06 feat: autonomous trading engine — full implementation
- Database migration 018 with 13 tables for trading engine state
- Trading engine service (services/trading/) with 12 pure computation modules:
  position sizer, stop-loss manager, reserve pool, circuit breaker,
  risk tier controller, correlation matrix, tax lots, trading window,
  gradual entry, notifications, micro-trading, backtester
- Core TradingEngine with pre-trade evaluation pipeline and integration wiring
- FastAPI HTTP service with 14 endpoints (health, config, decisions, metrics, backtest)
- Performance tracker with Sharpe ratio, drawdown, profit factor computation
- 194 Python tests (165 property-based + 29 integration)
- Frontend: 13 TanStack Query hooks, 7 dashboard panels, tabbed Trading Engine page
- Helm chart entry, network policy, nginx proxy, ingress for trading-engine
- Shared infrastructure: enums, Redis keys, TradingConfig in AppConfig
2026-04-15 16:12:22 +00:00
Celes Renata fd35e12d5e phase 17: switch Ollama to external proxy at 10.1.1.12:2701 2026-04-12 11:37:23 -07:00
Celes Renata 80e0f0976f phase 17: switch back to qwen3.5:9b-fast (Ollama restarted, model available again) 2026-04-12 11:05:42 -07:00
Celes Renata a3e8009fa9 phase 17: revert to qwen3.5:9b (9b-fast was removed from Ollama), add retry script 2026-04-12 10:58:53 -07:00
Celes Renata 109a2485cf phase 17: increase Ollama timeout to 300s for qwen3.5:9b-fast 32k context 2026-04-12 10:32:13 -07:00
Celes Renata 7ee1d0f050 phase 17: switch to qwen3.5:9b-fast (32k context), add queue management scripts 2026-04-12 10:19:28 -07:00
Celes Renata 608ccc8b68 phase 17: revert to qwen3.5:9b, keep improved prompt style 2026-04-12 10:06:13 -07:00
Celes Renata 66ed38bf18 phase 17: switch to gemma4:e4b, rewrite prompts for fill-the-fields style with forced ticker inclusion 2026-04-12 10:05:31 -07:00
Celes Renata 34787ad825 phase 17: fix Trino hive catalog — use native S3 filesystem, remove defunct hive.s3 props 2026-04-12 08:18:18 -07:00
Celes Renata 999648d90b phase 17: add s3.region to Trino catalog config for MinIO (fixes AWS SDK region error) 2026-04-12 08:16:14 -07:00
Celes Renata 226cc3ff44 phase 17: switch Ollama model to qwen3.5:9b (available on cluster) 2026-04-12 03:10:49 -07:00
Celes Renata f2b9d6c00a phase 17: fix scheduler config parsing, worker entry points, and seed data for Polygon sources 2026-04-12 02:45:37 -07:00
Celes Renata 6f5b2231a2 phase 16: add registry/risk nginx proxies, add company form, network policies 2026-04-11 19:12:07 -07:00
Celes Renata 4cd8961db6 phase 16: add dashboard network policy, allow query-api from dashboard 2026-04-11 18:20:48 -07:00
Celes Renata cc7014e33d phase 16: fix superset - trino driver in venv, psycopg2 metadata db, core secrets 2026-04-11 17:37:39 -07:00
Celes Renata 5f87cbe464 phase 16: custom superset image with trino driver, fix security context 2026-04-11 17:18:17 -07:00
Celes Renata 59da3fe89e phase 16: nginx-unprivileged on 8080, helm dashboard deployment 2026-04-11 16:37:59 -07:00
Celes Renata faccb0b8db phase 16: React dashboard with full platform control and analytics 2026-04-11 16:19:46 -07:00
Celes Renata fe3d6c0cb0 fix: trino iceberg catalog s3 props, superset port env override 2026-04-11 14:21:44 -07:00
Celes Renata 273a86e2bd fix: risk engine command points to services.risk.app, redis password, window quoting 2026-04-11 14:15:47 -07:00
Celes Renata 7524aa5f2c phase 15: helm chart for stonks-oracle deployment 2026-04-11 12:21:49 -07:00