66 Commits

Include renames

Author	SHA1	Message	Date
Celes Renata	021efba294	feat: auto-run migrations via psql init container on scheduler startup	2026-04-19 22:37:50 +00:00
Celes Renata	5c63264393	feat: stage-isolated infrastructure — separate Postgres DBs, Redis DBs, and MinIO bucket prefixes per stage	2026-04-19 22:20:03 +00:00
Celes Renata	2621b3c5c5	feat: add stage-specific ingress hostnames for beta and paper	2026-04-19 22:00:47 +00:00
Celes Renata	827be709df	fix: use Recreate strategy for hive-metastore and superset (RWO PVC)	2026-04-19 20:41:22 +00:00
Celes Renata	a9be904afe	fix: guard ghcr-secret template against nil ghcrAuth values	2026-04-19 19:51:29 +00:00
Celes Renata	1f69a27e3b	fix: replace mktemp with PID-based temp path for BusyBox compat ... BusyBox mktemp in alpine/k8s doesn't support .json suffix in template. The mktemp failure triggered set -e, causing pipeline to report failure despite all 93 tests passing.	2026-04-19 19:35:02 +00:00
Celes Renata	4df513d096	fix: remove bucket-init job, wait for pods before readiness check ... - Remove minio-bucket-init Job entirely (seed_minio.py creates bucket) - Wait for pods to exist before kubectl wait --for=condition=ready - Fixes 'no matching resources found' race when pods are still ContainerCreating	2026-04-19 19:25:49 +00:00
Celes Renata	b2b8aca7c6	fix: inttest runner crash and minio bucket-init proxy issue ... - Remove --profiling-output arg from runner.yaml (plugin uses default path) - Inline profiling hooks in root conftest.py with graceful fallback - Replace mc-based bucket-init with Python urllib (no proxy interference) - Add explicit ProxyHandler({}) to guarantee no proxy usage in bucket-init	2026-04-19 19:15:20 +00:00
Celes Renata	4ebf75134f	ci: clear proxy env in minio-bucket-init, capture seed pod logs on failure	2026-04-19 08:55:52 +00:00
Celes Renata	5be3ce2db9	feat: migrate CI/CD from GHCR to local Harbor registry ... - Makefile: GHCR -> registry.celestium.life/stonks-oracle - GitHub Actions: login to Harbor, use HARBOR_PASSWORD secret - infra/k8s/*.yaml: all image refs -> registry.celestium.life - inttest pipeline: remove GHCR pull secret (local registry, no auth) - Steering docs: update registry/git endpoints	2026-04-19 07:34:28 +00:00
Celes Renata	c2372ccd1e	ci: add NO_PROXY to minio-bucket-init to bypass proxy for internal services	2026-04-19 07:02:27 +00:00
Celes Renata	2d40d70975	ci: remove remaining ghcr-credentials from inttest seed/minio pod overrides	2026-04-19 06:45:46 +00:00
Celes Renata	ebafe795c1	fix: bump seed pod timeout to 5m and add debug diagnostics on pipeline failures	2026-04-19 06:34:58 +00:00
Celes Renata	19b63dd369	ci: migrate inttest images from GHCR to local registry, remove ghcr-credentials	2026-04-19 06:22:35 +00:00
Celes Renata	e3e1531847	ci: add Docker Hub auth + proxy CA to inttest namespace, fix MinIO pull secret	2026-04-19 06:09:56 +00:00
Celes Renata	5f6d23888a	ci: fix lint errors across project, update ruff.toml per-file ignores	2026-04-18 21:02:28 +00:00
Celes Renata	c85c0068a2	fix: clean up utcnow deprecation warnings, fix 12 failing tests, add CI/CD pipeline manifests ... - Replace all datetime.utcnow() with datetime.now(tz=timezone.utc) across 8 files - Fix 12 failing tests to match current implementation behavior - Fix pytest_plugins in non-top-level conftest (moved to root conftest.py) - Auto-fix 189 lint issues (import sorting, unused imports) - Add CI/CD pipeline infrastructure (ARC, ArgoCD, Kargo manifests) - Add values-beta.yaml and values-paper.yaml for staged deployments - Update GitHub Actions workflow to use self-hosted-gremlin runners - Add integration-test job to CI pipeline Result: 1596 passed, 0 failed, 0 warnings	2026-04-18 03:59:28 +00:00
Celes Renata	7a541aa693	fix: trend_windows.id missing gen_random_uuid() default	2026-04-17 17:27:41 +00:00
Celes Renata	84b3c06b2f	fix: sync agent DB prompts with code defaults + update model to qwen3.5:9b-fast	2026-04-17 17:12:09 +00:00
Celes Renata	c5b7bddadb	fix: backfill recommendation evidence for existing recommendations ... Migration 028: For each recommendation with no evidence rows, finds the closest matching trend_window (by ticker + time_horizon + timestamp) and re-inserts evidence from top_supporting/opposing_evidence arrays. Filters out non-UUID pattern IDs and verifies documents exist. This fixes 'No evidence linked' on recommendations created before the UUID filtering fix in persist_recommendation.	2026-04-17 07:37:14 +00:00
Celes Renata	7c23c044d7	feat: agent variants — migration, API, service integration, frontend, tests ... - Migration 027: agent_variants table with single-active enforcement, variant_id column on agent_performance_log - API: full CRUD, clone from agent/variant, activate/deactivate, per-variant performance metrics and history endpoints - Services: extractor, event classifier, thesis rewriter all wired to AgentConfigResolver with variant override support - Frontend: variant list, comparison view, create/edit/clone forms, activate/delete actions on Agents page - Tests: API tests + 5 property-based tests (single-active invariant, clone preservation, config resolution, slug determinism, update idempotence) - Spec files for agent-variants feature	2026-04-17 05:15:42 +00:00
Celes Renata	c501ccea40	fix: default model to qwen3.5:9b + improve event classifier prompt ... - Migration 026 and OllamaConfig now default to qwen3.5:9b instead of llama3.1:8b. Existing deployments keep their current model (qwen3.5:9b-fast) since the migration uses WHERE NOT EXISTS on slug. - Event classifier system prompt expanded with macro-vs-company filtering: explicitly instructs the model to NOT classify single-company news (lawsuits, earnings, management changes, debt crises) as macro events. Sets severity=low and confidence<0.3 for company-specific articles. Reserves 'critical' severity for multi-country/global market events. Prevents over-tagging event_types by requiring direct description. - Updated test_system_prompt_is_concise threshold to accommodate the expanded prompt (300 → 1000 chars).	2026-04-17 02:53:38 +00:00
Celes Renata	45752b9a29	feat: AI Agents management page with per-agent performance tracking ... New Agents tab in the sidebar (Ops group) for viewing, editing, and creating AI agent configurations: Database (migration 026): - ai_agents table: editable configs for each LLM agent (model, prompts, temperature, tokens, retries). source='system' for built-in, source='user' for custom. Seeds 3 system agents (Document Extractor, Event Classifier, Thesis Rewriter) using WHERE NOT EXISTS to never overwrite user edits across reinstalls. - agent_performance_log table: per-invocation metrics (duration, confidence, retries, tokens, errors) linked to agent config. API endpoints: - GET/POST /api/agents — list and create agents - GET/PUT/DELETE /api/agents/{id} — view, edit, delete (system agents can be edited but not deleted) - GET /api/agents/{id}/performance — aggregated metrics (success rate, avg/p95 latency, confidence, token usage) - GET /api/agents/{id}/performance/history — hourly time series Frontend: - AgentsPage with sidebar list + detail panel - Agent detail: config display, system prompt viewer, performance dashboard with metrics cards and time-series chart - Edit form: all config fields editable including system prompt, model, temperature, tokens, retries - Create form: new user-defined agents with auto-slug generation - System agents show blue badge, user agents show green badge	2026-04-17 01:24:35 +00:00
Celes Renata	86b549e5e1	fix: migrations preserve trend history across reinstalls ... Migration 023 was deleting all but the latest trend_windows row per entity before 024 could save them to trend_history. On reinstall, this wiped the entire history every time. Fixed by restructuring: - 023 now creates trend_history FIRST and copies all trend_windows rows into it before deduplicating trend_windows down to latest-only. Uses NOT EXISTS to avoid duplicating rows on re-runs. - 024 is now idempotent: ensures table/indexes exist and backfills from recommendations (last 7 days, 1 point per ticker/window/hour) to reconstruct approximate history even if trend_windows was sparse. Both migrations are safe to re-run on existing databases.	2026-04-17 01:15:28 +00:00
Celes Renata	2360c501e4	feat: intraday hourly price bars via Polygon range endpoint ... - New 'intraday_bars' endpoint in PolygonMarketAdapter: fetches hourly bars for today using range_bars URL with timespan=hour, sort=asc - Scheduler expands intraday_bars global source into per-ticker jobs for all active companies (every 15 minutes via polling_interval) - Migration 025 inserts the intraday source with 900s cadence - Frontend price matching uses closest-timestamp instead of date-string matching, with 2h tolerance for intraday and 36h for daily windows - Bumped market price fetch limit to 200 for intraday granularity	2026-04-17 01:13:24 +00:00
Celes Renata	7c589353f8	fix: blank company charts + competitor GUIDs instead of tickers ... Trend charts blank: - trend_windows uses upsert (1 row per ticker/window), so charts had at most 1 data point. Added trend_history table (migration 024) that appends every snapshot. New /api/trends/history endpoint serves the time series. Frontend now uses useTrendHistory for charts and useTrends for the latest summary card. Competitor GUIDs: - list_competitors query returned raw company_b_id UUIDs without joining companies table. Added LEFT JOIN with CASE to resolve the other company's ticker and legal_name. Updated Pydantic model to include enriched fields. Frontend fallback changed from truncated UUID to ticker/legal_name/Unknown.	2026-04-17 00:42:55 +00:00
Celes Renata	3a856cf6ff	fix: reduce Ollama timeout from 300s to 240s (4 min)	2026-04-16 18:43:50 +00:00
Celes Renata	6bab199159	fix: trend_windows now upserts instead of accumulating (7.5GB→4MB), add competitive signal retention cleanup	2026-04-16 14:32:24 +00:00
Celes Renata	58a8726306	feat: add paper trading capital controls — API endpoint + UI with presets, fix status/metrics to read real state, fix migration duplicates	2026-04-16 14:06:30 +00:00
Celes Renata	540d54c3f7	feat: scale aggregation to 4 replicas across cluster nodes	2026-04-16 09:26:22 +00:00
Celes Renata	0b3ab4ed90	feat: add 11 new saved queries, fix window quoting and cross-table join in samples	2026-04-16 07:14:44 +00:00
Celes Renata	f1e32e9186	fix: add round(double precision, integer) overload so ad-hoc queries work without ::numeric casts	2026-04-16 07:10:23 +00:00
Celes Renata	981e16a27f	fix: quote reserved keyword 'window' in trend query, add Top Movers sample, make seed upsert	2026-04-16 06:25:38 +00:00
Celes Renata	d28787a8ee	fix: add unique constraint on saved_queries.name to prevent duplicates ... The migration ran on every deploy, inserting duplicate queries each time (96 instead of 12). Added UNIQUE constraint on name and changed ON CONFLICT to reference it. Cleaned up 84 duplicates in DB.	2026-04-16 05:16:18 +00:00
Celes Renata	a3b2e97c2c	fix: allow SMTP port 587 egress for Gmail notifications ... The trading engine network policy only allowed egress on ports 443 (HTTPS) and 53 (DNS). Gmail SMTP uses port 587 (STARTTLS), causing 'Network is unreachable' when sending notifications.	2026-04-16 05:04:08 +00:00
Celes Renata	c4666c071b	feat: wire Gmail SMTP notifications with app password ... Replaced the Gmail API (OAuth2) notification delivery with plain SMTP using a Gmail app password. Much simpler setup — no Google Cloud project, no OAuth2 flow, no extra dependencies. - Rewrote _send_gmail() to use smtplib with smtp.gmail.com:587 TLS - Added stonks-gmail-secrets to Helm chart (GMAIL_SENDER, GMAIL_RECIPIENT, GMAIL_APP_PASSWORD) - Added gmail secret to trading-engine deployment - Updated runmefirst.sh to read gmail.app from kube dir - Sender/recipient: celes@celestium.life	2026-04-16 02:37:40 +00:00
Celes Renata	00ea917fc0	fix: add broker secrets to ingestion worker deployment ... The ingestion worker creates an AlpacaBrokerAdapter but the pod didn't have BROKER_API_KEY/BROKER_API_SECRET env vars, causing 401 Unauthorized on every broker source fetch. Added stonks-broker-secrets to the ingestion service's secrets list.	2026-04-16 01:25:54 +00:00
Celes Renata	949324dc89	feat: SQL Explorer with PostgreSQL schema browser and pre-built queries ... The SQL Explorer was querying Trino which has zero tables. Rewrote to use PostgreSQL directly: Backend: - GET /api/analytics/pg-schema: returns all public tables with column names, types, and nullability from information_schema - POST /api/analytics/pg-query: read-only SQL execution against PostgreSQL with SELECT-only enforcement, auto LIMIT, and descriptive error messages for syntax/table/query errors Frontend: - Schema browser shows all PostgreSQL tables with columns and types - Click a table name → generates SELECT * FROM table LIMIT 100 - Pre-built Queries section with 12 seeded queries covering companies, recommendations, trends, market prices, documents, global events, trading decisions, ingestion health, reserve pool, sector exposure - User-saved queries shown separately with delete buttons - Chart builder, Monaco editor, and save functionality preserved Migration 021: seeds 12 pre-built saved queries	2026-04-16 01:06:49 +00:00
Celes Renata	88c2bc84a1	feat: upgrade paper trading to $100k moderate tier ... Paper money has no downside — bigger capital exposes more model behavior: position sizing, diversification, sector exposure, correlation checks, circuit breakers, reserve pool siphoning, and risk tier auto-adjustment all become meaningful. - risk_tier: conservative → moderate (min_confidence 0.55) - absolute_position_cap: $25 → $10,000 - max_open_positions: 5 → 10 - initial portfolio value: $500 → $100,000 - Updated migration 019, Helm values, and engine default	2026-04-16 00:37:35 +00:00
Celes Renata	4501bbebd4	feat: add Polygon grouped daily endpoint for broad market data ... Two tiers of market data: 1. Per-ticker prev bars (existing 50 sources, 15-min cadence) for watchlist detail — trading decisions, stop-loss, position sizing 2. Grouped daily (new single source, once per day) for broad market context — correlation analysis, sector rotation, competitive intel Changes: - Add grouped_daily endpoint to PolygonMarketAdapter with auto date calculation (previous trading day, skip weekends) - Add fetch_global_market_sources() to scheduler for sources without company_id, scheduled once daily (86400s cadence) - Update _persist_market_items to use item-level ticker from T field and look up company_id dynamically for grouped daily bars - Migration 020: make company_id nullable on sources and market_snapshots tables, add grouped daily source row - Fix backtest replay to query market_snapshots data->>'c' for prices	2026-04-15 22:38:18 +00:00
Celes Renata	70bad7709a	feat: wire live decision loop and enable paper trading ... Phase 2 of the autonomous trading engine: - Replace start()/stop() stubs with real async implementations - Decision loop: polls recommendations from PostgreSQL, deduplicates via Redis, evaluates through the full pipeline, submits orders to stonks:queue:broker_orders - Stop-loss monitor: fetches prices from Polygon API, checks crossings, submits immediate sell orders, safety sell after 15 min without data - Performance loop: computes metrics every 5 min during market hours, persists daily snapshots at market close - Risk tier scheduler: evaluates daily at 16:00 ET, persists tier changes - Rebalance scheduler: evaluates Monday 09:45 ET, respects circuit breaker - Notification dispatch: SNS + Gmail with rate limiting and retry - Backtest replay: fetches historical data, simulates decisions, persists - Real asyncpg/redis connections in FastAPI lifespan (graceful degradation) - Migration 019: enable paper trading with conservative tier, 5 cap - Added max_open_positions to TradingConfig with env var loading - Phase 2 tasks added to autonomous-trading-engine spec	2026-04-15 20:52:28 +00:00
Celes Renata	4ffde8cc06	feat: autonomous trading engine — full implementation ... - Database migration 018 with 13 tables for trading engine state - Trading engine service (services/trading/) with 12 pure computation modules: position sizer, stop-loss manager, reserve pool, circuit breaker, risk tier controller, correlation matrix, tax lots, trading window, gradual entry, notifications, micro-trading, backtester - Core TradingEngine with pre-trade evaluation pipeline and integration wiring - FastAPI HTTP service with 14 endpoints (health, config, decisions, metrics, backtest) - Performance tracker with Sharpe ratio, drawdown, profit factor computation - 194 Python tests (165 property-based + 29 integration) - Frontend: 13 TanStack Query hooks, 7 dashboard panels, tabbed Trading Engine page - Helm chart entry, network policy, nginx proxy, ingress for trading-engine - Shared infrastructure: enums, Redis keys, TradingConfig in AppConfig	2026-04-15 16:12:22 +00:00
Celes Renata	f7a11d14ea	feat: competitive intelligence & historical pattern matching layer	2026-04-14 19:42:48 +00:00
Celes Renata	fd35e12d5e	phase 17: switch Ollama to external proxy at 10.1.1.12:2701	2026-04-12 11:37:23 -07:00
Celes Renata	80e0f0976f	phase 17: switch back to qwen3.5:9b-fast (Ollama restarted, model available again)	2026-04-12 11:05:42 -07:00
Celes Renata	a3e8009fa9	phase 17: revert to qwen3.5:9b (9b-fast was removed from Ollama), add retry script	2026-04-12 10:58:53 -07:00
Celes Renata	109a2485cf	phase 17: increase Ollama timeout to 300s for qwen3.5:9b-fast 32k context	2026-04-12 10:32:13 -07:00
Celes Renata	7ee1d0f050	phase 17: switch to qwen3.5:9b-fast (32k context), add queue management scripts	2026-04-12 10:19:28 -07:00
Celes Renata	608ccc8b68	phase 17: revert to qwen3.5:9b, keep improved prompt style	2026-04-12 10:06:13 -07:00
Celes Renata	66ed38bf18	phase 17: switch to gemma4:e4b, rewrite prompts for fill-the-fields style with forced ticker inclusion	2026-04-12 10:05:31 -07:00