diff --git a/.kiro/specs/stonks-oracle/design.md b/.kiro/specs/stonks-oracle/design.md index a44e003..7967f62 100644 --- a/.kiro/specs/stonks-oracle/design.md +++ b/.kiro/specs/stonks-oracle/design.md @@ -166,6 +166,30 @@ Responsibilities: - provide Athena-like SQL access to MinIO-hosted tables - support dashboard datasets and ad hoc exploration - support joins between market facts, AI predictions, and executed trades + +### 4.13 Web Dashboard (Frontend) +Technology: +- React 18+ with TypeScript +- Vite for build tooling +- TanStack Router for client-side routing +- TanStack Query for API data fetching and caching +- Recharts for time-series and analytical charts +- Monaco Editor for SQL query editing +- Tailwind CSS for styling +- Served as a static SPA from the query-api container or a dedicated nginx container + +Responsibilities: +- provide a unified web UI for all operational and analytical functions +- company, watchlist, alias, and source CRUD management +- document timeline browsing with intelligence drill-down +- trend summary visualization with evidence chain navigation +- recommendation review with full provenance display +- order and position tracking with audit trail views +- trading mode controls, risk configuration, approval workflow, and lockout management +- DevOps dashboards for pipeline health, ingestion throughput, model performance, and source coverage +- interactive SQL query explorer with result visualization (Athena-like) +- pre-built analytical dashboards with chart interactions (QuickSight-like) +- all API interactions go through the existing Query API, Symbol Registry, and Risk Engine endpoints # # 5. Storage Model @@ -431,6 +455,63 @@ Suggested starter dashboards: - model extraction quality - source coverage and ingestion lag +### 9.3 Web Dashboard Architecture + +#### Page structure +The SPA is organized into the following top-level sections accessible from a persistent sidebar: + +| Section | Route prefix | Primary API source | +|---------|-------------|-------------------| +| Home / Overview | `/` | Query API `/api/ops/pipeline/health`, `/api/ops/ingestion/summary` | +| Companies | `/companies` | Symbol Registry `/companies`, Query API `/api/companies` | +| Documents | `/documents` | Query API `/api/documents` | +| Trends | `/trends` | Query API `/api/trends` | +| Recommendations | `/recommendations` | Query API `/api/recommendations` | +| Orders | `/orders` | Query API `/api/orders` | +| Positions | `/positions` | Query API `/api/positions` | +| Trading Controls | `/trading` | Query API `/api/admin/trading/*`, Risk Engine `/approvals/*` | +| Source Management | `/sources` | Query API `/api/admin/sources/*` | +| Pipeline Health | `/ops/pipeline` | Query API `/api/ops/pipeline/health` | +| Ingestion Monitor | `/ops/ingestion` | Query API `/api/ops/ingestion/*` | +| Model Performance | `/ops/model` | Query API `/api/ops/model/*` | +| Source Coverage | `/ops/coverage` | Query API `/api/ops/sources/coverage-gaps` | +| SQL Explorer | `/analytics/query` | Trino via Query API proxy endpoint | +| Dashboards | `/analytics/dashboards` | Trino via Query API proxy endpoint | + +#### SQL Explorer design +The SQL explorer provides an Athena-like experience: +- Monaco Editor instance with SQL syntax highlighting +- Schema browser sidebar listing Trino catalogs, schemas, and tables +- Query execution via a new `/api/analytics/query` proxy endpoint on the Query API that forwards SQL to Trino +- Results rendered in a virtual-scrolling table with sortable columns +- Chart builder that maps result columns to axes and renders via Recharts +- Saved queries stored in PostgreSQL with name, description, and SQL text + +#### Pre-built dashboards +Each dashboard is a React component that fetches data from the SQL explorer proxy or dedicated API endpoints: +- Symbol Overview: company card grid with trend direction, latest recommendation, position status +- Sentiment Heatmap: sector × time matrix colored by aggregated sentiment +- Prediction Accuracy: scatter plot of predicted confidence vs realized price move +- Paper Trading PnL: equity curve, daily PnL bars, win rate metrics +- Model Quality: extraction success rate over time, latency distribution, retry rate +- Source Coverage: company × source type matrix with health indicators + +#### API proxy for Trino +A new endpoint on the Query API: +``` +POST /api/analytics/query +Body: { "sql": "SELECT ...", "limit": 1000 } +Response: { "columns": [...], "rows": [...], "row_count": N, "elapsed_ms": N } +``` +This proxies SQL to Trino, enforces row limits, and returns structured results. + +#### Deployment +The frontend is built as a static SPA and served either: +- From the query-api container via FastAPI's `StaticFiles` mount +- From a dedicated lightweight nginx container + +The Helm chart adds a `dashboard` service entry when the frontend is enabled. + ## 10. Reliability and Safety - Broker submission must be idempotent. - Live trading must be disabled by default. @@ -458,6 +539,7 @@ Recommended Kubernetes workloads: - `trino-coordinator` - `trino-worker` - `superset-web` +- `dashboard` (React SPA served via nginx or query-api static mount) - `postgres` - `redis` - `minio` diff --git a/.kiro/specs/stonks-oracle/requirements.md b/.kiro/specs/stonks-oracle/requirements.md index aadaab0..791529c 100644 --- a/.kiro/specs/stonks-oracle/requirements.md +++ b/.kiro/specs/stonks-oracle/requirements.md @@ -243,6 +243,77 @@ THE SYSTEM SHALL surface schema failure rates, latency percentiles, token usage WHEN source coverage changes materially THE SYSTEM SHALL alert operators about sustained source failures, symbol coverage gaps, or analytical publication lag. +### 13. Web dashboard and control plane UI + +#### Requirement 13.1 +**User Story:** As an operator, I want a web-based dashboard so that I can view and control every aspect of the platform without using curl or raw API calls. + +##### Acceptance Criteria +1. WHEN an operator opens the dashboard URL, THE Dashboard SHALL render a navigation layout with sidebar links to all major sections including companies, documents, trends, recommendations, orders, positions, trading controls, pipeline health, source management, analytics explorer, and system settings. +2. WHEN the dashboard loads, THE Dashboard SHALL authenticate against the Query API and display the current system health status. + +#### Requirement 13.2 +**User Story:** As an operator, I want to manage companies, watchlists, aliases, and sources through the dashboard so that I can configure what the platform monitors. + +##### Acceptance Criteria +1. WHEN an operator navigates to the companies section, THE Dashboard SHALL display a searchable, sortable table of all tracked companies with ticker, name, sector, active status, and source count. +2. WHEN an operator clicks a company row, THE Dashboard SHALL display a detail view with editable fields for sector, industry, market cap bucket, and active toggle, plus tabs for aliases, sources, and document history. +3. WHEN an operator adds or edits a source for a company, THE Dashboard SHALL present a form with source type selection, configuration fields, credibility score slider, retention days, and access policy. +4. WHEN an operator manages watchlists, THE Dashboard SHALL provide create, list, and member management views with drag-and-drop or multi-select company assignment. + +#### Requirement 13.3 +**User Story:** As an analyst, I want to browse documents, intelligence extractions, and trend summaries through the dashboard so that I can review what the AI is producing. + +##### Acceptance Criteria +1. WHEN an analyst navigates to the documents section, THE Dashboard SHALL display a filterable timeline of documents with columns for title, type, source, ticker mentions, published date, parse quality, and extraction status. +2. WHEN an analyst clicks a document, THE Dashboard SHALL display the full document detail including intelligence extraction, company impacts, sentiment scores, key facts, risks, and links to raw artifacts in MinIO. +3. WHEN an analyst navigates to the trends section, THE Dashboard SHALL display trend summaries per company with direction indicators, strength bars, confidence scores, contradiction scores, and expandable evidence lists. +4. WHEN an analyst drills into a trend, THE Dashboard SHALL display the full evidence chain linking trend windows to contributing documents, intelligence objects, and raw sources. + +#### Requirement 13.4 +**User Story:** As a trader, I want to view recommendations, orders, positions, and PnL through the dashboard so that I can monitor trading activity and audit decisions. + +##### Acceptance Criteria +1. WHEN a trader navigates to the recommendations section, THE Dashboard SHALL display a filterable list with ticker, action, mode, confidence, thesis preview, and generation timestamp. +2. WHEN a trader clicks a recommendation, THE Dashboard SHALL display the full evidence drill-down including contributing documents, intelligence objects, risk evaluation, and any linked orders. +3. WHEN a trader navigates to the orders section, THE Dashboard SHALL display order history with status badges, fill information, and expandable audit trails. +4. WHEN a trader navigates to the positions section, THE Dashboard SHALL display current positions with unrealized and realized PnL, entry prices, and current prices. + +#### Requirement 13.5 +**User Story:** As a risk owner, I want to manage trading mode, risk configuration, operator approvals, and symbol lockouts through the dashboard so that I can control execution safety. + +##### Acceptance Criteria +1. WHEN a risk owner navigates to trading controls, THE Dashboard SHALL display the current trading mode with a toggle between paper, live, and disabled modes, with a confirmation dialog for mode changes. +2. WHEN a risk owner views pending approvals, THE Dashboard SHALL display a queue of pending operator approval requests with ticker, side, quantity, estimated value, and approve/reject buttons. +3. WHEN a risk owner edits risk configuration, THE Dashboard SHALL present a form for max position size, daily loss cap, sector exposure limits, and cooldown periods. +4. WHEN a risk owner views lockouts, THE Dashboard SHALL display active symbol lockouts with type, reason, and expiration time. + +#### Requirement 13.6 +**User Story:** As a platform owner, I want DevOps dashboards showing pipeline health, ingestion throughput, model performance, and source coverage so that I can monitor system operations. + +##### Acceptance Criteria +1. WHEN a platform owner navigates to the pipeline health section, THE Dashboard SHALL display document counts at each processing stage, parsing quality distribution, extraction validation rates, and trend generation counts. +2. WHEN a platform owner views ingestion throughput, THE Dashboard SHALL display time-series charts of ingestion runs, items fetched, success and failure rates, broken down by source type and time bucket. +3. WHEN a platform owner views model performance, THE Dashboard SHALL display success rates, latency percentiles, retry rates, confidence distributions, and recent extraction failures with error details. +4. WHEN a platform owner views source coverage, THE Dashboard SHALL display a matrix of companies versus source types showing coverage gaps and stale sources. + +#### Requirement 13.7 +**User Story:** As an analyst, I want an interactive SQL query explorer in the dashboard so that I can run ad-hoc queries against the lakehouse without needing a separate Trino client. + +##### Acceptance Criteria +1. WHEN an analyst opens the analytics explorer, THE Dashboard SHALL display a SQL editor with syntax highlighting, auto-complete for table and column names, and an execute button. +2. WHEN an analyst executes a query, THE Dashboard SHALL display results in a paginated, sortable table with column type indicators and row count. +3. WHEN an analyst wants to visualize query results, THE Dashboard SHALL provide chart type selection including line, bar, scatter, pie, and heatmap with configurable axis mappings. +4. WHEN an analyst saves a query, THE Dashboard SHALL persist the query with a name and description for later reuse, and provide a saved queries list. + +#### Requirement 13.8 +**User Story:** As a dashboard user, I want pre-built analytical dashboards for market intelligence review so that I can quickly assess signal quality and trading performance. + +##### Acceptance Criteria +1. WHEN a user opens the dashboards section, THE Dashboard SHALL display a gallery of pre-built dashboards including symbol overview, sentiment heatmap, prediction accuracy, paper trading PnL, and model quality. +2. WHEN a user opens a pre-built dashboard, THE Dashboard SHALL render interactive charts with date range selectors, ticker filters, and drill-down capability. +3. WHEN a user interacts with a chart, THE Dashboard SHALL support click-through navigation from aggregated metrics to underlying detail views. + ## Non-Functional Requirements #### Requirement N1 WHEN the system processes documents and market events concurrently diff --git a/.kiro/specs/stonks-oracle/tasks.md b/.kiro/specs/stonks-oracle/tasks.md index 1b25c95..1793329 100644 --- a/.kiro/specs/stonks-oracle/tasks.md +++ b/.kiro/specs/stonks-oracle/tasks.md @@ -180,4 +180,97 @@ - [ ] Generate 7-day company trend summaries with market context - [ ] Produce paper-trade recommendations only - [ ] Publish analytical facts for bars, signals, and paper trades into MinIO -- [ ] Expose a simple dashboard with evidence, trend cards, and prediction-vs-outcome views \ No newline at end of file +- [ ] Expose a simple dashboard with evidence, trend cards, and prediction-vs-outcome views + +## Phase 16 - Web Dashboard Frontend + +- [ ] 16. Build React web dashboard for full platform control and analytics +- [ ] 16.1 Scaffold React project with Vite, TypeScript, Tailwind, and routing + - Initialize `frontend/` directory with `npm create vite@latest` using the React-TS template + - Install core dependencies: `@tanstack/react-router`, `@tanstack/react-query`, `tailwindcss`, `recharts`, `@monaco-editor/react`, `lucide-react` + - Configure Tailwind with a dark-mode-friendly color palette + - Create the app shell with sidebar navigation layout and route definitions for all sections + - Set up TanStack Query provider with default stale/cache times + - Create a shared API client module that targets the Query API, Symbol Registry, and Risk Engine base URLs (configurable via env vars) + - _Requirements: 13.1_ + +- [ ] 16.2 Build the API client layer and shared components + - Create typed API hooks using TanStack Query for each API domain: companies, documents, trends, recommendations, orders, positions, admin/trading, admin/sources, ops endpoints + - Build reusable UI components: DataTable (sortable, filterable, paginated), StatusBadge, ConfidenceBar, TrendArrow, DateRangeSelector, TickerFilter, LoadingSpinner, ErrorBoundary + - Build a shared layout component with sidebar, breadcrumbs, and top-bar health indicator + - _Requirements: 13.1, 13.2_ + +- [ ] 16.3 Implement company and source management pages + - Build `/companies` list page with searchable, sortable table showing ticker, name, sector, active status, source count + - Build `/companies/:id` detail page with editable fields (sector, industry, market cap, active toggle), tabs for aliases, sources, and document history + - Build source add/edit form with source type selector, config JSON editor, credibility slider, retention days, access policy dropdown + - Build `/companies/:id/aliases` management with add/delete alias forms + - Wire watchlist CRUD pages at `/watchlists` with member management + - _Requirements: 13.2, 1.1, 1.2, 1.3_ + +- [ ] 16.4 Implement document timeline and intelligence drill-down pages + - Build `/documents` list page with filterable timeline: title, type, source, ticker mentions, published date, parse quality badge, extraction status + - Build `/documents/:id` detail page showing full intelligence extraction, company impacts with sentiment/score, key facts, risks, macro themes, and links to raw MinIO artifacts + - Add evidence chain visualization showing document → intelligence → impact records + - _Requirements: 13.3, 11.1, 11.2_ + +- [ ] 16.5 Implement trend summary and evidence chain pages + - Build `/trends` list page with company trend cards showing direction indicator, strength bar, confidence score, contradiction score, and window selector + - Build `/trends/:id` detail page with full evidence drill-down: contributing documents, intelligence objects, rank scores, weight breakdowns + - Add expandable evidence list on trend cards for quick preview + - _Requirements: 13.3, 6.5, 10.4_ + +- [ ] 16.6 Implement recommendation review and order tracking pages + - Build `/recommendations` list page with filterable table: ticker, action badge, mode, confidence bar, thesis preview, timestamp + - Build `/recommendations/:id` detail page with full evidence drill-down, risk evaluation display, and linked orders + - Build `/orders` list page with status badges, fill info, and expandable audit trail + - Build `/orders/:id` detail page with decision trace, order events timeline, and full audit trail + - Build `/positions` page with current positions table showing unrealized/realized PnL, entry/current prices + - _Requirements: 13.4, 11.1, 11.2, 11.3_ + +- [ ] 16.7 Implement trading controls and risk management pages + - Build `/trading` page with trading mode toggle (paper/live/disabled) with confirmation dialog + - Build pending approvals queue with approve/reject buttons and review note input + - Build risk configuration editor form for max position size, daily loss cap, sector exposure, cooldown periods + - Build active lockouts display with type, reason, and expiration countdown + - _Requirements: 13.5, 8.1, 8.2_ + +- [ ] 16.8 Implement DevOps monitoring dashboards + - Build `/ops/pipeline` page with pipeline health summary: document stage counts, parsing quality distribution, extraction validation rates, trend generation stats + - Build `/ops/ingestion` page with time-series charts (Recharts) for ingestion throughput, success/failure rates by source type, configurable time bucket selector + - Build `/ops/model` page with model performance metrics: success rate gauge, latency percentile chart, retry rate, confidence distribution histogram, recent failures table + - Build `/ops/coverage` page with company × source type coverage matrix, stale source indicators, and coverage gap alerts + - _Requirements: 13.6, 12.1, 12.2, 12.3_ + +- [ ] 16.9 Implement SQL query explorer (Athena-like) + - Add a `/api/analytics/query` proxy endpoint to the Query API that forwards SQL to Trino, enforces row limits, and returns structured `{columns, rows, row_count, elapsed_ms}` results + - Add a `/api/analytics/schema` endpoint that returns Trino catalog/schema/table/column metadata for the schema browser + - Build `/analytics/query` page with Monaco Editor SQL input, schema browser sidebar, execute button, and results table with virtual scrolling + - Add chart builder panel: select chart type (line, bar, scatter, pie, heatmap), map result columns to axes, render via Recharts + - Add saved queries: persist to PostgreSQL via a new `/api/analytics/saved-queries` CRUD endpoint, display saved query list with load/delete + - _Requirements: 13.7, 10.1, 10.3_ + +- [ ] 16.10 Implement pre-built analytical dashboards (QuickSight-like) + - Build `/analytics/dashboards` gallery page listing available dashboards with preview thumbnails + - Build Symbol Overview dashboard: company card grid with trend direction, latest recommendation, position status, sourced from API data + - Build Sentiment Heatmap dashboard: sector × time matrix colored by aggregated sentiment, sourced from Trino query + - Build Prediction Accuracy dashboard: scatter plot of predicted confidence vs realized price move, sourced from Trino `prediction_vs_outcome` table + - Build Paper Trading PnL dashboard: equity curve line chart, daily PnL bars, win rate metrics, sourced from Trino `pnl_daily` table + - Build Model Quality dashboard: extraction success rate over time, latency distribution, retry rate charts + - Add date range selector and ticker filter controls shared across all dashboards + - _Requirements: 13.8, 10.2, 10.4_ + +- [ ] 16.11 Build home/overview page + - Build `/` home page with system health summary card, recent activity feed, key metrics (active companies, documents today, recommendations today, pipeline status) + - Add quick-nav cards linking to each major section + - Add alert banner for critical issues (source failures, pipeline bottlenecks) + - _Requirements: 13.1, 13.6_ + +- [ ] 16.12 Add Dockerfile, CI build, Helm template, and deploy to cluster + - Create `frontend/Dockerfile` using multi-stage build: node for build, nginx for serve + - Add `dashboard` service to `.github/workflows/build.yml` matrix + - Add `dashboard` deployment, service, and ingress to Helm chart values and templates + - Configure ingress at `stonks.celestium.life` (or similar) + - Build, push, and deploy to cluster + - Verify dashboard is accessible and all pages load data from the APIs + - _Requirements: 13.1, N1_ \ No newline at end of file