spec: add frontend dashboard requirements, design, and tasks (phase 16)
This commit is contained in:
@@ -166,6 +166,30 @@ Responsibilities:
|
|||||||
- provide Athena-like SQL access to MinIO-hosted tables
|
- provide Athena-like SQL access to MinIO-hosted tables
|
||||||
- support dashboard datasets and ad hoc exploration
|
- support dashboard datasets and ad hoc exploration
|
||||||
- support joins between market facts, AI predictions, and executed trades
|
- support joins between market facts, AI predictions, and executed trades
|
||||||
|
|
||||||
|
### 4.13 Web Dashboard (Frontend)
|
||||||
|
Technology:
|
||||||
|
- React 18+ with TypeScript
|
||||||
|
- Vite for build tooling
|
||||||
|
- TanStack Router for client-side routing
|
||||||
|
- TanStack Query for API data fetching and caching
|
||||||
|
- Recharts for time-series and analytical charts
|
||||||
|
- Monaco Editor for SQL query editing
|
||||||
|
- Tailwind CSS for styling
|
||||||
|
- Served as a static SPA from the query-api container or a dedicated nginx container
|
||||||
|
|
||||||
|
Responsibilities:
|
||||||
|
- provide a unified web UI for all operational and analytical functions
|
||||||
|
- company, watchlist, alias, and source CRUD management
|
||||||
|
- document timeline browsing with intelligence drill-down
|
||||||
|
- trend summary visualization with evidence chain navigation
|
||||||
|
- recommendation review with full provenance display
|
||||||
|
- order and position tracking with audit trail views
|
||||||
|
- trading mode controls, risk configuration, approval workflow, and lockout management
|
||||||
|
- DevOps dashboards for pipeline health, ingestion throughput, model performance, and source coverage
|
||||||
|
- interactive SQL query explorer with result visualization (Athena-like)
|
||||||
|
- pre-built analytical dashboards with chart interactions (QuickSight-like)
|
||||||
|
- all API interactions go through the existing Query API, Symbol Registry, and Risk Engine endpoints
|
||||||
#
|
#
|
||||||
# 5. Storage Model
|
# 5. Storage Model
|
||||||
|
|
||||||
@@ -431,6 +455,63 @@ Suggested starter dashboards:
|
|||||||
- model extraction quality
|
- model extraction quality
|
||||||
- source coverage and ingestion lag
|
- source coverage and ingestion lag
|
||||||
|
|
||||||
|
### 9.3 Web Dashboard Architecture
|
||||||
|
|
||||||
|
#### Page structure
|
||||||
|
The SPA is organized into the following top-level sections accessible from a persistent sidebar:
|
||||||
|
|
||||||
|
| Section | Route prefix | Primary API source |
|
||||||
|
|---------|-------------|-------------------|
|
||||||
|
| Home / Overview | `/` | Query API `/api/ops/pipeline/health`, `/api/ops/ingestion/summary` |
|
||||||
|
| Companies | `/companies` | Symbol Registry `/companies`, Query API `/api/companies` |
|
||||||
|
| Documents | `/documents` | Query API `/api/documents` |
|
||||||
|
| Trends | `/trends` | Query API `/api/trends` |
|
||||||
|
| Recommendations | `/recommendations` | Query API `/api/recommendations` |
|
||||||
|
| Orders | `/orders` | Query API `/api/orders` |
|
||||||
|
| Positions | `/positions` | Query API `/api/positions` |
|
||||||
|
| Trading Controls | `/trading` | Query API `/api/admin/trading/*`, Risk Engine `/approvals/*` |
|
||||||
|
| Source Management | `/sources` | Query API `/api/admin/sources/*` |
|
||||||
|
| Pipeline Health | `/ops/pipeline` | Query API `/api/ops/pipeline/health` |
|
||||||
|
| Ingestion Monitor | `/ops/ingestion` | Query API `/api/ops/ingestion/*` |
|
||||||
|
| Model Performance | `/ops/model` | Query API `/api/ops/model/*` |
|
||||||
|
| Source Coverage | `/ops/coverage` | Query API `/api/ops/sources/coverage-gaps` |
|
||||||
|
| SQL Explorer | `/analytics/query` | Trino via Query API proxy endpoint |
|
||||||
|
| Dashboards | `/analytics/dashboards` | Trino via Query API proxy endpoint |
|
||||||
|
|
||||||
|
#### SQL Explorer design
|
||||||
|
The SQL explorer provides an Athena-like experience:
|
||||||
|
- Monaco Editor instance with SQL syntax highlighting
|
||||||
|
- Schema browser sidebar listing Trino catalogs, schemas, and tables
|
||||||
|
- Query execution via a new `/api/analytics/query` proxy endpoint on the Query API that forwards SQL to Trino
|
||||||
|
- Results rendered in a virtual-scrolling table with sortable columns
|
||||||
|
- Chart builder that maps result columns to axes and renders via Recharts
|
||||||
|
- Saved queries stored in PostgreSQL with name, description, and SQL text
|
||||||
|
|
||||||
|
#### Pre-built dashboards
|
||||||
|
Each dashboard is a React component that fetches data from the SQL explorer proxy or dedicated API endpoints:
|
||||||
|
- Symbol Overview: company card grid with trend direction, latest recommendation, position status
|
||||||
|
- Sentiment Heatmap: sector × time matrix colored by aggregated sentiment
|
||||||
|
- Prediction Accuracy: scatter plot of predicted confidence vs realized price move
|
||||||
|
- Paper Trading PnL: equity curve, daily PnL bars, win rate metrics
|
||||||
|
- Model Quality: extraction success rate over time, latency distribution, retry rate
|
||||||
|
- Source Coverage: company × source type matrix with health indicators
|
||||||
|
|
||||||
|
#### API proxy for Trino
|
||||||
|
A new endpoint on the Query API:
|
||||||
|
```
|
||||||
|
POST /api/analytics/query
|
||||||
|
Body: { "sql": "SELECT ...", "limit": 1000 }
|
||||||
|
Response: { "columns": [...], "rows": [...], "row_count": N, "elapsed_ms": N }
|
||||||
|
```
|
||||||
|
This proxies SQL to Trino, enforces row limits, and returns structured results.
|
||||||
|
|
||||||
|
#### Deployment
|
||||||
|
The frontend is built as a static SPA and served either:
|
||||||
|
- From the query-api container via FastAPI's `StaticFiles` mount
|
||||||
|
- From a dedicated lightweight nginx container
|
||||||
|
|
||||||
|
The Helm chart adds a `dashboard` service entry when the frontend is enabled.
|
||||||
|
|
||||||
## 10. Reliability and Safety
|
## 10. Reliability and Safety
|
||||||
- Broker submission must be idempotent.
|
- Broker submission must be idempotent.
|
||||||
- Live trading must be disabled by default.
|
- Live trading must be disabled by default.
|
||||||
@@ -458,6 +539,7 @@ Recommended Kubernetes workloads:
|
|||||||
- `trino-coordinator`
|
- `trino-coordinator`
|
||||||
- `trino-worker`
|
- `trino-worker`
|
||||||
- `superset-web`
|
- `superset-web`
|
||||||
|
- `dashboard` (React SPA served via nginx or query-api static mount)
|
||||||
- `postgres`
|
- `postgres`
|
||||||
- `redis`
|
- `redis`
|
||||||
- `minio`
|
- `minio`
|
||||||
|
|||||||
@@ -243,6 +243,77 @@ THE SYSTEM SHALL surface schema failure rates, latency percentiles, token usage
|
|||||||
WHEN source coverage changes materially
|
WHEN source coverage changes materially
|
||||||
THE SYSTEM SHALL alert operators about sustained source failures, symbol coverage gaps, or analytical publication lag.
|
THE SYSTEM SHALL alert operators about sustained source failures, symbol coverage gaps, or analytical publication lag.
|
||||||
|
|
||||||
|
### 13. Web dashboard and control plane UI
|
||||||
|
|
||||||
|
#### Requirement 13.1
|
||||||
|
**User Story:** As an operator, I want a web-based dashboard so that I can view and control every aspect of the platform without using curl or raw API calls.
|
||||||
|
|
||||||
|
##### Acceptance Criteria
|
||||||
|
1. WHEN an operator opens the dashboard URL, THE Dashboard SHALL render a navigation layout with sidebar links to all major sections including companies, documents, trends, recommendations, orders, positions, trading controls, pipeline health, source management, analytics explorer, and system settings.
|
||||||
|
2. WHEN the dashboard loads, THE Dashboard SHALL authenticate against the Query API and display the current system health status.
|
||||||
|
|
||||||
|
#### Requirement 13.2
|
||||||
|
**User Story:** As an operator, I want to manage companies, watchlists, aliases, and sources through the dashboard so that I can configure what the platform monitors.
|
||||||
|
|
||||||
|
##### Acceptance Criteria
|
||||||
|
1. WHEN an operator navigates to the companies section, THE Dashboard SHALL display a searchable, sortable table of all tracked companies with ticker, name, sector, active status, and source count.
|
||||||
|
2. WHEN an operator clicks a company row, THE Dashboard SHALL display a detail view with editable fields for sector, industry, market cap bucket, and active toggle, plus tabs for aliases, sources, and document history.
|
||||||
|
3. WHEN an operator adds or edits a source for a company, THE Dashboard SHALL present a form with source type selection, configuration fields, credibility score slider, retention days, and access policy.
|
||||||
|
4. WHEN an operator manages watchlists, THE Dashboard SHALL provide create, list, and member management views with drag-and-drop or multi-select company assignment.
|
||||||
|
|
||||||
|
#### Requirement 13.3
|
||||||
|
**User Story:** As an analyst, I want to browse documents, intelligence extractions, and trend summaries through the dashboard so that I can review what the AI is producing.
|
||||||
|
|
||||||
|
##### Acceptance Criteria
|
||||||
|
1. WHEN an analyst navigates to the documents section, THE Dashboard SHALL display a filterable timeline of documents with columns for title, type, source, ticker mentions, published date, parse quality, and extraction status.
|
||||||
|
2. WHEN an analyst clicks a document, THE Dashboard SHALL display the full document detail including intelligence extraction, company impacts, sentiment scores, key facts, risks, and links to raw artifacts in MinIO.
|
||||||
|
3. WHEN an analyst navigates to the trends section, THE Dashboard SHALL display trend summaries per company with direction indicators, strength bars, confidence scores, contradiction scores, and expandable evidence lists.
|
||||||
|
4. WHEN an analyst drills into a trend, THE Dashboard SHALL display the full evidence chain linking trend windows to contributing documents, intelligence objects, and raw sources.
|
||||||
|
|
||||||
|
#### Requirement 13.4
|
||||||
|
**User Story:** As a trader, I want to view recommendations, orders, positions, and PnL through the dashboard so that I can monitor trading activity and audit decisions.
|
||||||
|
|
||||||
|
##### Acceptance Criteria
|
||||||
|
1. WHEN a trader navigates to the recommendations section, THE Dashboard SHALL display a filterable list with ticker, action, mode, confidence, thesis preview, and generation timestamp.
|
||||||
|
2. WHEN a trader clicks a recommendation, THE Dashboard SHALL display the full evidence drill-down including contributing documents, intelligence objects, risk evaluation, and any linked orders.
|
||||||
|
3. WHEN a trader navigates to the orders section, THE Dashboard SHALL display order history with status badges, fill information, and expandable audit trails.
|
||||||
|
4. WHEN a trader navigates to the positions section, THE Dashboard SHALL display current positions with unrealized and realized PnL, entry prices, and current prices.
|
||||||
|
|
||||||
|
#### Requirement 13.5
|
||||||
|
**User Story:** As a risk owner, I want to manage trading mode, risk configuration, operator approvals, and symbol lockouts through the dashboard so that I can control execution safety.
|
||||||
|
|
||||||
|
##### Acceptance Criteria
|
||||||
|
1. WHEN a risk owner navigates to trading controls, THE Dashboard SHALL display the current trading mode with a toggle between paper, live, and disabled modes, with a confirmation dialog for mode changes.
|
||||||
|
2. WHEN a risk owner views pending approvals, THE Dashboard SHALL display a queue of pending operator approval requests with ticker, side, quantity, estimated value, and approve/reject buttons.
|
||||||
|
3. WHEN a risk owner edits risk configuration, THE Dashboard SHALL present a form for max position size, daily loss cap, sector exposure limits, and cooldown periods.
|
||||||
|
4. WHEN a risk owner views lockouts, THE Dashboard SHALL display active symbol lockouts with type, reason, and expiration time.
|
||||||
|
|
||||||
|
#### Requirement 13.6
|
||||||
|
**User Story:** As a platform owner, I want DevOps dashboards showing pipeline health, ingestion throughput, model performance, and source coverage so that I can monitor system operations.
|
||||||
|
|
||||||
|
##### Acceptance Criteria
|
||||||
|
1. WHEN a platform owner navigates to the pipeline health section, THE Dashboard SHALL display document counts at each processing stage, parsing quality distribution, extraction validation rates, and trend generation counts.
|
||||||
|
2. WHEN a platform owner views ingestion throughput, THE Dashboard SHALL display time-series charts of ingestion runs, items fetched, success and failure rates, broken down by source type and time bucket.
|
||||||
|
3. WHEN a platform owner views model performance, THE Dashboard SHALL display success rates, latency percentiles, retry rates, confidence distributions, and recent extraction failures with error details.
|
||||||
|
4. WHEN a platform owner views source coverage, THE Dashboard SHALL display a matrix of companies versus source types showing coverage gaps and stale sources.
|
||||||
|
|
||||||
|
#### Requirement 13.7
|
||||||
|
**User Story:** As an analyst, I want an interactive SQL query explorer in the dashboard so that I can run ad-hoc queries against the lakehouse without needing a separate Trino client.
|
||||||
|
|
||||||
|
##### Acceptance Criteria
|
||||||
|
1. WHEN an analyst opens the analytics explorer, THE Dashboard SHALL display a SQL editor with syntax highlighting, auto-complete for table and column names, and an execute button.
|
||||||
|
2. WHEN an analyst executes a query, THE Dashboard SHALL display results in a paginated, sortable table with column type indicators and row count.
|
||||||
|
3. WHEN an analyst wants to visualize query results, THE Dashboard SHALL provide chart type selection including line, bar, scatter, pie, and heatmap with configurable axis mappings.
|
||||||
|
4. WHEN an analyst saves a query, THE Dashboard SHALL persist the query with a name and description for later reuse, and provide a saved queries list.
|
||||||
|
|
||||||
|
#### Requirement 13.8
|
||||||
|
**User Story:** As a dashboard user, I want pre-built analytical dashboards for market intelligence review so that I can quickly assess signal quality and trading performance.
|
||||||
|
|
||||||
|
##### Acceptance Criteria
|
||||||
|
1. WHEN a user opens the dashboards section, THE Dashboard SHALL display a gallery of pre-built dashboards including symbol overview, sentiment heatmap, prediction accuracy, paper trading PnL, and model quality.
|
||||||
|
2. WHEN a user opens a pre-built dashboard, THE Dashboard SHALL render interactive charts with date range selectors, ticker filters, and drill-down capability.
|
||||||
|
3. WHEN a user interacts with a chart, THE Dashboard SHALL support click-through navigation from aggregated metrics to underlying detail views.
|
||||||
|
|
||||||
## Non-Functional Requirements
|
## Non-Functional Requirements
|
||||||
#### Requirement N1
|
#### Requirement N1
|
||||||
WHEN the system processes documents and market events concurrently
|
WHEN the system processes documents and market events concurrently
|
||||||
|
|||||||
@@ -180,4 +180,97 @@
|
|||||||
- [ ] Generate 7-day company trend summaries with market context
|
- [ ] Generate 7-day company trend summaries with market context
|
||||||
- [ ] Produce paper-trade recommendations only
|
- [ ] Produce paper-trade recommendations only
|
||||||
- [ ] Publish analytical facts for bars, signals, and paper trades into MinIO
|
- [ ] Publish analytical facts for bars, signals, and paper trades into MinIO
|
||||||
- [ ] Expose a simple dashboard with evidence, trend cards, and prediction-vs-outcome views
|
- [ ] Expose a simple dashboard with evidence, trend cards, and prediction-vs-outcome views
|
||||||
|
|
||||||
|
## Phase 16 - Web Dashboard Frontend
|
||||||
|
|
||||||
|
- [ ] 16. Build React web dashboard for full platform control and analytics
|
||||||
|
- [ ] 16.1 Scaffold React project with Vite, TypeScript, Tailwind, and routing
|
||||||
|
- Initialize `frontend/` directory with `npm create vite@latest` using the React-TS template
|
||||||
|
- Install core dependencies: `@tanstack/react-router`, `@tanstack/react-query`, `tailwindcss`, `recharts`, `@monaco-editor/react`, `lucide-react`
|
||||||
|
- Configure Tailwind with a dark-mode-friendly color palette
|
||||||
|
- Create the app shell with sidebar navigation layout and route definitions for all sections
|
||||||
|
- Set up TanStack Query provider with default stale/cache times
|
||||||
|
- Create a shared API client module that targets the Query API, Symbol Registry, and Risk Engine base URLs (configurable via env vars)
|
||||||
|
- _Requirements: 13.1_
|
||||||
|
|
||||||
|
- [ ] 16.2 Build the API client layer and shared components
|
||||||
|
- Create typed API hooks using TanStack Query for each API domain: companies, documents, trends, recommendations, orders, positions, admin/trading, admin/sources, ops endpoints
|
||||||
|
- Build reusable UI components: DataTable (sortable, filterable, paginated), StatusBadge, ConfidenceBar, TrendArrow, DateRangeSelector, TickerFilter, LoadingSpinner, ErrorBoundary
|
||||||
|
- Build a shared layout component with sidebar, breadcrumbs, and top-bar health indicator
|
||||||
|
- _Requirements: 13.1, 13.2_
|
||||||
|
|
||||||
|
- [ ] 16.3 Implement company and source management pages
|
||||||
|
- Build `/companies` list page with searchable, sortable table showing ticker, name, sector, active status, source count
|
||||||
|
- Build `/companies/:id` detail page with editable fields (sector, industry, market cap, active toggle), tabs for aliases, sources, and document history
|
||||||
|
- Build source add/edit form with source type selector, config JSON editor, credibility slider, retention days, access policy dropdown
|
||||||
|
- Build `/companies/:id/aliases` management with add/delete alias forms
|
||||||
|
- Wire watchlist CRUD pages at `/watchlists` with member management
|
||||||
|
- _Requirements: 13.2, 1.1, 1.2, 1.3_
|
||||||
|
|
||||||
|
- [ ] 16.4 Implement document timeline and intelligence drill-down pages
|
||||||
|
- Build `/documents` list page with filterable timeline: title, type, source, ticker mentions, published date, parse quality badge, extraction status
|
||||||
|
- Build `/documents/:id` detail page showing full intelligence extraction, company impacts with sentiment/score, key facts, risks, macro themes, and links to raw MinIO artifacts
|
||||||
|
- Add evidence chain visualization showing document → intelligence → impact records
|
||||||
|
- _Requirements: 13.3, 11.1, 11.2_
|
||||||
|
|
||||||
|
- [ ] 16.5 Implement trend summary and evidence chain pages
|
||||||
|
- Build `/trends` list page with company trend cards showing direction indicator, strength bar, confidence score, contradiction score, and window selector
|
||||||
|
- Build `/trends/:id` detail page with full evidence drill-down: contributing documents, intelligence objects, rank scores, weight breakdowns
|
||||||
|
- Add expandable evidence list on trend cards for quick preview
|
||||||
|
- _Requirements: 13.3, 6.5, 10.4_
|
||||||
|
|
||||||
|
- [ ] 16.6 Implement recommendation review and order tracking pages
|
||||||
|
- Build `/recommendations` list page with filterable table: ticker, action badge, mode, confidence bar, thesis preview, timestamp
|
||||||
|
- Build `/recommendations/:id` detail page with full evidence drill-down, risk evaluation display, and linked orders
|
||||||
|
- Build `/orders` list page with status badges, fill info, and expandable audit trail
|
||||||
|
- Build `/orders/:id` detail page with decision trace, order events timeline, and full audit trail
|
||||||
|
- Build `/positions` page with current positions table showing unrealized/realized PnL, entry/current prices
|
||||||
|
- _Requirements: 13.4, 11.1, 11.2, 11.3_
|
||||||
|
|
||||||
|
- [ ] 16.7 Implement trading controls and risk management pages
|
||||||
|
- Build `/trading` page with trading mode toggle (paper/live/disabled) with confirmation dialog
|
||||||
|
- Build pending approvals queue with approve/reject buttons and review note input
|
||||||
|
- Build risk configuration editor form for max position size, daily loss cap, sector exposure, cooldown periods
|
||||||
|
- Build active lockouts display with type, reason, and expiration countdown
|
||||||
|
- _Requirements: 13.5, 8.1, 8.2_
|
||||||
|
|
||||||
|
- [ ] 16.8 Implement DevOps monitoring dashboards
|
||||||
|
- Build `/ops/pipeline` page with pipeline health summary: document stage counts, parsing quality distribution, extraction validation rates, trend generation stats
|
||||||
|
- Build `/ops/ingestion` page with time-series charts (Recharts) for ingestion throughput, success/failure rates by source type, configurable time bucket selector
|
||||||
|
- Build `/ops/model` page with model performance metrics: success rate gauge, latency percentile chart, retry rate, confidence distribution histogram, recent failures table
|
||||||
|
- Build `/ops/coverage` page with company × source type coverage matrix, stale source indicators, and coverage gap alerts
|
||||||
|
- _Requirements: 13.6, 12.1, 12.2, 12.3_
|
||||||
|
|
||||||
|
- [ ] 16.9 Implement SQL query explorer (Athena-like)
|
||||||
|
- Add a `/api/analytics/query` proxy endpoint to the Query API that forwards SQL to Trino, enforces row limits, and returns structured `{columns, rows, row_count, elapsed_ms}` results
|
||||||
|
- Add a `/api/analytics/schema` endpoint that returns Trino catalog/schema/table/column metadata for the schema browser
|
||||||
|
- Build `/analytics/query` page with Monaco Editor SQL input, schema browser sidebar, execute button, and results table with virtual scrolling
|
||||||
|
- Add chart builder panel: select chart type (line, bar, scatter, pie, heatmap), map result columns to axes, render via Recharts
|
||||||
|
- Add saved queries: persist to PostgreSQL via a new `/api/analytics/saved-queries` CRUD endpoint, display saved query list with load/delete
|
||||||
|
- _Requirements: 13.7, 10.1, 10.3_
|
||||||
|
|
||||||
|
- [ ] 16.10 Implement pre-built analytical dashboards (QuickSight-like)
|
||||||
|
- Build `/analytics/dashboards` gallery page listing available dashboards with preview thumbnails
|
||||||
|
- Build Symbol Overview dashboard: company card grid with trend direction, latest recommendation, position status, sourced from API data
|
||||||
|
- Build Sentiment Heatmap dashboard: sector × time matrix colored by aggregated sentiment, sourced from Trino query
|
||||||
|
- Build Prediction Accuracy dashboard: scatter plot of predicted confidence vs realized price move, sourced from Trino `prediction_vs_outcome` table
|
||||||
|
- Build Paper Trading PnL dashboard: equity curve line chart, daily PnL bars, win rate metrics, sourced from Trino `pnl_daily` table
|
||||||
|
- Build Model Quality dashboard: extraction success rate over time, latency distribution, retry rate charts
|
||||||
|
- Add date range selector and ticker filter controls shared across all dashboards
|
||||||
|
- _Requirements: 13.8, 10.2, 10.4_
|
||||||
|
|
||||||
|
- [ ] 16.11 Build home/overview page
|
||||||
|
- Build `/` home page with system health summary card, recent activity feed, key metrics (active companies, documents today, recommendations today, pipeline status)
|
||||||
|
- Add quick-nav cards linking to each major section
|
||||||
|
- Add alert banner for critical issues (source failures, pipeline bottlenecks)
|
||||||
|
- _Requirements: 13.1, 13.6_
|
||||||
|
|
||||||
|
- [ ] 16.12 Add Dockerfile, CI build, Helm template, and deploy to cluster
|
||||||
|
- Create `frontend/Dockerfile` using multi-stage build: node for build, nginx for serve
|
||||||
|
- Add `dashboard` service to `.github/workflows/build.yml` matrix
|
||||||
|
- Add `dashboard` deployment, service, and ingress to Helm chart values and templates
|
||||||
|
- Configure ingress at `stonks.celestium.life` (or similar)
|
||||||
|
- Build, push, and deploy to cluster
|
||||||
|
- Verify dashboard is accessible and all pages load data from the APIs
|
||||||
|
- _Requirements: 13.1, N1_
|
||||||
Reference in New Issue
Block a user