admin/stonks-oracle
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: beta API integration test suite — 85 new tests across 6 modules

Browse Source 
Extends integration test coverage from 108 to 193 tests for the beta gate.

New test modules:
- test_query_api_extended.py (33 tests): documents, evidence, macro/competitive, ops/admin, agents, analytics
- test_registry_write_paths.py (16 tests): write paths, validation, duplicates, competitor/exposure CRUD
- test_risk_approval_lifecycle.py (8 tests): evaluation edge cases, full approval lifecycle
- test_trading_extended.py (12 tests): config round-trips, decision filtering, override validation
- test_cross_service_roundtrip.py (4 tests): cross-service data consistency
- test_error_handling.py (12 tests): 404s, 422s, empty states, health checks

Seed script extended with watchlists, approvals, lockouts, notifications,
ingestion runs, saved queries, and daily risk snapshots.
This commit is contained in:
Celes Renata
2026-04-20 02:34:19 +00:00
parent 8f67d326c9
commit 898f89926d
12 changed files with 2129 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.kiro/specs/beta-api-test-suite/.config.kiro
+1
View File
@@ -0,0 +1 @@
{"specId": "e433350c-baf0-4f4f-a30e-3724f6654090", "workflowType": "requirements-first", "specType": "feature"}
.kiro/specs/beta-api-test-suite/design.md
+216
View File
@@ -0,0 +1,216 @@
# Design Document: Beta API Test Suite
## Overview
This design describes a comprehensive integration test suite that validates every HTTP endpoint across the four Stonks Oracle API services (query-api, symbol-registry, risk-engine, trading-engine) that can be tested without external broker or news API dependencies. The suite extends the existing ~56 integration tests to achieve full endpoint coverage, serving as the beta gate that blocks promotion to paper-trading if any test fails.
The test suite operates against an ephemeral Kubernetes sandbox with seeded PostgreSQL data, using deterministic UUIDs for exact equality assertions. It validates read paths, write paths, round-trips, edge cases, error handling, pagination/filtering, and empty-state behavior.
## Architecture
```mermaid
graph TD
subgraph "K8s Sandbox Namespace"
PG[(PostgreSQL)]
Redis[(Redis)]
MinIO[(MinIO)]
QA[query-api :8000]
SR[symbol-registry :8000]
RE[risk-engine :8000]
TE[trading-engine :8000]
end
subgraph "Test Runner Job"
SEED[seed_sandbox.py]
CONF[conftest.py]
T1[test_query_api.py]
T2[test_registry_api.py]
T3[test_risk_api.py]
T4[test_trading_api.py]
T5[test_signal_flow.py]
T6[test_frontend_data_deps.py]
T7[test_query_api_extended.py]
T8[test_registry_write_paths.py]
T9[test_risk_approval_lifecycle.py]
T10[test_trading_extended.py]
T11[test_cross_service_roundtrip.py]
T12[test_error_handling.py]
end
SEED -->|INSERT| PG
CONF -->|httpx.AsyncClient| QA
CONF -->|httpx.AsyncClient| SR
CONF -->|httpx.AsyncClient| RE
CONF -->|httpx.AsyncClient| TE
QA --> PG
SR --> PG
RE --> PG
TE --> PG
TE --> Redis
```
The architecture preserves the existing test infrastructure:
- **Runner Job**: K8s Job (`infra/inttest/runner.yaml`) executes pytest in the sandbox namespace
- **Seed Script**: `tests/integration/seed_sandbox.py` populates deterministic data before tests run
- **Fixtures**: `tests/integration/conftest.py` provides `ProfiledAsyncClient` wrappers per service
- **New test files** extend coverage without modifying existing passing tests
## Components and Interfaces
### 1. Extended Seed Script (`seed_sandbox.py`)
New seed functions added to populate tables required by untested endpoints:
| Table | New Records | Purpose |
|-------|-------------|---------|
| `watchlists` | 2 watchlists | Watchlist CRUD tests |
| `watchlist_members` | 3 members | Watchlist membership tests |
| `operator_approvals` | 2 (pending + approved) | Approval lifecycle tests |
| `symbol_lockouts` | 2 (active + expired) | Lockout CRUD tests |
| `notifications` | 1 delivered | Notification history tests |
| `ingestion_runs` | 2 (completed + failed) | Ingestion summary tests |
| `saved_queries` | 1 | Saved query CRUD tests |
| `daily_risk_snapshots` | 1 | Risk snapshot tests |
All new records use deterministic UUIDs exported as named constants.
### 2. New Test Modules
| Module | Service | Coverage |
|--------|---------|----------|
| `test_query_api_extended.py` | query-api | Documents filtering, evidence drill-down, trend projections, macro/competitive endpoints, ops dashboard, agents CRUD, analytics |
| `test_registry_write_paths.py` | symbol-registry | Alias/source/watchlist/competitor/exposure write paths, validation errors, duplicates |
| `test_risk_approval_lifecycle.py` | risk-engine | Full approval lifecycle, evaluation edge cases, expiry |
| `test_trading_extended.py` | trading-engine | Config round-trips, decision filtering, notification CRUD, override validation |
| `test_cross_service_roundtrip.py` | cross-service | Write via one service, read via another |
| `test_error_handling.py` | all services | 404s, 422s, empty lists, structured JSON errors |
### 3. Conftest Extensions
New fixtures added to `conftest.py`:
- Export new seed IDs (watchlists, approvals, lockouts, notifications, ingestion runs, saved queries)
- No changes to existing client fixtures
## Data Models
### New Seed ID Constants
```python
# Watchlists
WATCHLIST_01 = UUID("00000000-0000-4000-ac00-000000000001")
WATCHLIST_02 = UUID("00000000-0000-4000-ac00-000000000002")
# Operator Approvals
APPROVAL_PENDING = UUID("00000000-0000-4000-ad00-000000000001")
APPROVAL_APPROVED = UUID("00000000-0000-4000-ad00-000000000002")
# Symbol Lockouts
LOCKOUT_ACTIVE = UUID("00000000-0000-4000-ae00-000000000001")
LOCKOUT_EXPIRED = UUID("00000000-0000-4000-ae00-000000000002")
# Notifications
NOTIFICATION_01 = UUID("00000000-0000-4000-af00-000000000001")
# Ingestion Runs
INGESTION_RUN_01 = UUID("00000000-0000-4000-b300-000000000001")
INGESTION_RUN_02 = UUID("00000000-0000-4000-b300-000000000002")
# Saved Queries
SAVED_QUERY_01 = UUID("00000000-0000-4000-b400-000000000001")
# Daily Risk Snapshot
RISK_SNAPSHOT_01 = UUID("00000000-0000-4000-b500-000000000001")
```
### Seed ID Export Dictionary Extension
```python
SEED_WATCHLIST_IDS = {
"WL_01": str(WATCHLIST_01),
"WL_02": str(WATCHLIST_02),
}
SEED_APPROVAL_IDS = {
"PENDING": str(APPROVAL_PENDING),
"APPROVED": str(APPROVAL_APPROVED),
}
SEED_LOCKOUT_IDS = {
"ACTIVE": str(LOCKOUT_ACTIVE),
"EXPIRED": str(LOCKOUT_EXPIRED),
}
SEED_NOTIFICATION_IDS = {"NOTIF_01": str(NOTIFICATION_01)}
SEED_INGESTION_RUN_IDS = {
"RUN_01": str(INGESTION_RUN_01),
"RUN_02": str(INGESTION_RUN_02),
}
SEED_SAVED_QUERY_IDS = {"SQ_01": str(SAVED_QUERY_01)}
```
## Error Handling
The test suite validates error handling across all services:
1. **404 Not Found**: All detail endpoints return JSON `{"detail": "..."}` for non-existent UUIDs
2. **422 Validation Error**: POST/PUT endpoints return structured validation errors for invalid bodies
3. **409 Conflict**: Duplicate creation attempts return conflict errors
4. **400 Bad Request**: Business logic violations (self-referencing competitors, etc.)
5. **503 Service Unavailable**: Graceful degradation when Redis is unavailable (trading engine override)
6. **Empty Lists**: List endpoints return `[]` with HTTP 200, never 404
Each error response is verified to be valid JSON (not HTML stack traces).
## Testing Strategy
### Approach: Integration Testing with Seeded Data
**Why PBT does not apply**: This is an integration test suite that validates HTTP API contracts against live services with a seeded database. The tests verify:
- Correct HTTP status codes
- Response schema conformance
- Data consistency across services
- Error handling behavior
These are integration tests with concrete, deterministic assertions — not pure functions with universal properties. The input space is fixed (seeded data), and behavior depends on external service state (PostgreSQL, Redis). Property-based testing would require mocking the entire service layer, defeating the purpose of integration testing.
### Test Organization
**Existing tests (preserved as-is)**:
- `test_query_api.py` — 17 tests covering core query API reads
- `test_registry_api.py` — 8 tests covering registry CRUD
- `test_risk_api.py` — 4 tests covering risk evaluation and approvals
- `test_trading_api.py` — 12 tests covering trading engine endpoints
- `test_signal_flow.py` — 11 tests covering cross-service contracts
- `test_frontend_data_deps.py` — 21 tests covering frontend page dependencies
**New tests (added by this spec)**:
- `test_query_api_extended.py` — ~25 tests for uncovered query API endpoints
- `test_registry_write_paths.py` — ~15 tests for registry write paths and edge cases
- `test_risk_approval_lifecycle.py` — ~10 tests for approval workflow and evaluation edge cases
- `test_trading_extended.py` — ~12 tests for trading config round-trips, filtering, notifications
- `test_cross_service_roundtrip.py` — ~6 tests for cross-service data consistency
- `test_error_handling.py` — ~10 tests for structured error responses
**Total target**: ~130+ integration tests (existing ~56 + ~75 new)
### Test Execution
- Framework: pytest + pytest-asyncio
- HTTP client: httpx.AsyncClient wrapped in ProfiledAsyncClient
- Timeout: 30s per request, 600s total Job deadline
- Parallelism: Sequential within each file (shared state from writes), files can run in parallel
- Idempotency: Seed script uses `ON CONFLICT DO NOTHING` for re-runnability
### Naming Convention
Test classes follow the pattern `Test{Service}{Feature}` with descriptive method names:
```python
class TestQueryAPIDocumentFiltering:
async def test_filter_documents_by_ticker(self, query_client, seed_ids): ...
async def test_filter_documents_by_doc_type(self, query_client, seed_ids): ...
```
### Assertions
- Status code assertions first (`assert resp.status_code == 200`)
- Schema assertions verify required fields exist
- Value assertions use deterministic seed data for exact equality
- Numeric assertions use approximate comparison where appropriate (portfolio math)
- List assertions verify minimum counts from seed data
.kiro/specs/beta-api-test-suite/requirements.md
+206
View File
@@ -0,0 +1,206 @@
# Requirements Document
## Introduction
Comprehensive API integration test suite for the Stonks Oracle beta gate. The suite validates every HTTP endpoint across all four API services (query-api, symbol-registry, risk-engine, trading-engine) that can be tested without external broker or news API dependencies. Tests run against a seeded PostgreSQL database with deterministic data, exercising read paths, write paths, edge cases, empty-state behavior, error handling, pagination/filtering, and round-trip fidelity. The suite extends the existing ~41 integration tests to provide full endpoint coverage suitable for blocking promotion from beta to paper-trading.
## Glossary
- **Test_Suite**: The collection of pytest-asyncio integration test modules under `tests/integration/` that validate API behavior against live sandbox services
- **Seed_Script**: The `tests/integration/seed_sandbox.py` module that populates the database with deterministic UUIDs, timestamps, and relationships for reproducible assertions
- **Query_API**: The FastAPI service at `services/api/app.py` exposing ~50 read/write endpoints for analytics, evidence drill-down, admin controls, macro/competitive layers, agent management, and operational dashboards
- **Symbol_Registry**: The FastAPI service at `services/symbol_registry/app.py` exposing CRUD endpoints for companies, aliases, watchlists, sources, competitor relationships, and exposure profiles
- **Risk_Engine**: The FastAPI service at `services/risk/app.py` exposing order risk evaluation, approval workflow, and approval expiry endpoints
- **Trading_Engine**: The FastAPI service at `services/trading/app.py` exposing engine control, configuration, decisions, metrics, notifications, backtest, and override order endpoints
- **Beta_Gate**: The promotion pipeline (`infra/inttest/promote.sh`) that deploys to beta namespace, seeds data, runs the Test_Suite, and promotes to paper-trading only if all tests pass
- **Sandbox**: The ephemeral Kubernetes namespace with PostgreSQL, Redis, and MinIO where integration tests execute
- **Deterministic_UUID**: A hardcoded UUID in the Seed_Script that enables exact equality assertions in tests
- **Round_Trip**: A test pattern where data is written via a POST/PUT endpoint and then read back via a GET endpoint to verify fidelity
## Requirements
### Requirement 1: Expanded Seed Data for Full Coverage
**User Story:** As a test engineer, I want the seed script to populate all database tables with enough variety to exercise every API code path, so that tests can validate filtering, pagination, edge cases, and empty-state behavior.
#### Acceptance Criteria
1. THE Seed_Script SHALL insert at least 2 watchlists with at least 3 watchlist members across the watchlists
2. THE Seed_Script SHALL insert at least 2 operator approval records: one with status "pending" and one with status "approved"
3. THE Seed_Script SHALL insert at least 2 symbol lockout records: one expired and one still active
4. THE Seed_Script SHALL insert at least 1 notification record with delivery_status "delivered"
5. THE Seed_Script SHALL insert at least 2 ingestion run records with different statuses ("completed" and "failed")
6. THE Seed_Script SHALL insert at least 1 saved query record
7. THE Seed_Script SHALL insert at least 1 daily risk snapshot record
8. THE Seed_Script SHALL export all new Deterministic_UUIDs as named constants and include them in the SEED lookup dictionaries for test assertion use
9. WHEN the Seed_Script is run against an already-seeded database, THE Seed_Script SHALL be idempotent and not raise errors due to duplicate key violations
### Requirement 2: Query API — Document and Evidence Endpoints
**User Story:** As a test engineer, I want comprehensive tests for the Query API document timeline, evidence drill-down, and trend projection endpoints, so that schema drift between services is caught before promotion.
#### Acceptance Criteria
1. WHEN a GET request is made to `/api/documents` with `ticker` filter parameter, THE Query_API SHALL return only documents mentioning that ticker
2. WHEN a GET request is made to `/api/documents` with `doc_type` filter parameter, THE Query_API SHALL return only documents of that type
3. WHEN a GET request is made to `/api/documents/{id}` for a seeded document, THE Query_API SHALL return the document with `intelligence`, `company_mentions`, and `title` fields populated
4. WHEN a GET request is made to `/api/documents/{id}` with a non-existent UUID, THE Query_API SHALL return HTTP 404
5. WHEN a GET request is made to `/api/recommendations/{id}/evidence` for a seeded recommendation, THE Query_API SHALL return an evidence drill-down with document and intelligence references
6. WHEN a GET request is made to `/api/trends/{id}/evidence` for a seeded trend, THE Query_API SHALL return an evidence drill-down with supporting and opposing document references
7. WHEN a GET request is made to `/api/trends/{id}/projection` for a seeded trend with a projection, THE Query_API SHALL return the trend projection with `projected_direction`, `projected_strength`, `projected_confidence`, and `macro_contribution_pct` fields
### Requirement 3: Query API — Macro and Competitive Layer Endpoints
**User Story:** As a test engineer, I want tests for the macro event, competitive signal, and pattern endpoints, so that the three-layer signal aggregation engine is validated end-to-end.
#### Acceptance Criteria
1. WHEN a GET request is made to `/api/macro/status`, THE Query_API SHALL return the macro layer toggle status with `enabled` field
2. WHEN a GET request is made to `/api/macro/events`, THE Query_API SHALL return at least 2 seeded global events with `event_types`, `severity`, `affected_regions`, and `summary` fields
3. WHEN a GET request is made to `/api/macro/events/{id}` for a seeded event, THE Query_API SHALL return the event detail with `macro_impacts` containing per-company impact records
4. WHEN a GET request is made to `/api/macro/impacts/{ticker}` for "AAPL", THE Query_API SHALL return at least 1 macro impact record with `macro_impact_score`, `impact_direction`, and `contributing_factors`
5. WHEN a GET request is made to `/api/competitive/status`, THE Query_API SHALL return the competitive layer toggle status with `enabled` field
6. WHEN a GET request is made to `/api/competitive/signals/{ticker}` for "AAPL", THE Query_API SHALL return competitive signal records with `source_ticker`, `target_ticker`, `signal_direction`, and `signal_strength` fields
7. WHEN a GET request is made to `/api/competitive/patterns/{ticker}` for "AAPL", THE Query_API SHALL return historical pattern data (may be empty if no patterns computed)
### Requirement 4: Query API — Operational and Admin Endpoints
**User Story:** As a test engineer, I want tests for the operational dashboard, source management, trading config, approval workflow, and lockout endpoints, so that admin functionality is validated before promotion.
#### Acceptance Criteria
1. WHEN a GET request is made to `/api/ops/pipeline/health`, THE Query_API SHALL return pipeline health with `document_stages`, `parsing`, `extraction`, `aggregation`, and `queue_depths` fields
2. WHEN a GET request is made to `/api/ops/ingestion/summary`, THE Query_API SHALL return ingestion statistics with `total_runs` and `by_source_type` fields
3. WHEN a GET request is made to `/api/ops/sources/coverage-gaps`, THE Query_API SHALL return gap analysis with `missing_source_types` and `stale_sources` lists
4. WHEN a PUT request is made to `/api/sources/{id}/toggle?active=false` for a seeded source, THE Query_API SHALL return HTTP 200 and the source active status SHALL be updated
5. WHEN a GET request is made to `/api/trading/config`, THE Query_API SHALL return the risk configuration with `trading_mode` and `config` fields
6. WHEN a GET request is made to `/api/approvals/pending`, THE Query_API SHALL return a list containing the seeded pending approval record
7. WHEN a GET request is made to `/api/lockouts/active`, THE Query_API SHALL return a list containing the seeded active lockout record
8. WHEN a POST request is made to `/api/lockouts` with a valid ticker and expiry, THE Query_API SHALL create a lockout and return HTTP 200 with the lockout details
9. WHEN a DELETE request is made to `/api/lockouts/{id}` for the created lockout, THE Query_API SHALL remove the lockout and return HTTP 200
### Requirement 5: Query API — Agent and Variant Management Endpoints
**User Story:** As a test engineer, I want tests for the AI agent CRUD, variant lifecycle, and performance endpoints, so that the agent management system is validated.
#### Acceptance Criteria
1. WHEN a GET request is made to `/api/agents`, THE Query_API SHALL return at least 3 seeded agents with `id`, `name`, `slug`, `model_name`, and `active` fields
2. WHEN a GET request is made to `/api/agents/{id}` for a seeded agent, THE Query_API SHALL return the agent detail with `system_prompt`, `temperature`, and `max_tokens` fields
3. WHEN a POST request is made to `/api/agents` with a valid agent body, THE Query_API SHALL create the agent and return HTTP 201 with the new agent including a generated `slug`
4. WHEN a PUT request is made to `/api/agents/{id}` with updated fields, THE Query_API SHALL update the agent and return the modified record
5. WHEN a GET request is made to `/api/agents/{id}/variants` for a seeded agent, THE Query_API SHALL return at least 1 variant with `variant_name`, `model_name`, and `is_active` fields
6. WHEN a POST request is made to `/api/agents/{id}/variants` with a valid variant body, THE Query_API SHALL create the variant and return HTTP 201
7. WHEN a POST request is made to `/api/agents/{id}/variants/{vid}/activate`, THE Query_API SHALL set the variant as active and deactivate any previously active variant for that agent
8. WHEN a GET request is made to `/api/agents/{id}/performance` for a seeded agent, THE Query_API SHALL return performance metrics derived from the agent performance log
9. WHEN a GET request is made to `/api/agents/{id}/variants/{vid}/performance` for a seeded variant, THE Query_API SHALL return variant-level performance metrics
### Requirement 6: Symbol Registry — Write Path and Edge Case Endpoints
**User Story:** As a test engineer, I want tests for the Symbol Registry write paths (create, update, delete) and edge cases (duplicates, not-found, validation), so that data integrity is validated.
#### Acceptance Criteria
1. WHEN a POST request is made to `/companies` with a duplicate ticker and exchange, THE Symbol_Registry SHALL return HTTP 409
2. WHEN a GET request is made to `/companies/{id}` with a non-existent UUID, THE Symbol_Registry SHALL return HTTP 404
3. WHEN a POST request is made to `/companies/{id}/aliases` with a valid alias, THE Symbol_Registry SHALL create the alias and return HTTP 201 with `id`, `alias`, and `alias_type`
4. WHEN a POST request is made to `/companies/{id}/sources` with a valid source body, THE Symbol_Registry SHALL create the source and return HTTP 201 with `id`, `source_type`, and `source_name`
5. WHEN a POST request is made to `/companies/{id}/sources` with an invalid `source_type`, THE Symbol_Registry SHALL return HTTP 422
6. WHEN a POST request is made to `/watchlists` with a valid name, THE Symbol_Registry SHALL create the watchlist and return HTTP 201
7. WHEN a POST request is made to `/watchlists/{id}/members/{company_id}` for a seeded company, THE Symbol_Registry SHALL add the member and return HTTP 201
8. WHEN a GET request is made to `/watchlists/{id}/members` after adding members, THE Symbol_Registry SHALL return the member companies with `ticker` and `legal_name` fields
9. WHEN a POST request is made to `/watchlists` with a duplicate name, THE Symbol_Registry SHALL return HTTP 409
### Requirement 7: Symbol Registry — Competitor and Exposure Write Paths
**User Story:** As a test engineer, I want tests for competitor relationship CRUD and exposure profile upsert, so that the competitive intelligence data layer is validated.
#### Acceptance Criteria
1. WHEN a POST request is made to `/companies/{id}/competitors` with a valid competitor body, THE Symbol_Registry SHALL create the relationship and return HTTP 201 with `relationship_type`, `strength`, and `bidirectional` fields
2. WHEN a POST request is made to `/companies/{id}/competitors` where company_id equals company_b_id, THE Symbol_Registry SHALL return HTTP 400 with a self-referencing error
3. WHEN a PUT request is made to `/companies/{id}/competitors/{rel_id}` with updated strength, THE Symbol_Registry SHALL update the relationship and return the modified record
4. WHEN a DELETE request is made to `/companies/{id}/competitors/{rel_id}`, THE Symbol_Registry SHALL soft-delete the relationship (set active=false) and return HTTP 200
5. WHEN a PUT request is made to `/companies/{id}/exposure` with a valid exposure profile, THE Symbol_Registry SHALL create or update the profile and return the new version with incremented `version` number
6. WHEN a GET request is made to `/companies/{id}/exposure/history` for a company with multiple profile versions, THE Symbol_Registry SHALL return all versions ordered by version descending
7. WHEN a GET request is made to `/companies/{id}/exposure` for a company with no exposure profile, THE Symbol_Registry SHALL return HTTP 404
### Requirement 8: Risk Engine — Evaluation Edge Cases and Approval Workflow
**User Story:** As a test engineer, I want tests for risk evaluation edge cases, the full approval lifecycle, and approval expiry, so that the risk gate is validated.
#### Acceptance Criteria
1. WHEN a POST request is made to `/evaluate` with a minimal order (only ticker), THE Risk_Engine SHALL return a valid evaluation with `evaluation_id`, `eligible`, and `rejection_reasons`
2. WHEN a POST request is made to `/evaluate` with an order exceeding the absolute position cap, THE Risk_Engine SHALL return `eligible: false` with at least one rejection reason
3. WHEN a POST request is made to `/evaluate` with a custom `config` overriding risk parameters, THE Risk_Engine SHALL use the provided config for evaluation
4. WHEN a GET request is made to `/approvals/pending`, THE Risk_Engine SHALL return a list of pending approval records from the seeded data
5. WHEN a GET request is made to `/approvals/{id}` for a seeded pending approval, THE Risk_Engine SHALL return the approval detail with `ticker`, `side`, `quantity`, `status`, and `expires_at` fields
6. WHEN a POST request is made to `/approvals/{id}/review` with `approved: true`, THE Risk_Engine SHALL update the approval status to "approved" and return the new status
7. WHEN a POST request is made to `/approvals/{id}/review` for a non-existent approval, THE Risk_Engine SHALL return HTTP 404
8. WHEN a POST request is made to `/approvals/expire`, THE Risk_Engine SHALL expire stale pending approvals and return the count of expired records
### Requirement 9: Trading Engine — Configuration, Metrics, and Notification Endpoints
**User Story:** As a test engineer, I want tests for trading engine configuration round-trips, metrics consistency, notification config CRUD, and notification history, so that the autonomous trading control plane is validated.
#### Acceptance Criteria
1. WHEN a PUT request is made to `/api/trading/config` with `risk_tier: "aggressive"` followed by a GET to `/api/trading/status`, THE Trading_Engine SHALL reflect the updated risk tier in the status response (Round_Trip)
2. WHEN a POST request is made to `/api/trading/pause` followed by a GET to `/api/trading/status`, THE Trading_Engine SHALL report `paused: true` in the status response
3. WHEN a POST request is made to `/api/trading/resume` followed by a GET to `/api/trading/status`, THE Trading_Engine SHALL report `paused: false` in the status response
4. WHEN a GET request is made to `/api/trading/metrics`, THE Trading_Engine SHALL return all numeric portfolio metrics where `total_portfolio_value` approximately equals `active_pool + reserve_pool + unrealized_pnl`
5. WHEN a GET request is made to `/api/trading/metrics/history`, THE Trading_Engine SHALL return a list of portfolio snapshots with `portfolio_value` and `snapshot_date` fields
6. WHEN a PUT request is made to `/api/trading/notifications/config` with `sms_enabled: true`, THE Trading_Engine SHALL update the notification config and a subsequent GET SHALL reflect the change (Round_Trip)
7. WHEN a GET request is made to `/api/trading/notifications/history`, THE Trading_Engine SHALL return a list of notification records (at least 1 from seed data)
8. WHEN a GET request is made to `/api/trading/decisions` with `ticker` filter, THE Trading_Engine SHALL return only decisions for that ticker
### Requirement 10: Trading Engine — Decision Filtering, Backtest, and Override Validation
**User Story:** As a test engineer, I want tests for decision audit trail filtering, backtest submission, and override order validation, so that the trading engine's write paths and input validation are verified.
#### Acceptance Criteria
1. WHEN a GET request is made to `/api/trading/decisions` with `limit=1`, THE Trading_Engine SHALL return at most 1 decision record
2. WHEN a GET request is made to `/api/trading/decisions` with `decision=act`, THE Trading_Engine SHALL return only decisions with `decision: "act"`
3. WHEN a POST request is made to `/api/trading/override/order` with an invalid ticker (lowercase or special characters), THE Trading_Engine SHALL return HTTP 422
4. WHEN a POST request is made to `/api/trading/override/order` with quantity of 0 or negative, THE Trading_Engine SHALL return HTTP 422
5. WHEN a POST request is made to `/api/trading/override/order` with a valid market order, THE Trading_Engine SHALL return HTTP 202 with `job_id`, `status: "queued"`, `ticker`, `side`, and `quantity` fields, or a structured error if Redis is unavailable
6. IF the Trading_Engine returns HTTP 503 for an override order due to Redis unavailability, THEN THE Trading_Engine SHALL return a JSON error body (not an unhandled exception)
### Requirement 11: Query API — Analytics, Saved Queries, and Schema Introspection
**User Story:** As a test engineer, I want tests for the analytics query engine, saved query CRUD, and schema introspection endpoints, so that the data exploration features are validated.
#### Acceptance Criteria
1. WHEN a GET request is made to `/api/analytics/schema`, THE Query_API SHALL return the Trino schema information (or a structured error if Trino is unavailable)
2. WHEN a GET request is made to `/api/pg/schema`, THE Query_API SHALL return the PostgreSQL schema with table and column information
3. WHEN a GET request is made to `/api/saved-queries`, THE Query_API SHALL return a list of saved queries (at least 1 from seed data or migrations)
4. WHEN a POST request is made to `/api/saved-queries` with a valid query body, THE Query_API SHALL create the saved query and return the record
5. WHEN a DELETE request is made to `/api/saved-queries/{id}` for the created query, THE Query_API SHALL delete the query and return HTTP 200
6. WHEN a POST request is made to `/api/pg/query` with a valid read-only SQL query, THE Query_API SHALL execute the query and return results
### Requirement 12: Cross-Service Round-Trip and Contract Validation
**User Story:** As a test engineer, I want round-trip tests that write data through one service and read it through another, so that cross-service data consistency is validated.
#### Acceptance Criteria
1. WHEN a company is created via POST to Symbol_Registry `/companies` and then queried via GET to Query_API `/api/companies`, THE company SHALL appear in the Query_API response with matching `ticker` and `legal_name`
2. WHEN an exposure profile is created via PUT to Symbol_Registry `/companies/{id}/exposure` and then queried via GET to Symbol_Registry `/companies/{id}/exposure`, THE profile SHALL match the submitted data with `geographic_revenue_mix`, `supply_chain_regions`, and `market_position_tier` fields preserved
3. WHEN a competitor relationship is created via POST to Symbol_Registry `/companies/{id}/competitors` and then queried from both sides, THE relationship SHALL be visible from both company_a and company_b perspectives
4. WHEN a risk evaluation is performed via POST to Risk_Engine `/evaluate` and then the same ticker's recommendation is queried via Query_API `/api/recommendations`, THE recommendation SHALL include a `risk_evaluation` field with matching evaluation structure
### Requirement 13: Error Handling and Empty-State Behavior
**User Story:** As a test engineer, I want tests that verify all endpoints return structured JSON errors (not HTML or stack traces) and handle empty-state gracefully, so that the frontend never receives unexpected response formats.
#### Acceptance Criteria
1. WHEN a GET request is made to any list endpoint with no matching data, THE service SHALL return HTTP 200 with an empty list (not HTTP 404 or an error)
2. WHEN a GET request is made to any detail endpoint with a non-existent UUID, THE service SHALL return HTTP 404 with a JSON body containing an error message
3. WHEN a POST request is made with an invalid JSON body (missing required fields), THE service SHALL return HTTP 422 with a JSON body containing validation error details
4. WHEN a GET request is made to `/api/trends/{id}/projection` for a trend with no projection, THE Query_API SHALL return HTTP 404 with a JSON error (not HTTP 500)
5. WHEN a GET request is made to `/api/macro/impacts/{ticker}` for a ticker with no macro impacts, THE Query_API SHALL return HTTP 200 with an empty list
6. THE Test_Suite SHALL verify that all health endpoints (`/health`) across all four services return `{"status": "ok"}` with HTTP 200
.kiro/specs/beta-api-test-suite/tasks.md
+124
View File
@@ -0,0 +1,124 @@
# Tasks
## Task 1: Extend Seed Script with New Test Data
- [x] 1.1 Add deterministic UUID constants for watchlists, approvals, lockouts, notifications, ingestion runs, saved queries, and daily risk snapshots
- [x] 1.2 Add `_seed_watchlists` function to insert 2 watchlists with 3 watchlist members
- [x] 1.3 Add `_seed_operator_approvals` function to insert 1 pending and 1 approved approval record
- [x] 1.4 Add `_seed_symbol_lockouts` function to insert 1 active and 1 expired lockout
- [x] 1.5 Add `_seed_notifications` function to insert 1 delivered notification record
- [x] 1.6 Add `_seed_ingestion_runs` function to insert 2 ingestion runs (completed + failed)
- [x] 1.7 Add `_seed_saved_queries` function to insert 1 saved query record
- [x] 1.8 Add `_seed_daily_risk_snapshots` function to insert 1 daily risk snapshot
- [x] 1.9 Export new seed ID lookup dictionaries (SEED_WATCHLIST_IDS, SEED_APPROVAL_IDS, SEED_LOCKOUT_IDS, etc.)
- [x] 1.10 Call all new seed functions from the main `seed()` function
- [x] 1.11 Update conftest.py to import and expose new seed IDs in the `seed_ids` fixture
## Task 2: Query API Extended Tests — Documents and Evidence
- [x] 2.1 Create `tests/integration/test_query_api_extended.py` with test class for document filtering (by ticker, by doc_type)
- [x] 2.2 Add test for document detail with non-existent UUID returning 404
- [x] 2.3 Add test for recommendation evidence drill-down endpoint (`/api/recommendations/{id}/evidence`)
- [x] 2.4 Add test for trend evidence drill-down endpoint (`/api/trends/{id}/evidence`)
- [x] 2.5 Add test for trend projection endpoint (`/api/trends/{id}/projection`)
- [x] 2.6 Add test for trend projection with no projection returning appropriate response
## Task 3: Query API Extended Tests — Macro and Competitive Layer
- [x] 3.1 Add test for macro status endpoint (`/api/admin/macro/status`)
- [x] 3.2 Add test for macro events list endpoint (`/api/macro/events`) verifying seeded events
- [x] 3.3 Add test for macro event detail endpoint (`/api/macro/events/{id}`) with impacts
- [x] 3.4 Add test for macro impacts by ticker endpoint (`/api/macro/impacts/AAPL`)
- [x] 3.5 Add test for competitive status endpoint (`/api/admin/competitive/status`)
- [x] 3.6 Add test for competitive signals endpoint (`/api/patterns/{ticker}/competitive-signals`)
- [x] 3.7 Add test for patterns endpoint (`/api/patterns/{ticker}`)
## Task 4: Query API Extended Tests — Operational and Admin Endpoints
- [x] 4.1 Add test for pipeline health endpoint verifying all required fields
- [x] 4.2 Add test for ingestion summary endpoint verifying `total_runs` and `by_source_type`
- [x] 4.3 Add test for coverage gaps endpoint verifying `missing_source_types` and `stale_sources`
- [x] 4.4 Add test for source toggle endpoint (`PUT /api/admin/sources/{id}/toggle`)
- [x] 4.5 Add test for trading config endpoint (`GET /api/admin/trading/config`)
- [x] 4.6 Add test for pending approvals endpoint (`GET /api/admin/trading/approvals`)
- [x] 4.7 Add test for active lockouts endpoint (`GET /api/admin/trading/lockouts`)
- [x] 4.8 Add test for lockout create and delete lifecycle (`POST` then `DELETE /api/admin/trading/lockouts`)
## Task 5: Query API Extended Tests — Agents and Analytics
- [x] 5.1 Add test for agent detail endpoint (`GET /api/agents/{id}`) verifying `system_prompt`, `temperature`, `max_tokens`
- [x] 5.2 Add test for agent create endpoint (`POST /api/agents`) verifying 201 and slug generation
- [x] 5.3 Add test for agent update endpoint (`PUT /api/agents/{id}`)
- [x] 5.4 Add test for variant create endpoint (`POST /api/agents/{id}/variants`) verifying 201
- [x] 5.5 Add test for variant activate endpoint (`POST /api/agents/{id}/variants/{vid}/activate`)
- [x] 5.6 Add test for agent performance endpoint (`GET /api/agents/{id}/performance`)
- [x] 5.7 Add test for variant performance endpoint (`GET /api/agents/{id}/variants/{vid}/performance`)
- [x] 5.8 Add test for PostgreSQL schema endpoint (`GET /api/analytics/pg-schema`)
- [x] 5.9 Add test for saved queries list endpoint (`GET /api/analytics/saved-queries`)
- [x] 5.10 Add test for saved query create and delete lifecycle
- [x] 5.11 Add test for pg-query endpoint with a valid read-only SQL query
## Task 6: Symbol Registry Write Path and Edge Case Tests
- [x] 6.1 Create `tests/integration/test_registry_write_paths.py` with test for duplicate company creation returning 409
- [x] 6.2 Add test for company not found returning 404
- [x] 6.3 Add test for alias creation (`POST /companies/{id}/aliases`) returning 201
- [x] 6.4 Add test for source creation (`POST /companies/{id}/sources`) returning 201
- [x] 6.5 Add test for source creation with invalid source_type returning 422
- [x] 6.6 Add test for watchlist creation (`POST /watchlists`) returning 201
- [x] 6.7 Add test for watchlist member addition (`POST /watchlists/{id}/members/{company_id}`) returning 201
- [x] 6.8 Add test for watchlist members list (`GET /watchlists/{id}/members`) with ticker and legal_name
- [x] 6.9 Add test for duplicate watchlist creation returning 409
## Task 7: Symbol Registry — Competitor and Exposure Write Paths
- [x] 7.1 Add test for competitor creation (`POST /companies/{id}/competitors`) returning 201
- [x] 7.2 Add test for self-referencing competitor returning 400
- [x] 7.3 Add test for competitor update (`PUT /companies/{id}/competitors/{rel_id}`)
- [x] 7.4 Add test for competitor soft-delete (`DELETE /companies/{id}/competitors/{rel_id}`) returning 200
- [x] 7.5 Add test for exposure profile upsert (`PUT /companies/{id}/exposure`) with version increment
- [x] 7.6 Add test for exposure history endpoint (`GET /companies/{id}/exposure/history`)
- [x] 7.7 Add test for exposure profile not found returning 404
## Task 8: Risk Engine — Evaluation Edge Cases and Approval Lifecycle
- [x] 8.1 Create `tests/integration/test_risk_approval_lifecycle.py` with test for minimal order evaluation
- [x] 8.2 Add test for order exceeding position cap returning `eligible: false`
- [x] 8.3 Add test for evaluation with custom config override
- [x] 8.4 Add test for pending approvals list from seeded data
- [x] 8.5 Add test for approval detail endpoint (`GET /approvals/{id}`) with seeded pending approval
- [x] 8.6 Add test for approval review (`POST /approvals/{id}/review` with `approved: true`)
- [x] 8.7 Add test for review of non-existent approval returning 404
- [x] 8.8 Add test for approval expiry endpoint (`POST /approvals/expire`)
## Task 9: Trading Engine — Configuration Round-Trips and Extended Tests
- [x] 9.1 Create `tests/integration/test_trading_extended.py` with config round-trip test (PUT config → GET status)
- [x] 9.2 Add test for pause/resume round-trip (POST pause → GET status → POST resume → GET status)
- [x] 9.3 Add test for metrics consistency (`total_portfolio_value ≈ active_pool + reserve_pool + unrealized_pnl`)
- [x] 9.4 Add test for metrics history returning portfolio snapshots
- [x] 9.5 Add test for notification config update round-trip (PUT → GET)
- [x] 9.6 Add test for notification history endpoint
- [x] 9.7 Add test for decisions filtering by ticker
- [x] 9.8 Add test for decisions filtering by limit
- [x] 9.9 Add test for decisions filtering by decision type
- [x] 9.10 Add test for override order with invalid ticker returning 422
- [x] 9.11 Add test for override order with zero/negative quantity returning 422
- [x] 9.12 Add test for valid override order returning 202 or structured error
## Task 10: Cross-Service Round-Trip Tests
- [x] 10.1 Create `tests/integration/test_cross_service_roundtrip.py` with test creating company via registry then reading via query API
- [x] 10.2 Add test for exposure profile round-trip (PUT via registry → GET via registry)
- [x] 10.3 Add test for competitor relationship bidirectional visibility
- [x] 10.4 Add test for risk evaluation schema matching what query API returns for recommendations
## Task 11: Error Handling and Empty-State Tests
- [x] 11.1 Create `tests/integration/test_error_handling.py` with test for empty list endpoints returning 200 with `[]`
- [x] 11.2 Add test for non-existent UUID detail endpoints returning 404 with JSON body
- [x] 11.3 Add test for invalid JSON body returning 422 with validation details
- [x] 11.4 Add test for trend projection with no projection returning appropriate response (not 500)
- [x] 11.5 Add test for macro impacts with no data returning 200 with empty impacts list
- [x] 11.6 Add test verifying all four service health endpoints return `{"status": "ok"}`
- [x] 11.7 Add test for override order structured error when Redis unavailable (503 with JSON, not unhandled exception)