admin/stonks-oracle
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
680e7ab1b8722441cf0a37b864f2895422cc2735
stonks-oracle/.kiro/specs/beta-api-test-suite/requirements.md
T
Celes Renata 898f89926d feat: beta API integration test suite — 85 new tests across 6 modules 
Extends integration test coverage from 108 to 193 tests for the beta gate.

New test modules:
- test_query_api_extended.py (33 tests): documents, evidence, macro/competitive, ops/admin, agents, analytics
- test_registry_write_paths.py (16 tests): write paths, validation, duplicates, competitor/exposure CRUD
- test_risk_approval_lifecycle.py (8 tests): evaluation edge cases, full approval lifecycle
- test_trading_extended.py (12 tests): config round-trips, decision filtering, override validation
- test_cross_service_roundtrip.py (4 tests): cross-service data consistency
- test_error_handling.py (12 tests): 404s, 422s, empty states, health checks

Seed script extended with watchlists, approvals, lockouts, notifications,
ingestion runs, saved queries, and daily risk snapshots.
2026-04-20 02:34:19 +00:00

207 lines
21 KiB
Markdown
Raw Blame History

# Requirements Document
## Introduction
Comprehensive API integration test suite for the Stonks Oracle beta gate. The suite validates every HTTP endpoint across all four API services (query-api, symbol-registry, risk-engine, trading-engine) that can be tested without external broker or news API dependencies. Tests run against a seeded PostgreSQL database with deterministic data, exercising read paths, write paths, edge cases, empty-state behavior, error handling, pagination/filtering, and round-trip fidelity. The suite extends the existing ~41 integration tests to provide full endpoint coverage suitable for blocking promotion from beta to paper-trading.
## Glossary
- **Test_Suite**: The collection of pytest-asyncio integration test modules under `tests/integration/` that validate API behavior against live sandbox services
- **Seed_Script**: The `tests/integration/seed_sandbox.py` module that populates the database with deterministic UUIDs, timestamps, and relationships for reproducible assertions
- **Query_API**: The FastAPI service at `services/api/app.py` exposing ~50 read/write endpoints for analytics, evidence drill-down, admin controls, macro/competitive layers, agent management, and operational dashboards
- **Symbol_Registry**: The FastAPI service at `services/symbol_registry/app.py` exposing CRUD endpoints for companies, aliases, watchlists, sources, competitor relationships, and exposure profiles
- **Risk_Engine**: The FastAPI service at `services/risk/app.py` exposing order risk evaluation, approval workflow, and approval expiry endpoints
- **Trading_Engine**: The FastAPI service at `services/trading/app.py` exposing engine control, configuration, decisions, metrics, notifications, backtest, and override order endpoints
- **Beta_Gate**: The promotion pipeline (`infra/inttest/promote.sh`) that deploys to beta namespace, seeds data, runs the Test_Suite, and promotes to paper-trading only if all tests pass
- **Sandbox**: The ephemeral Kubernetes namespace with PostgreSQL, Redis, and MinIO where integration tests execute
- **Deterministic_UUID**: A hardcoded UUID in the Seed_Script that enables exact equality assertions in tests
- **Round_Trip**: A test pattern where data is written via a POST/PUT endpoint and then read back via a GET endpoint to verify fidelity
## Requirements
### Requirement 1: Expanded Seed Data for Full Coverage
**User Story:** As a test engineer, I want the seed script to populate all database tables with enough variety to exercise every API code path, so that tests can validate filtering, pagination, edge cases, and empty-state behavior.
#### Acceptance Criteria
1. THE Seed_Script SHALL insert at least 2 watchlists with at least 3 watchlist members across the watchlists
2. THE Seed_Script SHALL insert at least 2 operator approval records: one with status "pending" and one with status "approved"
3. THE Seed_Script SHALL insert at least 2 symbol lockout records: one expired and one still active
4. THE Seed_Script SHALL insert at least 1 notification record with delivery_status "delivered"
5. THE Seed_Script SHALL insert at least 2 ingestion run records with different statuses ("completed" and "failed")
6. THE Seed_Script SHALL insert at least 1 saved query record
7. THE Seed_Script SHALL insert at least 1 daily risk snapshot record
8. THE Seed_Script SHALL export all new Deterministic_UUIDs as named constants and include them in the SEED lookup dictionaries for test assertion use
9. WHEN the Seed_Script is run against an already-seeded database, THE Seed_Script SHALL be idempotent and not raise errors due to duplicate key violations
### Requirement 2: Query API — Document and Evidence Endpoints
**User Story:** As a test engineer, I want comprehensive tests for the Query API document timeline, evidence drill-down, and trend projection endpoints, so that schema drift between services is caught before promotion.
#### Acceptance Criteria
1. WHEN a GET request is made to `/api/documents` with `ticker` filter parameter, THE Query_API SHALL return only documents mentioning that ticker
2. WHEN a GET request is made to `/api/documents` with `doc_type` filter parameter, THE Query_API SHALL return only documents of that type
3. WHEN a GET request is made to `/api/documents/{id}` for a seeded document, THE Query_API SHALL return the document with `intelligence`, `company_mentions`, and `title` fields populated
4. WHEN a GET request is made to `/api/documents/{id}` with a non-existent UUID, THE Query_API SHALL return HTTP 404
5. WHEN a GET request is made to `/api/recommendations/{id}/evidence` for a seeded recommendation, THE Query_API SHALL return an evidence drill-down with document and intelligence references
6. WHEN a GET request is made to `/api/trends/{id}/evidence` for a seeded trend, THE Query_API SHALL return an evidence drill-down with supporting and opposing document references
7. WHEN a GET request is made to `/api/trends/{id}/projection` for a seeded trend with a projection, THE Query_API SHALL return the trend projection with `projected_direction`, `projected_strength`, `projected_confidence`, and `macro_contribution_pct` fields
### Requirement 3: Query API — Macro and Competitive Layer Endpoints
**User Story:** As a test engineer, I want tests for the macro event, competitive signal, and pattern endpoints, so that the three-layer signal aggregation engine is validated end-to-end.
#### Acceptance Criteria
1. WHEN a GET request is made to `/api/macro/status`, THE Query_API SHALL return the macro layer toggle status with `enabled` field
2. WHEN a GET request is made to `/api/macro/events`, THE Query_API SHALL return at least 2 seeded global events with `event_types`, `severity`, `affected_regions`, and `summary` fields
3. WHEN a GET request is made to `/api/macro/events/{id}` for a seeded event, THE Query_API SHALL return the event detail with `macro_impacts` containing per-company impact records
4. WHEN a GET request is made to `/api/macro/impacts/{ticker}` for "AAPL", THE Query_API SHALL return at least 1 macro impact record with `macro_impact_score`, `impact_direction`, and `contributing_factors`
5. WHEN a GET request is made to `/api/competitive/status`, THE Query_API SHALL return the competitive layer toggle status with `enabled` field
6. WHEN a GET request is made to `/api/competitive/signals/{ticker}` for "AAPL", THE Query_API SHALL return competitive signal records with `source_ticker`, `target_ticker`, `signal_direction`, and `signal_strength` fields
7. WHEN a GET request is made to `/api/competitive/patterns/{ticker}` for "AAPL", THE Query_API SHALL return historical pattern data (may be empty if no patterns computed)
### Requirement 4: Query API — Operational and Admin Endpoints
**User Story:** As a test engineer, I want tests for the operational dashboard, source management, trading config, approval workflow, and lockout endpoints, so that admin functionality is validated before promotion.
#### Acceptance Criteria
1. WHEN a GET request is made to `/api/ops/pipeline/health`, THE Query_API SHALL return pipeline health with `document_stages`, `parsing`, `extraction`, `aggregation`, and `queue_depths` fields
2. WHEN a GET request is made to `/api/ops/ingestion/summary`, THE Query_API SHALL return ingestion statistics with `total_runs` and `by_source_type` fields
3. WHEN a GET request is made to `/api/ops/sources/coverage-gaps`, THE Query_API SHALL return gap analysis with `missing_source_types` and `stale_sources` lists
4. WHEN a PUT request is made to `/api/sources/{id}/toggle?active=false` for a seeded source, THE Query_API SHALL return HTTP 200 and the source active status SHALL be updated
5. WHEN a GET request is made to `/api/trading/config`, THE Query_API SHALL return the risk configuration with `trading_mode` and `config` fields
6. WHEN a GET request is made to `/api/approvals/pending`, THE Query_API SHALL return a list containing the seeded pending approval record
7. WHEN a GET request is made to `/api/lockouts/active`, THE Query_API SHALL return a list containing the seeded active lockout record
8. WHEN a POST request is made to `/api/lockouts` with a valid ticker and expiry, THE Query_API SHALL create a lockout and return HTTP 200 with the lockout details
9. WHEN a DELETE request is made to `/api/lockouts/{id}` for the created lockout, THE Query_API SHALL remove the lockout and return HTTP 200
### Requirement 5: Query API — Agent and Variant Management Endpoints
**User Story:** As a test engineer, I want tests for the AI agent CRUD, variant lifecycle, and performance endpoints, so that the agent management system is validated.
#### Acceptance Criteria
1. WHEN a GET request is made to `/api/agents`, THE Query_API SHALL return at least 3 seeded agents with `id`, `name`, `slug`, `model_name`, and `active` fields
2. WHEN a GET request is made to `/api/agents/{id}` for a seeded agent, THE Query_API SHALL return the agent detail with `system_prompt`, `temperature`, and `max_tokens` fields
3. WHEN a POST request is made to `/api/agents` with a valid agent body, THE Query_API SHALL create the agent and return HTTP 201 with the new agent including a generated `slug`
4. WHEN a PUT request is made to `/api/agents/{id}` with updated fields, THE Query_API SHALL update the agent and return the modified record
5. WHEN a GET request is made to `/api/agents/{id}/variants` for a seeded agent, THE Query_API SHALL return at least 1 variant with `variant_name`, `model_name`, and `is_active` fields
6. WHEN a POST request is made to `/api/agents/{id}/variants` with a valid variant body, THE Query_API SHALL create the variant and return HTTP 201
7. WHEN a POST request is made to `/api/agents/{id}/variants/{vid}/activate`, THE Query_API SHALL set the variant as active and deactivate any previously active variant for that agent
8. WHEN a GET request is made to `/api/agents/{id}/performance` for a seeded agent, THE Query_API SHALL return performance metrics derived from the agent performance log
9. WHEN a GET request is made to `/api/agents/{id}/variants/{vid}/performance` for a seeded variant, THE Query_API SHALL return variant-level performance metrics
### Requirement 6: Symbol Registry — Write Path and Edge Case Endpoints
**User Story:** As a test engineer, I want tests for the Symbol Registry write paths (create, update, delete) and edge cases (duplicates, not-found, validation), so that data integrity is validated.
#### Acceptance Criteria
1. WHEN a POST request is made to `/companies` with a duplicate ticker and exchange, THE Symbol_Registry SHALL return HTTP 409
2. WHEN a GET request is made to `/companies/{id}` with a non-existent UUID, THE Symbol_Registry SHALL return HTTP 404
3. WHEN a POST request is made to `/companies/{id}/aliases` with a valid alias, THE Symbol_Registry SHALL create the alias and return HTTP 201 with `id`, `alias`, and `alias_type`
4. WHEN a POST request is made to `/companies/{id}/sources` with a valid source body, THE Symbol_Registry SHALL create the source and return HTTP 201 with `id`, `source_type`, and `source_name`
5. WHEN a POST request is made to `/companies/{id}/sources` with an invalid `source_type`, THE Symbol_Registry SHALL return HTTP 422
6. WHEN a POST request is made to `/watchlists` with a valid name, THE Symbol_Registry SHALL create the watchlist and return HTTP 201
7. WHEN a POST request is made to `/watchlists/{id}/members/{company_id}` for a seeded company, THE Symbol_Registry SHALL add the member and return HTTP 201
8. WHEN a GET request is made to `/watchlists/{id}/members` after adding members, THE Symbol_Registry SHALL return the member companies with `ticker` and `legal_name` fields
9. WHEN a POST request is made to `/watchlists` with a duplicate name, THE Symbol_Registry SHALL return HTTP 409
### Requirement 7: Symbol Registry — Competitor and Exposure Write Paths
**User Story:** As a test engineer, I want tests for competitor relationship CRUD and exposure profile upsert, so that the competitive intelligence data layer is validated.
#### Acceptance Criteria
1. WHEN a POST request is made to `/companies/{id}/competitors` with a valid competitor body, THE Symbol_Registry SHALL create the relationship and return HTTP 201 with `relationship_type`, `strength`, and `bidirectional` fields
2. WHEN a POST request is made to `/companies/{id}/competitors` where company_id equals company_b_id, THE Symbol_Registry SHALL return HTTP 400 with a self-referencing error
3. WHEN a PUT request is made to `/companies/{id}/competitors/{rel_id}` with updated strength, THE Symbol_Registry SHALL update the relationship and return the modified record
4. WHEN a DELETE request is made to `/companies/{id}/competitors/{rel_id}`, THE Symbol_Registry SHALL soft-delete the relationship (set active=false) and return HTTP 200
5. WHEN a PUT request is made to `/companies/{id}/exposure` with a valid exposure profile, THE Symbol_Registry SHALL create or update the profile and return the new version with incremented `version` number
6. WHEN a GET request is made to `/companies/{id}/exposure/history` for a company with multiple profile versions, THE Symbol_Registry SHALL return all versions ordered by version descending
7. WHEN a GET request is made to `/companies/{id}/exposure` for a company with no exposure profile, THE Symbol_Registry SHALL return HTTP 404
### Requirement 8: Risk Engine — Evaluation Edge Cases and Approval Workflow
**User Story:** As a test engineer, I want tests for risk evaluation edge cases, the full approval lifecycle, and approval expiry, so that the risk gate is validated.
#### Acceptance Criteria
1. WHEN a POST request is made to `/evaluate` with a minimal order (only ticker), THE Risk_Engine SHALL return a valid evaluation with `evaluation_id`, `eligible`, and `rejection_reasons`
2. WHEN a POST request is made to `/evaluate` with an order exceeding the absolute position cap, THE Risk_Engine SHALL return `eligible: false` with at least one rejection reason
3. WHEN a POST request is made to `/evaluate` with a custom `config` overriding risk parameters, THE Risk_Engine SHALL use the provided config for evaluation
4. WHEN a GET request is made to `/approvals/pending`, THE Risk_Engine SHALL return a list of pending approval records from the seeded data
5. WHEN a GET request is made to `/approvals/{id}` for a seeded pending approval, THE Risk_Engine SHALL return the approval detail with `ticker`, `side`, `quantity`, `status`, and `expires_at` fields
6. WHEN a POST request is made to `/approvals/{id}/review` with `approved: true`, THE Risk_Engine SHALL update the approval status to "approved" and return the new status
7. WHEN a POST request is made to `/approvals/{id}/review` for a non-existent approval, THE Risk_Engine SHALL return HTTP 404
8. WHEN a POST request is made to `/approvals/expire`, THE Risk_Engine SHALL expire stale pending approvals and return the count of expired records
### Requirement 9: Trading Engine — Configuration, Metrics, and Notification Endpoints
**User Story:** As a test engineer, I want tests for trading engine configuration round-trips, metrics consistency, notification config CRUD, and notification history, so that the autonomous trading control plane is validated.
#### Acceptance Criteria
1. WHEN a PUT request is made to `/api/trading/config` with `risk_tier: "aggressive"` followed by a GET to `/api/trading/status`, THE Trading_Engine SHALL reflect the updated risk tier in the status response (Round_Trip)
2. WHEN a POST request is made to `/api/trading/pause` followed by a GET to `/api/trading/status`, THE Trading_Engine SHALL report `paused: true` in the status response
3. WHEN a POST request is made to `/api/trading/resume` followed by a GET to `/api/trading/status`, THE Trading_Engine SHALL report `paused: false` in the status response
4. WHEN a GET request is made to `/api/trading/metrics`, THE Trading_Engine SHALL return all numeric portfolio metrics where `total_portfolio_value` approximately equals `active_pool + reserve_pool + unrealized_pnl`
5. WHEN a GET request is made to `/api/trading/metrics/history`, THE Trading_Engine SHALL return a list of portfolio snapshots with `portfolio_value` and `snapshot_date` fields
6. WHEN a PUT request is made to `/api/trading/notifications/config` with `sms_enabled: true`, THE Trading_Engine SHALL update the notification config and a subsequent GET SHALL reflect the change (Round_Trip)
7. WHEN a GET request is made to `/api/trading/notifications/history`, THE Trading_Engine SHALL return a list of notification records (at least 1 from seed data)
8. WHEN a GET request is made to `/api/trading/decisions` with `ticker` filter, THE Trading_Engine SHALL return only decisions for that ticker
### Requirement 10: Trading Engine — Decision Filtering, Backtest, and Override Validation
**User Story:** As a test engineer, I want tests for decision audit trail filtering, backtest submission, and override order validation, so that the trading engine's write paths and input validation are verified.
#### Acceptance Criteria
1. WHEN a GET request is made to `/api/trading/decisions` with `limit=1`, THE Trading_Engine SHALL return at most 1 decision record
2. WHEN a GET request is made to `/api/trading/decisions` with `decision=act`, THE Trading_Engine SHALL return only decisions with `decision: "act"`
3. WHEN a POST request is made to `/api/trading/override/order` with an invalid ticker (lowercase or special characters), THE Trading_Engine SHALL return HTTP 422
4. WHEN a POST request is made to `/api/trading/override/order` with quantity of 0 or negative, THE Trading_Engine SHALL return HTTP 422
5. WHEN a POST request is made to `/api/trading/override/order` with a valid market order, THE Trading_Engine SHALL return HTTP 202 with `job_id`, `status: "queued"`, `ticker`, `side`, and `quantity` fields, or a structured error if Redis is unavailable
6. IF the Trading_Engine returns HTTP 503 for an override order due to Redis unavailability, THEN THE Trading_Engine SHALL return a JSON error body (not an unhandled exception)
### Requirement 11: Query API — Analytics, Saved Queries, and Schema Introspection
**User Story:** As a test engineer, I want tests for the analytics query engine, saved query CRUD, and schema introspection endpoints, so that the data exploration features are validated.
#### Acceptance Criteria
1. WHEN a GET request is made to `/api/analytics/schema`, THE Query_API SHALL return the Trino schema information (or a structured error if Trino is unavailable)
2. WHEN a GET request is made to `/api/pg/schema`, THE Query_API SHALL return the PostgreSQL schema with table and column information
3. WHEN a GET request is made to `/api/saved-queries`, THE Query_API SHALL return a list of saved queries (at least 1 from seed data or migrations)
4. WHEN a POST request is made to `/api/saved-queries` with a valid query body, THE Query_API SHALL create the saved query and return the record
5. WHEN a DELETE request is made to `/api/saved-queries/{id}` for the created query, THE Query_API SHALL delete the query and return HTTP 200
6. WHEN a POST request is made to `/api/pg/query` with a valid read-only SQL query, THE Query_API SHALL execute the query and return results
### Requirement 12: Cross-Service Round-Trip and Contract Validation
**User Story:** As a test engineer, I want round-trip tests that write data through one service and read it through another, so that cross-service data consistency is validated.
#### Acceptance Criteria
1. WHEN a company is created via POST to Symbol_Registry `/companies` and then queried via GET to Query_API `/api/companies`, THE company SHALL appear in the Query_API response with matching `ticker` and `legal_name`
2. WHEN an exposure profile is created via PUT to Symbol_Registry `/companies/{id}/exposure` and then queried via GET to Symbol_Registry `/companies/{id}/exposure`, THE profile SHALL match the submitted data with `geographic_revenue_mix`, `supply_chain_regions`, and `market_position_tier` fields preserved
3. WHEN a competitor relationship is created via POST to Symbol_Registry `/companies/{id}/competitors` and then queried from both sides, THE relationship SHALL be visible from both company_a and company_b perspectives
4. WHEN a risk evaluation is performed via POST to Risk_Engine `/evaluate` and then the same ticker's recommendation is queried via Query_API `/api/recommendations`, THE recommendation SHALL include a `risk_evaluation` field with matching evaluation structure
### Requirement 13: Error Handling and Empty-State Behavior
**User Story:** As a test engineer, I want tests that verify all endpoints return structured JSON errors (not HTML or stack traces) and handle empty-state gracefully, so that the frontend never receives unexpected response formats.
#### Acceptance Criteria
1. WHEN a GET request is made to any list endpoint with no matching data, THE service SHALL return HTTP 200 with an empty list (not HTTP 404 or an error)
2. WHEN a GET request is made to any detail endpoint with a non-existent UUID, THE service SHALL return HTTP 404 with a JSON body containing an error message
3. WHEN a POST request is made with an invalid JSON body (missing required fields), THE service SHALL return HTTP 422 with a JSON body containing validation error details
4. WHEN a GET request is made to `/api/trends/{id}/projection` for a trend with no projection, THE Query_API SHALL return HTTP 404 with a JSON error (not HTTP 500)
5. WHEN a GET request is made to `/api/macro/impacts/{ticker}` for a ticker with no macro impacts, THE Query_API SHALL return HTTP 200 with an empty list
6. THE Test_Suite SHALL verify that all health endpoints (`/health`) across all four services return `{"status": "ok"}` with HTTP 200
Reference in New Issue View Git Blame Copy Permalink