stonks-oracle/.kiro/specs/beta-api-test-suite/design.md

# Design Document: Beta API Test Suite

## Overview

This design describes a comprehensive integration test suite that validates every HTTP endpoint across the four Stonks Oracle API services (query-api, symbol-registry, risk-engine, trading-engine) that can be tested without external broker or news API dependencies. The suite extends the existing ~56 integration tests to achieve full endpoint coverage, serving as the beta gate that blocks promotion to paper-trading if any test fails.

The test suite operates against an ephemeral Kubernetes sandbox with seeded PostgreSQL data, using deterministic UUIDs for exact equality assertions. It validates read paths, write paths, round-trips, edge cases, error handling, pagination/filtering, and empty-state behavior.

## Architecture

```mermaid
graph TD
    subgraph "K8s Sandbox Namespace"
        PG[(PostgreSQL)]
        Redis[(Redis)]
        MinIO[(MinIO)]
        QA[query-api :8000]
        SR[symbol-registry :8000]
        RE[risk-engine :8000]
        TE[trading-engine :8000]
    end

    subgraph "Test Runner Job"
        SEED[seed_sandbox.py]
        CONF[conftest.py]
        T1[test_query_api.py]
        T2[test_registry_api.py]
        T3[test_risk_api.py]
        T4[test_trading_api.py]
        T5[test_signal_flow.py]
        T6[test_frontend_data_deps.py]
        T7[test_query_api_extended.py]
        T8[test_registry_write_paths.py]
        T9[test_risk_approval_lifecycle.py]
        T10[test_trading_extended.py]
        T11[test_cross_service_roundtrip.py]
        T12[test_error_handling.py]
    end

    SEED -->|INSERT| PG
    CONF -->|httpx.AsyncClient| QA
    CONF -->|httpx.AsyncClient| SR
    CONF -->|httpx.AsyncClient| RE
    CONF -->|httpx.AsyncClient| TE
    QA --> PG
    SR --> PG
    RE --> PG
    TE --> PG
    TE --> Redis
```

The architecture preserves the existing test infrastructure:
- **Runner Job**: K8s Job (`infra/inttest/runner.yaml`) executes pytest in the sandbox namespace
- **Seed Script**: `tests/integration/seed_sandbox.py` populates deterministic data before tests run
- **Fixtures**: `tests/integration/conftest.py` provides `ProfiledAsyncClient` wrappers per service
- **New test files** extend coverage without modifying existing passing tests

## Components and Interfaces

### 1. Extended Seed Script (`seed_sandbox.py`)

New seed functions added to populate tables required by untested endpoints:

| Table | New Records | Purpose |
|-------|-------------|---------|
| `watchlists` | 2 watchlists | Watchlist CRUD tests |
| `watchlist_members` | 3 members | Watchlist membership tests |
| `operator_approvals` | 2 (pending + approved) | Approval lifecycle tests |
| `symbol_lockouts` | 2 (active + expired) | Lockout CRUD tests |
| `notifications` | 1 delivered | Notification history tests |
| `ingestion_runs` | 2 (completed + failed) | Ingestion summary tests |
| `saved_queries` | 1 | Saved query CRUD tests |
| `daily_risk_snapshots` | 1 | Risk snapshot tests |

All new records use deterministic UUIDs exported as named constants.

### 2. New Test Modules

| Module | Service | Coverage |
|--------|---------|----------|
| `test_query_api_extended.py` | query-api | Documents filtering, evidence drill-down, trend projections, macro/competitive endpoints, ops dashboard, agents CRUD, analytics |
| `test_registry_write_paths.py` | symbol-registry | Alias/source/watchlist/competitor/exposure write paths, validation errors, duplicates |
| `test_risk_approval_lifecycle.py` | risk-engine | Full approval lifecycle, evaluation edge cases, expiry |
| `test_trading_extended.py` | trading-engine | Config round-trips, decision filtering, notification CRUD, override validation |
| `test_cross_service_roundtrip.py` | cross-service | Write via one service, read via another |
| `test_error_handling.py` | all services | 404s, 422s, empty lists, structured JSON errors |

### 3. Conftest Extensions

New fixtures added to `conftest.py`:
- Export new seed IDs (watchlists, approvals, lockouts, notifications, ingestion runs, saved queries)
- No changes to existing client fixtures

## Data Models

### New Seed ID Constants

```python
# Watchlists
WATCHLIST_01 = UUID("00000000-0000-4000-ac00-000000000001")
WATCHLIST_02 = UUID("00000000-0000-4000-ac00-000000000002")

# Operator Approvals
APPROVAL_PENDING = UUID("00000000-0000-4000-ad00-000000000001")
APPROVAL_APPROVED = UUID("00000000-0000-4000-ad00-000000000002")

# Symbol Lockouts
LOCKOUT_ACTIVE = UUID("00000000-0000-4000-ae00-000000000001")
LOCKOUT_EXPIRED = UUID("00000000-0000-4000-ae00-000000000002")

# Notifications
NOTIFICATION_01 = UUID("00000000-0000-4000-af00-000000000001")

# Ingestion Runs
INGESTION_RUN_01 = UUID("00000000-0000-4000-b300-000000000001")
INGESTION_RUN_02 = UUID("00000000-0000-4000-b300-000000000002")

# Saved Queries
SAVED_QUERY_01 = UUID("00000000-0000-4000-b400-000000000001")

# Daily Risk Snapshot
RISK_SNAPSHOT_01 = UUID("00000000-0000-4000-b500-000000000001")
```

### Seed ID Export Dictionary Extension

```python
SEED_WATCHLIST_IDS = {
    "WL_01": str(WATCHLIST_01),
    "WL_02": str(WATCHLIST_02),
}
SEED_APPROVAL_IDS = {
    "PENDING": str(APPROVAL_PENDING),
    "APPROVED": str(APPROVAL_APPROVED),
}
SEED_LOCKOUT_IDS = {
    "ACTIVE": str(LOCKOUT_ACTIVE),
    "EXPIRED": str(LOCKOUT_EXPIRED),
}
SEED_NOTIFICATION_IDS = {"NOTIF_01": str(NOTIFICATION_01)}
SEED_INGESTION_RUN_IDS = {
    "RUN_01": str(INGESTION_RUN_01),
    "RUN_02": str(INGESTION_RUN_02),
}
SEED_SAVED_QUERY_IDS = {"SQ_01": str(SAVED_QUERY_01)}
```

## Error Handling

The test suite validates error handling across all services:

1. **404 Not Found**: All detail endpoints return JSON `{"detail": "..."}` for non-existent UUIDs
2. **422 Validation Error**: POST/PUT endpoints return structured validation errors for invalid bodies
3. **409 Conflict**: Duplicate creation attempts return conflict errors
4. **400 Bad Request**: Business logic violations (self-referencing competitors, etc.)
5. **503 Service Unavailable**: Graceful degradation when Redis is unavailable (trading engine override)
6. **Empty Lists**: List endpoints return `[]` with HTTP 200, never 404

Each error response is verified to be valid JSON (not HTML stack traces).

## Testing Strategy

### Approach: Integration Testing with Seeded Data

**Why PBT does not apply**: This is an integration test suite that validates HTTP API contracts against live services with a seeded database. The tests verify:
- Correct HTTP status codes
- Response schema conformance
- Data consistency across services
- Error handling behavior

These are integration tests with concrete, deterministic assertions — not pure functions with universal properties. The input space is fixed (seeded data), and behavior depends on external service state (PostgreSQL, Redis). Property-based testing would require mocking the entire service layer, defeating the purpose of integration testing.

### Test Organization

**Existing tests (preserved as-is)**:
- `test_query_api.py` — 17 tests covering core query API reads
- `test_registry_api.py` — 8 tests covering registry CRUD
- `test_risk_api.py` — 4 tests covering risk evaluation and approvals
- `test_trading_api.py` — 12 tests covering trading engine endpoints
- `test_signal_flow.py` — 11 tests covering cross-service contracts
- `test_frontend_data_deps.py` — 21 tests covering frontend page dependencies

**New tests (added by this spec)**:
- `test_query_api_extended.py` — ~25 tests for uncovered query API endpoints
- `test_registry_write_paths.py` — ~15 tests for registry write paths and edge cases
- `test_risk_approval_lifecycle.py` — ~10 tests for approval workflow and evaluation edge cases
- `test_trading_extended.py` — ~12 tests for trading config round-trips, filtering, notifications
- `test_cross_service_roundtrip.py` — ~6 tests for cross-service data consistency
- `test_error_handling.py` — ~10 tests for structured error responses

**Total target**: ~130+ integration tests (existing ~56 + ~75 new)

### Test Execution

- Framework: pytest + pytest-asyncio
- HTTP client: httpx.AsyncClient wrapped in ProfiledAsyncClient
- Timeout: 30s per request, 600s total Job deadline
- Parallelism: Sequential within each file (shared state from writes), files can run in parallel
- Idempotency: Seed script uses `ON CONFLICT DO NOTHING` for re-runnability

### Naming Convention

Test classes follow the pattern `Test{Service}{Feature}` with descriptive method names:
```python
class TestQueryAPIDocumentFiltering:
    async def test_filter_documents_by_ticker(self, query_client, seed_ids): ...
    async def test_filter_documents_by_doc_type(self, query_client, seed_ids): ...
```

### Assertions

- Status code assertions first (`assert resp.status_code == 200`)
- Schema assertions verify required fields exist
- Value assertions use deterministic seed data for exact equality
- Numeric assertions use approximate comparison where appropriate (portfolio math)
- List assertions verify minimum counts from seed data