stonks-oracle/.kiro/specs/beta-api-test-suite/design.md

Design Document: Beta API Test Suite

Overview

This design describes a comprehensive integration test suite that validates every HTTP endpoint across the four Stonks Oracle API services (query-api, symbol-registry, risk-engine, trading-engine) that can be tested without external broker or news API dependencies. The suite extends the existing ~56 integration tests to achieve full endpoint coverage, serving as the beta gate that blocks promotion to paper-trading if any test fails.

The test suite operates against an ephemeral Kubernetes sandbox with seeded PostgreSQL data, using deterministic UUIDs for exact equality assertions. It validates read paths, write paths, round-trips, edge cases, error handling, pagination/filtering, and empty-state behavior.

Architecture

graph TD
    subgraph "K8s Sandbox Namespace"
        PG[(PostgreSQL)]
        Redis[(Redis)]
        MinIO[(MinIO)]
        QA[query-api :8000]
        SR[symbol-registry :8000]
        RE[risk-engine :8000]
        TE[trading-engine :8000]
    end

    subgraph "Test Runner Job"
        SEED[seed_sandbox.py]
        CONF[conftest.py]
        T1[test_query_api.py]
        T2[test_registry_api.py]
        T3[test_risk_api.py]
        T4[test_trading_api.py]
        T5[test_signal_flow.py]
        T6[test_frontend_data_deps.py]
        T7[test_query_api_extended.py]
        T8[test_registry_write_paths.py]
        T9[test_risk_approval_lifecycle.py]
        T10[test_trading_extended.py]
        T11[test_cross_service_roundtrip.py]
        T12[test_error_handling.py]
    end

    SEED -->|INSERT| PG
    CONF -->|httpx.AsyncClient| QA
    CONF -->|httpx.AsyncClient| SR
    CONF -->|httpx.AsyncClient| RE
    CONF -->|httpx.AsyncClient| TE
    QA --> PG
    SR --> PG
    RE --> PG
    TE --> PG
    TE --> Redis

The architecture preserves the existing test infrastructure:

• Runner Job: K8s Job (infra/inttest/runner.yaml) executes pytest in the sandbox namespace
• Seed Script: tests/integration/seed_sandbox.py populates deterministic data before tests run
• Fixtures: tests/integration/conftest.py provides ProfiledAsyncClient wrappers per service
• New test files extend coverage without modifying existing passing tests

Components and Interfaces

1. Extended Seed Script (seed_sandbox.py)

New seed functions added to populate tables required by untested endpoints:

Table	New Records	Purpose
watchlists	2 watchlists	Watchlist CRUD tests
watchlist_members	3 members	Watchlist membership tests
operator_approvals	2 (pending + approved)	Approval lifecycle tests
symbol_lockouts	2 (active + expired)	Lockout CRUD tests
notifications	1 delivered	Notification history tests
ingestion_runs	2 (completed + failed)	Ingestion summary tests
saved_queries	1	Saved query CRUD tests
daily_risk_snapshots	1	Risk snapshot tests

All new records use deterministic UUIDs exported as named constants.

2. New Test Modules

Module	Service	Coverage
test_query_api_extended.py	query-api	Documents filtering, evidence drill-down, trend projections, macro/competitive endpoints, ops dashboard, agents CRUD, analytics
test_registry_write_paths.py	symbol-registry	Alias/source/watchlist/competitor/exposure write paths, validation errors, duplicates
test_risk_approval_lifecycle.py	risk-engine	Full approval lifecycle, evaluation edge cases, expiry
test_trading_extended.py	trading-engine	Config round-trips, decision filtering, notification CRUD, override validation
test_cross_service_roundtrip.py	cross-service	Write via one service, read via another
test_error_handling.py	all services	404s, 422s, empty lists, structured JSON errors

3. Conftest Extensions

New fixtures added to conftest.py:

• Export new seed IDs (watchlists, approvals, lockouts, notifications, ingestion runs, saved queries)
• No changes to existing client fixtures

Data Models

New Seed ID Constants

# Watchlists
WATCHLIST_01 = UUID("00000000-0000-4000-ac00-000000000001")
WATCHLIST_02 = UUID("00000000-0000-4000-ac00-000000000002")

# Operator Approvals
APPROVAL_PENDING = UUID("00000000-0000-4000-ad00-000000000001")
APPROVAL_APPROVED = UUID("00000000-0000-4000-ad00-000000000002")

# Symbol Lockouts
LOCKOUT_ACTIVE = UUID("00000000-0000-4000-ae00-000000000001")
LOCKOUT_EXPIRED = UUID("00000000-0000-4000-ae00-000000000002")

# Notifications
NOTIFICATION_01 = UUID("00000000-0000-4000-af00-000000000001")

# Ingestion Runs
INGESTION_RUN_01 = UUID("00000000-0000-4000-b300-000000000001")
INGESTION_RUN_02 = UUID("00000000-0000-4000-b300-000000000002")

# Saved Queries
SAVED_QUERY_01 = UUID("00000000-0000-4000-b400-000000000001")

# Daily Risk Snapshot
RISK_SNAPSHOT_01 = UUID("00000000-0000-4000-b500-000000000001")

Seed ID Export Dictionary Extension

SEED_WATCHLIST_IDS = {
    "WL_01": str(WATCHLIST_01),
    "WL_02": str(WATCHLIST_02),
}
SEED_APPROVAL_IDS = {
    "PENDING": str(APPROVAL_PENDING),
    "APPROVED": str(APPROVAL_APPROVED),
}
SEED_LOCKOUT_IDS = {
    "ACTIVE": str(LOCKOUT_ACTIVE),
    "EXPIRED": str(LOCKOUT_EXPIRED),
}
SEED_NOTIFICATION_IDS = {"NOTIF_01": str(NOTIFICATION_01)}
SEED_INGESTION_RUN_IDS = {
    "RUN_01": str(INGESTION_RUN_01),
    "RUN_02": str(INGESTION_RUN_02),
}
SEED_SAVED_QUERY_IDS = {"SQ_01": str(SAVED_QUERY_01)}

Error Handling

The test suite validates error handling across all services:

1. 404 Not Found: All detail endpoints return JSON {"detail": "..."} for non-existent UUIDs
2. 422 Validation Error: POST/PUT endpoints return structured validation errors for invalid bodies
3. 409 Conflict: Duplicate creation attempts return conflict errors
4. 400 Bad Request: Business logic violations (self-referencing competitors, etc.)
5. 503 Service Unavailable: Graceful degradation when Redis is unavailable (trading engine override)
6. Empty Lists: List endpoints return [] with HTTP 200, never 404

Each error response is verified to be valid JSON (not HTML stack traces).

Testing Strategy

Approach: Integration Testing with Seeded Data

Why PBT does not apply: This is an integration test suite that validates HTTP API contracts against live services with a seeded database. The tests verify:

• Correct HTTP status codes
• Response schema conformance
• Data consistency across services
• Error handling behavior

These are integration tests with concrete, deterministic assertions — not pure functions with universal properties. The input space is fixed (seeded data), and behavior depends on external service state (PostgreSQL, Redis). Property-based testing would require mocking the entire service layer, defeating the purpose of integration testing.

Test Organization

Existing tests (preserved as-is):

• test_query_api.py — 17 tests covering core query API reads
• test_registry_api.py — 8 tests covering registry CRUD
• test_risk_api.py — 4 tests covering risk evaluation and approvals
• test_trading_api.py — 12 tests covering trading engine endpoints
• test_signal_flow.py — 11 tests covering cross-service contracts
• test_frontend_data_deps.py — 21 tests covering frontend page dependencies

New tests (added by this spec):

• test_query_api_extended.py — ~25 tests for uncovered query API endpoints
• test_registry_write_paths.py — ~15 tests for registry write paths and edge cases
• test_risk_approval_lifecycle.py — ~10 tests for approval workflow and evaluation edge cases
• test_trading_extended.py — ~12 tests for trading config round-trips, filtering, notifications
• test_cross_service_roundtrip.py — ~6 tests for cross-service data consistency
• test_error_handling.py — ~10 tests for structured error responses

Total target: ~130+ integration tests (existing ~56 + ~75 new)

Test Execution

• Framework: pytest + pytest-asyncio
• HTTP client: httpx.AsyncClient wrapped in ProfiledAsyncClient
• Timeout: 30s per request, 600s total Job deadline
• Parallelism: Sequential within each file (shared state from writes), files can run in parallel
• Idempotency: Seed script uses ON CONFLICT DO NOTHING for re-runnability

Naming Convention

Test classes follow the pattern Test{Service}{Feature} with descriptive method names:

class TestQueryAPIDocumentFiltering:
    async def test_filter_documents_by_ticker(self, query_client, seed_ids): ...
    async def test_filter_documents_by_doc_type(self, query_client, seed_ids): ...

Assertions

• Status code assertions first (assert resp.status_code == 200)
• Schema assertions verify required fields exist
• Value assertions use deterministic seed data for exact equality
• Numeric assertions use approximate comparison where appropriate (portfolio math)
• List assertions verify minimum counts from seed data