stonks-oracle/.kiro/specs/agent-variants/tasks.md

Implementation Tasks: Agent Variants

Phase 1: Database Schema

• 1.1 Create migration infra/migrations/027_agent_variants.sql
  • 1.1.1 Create agent_variants table with columns: id (UUID PK), agent_id (UUID FK to ai_agents ON DELETE CASCADE), variant_name (VARCHAR 200 NOT NULL), variant_slug (VARCHAR 200 NOT NULL), description (TEXT DEFAULT ''), model_provider (VARCHAR 50 DEFAULT 'ollama'), model_name (VARCHAR 200 NOT NULL), system_prompt (TEXT DEFAULT ''), user_prompt_template (TEXT DEFAULT ''), prompt_version (VARCHAR 100 DEFAULT ''), temperature (FLOAT DEFAULT 0.0), max_tokens (INTEGER DEFAULT 32768), context_window (INTEGER DEFAULT 0), input_token_limit (INTEGER DEFAULT 0), token_budget (INTEGER DEFAULT 0), timeout_seconds (INTEGER DEFAULT 120), max_retries (INTEGER DEFAULT 2), is_active (BOOLEAN DEFAULT FALSE), created_at (TIMESTAMPTZ DEFAULT NOW()), updated_at (TIMESTAMPTZ DEFAULT NOW())
  • 1.1.2 Create unique index idx_agent_variants_slug on (agent_id, variant_slug) to prevent duplicate slugs per agent
  • 1.1.3 Create partial unique index idx_agent_variants_active on (agent_id) WHERE is_active = TRUE to enforce at most one active variant per agent
  • 1.1.4 Create index idx_agent_variants_agent on (agent_id) for fast lookup
  • 1.1.5 Add nullable variant_id column (UUID FK to agent_variants ON DELETE SET NULL) to agent_performance_log table
  • 1.1.6 Create index idx_agent_perf_variant on (variant_id, recorded_at DESC) on agent_performance_log

Phase 2: Config Resolution Module

• 2.1 Create services/shared/agent_config.py with AgentConfigResolver
  • 2.1.1 Define ResolvedAgentConfig dataclass with fields: agent_id, variant_id (optional), model_provider, model_name, system_prompt, user_prompt_template, prompt_version, temperature, max_tokens, context_window, input_token_limit, token_budget, timeout_seconds, max_retries
  • 2.1.2 Implement AgentConfigResolver.__init__ accepting asyncpg.Pool and ttl_seconds (default 60)
  • 2.1.3 Implement AgentConfigResolver.resolve(agent_slug) that queries ai_agents LEFT JOIN agent_variants (WHERE is_active = TRUE) using COALESCE to prefer variant values, with TTL-based in-memory cache
  • 2.1.4 Implement cache invalidation: entries expire after ttl_seconds; return None and log warning if DB query fails (callers fall back to env-var defaults)

Phase 3: Backend API Endpoints

• 3.1 Add Pydantic models for variant operations in services/api/app.py

  • 3.1.1 Add VariantCreateBody model with fields: variant_name (str), variant_slug (str | None), description (str = ""), model_provider (str = "ollama"), model_name (str), system_prompt (str = ""), user_prompt_template (str = ""), prompt_version (str = ""), temperature (float = 0.0), max_tokens (int = 32768), context_window (int = 0), input_token_limit (int = 0), token_budget (int = 0), timeout_seconds (int = 120), max_retries (int = 2)
  • 3.1.2 Add VariantUpdateBody model with all config fields optional (str | None, float | None, int | None)
  • 3.1.3 Add VariantCloneBody model with variant_name (str), variant_slug (str | None), and all config fields optional for overrides
  • 3.1.4 Add slug auto-generation helper: _slugify(name: str) -> str that lowercases, replaces non-alphanumeric with hyphens, strips leading/trailing hyphens

• 3.2 Add variant CRUD endpoints in services/api/app.py

  • 3.2.1 GET /api/agents/{agent_id}/variants — list all variants for an agent ordered by created_at ASC, return list of variant dicts
  • 3.2.2 GET /api/agents/{agent_id}/variants/{variant_id} — get single variant, return 404 if not found or agent mismatch
  • 3.2.3 POST /api/agents/{agent_id}/variants — create variant with VariantCreateBody, auto-generate slug if not provided, return 409 on duplicate slug, return 201 with created variant
  • 3.2.4 PUT /api/agents/{agent_id}/variants/{variant_id} — partial update with VariantUpdateBody, set updated_at = NOW(), return 404 if not found
  • 3.2.5 DELETE /api/agents/{agent_id}/variants/{variant_id} — delete variant, return 400 if variant is currently active (is_active = TRUE), return 404 if not found

• 3.3 Add clone endpoints in services/api/app.py

  • 3.3.1 POST /api/agents/{agent_id}/clone — clone agent as variant: fetch agent by id, build variant fields from agent config with VariantCloneBody overrides, auto-generate slug, insert into agent_variants, return 201
  • 3.3.2 POST /api/agents/{agent_id}/variants/{variant_id}/clone — clone variant as new variant: fetch source variant, build new variant fields with overrides, insert, return 201

• 3.4 Add activate/deactivate endpoints in services/api/app.py

  • 3.4.1 POST /api/agents/{agent_id}/variants/{variant_id}/activate — within a transaction: set is_active = FALSE on current active variant for agent, then set is_active = TRUE on target variant, return updated variant
  • 3.4.2 POST /api/agents/{agent_id}/variants/deactivate — set is_active = FALSE on current active variant for agent, return {"deactivated": true}

• 3.5 Add per-variant performance endpoints in services/api/app.py

  • 3.5.1 GET /api/agents/{agent_id}/variants/{variant_id}/performance — aggregated metrics (total invocations, successes, failures, avg/p95 duration, avg confidence, avg retries, total tokens, success rate) filtered by variant_id and time window
  • 3.5.2 GET /api/agents/{agent_id}/variants/{variant_id}/performance/history — hourly time-series filtered by variant_id

Phase 4: Service Integration

• 4.1 Integrate config resolver into Document Extractor

  • 4.1.1 In the extractor service startup or invocation path, instantiate AgentConfigResolver and call resolve("document-extractor") to get runtime config; fall back to OllamaConfig() from env vars if resolve returns None or raises
  • 4.1.2 Build OllamaConfig from ResolvedAgentConfig fields (model_name → config.model, system_prompt, temperature, max_tokens, timeout, max_retries)
  • 4.1.3 Update performance logging to include variant_id from the resolved config when inserting into agent_performance_log

• 4.2 Integrate config resolver into Event Classifier

  • 4.2.1 In classify_global_event or its caller, resolve config for "event-classifier" slug; fall back to existing OllamaConfig if resolution fails
  • 4.2.2 Use resolved model_name and system_prompt when building the Ollama request
  • 4.2.3 Pass variant_id to performance log entries

• 4.3 Integrate config resolver into Thesis Rewriter

  • 4.3.1 In rewrite_thesis_with_llm or its caller, resolve config for "thesis-rewriter" slug; fall back to passed OllamaConfig if resolution fails
  • 4.3.2 Override OllamaConfig fields with resolved values (model, timeout, max_retries)
  • 4.3.3 Pass variant_id to performance log entries if performance logging exists for thesis rewrites

Phase 5: Frontend — Variant List and Detail

• 5.1 Add TypeScript types and API hooks for variants

  • 5.1.1 Add AgentVariant interface to Agents.tsx with fields: id, agent_id, variant_name, variant_slug, description, model_provider, model_name, system_prompt, user_prompt_template, prompt_version, temperature, max_tokens, context_window, input_token_limit, token_budget, timeout_seconds, max_retries, is_active, created_at, updated_at
  • 5.1.2 Add useAgentVariants(agentId) query hook fetching GET /api/agents/{agentId}/variants
  • 5.1.3 Add useVariantPerformance(agentId, variantId, hours) query hook
  • 5.1.4 Add useVariantPerfHistory(agentId, variantId, hours) query hook
  • 5.1.5 Add mutation hooks: useCloneAgentAsVariant, useCreateVariant, useUpdateVariant, useDeleteVariant, useActivateVariant, useDeactivateVariants — each invalidates ['agent-variants'] query on success

• 5.2 Add VariantList component to AgentDetail

  • 5.2.1 Create VariantList component that renders a table/list of variants for the selected agent showing: variant_name, model_name, is_active badge, created_at, and action buttons (Edit, Clone, Delete, Activate)
  • 5.2.2 Add "Clone as Variant" button to the AgentDetail header that opens the clone form pre-filled with agent config
  • 5.2.3 Highlight the active variant row with a distinct badge (e.g. green "Active" StatusBadge)
  • 5.2.4 Wire Activate button to call the activate endpoint and invalidate queries; disable on the already-active variant

• 5.3 Add Variant Create/Edit/Clone forms

  • 5.3.1 Create VariantCloneForm component pre-filled with source (agent or variant) config fields, allowing modification before submit; uses useCloneAgentAsVariant or clone-variant mutation
  • 5.3.2 Create VariantEditForm component pre-filled with current variant config, using useUpdateVariant mutation
  • 5.3.3 Add delete confirmation dialog: show warning, call useDeleteVariant on confirm; show error toast if variant is active (400 response)

Phase 6: Frontend — Comparison View

• 6.1 Add variant comparison UI
  • 6.1.1 Add checkbox selection to each VariantRow; track selected variant IDs in component state
  • 6.1.2 Create VariantCompare component that renders when 2+ variants are selected: shows a table with one column per selected variant, rows for each metric (success rate, avg latency, p95 latency, avg confidence, total tokens)
  • 6.1.3 Add an overlay performance chart in VariantCompare: use Recharts LineChart with one Line per selected variant, shared XAxis (hour), showing success rate or latency over time
  • 6.1.4 Add "Activate" button in comparison view for each non-active variant column

Phase 7: Tests

• 7.1 Add backend tests for variant API

  • 7.1.1 Add tests/test_agent_variants_api.py with example tests for: create variant, clone from agent, clone from variant, list variants, get variant, update variant, delete variant, delete active variant (expect 400), activate/deactivate, variant performance queries
  • 7.1.2 Add edge-case tests: duplicate slug (409), non-existent agent (404), non-existent variant (404), empty model_name validation

• 7.2 Add property-based tests for variant logic

  • 7.2.1 [PBT] Property: Single active variant invariant — generate random sequences of activate/deactivate operations, assert at most one active variant per agent after each operation
  • 7.2.2 [PBT] Property: Clone preserves unoverridden fields — generate random agent configs and random override subsets, assert non-overridden fields match source
  • 7.2.3 [PBT] Property: Config resolution prefers active variant — generate agent + variants with random active state, assert resolver returns correct config source
  • 7.2.4 [PBT] Property: Slug auto-generation determinism — generate random names, assert slugify is deterministic, produces valid kebab-case, and is non-empty for non-empty input
  • 7.2.5 [PBT] Property: Partial update idempotence — generate variant + random update subset, apply twice, assert fields match (excluding updated_at)

• 7.3 Add frontend tests

  • 7.3.1 Add MSW handlers in frontend/src/test/mocks/handlers.ts for all variant endpoints (list, get, create, clone, update, delete, activate, deactivate, performance, history)
  • 7.3.2 Add test in frontend/src/test/pages.test.tsx verifying the Agents page renders variant list when an agent is selected, and that the comparison view appears when multiple variants are checked