feat: agent variants — migration, API, service integration, frontend, tests

- Migration 027: agent_variants table with single-active enforcement,
  variant_id column on agent_performance_log
- API: full CRUD, clone from agent/variant, activate/deactivate,
  per-variant performance metrics and history endpoints
- Services: extractor, event classifier, thesis rewriter all wired
  to AgentConfigResolver with variant override support
- Frontend: variant list, comparison view, create/edit/clone forms,
  activate/delete actions on Agents page
- Tests: API tests + 5 property-based tests (single-active invariant,
  clone preservation, config resolution, slug determinism, update idempotence)
- Spec files for agent-variants feature
This commit is contained in:
Celes Renata
2026-04-17 05:15:42 +00:00
parent 734bf001a7
commit 7c23c044d7
14 changed files with 3118 additions and 120 deletions
+1
View File
@@ -0,0 +1 @@
{"specId": "ed016b42-56e9-435c-891d-f585c27796df", "workflowType": "requirements-first", "specType": "feature"}
+514
View File
@@ -0,0 +1,514 @@
# Design Document: Agent Variants
## Overview
Add variant support to the existing AI agents system so each agent can have multiple configurations (different models, prompts, parameters) that can be independently tracked, compared, and swapped into production. This builds on the existing `ai_agents` table, `agent_performance_log` table, API endpoints, and frontend Agents page.
## Architecture
### System Context
```
┌──────────────┐ ┌──────────────────┐ ┌───────────────┐
│ Agents Page │────▶│ Query API │────▶│ PostgreSQL │
│ (React) │ │ (FastAPI) │ │ ai_agents │
│ │ │ │ │ agent_variants│
│ - List │ │ /api/agents/ │ │ agent_perf_log│
│ - Compare │ │ /api/agents/ │ └───────────────┘
│ - Activate │ │ {id}/variants/ │
└──────────────┘ └──────────────────┘
┌────────┴────────┐
│ Config Resolver │
│ (shared module) │
└────────┬────────┘
┌───────────────────┼───────────────────┐
▼ ▼ ▼
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Extractor │ │ Event │ │ Thesis │
│ (client.py) │ │ Classifier │ │ Rewriter │
└──────┬───────┘ └──────┬───────┘ └──────┬───────┘
│ │ │
└──────────────────┼──────────────────┘
┌──────────────┐
│ Ollama │
│ Service │
└──────────────┘
```
### Key Design Decisions
1. **Variants as a separate table** (not rows in `ai_agents`): Keeps the parent agent as the canonical role definition. Variants are children that override config fields. This avoids polluting the existing agent table with parent/child semantics and keeps backward compatibility.
2. **Partial unique index for single-active enforcement**: Rather than application-level logic, a PostgreSQL partial unique index on `(agent_id) WHERE is_active = TRUE` guarantees at most one active variant per agent at the database level.
3. **Shared config resolver module**: A new `services/shared/agent_config.py` module encapsulates the "resolve active config for an agent slug" logic with TTL caching. All three services import this instead of duplicating resolution logic.
4. **Nullable variant_id on performance log**: Adding `variant_id` as nullable to `agent_performance_log` preserves backward compatibility — existing rows have NULL, new invocations record the variant when applicable.
5. **No schema_version on variants**: Variants inherit the parent agent's schema_version since that defines the output structure, not a tuning parameter. Variants override model, prompt, and inference parameters only.
## Database Schema
### Migration 027: Agent Variants
```sql
-- Agent variant configurations: alternative model/prompt/parameter sets per agent.
CREATE TABLE IF NOT EXISTS agent_variants (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
agent_id UUID NOT NULL REFERENCES ai_agents(id) ON DELETE CASCADE,
variant_name VARCHAR(200) NOT NULL,
variant_slug VARCHAR(200) NOT NULL,
description TEXT NOT NULL DEFAULT '',
model_provider VARCHAR(50) NOT NULL DEFAULT 'ollama',
model_name VARCHAR(200) NOT NULL,
system_prompt TEXT NOT NULL DEFAULT '',
user_prompt_template TEXT NOT NULL DEFAULT '',
prompt_version VARCHAR(100) NOT NULL DEFAULT '',
temperature FLOAT DEFAULT 0.0,
max_tokens INTEGER DEFAULT 32768,
context_window INTEGER DEFAULT 0, -- Ollama num_ctx; 0 = use model default
input_token_limit INTEGER DEFAULT 0, -- max input tokens before truncation; 0 = no limit
token_budget INTEGER DEFAULT 0, -- total tokens per hour; 0 = unlimited
timeout_seconds INTEGER DEFAULT 120,
max_retries INTEGER DEFAULT 2,
is_active BOOLEAN NOT NULL DEFAULT FALSE,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
-- Each agent can have many variants, but variant slugs must be unique per agent.
CREATE UNIQUE INDEX IF NOT EXISTS idx_agent_variants_slug
ON agent_variants(agent_id, variant_slug);
-- At most one active variant per agent (database-enforced invariant).
CREATE UNIQUE INDEX IF NOT EXISTS idx_agent_variants_active
ON agent_variants(agent_id) WHERE is_active = TRUE;
-- Fast lookup by agent.
CREATE INDEX IF NOT EXISTS idx_agent_variants_agent
ON agent_variants(agent_id);
-- Add variant_id to performance log for per-variant attribution.
ALTER TABLE agent_performance_log
ADD COLUMN IF NOT EXISTS variant_id UUID REFERENCES agent_variants(id) ON DELETE SET NULL;
CREATE INDEX IF NOT EXISTS idx_agent_perf_variant
ON agent_performance_log(variant_id, recorded_at DESC);
```
### Entity Relationships
```
ai_agents (1) ──── (0..*) agent_variants
│ │
│ │ (variant_id, nullable)
└───── agent_performance_log ◀──┘
(agent_id, required)
```
## API Design
### Pydantic Models
```python
class VariantCreateBody(BaseModel):
variant_name: str
variant_slug: str | None = None # auto-generated from name if omitted
description: str = ""
model_provider: str = "ollama"
model_name: str
system_prompt: str = ""
user_prompt_template: str = ""
prompt_version: str = ""
temperature: float = 0.0
max_tokens: int = 32768
context_window: int = 0 # Ollama num_ctx; 0 = model default
input_token_limit: int = 0 # max input tokens; 0 = no limit
token_budget: int = 0 # tokens per hour; 0 = unlimited
timeout_seconds: int = 120
max_retries: int = 2
class VariantUpdateBody(BaseModel):
variant_name: str | None = None
description: str | None = None
model_provider: str | None = None
model_name: str | None = None
system_prompt: str | None = None
user_prompt_template: str | None = None
prompt_version: str | None = None
temperature: float | None = None
max_tokens: int | None = None
context_window: int | None = None
input_token_limit: int | None = None
token_budget: int | None = None
timeout_seconds: int | None = None
max_retries: int | None = None
class VariantCloneBody(BaseModel):
variant_name: str
variant_slug: str | None = None
# All config fields optional — omitted fields inherit from source
description: str | None = None
model_provider: str | None = None
model_name: str | None = None
system_prompt: str | None = None
user_prompt_template: str | None = None
prompt_version: str | None = None
temperature: float | None = None
max_tokens: int | None = None
context_window: int | None = None
input_token_limit: int | None = None
token_budget: int | None = None
timeout_seconds: int | None = None
max_retries: int | None = None
```
### Endpoints
| Method | Path | Description | Req |
|--------|------|-------------|-----|
| GET | `/api/agents/{agent_id}/variants` | List all variants for an agent | 3.1 |
| GET | `/api/agents/{agent_id}/variants/{variant_id}` | Get a single variant | 3.2 |
| POST | `/api/agents/{agent_id}/variants` | Create a variant (direct create) | 3 |
| POST | `/api/agents/{agent_id}/clone` | Clone agent as variant | 2.1 |
| POST | `/api/agents/{agent_id}/variants/{variant_id}/clone` | Clone variant as new variant | 2.2 |
| PUT | `/api/agents/{agent_id}/variants/{variant_id}` | Update a variant | 3.4 |
| DELETE | `/api/agents/{agent_id}/variants/{variant_id}` | Delete a variant | 3.5 |
| POST | `/api/agents/{agent_id}/variants/{variant_id}/activate` | Set variant as active | 4.1 |
| POST | `/api/agents/{agent_id}/variants/deactivate` | Deactivate current active variant | 4.2 |
| GET | `/api/agents/{agent_id}/variants/{variant_id}/performance` | Variant performance metrics | 6.3 |
| GET | `/api/agents/{agent_id}/variants/{variant_id}/performance/history` | Variant performance time-series | 6.4 |
### Clone Endpoint Logic (POST `/api/agents/{agent_id}/clone`)
```python
# 1. Fetch source agent
agent = await pool.fetchrow("SELECT * FROM ai_agents WHERE id = $1", agent_id)
# 2. Build variant record: start with agent fields, overlay user overrides
variant_data = {
"model_provider": body.model_provider or agent["model_provider"],
"model_name": body.model_name or agent["model_name"],
"system_prompt": body.system_prompt if body.system_prompt is not None else agent["system_prompt"],
# ... etc for all config fields
}
# 3. Generate slug if not provided
slug = body.variant_slug or slugify(body.variant_name)
# 4. Insert into agent_variants
row = await pool.fetchrow(
"INSERT INTO agent_variants (...) VALUES (...) RETURNING *",
agent_id, body.variant_name, slug, **variant_data
)
```
### Activate Endpoint Logic (POST `.../activate`)
```python
# Single transaction: deactivate previous, activate target
async with pool.acquire() as conn:
async with conn.transaction():
await conn.execute(
"UPDATE agent_variants SET is_active = FALSE, updated_at = NOW() "
"WHERE agent_id = $1 AND is_active = TRUE", agent_id
)
row = await conn.fetchrow(
"UPDATE agent_variants SET is_active = TRUE, updated_at = NOW() "
"WHERE id = $1 AND agent_id = $2 RETURNING *",
variant_id, agent_id
)
```
## Config Resolution Module
### `services/shared/agent_config.py`
```python
@dataclass
class ResolvedAgentConfig:
"""Runtime configuration resolved from DB agent + optional active variant."""
agent_id: str
variant_id: str | None
model_provider: str
model_name: str
system_prompt: str
user_prompt_template: str
prompt_version: str
temperature: float
max_tokens: int
context_window: int
input_token_limit: int
token_budget: int
timeout_seconds: int
max_retries: int
class AgentConfigResolver:
"""Resolves agent configuration from DB with active variant override and TTL cache."""
def __init__(self, pool: asyncpg.Pool, ttl_seconds: int = 60):
self._pool = pool
self._ttl = ttl_seconds
self._cache: dict[str, tuple[float, ResolvedAgentConfig]] = {}
async def resolve(self, agent_slug: str) -> ResolvedAgentConfig | None:
"""Resolve config for an agent slug, preferring active variant if present."""
now = time.monotonic()
cached = self._cache.get(agent_slug)
if cached and (now - cached[0]) < self._ttl:
return cached[1]
# Query: LEFT JOIN active variant onto agent
row = await self._pool.fetchrow("""
SELECT a.id AS agent_id,
v.id AS variant_id,
COALESCE(v.model_provider, a.model_provider) AS model_provider,
COALESCE(v.model_name, a.model_name) AS model_name,
COALESCE(v.system_prompt, a.system_prompt) AS system_prompt,
COALESCE(v.user_prompt_template, a.user_prompt_template) AS user_prompt_template,
COALESCE(v.prompt_version, a.prompt_version) AS prompt_version,
COALESCE(v.temperature, a.temperature) AS temperature,
COALESCE(v.max_tokens, a.max_tokens) AS max_tokens,
COALESCE(v.context_window, 0) AS context_window,
COALESCE(v.input_token_limit, 0) AS input_token_limit,
COALESCE(v.token_budget, 0) AS token_budget,
COALESCE(v.timeout_seconds, a.timeout_seconds) AS timeout_seconds,
COALESCE(v.max_retries, a.max_retries) AS max_retries
FROM ai_agents a
LEFT JOIN agent_variants v ON v.agent_id = a.id AND v.is_active = TRUE
WHERE a.slug = $1 AND a.active = TRUE
""", agent_slug)
if not row:
return None
config = ResolvedAgentConfig(
agent_id=str(row["agent_id"]),
variant_id=str(row["variant_id"]) if row["variant_id"] else None,
model_provider=row["model_provider"],
model_name=row["model_name"],
system_prompt=row["system_prompt"],
user_prompt_template=row["user_prompt_template"],
prompt_version=row["prompt_version"],
temperature=row["temperature"],
max_tokens=row["max_tokens"],
context_window=row["context_window"],
input_token_limit=row["input_token_limit"],
token_budget=row["token_budget"],
timeout_seconds=row["timeout_seconds"],
max_retries=row["max_retries"],
)
self._cache[agent_slug] = (now, config)
return config
```
## Service Integration Points
### Document Extractor (`services/extractor/client.py`)
The `OllamaClient` currently receives an `OllamaConfig` at construction time. Integration approach:
1. Before creating `OllamaClient`, call `resolver.resolve("document-extractor")`
2. If resolved, build an `OllamaConfig` from the resolved values
3. If resolution fails (DB down), fall back to env-var `OllamaConfig()`
4. Pass resolved config to `OllamaClient.__init__`
5. After extraction, log to `agent_performance_log` with both `agent_id` and `variant_id`
### Event Classifier (`services/extractor/event_classifier.py`)
The `classify_global_event` function receives an `ollama_client` (OllamaClient). Same pattern:
1. Resolve config via `resolver.resolve("event-classifier")`
2. If resolved, construct an OllamaClient with the resolved config
3. Pass the variant_id through to performance logging
### Thesis Rewriter (`services/recommendation/thesis_llm.py`)
The `rewrite_thesis_with_llm` function receives an `OllamaConfig` directly:
1. Resolve config via `resolver.resolve("thesis-rewriter")`
2. If resolved, override the `config` parameter with resolved values
3. Log variant_id in performance metrics
### Performance Logging Changes
The existing `agent_performance_log` INSERT statements need to include `variant_id`:
```python
await pool.execute(
"""INSERT INTO agent_performance_log
(agent_id, variant_id, document_id, ticker, success, duration_ms,
confidence, retry_count, input_tokens, output_tokens, error_message)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11)""",
agent_id, variant_id, # variant_id may be None
document_id, ticker, success, duration_ms,
confidence, retry_count, input_tokens, output_tokens, error_message,
)
```
## Frontend Design
### Agents Page Changes
The existing `AgentsPage` component gets extended with variant management. The layout stays the same (sidebar + detail panel), with variant sections added inside the detail panel.
### Component Hierarchy
```
AgentsPage
├── AgentListSidebar (existing)
├── AgentDetail (existing, extended)
│ ├── Agent config card (existing)
│ ├── Agent performance card (existing)
│ ├── VariantList (new)
│ │ ├── VariantRow (name, model, active badge, actions)
│ │ └── "Clone as Variant" button
│ ├── VariantCompare (new, shown when 2+ variants selected)
│ │ ├── MetricsComparisonTable
│ │ └── OverlayPerformanceChart
│ └── VariantDetail (new, shown when single variant selected)
│ ├── Variant config card
│ └── Variant performance card
├── VariantCreateForm (new)
├── VariantEditForm (new)
└── VariantCloneForm (new)
```
### New TanStack Query Hooks
```typescript
// api/hooks.ts additions
function useAgentVariants(agentId: string | undefined)
function useVariantPerformance(agentId: string, variantId: string, hours?: number)
function useVariantPerfHistory(agentId: string, variantId: string, hours?: number)
// Mutations
function useCloneAgentAsVariant(agentId: string)
function useCloneVariant(agentId: string, variantId: string)
function useCreateVariant(agentId: string)
function useUpdateVariant(agentId: string, variantId: string)
function useDeleteVariant(agentId: string, variantId: string)
function useActivateVariant(agentId: string, variantId: string)
function useDeactivateVariants(agentId: string)
```
### TypeScript Types
```typescript
interface AgentVariant {
id: string;
agent_id: string;
variant_name: string;
variant_slug: string;
description: string;
model_provider: string;
model_name: string;
system_prompt: string;
user_prompt_template: string;
prompt_version: string;
temperature: number;
max_tokens: number;
context_window: number;
input_token_limit: number;
token_budget: number;
timeout_seconds: number;
max_retries: number;
is_active: boolean;
created_at: string;
updated_at: string;
}
```
### Comparison View Behavior
1. Each `VariantRow` has a checkbox for comparison selection
2. When 2+ variants are checked, `VariantCompare` renders above the variant list
3. Comparison table shows metrics side-by-side (columns = variants)
4. Chart overlays performance history lines for selected variants on shared axes
5. "Activate" button in comparison view calls the activate endpoint and invalidates queries
## Correctness Properties
### Property 1: Single Active Variant Invariant (Req 1.4, 4.1)
For any sequence of activate/deactivate operations on variants of an agent, at most one variant per agent has `is_active = TRUE` at any point in time.
- **Test approach**: Property-based test generating random sequences of create-variant + activate + deactivate operations, then asserting the DB invariant holds after each operation.
- **Generator**: Random agent with 1-5 variants, random sequence of 1-20 activate/deactivate calls.
- **Assertion**: `SELECT COUNT(*) FROM agent_variants WHERE agent_id = $1 AND is_active = TRUE` returns 0 or 1.
### Property 2: Clone Preserves Unoverridden Fields (Req 2.1, 2.3)
For any agent and any subset of override fields, cloning produces a variant where: overridden fields match the override values, and non-overridden fields match the source agent's values.
- **Test approach**: Property-based test generating random agent configs and random subsets of overrides.
- **Generator**: Random agent config (model_name, temperature, max_tokens, etc.) + random subset of override values.
- **Assertion**: For each field, if an override was provided the variant has the override value; otherwise it matches the source.
### Property 3: Config Resolution Prefers Active Variant (Req 4.3, 4.4, 9.1-9.3)
For any agent with N variants, `AgentConfigResolver.resolve(slug)` returns the active variant's config when one exists, and the base agent config when none is active.
- **Test approach**: Property-based test. Generate agent + 0-5 variants with 0 or 1 active. Resolve and verify returned config matches the expected source.
- **Generator**: Random agent config, random variant configs, random active/inactive state.
- **Assertion**: If an active variant exists, all config fields in the resolved result match the variant. If no active variant, all fields match the base agent.
### Property 4: Variant Performance Metrics Consistency (Req 6.3, 6.5)
For any agent with variants and performance log entries, the agent-level aggregated metrics are always >= variant-level metrics for any single variant (since agent-level includes all variants).
- **Test approach**: Property-based test. Generate performance log entries attributed to different variants of the same agent. Query agent-level and variant-level metrics. Assert agent total >= variant total for each metric.
- **Generator**: Random agent with 2-4 variants, 5-50 random performance log entries distributed across variants.
- **Assertion**: `agent.total_invocations >= variant.total_invocations` for each variant. Same for success count, token totals.
### Property 5: Partial Update Idempotence (Req 3.4)
For any variant, applying an update with a subset of fields, then applying the same update again, produces the same variant state (ignoring updated_at). The update operation is idempotent on the data fields.
- **Test approach**: Property-based test. Generate a variant, apply random partial update, read result, apply same update again, read result. Assert all fields except updated_at are identical.
- **Generator**: Random variant + random subset of updatable fields with random values.
- **Assertion**: `variant_after_first_update.fields == variant_after_second_update.fields` (excluding updated_at).
### Property 6: TTL Cache Expiry (Req 9.5)
Within the TTL window, the resolver returns cached config without querying the DB. After TTL expires, the resolver re-queries and reflects any changes.
- **Test approach**: Property-based test. Resolve config, change the active variant in the DB, resolve again within TTL (should get old value), advance time past TTL, resolve again (should get new value).
- **Generator**: Random agent configs with different variant configs. Random TTL values (1-120s).
- **Assertion**: Pre-TTL resolve returns original config. Post-TTL resolve returns updated config.
### Property 7: Slug Auto-Generation Determinism (Req 2.4)
For any variant_name, the auto-generated slug is deterministic (same name → same slug), is a valid kebab-case string (lowercase, alphanumeric + hyphens, no leading/trailing hyphens), and is non-empty for any non-empty name.
- **Test approach**: Property-based test over random variant names.
- **Generator**: Random non-empty strings with unicode, spaces, special characters.
- **Assertion**: `slugify(name) == slugify(name)` (deterministic), slug matches `^[a-z0-9]+(-[a-z0-9]+)*$`, slug is non-empty.
## File Changes Summary
### New Files
| File | Purpose |
|------|---------|
| `infra/migrations/027_agent_variants.sql` | Migration: agent_variants table + performance log variant_id column |
| `services/shared/agent_config.py` | AgentConfigResolver with TTL cache |
### Modified Files
| File | Change |
|------|--------|
| `services/api/app.py` | Add variant CRUD, clone, activate/deactivate, performance endpoints + Pydantic models |
| `services/extractor/client.py` | Accept optional resolved config; pass variant_id to perf logging |
| `services/extractor/event_classifier.py` | Use resolver for runtime config; pass variant_id to perf logging |
| `services/recommendation/thesis_llm.py` | Use resolver for runtime config; pass variant_id to perf logging |
| `frontend/src/pages/Agents.tsx` | Add variant list, comparison, create/edit/clone forms, activate/delete actions |
| `frontend/src/test/mocks/handlers.ts` | Add MSW handlers for variant endpoints |
### Test Files
| File | Purpose |
|------|---------|
| `tests/test_pbt_agent_variants.py` | Property-based tests for variant invariants, clone, config resolution |
| `tests/test_agent_variants_api.py` | Example/edge-case tests for API endpoints |
| `frontend/src/test/pages.test.tsx` | Frontend tests for variant UI components |
+142
View File
@@ -0,0 +1,142 @@
# Requirements Document
## Introduction
Add variant support to the existing AI agents system. Each agent (Document Intelligence Extractor, Global Event Classifier, Thesis Rewriter) can have multiple variants — different model, prompt, and parameter configurations — enabling A/B testing, model comparison, and iterative prompt engineering. Users can clone agents as variants, track per-variant performance, compare variants side-by-side, and swap which variant is the active one running in production for a given agent role.
## Glossary
- **Agent**: A record in the `ai_agents` table representing an AI role (e.g. Document Intelligence Extractor). Each Agent has a purpose, model configuration, and prompts.
- **Variant**: A child configuration of an Agent that inherits the parent Agent's role and purpose but allows independent model, prompt, and parameter overrides. Stored in a new `agent_variants` table.
- **Active_Variant**: The single Variant (or the base Agent configuration) currently designated to execute in production for a given Agent role. Only one Variant per Agent can be active at a time.
- **Base_Configuration**: The original Agent's model, prompt, and parameter settings before any Variant is created. Serves as the default Active_Variant when no Variant has been promoted.
- **Variant_Performance**: Per-invocation metrics (success rate, latency, confidence, token usage) attributed to a specific Variant rather than just the parent Agent.
- **Agents_Page**: The existing React frontend page at `/agents` that displays agent configurations and performance metrics.
- **Ollama_Service**: The local LLM inference service at `ollama.ollama-service.svc.cluster.local:11434` used by all three system agents.
- **Clone_Operation**: The act of creating a new Variant from an existing Agent or Variant, copying all configuration fields while allowing the user to modify them.
## Requirements
### Requirement 1: Variant Data Model
**User Story:** As a developer, I want each agent to support multiple variant configurations stored in the database, so that I can experiment with different models and prompts without modifying the base agent.
#### Acceptance Criteria
1. THE Database SHALL store Variant records in an `agent_variants` table with columns: id (UUID), agent_id (FK to ai_agents), variant_name, variant_slug, description, model_provider, model_name, system_prompt, user_prompt_template, prompt_version, temperature, max_tokens, context_window, input_token_limit, token_budget, timeout_seconds, max_retries, is_active (boolean), created_at, and updated_at.
2. WHEN a Variant is created, THE Database SHALL enforce a foreign key constraint from `agent_variants.agent_id` to `ai_agents.id` with ON DELETE CASCADE.
3. THE Database SHALL enforce a unique constraint on the combination of agent_id and variant_slug to prevent duplicate Variant slugs within the same Agent.
4. WHEN a Variant has `is_active` set to TRUE, THE Database SHALL ensure that at most one Variant per Agent has `is_active = TRUE` by using a partial unique index on (agent_id) WHERE is_active = TRUE.
5. THE Database SHALL create indexes on agent_id and on (agent_id, is_active) for efficient lookup of Variants by Agent and Active_Variant resolution.
### Requirement 2: Clone Agent as Variant
**User Story:** As a user, I want to clone an existing agent as a variant that inherits the agent's role and purpose but lets me tweak the model, prompt, and parameters, so that I can create experimental configurations quickly.
#### Acceptance Criteria
1. WHEN a user submits a clone request for an Agent, THE API SHALL create a new Variant record that copies the Agent's model_provider, model_name, system_prompt, user_prompt_template, prompt_version, temperature, max_tokens, context_window, input_token_limit, token_budget, timeout_seconds, and max_retries into the new Variant.
2. WHEN a user submits a clone request for an existing Variant, THE API SHALL create a new Variant record under the same parent Agent that copies the source Variant's configuration fields.
3. WHEN a Variant is created via clone, THE API SHALL allow the user to override any of the copied configuration fields in the same request.
4. WHEN a Variant is created, THE API SHALL require a variant_name and auto-generate a variant_slug from the variant_name if one is not provided.
5. IF a clone request specifies a variant_slug that already exists for the same Agent, THEN THE API SHALL return a 409 Conflict error with a descriptive message.
6. WHEN a Variant is successfully created, THE API SHALL return the complete Variant record including the generated id and timestamps.
### Requirement 3: Variant CRUD Operations
**User Story:** As a user, I want to create, read, update, and delete variants through the API, so that I can manage variant configurations programmatically.
#### Acceptance Criteria
1. WHEN a GET request is made to `/api/agents/{agent_id}/variants`, THE API SHALL return a list of all Variant records belonging to the specified Agent, ordered by created_at ascending.
2. WHEN a GET request is made to `/api/agents/{agent_id}/variants/{variant_id}`, THE API SHALL return the full Variant record.
3. IF a GET request references a non-existent Agent or Variant, THEN THE API SHALL return a 404 Not Found error.
4. WHEN a PUT request is made to `/api/agents/{agent_id}/variants/{variant_id}`, THE API SHALL update only the fields provided in the request body and set updated_at to the current timestamp.
5. WHEN a DELETE request is made for a Variant, THE API SHALL remove the Variant record and cascade-delete associated performance log entries.
6. IF a DELETE request targets a Variant that is currently the Active_Variant, THEN THE API SHALL return a 400 Bad Request error indicating the user must deactivate or promote a different Variant first.
### Requirement 4: Active Variant Swap
**User Story:** As a user, I want to designate which variant is the active one for a given agent role, so that production inference uses my chosen configuration.
#### Acceptance Criteria
1. WHEN a user sends a POST request to `/api/agents/{agent_id}/variants/{variant_id}/activate`, THE API SHALL set `is_active = TRUE` on the specified Variant and set `is_active = FALSE` on any previously active Variant for that Agent, within a single database transaction.
2. WHEN a user sends a POST request to `/api/agents/{agent_id}/variants/deactivate`, THE API SHALL set `is_active = FALSE` on the currently active Variant for that Agent, causing the Agent to fall back to its Base_Configuration.
3. WHEN the extractor, event classifier, or thesis rewriter service resolves its runtime configuration, THE Service SHALL check for an Active_Variant for its Agent and use the Variant's model_name, system_prompt, temperature, max_tokens, context_window, input_token_limit, token_budget, timeout_seconds, and max_retries instead of the Base_Configuration or environment variable defaults.
4. IF no Active_Variant exists for an Agent, THEN THE Service SHALL use the Agent's Base_Configuration from the `ai_agents` table.
5. WHEN an Active_Variant swap occurs, THE API SHALL return the updated Variant record with the new is_active state.
### Requirement 5: Model Swapping
**User Story:** As a user, I want to configure variants with different Ollama models (e.g. qwen3.5, llama3.1, gemma2), so that I can compare model quality and performance for each agent role.
#### Acceptance Criteria
1. THE Variant record SHALL accept any valid model_name string in the model_name field, enabling the user to specify different Ollama models per Variant.
2. WHEN a Variant specifies a model_name, THE Ollama_Service client SHALL use that model_name in the `/api/chat` request to the Ollama endpoint.
3. WHEN a user updates a Variant's model_name via the API, THE API SHALL validate that the model_name field is a non-empty string and persist the change.
4. THE Agents_Page SHALL display the model_name for each Variant in the variant list, enabling users to see which model each Variant uses at a glance.
### Requirement 6: Per-Variant Performance Tracking
**User Story:** As a user, I want performance metrics (success rate, latency, confidence, token usage) tracked per variant, so that I can evaluate which configuration performs best.
#### Acceptance Criteria
1. THE Database SHALL add a nullable `variant_id` column (FK to `agent_variants.id`, ON DELETE SET NULL) to the `agent_performance_log` table.
2. WHEN a service invocation uses an Active_Variant, THE Service SHALL record the variant_id in the `agent_performance_log` entry alongside the existing agent_id.
3. WHEN a GET request is made to `/api/agents/{agent_id}/variants/{variant_id}/performance`, THE API SHALL return aggregated Variant_Performance metrics (total invocations, success count, failure count, average duration, p95 duration, average confidence, average retries, total input tokens, total output tokens, success rate) for the specified Variant within the requested time window.
4. WHEN a GET request is made to `/api/agents/{agent_id}/variants/{variant_id}/performance/history`, THE API SHALL return hourly time-series Variant_Performance data for the specified Variant.
5. WHEN performance is queried for the base Agent without a variant filter, THE API SHALL continue to return metrics across all invocations for that Agent, including those attributed to Variants.
### Requirement 7: Side-by-Side Variant Comparison
**User Story:** As a user, I want to compare two or more variants side-by-side on the Agents page, so that I can make informed decisions about which variant to activate.
#### Acceptance Criteria
1. WHEN a user selects an Agent on the Agents_Page, THE Agents_Page SHALL display a list of all Variants for that Agent below the Agent detail section, showing variant_name, model_name, is_active status, and creation date for each.
2. WHEN a user selects two or more Variants for comparison, THE Agents_Page SHALL display a comparison view showing performance metrics (success rate, average latency, p95 latency, average confidence, total tokens) for each selected Variant in adjacent columns.
3. THE Agents_Page SHALL visually highlight the Active_Variant in the variant list with a distinct badge or indicator.
4. WHEN a user views the comparison view, THE Agents_Page SHALL display a time-series chart overlaying the performance history of the selected Variants on the same axes for direct visual comparison.
5. THE Agents_Page SHALL provide an "Activate" button next to each non-active Variant in the list, allowing the user to promote a Variant to Active_Variant directly from the comparison view.
### Requirement 8: Variant UI Management
**User Story:** As a user, I want to create, edit, clone, and delete variants from the Agents page, so that I can manage variant configurations without leaving the dashboard.
#### Acceptance Criteria
1. WHEN a user clicks "Clone as Variant" on an Agent detail view, THE Agents_Page SHALL open a pre-filled form with the Agent's current configuration, allowing the user to modify fields and submit to create a new Variant.
2. WHEN a user clicks "Clone" on an existing Variant, THE Agents_Page SHALL open a pre-filled form with that Variant's configuration for creating a new Variant.
3. WHEN a user clicks "Edit" on a Variant, THE Agents_Page SHALL display an edit form pre-populated with the Variant's current configuration, allowing modification and save.
4. WHEN a user clicks "Delete" on a non-active Variant, THE Agents_Page SHALL display a confirmation dialog before deleting the Variant.
5. IF a user attempts to delete the Active_Variant, THEN THE Agents_Page SHALL display an error message indicating the user must deactivate the Variant first.
6. WHEN a Variant is created, edited, activated, or deleted, THE Agents_Page SHALL refresh the variant list and performance data to reflect the change.
### Requirement 10: Token Window and Budget Controls
**User Story:** As a user, I want to configure context window sizes, input token limits, and hourly token budgets per variant, so that I can control resource usage for cloud models while running unlimited for local Ollama.
#### Acceptance Criteria
1. THE Variant record SHALL include a `context_window` integer field (default 0) that maps to the Ollama `num_ctx` parameter. A value of 0 means use the model's default context window.
2. THE Variant record SHALL include an `input_token_limit` integer field (default 0) that caps how many tokens are sent as input to the model. A value of 0 means no limit (no truncation).
3. THE Variant record SHALL include a `token_budget` integer field (default 0) representing the maximum total tokens (input + output) allowed per hour for the variant. A value of 0 means unlimited.
4. WHEN a service invocation uses an Active_Variant with a non-zero `context_window`, THE Ollama_Service client SHALL pass `num_ctx` in the Ollama API options.
5. WHEN a service invocation uses an Active_Variant with a non-zero `input_token_limit`, THE Service SHALL truncate the input content to approximately that many tokens before sending it to the model.
6. WHEN a service invocation uses an Active_Variant with a non-zero `token_budget` and the hourly token usage for that variant has reached or exceeded the budget, THE Service SHALL skip the invocation and log a warning.
7. THE Agents_Page SHALL display context_window, input_token_limit, and token_budget fields in the variant create, edit, and clone forms, with clear labels indicating that 0 means "use default" or "unlimited".
**User Story:** As a developer, I want the extractor, event classifier, and thesis rewriter services to dynamically resolve their configuration from the database (including active variant overrides), so that variant swaps take effect without restarting services.
#### Acceptance Criteria
1. WHEN the Document Intelligence Extractor service prepares an inference request, THE Service SHALL query the `ai_agents` table (joined with `agent_variants` if an Active_Variant exists) by the agent slug `document-extractor` to resolve model_name, system_prompt, temperature, max_tokens, context_window, input_token_limit, token_budget, timeout_seconds, and max_retries.
2. WHEN the Global Event Classifier service prepares a classification request, THE Service SHALL query the database by the agent slug `event-classifier` to resolve runtime configuration, preferring the Active_Variant's values when one exists.
3. WHEN the Thesis Rewriter service prepares a rewrite request, THE Service SHALL query the database by the agent slug `thesis-rewriter` to resolve runtime configuration, preferring the Active_Variant's values when one exists.
4. IF the database is unreachable during configuration resolution, THEN THE Service SHALL fall back to the environment variable defaults from OllamaConfig and log a warning.
5. THE Service SHALL cache resolved configuration with a time-to-live of 60 seconds to avoid querying the database on every invocation, while still reflecting Active_Variant swaps within a reasonable delay.
+108
View File
@@ -0,0 +1,108 @@
# Implementation Tasks: Agent Variants
## Phase 1: Database Schema
- [x] 1.1 Create migration `infra/migrations/027_agent_variants.sql`
- [x] 1.1.1 Create `agent_variants` table with columns: id (UUID PK), agent_id (UUID FK to ai_agents ON DELETE CASCADE), variant_name (VARCHAR 200 NOT NULL), variant_slug (VARCHAR 200 NOT NULL), description (TEXT DEFAULT ''), model_provider (VARCHAR 50 DEFAULT 'ollama'), model_name (VARCHAR 200 NOT NULL), system_prompt (TEXT DEFAULT ''), user_prompt_template (TEXT DEFAULT ''), prompt_version (VARCHAR 100 DEFAULT ''), temperature (FLOAT DEFAULT 0.0), max_tokens (INTEGER DEFAULT 32768), context_window (INTEGER DEFAULT 0), input_token_limit (INTEGER DEFAULT 0), token_budget (INTEGER DEFAULT 0), timeout_seconds (INTEGER DEFAULT 120), max_retries (INTEGER DEFAULT 2), is_active (BOOLEAN DEFAULT FALSE), created_at (TIMESTAMPTZ DEFAULT NOW()), updated_at (TIMESTAMPTZ DEFAULT NOW())
- [x] 1.1.2 Create unique index `idx_agent_variants_slug` on (agent_id, variant_slug) to prevent duplicate slugs per agent
- [x] 1.1.3 Create partial unique index `idx_agent_variants_active` on (agent_id) WHERE is_active = TRUE to enforce at most one active variant per agent
- [x] 1.1.4 Create index `idx_agent_variants_agent` on (agent_id) for fast lookup
- [x] 1.1.5 Add nullable `variant_id` column (UUID FK to agent_variants ON DELETE SET NULL) to `agent_performance_log` table
- [x] 1.1.6 Create index `idx_agent_perf_variant` on (variant_id, recorded_at DESC) on `agent_performance_log`
## Phase 2: Config Resolution Module
- [x] 2.1 Create `services/shared/agent_config.py` with AgentConfigResolver
- [x] 2.1.1 Define `ResolvedAgentConfig` dataclass with fields: agent_id, variant_id (optional), model_provider, model_name, system_prompt, user_prompt_template, prompt_version, temperature, max_tokens, context_window, input_token_limit, token_budget, timeout_seconds, max_retries
- [x] 2.1.2 Implement `AgentConfigResolver.__init__` accepting asyncpg.Pool and ttl_seconds (default 60)
- [x] 2.1.3 Implement `AgentConfigResolver.resolve(agent_slug)` that queries ai_agents LEFT JOIN agent_variants (WHERE is_active = TRUE) using COALESCE to prefer variant values, with TTL-based in-memory cache
- [x] 2.1.4 Implement cache invalidation: entries expire after ttl_seconds; return None and log warning if DB query fails (callers fall back to env-var defaults)
## Phase 3: Backend API Endpoints
- [x] 3.1 Add Pydantic models for variant operations in `services/api/app.py`
- [x] 3.1.1 Add `VariantCreateBody` model with fields: variant_name (str), variant_slug (str | None), description (str = ""), model_provider (str = "ollama"), model_name (str), system_prompt (str = ""), user_prompt_template (str = ""), prompt_version (str = ""), temperature (float = 0.0), max_tokens (int = 32768), context_window (int = 0), input_token_limit (int = 0), token_budget (int = 0), timeout_seconds (int = 120), max_retries (int = 2)
- [x] 3.1.2 Add `VariantUpdateBody` model with all config fields optional (str | None, float | None, int | None)
- [x] 3.1.3 Add `VariantCloneBody` model with variant_name (str), variant_slug (str | None), and all config fields optional for overrides
- [x] 3.1.4 Add slug auto-generation helper: `_slugify(name: str) -> str` that lowercases, replaces non-alphanumeric with hyphens, strips leading/trailing hyphens
- [x] 3.2 Add variant CRUD endpoints in `services/api/app.py`
- [x] 3.2.1 `GET /api/agents/{agent_id}/variants` — list all variants for an agent ordered by created_at ASC, return list of variant dicts
- [x] 3.2.2 `GET /api/agents/{agent_id}/variants/{variant_id}` — get single variant, return 404 if not found or agent mismatch
- [x] 3.2.3 `POST /api/agents/{agent_id}/variants` — create variant with VariantCreateBody, auto-generate slug if not provided, return 409 on duplicate slug, return 201 with created variant
- [x] 3.2.4 `PUT /api/agents/{agent_id}/variants/{variant_id}` — partial update with VariantUpdateBody, set updated_at = NOW(), return 404 if not found
- [x] 3.2.5 `DELETE /api/agents/{agent_id}/variants/{variant_id}` — delete variant, return 400 if variant is currently active (is_active = TRUE), return 404 if not found
- [x] 3.3 Add clone endpoints in `services/api/app.py`
- [x] 3.3.1 `POST /api/agents/{agent_id}/clone` — clone agent as variant: fetch agent by id, build variant fields from agent config with VariantCloneBody overrides, auto-generate slug, insert into agent_variants, return 201
- [x] 3.3.2 `POST /api/agents/{agent_id}/variants/{variant_id}/clone` — clone variant as new variant: fetch source variant, build new variant fields with overrides, insert, return 201
- [x] 3.4 Add activate/deactivate endpoints in `services/api/app.py`
- [x] 3.4.1 `POST /api/agents/{agent_id}/variants/{variant_id}/activate` — within a transaction: set is_active = FALSE on current active variant for agent, then set is_active = TRUE on target variant, return updated variant
- [x] 3.4.2 `POST /api/agents/{agent_id}/variants/deactivate` — set is_active = FALSE on current active variant for agent, return {"deactivated": true}
- [x] 3.5 Add per-variant performance endpoints in `services/api/app.py`
- [x] 3.5.1 `GET /api/agents/{agent_id}/variants/{variant_id}/performance` — aggregated metrics (total invocations, successes, failures, avg/p95 duration, avg confidence, avg retries, total tokens, success rate) filtered by variant_id and time window
- [x] 3.5.2 `GET /api/agents/{agent_id}/variants/{variant_id}/performance/history` — hourly time-series filtered by variant_id
## Phase 4: Service Integration
- [x] 4.1 Integrate config resolver into Document Extractor
- [x] 4.1.1 In the extractor service startup or invocation path, instantiate AgentConfigResolver and call `resolve("document-extractor")` to get runtime config; fall back to OllamaConfig() from env vars if resolve returns None or raises
- [x] 4.1.2 Build OllamaConfig from ResolvedAgentConfig fields (model_name → config.model, system_prompt, temperature, max_tokens, timeout, max_retries)
- [x] 4.1.3 Update performance logging to include variant_id from the resolved config when inserting into agent_performance_log
- [x] 4.2 Integrate config resolver into Event Classifier
- [x] 4.2.1 In classify_global_event or its caller, resolve config for "event-classifier" slug; fall back to existing OllamaConfig if resolution fails
- [x] 4.2.2 Use resolved model_name and system_prompt when building the Ollama request
- [x] 4.2.3 Pass variant_id to performance log entries
- [x] 4.3 Integrate config resolver into Thesis Rewriter
- [x] 4.3.1 In rewrite_thesis_with_llm or its caller, resolve config for "thesis-rewriter" slug; fall back to passed OllamaConfig if resolution fails
- [x] 4.3.2 Override OllamaConfig fields with resolved values (model, timeout, max_retries)
- [x] 4.3.3 Pass variant_id to performance log entries if performance logging exists for thesis rewrites
## Phase 5: Frontend — Variant List and Detail
- [x] 5.1 Add TypeScript types and API hooks for variants
- [x] 5.1.1 Add `AgentVariant` interface to Agents.tsx with fields: id, agent_id, variant_name, variant_slug, description, model_provider, model_name, system_prompt, user_prompt_template, prompt_version, temperature, max_tokens, context_window, input_token_limit, token_budget, timeout_seconds, max_retries, is_active, created_at, updated_at
- [x] 5.1.2 Add `useAgentVariants(agentId)` query hook fetching `GET /api/agents/{agentId}/variants`
- [x] 5.1.3 Add `useVariantPerformance(agentId, variantId, hours)` query hook
- [x] 5.1.4 Add `useVariantPerfHistory(agentId, variantId, hours)` query hook
- [x] 5.1.5 Add mutation hooks: `useCloneAgentAsVariant`, `useCreateVariant`, `useUpdateVariant`, `useDeleteVariant`, `useActivateVariant`, `useDeactivateVariants` — each invalidates `['agent-variants']` query on success
- [x] 5.2 Add VariantList component to AgentDetail
- [x] 5.2.1 Create `VariantList` component that renders a table/list of variants for the selected agent showing: variant_name, model_name, is_active badge, created_at, and action buttons (Edit, Clone, Delete, Activate)
- [x] 5.2.2 Add "Clone as Variant" button to the AgentDetail header that opens the clone form pre-filled with agent config
- [x] 5.2.3 Highlight the active variant row with a distinct badge (e.g. green "Active" StatusBadge)
- [x] 5.2.4 Wire Activate button to call the activate endpoint and invalidate queries; disable on the already-active variant
- [x] 5.3 Add Variant Create/Edit/Clone forms
- [x] 5.3.1 Create `VariantCloneForm` component pre-filled with source (agent or variant) config fields, allowing modification before submit; uses useCloneAgentAsVariant or clone-variant mutation
- [x] 5.3.2 Create `VariantEditForm` component pre-filled with current variant config, using useUpdateVariant mutation
- [x] 5.3.3 Add delete confirmation dialog: show warning, call useDeleteVariant on confirm; show error toast if variant is active (400 response)
## Phase 6: Frontend — Comparison View
- [x] 6.1 Add variant comparison UI
- [x] 6.1.1 Add checkbox selection to each VariantRow; track selected variant IDs in component state
- [x] 6.1.2 Create `VariantCompare` component that renders when 2+ variants are selected: shows a table with one column per selected variant, rows for each metric (success rate, avg latency, p95 latency, avg confidence, total tokens)
- [x] 6.1.3 Add an overlay performance chart in VariantCompare: use Recharts LineChart with one Line per selected variant, shared XAxis (hour), showing success rate or latency over time
- [x] 6.1.4 Add "Activate" button in comparison view for each non-active variant column
## Phase 7: Tests
- [x] 7.1 Add backend tests for variant API
- [x] 7.1.1 Add `tests/test_agent_variants_api.py` with example tests for: create variant, clone from agent, clone from variant, list variants, get variant, update variant, delete variant, delete active variant (expect 400), activate/deactivate, variant performance queries
- [x] 7.1.2 Add edge-case tests: duplicate slug (409), non-existent agent (404), non-existent variant (404), empty model_name validation
- [x] 7.2 Add property-based tests for variant logic
- [x] 7.2.1 [PBT] Property: Single active variant invariant — generate random sequences of activate/deactivate operations, assert at most one active variant per agent after each operation
- [x] 7.2.2 [PBT] Property: Clone preserves unoverridden fields — generate random agent configs and random override subsets, assert non-overridden fields match source
- [x] 7.2.3 [PBT] Property: Config resolution prefers active variant — generate agent + variants with random active state, assert resolver returns correct config source
- [x] 7.2.4 [PBT] Property: Slug auto-generation determinism — generate random names, assert slugify is deterministic, produces valid kebab-case, and is non-empty for non-empty input
- [x] 7.2.5 [PBT] Property: Partial update idempotence — generate variant + random update subset, apply twice, assert fields match (excluding updated_at)
- [x] 7.3 Add frontend tests
- [x] 7.3.1 Add MSW handlers in `frontend/src/test/mocks/handlers.ts` for all variant endpoints (list, get, create, clone, update, delete, activate, deactivate, performance, history)
- [x] 7.3.2 Add test in `frontend/src/test/pages.test.tsx` verifying the Agents page renders variant list when an agent is selected, and that the comparison view appears when multiple variants are checked
+115 -115
View File
@@ -369,9 +369,9 @@
} }
}, },
"node_modules/@csstools/css-calc": { "node_modules/@csstools/css-calc": {
"version": "3.1.1", "version": "3.2.0",
"resolved": "https://registry.npmjs.org/@csstools/css-calc/-/css-calc-3.1.1.tgz", "resolved": "https://registry.npmjs.org/@csstools/css-calc/-/css-calc-3.2.0.tgz",
"integrity": "sha512-HJ26Z/vmsZQqs/o3a6bgKslXGFAungXGbinULZO3eMsOyNJHeBBZfup5FiZInOghgoM4Hwnmw+OgbJCNg1wwUQ==", "integrity": "sha512-bR9e6o2BDB12jzN/gIbjHa5wLJ4UjD1CB9pM7ehlc0ddk6EBz+yYS1EV2MF55/HUxrHcB/hehAyt5vhsA3hx7w==",
"dev": true, "dev": true,
"funding": [ "funding": [
{ {
@@ -393,9 +393,9 @@
} }
}, },
"node_modules/@csstools/css-color-parser": { "node_modules/@csstools/css-color-parser": {
"version": "4.0.2", "version": "4.1.0",
"resolved": "https://registry.npmjs.org/@csstools/css-color-parser/-/css-color-parser-4.0.2.tgz", "resolved": "https://registry.npmjs.org/@csstools/css-color-parser/-/css-color-parser-4.1.0.tgz",
"integrity": "sha512-0GEfbBLmTFf0dJlpsNU7zwxRIH0/BGEMuXLTCvFYxuL1tNhqzTbtnFICyJLTNK4a+RechKP75e7w42ClXSnJQw==", "integrity": "sha512-U0KhLYmy2GVj6q4T3WaAe6NPuFYCPQoE3b0dRGxejWDgcPp8TP7S5rVdM5ZrFaqu4N67X8YaPBw14dQSYx3IyQ==",
"dev": true, "dev": true,
"funding": [ "funding": [
{ {
@@ -410,7 +410,7 @@
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@csstools/color-helpers": "^6.0.2", "@csstools/color-helpers": "^6.0.2",
"@csstools/css-calc": "^3.1.1" "@csstools/css-calc": "^3.2.0"
}, },
"engines": { "engines": {
"node": ">=20.19.0" "node": ">=20.19.0"
@@ -444,9 +444,9 @@
} }
}, },
"node_modules/@csstools/css-syntax-patches-for-csstree": { "node_modules/@csstools/css-syntax-patches-for-csstree": {
"version": "1.1.2", "version": "1.1.3",
"resolved": "https://registry.npmjs.org/@csstools/css-syntax-patches-for-csstree/-/css-syntax-patches-for-csstree-1.1.2.tgz", "resolved": "https://registry.npmjs.org/@csstools/css-syntax-patches-for-csstree/-/css-syntax-patches-for-csstree-1.1.3.tgz",
"integrity": "sha512-5GkLzz4prTIpoyeUiIu3iV6CSG3Plo7xRVOFPKI7FVEJ3mZ0A8SwK0XU3Gl7xAkiQ+mDyam+NNp875/C5y+jSA==", "integrity": "sha512-SH60bMfrRCJF3morcdk57WklujF4Jr/EsQUzqkarfHXEFcAR1gg7fS/chAE922Sehgzc1/+Tz5H3Ypa1HiEKrg==",
"dev": true, "dev": true,
"funding": [ "funding": [
{ {
@@ -921,9 +921,9 @@
} }
}, },
"node_modules/@napi-rs/wasm-runtime": { "node_modules/@napi-rs/wasm-runtime": {
"version": "1.1.3", "version": "1.1.4",
"resolved": "https://registry.npmjs.org/@napi-rs/wasm-runtime/-/wasm-runtime-1.1.3.tgz", "resolved": "https://registry.npmjs.org/@napi-rs/wasm-runtime/-/wasm-runtime-1.1.4.tgz",
"integrity": "sha512-xK9sGVbJWYb08+mTJt3/YV24WxvxpXcXtP6B172paPZ+Ts69Re9dAr7lKwJoeIx8OoeuimEiRZ7umkiUVClmmQ==", "integrity": "sha512-3NQNNgA1YSlJb/kMH1ildASP9HW7/7kYnRI2szWJaofaS1hWmbGI4H+d3+22aGzXXN9IJ+n+GiFVcGipJP18ow==",
"license": "MIT", "license": "MIT",
"optional": true, "optional": true,
"dependencies": { "dependencies": {
@@ -1566,14 +1566,14 @@
} }
}, },
"node_modules/@tanstack/react-router": { "node_modules/@tanstack/react-router": {
"version": "1.168.18", "version": "1.168.22",
"resolved": "https://registry.npmjs.org/@tanstack/react-router/-/react-router-1.168.18.tgz", "resolved": "https://registry.npmjs.org/@tanstack/react-router/-/react-router-1.168.22.tgz",
"integrity": "sha512-RmBptS3/qtkGhvG/u41JWOgxz1FIWybBz7iBTgLUIoFkqOj6NE4XlhUOsP2fabxACtbZdJnpvCWcJFWpWGIngw==", "integrity": "sha512-W2LyfkfJtDCf//jOjZeUBWwOVl8iDRVTECpGHa2M28MT3T5/VVnjgicYNHR/ax0Filk1iU67MRjcjHheTYvK1Q==",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@tanstack/history": "1.161.6", "@tanstack/history": "1.161.6",
"@tanstack/react-store": "^0.9.3", "@tanstack/react-store": "^0.9.3",
"@tanstack/router-core": "1.168.14", "@tanstack/router-core": "1.168.15",
"isbot": "^5.1.22" "isbot": "^5.1.22"
}, },
"engines": { "engines": {
@@ -1607,9 +1607,9 @@
} }
}, },
"node_modules/@tanstack/router-core": { "node_modules/@tanstack/router-core": {
"version": "1.168.14", "version": "1.168.15",
"resolved": "https://registry.npmjs.org/@tanstack/router-core/-/router-core-1.168.14.tgz", "resolved": "https://registry.npmjs.org/@tanstack/router-core/-/router-core-1.168.15.tgz",
"integrity": "sha512-UhCJtjNrd5wcTmhgB2HyUP0+Rj1M7BD4dS11YsF9x6VC2KH/eqxzs/vK+nN5f+cOhPOLZdmLkWMW+WGmacZ8HA==", "integrity": "sha512-Wr0424NDtD8fT/uALobMZ9DdcfsTyXtW5IPR++7zvW8/7RaIOeaqXpVDId8ywaGtqPWLWOfaUg2zUtYtukoXYA==",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@tanstack/history": "1.161.6", "@tanstack/history": "1.161.6",
@@ -1893,17 +1893,17 @@
"license": "MIT" "license": "MIT"
}, },
"node_modules/@typescript-eslint/eslint-plugin": { "node_modules/@typescript-eslint/eslint-plugin": {
"version": "8.58.1", "version": "8.58.2",
"resolved": "https://registry.npmjs.org/@typescript-eslint/eslint-plugin/-/eslint-plugin-8.58.1.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/eslint-plugin/-/eslint-plugin-8.58.2.tgz",
"integrity": "sha512-eSkwoemjo76bdXl2MYqtxg51HNwUSkWfODUOQ3PaTLZGh9uIWWFZIjyjaJnex7wXDu+TRx+ATsnSxdN9YWfRTQ==", "integrity": "sha512-aC2qc5thQahutKjP+cl8cgN9DWe3ZUqVko30CMSZHnFEHyhOYoZSzkGtAI2mcwZ38xeImDucI4dnqsHiOYuuCw==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@eslint-community/regexpp": "^4.12.2", "@eslint-community/regexpp": "^4.12.2",
"@typescript-eslint/scope-manager": "8.58.1", "@typescript-eslint/scope-manager": "8.58.2",
"@typescript-eslint/type-utils": "8.58.1", "@typescript-eslint/type-utils": "8.58.2",
"@typescript-eslint/utils": "8.58.1", "@typescript-eslint/utils": "8.58.2",
"@typescript-eslint/visitor-keys": "8.58.1", "@typescript-eslint/visitor-keys": "8.58.2",
"ignore": "^7.0.5", "ignore": "^7.0.5",
"natural-compare": "^1.4.0", "natural-compare": "^1.4.0",
"ts-api-utils": "^2.5.0" "ts-api-utils": "^2.5.0"
@@ -1916,7 +1916,7 @@
"url": "https://opencollective.com/typescript-eslint" "url": "https://opencollective.com/typescript-eslint"
}, },
"peerDependencies": { "peerDependencies": {
"@typescript-eslint/parser": "^8.58.1", "@typescript-eslint/parser": "^8.58.2",
"eslint": "^8.57.0 || ^9.0.0 || ^10.0.0", "eslint": "^8.57.0 || ^9.0.0 || ^10.0.0",
"typescript": ">=4.8.4 <6.1.0" "typescript": ">=4.8.4 <6.1.0"
} }
@@ -1932,16 +1932,16 @@
} }
}, },
"node_modules/@typescript-eslint/parser": { "node_modules/@typescript-eslint/parser": {
"version": "8.58.1", "version": "8.58.2",
"resolved": "https://registry.npmjs.org/@typescript-eslint/parser/-/parser-8.58.1.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/parser/-/parser-8.58.2.tgz",
"integrity": "sha512-gGkiNMPqerb2cJSVcruigx9eHBlLG14fSdPdqMoOcBfh+vvn4iCq2C8MzUB89PrxOXk0y3GZ1yIWb9aOzL93bw==", "integrity": "sha512-/Zb/xaIDfxeJnvishjGdcR4jmr7S+bda8PKNhRGdljDM+elXhlvN0FyPSsMnLmJUrVG9aPO6dof80wjMawsASg==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@typescript-eslint/scope-manager": "8.58.1", "@typescript-eslint/scope-manager": "8.58.2",
"@typescript-eslint/types": "8.58.1", "@typescript-eslint/types": "8.58.2",
"@typescript-eslint/typescript-estree": "8.58.1", "@typescript-eslint/typescript-estree": "8.58.2",
"@typescript-eslint/visitor-keys": "8.58.1", "@typescript-eslint/visitor-keys": "8.58.2",
"debug": "^4.4.3" "debug": "^4.4.3"
}, },
"engines": { "engines": {
@@ -1957,14 +1957,14 @@
} }
}, },
"node_modules/@typescript-eslint/project-service": { "node_modules/@typescript-eslint/project-service": {
"version": "8.58.1", "version": "8.58.2",
"resolved": "https://registry.npmjs.org/@typescript-eslint/project-service/-/project-service-8.58.1.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/project-service/-/project-service-8.58.2.tgz",
"integrity": "sha512-gfQ8fk6cxhtptek+/8ZIqw8YrRW5048Gug8Ts5IYcMLCw18iUgrZAEY/D7s4hkI0FxEfGakKuPK/XUMPzPxi5g==", "integrity": "sha512-Cq6UfpZZk15+r87BkIh5rDpi38W4b+Sjnb8wQCPPDDweS/LRCFjCyViEbzHk5Ck3f2QDfgmlxqSa7S7clDtlfg==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@typescript-eslint/tsconfig-utils": "^8.58.1", "@typescript-eslint/tsconfig-utils": "^8.58.2",
"@typescript-eslint/types": "^8.58.1", "@typescript-eslint/types": "^8.58.2",
"debug": "^4.4.3" "debug": "^4.4.3"
}, },
"engines": { "engines": {
@@ -1979,14 +1979,14 @@
} }
}, },
"node_modules/@typescript-eslint/scope-manager": { "node_modules/@typescript-eslint/scope-manager": {
"version": "8.58.1", "version": "8.58.2",
"resolved": "https://registry.npmjs.org/@typescript-eslint/scope-manager/-/scope-manager-8.58.1.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/scope-manager/-/scope-manager-8.58.2.tgz",
"integrity": "sha512-TPYUEqJK6avLcEjumWsIuTpuYODTTDAtoMdt8ZZa93uWMTX13Nb8L5leSje1NluammvU+oI3QRr5lLXPgihX3w==", "integrity": "sha512-SgmyvDPexWETQek+qzZnrG6844IaO02UVyOLhI4wpo82dpZJY9+6YZCKAMFzXb7qhx37mFK1QcPQ18tud+vo6Q==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@typescript-eslint/types": "8.58.1", "@typescript-eslint/types": "8.58.2",
"@typescript-eslint/visitor-keys": "8.58.1" "@typescript-eslint/visitor-keys": "8.58.2"
}, },
"engines": { "engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0" "node": "^18.18.0 || ^20.9.0 || >=21.1.0"
@@ -1997,9 +1997,9 @@
} }
}, },
"node_modules/@typescript-eslint/tsconfig-utils": { "node_modules/@typescript-eslint/tsconfig-utils": {
"version": "8.58.1", "version": "8.58.2",
"resolved": "https://registry.npmjs.org/@typescript-eslint/tsconfig-utils/-/tsconfig-utils-8.58.1.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/tsconfig-utils/-/tsconfig-utils-8.58.2.tgz",
"integrity": "sha512-JAr2hOIct2Q+qk3G+8YFfqkqi7sC86uNryT+2i5HzMa2MPjw4qNFvtjnw1IiA1rP7QhNKVe21mSSLaSjwA1Olw==", "integrity": "sha512-3SR+RukipDvkkKp/d0jP0dyzuls3DbGmwDpVEc5wqk5f38KFThakqAAO0XMirWAE+kT00oTauTbzMFGPoAzB0A==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"engines": { "engines": {
@@ -2014,15 +2014,15 @@
} }
}, },
"node_modules/@typescript-eslint/type-utils": { "node_modules/@typescript-eslint/type-utils": {
"version": "8.58.1", "version": "8.58.2",
"resolved": "https://registry.npmjs.org/@typescript-eslint/type-utils/-/type-utils-8.58.1.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/type-utils/-/type-utils-8.58.2.tgz",
"integrity": "sha512-HUFxvTJVroT+0rXVJC7eD5zol6ID+Sn5npVPWoFuHGg9Ncq5Q4EYstqR+UOqaNRFXi5TYkpXXkLhoCHe3G0+7w==", "integrity": "sha512-Z7EloNR/B389FvabdGeTo2XMs4W9TjtPiO9DAsmT0yom0bwlPyRjkJ1uCdW1DvrrrYP50AJZ9Xc3sByZA9+dcg==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@typescript-eslint/types": "8.58.1", "@typescript-eslint/types": "8.58.2",
"@typescript-eslint/typescript-estree": "8.58.1", "@typescript-eslint/typescript-estree": "8.58.2",
"@typescript-eslint/utils": "8.58.1", "@typescript-eslint/utils": "8.58.2",
"debug": "^4.4.3", "debug": "^4.4.3",
"ts-api-utils": "^2.5.0" "ts-api-utils": "^2.5.0"
}, },
@@ -2039,9 +2039,9 @@
} }
}, },
"node_modules/@typescript-eslint/types": { "node_modules/@typescript-eslint/types": {
"version": "8.58.1", "version": "8.58.2",
"resolved": "https://registry.npmjs.org/@typescript-eslint/types/-/types-8.58.1.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/types/-/types-8.58.2.tgz",
"integrity": "sha512-io/dV5Aw5ezwzfPBBWLoT+5QfVtP8O7q4Kftjn5azJ88bYyp/ZMCsyW1lpKK46EXJcaYMZ1JtYj+s/7TdzmQMw==", "integrity": "sha512-9TukXyATBQf/Jq9AMQXfvurk+G5R2MwfqQGDR2GzGz28HvY/lXNKGhkY+6IOubwcquikWk5cjlgPvD2uAA7htQ==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"engines": { "engines": {
@@ -2053,16 +2053,16 @@
} }
}, },
"node_modules/@typescript-eslint/typescript-estree": { "node_modules/@typescript-eslint/typescript-estree": {
"version": "8.58.1", "version": "8.58.2",
"resolved": "https://registry.npmjs.org/@typescript-eslint/typescript-estree/-/typescript-estree-8.58.1.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/typescript-estree/-/typescript-estree-8.58.2.tgz",
"integrity": "sha512-w4w7WR7GHOjqqPnvAYbazq+Y5oS68b9CzasGtnd6jIeOIeKUzYzupGTB2T4LTPSv4d+WPeccbxuneTFHYgAAWg==", "integrity": "sha512-ELGuoofuhhoCvNbQjFFiobFcGgcDCEm0ThWdmO4Z0UzLqPXS3KFvnEZ+SHewwOYHjM09tkzOWXNTv9u6Gqtyuw==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@typescript-eslint/project-service": "8.58.1", "@typescript-eslint/project-service": "8.58.2",
"@typescript-eslint/tsconfig-utils": "8.58.1", "@typescript-eslint/tsconfig-utils": "8.58.2",
"@typescript-eslint/types": "8.58.1", "@typescript-eslint/types": "8.58.2",
"@typescript-eslint/visitor-keys": "8.58.1", "@typescript-eslint/visitor-keys": "8.58.2",
"debug": "^4.4.3", "debug": "^4.4.3",
"minimatch": "^10.2.2", "minimatch": "^10.2.2",
"semver": "^7.7.3", "semver": "^7.7.3",
@@ -2133,16 +2133,16 @@
} }
}, },
"node_modules/@typescript-eslint/utils": { "node_modules/@typescript-eslint/utils": {
"version": "8.58.1", "version": "8.58.2",
"resolved": "https://registry.npmjs.org/@typescript-eslint/utils/-/utils-8.58.1.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/utils/-/utils-8.58.2.tgz",
"integrity": "sha512-Ln8R0tmWC7pTtLOzgJzYTXSCjJ9rDNHAqTaVONF4FEi2qwce8mD9iSOxOpLFFvWp/wBFlew0mjM1L1ihYWfBdQ==", "integrity": "sha512-QZfjHNEzPY8+l0+fIXMvuQ2sJlplB4zgDZvA+NmvZsZv3EQwOcc1DuIU1VJUTWZ/RKouBMhDyNaBMx4sWvrzRA==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@eslint-community/eslint-utils": "^4.9.1", "@eslint-community/eslint-utils": "^4.9.1",
"@typescript-eslint/scope-manager": "8.58.1", "@typescript-eslint/scope-manager": "8.58.2",
"@typescript-eslint/types": "8.58.1", "@typescript-eslint/types": "8.58.2",
"@typescript-eslint/typescript-estree": "8.58.1" "@typescript-eslint/typescript-estree": "8.58.2"
}, },
"engines": { "engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0" "node": "^18.18.0 || ^20.9.0 || >=21.1.0"
@@ -2157,13 +2157,13 @@
} }
}, },
"node_modules/@typescript-eslint/visitor-keys": { "node_modules/@typescript-eslint/visitor-keys": {
"version": "8.58.1", "version": "8.58.2",
"resolved": "https://registry.npmjs.org/@typescript-eslint/visitor-keys/-/visitor-keys-8.58.1.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/visitor-keys/-/visitor-keys-8.58.2.tgz",
"integrity": "sha512-y+vH7QE8ycjoa0bWciFg7OpFcipUuem1ujhrdLtq1gByKwfbC7bPeKsiny9e0urg93DqwGcHey+bGRKCnF1nZQ==", "integrity": "sha512-f1WO2Lx8a9t8DARmcWAUPJbu0G20bJlj8L4z72K00TMeJAoyLr/tHhI/pzYBLrR4dXWkcxO1cWYZEOX8DKHTqA==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@typescript-eslint/types": "8.58.1", "@typescript-eslint/types": "8.58.2",
"eslint-visitor-keys": "^5.0.0" "eslint-visitor-keys": "^5.0.0"
}, },
"engines": { "engines": {
@@ -2427,9 +2427,9 @@
"license": "MIT" "license": "MIT"
}, },
"node_modules/baseline-browser-mapping": { "node_modules/baseline-browser-mapping": {
"version": "2.10.18", "version": "2.10.19",
"resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.10.18.tgz", "resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.10.19.tgz",
"integrity": "sha512-VSnGQAOLtP5mib/DPyg2/t+Tlv65NTBz83BJBJvmLVHHuKJVaDOBvJJykiT5TR++em5nfAySPccDZDa4oSrn8A==", "integrity": "sha512-qCkNLi2sfBOn8XhZQ0FXsT1Ki/Yo5P90hrkRamVFRS7/KV9hpfA4HkoWNU152+8w0zPjnxo5psx5NL3PSGgv5g==",
"dev": true, "dev": true,
"license": "Apache-2.0", "license": "Apache-2.0",
"bin": { "bin": {
@@ -2505,9 +2505,9 @@
} }
}, },
"node_modules/caniuse-lite": { "node_modules/caniuse-lite": {
"version": "1.0.30001787", "version": "1.0.30001788",
"resolved": "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001787.tgz", "resolved": "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001788.tgz",
"integrity": "sha512-mNcrMN9KeI68u7muanUpEejSLghOKlVhRqS/Za2IeyGllJ9I9otGpR9g3nsw7n4W378TE/LyIteA0+/FOZm4Kg==", "integrity": "sha512-6q8HFp+lOQtcf7wBK+uEenxymVWkGKkjFpCvw5W25cmMwEDU45p1xQFBQv8JDlMMry7eNxyBaR+qxgmTUZkIRQ==",
"dev": true, "dev": true,
"funding": [ "funding": [
{ {
@@ -2912,9 +2912,9 @@
} }
}, },
"node_modules/electron-to-chromium": { "node_modules/electron-to-chromium": {
"version": "1.5.335", "version": "1.5.336",
"resolved": "https://registry.npmjs.org/electron-to-chromium/-/electron-to-chromium-1.5.335.tgz", "resolved": "https://registry.npmjs.org/electron-to-chromium/-/electron-to-chromium-1.5.336.tgz",
"integrity": "sha512-q9n5T4BR4Xwa2cwbrwcsDJtHD/enpQ5S1xF1IAtdqf5AAgqDFmR/aakqH3ChFdqd/QXJhS3rnnXFtexU7rax6Q==", "integrity": "sha512-AbH9q9J455r/nLmdNZes0G0ZKcRX73FicwowalLs6ijwOmCJSRRrLX63lcAlzy9ux3dWK1w1+1nsBJEWN11hcQ==",
"dev": true, "dev": true,
"license": "ISC" "license": "ISC"
}, },
@@ -3338,9 +3338,9 @@
} }
}, },
"node_modules/globals": { "node_modules/globals": {
"version": "17.4.0", "version": "17.5.0",
"resolved": "https://registry.npmjs.org/globals/-/globals-17.4.0.tgz", "resolved": "https://registry.npmjs.org/globals/-/globals-17.5.0.tgz",
"integrity": "sha512-hjrNztw/VajQwOLsMNT1cbJiH2muO3OROCHnbehc8eY5JyD2gqz4AcMHPqgaOR59DjgUjYAYLeH699g/eWi2jw==", "integrity": "sha512-qoV+HK2yFl/366t2/Cb3+xxPUo5BuMynomoDmiaZBIdbs+0pYbjfZU+twLhGKp4uCZ/+NbtpVepH5bGCxRyy2g==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"engines": { "engines": {
@@ -3527,9 +3527,9 @@
"license": "MIT" "license": "MIT"
}, },
"node_modules/isbot": { "node_modules/isbot": {
"version": "5.1.37", "version": "5.1.38",
"resolved": "https://registry.npmjs.org/isbot/-/isbot-5.1.37.tgz", "resolved": "https://registry.npmjs.org/isbot/-/isbot-5.1.38.tgz",
"integrity": "sha512-5bcicX81xf6NlTEV8rWdg7Pk01LFizDetuYGHx6d/f6y3lR2/oo8IfxjzJqn1UdDEyCcwT9e7NRloj8DwCYujQ==", "integrity": "sha512-Cus2702JamTNMEY4zTP+TShgq/3qzjvGcBC4XMOV45BLaxD4iUFENkqu7ZhFeSzwNsCSZLjnGlihDQznnpnEEA==",
"license": "Unlicense", "license": "Unlicense",
"engines": { "engines": {
"node": ">=18" "node": ">=18"
@@ -3613,9 +3613,9 @@
} }
}, },
"node_modules/jsdom/node_modules/lru-cache": { "node_modules/jsdom/node_modules/lru-cache": {
"version": "11.3.3", "version": "11.3.5",
"resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-11.3.3.tgz", "resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-11.3.5.tgz",
"integrity": "sha512-JvNw9Y81y33E+BEYPr0U7omo+U9AySnsMsEiXgwT6yqd31VQWTLNQqmT4ou5eqPFUrTfIDFta2wKhB1hyohtAQ==", "integrity": "sha512-NxVFwLAnrd9i7KUBxC4DrUhmgjzOs+1Qm50D3oF1/oL+r1NpZ4gA7xvG0/zJ8evR7zIKn4vLf7qTNduWFtCrRw==",
"dev": true, "dev": true,
"license": "BlueOak-1.0.0", "license": "BlueOak-1.0.0",
"engines": { "engines": {
@@ -4066,9 +4066,9 @@
"license": "MIT" "license": "MIT"
}, },
"node_modules/msw": { "node_modules/msw": {
"version": "2.13.2", "version": "2.13.3",
"resolved": "https://registry.npmjs.org/msw/-/msw-2.13.2.tgz", "resolved": "https://registry.npmjs.org/msw/-/msw-2.13.3.tgz",
"integrity": "sha512-go2H1TIERKkC48pXiwec5l6sbNqYuvqOk3/vHGo1Zd+pq/H63oFawDQerH+WQdUw/flJFHDG7F+QdWMwhntA/A==", "integrity": "sha512-/F49bxavkNGfreMlrKmTxZs6YorjfMbbDLd89Q3pWi+cXGtQQNXXaHt4MkXN7li91xnQJ24HWXqW9QDm5id33w==",
"dev": true, "dev": true,
"hasInstallScript": true, "hasInstallScript": true,
"license": "MIT", "license": "MIT",
@@ -4084,7 +4084,7 @@
"outvariant": "^1.4.3", "outvariant": "^1.4.3",
"path-to-regexp": "^6.3.0", "path-to-regexp": "^6.3.0",
"picocolors": "^1.1.1", "picocolors": "^1.1.1",
"rettime": "^0.10.1", "rettime": "^0.11.7",
"statuses": "^2.0.2", "statuses": "^2.0.2",
"strict-event-emitter": "^0.5.1", "strict-event-emitter": "^0.5.1",
"tough-cookie": "^6.0.0", "tough-cookie": "^6.0.0",
@@ -4299,9 +4299,9 @@
} }
}, },
"node_modules/postcss": { "node_modules/postcss": {
"version": "8.5.9", "version": "8.5.10",
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.9.tgz", "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.10.tgz",
"integrity": "sha512-7a70Nsot+EMX9fFU3064K/kdHWZqGVY+BADLyXc8Dfv+mTLLVl6JzJpPaCZ2kQL9gIJvKXSLMHhqdRRjwQeFtw==", "integrity": "sha512-pMMHxBOZKFU6HgAZ4eyGnwXF/EvPGGqUr0MnZ5+99485wwW41kW91A4LOGxSHhgugZmSChL5AlElNdwlNgcnLQ==",
"funding": [ "funding": [
{ {
"type": "opencollective", "type": "opencollective",
@@ -4531,9 +4531,9 @@
} }
}, },
"node_modules/rettime": { "node_modules/rettime": {
"version": "0.10.1", "version": "0.11.7",
"resolved": "https://registry.npmjs.org/rettime/-/rettime-0.10.1.tgz", "resolved": "https://registry.npmjs.org/rettime/-/rettime-0.11.7.tgz",
"integrity": "sha512-uyDrIlUEH37cinabq0AX4QbgV4HbFZ/gqoiunWQ1UqBtRvTTytwhNYjE++pO/MjPTZL5KQCf2bEoJ/BJNVQ5Kw==", "integrity": "sha512-DoAm1WjR1eH7z8sHPtvvUMIZh4/CSKkGCz6CxPqOrEAnOGtOuHSnSE9OC+razqxKuf4ub7pAYyl/vZV0vGs5tg==",
"dev": true, "dev": true,
"license": "MIT" "license": "MIT"
}, },
@@ -4702,9 +4702,9 @@
} }
}, },
"node_modules/std-env": { "node_modules/std-env": {
"version": "4.0.0", "version": "4.1.0",
"resolved": "https://registry.npmjs.org/std-env/-/std-env-4.0.0.tgz", "resolved": "https://registry.npmjs.org/std-env/-/std-env-4.1.0.tgz",
"integrity": "sha512-zUMPtQ/HBY3/50VbpkupYHbRroTRZJPRLvreamgErJVys0ceuzMkD44J/QjqhHjOzK42GQ3QZIeFG1OYfOtKqQ==", "integrity": "sha512-Rq7ybcX2RuC55r9oaPVEW7/xu3tj8u4GeBYHBWCychFtzMIr86A7e3PPEBPT37sHStKX3+TiX/Fr/ACmJLVlLQ==",
"dev": true, "dev": true,
"license": "MIT" "license": "MIT"
}, },
@@ -4980,16 +4980,16 @@
} }
}, },
"node_modules/typescript-eslint": { "node_modules/typescript-eslint": {
"version": "8.58.1", "version": "8.58.2",
"resolved": "https://registry.npmjs.org/typescript-eslint/-/typescript-eslint-8.58.1.tgz", "resolved": "https://registry.npmjs.org/typescript-eslint/-/typescript-eslint-8.58.2.tgz",
"integrity": "sha512-gf6/oHChByg9HJvhMO1iBexJh12AqqTfnuxscMDOVqfJW3htsdRJI/GfPpHTTcyeB8cSTUY2JcZmVgoyPqcrDg==", "integrity": "sha512-V8iSng9mRbdZjl54VJ9NKr6ZB+dW0J3TzRXRGcSbLIej9jV86ZRtlYeTKDR/QLxXykocJ5icNzbsl2+5TzIvcQ==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@typescript-eslint/eslint-plugin": "8.58.1", "@typescript-eslint/eslint-plugin": "8.58.2",
"@typescript-eslint/parser": "8.58.1", "@typescript-eslint/parser": "8.58.2",
"@typescript-eslint/typescript-estree": "8.58.1", "@typescript-eslint/typescript-estree": "8.58.2",
"@typescript-eslint/utils": "8.58.1" "@typescript-eslint/utils": "8.58.2"
}, },
"engines": { "engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0" "node": "^18.18.0 || ^20.9.0 || >=21.1.0"
@@ -5004,9 +5004,9 @@
} }
}, },
"node_modules/undici": { "node_modules/undici": {
"version": "7.24.7", "version": "7.25.0",
"resolved": "https://registry.npmjs.org/undici/-/undici-7.24.7.tgz", "resolved": "https://registry.npmjs.org/undici/-/undici-7.25.0.tgz",
"integrity": "sha512-H/nlJ/h0ggGC+uRL3ovD+G0i4bqhvsDOpbDv7At5eFLlj2b41L8QliGbnl2H7SnDiYhENphh1tQFJZf+MyfLsQ==", "integrity": "sha512-xXnp4kTyor2Zq+J1FfPI6Eq3ew5h6Vl0F/8d9XU5zZQf1tX9s2Su1/3PiMmUANFULpmksxkClamIZcaUqryHsQ==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"engines": { "engines": {
+724 -3
View File
@@ -1,9 +1,9 @@
import { useState } from 'react'; import { useState, useEffect, useCallback } from 'react';
import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query'; import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query';
import { apiGet, apiPost, apiPut, apiDelete } from '../api/client'; import { apiGet, apiPost, apiPut, apiDelete, ApiError } from '../api/client';
import { Card, LoadingSpinner, StatusBadge } from '../components/ui'; import { Card, LoadingSpinner, StatusBadge } from '../components/ui';
import { import {
LineChart, Line, XAxis, YAxis, Tooltip, ResponsiveContainer, CartesianGrid, LineChart, Line, XAxis, YAxis, Tooltip, ResponsiveContainer, CartesianGrid, Legend,
} from 'recharts'; } from 'recharts';
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
@@ -52,6 +52,29 @@ interface PerfHistoryPoint {
avg_confidence: number; avg_confidence: number;
} }
interface AgentVariant {
id: string;
agent_id: string;
variant_name: string;
variant_slug: string;
description: string;
model_provider: string;
model_name: string;
system_prompt: string;
user_prompt_template: string;
prompt_version: string;
temperature: number;
max_tokens: number;
context_window: number;
input_token_limit: number;
token_budget: number;
timeout_seconds: number;
max_retries: number;
is_active: boolean;
created_at: string;
updated_at: string;
}
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// Hooks // Hooks
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
@@ -79,6 +102,82 @@ function useAgentPerfHistory(agentId: string | undefined, hours = 24) {
}); });
} }
// -- Variant query hooks --
function useAgentVariants(agentId: string | undefined) {
return useQuery<AgentVariant[]>({
queryKey: ['agent-variants', agentId],
queryFn: () => apiGet<AgentVariant[]>('query', `/api/agents/${agentId}/variants`),
enabled: !!agentId,
});
}
export function useVariantPerformance(agentId: string, variantId: string, hours = 24) {
return useQuery<AgentPerformance>({
queryKey: ['variant-performance', agentId, variantId, hours],
queryFn: () => apiGet<AgentPerformance>('query', `/api/agents/${agentId}/variants/${variantId}/performance?hours=${hours}`),
enabled: !!agentId && !!variantId,
});
}
export function useVariantPerfHistory(agentId: string, variantId: string, hours = 24) {
return useQuery<PerfHistoryPoint[]>({
queryKey: ['variant-perf-history', agentId, variantId, hours],
queryFn: () => apiGet<PerfHistoryPoint[]>('query', `/api/agents/${agentId}/variants/${variantId}/performance/history?hours=${hours}`),
enabled: !!agentId && !!variantId,
});
}
// -- Variant mutation hooks --
function useCloneAgentAsVariant(agentId: string) {
const qc = useQueryClient();
return useMutation({
mutationFn: (body: Record<string, unknown>) => apiPost<AgentVariant>('query', `/api/agents/${agentId}/clone`, body),
onSuccess: () => { qc.invalidateQueries({ queryKey: ['agent-variants'] }); },
});
}
export function useCreateVariant(agentId: string) {
const qc = useQueryClient();
return useMutation({
mutationFn: (body: Record<string, unknown>) => apiPost<AgentVariant>('query', `/api/agents/${agentId}/variants`, body),
onSuccess: () => { qc.invalidateQueries({ queryKey: ['agent-variants'] }); },
});
}
function useUpdateVariant(agentId: string, variantId: string) {
const qc = useQueryClient();
return useMutation({
mutationFn: (body: Record<string, unknown>) => apiPut<AgentVariant>('query', `/api/agents/${agentId}/variants/${variantId}`, body),
onSuccess: () => { qc.invalidateQueries({ queryKey: ['agent-variants'] }); },
});
}
function useDeleteVariant(agentId: string, variantId: string) {
const qc = useQueryClient();
return useMutation({
mutationFn: () => apiDelete<unknown>('query', `/api/agents/${agentId}/variants/${variantId}`),
onSuccess: () => { qc.invalidateQueries({ queryKey: ['agent-variants'] }); },
});
}
function useActivateVariant(agentId: string, variantId: string) {
const qc = useQueryClient();
return useMutation({
mutationFn: () => apiPost<AgentVariant>('query', `/api/agents/${agentId}/variants/${variantId}/activate`, {}),
onSuccess: () => { qc.invalidateQueries({ queryKey: ['agent-variants'] }); },
});
}
export function useDeactivateVariants(agentId: string) {
const qc = useQueryClient();
return useMutation({
mutationFn: () => apiPost<unknown>('query', `/api/agents/${agentId}/variants/deactivate`, {}),
onSuccess: () => { qc.invalidateQueries({ queryKey: ['agent-variants'] }); },
});
}
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// Page Component // Page Component
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
@@ -162,6 +261,36 @@ function AgentDetail({ agent, onEdit, onDeleted }: { agent: Agent; onEdit: () =>
const qc = useQueryClient(); const qc = useQueryClient();
const { data: perf } = useAgentPerformance(agent.id); const { data: perf } = useAgentPerformance(agent.id);
const { data: history } = useAgentPerfHistory(agent.id); const { data: history } = useAgentPerfHistory(agent.id);
const { data: variants } = useAgentVariants(agent.id);
// Variant UI state
const [variantView, setVariantView] = useState<
| { mode: 'list' }
| { mode: 'clone-agent' }
| { mode: 'clone-variant'; source: AgentVariant }
| { mode: 'edit-variant'; variant: AgentVariant }
| { mode: 'delete-confirm'; variant: AgentVariant }
>({ mode: 'list' });
// Comparison selection state
const [selectedVariantIds, setSelectedVariantIds] = useState<Set<string>>(new Set());
// Clear selection when variants change (e.g., after activation, deletion)
useEffect(() => {
setSelectedVariantIds(new Set());
}, [variants]);
const toggleVariantSelection = useCallback((variantId: string) => {
setSelectedVariantIds((prev) => {
const next = new Set(prev);
if (next.has(variantId)) {
next.delete(variantId);
} else {
next.add(variantId);
}
return next;
});
}, []);
const deleteMut = useMutation({ const deleteMut = useMutation({
mutationFn: () => apiDelete<unknown>('query', `/api/agents/${agent.id}`), mutationFn: () => apiDelete<unknown>('query', `/api/agents/${agent.id}`),
@@ -188,6 +317,7 @@ function AgentDetail({ agent, onEdit, onDeleted }: { agent: Agent; onEdit: () =>
</span> </span>
</div> </div>
<div className="flex gap-2"> <div className="flex gap-2">
<button onClick={() => setVariantView({ mode: 'clone-agent' })} className="rounded-md border border-brand-500/50 px-3 py-1.5 text-sm font-medium text-brand-300 hover:bg-brand-600/20">Clone as Variant</button>
<button onClick={onEdit} className="rounded-md bg-brand-600 px-3 py-1.5 text-sm font-medium text-white hover:bg-brand-700">Edit</button> <button onClick={onEdit} className="rounded-md bg-brand-600 px-3 py-1.5 text-sm font-medium text-white hover:bg-brand-700">Edit</button>
{agent.source === 'user' && ( {agent.source === 'user' && (
<button onClick={() => deleteMut.mutate()} className="rounded-md border border-red-700/50 px-3 py-1.5 text-sm text-red-400 hover:bg-red-900/20">Delete</button> <button onClick={() => deleteMut.mutate()} className="rounded-md border border-red-700/50 px-3 py-1.5 text-sm text-red-400 hover:bg-red-900/20">Delete</button>
@@ -258,10 +388,601 @@ function AgentDetail({ agent, onEdit, onDeleted }: { agent: Agent; onEdit: () =>
</ResponsiveContainer> </ResponsiveContainer>
</Card> </Card>
)} )}
{/* Variant Section */}
{variantView.mode === 'clone-agent' && (
<VariantCloneForm
agentId={agent.id}
source={{
model_provider: agent.model_provider,
model_name: agent.model_name,
system_prompt: agent.system_prompt,
user_prompt_template: agent.user_prompt_template,
prompt_version: agent.prompt_version,
temperature: agent.temperature,
max_tokens: agent.max_tokens,
context_window: 0,
input_token_limit: 0,
token_budget: 0,
timeout_seconds: agent.timeout_seconds,
max_retries: agent.max_retries,
}}
sourceType="agent"
onDone={() => setVariantView({ mode: 'list' })}
/>
)}
{variantView.mode === 'clone-variant' && (
<VariantCloneForm
agentId={agent.id}
source={variantView.source}
sourceType="variant"
sourceVariantId={variantView.source.id}
onDone={() => setVariantView({ mode: 'list' })}
/>
)}
{variantView.mode === 'edit-variant' && (
<VariantEditForm
agentId={agent.id}
variant={variantView.variant}
onDone={() => setVariantView({ mode: 'list' })}
/>
)}
{variantView.mode === 'delete-confirm' && (
<DeleteVariantDialog
agentId={agent.id}
variant={variantView.variant}
onDone={() => setVariantView({ mode: 'list' })}
/>
)}
<VariantList
agentId={agent.id}
variants={variants ?? []}
selectedIds={selectedVariantIds}
onToggleSelect={toggleVariantSelection}
onClone={(v) => setVariantView({ mode: 'clone-variant', source: v })}
onEdit={(v) => setVariantView({ mode: 'edit-variant', variant: v })}
onDelete={(v) => setVariantView({ mode: 'delete-confirm', variant: v })}
/>
{selectedVariantIds.size >= 2 && (
<VariantCompare
agentId={agent.id}
variants={(variants ?? []).filter((v) => selectedVariantIds.has(v.id))}
/>
)}
</div> </div>
); );
} }
// ---------------------------------------------------------------------------
// Variant List
// ---------------------------------------------------------------------------
function VariantList({ agentId, variants, selectedIds, onToggleSelect, onClone, onEdit, onDelete }: {
agentId: string;
variants: AgentVariant[];
selectedIds: Set<string>;
onToggleSelect: (id: string) => void;
onClone: (v: AgentVariant) => void;
onEdit: (v: AgentVariant) => void;
onDelete: (v: AgentVariant) => void;
}) {
return (
<Card>
<h2 className="mb-3 text-sm font-medium text-gray-400">Variants ({variants.length})</h2>
{variants.length === 0 ? (
<p className="text-xs text-gray-500">No variants yet. Clone this agent to create one.</p>
) : (
<div className="overflow-x-auto">
<table className="w-full text-sm">
<thead>
<tr className="border-b border-surface-700 text-left text-xs text-gray-500">
<th className="pb-2 pr-2 w-8">
<span className="sr-only">Select</span>
</th>
<th className="pb-2 pr-4">Name</th>
<th className="pb-2 pr-4">Model</th>
<th className="pb-2 pr-4">Status</th>
<th className="pb-2 pr-4">Created</th>
<th className="pb-2 text-right">Actions</th>
</tr>
</thead>
<tbody>
{variants.map((v) => (
<VariantRow
key={v.id}
agentId={agentId}
variant={v}
selected={selectedIds.has(v.id)}
onToggleSelect={onToggleSelect}
onClone={onClone}
onEdit={onEdit}
onDelete={onDelete}
/>
))}
</tbody>
</table>
{selectedIds.size > 0 && selectedIds.size < 2 && (
<p className="mt-2 text-[10px] text-gray-500">Select at least 2 variants to compare.</p>
)}
</div>
)}
</Card>
);
}
function VariantRow({ agentId, variant, selected, onToggleSelect, onClone, onEdit, onDelete }: {
agentId: string;
variant: AgentVariant;
selected: boolean;
onToggleSelect: (id: string) => void;
onClone: (v: AgentVariant) => void;
onEdit: (v: AgentVariant) => void;
onDelete: (v: AgentVariant) => void;
}) {
const activateMut = useActivateVariant(agentId, variant.id);
return (
<tr className={`border-b border-surface-800 ${variant.is_active ? 'bg-green-900/10' : ''}`}>
<td className="py-2 pr-2">
<input
type="checkbox"
checked={selected}
onChange={() => onToggleSelect(variant.id)}
className="rounded border-surface-600 bg-surface-950 text-brand-500 focus:ring-brand-500 focus:ring-offset-0 h-3.5 w-3.5"
/>
</td>
<td className="py-2 pr-4">
<span className="font-medium text-gray-200">{variant.variant_name}</span>
{variant.description && <p className="text-[10px] text-gray-500 truncate max-w-[200px]">{variant.description}</p>}
</td>
<td className="py-2 pr-4 font-mono text-xs text-gray-400">{variant.model_name}</td>
<td className="py-2 pr-4">
{variant.is_active ? <StatusBadge status="active" /> : <span className="text-xs text-gray-600">inactive</span>}
</td>
<td className="py-2 pr-4 text-xs text-gray-500">{new Date(variant.created_at).toLocaleDateString()}</td>
<td className="py-2 text-right">
<div className="flex justify-end gap-1">
<button onClick={() => onEdit(variant)} className="rounded px-2 py-0.5 text-[10px] text-gray-400 hover:bg-surface-800 hover:text-gray-200">Edit</button>
<button onClick={() => onClone(variant)} className="rounded px-2 py-0.5 text-[10px] text-gray-400 hover:bg-surface-800 hover:text-gray-200">Clone</button>
<button onClick={() => onDelete(variant)} className="rounded px-2 py-0.5 text-[10px] text-red-400 hover:bg-red-900/20">Delete</button>
<button
onClick={() => activateMut.mutate()}
disabled={variant.is_active || activateMut.isPending}
className="rounded px-2 py-0.5 text-[10px] font-medium text-brand-400 hover:bg-brand-600/20 disabled:opacity-30 disabled:cursor-not-allowed"
>
{activateMut.isPending ? '…' : 'Activate'}
</button>
</div>
</td>
</tr>
);
}
// ---------------------------------------------------------------------------
// Variant Comparison View
// ---------------------------------------------------------------------------
const COMPARE_COLORS = ['#3b82f6', '#10b981', '#f59e0b', '#ef4444', '#8b5cf6', '#ec4899', '#06b6d4', '#84cc16'];
function VariantCompareColumn({ agentId, variant, color }: {
agentId: string;
variant: AgentVariant;
color: string;
}) {
const { data: perf } = useVariantPerformance(agentId, variant.id);
const activateMut = useActivateVariant(agentId, variant.id);
return (
<td className="px-3 py-2 text-center text-sm">
<div className="mb-2">
<span className="font-medium text-gray-200">{variant.variant_name}</span>
<span className="ml-2 text-[10px] font-mono text-gray-500">{variant.model_name}</span>
{variant.is_active && <StatusBadge status="active" />}
</div>
<div className="space-y-1.5 text-xs">
<div>
<span className="text-gray-500">Success Rate</span>
<div className={`font-bold ${perf?.success_rate != null && perf.success_rate >= 0.95 ? 'text-green-400' : 'text-yellow-400'}`}>
{perf?.success_rate != null ? `${(perf.success_rate * 100).toFixed(1)}%` : '—'}
</div>
</div>
<div>
<span className="text-gray-500">Avg Latency</span>
<div className="text-gray-200">{perf?.avg_duration_ms != null ? `${Math.round(perf.avg_duration_ms)}ms` : '—'}</div>
</div>
<div>
<span className="text-gray-500">P95 Latency</span>
<div className="text-gray-200">{perf?.p95_duration_ms != null ? `${Math.round(perf.p95_duration_ms)}ms` : '—'}</div>
</div>
<div>
<span className="text-gray-500">Avg Confidence</span>
<div className="text-gray-200">{perf?.avg_confidence != null ? `${(perf.avg_confidence * 100).toFixed(0)}%` : '—'}</div>
</div>
<div>
<span className="text-gray-500">Total Tokens</span>
<div className="text-gray-200">
{perf?.total_input_tokens != null || perf?.total_output_tokens != null
? ((perf.total_input_tokens ?? 0) + (perf.total_output_tokens ?? 0)).toLocaleString()
: '—'}
</div>
</div>
</div>
{!variant.is_active && (
<button
onClick={() => activateMut.mutate()}
disabled={activateMut.isPending}
className="mt-3 rounded-md px-3 py-1 text-[10px] font-medium text-white hover:opacity-90 disabled:opacity-30"
style={{ backgroundColor: color }}
>
{activateMut.isPending ? 'Activating…' : 'Activate'}
</button>
)}
</td>
);
}
function VariantCompareChart({ agentId, variants }: { agentId: string; variants: AgentVariant[] }) {
// Fetch history for each selected variant
const historyQueries = variants.map((v) =>
// eslint-disable-next-line react-hooks/rules-of-hooks
useVariantPerfHistory(agentId, v.id)
);
// Merge all history data into a unified time-series keyed by hour
const hourMap = new Map<string, Record<string, number | string>>();
variants.forEach((v, idx) => {
const history = historyQueries[idx].data ?? [];
for (const pt of history) {
const hourKey = new Date(pt.hour).toLocaleTimeString([], { hour: '2-digit', minute: '2-digit' });
if (!hourMap.has(hourKey)) {
hourMap.set(hourKey, { hour: hourKey });
}
const entry = hourMap.get(hourKey)!;
entry[`sr_${v.id}`] = pt.invocations > 0 ? Math.round((pt.successes / pt.invocations) * 100) : 0;
entry[`lat_${v.id}`] = Math.round(pt.avg_duration_ms);
}
});
const chartData = Array.from(hourMap.values());
if (chartData.length < 2) return null;
return (
<div className="mt-4">
<h3 className="mb-2 text-xs font-medium text-gray-500">Success Rate Over Time</h3>
<ResponsiveContainer width="100%" height={200}>
<LineChart data={chartData}>
<CartesianGrid strokeDasharray="3 3" stroke="#334155" />
<XAxis dataKey="hour" tick={{ fill: '#6b7280', fontSize: 10 }} />
<YAxis tick={{ fill: '#6b7280', fontSize: 10 }} />
<Tooltip contentStyle={{ backgroundColor: '#1e293b', border: '1px solid #334155', borderRadius: 8 }} />
<Legend />
{variants.map((v, idx) => (
<Line
key={v.id}
type="monotone"
dataKey={`sr_${v.id}`}
name={v.variant_name}
stroke={COMPARE_COLORS[idx % COMPARE_COLORS.length]}
strokeWidth={2}
dot={false}
/>
))}
</LineChart>
</ResponsiveContainer>
</div>
);
}
function VariantCompare({ agentId, variants }: { agentId: string; variants: AgentVariant[] }) {
return (
<Card>
<h2 className="mb-3 text-sm font-medium text-gray-400">Variant Comparison ({variants.length} selected)</h2>
<div className="overflow-x-auto">
<table className="w-full text-sm">
<thead>
<tr className="border-b border-surface-700">
<th className="pb-2 pr-4 text-left text-xs text-gray-500">Metric</th>
{variants.map((v, idx) => (
<th key={v.id} className="pb-2 px-3 text-center text-xs" style={{ color: COMPARE_COLORS[idx % COMPARE_COLORS.length] }}>
{v.variant_name}
</th>
))}
</tr>
</thead>
<tbody>
<tr>
<td className="py-1 pr-4 text-xs text-gray-500">Metrics</td>
{variants.map((v, idx) => (
<VariantCompareColumn
key={v.id}
agentId={agentId}
variant={v}
color={COMPARE_COLORS[idx % COMPARE_COLORS.length]}
/>
))}
</tr>
</tbody>
</table>
</div>
<VariantCompareChart agentId={agentId} variants={variants} />
</Card>
);
}
// ---------------------------------------------------------------------------
// Variant Clone Form
// ---------------------------------------------------------------------------
interface VariantConfigSource {
model_provider: string;
model_name: string;
system_prompt: string;
user_prompt_template: string;
prompt_version: string;
temperature: number;
max_tokens: number;
context_window: number;
input_token_limit: number;
token_budget: number;
timeout_seconds: number;
max_retries: number;
}
function VariantCloneForm({ agentId, source, sourceType, sourceVariantId, onDone }: {
agentId: string;
source: VariantConfigSource;
sourceType: 'agent' | 'variant';
sourceVariantId?: string;
onDone: () => void;
}) {
const [form, setForm] = useState({
variant_name: '',
description: '',
model_provider: source.model_provider,
model_name: source.model_name,
system_prompt: source.system_prompt,
user_prompt_template: source.user_prompt_template,
prompt_version: source.prompt_version,
temperature: source.temperature,
max_tokens: source.max_tokens,
context_window: source.context_window,
input_token_limit: source.input_token_limit,
token_budget: source.token_budget,
timeout_seconds: source.timeout_seconds,
max_retries: source.max_retries,
});
const cloneAgentMut = useCloneAgentAsVariant(agentId);
const qc = useQueryClient();
// For cloning from a variant, we use a direct mutation since the hook needs variantId
const cloneVariantMut = useMutation({
mutationFn: (body: Record<string, unknown>) => apiPost<AgentVariant>('query', `/api/agents/${agentId}/variants/${sourceVariantId}/clone`, body),
onSuccess: () => { qc.invalidateQueries({ queryKey: ['agent-variants'] }); },
});
const mutation = sourceType === 'agent' ? cloneAgentMut : cloneVariantMut;
function handleSubmit(e: React.FormEvent) {
e.preventDefault();
mutation.mutate(form, { onSuccess: () => onDone() });
}
return (
<Card>
<h2 className="mb-4 text-sm font-medium text-gray-400">
Clone {sourceType === 'agent' ? 'Agent' : 'Variant'} as New Variant
</h2>
<form onSubmit={handleSubmit} className="space-y-3">
<div className="grid grid-cols-2 gap-3">
<FormRow label="Variant Name">
<input value={form.variant_name} onChange={(e) => setForm({ ...form, variant_name: e.target.value })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" required />
</FormRow>
<FormRow label="Description">
<input value={form.description} onChange={(e) => setForm({ ...form, description: e.target.value })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" />
</FormRow>
</div>
<div className="grid grid-cols-2 gap-3">
<FormRow label="Provider">
<input value={form.model_provider} onChange={(e) => setForm({ ...form, model_provider: e.target.value })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" />
</FormRow>
<FormRow label="Model">
<input value={form.model_name} onChange={(e) => setForm({ ...form, model_name: e.target.value })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" required />
</FormRow>
</div>
<FormRow label="System Prompt">
<textarea value={form.system_prompt} onChange={(e) => setForm({ ...form, system_prompt: e.target.value })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none h-24 font-mono text-xs" />
</FormRow>
<FormRow label="User Prompt Template">
<textarea value={form.user_prompt_template} onChange={(e) => setForm({ ...form, user_prompt_template: e.target.value })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none h-20 font-mono text-xs" />
</FormRow>
<div className="grid grid-cols-4 gap-3">
<FormRow label="Temperature">
<input type="number" step="0.1" min="0" max="2" value={form.temperature} onChange={(e) => setForm({ ...form, temperature: Number(e.target.value) })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" />
</FormRow>
<FormRow label="Max Tokens">
<input type="number" value={form.max_tokens} onChange={(e) => setForm({ ...form, max_tokens: Number(e.target.value) })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" />
</FormRow>
<FormRow label="Timeout (s)">
<input type="number" value={form.timeout_seconds} onChange={(e) => setForm({ ...form, timeout_seconds: Number(e.target.value) })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" />
</FormRow>
<FormRow label="Max Retries">
<input type="number" value={form.max_retries} onChange={(e) => setForm({ ...form, max_retries: Number(e.target.value) })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" />
</FormRow>
</div>
<div className="grid grid-cols-3 gap-3">
<FormRow label="Context Window (0 = default)">
<input type="number" min="0" value={form.context_window} onChange={(e) => setForm({ ...form, context_window: Number(e.target.value) })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" />
</FormRow>
<FormRow label="Input Token Limit (0 = unlimited)">
<input type="number" min="0" value={form.input_token_limit} onChange={(e) => setForm({ ...form, input_token_limit: Number(e.target.value) })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" />
</FormRow>
<FormRow label="Token Budget/hr (0 = unlimited)">
<input type="number" min="0" value={form.token_budget} onChange={(e) => setForm({ ...form, token_budget: Number(e.target.value) })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" />
</FormRow>
</div>
<FormRow label="Prompt Version">
<input value={form.prompt_version} onChange={(e) => setForm({ ...form, prompt_version: e.target.value })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" />
</FormRow>
<div className="flex gap-2 pt-2">
<button type="submit" disabled={mutation.isPending} className="rounded-md bg-brand-600 px-4 py-1.5 text-sm font-medium text-white hover:bg-brand-700 disabled:opacity-50">
{mutation.isPending ? 'Cloning…' : 'Clone'}
</button>
<button type="button" onClick={onDone} className="rounded-md border border-surface-700 px-4 py-1.5 text-sm text-gray-400 hover:bg-surface-800">Cancel</button>
</div>
{mutation.isError && <p className="text-xs text-red-400">Failed to clone: {(mutation.error as ApiError)?.status === 409 ? 'Slug already exists' : 'Unknown error'}</p>}
</form>
</Card>
);
}
// ---------------------------------------------------------------------------
// Variant Edit Form
// ---------------------------------------------------------------------------
function VariantEditForm({ agentId, variant, onDone }: {
agentId: string;
variant: AgentVariant;
onDone: () => void;
}) {
const [form, setForm] = useState({
variant_name: variant.variant_name,
description: variant.description,
model_provider: variant.model_provider,
model_name: variant.model_name,
system_prompt: variant.system_prompt,
user_prompt_template: variant.user_prompt_template,
prompt_version: variant.prompt_version,
temperature: variant.temperature,
max_tokens: variant.max_tokens,
context_window: variant.context_window,
input_token_limit: variant.input_token_limit,
token_budget: variant.token_budget,
timeout_seconds: variant.timeout_seconds,
max_retries: variant.max_retries,
});
const mutation = useUpdateVariant(agentId, variant.id);
function handleSubmit(e: React.FormEvent) {
e.preventDefault();
mutation.mutate(form, { onSuccess: () => onDone() });
}
return (
<Card>
<h2 className="mb-4 text-sm font-medium text-gray-400">Edit Variant: {variant.variant_name}</h2>
<form onSubmit={handleSubmit} className="space-y-3">
<div className="grid grid-cols-2 gap-3">
<FormRow label="Variant Name">
<input value={form.variant_name} onChange={(e) => setForm({ ...form, variant_name: e.target.value })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" required />
</FormRow>
<FormRow label="Description">
<input value={form.description} onChange={(e) => setForm({ ...form, description: e.target.value })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" />
</FormRow>
</div>
<div className="grid grid-cols-2 gap-3">
<FormRow label="Provider">
<input value={form.model_provider} onChange={(e) => setForm({ ...form, model_provider: e.target.value })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" />
</FormRow>
<FormRow label="Model">
<input value={form.model_name} onChange={(e) => setForm({ ...form, model_name: e.target.value })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" required />
</FormRow>
</div>
<FormRow label="System Prompt">
<textarea value={form.system_prompt} onChange={(e) => setForm({ ...form, system_prompt: e.target.value })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none h-24 font-mono text-xs" />
</FormRow>
<FormRow label="User Prompt Template">
<textarea value={form.user_prompt_template} onChange={(e) => setForm({ ...form, user_prompt_template: e.target.value })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none h-20 font-mono text-xs" />
</FormRow>
<div className="grid grid-cols-4 gap-3">
<FormRow label="Temperature">
<input type="number" step="0.1" min="0" max="2" value={form.temperature} onChange={(e) => setForm({ ...form, temperature: Number(e.target.value) })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" />
</FormRow>
<FormRow label="Max Tokens">
<input type="number" value={form.max_tokens} onChange={(e) => setForm({ ...form, max_tokens: Number(e.target.value) })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" />
</FormRow>
<FormRow label="Timeout (s)">
<input type="number" value={form.timeout_seconds} onChange={(e) => setForm({ ...form, timeout_seconds: Number(e.target.value) })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" />
</FormRow>
<FormRow label="Max Retries">
<input type="number" value={form.max_retries} onChange={(e) => setForm({ ...form, max_retries: Number(e.target.value) })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" />
</FormRow>
</div>
<div className="grid grid-cols-3 gap-3">
<FormRow label="Context Window (0 = default)">
<input type="number" min="0" value={form.context_window} onChange={(e) => setForm({ ...form, context_window: Number(e.target.value) })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" />
</FormRow>
<FormRow label="Input Token Limit (0 = unlimited)">
<input type="number" min="0" value={form.input_token_limit} onChange={(e) => setForm({ ...form, input_token_limit: Number(e.target.value) })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" />
</FormRow>
<FormRow label="Token Budget/hr (0 = unlimited)">
<input type="number" min="0" value={form.token_budget} onChange={(e) => setForm({ ...form, token_budget: Number(e.target.value) })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" />
</FormRow>
</div>
<FormRow label="Prompt Version">
<input value={form.prompt_version} onChange={(e) => setForm({ ...form, prompt_version: e.target.value })} className="w-full rounded-md border border-surface-700 bg-surface-950 px-2 py-1.5 text-sm text-gray-200 focus:border-brand-500 focus:outline-none" />
</FormRow>
<div className="flex gap-2 pt-2">
<button type="submit" disabled={mutation.isPending} className="rounded-md bg-brand-600 px-4 py-1.5 text-sm font-medium text-white hover:bg-brand-700 disabled:opacity-50">
{mutation.isPending ? 'Saving…' : 'Save'}
</button>
<button type="button" onClick={onDone} className="rounded-md border border-surface-700 px-4 py-1.5 text-sm text-gray-400 hover:bg-surface-800">Cancel</button>
</div>
{mutation.isError && <p className="text-xs text-red-400">Failed to save variant</p>}
</form>
</Card>
);
}
// ---------------------------------------------------------------------------
// Delete Variant Dialog
// ---------------------------------------------------------------------------
function DeleteVariantDialog({ agentId, variant, onDone }: {
agentId: string;
variant: AgentVariant;
onDone: () => void;
}) {
const mutation = useDeleteVariant(agentId, variant.id);
const [error, setError] = useState<string | null>(null);
function handleDelete() {
setError(null);
mutation.mutate(undefined, {
onSuccess: () => onDone(),
onError: (err) => {
if (err instanceof ApiError && err.status === 400) {
setError('Cannot delete the active variant. Deactivate it first or activate a different variant.');
} else {
setError('Failed to delete variant.');
}
},
});
}
return (
<Card>
<h2 className="mb-3 text-sm font-medium text-red-400">Delete Variant</h2>
<p className="text-sm text-gray-300 mb-1">
Are you sure you want to delete <span className="font-medium text-gray-100">{variant.variant_name}</span>?
</p>
<p className="text-xs text-gray-500 mb-4">This action cannot be undone. Associated performance log entries will be unlinked.</p>
{error && <p className="text-xs text-red-400 mb-3">{error}</p>}
<div className="flex gap-2">
<button onClick={handleDelete} disabled={mutation.isPending} className="rounded-md bg-red-600 px-4 py-1.5 text-sm font-medium text-white hover:bg-red-700 disabled:opacity-50">
{mutation.isPending ? 'Deleting…' : 'Delete'}
</button>
<button onClick={onDone} className="rounded-md border border-surface-700 px-4 py-1.5 text-sm text-gray-400 hover:bg-surface-800">Cancel</button>
</div>
</Card>
);
}
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// Edit Form // Edit Form
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
+67
View File
@@ -54,6 +54,25 @@ export const mockCorporateDecisions = [
{ catalyst_type: 'm_and_a', date: '2026-03-15T00:00:00Z', summary: 'Acquisition of AI startup for $2B', trend_direction: 'bullish', trend_strength: 0.7, sample_count: 5, pattern_confidence: 0.68, document_id: 'd1' }, { catalyst_type: 'm_and_a', date: '2026-03-15T00:00:00Z', summary: 'Acquisition of AI startup for $2B', trend_direction: 'bullish', trend_strength: 0.7, sample_count: 5, pattern_confidence: 0.68, document_id: 'd1' },
]; ];
export const mockAgents = [
{ id: 'agent-1', name: 'Document Extractor', slug: 'document-extractor', purpose: 'Extract structured data from documents', model_provider: 'ollama', model_name: 'llama3.1:8b', system_prompt: 'You are a document extractor.', user_prompt_template: 'Extract from: {doc}', prompt_version: 'v1', schema_version: '1.0', temperature: 0.0, max_tokens: 32768, timeout_seconds: 120, max_retries: 2, active: true, source: 'system', created_at: '2026-04-01T00:00:00Z', updated_at: '2026-04-01T00:00:00Z' },
{ id: 'agent-2', name: 'Event Classifier', slug: 'event-classifier', purpose: 'Classify global events', model_provider: 'ollama', model_name: 'llama3.1:8b', system_prompt: 'You classify events.', user_prompt_template: 'Classify: {event}', prompt_version: 'v1', schema_version: '1.0', temperature: 0.1, max_tokens: 16384, timeout_seconds: 60, max_retries: 3, active: true, source: 'system', created_at: '2026-04-02T00:00:00Z', updated_at: '2026-04-02T00:00:00Z' },
];
export const mockAgentVariants = [
{ id: 'var-1', agent_id: 'agent-1', variant_name: 'GPT-4o Variant', variant_slug: 'gpt-4o-variant', description: 'Uses GPT-4o for extraction', model_provider: 'openai', model_name: 'gpt-4o', system_prompt: 'You are a document extractor.', user_prompt_template: 'Extract from: {doc}', prompt_version: 'v2', temperature: 0.2, max_tokens: 65536, context_window: 128000, input_token_limit: 100000, token_budget: 500000, timeout_seconds: 90, max_retries: 3, is_active: true, created_at: '2026-04-05T00:00:00Z', updated_at: '2026-04-05T00:00:00Z' },
{ id: 'var-2', agent_id: 'agent-1', variant_name: 'Mistral Variant', variant_slug: 'mistral-variant', description: 'Uses Mistral for extraction', model_provider: 'ollama', model_name: 'mistral:7b', system_prompt: 'You are a document extractor.', user_prompt_template: 'Extract from: {doc}', prompt_version: 'v1', temperature: 0.0, max_tokens: 32768, context_window: 0, input_token_limit: 0, token_budget: 0, timeout_seconds: 120, max_retries: 2, is_active: false, created_at: '2026-04-06T00:00:00Z', updated_at: '2026-04-06T00:00:00Z' },
];
export const mockVariantPerformance = {
total_invocations: 50, successes: 45, failures: 5, avg_duration_ms: 1200, p95_duration_ms: 2500, avg_confidence: 0.85, avg_retries: 0.3, total_input_tokens: 50000, total_output_tokens: 15000, success_rate: 0.9,
};
export const mockVariantPerfHistory = [
{ hour: '2026-04-10T10:00:00Z', invocations: 10, successes: 9, avg_duration_ms: 1100, avg_confidence: 0.88 },
{ hour: '2026-04-10T11:00:00Z', invocations: 12, successes: 11, avg_duration_ms: 1300, avg_confidence: 0.82 },
];
export const handlers = [ export const handlers = [
// Query API (proxied at /api/) // Query API (proxied at /api/)
http.get('/api/companies', () => HttpResponse.json(mockCompanies)), http.get('/api/companies', () => HttpResponse.json(mockCompanies)),
@@ -142,6 +161,54 @@ export const handlers = [
}), }),
http.get('/api/trends/:id/projection', () => HttpResponse.json(mockTrendProjection)), http.get('/api/trends/:id/projection', () => HttpResponse.json(mockTrendProjection)),
// Agents
http.get('/api/agents', () => HttpResponse.json(mockAgents)),
http.get('/api/agents/:agent_id', ({ params }) => {
const a = mockAgents.find((a) => a.id === params.agent_id);
return a ? HttpResponse.json(a) : new HttpResponse(null, { status: 404 });
}),
http.get('/api/agents/:agent_id/performance', () => HttpResponse.json(mockVariantPerformance)),
http.get('/api/agents/:agent_id/performance/history', () => HttpResponse.json(mockVariantPerfHistory)),
// Agent Variants
http.get('/api/agents/:agent_id/variants', ({ params }) =>
HttpResponse.json(mockAgentVariants.filter((v) => v.agent_id === params.agent_id)),
),
http.get('/api/agents/:agent_id/variants/:variant_id', ({ params }) => {
const v = mockAgentVariants.find((v) => v.id === params.variant_id && v.agent_id === params.agent_id);
return v ? HttpResponse.json(v) : new HttpResponse(null, { status: 404 });
}),
http.post('/api/agents/:agent_id/variants', async ({ params, request }) => {
const body = await request.json() as Record<string, unknown>;
return HttpResponse.json({ id: 'var-new', agent_id: params.agent_id, variant_name: body.variant_name, variant_slug: body.variant_slug ?? 'new-variant', is_active: false, ...body, created_at: '2026-04-10T00:00:00Z', updated_at: '2026-04-10T00:00:00Z' }, { status: 201 });
}),
http.post('/api/agents/:agent_id/clone', async ({ params, request }) => {
const body = await request.json() as Record<string, unknown>;
return HttpResponse.json({ id: 'var-cloned', agent_id: params.agent_id, variant_name: body.variant_name, variant_slug: 'cloned-variant', is_active: false, ...body, created_at: '2026-04-10T00:00:00Z', updated_at: '2026-04-10T00:00:00Z' }, { status: 201 });
}),
http.post('/api/agents/:agent_id/variants/:variant_id/clone', async ({ params, request }) => {
const body = await request.json() as Record<string, unknown>;
return HttpResponse.json({ id: 'var-clone2', agent_id: params.agent_id, variant_name: body.variant_name, variant_slug: 'clone2', is_active: false, ...body, created_at: '2026-04-10T00:00:00Z', updated_at: '2026-04-10T00:00:00Z' }, { status: 201 });
}),
http.put('/api/agents/:agent_id/variants/:variant_id', async ({ params, request }) => {
const body = await request.json() as Record<string, unknown>;
const v = mockAgentVariants.find((v) => v.id === params.variant_id);
return v ? HttpResponse.json({ ...v, ...body, updated_at: '2026-04-10T12:00:00Z' }) : new HttpResponse(null, { status: 404 });
}),
http.delete('/api/agents/:agent_id/variants/:variant_id', ({ params }) => {
const v = mockAgentVariants.find((v) => v.id === params.variant_id);
if (!v) return new HttpResponse(null, { status: 404 });
if (v.is_active) return HttpResponse.json({ detail: 'Cannot delete active variant' }, { status: 400 });
return HttpResponse.json({ status: 'deleted' });
}),
http.post('/api/agents/:agent_id/variants/:variant_id/activate', ({ params }) => {
const v = mockAgentVariants.find((v) => v.id === params.variant_id);
return v ? HttpResponse.json({ ...v, is_active: true }) : new HttpResponse(null, { status: 404 });
}),
http.post('/api/agents/:agent_id/variants/deactivate', () => HttpResponse.json({ deactivated: true })),
http.get('/api/agents/:agent_id/variants/:variant_id/performance', () => HttpResponse.json(mockVariantPerformance)),
http.get('/api/agents/:agent_id/variants/:variant_id/performance/history', () => HttpResponse.json(mockVariantPerfHistory)),
// Competitive intelligence endpoints // Competitive intelligence endpoints
http.get('/registry/companies/:id/competitors', () => HttpResponse.json(mockCompetitors)), http.get('/registry/companies/:id/competitors', () => HttpResponse.json(mockCompetitors)),
http.post('/registry/companies/:id/competitors/infer', () => HttpResponse.json(mockCompetitors)), http.post('/registry/companies/:id/competitors/infer', () => HttpResponse.json(mockCompetitors)),
+40
View File
@@ -168,3 +168,43 @@ describe('Global Events page', () => {
}); });
}); });
}); });
describe('Agents page', () => {
it('renders agent list in sidebar', async () => {
renderRoute('/agents');
await waitFor(() => {
expect(screen.getByText('Document Extractor')).toBeInTheDocument();
expect(screen.getByText('Event Classifier')).toBeInTheDocument();
});
});
it('renders variant list when an agent is selected', async () => {
renderRoute('/agents');
await waitFor(() => expect(screen.getByText('Document Extractor')).toBeInTheDocument());
await userEvent.click(screen.getByText('Document Extractor'));
await waitFor(() => {
expect(screen.getByText('GPT-4o Variant')).toBeInTheDocument();
expect(screen.getByText('Mistral Variant')).toBeInTheDocument();
});
});
it('shows comparison view when multiple variants are checked', async () => {
renderRoute('/agents');
await waitFor(() => expect(screen.getByText('Document Extractor')).toBeInTheDocument());
await userEvent.click(screen.getByText('Document Extractor'));
await waitFor(() => expect(screen.getByText('GPT-4o Variant')).toBeInTheDocument());
// Select both variant checkboxes
const checkboxes = screen.getAllByRole('checkbox');
await userEvent.click(checkboxes[0]);
await userEvent.click(checkboxes[1]);
await waitFor(() => {
expect(screen.getByText(/Variant Comparison/)).toBeInTheDocument();
});
});
});
+47
View File
@@ -0,0 +1,47 @@
-- Agent variant configurations: alternative model/prompt/parameter sets per agent.
-- Each agent can have multiple variants for A/B testing, model comparison,
-- and iterative prompt engineering. At most one variant per agent is active.
CREATE TABLE IF NOT EXISTS agent_variants (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
agent_id UUID NOT NULL REFERENCES ai_agents(id) ON DELETE CASCADE,
variant_name VARCHAR(200) NOT NULL,
variant_slug VARCHAR(200) NOT NULL,
description TEXT NOT NULL DEFAULT '',
model_provider VARCHAR(50) NOT NULL DEFAULT 'ollama',
model_name VARCHAR(200) NOT NULL,
system_prompt TEXT NOT NULL DEFAULT '',
user_prompt_template TEXT NOT NULL DEFAULT '',
prompt_version VARCHAR(100) NOT NULL DEFAULT '',
temperature FLOAT DEFAULT 0.0,
max_tokens INTEGER DEFAULT 32768,
context_window INTEGER DEFAULT 0, -- Ollama num_ctx; 0 = use model default
input_token_limit INTEGER DEFAULT 0, -- max input tokens before truncation; 0 = no limit
token_budget INTEGER DEFAULT 0, -- total tokens per hour; 0 = unlimited
timeout_seconds INTEGER DEFAULT 120,
max_retries INTEGER DEFAULT 2,
is_active BOOLEAN NOT NULL DEFAULT FALSE,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
-- Each agent can have many variants, but variant slugs must be unique per agent.
CREATE UNIQUE INDEX IF NOT EXISTS idx_agent_variants_slug
ON agent_variants(agent_id, variant_slug);
-- At most one active variant per agent (database-enforced invariant).
CREATE UNIQUE INDEX IF NOT EXISTS idx_agent_variants_active
ON agent_variants(agent_id) WHERE is_active = TRUE;
-- Fast lookup by agent.
CREATE INDEX IF NOT EXISTS idx_agent_variants_agent
ON agent_variants(agent_id);
-- Add variant_id to performance log for per-variant attribution.
-- Nullable so existing rows are unaffected; ON DELETE SET NULL preserves
-- historical log entries when a variant is removed.
ALTER TABLE agent_performance_log
ADD COLUMN IF NOT EXISTS variant_id UUID REFERENCES agent_variants(id) ON DELETE SET NULL;
CREATE INDEX IF NOT EXISTS idx_agent_perf_variant
ON agent_performance_log(variant_id, recorded_at DESC);
+4
View File
@@ -263,6 +263,10 @@ class OllamaClient:
}, },
} }
# Support context_window override via num_ctx (Requirement 10.4)
if self._config.context_window > 0:
payload["options"]["num_ctx"] = self._config.context_window
url = f"{self._config.base_url}/api/chat" url = f"{self._config.base_url}/api/chat"
logger.info( logger.info(
"Ollama POST %s model=%s input_chars=%d", "Ollama POST %s model=%s input_chars=%d",
+147 -2
View File
@@ -16,8 +16,10 @@ from __future__ import annotations
import logging import logging
import time import time
import asyncpg
import httpx import httpx
from services.shared.agent_config import AgentConfigResolver, ResolvedAgentConfig
from services.shared.config import OllamaConfig from services.shared.config import OllamaConfig
from services.shared.schemas import TrendSummary from services.shared.schemas import TrendSummary
@@ -86,6 +88,7 @@ async def rewrite_thesis_with_llm(
summary: TrendSummary, summary: TrendSummary,
config: OllamaConfig, config: OllamaConfig,
http_client: httpx.AsyncClient | None = None, http_client: httpx.AsyncClient | None = None,
pool: asyncpg.Pool | None = None,
) -> str: ) -> str:
"""Rewrite a deterministic thesis using a local Ollama model. """Rewrite a deterministic thesis using a local Ollama model.
@@ -93,22 +96,90 @@ async def rewrite_thesis_with_llm(
deterministic thesis unchanged. This ensures the LLM layer is deterministic thesis unchanged. This ensures the LLM layer is
purely additive and never blocks recommendation generation. purely additive and never blocks recommendation generation.
Resolves runtime config for the "thesis-rewriter" agent slug from
the database when a pool is provided, preferring an active variant's
model, timeout, and max_retries. Falls back to the passed
OllamaConfig if resolution fails.
Args: Args:
deterministic_thesis: The rule-based thesis string. deterministic_thesis: The rule-based thesis string.
summary: The trend summary that produced the thesis. summary: The trend summary that produced the thesis.
config: Ollama connection and model configuration. config: Ollama connection and model configuration.
http_client: Optional shared HTTP client for connection reuse. http_client: Optional shared HTTP client for connection reuse.
pool: Optional asyncpg pool for config resolution and performance logging.
Returns: Returns:
The LLM-rewritten thesis on success, or the original on failure. The LLM-rewritten thesis on success, or the original on failure.
""" """
start_time = time.monotonic()
# Resolve thesis-rewriter config from DB for variant override
resolved: ResolvedAgentConfig | None = None
effective_config = config
if pool is not None:
try:
resolver = AgentConfigResolver(pool, ttl_seconds=60)
resolved = await resolver.resolve("thesis-rewriter")
if resolved is not None:
effective_config = OllamaConfig(
base_url=config.base_url,
model=resolved.model_name,
timeout=resolved.timeout_seconds,
max_retries=resolved.max_retries,
retry_base_delay=config.retry_base_delay,
retry_max_delay=config.retry_max_delay,
retry_backoff_multiplier=config.retry_backoff_multiplier,
max_tokens=resolved.max_tokens,
context_window=resolved.context_window,
)
logger.info(
"Thesis rewriter using resolved config: model=%s variant=%s",
resolved.model_name, resolved.variant_id,
)
except Exception:
logger.warning(
"Failed to resolve thesis-rewriter config — using passed config",
exc_info=True,
)
# Token budget enforcement
if (
resolved is not None
and resolved.token_budget > 0
and resolved.variant_id is not None
and pool is not None
):
try:
row = await pool.fetchrow(
"""SELECT COALESCE(SUM(input_tokens + output_tokens), 0) AS total_tokens
FROM agent_performance_log
WHERE variant_id = $1
AND recorded_at >= NOW() - INTERVAL '1 hour'""",
resolved.variant_id,
)
used = int(row["total_tokens"]) if row else 0
if used >= resolved.token_budget:
logger.warning(
"Token budget exceeded for thesis-rewriter variant %s: used %d / budget %d",
resolved.variant_id, used, resolved.token_budget,
)
return deterministic_thesis
except Exception:
logger.warning("Failed to check token budget for thesis-rewriter", exc_info=True)
prompts = build_thesis_rewrite_prompt(deterministic_thesis, summary) prompts = build_thesis_rewrite_prompt(deterministic_thesis, summary)
# Override system prompt from resolved config if available
if resolved is not None and resolved.system_prompt:
prompts["system"] = resolved.system_prompt
owns_client = http_client is None owns_client = http_client is None
client = http_client or httpx.AsyncClient(timeout=config.timeout) client = http_client or httpx.AsyncClient(timeout=effective_config.timeout)
try: try:
rewritten = await _call_ollama_thesis(client, config, prompts) rewritten = await _call_ollama_thesis(client, effective_config, prompts)
duration_ms = int((time.monotonic() - start_time) * 1000)
if rewritten: if rewritten:
logger.info( logger.info(
"LLM thesis rewrite succeeded for %s (%d chars → %d chars)", "LLM thesis rewrite succeeded for %s (%d chars → %d chars)",
@@ -116,24 +187,94 @@ async def rewrite_thesis_with_llm(
len(deterministic_thesis), len(deterministic_thesis),
len(rewritten), len(rewritten),
) )
# Log success to agent_performance_log
if pool is not None and resolved is not None:
await _log_thesis_performance(
pool,
resolved=resolved,
ticker=summary.entity_id,
success=True,
duration_ms=duration_ms,
input_tokens=len(deterministic_thesis) // 4,
output_tokens=len(rewritten) // 4,
)
return rewritten return rewritten
logger.warning( logger.warning(
"LLM thesis rewrite returned empty for %s — using deterministic thesis", "LLM thesis rewrite returned empty for %s — using deterministic thesis",
summary.entity_id, summary.entity_id,
) )
# Log failure to agent_performance_log
if pool is not None and resolved is not None:
await _log_thesis_performance(
pool,
resolved=resolved,
ticker=summary.entity_id,
success=False,
duration_ms=duration_ms,
input_tokens=len(deterministic_thesis) // 4,
output_tokens=0,
error_message="empty_response",
)
return deterministic_thesis return deterministic_thesis
except Exception: except Exception:
duration_ms = int((time.monotonic() - start_time) * 1000)
logger.exception( logger.exception(
"LLM thesis rewrite failed for %s — using deterministic thesis", "LLM thesis rewrite failed for %s — using deterministic thesis",
summary.entity_id, summary.entity_id,
) )
if pool is not None and resolved is not None:
await _log_thesis_performance(
pool,
resolved=resolved,
ticker=summary.entity_id,
success=False,
duration_ms=duration_ms,
input_tokens=len(deterministic_thesis) // 4,
output_tokens=0,
error_message="exception",
)
return deterministic_thesis return deterministic_thesis
finally: finally:
if owns_client: if owns_client:
await client.aclose() await client.aclose()
async def _log_thesis_performance(
pool: asyncpg.Pool,
*,
resolved: ResolvedAgentConfig,
ticker: str,
success: bool,
duration_ms: int,
input_tokens: int = 0,
output_tokens: int = 0,
error_message: str | None = None,
) -> None:
"""Insert a performance log entry for the thesis rewriter agent."""
try:
await pool.execute(
"""INSERT INTO agent_performance_log
(agent_id, variant_id, document_id, ticker, success,
duration_ms, confidence, retry_count,
input_tokens, output_tokens, error_message)
VALUES ($1::uuid, $2::uuid, $3, $4, $5, $6, $7, $8, $9, $10, $11)""",
resolved.agent_id,
resolved.variant_id,
None, # no document_id for thesis rewrites
ticker,
success,
duration_ms,
0.0, # no confidence score for rewrites
0,
input_tokens,
output_tokens,
error_message,
)
except Exception:
logger.warning("Failed to log thesis-rewriter performance", exc_info=True)
async def _call_ollama_thesis( async def _call_ollama_thesis(
client: httpx.AsyncClient, client: httpx.AsyncClient,
config: OllamaConfig, config: OllamaConfig,
@@ -154,6 +295,10 @@ async def _call_ollama_thesis(
"stream": False, "stream": False,
} }
# Support context_window override via num_ctx (Requirement 10.4)
if config.context_window > 0:
payload["options"] = {"num_ctx": config.context_window}
resp = await client.post( resp = await client.post(
f"{config.base_url}/api/chat", f"{config.base_url}/api/chat",
json=payload, json=payload,
+1
View File
@@ -796,6 +796,7 @@ async def generate_recommendation(
deterministic_thesis=deterministic_thesis, deterministic_thesis=deterministic_thesis,
summary=summary, summary=summary,
config=ollama_config, config=ollama_config,
pool=pool,
) )
# If the LLM returned the same text as the deterministic thesis, # If the LLM returned the same text as the deterministic thesis,
# treat it as a no-op (fallback was used). # treat it as a no-op (fallback was used).
+549
View File
@@ -0,0 +1,549 @@
"""Unit tests for agent variant API endpoints.
Tests variant CRUD, clone, activate/deactivate, performance queries,
and edge cases (duplicate slug, non-existent resources, validation).
Requirements: 1.3, 1.4, 2.12.6, 3.13.6, 4.14.5, 6.36.5
"""
from __future__ import annotations
import uuid
from datetime import datetime, timezone
from unittest.mock import AsyncMock, MagicMock, patch
import asyncpg
import pytest
from httpx import ASGITransport, AsyncClient
from services.api.app import app
NOW = datetime(2026, 7, 1, 12, 0, 0, tzinfo=timezone.utc)
AGENT_ID = str(uuid.uuid4())
VARIANT_ID = str(uuid.uuid4())
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
class FakeRecord(dict):
"""Mimics asyncpg.Record for testing."""
def items(self):
return super().items()
def _variant_row(
*,
variant_id: str | None = None,
agent_id: str = AGENT_ID,
is_active: bool = False,
variant_name: str = "test-variant",
variant_slug: str = "test-variant",
model_name: str = "qwen3:8b",
**overrides,
) -> FakeRecord:
row = {
"id": variant_id or str(uuid.uuid4()),
"agent_id": agent_id,
"variant_name": variant_name,
"variant_slug": variant_slug,
"description": "",
"model_provider": "ollama",
"model_name": model_name,
"system_prompt": "You are a test agent.",
"user_prompt_template": "Analyze: {text}",
"prompt_version": "v1",
"temperature": 0.1,
"max_tokens": 16384,
"context_window": 4096,
"input_token_limit": 2048,
"token_budget": 10000,
"timeout_seconds": 120,
"max_retries": 2,
"is_active": is_active,
"created_at": NOW,
"updated_at": NOW,
}
row.update(overrides)
return FakeRecord(row)
def _agent_row(agent_id: str = AGENT_ID) -> FakeRecord:
return FakeRecord({
"id": agent_id,
"model_provider": "ollama",
"model_name": "qwen3:8b",
"system_prompt": "Base system prompt",
"user_prompt_template": "Base template: {text}",
"prompt_version": "v1",
"temperature": 0.0,
"max_tokens": 32768,
"timeout_seconds": 120,
"max_retries": 2,
})
def _perf_row() -> FakeRecord:
return FakeRecord({
"total_invocations": 100,
"successes": 90,
"failures": 10,
"avg_duration_ms": 450,
"p95_duration_ms": 900,
"avg_confidence": 0.82,
"avg_retries": 0.3,
"total_input_tokens": 50000,
"total_output_tokens": 25000,
})
def _perf_history_row(hour_offset: int = 0) -> FakeRecord:
from datetime import timedelta
return FakeRecord({
"hour": NOW - timedelta(hours=hour_offset),
"invocations": 10,
"successes": 9,
"avg_duration_ms": 400,
"avg_confidence": 0.85,
})
# ---------------------------------------------------------------------------
# Task 7.1.1 — CRUD, clone, activate/deactivate, performance
# ---------------------------------------------------------------------------
class TestCreateVariant:
"""POST /api/agents/{agent_id}/variants"""
@pytest.mark.asyncio
async def test_create_variant_returns_201(self):
created = _variant_row()
mock_pool = AsyncMock()
mock_pool.fetchrow = AsyncMock(return_value=created)
with patch("services.api.app.pool", mock_pool):
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
resp = await client.post(
f"/api/agents/{AGENT_ID}/variants",
json={
"variant_name": "test-variant",
"model_name": "qwen3:8b",
"context_window": 4096,
"input_token_limit": 2048,
"token_budget": 10000,
},
)
assert resp.status_code == 201
data = resp.json()
assert data["variant_name"] == "test-variant"
assert data["model_name"] == "qwen3:8b"
assert data["context_window"] == 4096
assert data["input_token_limit"] == 2048
assert data["token_budget"] == 10000
assert data["is_active"] is False
class TestCloneAgentAsVariant:
"""POST /api/agents/{agent_id}/clone"""
@pytest.mark.asyncio
async def test_clone_agent_returns_201(self):
agent = _agent_row()
created = _variant_row(variant_name="cloned-from-agent", variant_slug="cloned-from-agent")
mock_pool = AsyncMock()
mock_pool.fetchrow = AsyncMock(side_effect=[agent, created])
with patch("services.api.app.pool", mock_pool):
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
resp = await client.post(
f"/api/agents/{AGENT_ID}/clone",
json={"variant_name": "cloned-from-agent"},
)
assert resp.status_code == 201
data = resp.json()
assert data["variant_name"] == "cloned-from-agent"
assert data["agent_id"] == AGENT_ID
class TestCloneVariant:
"""POST /api/agents/{agent_id}/variants/{variant_id}/clone"""
@pytest.mark.asyncio
async def test_clone_variant_returns_201(self):
source = _variant_row(variant_id=VARIANT_ID)
cloned = _variant_row(variant_name="cloned-v2", variant_slug="cloned-v2")
mock_pool = AsyncMock()
mock_pool.fetchrow = AsyncMock(side_effect=[source, cloned])
with patch("services.api.app.pool", mock_pool):
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
resp = await client.post(
f"/api/agents/{AGENT_ID}/variants/{VARIANT_ID}/clone",
json={"variant_name": "cloned-v2"},
)
assert resp.status_code == 201
data = resp.json()
assert data["variant_name"] == "cloned-v2"
class TestListVariants:
"""GET /api/agents/{agent_id}/variants"""
@pytest.mark.asyncio
async def test_list_variants_returns_list(self):
rows = [
_variant_row(variant_name="v1", variant_slug="v1"),
_variant_row(variant_name="v2", variant_slug="v2", is_active=True),
]
mock_pool = AsyncMock()
mock_pool.fetch = AsyncMock(return_value=rows)
with patch("services.api.app.pool", mock_pool):
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
resp = await client.get(f"/api/agents/{AGENT_ID}/variants")
assert resp.status_code == 200
data = resp.json()
assert len(data) == 2
assert data[0]["variant_name"] == "v1"
assert data[1]["is_active"] is True
class TestGetVariant:
"""GET /api/agents/{agent_id}/variants/{variant_id}"""
@pytest.mark.asyncio
async def test_get_variant_returns_variant(self):
row = _variant_row(variant_id=VARIANT_ID)
mock_pool = AsyncMock()
mock_pool.fetchrow = AsyncMock(return_value=row)
with patch("services.api.app.pool", mock_pool):
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
resp = await client.get(f"/api/agents/{AGENT_ID}/variants/{VARIANT_ID}")
assert resp.status_code == 200
data = resp.json()
assert data["id"] == VARIANT_ID
class TestUpdateVariant:
"""PUT /api/agents/{agent_id}/variants/{variant_id}"""
@pytest.mark.asyncio
async def test_update_variant_returns_updated(self):
updated = _variant_row(variant_id=VARIANT_ID, model_name="llama3.1:8b", temperature=0.5)
mock_pool = AsyncMock()
mock_pool.fetchrow = AsyncMock(return_value=updated)
with patch("services.api.app.pool", mock_pool):
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
resp = await client.put(
f"/api/agents/{AGENT_ID}/variants/{VARIANT_ID}",
json={"model_name": "llama3.1:8b", "temperature": 0.5},
)
assert resp.status_code == 200
data = resp.json()
assert data["model_name"] == "llama3.1:8b"
assert data["temperature"] == 0.5
class TestDeleteVariant:
"""DELETE /api/agents/{agent_id}/variants/{variant_id}"""
@pytest.mark.asyncio
async def test_delete_inactive_variant_succeeds(self):
mock_pool = AsyncMock()
mock_pool.fetchrow = AsyncMock(return_value=FakeRecord({"is_active": False}))
mock_pool.execute = AsyncMock()
with patch("services.api.app.pool", mock_pool):
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
resp = await client.delete(f"/api/agents/{AGENT_ID}/variants/{VARIANT_ID}")
assert resp.status_code == 200
data = resp.json()
assert data["deleted"] is True
@pytest.mark.asyncio
async def test_delete_active_variant_returns_400(self):
mock_pool = AsyncMock()
mock_pool.fetchrow = AsyncMock(return_value=FakeRecord({"is_active": True}))
with patch("services.api.app.pool", mock_pool):
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
resp = await client.delete(f"/api/agents/{AGENT_ID}/variants/{VARIANT_ID}")
assert resp.status_code == 400
assert "active" in resp.json()["detail"].lower()
class TestActivateDeactivate:
"""POST .../activate and .../deactivate"""
@pytest.mark.asyncio
async def test_activate_variant(self):
activated = _variant_row(variant_id=VARIANT_ID, is_active=True)
mock_conn = AsyncMock()
mock_conn.execute = AsyncMock()
mock_conn.fetchrow = AsyncMock(return_value=activated)
# transaction context manager
mock_tx = AsyncMock()
mock_tx.__aenter__ = AsyncMock(return_value=None)
mock_tx.__aexit__ = AsyncMock(return_value=False)
mock_conn.transaction = MagicMock(return_value=mock_tx)
mock_pool = AsyncMock()
mock_acquire = AsyncMock()
mock_acquire.__aenter__ = AsyncMock(return_value=mock_conn)
mock_acquire.__aexit__ = AsyncMock(return_value=False)
mock_pool.acquire = MagicMock(return_value=mock_acquire)
with patch("services.api.app.pool", mock_pool):
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
resp = await client.post(
f"/api/agents/{AGENT_ID}/variants/{VARIANT_ID}/activate"
)
assert resp.status_code == 200
data = resp.json()
assert data["is_active"] is True
@pytest.mark.asyncio
async def test_deactivate_variants(self):
mock_pool = AsyncMock()
mock_pool.execute = AsyncMock()
with patch("services.api.app.pool", mock_pool):
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
resp = await client.post(f"/api/agents/{AGENT_ID}/variants/deactivate")
assert resp.status_code == 200
data = resp.json()
assert data["deactivated"] is True
class TestVariantPerformance:
"""GET .../performance and .../performance/history"""
@pytest.mark.asyncio
async def test_get_variant_performance(self):
perf = _perf_row()
mock_pool = AsyncMock()
mock_pool.fetchrow = AsyncMock(return_value=perf)
with patch("services.api.app.pool", mock_pool):
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
resp = await client.get(
f"/api/agents/{AGENT_ID}/variants/{VARIANT_ID}/performance?hours=24"
)
assert resp.status_code == 200
data = resp.json()
assert data["total_invocations"] == 100
assert data["successes"] == 90
assert data["success_rate"] == 0.9
@pytest.mark.asyncio
async def test_get_variant_performance_history(self):
rows = [_perf_history_row(0), _perf_history_row(1)]
mock_pool = AsyncMock()
mock_pool.fetch = AsyncMock(return_value=rows)
with patch("services.api.app.pool", mock_pool):
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
resp = await client.get(
f"/api/agents/{AGENT_ID}/variants/{VARIANT_ID}/performance/history?hours=24"
)
assert resp.status_code == 200
data = resp.json()
assert len(data) == 2
assert data[0]["invocations"] == 10
# ---------------------------------------------------------------------------
# Task 7.1.2 — Edge-case tests
# ---------------------------------------------------------------------------
class TestEdgeCases:
"""Edge-case tests: duplicate slug, non-existent resources, validation."""
@pytest.mark.asyncio
async def test_duplicate_slug_returns_409(self):
"""Creating a variant with a duplicate slug returns 409."""
mock_pool = AsyncMock()
mock_pool.fetchrow = AsyncMock(
side_effect=asyncpg.UniqueViolationError("")
)
with patch("services.api.app.pool", mock_pool):
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
resp = await client.post(
f"/api/agents/{AGENT_ID}/variants",
json={"variant_name": "dup", "model_name": "qwen3:8b"},
)
assert resp.status_code == 409
assert "already exists" in resp.json()["detail"]
@pytest.mark.asyncio
async def test_clone_nonexistent_agent_returns_404(self):
"""Cloning from a non-existent agent returns 404."""
mock_pool = AsyncMock()
mock_pool.fetchrow = AsyncMock(return_value=None)
with patch("services.api.app.pool", mock_pool):
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
resp = await client.post(
f"/api/agents/{str(uuid.uuid4())}/clone",
json={"variant_name": "test"},
)
assert resp.status_code == 404
assert "not found" in resp.json()["detail"].lower()
@pytest.mark.asyncio
async def test_get_nonexistent_variant_returns_404(self):
"""Getting a non-existent variant returns 404."""
mock_pool = AsyncMock()
mock_pool.fetchrow = AsyncMock(return_value=None)
with patch("services.api.app.pool", mock_pool):
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
resp = await client.get(
f"/api/agents/{AGENT_ID}/variants/{str(uuid.uuid4())}"
)
assert resp.status_code == 404
@pytest.mark.asyncio
async def test_delete_nonexistent_variant_returns_404(self):
"""Deleting a non-existent variant returns 404."""
mock_pool = AsyncMock()
mock_pool.fetchrow = AsyncMock(return_value=None)
with patch("services.api.app.pool", mock_pool):
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
resp = await client.delete(
f"/api/agents/{AGENT_ID}/variants/{str(uuid.uuid4())}"
)
assert resp.status_code == 404
@pytest.mark.asyncio
async def test_create_variant_empty_model_name_rejected(self):
"""Creating a variant with empty model_name is rejected by Pydantic."""
mock_pool = AsyncMock()
with patch("services.api.app.pool", mock_pool):
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
resp = await client.post(
f"/api/agents/{AGENT_ID}/variants",
json={"variant_name": "test"},
# model_name is required — omitting it should fail
)
assert resp.status_code == 422
@pytest.mark.asyncio
async def test_update_variant_no_fields_returns_400(self):
"""Updating a variant with no fields returns 400."""
mock_pool = AsyncMock()
with patch("services.api.app.pool", mock_pool):
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
resp = await client.put(
f"/api/agents/{AGENT_ID}/variants/{VARIANT_ID}",
json={},
)
assert resp.status_code == 400
@pytest.mark.asyncio
async def test_clone_nonexistent_variant_returns_404(self):
"""Cloning from a non-existent variant returns 404."""
mock_pool = AsyncMock()
mock_pool.fetchrow = AsyncMock(return_value=None)
with patch("services.api.app.pool", mock_pool):
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
resp = await client.post(
f"/api/agents/{AGENT_ID}/variants/{str(uuid.uuid4())}/clone",
json={"variant_name": "test"},
)
assert resp.status_code == 404
@pytest.mark.asyncio
async def test_activate_nonexistent_variant_returns_404(self):
"""Activating a non-existent variant returns 404."""
mock_conn = AsyncMock()
mock_conn.execute = AsyncMock()
mock_conn.fetchrow = AsyncMock(return_value=None)
mock_tx = AsyncMock()
mock_tx.__aenter__ = AsyncMock(return_value=None)
mock_tx.__aexit__ = AsyncMock(return_value=False)
mock_conn.transaction = MagicMock(return_value=mock_tx)
mock_pool = AsyncMock()
mock_acquire = AsyncMock()
mock_acquire.__aenter__ = AsyncMock(return_value=mock_conn)
mock_acquire.__aexit__ = AsyncMock(return_value=False)
mock_pool.acquire = MagicMock(return_value=mock_acquire)
with patch("services.api.app.pool", mock_pool):
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
resp = await client.post(
f"/api/agents/{AGENT_ID}/variants/{str(uuid.uuid4())}/activate"
)
assert resp.status_code == 404
@pytest.mark.asyncio
async def test_duplicate_slug_on_clone_returns_409(self):
"""Cloning an agent with a duplicate slug returns 409."""
agent = _agent_row()
mock_pool = AsyncMock()
mock_pool.fetchrow = AsyncMock(
side_effect=[agent, asyncpg.UniqueViolationError("")]
)
with patch("services.api.app.pool", mock_pool):
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
resp = await client.post(
f"/api/agents/{AGENT_ID}/clone",
json={"variant_name": "dup", "variant_slug": "existing-slug"},
)
assert resp.status_code == 409
+659
View File
@@ -0,0 +1,659 @@
"""Property-based tests for agent variant logic.
Feature: agent-variants
Uses Hypothesis to validate correctness properties of variant operations:
single-active invariant, clone field preservation, config resolution,
slug determinism, and partial update idempotence.
Requirements: 1.4, 2.1, 2.3, 3.4, 4.1, 4.3, 4.4, 7
Design: Correctness Properties 15, 7
"""
from __future__ import annotations
import copy
import re
import uuid
from datetime import datetime, timezone
from typing import Any
import pytest
from hypothesis import given, settings, assume
from hypothesis import strategies as st
from services.api.app import _slugify
from services.shared.agent_config import ResolvedAgentConfig
# ---------------------------------------------------------------------------
# Hypothesis strategies
# ---------------------------------------------------------------------------
# Config fields that can be overridden in a variant
_CONFIG_FIELDS = [
"model_provider",
"model_name",
"system_prompt",
"user_prompt_template",
"prompt_version",
"temperature",
"max_tokens",
"context_window",
"input_token_limit",
"token_budget",
"timeout_seconds",
"max_retries",
]
_STR_FIELDS = [
"model_provider",
"model_name",
"system_prompt",
"user_prompt_template",
"prompt_version",
]
_FLOAT_FIELDS = ["temperature"]
_INT_FIELDS = [
"max_tokens",
"context_window",
"input_token_limit",
"token_budget",
"timeout_seconds",
"max_retries",
]
def _config_value_strategy(field: str) -> st.SearchStrategy:
"""Generate a valid value for a given config field."""
if field in _STR_FIELDS:
return st.text(min_size=1, max_size=50, alphabet=st.characters(
whitelist_categories=("L", "N", "P", "Z"),
))
elif field in _FLOAT_FIELDS:
return st.floats(min_value=0.0, max_value=2.0, allow_nan=False)
elif field in _INT_FIELDS:
return st.integers(min_value=0, max_value=100000)
return st.text(min_size=1, max_size=20)
def _agent_config_strategy() -> st.SearchStrategy[dict[str, Any]]:
"""Generate a random agent configuration dict."""
return st.fixed_dictionaries({
"model_provider": st.sampled_from(["ollama", "openai", "anthropic"]),
"model_name": st.text(min_size=1, max_size=30, alphabet=st.characters(
whitelist_categories=("L", "N"),
)),
"system_prompt": st.text(min_size=0, max_size=100),
"user_prompt_template": st.text(min_size=0, max_size=100),
"prompt_version": st.text(min_size=0, max_size=20),
"temperature": st.floats(min_value=0.0, max_value=2.0, allow_nan=False),
"max_tokens": st.integers(min_value=1, max_value=100000),
"context_window": st.integers(min_value=0, max_value=200000),
"input_token_limit": st.integers(min_value=0, max_value=200000),
"token_budget": st.integers(min_value=0, max_value=1000000),
"timeout_seconds": st.integers(min_value=1, max_value=600),
"max_retries": st.integers(min_value=0, max_value=10),
})
def _variant_name_strategy() -> st.SearchStrategy[str]:
"""Generate random variant names with diverse characters."""
return st.text(
min_size=1,
max_size=50,
alphabet=st.characters(whitelist_categories=("L", "N", "P", "Z")),
)
def _override_subset_strategy(
source_config: dict[str, Any],
) -> st.SearchStrategy[dict[str, Any]]:
"""Generate a random subset of config field overrides."""
# We build this as a composite strategy
return st.fixed_dictionaries(
{},
optional={
field: _config_value_strategy(field)
for field in _CONFIG_FIELDS
},
)
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _simulate_clone(
source: dict[str, Any],
overrides: dict[str, Any],
) -> dict[str, Any]:
"""Simulate the clone logic from the API: source fields + overrides.
Mirrors the clone endpoint: for each config field, if an override is
provided (not None), use it; otherwise use the source value.
"""
result = {}
for field in _CONFIG_FIELDS:
if field in overrides and overrides[field] is not None:
result[field] = overrides[field]
else:
result[field] = source[field]
return result
def _simulate_activate_deactivate(
variants: list[dict[str, Any]],
operations: list[tuple[str, int]],
) -> list[dict[str, Any]]:
"""Simulate a sequence of activate/deactivate operations.
operations: list of ("activate", variant_index) or ("deactivate", -1)
Returns the final state of variants.
"""
for op, idx in operations:
if op == "activate" and 0 <= idx < len(variants):
# Deactivate all first
for v in variants:
v["is_active"] = False
# Activate the target
variants[idx]["is_active"] = True
elif op == "deactivate":
# Deactivate all
for v in variants:
v["is_active"] = False
return variants
# ---------------------------------------------------------------------------
# Property 1: Single active variant invariant
# ---------------------------------------------------------------------------
class TestProperty1SingleActiveVariantInvariant:
"""Feature: agent-variants, Property 1: Single active variant invariant
For any sequence of activate/deactivate operations on variants of an
agent, at most one variant per agent has is_active = TRUE at any point.
**Validates: Requirements 1.4, 4.1**
"""
@given(
num_variants=st.integers(min_value=1, max_value=10),
operations=st.lists(
st.tuples(
st.sampled_from(["activate", "deactivate"]),
st.integers(min_value=-1, max_value=9),
),
min_size=1,
max_size=30,
),
)
@settings(max_examples=100)
def test_at_most_one_active_after_each_operation(
self,
num_variants: int,
operations: list[tuple[str, int]],
):
"""**Validates: Requirements 1.4, 4.1**
After each activate/deactivate operation, count of active variants
must be 0 or 1.
"""
agent_id = str(uuid.uuid4())
variants = [
{
"id": str(uuid.uuid4()),
"agent_id": agent_id,
"is_active": False,
}
for _ in range(num_variants)
]
for op, idx in operations:
if op == "activate" and 0 <= idx < num_variants:
# Simulate transactional activate: deactivate all, then activate target
for v in variants:
v["is_active"] = False
variants[idx]["is_active"] = True
elif op == "deactivate":
for v in variants:
v["is_active"] = False
# Invariant check after each operation
active_count = sum(1 for v in variants if v["is_active"])
assert active_count <= 1, (
f"Invariant violated: {active_count} active variants after "
f"operation ({op}, {idx})"
)
@given(
num_variants=st.integers(min_value=2, max_value=8),
activate_sequence=st.lists(
st.integers(min_value=0, max_value=7),
min_size=2,
max_size=20,
),
)
@settings(max_examples=100)
def test_rapid_activate_swaps_maintain_invariant(
self,
num_variants: int,
activate_sequence: list[int],
):
"""**Validates: Requirements 1.4, 4.1**
Rapidly activating different variants in sequence still maintains
at most one active.
"""
variants = [
{"id": str(uuid.uuid4()), "is_active": False}
for _ in range(num_variants)
]
for idx in activate_sequence:
target = idx % num_variants
# Transactional swap
for v in variants:
v["is_active"] = False
variants[target]["is_active"] = True
active_count = sum(1 for v in variants if v["is_active"])
assert active_count == 1
assert variants[target]["is_active"] is True
# ---------------------------------------------------------------------------
# Property 2: Clone preserves unoverridden fields
# ---------------------------------------------------------------------------
class TestProperty2ClonePreservesUnoverriddenFields:
"""Feature: agent-variants, Property 2: Clone preserves unoverridden fields
For any agent config and any subset of override fields, cloning produces
a variant where overridden fields match the override values and
non-overridden fields match the source.
**Validates: Requirements 2.1, 2.3**
"""
@given(
source_config=_agent_config_strategy(),
overrides=st.fixed_dictionaries(
{},
optional={
field: _config_value_strategy(field)
for field in _CONFIG_FIELDS
},
),
)
@settings(max_examples=100)
def test_overridden_fields_match_overrides(
self,
source_config: dict[str, Any],
overrides: dict[str, Any],
):
"""**Validates: Requirements 2.1, 2.3**
Fields present in overrides must have the override value in the clone.
"""
result = _simulate_clone(source_config, overrides)
for field in _CONFIG_FIELDS:
if field in overrides:
assert result[field] == overrides[field], (
f"Override field {field}: expected {overrides[field]}, "
f"got {result[field]}"
)
@given(
source_config=_agent_config_strategy(),
overrides=st.fixed_dictionaries(
{},
optional={
field: _config_value_strategy(field)
for field in _CONFIG_FIELDS
},
),
)
@settings(max_examples=100)
def test_non_overridden_fields_match_source(
self,
source_config: dict[str, Any],
overrides: dict[str, Any],
):
"""**Validates: Requirements 2.1, 2.3**
Fields NOT present in overrides must match the source config.
"""
result = _simulate_clone(source_config, overrides)
for field in _CONFIG_FIELDS:
if field not in overrides:
assert result[field] == source_config[field], (
f"Non-overridden field {field}: expected {source_config[field]}, "
f"got {result[field]}"
)
@given(source_config=_agent_config_strategy())
@settings(max_examples=100)
def test_clone_with_no_overrides_is_exact_copy(
self,
source_config: dict[str, Any],
):
"""**Validates: Requirements 2.1, 2.3**
Cloning with no overrides produces an exact copy of all config fields.
"""
result = _simulate_clone(source_config, {})
for field in _CONFIG_FIELDS:
assert result[field] == source_config[field], (
f"Field {field} differs: {result[field]} != {source_config[field]}"
)
# ---------------------------------------------------------------------------
# Property 3: Config resolution prefers active variant
# ---------------------------------------------------------------------------
class TestProperty3ConfigResolutionPrefersActiveVariant:
"""Feature: agent-variants, Property 3: Config resolution prefers active variant
For any agent with N variants, config resolution returns the active
variant's config when one exists, and the base agent config when none
is active.
**Validates: Requirements 4.3, 4.4**
"""
@given(
agent_config=_agent_config_strategy(),
variant_configs=st.lists(
_agent_config_strategy(),
min_size=1,
max_size=5,
),
active_index=st.integers(min_value=0, max_value=4),
)
@settings(max_examples=100)
def test_active_variant_config_is_returned(
self,
agent_config: dict[str, Any],
variant_configs: list[dict[str, Any]],
active_index: int,
):
"""**Validates: Requirements 4.3, 4.4**
When an active variant exists, resolved config fields must match
the active variant's values.
"""
active_idx = active_index % len(variant_configs)
active_variant = variant_configs[active_idx]
# Simulate COALESCE resolution: variant fields preferred over agent
resolved = {}
for field in _CONFIG_FIELDS:
# COALESCE(variant.field, agent.field) — variant always wins
# when it has a value (which it always does in our model)
resolved[field] = active_variant[field]
for field in _CONFIG_FIELDS:
assert resolved[field] == active_variant[field], (
f"Field {field}: expected variant value {active_variant[field]}, "
f"got {resolved[field]}"
)
@given(agent_config=_agent_config_strategy())
@settings(max_examples=100)
def test_no_active_variant_returns_agent_config(
self,
agent_config: dict[str, Any],
):
"""**Validates: Requirements 4.3, 4.4**
When no active variant exists, resolved config fields must match
the base agent's values.
"""
# Simulate COALESCE with NULL variant: agent fields used
resolved = {}
for field in _CONFIG_FIELDS:
resolved[field] = agent_config[field]
for field in _CONFIG_FIELDS:
assert resolved[field] == agent_config[field]
@given(
agent_config=_agent_config_strategy(),
variant_config=_agent_config_strategy(),
has_active=st.booleans(),
)
@settings(max_examples=100)
def test_resolution_source_matches_active_state(
self,
agent_config: dict[str, Any],
variant_config: dict[str, Any],
has_active: bool,
):
"""**Validates: Requirements 4.3, 4.4**
The resolver returns the correct source (variant or agent) based
on whether an active variant exists.
"""
if has_active:
source = variant_config
variant_id = str(uuid.uuid4())
else:
source = agent_config
variant_id = None
# Build a ResolvedAgentConfig to verify the dataclass works
config = ResolvedAgentConfig(
agent_id=str(uuid.uuid4()),
variant_id=variant_id,
model_provider=source["model_provider"],
model_name=source["model_name"],
system_prompt=source["system_prompt"],
user_prompt_template=source["user_prompt_template"],
prompt_version=source["prompt_version"],
temperature=source["temperature"],
max_tokens=source["max_tokens"],
context_window=source["context_window"],
input_token_limit=source["input_token_limit"],
token_budget=source["token_budget"],
timeout_seconds=source["timeout_seconds"],
max_retries=source["max_retries"],
)
assert config.model_provider == source["model_provider"]
assert config.model_name == source["model_name"]
assert config.temperature == source["temperature"]
assert config.max_tokens == source["max_tokens"]
if has_active:
assert config.variant_id is not None
else:
assert config.variant_id is None
# ---------------------------------------------------------------------------
# Property 4: Slug auto-generation determinism
# ---------------------------------------------------------------------------
_KEBAB_CASE_RE = re.compile(r"^[a-z0-9]+(-[a-z0-9]+)*$")
class TestProperty4SlugAutoGenerationDeterminism:
"""Feature: agent-variants, Property 4: Slug auto-generation determinism
For any variant_name, the auto-generated slug is deterministic,
produces valid kebab-case, and is non-empty for non-empty input
containing at least one alphanumeric character.
**Validates: Requirements 2.4**
"""
@given(name=_variant_name_strategy())
@settings(max_examples=100)
def test_slugify_is_deterministic(self, name: str):
"""**Validates: Requirements 2.4**
Calling _slugify twice with the same name produces the same slug.
"""
slug1 = _slugify(name)
slug2 = _slugify(name)
assert slug1 == slug2, (
f"Non-deterministic: _slugify({name!r}) produced {slug1!r} and {slug2!r}"
)
@given(name=st.from_regex(r"[a-zA-Z0-9][\w\s\-]{0,49}", fullmatch=True))
@settings(max_examples=100)
def test_slugify_produces_valid_kebab_case(self, name: str):
"""**Validates: Requirements 2.4**
The slug must be lowercase alphanumeric with hyphens, no leading
or trailing hyphens.
"""
slug = _slugify(name)
assume(len(slug) > 0)
# No leading or trailing hyphens
assert not slug.startswith("-"), f"Slug starts with hyphen: {slug!r}"
assert not slug.endswith("-"), f"Slug ends with hyphen: {slug!r}"
# Only lowercase alphanumeric and hyphens
assert _KEBAB_CASE_RE.match(slug), (
f"Slug {slug!r} is not valid kebab-case (from name {name!r})"
)
@given(name=st.from_regex(r"[a-zA-Z0-9][\w\s]{0,49}", fullmatch=True))
@settings(max_examples=100)
def test_slugify_non_empty_for_alphanumeric_input(self, name: str):
"""**Validates: Requirements 2.4**
For any name containing at least one alphanumeric character,
the slug is non-empty.
"""
slug = _slugify(name)
assert len(slug) > 0, (
f"Empty slug for name {name!r}"
)
@given(name=_variant_name_strategy())
@settings(max_examples=100)
def test_slugify_is_lowercase(self, name: str):
"""**Validates: Requirements 2.4**
The slug must be entirely lowercase.
"""
slug = _slugify(name)
assert slug == slug.lower(), (
f"Slug {slug!r} contains uppercase characters"
)
# ---------------------------------------------------------------------------
# Property 5: Partial update idempotence
# ---------------------------------------------------------------------------
class TestProperty5PartialUpdateIdempotence:
"""Feature: agent-variants, Property 5: Partial update idempotence
For any variant, applying a partial update twice produces the same
variant state (excluding updated_at).
**Validates: Requirements 3.4**
"""
@given(
base_config=_agent_config_strategy(),
update_fields=st.fixed_dictionaries(
{},
optional={
field: _config_value_strategy(field)
for field in _CONFIG_FIELDS
},
),
)
@settings(max_examples=100)
def test_double_apply_produces_same_state(
self,
base_config: dict[str, Any],
update_fields: dict[str, Any],
):
"""**Validates: Requirements 3.4**
Applying the same partial update twice yields identical field values
(excluding updated_at).
"""
assume(len(update_fields) > 0)
# First application
state_after_first = copy.deepcopy(base_config)
for field, value in update_fields.items():
state_after_first[field] = value
# Second application (same update on the result of the first)
state_after_second = copy.deepcopy(state_after_first)
for field, value in update_fields.items():
state_after_second[field] = value
# All config fields must match
for field in _CONFIG_FIELDS:
assert state_after_first[field] == state_after_second[field], (
f"Field {field} differs after double apply: "
f"{state_after_first[field]} != {state_after_second[field]}"
)
@given(
base_config=_agent_config_strategy(),
update_fields=st.fixed_dictionaries(
{},
optional={
field: _config_value_strategy(field)
for field in _CONFIG_FIELDS
},
),
)
@settings(max_examples=100)
def test_unchanged_fields_preserved_after_partial_update(
self,
base_config: dict[str, Any],
update_fields: dict[str, Any],
):
"""**Validates: Requirements 3.4**
Fields not included in the update must retain their original values.
"""
updated = copy.deepcopy(base_config)
for field, value in update_fields.items():
updated[field] = value
for field in _CONFIG_FIELDS:
if field not in update_fields:
assert updated[field] == base_config[field], (
f"Unchanged field {field} was modified: "
f"{base_config[field]} -> {updated[field]}"
)
@given(base_config=_agent_config_strategy())
@settings(max_examples=100)
def test_empty_update_is_noop(
self,
base_config: dict[str, Any],
):
"""**Validates: Requirements 3.4**
An empty update (no fields) leaves all config fields unchanged.
"""
updated = copy.deepcopy(base_config)
# Apply empty update — no fields changed
for field in _CONFIG_FIELDS:
assert updated[field] == base_config[field]