feat: trading feedback engine — periodic performance reports with AI summarization
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
- Migration 038: trading_reports table + report-summarizer agent seed
- 6 reporting modules: models, collector, sections, validator, summarizer, generator
- API endpoints: GET /api/reports (paginated, filterable), GET /api/reports/{id}
- Frontend hooks: useReports, useReport with TanStack Query
- Scheduler: daily (after 16:30 ET) and weekly (Saturday) report triggers
- Redis queue consumer for async report generation with retry/dedup
- 5 property-based tests (chunking, serialization, validation, accuracy, deltas)
- 109 unit/integration tests across all modules
- 6 frontend hook tests with MSW mocks
This commit is contained in:
@@ -0,0 +1,127 @@
|
||||
# Feature: trading-feedback-engine, Property 3: Validation discrepancy detection correctness
|
||||
"""Property-based tests for report validation discrepancy detection.
|
||||
|
||||
Feature: trading-feedback-engine
|
||||
|
||||
Tests the validation discrepancy detection correctness property from the
|
||||
design specification: for any pair of computed metric value and snapshot
|
||||
metric value (both finite, non-negative floats), the validation function
|
||||
SHALL produce a warning if and only if the percentage difference exceeds 5%.
|
||||
The percentage difference SHALL be computed as |computed - snapshot| /
|
||||
snapshot * 100 when snapshot > 0, and SHALL flag any non-zero computed value
|
||||
when snapshot is 0.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
|
||||
from hypothesis import given, settings
|
||||
from hypothesis import strategies as st
|
||||
|
||||
from services.reporting.validator import (
|
||||
DISCREPANCY_THRESHOLD_PCT,
|
||||
_check_discrepancy,
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Property 3: Validation Discrepancy Detection Correctness
|
||||
# Validates: Requirements 4.1, 4.2, 4.3, 4.4
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Strategy: finite, non-negative floats in [0, 1e6]
|
||||
_metric_float = st.floats(
|
||||
min_value=0, max_value=1e6, allow_nan=False, allow_infinity=False,
|
||||
)
|
||||
|
||||
|
||||
@given(computed=_metric_float, snapshot=_metric_float)
|
||||
@settings(max_examples=100)
|
||||
def test_discrepancy_detection_correctness(
|
||||
computed: float,
|
||||
snapshot: float,
|
||||
) -> None:
|
||||
"""**Validates: Requirements 4.1, 4.2, 4.3, 4.4**
|
||||
|
||||
For any pair of computed and snapshot values (finite, non-negative):
|
||||
- Both zero → no warning
|
||||
- Snapshot zero, computed non-zero → warning (100% discrepancy)
|
||||
- Snapshot > 0 → warning iff |computed - snapshot| / snapshot * 100 > 5%
|
||||
"""
|
||||
result = _check_discrepancy("test_field", computed, snapshot)
|
||||
|
||||
if snapshot == 0.0 and computed == 0.0:
|
||||
# Both zero → no discrepancy
|
||||
assert result is None, (
|
||||
f"Expected no warning when both values are 0, got {result}"
|
||||
)
|
||||
elif snapshot == 0.0:
|
||||
# Non-zero computed with zero snapshot → always a warning
|
||||
assert result is not None, (
|
||||
f"Expected warning for non-zero computed={computed} with "
|
||||
f"snapshot=0, got None"
|
||||
)
|
||||
assert result.pct_difference == 100.0, (
|
||||
f"Expected 100% discrepancy for zero snapshot, "
|
||||
f"got {result.pct_difference}%"
|
||||
)
|
||||
else:
|
||||
# Normal case: snapshot > 0
|
||||
expected_pct = abs(computed - snapshot) / snapshot * 100.0
|
||||
if expected_pct > DISCREPANCY_THRESHOLD_PCT:
|
||||
assert result is not None, (
|
||||
f"Expected warning for {expected_pct:.4f}% discrepancy "
|
||||
f"(computed={computed}, snapshot={snapshot}), got None"
|
||||
)
|
||||
# When expected_pct is inf (very small snapshot), both should be inf
|
||||
if math.isinf(expected_pct):
|
||||
assert math.isinf(result.pct_difference), (
|
||||
f"Expected inf pct_difference, got {result.pct_difference}"
|
||||
)
|
||||
else:
|
||||
assert abs(result.pct_difference - round(expected_pct, 4)) < 1e-6, (
|
||||
f"Percentage difference mismatch: "
|
||||
f"expected {round(expected_pct, 4)}, "
|
||||
f"got {result.pct_difference}"
|
||||
)
|
||||
else:
|
||||
assert result is None, (
|
||||
f"Expected no warning for {expected_pct:.4f}% discrepancy "
|
||||
f"(computed={computed}, snapshot={snapshot}), "
|
||||
f"got warning with pct_difference={result.pct_difference}"
|
||||
)
|
||||
|
||||
|
||||
@given(computed=_metric_float, snapshot=_metric_float)
|
||||
@settings(max_examples=100)
|
||||
def test_discrepancy_threshold_is_five_percent(
|
||||
computed: float,
|
||||
snapshot: float,
|
||||
) -> None:
|
||||
"""**Validates: Requirements 4.1, 4.2, 4.3, 4.4**
|
||||
|
||||
Verify that DISCREPANCY_THRESHOLD_PCT = 5.0 is the threshold used:
|
||||
the function produces a warning if and only if the discrepancy
|
||||
exceeds exactly 5%.
|
||||
"""
|
||||
assert DISCREPANCY_THRESHOLD_PCT == 5.0, (
|
||||
f"Expected threshold of 5.0%, got {DISCREPANCY_THRESHOLD_PCT}%"
|
||||
)
|
||||
|
||||
result = _check_discrepancy("threshold_check", computed, snapshot)
|
||||
|
||||
if snapshot == 0.0 and computed == 0.0:
|
||||
assert result is None
|
||||
elif snapshot == 0.0:
|
||||
# 100% > 5% → always warning
|
||||
assert result is not None
|
||||
else:
|
||||
pct = abs(computed - snapshot) / snapshot * 100.0
|
||||
should_warn = pct > 5.0
|
||||
if should_warn:
|
||||
assert result is not None, (
|
||||
f"Discrepancy {pct:.4f}% > 5% but no warning produced"
|
||||
)
|
||||
else:
|
||||
assert result is None, (
|
||||
f"Discrepancy {pct:.4f}% <= 5% but warning produced"
|
||||
)
|
||||
Reference in New Issue
Block a user