feat: trading feedback engine — periodic performance reports with AI summarization
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
- Migration 038: trading_reports table + report-summarizer agent seed
- 6 reporting modules: models, collector, sections, validator, summarizer, generator
- API endpoints: GET /api/reports (paginated, filterable), GET /api/reports/{id}
- Frontend hooks: useReports, useReport with TanStack Query
- Scheduler: daily (after 16:30 ET) and weekly (Saturday) report triggers
- Redis queue consumer for async report generation with retry/dedup
- 5 property-based tests (chunking, serialization, validation, accuracy, deltas)
- 109 unit/integration tests across all modules
- 6 frontend hook tests with MSW mocks
This commit is contained in:
@@ -0,0 +1,245 @@
|
||||
# Feature: trading-feedback-engine, Property 2: Report serialization round-trip
|
||||
"""Property-based tests for report serialization round-trip.
|
||||
|
||||
Feature: trading-feedback-engine
|
||||
|
||||
Tests the report serialization round-trip property from the design
|
||||
specification: for any valid ReportData object (with valid P&L,
|
||||
recommendation accuracy, position performance, risk metrics, and model
|
||||
quality sections), serializing to JSON and then deserializing back SHALL
|
||||
produce a ReportData object equivalent to the original. All datetime fields
|
||||
in the serialized JSON SHALL be in ISO 8601 format.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
from datetime import date, datetime, timezone
|
||||
|
||||
from hypothesis import given, settings
|
||||
from hypothesis import strategies as st
|
||||
|
||||
from services.reporting.models import (
|
||||
ModelQualitySection,
|
||||
ModelQualityWindow,
|
||||
PLSection,
|
||||
PositionDetail,
|
||||
PositionPerformanceSection,
|
||||
RecommendationAccuracySection,
|
||||
ReportData,
|
||||
ReportType,
|
||||
RiskMetricsSection,
|
||||
ValidationStatus,
|
||||
ValidationWarning,
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Property 2: Report Serialization Round-Trip
|
||||
# Validates: Requirements 8.1, 8.2, 8.3, 8.4
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# ISO 8601 datetime pattern (covers both datetime and date formats)
|
||||
_ISO8601_DATETIME_RE = re.compile(
|
||||
r"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}" # YYYY-MM-DDTHH:MM:SS
|
||||
r"(?:\.\d+)?" # optional fractional seconds
|
||||
r"(?:Z|[+-]\d{2}:\d{2})?$" # optional timezone
|
||||
)
|
||||
_ISO8601_DATE_RE = re.compile(r"^\d{4}-\d{2}-\d{2}$")
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Hypothesis strategies for each model
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_finite_float = st.floats(allow_nan=False, allow_infinity=False)
|
||||
_non_negative_finite_float = st.floats(
|
||||
min_value=0.0, allow_nan=False, allow_infinity=False,
|
||||
)
|
||||
_rate_float = st.floats(
|
||||
min_value=0.0, max_value=1.0, allow_nan=False, allow_infinity=False,
|
||||
)
|
||||
_optional_finite_float = st.one_of(st.none(), _finite_float)
|
||||
|
||||
_validation_warning_strategy = st.builds(
|
||||
ValidationWarning,
|
||||
field_name=st.text(min_size=1, max_size=50),
|
||||
computed_value=_finite_float,
|
||||
snapshot_value=_finite_float,
|
||||
pct_difference=_non_negative_finite_float,
|
||||
)
|
||||
|
||||
_pnl_section_strategy = st.builds(
|
||||
PLSection,
|
||||
realized_pnl=_finite_float,
|
||||
unrealized_pnl=_finite_float,
|
||||
daily_return=_finite_float,
|
||||
cumulative_return=_finite_float,
|
||||
win_count=st.integers(min_value=0, max_value=10000),
|
||||
loss_count=st.integers(min_value=0, max_value=10000),
|
||||
win_rate=_rate_float,
|
||||
profit_factor=_non_negative_finite_float,
|
||||
sharpe_ratio=_finite_float,
|
||||
summary=st.text(max_size=200),
|
||||
validation_warnings=st.lists(
|
||||
_validation_warning_strategy, min_size=0, max_size=3,
|
||||
),
|
||||
)
|
||||
|
||||
_recommendation_accuracy_strategy = st.builds(
|
||||
RecommendationAccuracySection,
|
||||
total_evaluated=st.integers(min_value=0, max_value=10000),
|
||||
act_count=st.integers(min_value=0, max_value=10000),
|
||||
skip_count=st.integers(min_value=0, max_value=10000),
|
||||
acted_win_rate=_rate_float,
|
||||
avg_confidence_acted=_rate_float,
|
||||
avg_confidence_skipped=_rate_float,
|
||||
summary=st.text(max_size=200),
|
||||
validation_warnings=st.lists(
|
||||
_validation_warning_strategy, min_size=0, max_size=3,
|
||||
),
|
||||
)
|
||||
|
||||
_position_detail_strategy = st.builds(
|
||||
PositionDetail,
|
||||
ticker=st.text(min_size=1, max_size=10),
|
||||
entry_price=_finite_float,
|
||||
current_or_exit_price=_finite_float,
|
||||
pnl=_finite_float,
|
||||
pnl_pct=_finite_float,
|
||||
hold_duration_hours=_non_negative_finite_float,
|
||||
status=st.sampled_from(["open", "closed"]),
|
||||
)
|
||||
|
||||
_position_performance_strategy = st.builds(
|
||||
PositionPerformanceSection,
|
||||
positions=st.lists(_position_detail_strategy, min_size=0, max_size=5),
|
||||
summary=st.text(max_size=200),
|
||||
)
|
||||
|
||||
_risk_metrics_strategy = st.builds(
|
||||
RiskMetricsSection,
|
||||
current_risk_tier=st.sampled_from(["low", "moderate", "high", "critical"]),
|
||||
portfolio_heat=_non_negative_finite_float,
|
||||
max_drawdown=_non_negative_finite_float,
|
||||
current_drawdown_pct=_non_negative_finite_float,
|
||||
reserve_pool_balance=_non_negative_finite_float,
|
||||
circuit_breaker_event_count=st.integers(min_value=0, max_value=100),
|
||||
summary=st.text(max_size=200),
|
||||
)
|
||||
|
||||
_model_quality_window_strategy = st.builds(
|
||||
ModelQualityWindow,
|
||||
lookback=st.sampled_from(["7d", "30d", "90d"]),
|
||||
win_rate=_optional_finite_float,
|
||||
directional_accuracy=_optional_finite_float,
|
||||
information_coefficient=_optional_finite_float,
|
||||
calibration_error=_optional_finite_float,
|
||||
brier_score=_optional_finite_float,
|
||||
)
|
||||
|
||||
_model_quality_strategy = st.builds(
|
||||
ModelQualitySection,
|
||||
windows=st.lists(_model_quality_window_strategy, min_size=0, max_size=3),
|
||||
summary=st.text(max_size=200),
|
||||
validation_warnings=st.lists(
|
||||
_validation_warning_strategy, min_size=0, max_size=3,
|
||||
),
|
||||
)
|
||||
|
||||
# Use timezone-aware datetimes for generated_at
|
||||
_aware_datetime_strategy = st.datetimes(
|
||||
min_value=datetime(2020, 1, 1),
|
||||
max_value=datetime(2030, 12, 31),
|
||||
timezones=st.just(timezone.utc),
|
||||
)
|
||||
|
||||
_date_strategy = st.dates(
|
||||
min_value=date(2020, 1, 1),
|
||||
max_value=date(2030, 12, 31),
|
||||
)
|
||||
|
||||
_report_data_strategy = st.builds(
|
||||
ReportData,
|
||||
pnl=_pnl_section_strategy,
|
||||
recommendation_accuracy=_recommendation_accuracy_strategy,
|
||||
position_performance=_position_performance_strategy,
|
||||
risk_metrics=_risk_metrics_strategy,
|
||||
model_quality=_model_quality_strategy,
|
||||
executive_summary=st.text(max_size=300),
|
||||
validation_status=st.sampled_from(list(ValidationStatus)),
|
||||
generated_at=_aware_datetime_strategy,
|
||||
period_start=_date_strategy,
|
||||
period_end=_date_strategy,
|
||||
report_type=st.sampled_from(list(ReportType)),
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helper: recursively find all datetime-like string values in parsed JSON
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_DATETIME_FIELD_NAMES = {"generated_at"}
|
||||
_DATE_FIELD_NAMES = {"period_start", "period_end"}
|
||||
|
||||
|
||||
def _collect_datetime_strings(
|
||||
obj: object,
|
||||
key: str | None = None,
|
||||
) -> list[tuple[str, str]]:
|
||||
"""Walk parsed JSON and collect (field_name, value) for datetime fields."""
|
||||
results: list[tuple[str, str]] = []
|
||||
if isinstance(obj, dict):
|
||||
for k, v in obj.items():
|
||||
results.extend(_collect_datetime_strings(v, k))
|
||||
elif isinstance(obj, list):
|
||||
for item in obj:
|
||||
results.extend(_collect_datetime_strings(item, key))
|
||||
elif isinstance(obj, str) and key is not None:
|
||||
if key in _DATETIME_FIELD_NAMES or key in _DATE_FIELD_NAMES:
|
||||
results.append((key, obj))
|
||||
return results
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Property tests
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given(report=_report_data_strategy)
|
||||
@settings(max_examples=100)
|
||||
def test_report_serialization_round_trip(report: ReportData) -> None:
|
||||
"""**Validates: Requirements 8.1, 8.2, 8.3, 8.4**
|
||||
|
||||
For any valid ReportData object, serializing to JSON and then
|
||||
deserializing back SHALL produce a ReportData object equivalent
|
||||
to the original.
|
||||
"""
|
||||
json_str = report.model_dump_json()
|
||||
restored = ReportData.model_validate_json(json_str)
|
||||
assert restored == report, (
|
||||
f"Round-trip failed: deserialized report differs from original.\n"
|
||||
f" report_type: {report.report_type}\n"
|
||||
f" period: {report.period_start} → {report.period_end}\n"
|
||||
f" generated_at: {report.generated_at}"
|
||||
)
|
||||
|
||||
|
||||
@given(report=_report_data_strategy)
|
||||
@settings(max_examples=100)
|
||||
def test_report_datetime_fields_iso8601(report: ReportData) -> None:
|
||||
"""**Validates: Requirements 8.4**
|
||||
|
||||
All datetime fields in the serialized JSON SHALL be in ISO 8601 format.
|
||||
"""
|
||||
json_str = report.model_dump_json()
|
||||
parsed = json.loads(json_str)
|
||||
dt_fields = _collect_datetime_strings(parsed)
|
||||
|
||||
for field_name, value in dt_fields:
|
||||
if field_name in _DATETIME_FIELD_NAMES:
|
||||
assert _ISO8601_DATETIME_RE.match(value), (
|
||||
f"Datetime field '{field_name}' is not ISO 8601: {value!r}"
|
||||
)
|
||||
elif field_name in _DATE_FIELD_NAMES:
|
||||
assert _ISO8601_DATE_RE.match(value), (
|
||||
f"Date field '{field_name}' is not ISO 8601: {value!r}"
|
||||
)
|
||||
Reference in New Issue
Block a user