feat: trading feedback engine — periodic performance reports with AI summarization
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
- Migration 038: trading_reports table + report-summarizer agent seed
- 6 reporting modules: models, collector, sections, validator, summarizer, generator
- API endpoints: GET /api/reports (paginated, filterable), GET /api/reports/{id}
- Frontend hooks: useReports, useReport with TanStack Query
- Scheduler: daily (after 16:30 ET) and weekly (Saturday) report triggers
- Redis queue consumer for async report generation with retry/dedup
- 5 property-based tests (chunking, serialization, validation, accuracy, deltas)
- 109 unit/integration tests across all modules
- 6 frontend hook tests with MSW mocks
This commit is contained in:
@@ -0,0 +1,203 @@
|
||||
"""Unit tests for AI summarizer.
|
||||
|
||||
Tests the deterministic fallback summary generation and chunk_data edge cases
|
||||
from services.reporting.summarizer.
|
||||
|
||||
Requirements validated: 2.2, 2.6
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from services.reporting.summarizer import build_deterministic_summary, chunk_data
|
||||
|
||||
# ═══════════════════════════════════════════════════════════════════════
|
||||
# 1. chunk_data — edge cases
|
||||
# Requirements validated: 2.2
|
||||
# ═══════════════════════════════════════════════════════════════════════
|
||||
|
||||
|
||||
class TestChunkDataEdgeCases:
|
||||
"""Tests for chunk_data edge cases."""
|
||||
|
||||
def test_empty_input_returns_single_empty_chunk(self) -> None:
|
||||
"""Empty input produces exactly one empty-string chunk."""
|
||||
result = chunk_data("", max_chars=100)
|
||||
assert result == [""]
|
||||
|
||||
def test_single_character_returns_one_chunk(self) -> None:
|
||||
"""A single character fits in one chunk."""
|
||||
result = chunk_data("x", max_chars=100)
|
||||
assert result == ["x"]
|
||||
|
||||
def test_exactly_at_limit_returns_one_chunk(self) -> None:
|
||||
"""A string exactly at the limit fits in one chunk."""
|
||||
data = "a" * 50
|
||||
result = chunk_data(data, max_chars=50)
|
||||
assert result == [data]
|
||||
|
||||
def test_one_char_over_limit_with_newline_returns_two_chunks(self) -> None:
|
||||
"""A string one char over the limit (with a newline) splits into two chunks."""
|
||||
# 25 chars + newline + 25 chars = 51 chars total, limit=50
|
||||
data = "a" * 25 + "\n" + "b" * 25
|
||||
result = chunk_data(data, max_chars=50)
|
||||
assert len(result) == 2
|
||||
# First chunk: "aaa...a\n" (26 chars), second chunk: "bbb...b" (25 chars)
|
||||
assert result[0] == "a" * 25 + "\n"
|
||||
assert result[1] == "b" * 25
|
||||
# Round-trip: concatenation reconstructs original
|
||||
assert "".join(result) == data
|
||||
|
||||
def test_no_newlines_in_long_string_returns_one_chunk(self) -> None:
|
||||
"""A long string with no newlines is never broken mid-line — stays as one chunk."""
|
||||
data = "x" * 200
|
||||
result = chunk_data(data, max_chars=50)
|
||||
# No newlines means no split points, so the entire string is one chunk
|
||||
assert result == [data]
|
||||
|
||||
def test_multiple_newlines_proper_splitting(self) -> None:
|
||||
"""Multiple newlines produce proper splitting at line boundaries."""
|
||||
# 3 lines of 30 chars each (including newlines): "aaa...\n" "bbb...\n" "ccc..."
|
||||
line_a = "a" * 29 + "\n" # 30 chars
|
||||
line_b = "b" * 29 + "\n" # 30 chars
|
||||
line_c = "c" * 29 # 29 chars
|
||||
data = line_a + line_b + line_c # 89 chars total
|
||||
result = chunk_data(data, max_chars=60)
|
||||
# First chunk: line_a + line_b = 60 chars (exactly at limit)
|
||||
# Second chunk: line_c = 29 chars
|
||||
assert len(result) == 2
|
||||
assert result[0] == line_a + line_b
|
||||
assert result[1] == line_c
|
||||
assert "".join(result) == data
|
||||
|
||||
def test_round_trip_concatenation(self) -> None:
|
||||
"""Concatenating all chunks reconstructs the original string."""
|
||||
data = "line1\nline2\nline3\nline4\n"
|
||||
result = chunk_data(data, max_chars=12)
|
||||
assert "".join(result) == data
|
||||
|
||||
def test_max_chars_one(self) -> None:
|
||||
"""With max_chars=1, each line-segment becomes its own chunk."""
|
||||
data = "a\nb"
|
||||
result = chunk_data(data, max_chars=1)
|
||||
# "a\n" is 2 chars but no split point within it, so it's one chunk
|
||||
# "b" is 1 char, another chunk
|
||||
assert "".join(result) == data
|
||||
assert len(result) >= 2
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════════════════
|
||||
# 2. build_deterministic_summary — section type templates
|
||||
# Requirements validated: 2.6
|
||||
# ═══════════════════════════════════════════════════════════════════════
|
||||
|
||||
|
||||
class TestBuildDeterministicSummary:
|
||||
"""Tests for build_deterministic_summary with each section type."""
|
||||
|
||||
def test_pnl_section(self) -> None:
|
||||
"""P&L section uses the pnl template with realized_pnl, unrealized_pnl, etc."""
|
||||
data = {
|
||||
"realized_pnl": 125.50,
|
||||
"unrealized_pnl": -30.20,
|
||||
"daily_return": 1.2,
|
||||
"win_rate": 72.7,
|
||||
}
|
||||
result = build_deterministic_summary("pnl", data)
|
||||
assert "125.5" in result
|
||||
assert "-30.2" in result
|
||||
assert "1.2" in result
|
||||
assert "72.7" in result
|
||||
assert result.startswith("P&L Summary:")
|
||||
|
||||
def test_recommendation_accuracy_section(self) -> None:
|
||||
"""Recommendation accuracy section uses the template with total_evaluated, act_count, etc."""
|
||||
data = {
|
||||
"total_evaluated": 15,
|
||||
"act_count": 8,
|
||||
"acted_win_rate": 75.0,
|
||||
"skip_count": 7,
|
||||
"avg_confidence_acted": 0.72,
|
||||
"avg_confidence_skipped": 0.48,
|
||||
}
|
||||
result = build_deterministic_summary("recommendation_accuracy", data)
|
||||
assert "15" in result
|
||||
assert "8" in result
|
||||
assert "75.0" in result or "75" in result
|
||||
assert "7" in result
|
||||
assert result.startswith("Recommendation Accuracy:")
|
||||
|
||||
def test_position_performance_section(self) -> None:
|
||||
"""Position performance section uses the template with position count."""
|
||||
data = {
|
||||
"positions": [
|
||||
{"ticker": "AAPL", "pnl": 68.0},
|
||||
{"ticker": "MSFT", "pnl": -12.0},
|
||||
{"ticker": "GOOG", "pnl": 25.0},
|
||||
],
|
||||
}
|
||||
result = build_deterministic_summary("position_performance", data)
|
||||
assert "3" in result
|
||||
assert "Position Performance:" in result
|
||||
|
||||
def test_position_performance_empty_positions(self) -> None:
|
||||
"""Position performance with no positions reports 0."""
|
||||
data = {"positions": []}
|
||||
result = build_deterministic_summary("position_performance", data)
|
||||
assert "0" in result
|
||||
|
||||
def test_risk_metrics_section(self) -> None:
|
||||
"""Risk metrics section uses the template with risk_tier, portfolio_heat, etc."""
|
||||
data = {
|
||||
"current_risk_tier": "moderate",
|
||||
"portfolio_heat": 0.12,
|
||||
"max_drawdown": 0.08,
|
||||
"current_drawdown_pct": 3.0,
|
||||
"reserve_pool_balance": 450.00,
|
||||
"circuit_breaker_event_count": 1,
|
||||
}
|
||||
result = build_deterministic_summary("risk_metrics", data)
|
||||
assert "moderate" in result
|
||||
assert "0.12" in result
|
||||
assert "0.08" in result
|
||||
assert "3.0" in result or "3" in result
|
||||
assert "450" in result
|
||||
assert "1" in result
|
||||
assert result.startswith("Risk Metrics:")
|
||||
|
||||
def test_model_quality_section(self) -> None:
|
||||
"""Model quality section uses the template with window count."""
|
||||
data = {
|
||||
"windows": [
|
||||
{"lookback": "7d"},
|
||||
{"lookback": "30d"},
|
||||
{"lookback": "90d"},
|
||||
],
|
||||
}
|
||||
result = build_deterministic_summary("model_quality", data)
|
||||
assert "3" in result
|
||||
assert "Model Quality:" in result
|
||||
|
||||
def test_model_quality_no_windows(self) -> None:
|
||||
"""Model quality with no windows reports 0."""
|
||||
data = {"windows": []}
|
||||
result = build_deterministic_summary("model_quality", data)
|
||||
assert "0" in result
|
||||
|
||||
def test_unknown_section_generic_fallback(self) -> None:
|
||||
"""An unknown section name produces a generic fallback summary."""
|
||||
data = {"metric_a": 1, "metric_b": 2, "metric_c": 3}
|
||||
result = build_deterministic_summary("unknown_section", data)
|
||||
assert "unknown_section" in result
|
||||
assert "3 metrics reported" in result
|
||||
|
||||
def test_unknown_section_empty_data(self) -> None:
|
||||
"""An unknown section with empty data reports 0 metrics."""
|
||||
result = build_deterministic_summary("totally_new", {})
|
||||
assert "totally_new" in result
|
||||
assert "0 metrics reported" in result
|
||||
|
||||
def test_pnl_missing_key_falls_back(self) -> None:
|
||||
"""P&L template with missing keys falls back to error message."""
|
||||
data = {"realized_pnl": 100.0} # missing other keys
|
||||
result = build_deterministic_summary("pnl", data)
|
||||
# Should fall back to the error message since template.format() will raise KeyError
|
||||
assert "template formatting failed" in result
|
||||
Reference in New Issue
Block a user