fix: filter correlation matrix to tracked companies and make non-blocking
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
The correlation query was fetching 12K+ tickers from market_snapshots instead of just the 50 tracked companies, causing OOM during startup. - JOIN with companies table to filter to active tracked tickers only - Move correlation computation to background task so engine starts trading immediately - Add 30s timeout to prevent indefinite hangs
This commit is contained in:
@@ -598,8 +598,9 @@ class TradingEngine:
|
||||
initial_capital, invested, available, reserve_balance, open_count,
|
||||
)
|
||||
|
||||
# Compute initial correlation matrix from market data
|
||||
await self._compute_correlation_matrix()
|
||||
# Compute initial correlation matrix from market data (non-blocking)
|
||||
# Runs in background so the engine can start trading immediately
|
||||
asyncio.create_task(self._compute_correlation_matrix(), name="correlation_matrix")
|
||||
|
||||
async def _decision_loop(self) -> None:
|
||||
"""Poll recommendations and evaluate them in a continuous loop.
|
||||
@@ -1351,12 +1352,19 @@ class TradingEngine:
|
||||
return
|
||||
|
||||
try:
|
||||
rows = await self.pool.fetch(
|
||||
"SELECT ticker, captured_at::date AS dt, (data->>'c')::float AS close "
|
||||
"FROM market_snapshots "
|
||||
"WHERE snapshot_type = 'bar' AND captured_at > NOW() - INTERVAL '30 days' "
|
||||
"ORDER BY ticker, captured_at"
|
||||
rows = await asyncio.wait_for(
|
||||
self.pool.fetch(
|
||||
"SELECT ms.ticker, ms.captured_at::date AS dt, (ms.data->>'c')::float AS close "
|
||||
"FROM market_snapshots ms "
|
||||
"JOIN companies c ON c.ticker = ms.ticker AND c.active = TRUE "
|
||||
"WHERE ms.snapshot_type = 'bar' AND ms.captured_at > NOW() - INTERVAL '30 days' "
|
||||
"ORDER BY ms.ticker, ms.captured_at"
|
||||
),
|
||||
timeout=30.0,
|
||||
)
|
||||
except asyncio.TimeoutError:
|
||||
logger.warning("Correlation matrix query timed out — skipping")
|
||||
return
|
||||
except Exception:
|
||||
logger.debug("Could not query market_snapshots for correlation matrix")
|
||||
return
|
||||
|
||||
Reference in New Issue
Block a user