feat: add Polygon grouped daily endpoint for broad market data

Two tiers of market data:
1. Per-ticker prev bars (existing 50 sources, 15-min cadence) for
   watchlist detail — trading decisions, stop-loss, position sizing
2. Grouped daily (new single source, once per day) for broad market
   context — correlation analysis, sector rotation, competitive intel

Changes:
- Add grouped_daily endpoint to PolygonMarketAdapter with auto date
  calculation (previous trading day, skip weekends)
- Add fetch_global_market_sources() to scheduler for sources without
  company_id, scheduled once daily (86400s cadence)
- Update _persist_market_items to use item-level ticker from T field
  and look up company_id dynamically for grouped daily bars
- Migration 020: make company_id nullable on sources and
  market_snapshots tables, add grouped daily source row
- Fix backtest replay to query market_snapshots data->>'c' for prices
This commit is contained in:
Celes Renata
2026-04-15 22:38:18 +00:00
parent ea6c2b3f54
commit 4501bbebd4
4 changed files with 133 additions and 4 deletions
+20 -3
View File
@@ -302,7 +302,11 @@ async def _persist_market_items(
provider: str,
content_hash: str,
) -> tuple[int, list[str]]:
"""Persist market data items as market_snapshots rows."""
"""Persist market data items as market_snapshots rows.
For grouped daily responses, each item contains a 'T' field with the
ticker. When present, the item's ticker overrides the job-level ticker.
"""
ids: list[str] = []
for item in items:
item_hash = content_hash_str(json.dumps(item, sort_keys=True))
@@ -313,11 +317,24 @@ async def _persist_market_items(
if exists:
continue
# Use item-level ticker if available (grouped daily), else job-level
item_ticker = item.get("T", ticker) or ticker
snapshot_type = _infer_market_snapshot_type(item)
# For grouped daily items, look up company_id by ticker
item_company_id = company_id
if item.get("T") and not company_id:
cid = await pool.fetchval(
"SELECT id FROM companies WHERE ticker = $1", item_ticker
)
if cid:
item_company_id = str(cid)
# If not in our companies table, store with company_id=NULL
row_id = await persist_market_snapshot(
pool,
company_id=company_id,
ticker=ticker,
company_id=item_company_id,
ticker=item_ticker,
snapshot_type=snapshot_type,
data=item,
source_provider=provider,