feat: add Polygon grouped daily endpoint for broad market data
Two tiers of market data: 1. Per-ticker prev bars (existing 50 sources, 15-min cadence) for watchlist detail — trading decisions, stop-loss, position sizing 2. Grouped daily (new single source, once per day) for broad market context — correlation analysis, sector rotation, competitive intel Changes: - Add grouped_daily endpoint to PolygonMarketAdapter with auto date calculation (previous trading day, skip weekends) - Add fetch_global_market_sources() to scheduler for sources without company_id, scheduled once daily (86400s cadence) - Update _persist_market_items to use item-level ticker from T field and look up company_id dynamically for grouped daily bars - Migration 020: make company_id nullable on sources and market_snapshots tables, add grouped daily source row - Fix backtest replay to query market_snapshots data->>'c' for prices
This commit is contained in:
@@ -302,7 +302,11 @@ async def _persist_market_items(
|
||||
provider: str,
|
||||
content_hash: str,
|
||||
) -> tuple[int, list[str]]:
|
||||
"""Persist market data items as market_snapshots rows."""
|
||||
"""Persist market data items as market_snapshots rows.
|
||||
|
||||
For grouped daily responses, each item contains a 'T' field with the
|
||||
ticker. When present, the item's ticker overrides the job-level ticker.
|
||||
"""
|
||||
ids: list[str] = []
|
||||
for item in items:
|
||||
item_hash = content_hash_str(json.dumps(item, sort_keys=True))
|
||||
@@ -313,11 +317,24 @@ async def _persist_market_items(
|
||||
if exists:
|
||||
continue
|
||||
|
||||
# Use item-level ticker if available (grouped daily), else job-level
|
||||
item_ticker = item.get("T", ticker) or ticker
|
||||
snapshot_type = _infer_market_snapshot_type(item)
|
||||
|
||||
# For grouped daily items, look up company_id by ticker
|
||||
item_company_id = company_id
|
||||
if item.get("T") and not company_id:
|
||||
cid = await pool.fetchval(
|
||||
"SELECT id FROM companies WHERE ticker = $1", item_ticker
|
||||
)
|
||||
if cid:
|
||||
item_company_id = str(cid)
|
||||
# If not in our companies table, store with company_id=NULL
|
||||
|
||||
row_id = await persist_market_snapshot(
|
||||
pool,
|
||||
company_id=company_id,
|
||||
ticker=ticker,
|
||||
company_id=item_company_id,
|
||||
ticker=item_ticker,
|
||||
snapshot_type=snapshot_type,
|
||||
data=item,
|
||||
source_provider=provider,
|
||||
|
||||
Reference in New Issue
Block a user