Document Stages now shows time-filtered counts from /health endpoint
(consistent with Parsing/Extraction/Trends sections) with all-time
totals shown as a subtle subtitle from the SSE stream. Added "(last Xh)"
labels to all sections so the time window context is always visible.
_process_macro_classification never updated document status, leaving
macro_event docs stuck in parsed forever. Now marks them extracted on
success or extraction_failed on error.
- Replace setdefault() with explicit None checks in _normalize_extraction_data()
- Coerce null top-level fields (summary, novelty_score, confidence) to defaults
- Coerce null company fields (ticker, impact_score, impact_horizon, etc.) to defaults
- Filter out company entries with empty ticker after normalization
- Prevents schema validation failures when model returns null for required fields
- Strategy: Recreate (GPU can only be used by one pod at a time)
- nuextract-external targetPort: 8080 (nginx proxy port)
- served-model-name: numind/NuExtract3
- DB ai_agents updated to model=numind/NuExtract3 provider=vllm
- Ollama scaled to 0 — vLLM gets entire 16GB GPU
- NuExtract3 (9B) running at 80% GPU utilization
- Both OLLAMA_BASE_URL and VLLM_BASE_URL point to vLLM service
- Fixed gremlin-1 missing default route (no internet access)
- Removed GPU resource limit (time-slicing handles allocation)
- Removed init container (direct download works with internet fix)
- vLLM image mirrored to registry.celestium.life/stonks-oracle/vllm-openai
- Deployment uses Harbor image (Docker Hub IPv6 unreachable from cluster)
- All 3 pipelines use vllm-external.vllm-service.svc.cluster.local:2701
- K8s manifests at infra/kube-vllm/ and synced to ~/sources/kube/vllm
K8s vLLM deployment blocked by Docker Hub IPv6 connectivity issue.
Falling back to NixOS systemd vLLM service on esnixi.
K8s manifests remain in infra/kube-vllm/ for future use.
Sentiment Heatmap and Prediction Accuracy passed limit=500 but
the trends and recommendations endpoints enforce limit<=200,
causing a 422 validation error that made both panels show blank.
New ArgoCD stage 'stonks-live-math' running the dual-pipeline signal
engine against a separate Alpaca paper account for production validation.
- Namespace: stonks-live-math
- Database: stonks_live_math (Redis DB 3)
- Broker: PK64RS7NH24XPBI3IDEU3BB72Y (paper-api.alpaca.markets)
- Ingress: stonks-math-*.celestium.life
- Deploys alongside existing live stage (same image tag)
- Reports list page with type filter (daily/weekly)
- Report detail page with all sections: P&L, recommendation accuracy,
position performance table, risk metrics, model quality windows
- Executive summary card, validation warnings display
- Nav item under Trading group
- Routes: /reports and /reports/:id
- Add yAxisId='left' to vertical ReferenceLine (required by Recharts)
- Increase bottom margin from 40 to 70 for date label + legend space
- Legend uses verticalAlign='bottom' with 24px top padding
- Dockerfile now copies scripts/ directory into /app/scripts/
- Init container uses sh -c with fallback so missing script
doesn't crash the pod (graceful degradation until image rebuilds)
Inline Python with f-strings and colons broke Helm YAML parsing.
Moved to scripts/backfill_market_data.py and call it directly
from the init container command.
- Click 'Expand' button to view the trend chart fullscreen
- Press Escape or click outside to close
- Fullscreen uses full viewport height for better readability
- Green dashed vertical lines at 9:30 AM ET (market open)
- Red dashed vertical lines at 4:00 PM ET (market close)
- Markers only shown on intraday and 1d windows (hidden on 7d+)
- Chart content extracted into shared function to avoid duplication
Alpaca's current_price reflects extended hours trading which is more
current than Polygon's regular session close. Keep it as the display
price. Add polygon_price as a reference field in the API response.
Alpaca paper trading returns inaccurate current_price values.
The positions endpoint now uses the latest Polygon close from
market_snapshots and recomputes unrealized P&L from that.
1. Add backfill-market-data init container to scheduler deployment.
On vanilla start (< 50 bars in market_snapshots), fetches 90 days
of daily OHLCV from Polygon directly via asyncpg + httpx.
2. Fix trading engine dedup key placement. Previously the Redis dedup
key was set BEFORE evaluate_recommendation(), so recs skipped as
outside_trading_window were permanently deduped and never retried
when the market opened. Now the dedup key is only set AFTER the
decision for non-retryable outcomes (act, confidence_too_low, etc).
outside_trading_window skips are retryable.
After scale_up, restart the scheduler and wait for it to be ready
before other services start. The scheduler's init containers run
migrations, so this ensures tables exist before ingestion/extractor
pods try to query them.
Backend:
- GET /api/market/prices/{ticker} now returns { bars, range_90d }
with 90-day low/high computed from market_snapshots
- POST /api/market/backfill/{ticker} fetches 90 days of daily bars
from Polygon and inserts missing bars into market_snapshots
- POST /api/market/backfill-all does the same for all active tickers
Frontend:
- Right Y-axis domain scaled to 90-day min/max (with 3% padding)
- Green dashed reference line at 90-day high
- Red dashed reference line at 90-day low
- Labels show exact price on each reference line
- Default limit bumped to 200 bars
- Add initial capital input (toggle between broker balance or custom amount)
- Add reserve/active pool split slider (0-50%, default 20%)
- Backend accepts reserve_pct in reset request body
- Note in UI that Alpaca balance reset requires Alpaca dashboard
- Confirmation dialog shows exact capital and split being applied
Changed intraday_bars default from 1-hour bars to 15-minute bars.
This gives ~26 price points per trading day per ticker (6.5h market
× 4 bars/hour) instead of ~7 hourly bars. Limit raised to 100 to
accommodate the higher bar count.
The market_api sources were configured with endpoint='prev_bars' which
only fetches a single previous-day bar per ticker. Changed to
'intraday_bars' which fetches hourly bars for today from Polygon's
/v2/aggs/ticker/{ticker}/range/1/hour/{today}/{today} endpoint.
Updated: seed script, beta DB (50 sources), production DB (50 sources).
This gives ~7-8 hourly price bars per trading day per ticker instead
of 1 daily bar.
Only ~9 price bars per ticker (Polygon returns daily bars, not
intraday). Widened gap tolerance to 6h for intraday, 12h for 1d,
etc. Also skip time-range filtering when price data is sparse
(≤20 bars) to avoid showing no prices at all.
Was showing every data point timestamp. Now:
- Recharts generates max 8 evenly-spaced ticks
- Each tick shows 'Apr 29' in bold white + '2 PM' in gray
- All labels at -35° angle to avoid overlap
- Simplified tick component (no hour-boundary filtering needed)
Replaced string-based X-axis with numeric timestamp axis:
- Custom ChartXTick component renders hour marks at -35° angle
- New day boundaries shown in bold (e.g., 'Apr 29')
- Hour marks shown as '9:00 AM', '10:00 AM' etc.
- Tooltip shows full date+time on hover
- Direction timeline uses formatted timestamps
- Bottom margin increased to accommodate angled labels
Two fixes for missing intraday data:
1. Frontend: lifted selectedWindow state to page level so useTrendHistory
passes window param to the API. Previously fetched all windows with
limit=500 which exhausted the limit before reaching recent intraday
data. Now fetches only the selected window's data.
2. Scheduler: removed market-hours-only restriction from periodic
aggregation. Runs every 15 minutes 24/7 so intraday data is always
populated for backtesting regardless of market state.
The aggregation engine only ran when new documents were ingested,
leaving intraday trend data stale for long periods. Now the scheduler
enqueues all 50 tickers for re-aggregation every ~15 minutes during
US market hours (Mon-Fri, 6:30 AM - 1:30 PM PT). This ensures
continuous intraday trend updates based on existing signals and
market price changes.
Intraday was showing only 12h of data (9 sparse points). Widened to
24h to show a full day of intraday trend history. Also widened 1d
from 24h to 48h for better context.
Displays an 'Open Position' card above the trend charts when we hold
a position in that ticker. Shows shares, avg entry price, current
price, market value, and unrealized P&L with green/red coloring.
Card is hidden when no position exists for the ticker.
Containers default to UTC. The host is PDT but pods don't inherit
the host timezone. Adding TZ to the ConfigMap ensures all services
log and compute timestamps in Pacific time.
Document the TZ config option in both docker-deployment.md and
helm-reference.md. Default is America/Los_Angeles. Frontend uses
the browser's local timezone for display.
Replace raw UUID with a linked document title in both the collapsed
row (using the empty space on the right) and the expanded detail view.
Uses useDocument hook to fetch the title, falls back to truncated UUID
while loading. Clicking the link navigates to the document detail page.
The /api/patterns/{ticker}/competitive-signals endpoint returns
{competitive_signals: [...], count: N} but the hook was typed as
returning a raw array. The component called .map() on the object,
causing 'e.map is not a function'. Now extracts the array from the
response wrapper.
Variable was used before declaration (temporal dead zone error).
Moved windowHours/hoursBack/cutoffTs above the filtered const that
references cutoffTs.
The trend history chart was showing all historical data regardless of
which window was selected — only filtering by window name but not by
time range. Now filters both trend data and price data to the time
range matching the selected window (e.g., 7d shows last 7 days only,
30d shows last 30 days).
The TickerFilter triggered a query on every keystroke, causing re-renders
that stole focus from the input. Now uses a local input state with a
300ms debounce before updating the query, keeping focus on the text box
while typing.
chat_template_kwargs isn't being respected by the vLLM deployment.
Qwen3 models support /no_think as an inline suffix in the user message
to disable thinking mode. This is the most reliable method across all
serving backends (vLLM, Ollama, SGLang).
The thesis rewriter uses vLLM (not Ollama) in production. The previous
fix only added think=False to the Ollama payload. For vLLM's
OpenAI-compatible API with Qwen3 models, thinking mode is disabled via
chat_template_kwargs: {enable_thinking: false} in the request body.
The model outputs 'Thinking Process:' as plain text (not in <think> tags).
Updated _strip_thinking_block to handle both XML tags and plain-text
reasoning patterns. Also:
- Added rule 7 to system prompt: 'Do NOT show your thinking process'
- Set think=False in Ollama payload to disable Qwen3 thinking mode
- Added fallback regex to extract thesis from after thinking blocks
Qwen3.5 in thinking mode emits <think>...</think> chain-of-thought
before the actual response. The thesis rewriter was returning the raw
output including the entire reasoning block. Now strips thinking tags
from both Ollama and vLLM response paths.
Add sections 1B, 2B, 3B, 4B, 5B, 7B covering all new probabilistic
formulas: sigmoid gate, info gain, adaptive decay, regime multiplier,
source accuracy, Bayesian posterior, entropy direction, weighted
disagreement entropy, multiplicative macro exposure, conditional macro
integration, graph-distance attenuation, EW momentum, and EV gate.
Updated constants summary with all new parameters.
The sync_positions loop only upserted positions from Alpaca but never
deleted DB rows for positions that were closed/liquidated on the broker
side. After a paper reset, the next sync would not remove the stale
positions because they simply weren't in Alpaca's response anymore.
Now performs full reconciliation: after upserting what Alpaca reports,
deletes any DB positions for the account that Alpaca no longer holds.
Workers (ingestion, parser, extractor, aggregation, recommendation,
broker, lake-publisher) now check the pipeline:enabled Redis flag on
each loop iteration and sleep when disabled.
The toggle endpoint flushes all pipeline queues on disable so queued
jobs don't resume when workers eventually check. Broker/trading queues
are excluded from flush to avoid dropping in-flight orders.
- thesis_llm.py: add _call_vllm_thesis() using /v1/chat/completions
- thesis_llm.py: check resolved model_provider and route accordingly
- values.yaml: set OLLAMA_BASE_URL to http://10.1.1.12:2701
Celes Renata