162 Commits

Author	SHA1	Message	Date
Celes Renata	2538da3f1e	fix: switch market data sources from prev_bars to intraday_bars ... The market_api sources were configured with endpoint='prev_bars' which only fetches a single previous-day bar per ticker. Changed to 'intraday_bars' which fetches hourly bars for today from Polygon's /v2/aggs/ticker/{ticker}/range/1/hour/{today}/{today} endpoint. Updated: seed script, beta DB (50 sources), production DB (50 sources). This gives ~7-8 hourly price bars per trading day per ticker instead of 1 daily bar.	2026-04-29 22:07:47 +00:00
Celes Renata	cb3eb230d6	fix: fetch trend history per-window and run aggregation 24/7 ... Two fixes for missing intraday data: 1. Frontend: lifted selectedWindow state to page level so useTrendHistory passes window param to the API. Previously fetched all windows with limit=500 which exhausted the limit before reaching recent intraday data. Now fetches only the selected window's data. 2. Scheduler: removed market-hours-only restriction from periodic aggregation. Runs every 15 minutes 24/7 so intraday data is always populated for backtesting regardless of market state.	2026-04-29 21:11:46 +00:00
Celes Renata	6f54fd07fa	feat: periodic aggregation every 15 minutes during market hours ... The aggregation engine only ran when new documents were ingested, leaving intraday trend data stale for long periods. Now the scheduler enqueues all 50 tickers for re-aggregation every ~15 minutes during US market hours (Mon-Fri, 6:30 AM - 1:30 PM PT). This ensures continuous intraday trend updates based on existing signals and market price changes.	2026-04-29 18:27:49 +00:00
Celes Renata	6880f11c26	fix: add /no_think inline tag to disable Qwen3 thinking mode ... chat_template_kwargs isn't being respected by the vLLM deployment. Qwen3 models support /no_think as an inline suffix in the user message to disable thinking mode. This is the most reliable method across all serving backends (vLLM, Ollama, SGLang).	2026-04-29 16:11:44 +00:00
Celes Renata	eead4f1381	fix: disable thinking mode on vLLM path with chat_template_kwargs ... The thesis rewriter uses vLLM (not Ollama) in production. The previous fix only added think=False to the Ollama payload. For vLLM's OpenAI-compatible API with Qwen3 models, thinking mode is disabled via chat_template_kwargs: {enable_thinking: false} in the request body.	2026-04-29 16:04:04 +00:00
Celes Renata	007189c0a5	fix: handle plain-text thinking blocks and disable think mode ... The model outputs 'Thinking Process:' as plain text (not in <think> tags). Updated _strip_thinking_block to handle both XML tags and plain-text reasoning patterns. Also: - Added rule 7 to system prompt: 'Do NOT show your thinking process' - Set think=False in Ollama payload to disable Qwen3 thinking mode - Added fallback regex to extract thesis from after thinking blocks	2026-04-29 15:50:49 +00:00
Celes Renata	f9ee1532dc	fix: strip <think> reasoning blocks from thesis LLM output ... Qwen3.5 in thinking mode emits <think>...</think> chain-of-thought before the actual response. The thesis rewriter was returning the raw output including the entire reasoning block. Now strips thinking tags from both Ollama and vLLM response paths.	2026-04-29 15:25:04 +00:00
Celes Renata	bb40a3cb8e	fix: position sync now reconciles — removes positions broker no longer holds ... The sync_positions loop only upserted positions from Alpaca but never deleted DB rows for positions that were closed/liquidated on the broker side. After a paper reset, the next sync would not remove the stale positions because they simply weren't in Alpaca's response anymore. Now performs full reconciliation: after upserting what Alpaca reports, deletes any DB positions for the account that Alpaca no longer holds.	2026-04-29 12:02:57 +00:00
Celes Renata	4e010bc048	feat: signal math upgrade — probabilistic, regime-aware scoring pipeline ... Implement full probabilistic signal processing pipeline gated behind probabilistic_scoring_enabled feature flag in risk_configs: - Bayesian log-likelihood accumulator with Beta posterior and entropy - Regime detector (trend-following, panic, mean-reversion, uncertainty) - Source accuracy tracker with per-source historical prediction accuracy - Sigmoid confidence gate replacing binary gate - Information gain surprise weighting for rare events - Adaptive recency decay with event-specific half-lives - Regime multiplier replacing market context multiplier - Weighted disagreement entropy for contradiction detection - Multiplicative macro exposure with conditional integration - Graph-distance attenuated competitive signal propagation - Exponentially weighted momentum with volatility scaling - Expected value recommendation gate All changes backward-compatible: flag=false preserves exact current behavior. New outputs stored in existing JSONB columns (no schema changes except source_accuracy table via migration 034). Tests: 26 property-based tests (14 correctness properties), 99 unit tests, 1789 total tests passing with zero regressions.	2026-04-29 11:41:48 +00:00
Celes Renata	8c3c1aab43	fix: pipeline stop now halts all workers and flushes queues ... Workers (ingestion, parser, extractor, aggregation, recommendation, broker, lake-publisher) now check the pipeline:enabled Redis flag on each loop iteration and sleep when disabled. The toggle endpoint flushes all pipeline queues on disable so queued jobs don't resume when workers eventually check. Broker/trading queues are excluded from flush to avoid dropping in-flight orders.	2026-04-29 07:59:35 +00:00
Celes Renata	f264e924f0	fix: thesis rewriter now routes to vLLM when provider is vllm ... - thesis_llm.py: add _call_vllm_thesis() using /v1/chat/completions - thesis_llm.py: check resolved model_provider and route accordingly - values.yaml: set OLLAMA_BASE_URL to http://10.1.1.12:2701	2026-04-29 05:42:10 +00:00
Celes Renata	b38fb24f14	fix: ensure production uses DB-configured model/provider from UI ... - Migration 026: update seed defaults from ollama to vllm/AxionML - Migration 031: fix existing rows still on old ollama defaults - Helm values: set OLLAMA_BASE_URL to cluster ollama endpoint (was empty) - Extractor: guard against switching to ollama when base_url is empty - OllamaClient: validate base_url on construction to fail fast	2026-04-29 04:33:21 +00:00
Celes Renata	49bff9de50	fix: delete stale trend_evidence before inserting to prevent duplicate accumulation	2026-04-28 19:45:15 +00:00
Celes Renata	27b84fcd2e	fix: return open_position_count in trading status for dashboard display	2026-04-28 19:27:40 +00:00
Celes Renata	7e8d518946	feat: add max open positions and position cap controls to trading dashboard	2026-04-28 19:16:50 +00:00
Celes Renata	c188677330	fix: route macro_news documents as macro_event so global event classification runs	2026-04-28 16:18:02 +00:00
Celes Renata	226d799eb2	feat: auto-clamp buy orders to fit within position limits instead of hard-rejecting	2026-04-28 14:20:44 +00:00
Celes Renata	e360b66c3e	fix: beta trading pipeline — max_tokens default, approval re-enqueue, credentials ... - Migration 031: change ai_agents/agent_variants max_tokens default from 32768 to 4096 (32768 exceeds vLLM context window, causing HTTP 400 on every extraction) - API: re-enqueue approved orders to broker queue — previously approved orders sat in DB with nothing to execute them - values-beta: enable TRADING_ENABLED, update Alpaca paper keys	2026-04-28 14:13:58 +00:00
Celes Renata	0437943863	fix: reduce vLLM default max_tokens to 4096, update model to AxionML/Qwen3.5-9B-NVFP4 ... The model's max_model_len is 16384 — requesting 32768 output tokens caused HTTP 400 from vLLM. 4096 is a safe default for extraction output.	2026-04-23 19:49:34 +00:00
Celes Renata	f7ae34ef3b	fix: add extract() method to VLLMClient for extraction pipeline compatibility	2026-04-23 19:32:33 +00:00
Celes Renata	117b693b19	feat: add remote vLLM support with provider abstraction layer ... - LLMClient Protocol for provider-agnostic inference - VLLMClient for OpenAI-compatible /v1/chat/completions API - LLM client factory with provider routing (ollama/vllm) - VLLMConfig with VLLM_* environment variable loading - Updated extractor worker with health check and provider switching - Updated event classifier to use LLMClient protocol - Helm values for vLLM configuration - 18 unit tests + 6 property-based tests - Full backward compatibility preserved	2026-04-23 08:17:23 +00:00
Celes Renata	f251c53f92	fix: risk engine blocking sell orders on over-concentrated positions ... Two bugs: (1) trading engine omitted estimated_value from sell order jobs, causing risk engine to compute 0 reduction; (2) risk engine applied position size limits to sells, trapping users in positions they couldn't exit. Sells now always pass position value/pct checks.	2026-04-22 02:07:24 +00:00
Celes Renata	3b49aa2fa2	fix: risk engine now allows sells on over-concentrated positions	2026-04-21 20:25:02 +00:00
Celes Renata	5d0635a291	feat: beta deploys all services with pipeline toggle defaulting to OFF ... - pipelineEnabled: true in beta so all pods run (Kargo happy) - PIPELINE_DEFAULT_OFF=true in beta config — scheduler initializes the Redis toggle to OFF on first boot - Shared Ollama (10.1.1.12:2701) between beta and paper - Flip pipeline ON from the UI when testing, OFF when done - Optimistic UI update for the toggle button	2026-04-21 03:54:00 +00:00
Celes Renata	be526ae614	feat: pipeline on/off toggle with per-stage Helm control ... - Added pipelineEnabled flag to Helm values (default: true) - Worker services (scheduler, ingestion, parser, extractor, aggregation, recommendation, broker-adapter, lake-publisher) scale to 0 when disabled - API services always run regardless of toggle - Redis-based runtime toggle: POST /api/ops/pipeline/toggle - Scheduler checks the flag before each cycle - Frontend: green/red Pipeline ON/OFF button on the pipeline page - Beta defaults to pipelineEnabled: false - Base values.yaml: blanked external URLs (Ollama, Polygon, Alpaca) so stages only connect to what they explicitly configure	2026-04-21 00:21:53 +00:00
Celes Renata	7071bba92d	fix: increase stale threshold to 4h to prevent duplicate enqueuing ... The 30-minute threshold was shorter than the queue drain time, causing the recovery sweep to re-enqueue docs that were already queued but not yet processed. Bumped to 4 hours with matching marker TTL.	2026-04-20 18:05:30 +00:00
Celes Renata	46c24aefab	fix: prevent duplicate queue entries with Redis SET markers ... Recovery sweeps and the retry endpoint now check a per-document Redis key (SET NX, 1h TTL) before pushing to the queue. If the marker exists, the doc is already enqueued and gets skipped. This prevents the scheduler from re-enqueuing the same parsed docs every 5 minutes.	2026-04-20 17:24:53 +00:00
Celes Renata	288c5333b5	fix: use queue_key() for stage-prefixed Redis queue names in pipeline endpoints ... The pipeline health, SSE stream, and retry endpoints were hardcoding 'stonks:queue:{name}' but services use DEPLOY_STAGE prefix ('stonks:paper:queue:{name}'). Now uses queue_key() from redis_keys.py.	2026-04-20 13:16:11 +00:00
Celes Renata	de35279269	feat: retry failed extractions button on pipeline page ... - POST /api/ops/pipeline/retry-failed endpoint resets extraction_failed docs to parsed, deletes failed intelligence rows, and re-enqueues them (batch of 200) - Scheduler now auto-retries extraction_failed docs every ~10 minutes (100 per cycle, 60-min cooldown per doc) - Pipeline page shows 'Retry Failed (N)' button when extraction_failed count > 0, with pending/success/error states	2026-04-20 08:09:29 +00:00
Celes Renata	950ff03f7e	fix: join through document_intelligence in patterns endpoint ... The inline catalyst_type query in GET /api/patterns/{ticker} referenced dir.document_id which does not exist on document_impact_records. The table links to documents via intelligence_id -> document_intelligence -> document_id. Added the missing JOIN to match the pattern used in _SELF_PATTERN_QUERY.	2026-04-20 07:12:13 +00:00
Celes Renata	5acb2fb43e	fix: resolve 6 integration test failures ... 1. patterns endpoint: fix query referencing non-existent column di.catalyst_type → dir.catalyst_type (column is on document_impact_records) 2. lockouts seed: use relative timestamps (now + 7d) so active lockout is always in the future regardless of when tests run 3. create_agent: make slug optional with auto-generation from name 4. create_source: json.dumps(config) + ::jsonb cast for asyncpg JSONB compat 5. approval_expiry: return count as int (len(expired)) not the list itself 6. metrics_consistency: fix test assertion to match API contract (total >= active + reserve, not total == active + reserve + unrealized)	2026-04-20 04:30:13 +00:00
Celes Renata	8f67d326c9	feat: derive POSTGRES_DB and Redis prefix from DEPLOY_STAGE for pipeline isolation	2026-04-20 01:33:14 +00:00
Celes Renata	d64ce82649	fix: scheduler timezone-aware datetime subtraction in is_source_due	2026-04-20 00:47:26 +00:00
Celes Renata	47f10cd3cf	fix: use Python asyncpg migration runner instead of psql, remove postgresql-client from image	2026-04-19 22:54:01 +00:00
Celes Renata	5c63264393	feat: stage-isolated infrastructure — separate Postgres DBs, Redis DBs, and MinIO bucket prefixes per stage	2026-04-19 22:20:03 +00:00
Celes Renata	c85c0068a2	fix: clean up utcnow deprecation warnings, fix 12 failing tests, add CI/CD pipeline manifests ... - Replace all datetime.utcnow() with datetime.now(tz=timezone.utc) across 8 files - Fix 12 failing tests to match current implementation behavior - Fix pytest_plugins in non-top-level conftest (moved to root conftest.py) - Auto-fix 189 lint issues (import sorting, unused imports) - Add CI/CD pipeline infrastructure (ARC, ArgoCD, Kargo manifests) - Add values-beta.yaml and values-paper.yaml for staged deployments - Update GitHub Actions workflow to use self-hosted-gremlin runners - Add integration-test job to CI pipeline Result: 1596 passed, 0 failed, 0 warnings	2026-04-18 03:59:28 +00:00
Celes Renata	ee5fd30398	fix: backtest force-closes open positions at end + uses real market prices for exits	2026-04-18 00:03:51 +00:00
Celes Renata	82da8af02b	fix: dedup recommendation queue at aggregation level — prevent duplicate ticker+window flooding	2026-04-17 22:11:59 +00:00
Celes Renata	a6189fce6b	fix: skip LLM thesis rewrite for informational/suppressed recs to prevent queue buildup	2026-04-17 19:29:41 +00:00
Celes Renata	7736554988	fix: recommendation worker now respects thesis-rewriter active=false toggle	2026-04-17 19:22:31 +00:00
Celes Renata	d8f1b615c8	fix: add order status sync loop to broker adapter — pending orders now update to filled/cancelled	2026-04-17 17:38:10 +00:00
Celes Renata	aa67523acd	fix: ensure JSON output instruction in system prompt override + retry on ValueError	2026-04-17 17:03:58 +00:00
Celes Renata	523d3ea749	fix: catch ValueError in classification retry loop + add debug logging for raw output	2026-04-17 17:00:32 +00:00
Celes Renata	f054e97b5b	fix: event classifier unwraps single-element list from model output ... Model sometimes returns [{...}] instead of {...}. Now unwraps single-element lists before parsing the event fields.	2026-04-17 16:44:57 +00:00
Celes Renata	76ff7ae00a	fix: ruff import sort order	2026-04-17 16:37:30 +00:00
Celes Renata	1394e6168b	fix: event classifier now strips markdown fences and repairs JSON ... _parse_classification_response receives raw model output (with thinking tags, markdown fences, etc.) but was calling json.loads directly. Now uses _strip_markdown_fences + _repair_json from the client module before parsing, matching what _call_ollama does for extractions.	2026-04-17 16:35:13 +00:00
Celes Renata	759d868e3b	fix: event classifier was blocked by extraction schema validation ... _call_ollama validates against the document extraction schema, which doesn't match event classification output. The event classifier was checking 'if attempt.error is None' before trying its own parsing, so it never got to parse the valid event JSON — 956 consecutive failures. Now tries _parse_classification_response whenever raw_output exists, regardless of the extraction validation error.	2026-04-17 16:28:39 +00:00
Celes Renata	5efccb1e03	fix: deduplicate evidence refs in trend summaries ... Backend: assemble_trend_with_evidence now deduplicates document IDs via dict.fromkeys() (the rollup code already did this, but the base assembly didn't — same doc could appear multiple times from different intelligence extractions). Frontend: Trends.tsx deduplicates via Set before rendering as a safety net for existing data already stored with duplicates.	2026-04-17 07:25:32 +00:00
Celes Renata	419cf7558a	fix: evidence articles missing on recommendations + Lucide title prop CI failure ... - recommendation worker: filter out non-UUID document IDs (synthetic pattern:* IDs from competitive signals) before inserting into recommendation_evidence table — the uuid cast was failing and silently dropping all evidence rows - wrap executemany in try/except so partial failures don't lose all evidence - SqlExplorer: wrap Lucide icons in <span title=...> instead of passing title prop directly (not supported by lucide-react, broke CI build)	2026-04-17 07:10:21 +00:00
Celes Renata	913fe8b0b3	feat: override trade tab — manual order entry with auto-registration ... Backend: - OverrideOrderRequest/Response Pydantic models with ticker, quantity, price validators - POST /api/trading/override/order endpoint (enqueue to Redis broker queue) - auto_register_symbol() module for untracked ticker registration via Symbol Registry - Unit tests (17) and property-based tests (3 x 100 examples) Frontend: - OverrideTradePanel component (order form + positions display) - Override tab in TradingEngine page with URL search param navigation - Override Trade button on Trading Controls page - useSubmitOverrideOrder mutation hook - MSW handler and 13 component/integration tests Steering: - Updated steering docs for Ubuntu dev machine with nvm/Node 24	2026-04-17 07:02:30 +00:00