8d227b62f6
scale extractor to 4 replicas
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/finalize Pipeline failed
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
2026-04-28 21:54:38 -07:00
b38fb24f14
fix: ensure production uses DB-configured model/provider from UI
...
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
- Migration 026: update seed defaults from ollama to vllm/AxionML
- Migration 031: fix existing rows still on old ollama defaults
- Helm values: set OLLAMA_BASE_URL to cluster ollama endpoint (was empty)
- Extractor: guard against switching to ollama when base_url is empty
- OllamaClient: validate base_url on construction to fail fast
2026-04-29 04:33:21 +00:00
23f2134754
fix: stop migrations from resetting ai_agents model_name, default to vllm/AxionML/Qwen3.5-9B-NVFP4
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
2026-04-28 18:58:13 +00:00
3b22f5e1fc
feat: seed default risk_configs with macro and competitive layers enabled
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
2026-04-28 16:34:40 +00:00
e360b66c3e
fix: beta trading pipeline — max_tokens default, approval re-enqueue, credentials
...
- Migration 031: change ai_agents/agent_variants max_tokens default
from 32768 to 4096 (32768 exceeds vLLM context window, causing
HTTP 400 on every extraction)
- API: re-enqueue approved orders to broker queue — previously
approved orders sat in DB with nothing to execute them
- values-beta: enable TRADING_ENABLED, update Alpaca paper keys
2026-04-28 14:13:58 +00:00
4bee7a7874
fix: update vLLM model to AxionML/Qwen3.5-9B-NVFP4
2026-04-23 19:30:22 +00:00
6909ac5e50
feat: add vLLM config to beta values overlay
2026-04-23 08:19:03 +00:00
117b693b19
feat: add remote vLLM support with provider abstraction layer
...
- LLMClient Protocol for provider-agnostic inference
- VLLMClient for OpenAI-compatible /v1/chat/completions API
- LLM client factory with provider routing (ollama/vllm)
- VLLMConfig with VLLM_* environment variable loading
- Updated extractor worker with health check and provider switching
- Updated event classifier to use LLMClient protocol
- Helm values for vLLM configuration
- 18 unit tests + 6 property-based tests
- Full backward compatibility preserved
2026-04-23 08:17:23 +00:00
5dcbd286e8
fix: update beta Alpaca broker credentials
2026-04-21 16:43:54 +00:00
d407888caa
feat: set beta Ollama model to qwen3.6
2026-04-21 09:55:52 +00:00
abdf639925
feat: isolate beta on stonks_beta database, stop sharing stonks_paper
2026-04-21 09:50:22 +00:00
a99aaa29ca
feat: point beta Ollama to 192.168.42.254:11434
2026-04-21 09:36:03 +00:00
08b93ad119
fix: add beta Alpaca broker credentials for separate paper account
2026-04-21 05:32:13 +00:00
19f39301d3
fix: skip superset/trino ingress when hostname is empty (fixes Kargo beta promotion)
2026-04-21 05:09:08 +00:00
9275351279
fix: beta stage uses paper DB to resolve permission denied errors
...
Beta was pointing at stonks_beta DB where tables were owned by postgres
superuser, causing permission denied for the stonks app user. Switch to
sharing stonks_paper DB/user (already has proper grants). DEPLOY_STAGE=beta
still isolates Redis keys and MinIO buckets. Added market data API key
so beta can test ingestion when pipeline is toggled ON.
2026-04-21 04:57:29 +00:00
5d0635a291
feat: beta deploys all services with pipeline toggle defaulting to OFF
...
- pipelineEnabled: true in beta so all pods run (Kargo happy)
- PIPELINE_DEFAULT_OFF=true in beta config — scheduler initializes
the Redis toggle to OFF on first boot
- Shared Ollama (10.1.1.12:2701) between beta and paper
- Flip pipeline ON from the UI when testing, OFF when done
- Optimistic UI update for the toggle button
2026-04-21 03:54:00 +00:00
6703b20b59
fix: wait for infra pods to exist before checking readiness
...
kubectl wait fails immediately with 'no matching resources found' if
pods haven't been created yet. Added a poll loop to wait for all 3
infra pods (postgres, redis, minio) to exist before running wait.
2026-04-21 02:42:20 +00:00
c166aafc40
fix: reduce integration test timeout to 5 minutes
...
Tests complete in ~7s. The 10-minute timeout was causing unnecessary
wait time on failures. Reduced Job activeDeadlineSeconds and kubectl
wait timeout to 300s.
2026-04-21 01:52:11 +00:00
be526ae614
feat: pipeline on/off toggle with per-stage Helm control
...
- Added pipelineEnabled flag to Helm values (default: true)
- Worker services (scheduler, ingestion, parser, extractor, aggregation,
recommendation, broker-adapter, lake-publisher) scale to 0 when disabled
- API services always run regardless of toggle
- Redis-based runtime toggle: POST /api/ops/pipeline/toggle
- Scheduler checks the flag before each cycle
- Frontend: green/red Pipeline ON/OFF button on the pipeline page
- Beta defaults to pipelineEnabled: false
- Base values.yaml: blanked external URLs (Ollama, Polygon, Alpaca)
so stages only connect to what they explicitly configure
2026-04-21 00:21:53 +00:00
a19ed086fe
fix: blank external URLs in base values, set only in stage overrides
...
Base values.yaml now has empty OLLAMA_BASE_URL, MARKET_DATA_BASE_URL,
and BROKER_PROVIDER. Only paper (and eventually live) set the real
URLs. Beta inherits empty defaults so it can't reach external services.
2026-04-21 00:15:22 +00:00
d8fce71178
fix: disable pipeline workers and blank API keys in beta stage
...
Beta is for API testing only. Scale scheduler, ingestion, parser,
extractor, aggregation, recommendation, broker-adapter, and
lake-publisher to 0 replicas. Blank out Polygon and Alpaca keys.
Infra secrets (postgres, redis, minio) kept so API services work.
2026-04-21 00:14:02 +00:00
7eab50fda9
fix: disable all external connections and pipeline workers in beta
...
Beta is for API testing only. Blanked out Polygon/Alpaca/Ollama
credentials, set OLLAMA_BASE_URL to localhost:99999, and scaled
scheduler/ingestion/parser/extractor/aggregation/recommendation/
broker-adapter/lake-publisher to 0 replicas.
2026-04-21 00:13:42 +00:00
20faa8e20d
fix: bake secrets into values-paper.yaml and auto-seed on empty DB
...
- All paper stage credentials now in values-paper.yaml so ArgoCD
renders them correctly on every sync (no more empty secrets)
- Added seed-if-empty init container to scheduler: runs the seed
script if the companies table is empty after migrations
2026-04-20 17:40:41 +00:00
740ddc1c54
fix: revert extractor to 1 replica (single GPU bottleneck)
2026-04-20 12:16:51 +00:00
f1f0b7e34c
fix: scale extractor to 3 replicas in paper stage
...
The extraction queue had 3000+ SEC filings backed up with a single
extractor pod processing them at 10-115s each. Ollama handles
concurrent requests so multiple extractor pods can share the GPU.
2026-04-20 10:59:05 +00:00
5289f0f195
fix: use kubectl wait for job completion detection in inttest pipeline
...
The polling loop checked conditions[0].type which missed the Complete
condition when it wasn't at index 0. Switch to kubectl wait
--for=condition=complete which handles condition matching reliably.
2026-04-20 07:47:10 +00:00
422326bf83
fix: inttest pipeline timeout and busybox grep compatibility
...
- Poll job status instead of kubectl wait (catches Failed condition
immediately instead of waiting 600s for Complete that never comes)
- Replace grep -oP (Perl regex) with POSIX grep -o (BusyBox compat)
2026-04-20 04:23:19 +00:00
f3aac0ac3d
fix: superset config uses POSTGRES_DB and REDIS_DB env vars for stage isolation
2026-04-19 23:49:11 +00:00
0f2f0460a6
fix: dedicated scheduler Dockerfile with psql for migrations, remove Python splitter
2026-04-19 23:35:00 +00:00
48fed18078
feat: per-stage PostgreSQL users for database isolation (stonks_beta, stonks_paper)
2026-04-19 23:17:22 +00:00
47f10cd3cf
fix: use Python asyncpg migration runner instead of psql, remove postgresql-client from image
2026-04-19 22:54:01 +00:00
021efba294
feat: auto-run migrations via psql init container on scheduler startup
2026-04-19 22:37:50 +00:00
5c63264393
feat: stage-isolated infrastructure — separate Postgres DBs, Redis DBs, and MinIO bucket prefixes per stage
2026-04-19 22:20:03 +00:00
2621b3c5c5
feat: add stage-specific ingress hostnames for beta and paper
2026-04-19 22:00:47 +00:00
827be709df
fix: use Recreate strategy for hive-metastore and superset (RWO PVC)
2026-04-19 20:41:22 +00:00
a9be904afe
fix: guard ghcr-secret template against nil ghcrAuth values
2026-04-19 19:51:29 +00:00
1f69a27e3b
fix: replace mktemp with PID-based temp path for BusyBox compat
...
BusyBox mktemp in alpine/k8s doesn't support .json suffix in template.
The mktemp failure triggered set -e, causing pipeline to report failure
despite all 93 tests passing.
2026-04-19 19:35:02 +00:00
4df513d096
fix: remove bucket-init job, wait for pods before readiness check
...
- Remove minio-bucket-init Job entirely (seed_minio.py creates bucket)
- Wait for pods to exist before kubectl wait --for=condition=ready
- Fixes 'no matching resources found' race when pods are still ContainerCreating
2026-04-19 19:25:49 +00:00
b2b8aca7c6
fix: inttest runner crash and minio bucket-init proxy issue
...
- Remove --profiling-output arg from runner.yaml (plugin uses default path)
- Inline profiling hooks in root conftest.py with graceful fallback
- Replace mc-based bucket-init with Python urllib (no proxy interference)
- Add explicit ProxyHandler({}) to guarantee no proxy usage in bucket-init
2026-04-19 19:15:20 +00:00
4ebf75134f
ci: clear proxy env in minio-bucket-init, capture seed pod logs on failure
2026-04-19 08:55:52 +00:00
5be3ce2db9
feat: migrate CI/CD from GHCR to local Harbor registry
...
- Makefile: GHCR -> registry.celestium.life/stonks-oracle
- GitHub Actions: login to Harbor, use HARBOR_PASSWORD secret
- infra/k8s/*.yaml: all image refs -> registry.celestium.life
- inttest pipeline: remove GHCR pull secret (local registry, no auth)
- Steering docs: update registry/git endpoints
2026-04-19 07:34:28 +00:00
c2372ccd1e
ci: add NO_PROXY to minio-bucket-init to bypass proxy for internal services
2026-04-19 07:02:27 +00:00
2d40d70975
ci: remove remaining ghcr-credentials from inttest seed/minio pod overrides
2026-04-19 06:45:46 +00:00
ebafe795c1
fix: bump seed pod timeout to 5m and add debug diagnostics on pipeline failures
2026-04-19 06:34:58 +00:00
19b63dd369
ci: migrate inttest images from GHCR to local registry, remove ghcr-credentials
2026-04-19 06:22:35 +00:00
e3e1531847
ci: add Docker Hub auth + proxy CA to inttest namespace, fix MinIO pull secret
2026-04-19 06:09:56 +00:00
5f6d23888a
ci: fix lint errors across project, update ruff.toml per-file ignores
2026-04-18 21:02:28 +00:00
c85c0068a2
fix: clean up utcnow deprecation warnings, fix 12 failing tests, add CI/CD pipeline manifests
...
- Replace all datetime.utcnow() with datetime.now(tz=timezone.utc) across 8 files
- Fix 12 failing tests to match current implementation behavior
- Fix pytest_plugins in non-top-level conftest (moved to root conftest.py)
- Auto-fix 189 lint issues (import sorting, unused imports)
- Add CI/CD pipeline infrastructure (ARC, ArgoCD, Kargo manifests)
- Add values-beta.yaml and values-paper.yaml for staged deployments
- Update GitHub Actions workflow to use self-hosted-gremlin runners
- Add integration-test job to CI pipeline
Result: 1596 passed, 0 failed, 0 warnings
2026-04-18 03:59:28 +00:00
7a541aa693
fix: trend_windows.id missing gen_random_uuid() default
2026-04-17 17:27:41 +00:00
84b3c06b2f
fix: sync agent DB prompts with code defaults + update model to qwen3.5:9b-fast
2026-04-17 17:12:09 +00:00
Celes Renata