Commit Graph

101 Commits

Author SHA1 Message Date
Celes Renata aaf8cee927 feat: scale extractor to 8 replicas
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
2026-04-29 05:54:00 +00:00
Celes Renata f264e924f0 fix: thesis rewriter now routes to vLLM when provider is vllm
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
- thesis_llm.py: add _call_vllm_thesis() using /v1/chat/completions
- thesis_llm.py: check resolved model_provider and route accordingly
- values.yaml: set OLLAMA_BASE_URL to http://10.1.1.12:2701
2026-04-29 05:42:10 +00:00
Celes Renata a36702e5f3 fix: restore REDIS_PASSWORD and MINIO credentials
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
2026-04-28 22:02:50 -07:00
Celes Renata 01d77c153d fix: set POSTGRES_PASSWORD in core secrets
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/finalize unknown status
ci/woodpecker/push/build-3 Pipeline failed
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
2026-04-28 21:59:30 -07:00
Celes Renata 8d227b62f6 scale extractor to 4 replicas
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/finalize Pipeline failed
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
2026-04-28 21:54:38 -07:00
Celes Renata b38fb24f14 fix: ensure production uses DB-configured model/provider from UI
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
- Migration 026: update seed defaults from ollama to vllm/AxionML
- Migration 031: fix existing rows still on old ollama defaults
- Helm values: set OLLAMA_BASE_URL to cluster ollama endpoint (was empty)
- Extractor: guard against switching to ollama when base_url is empty
- OllamaClient: validate base_url on construction to fail fast
2026-04-29 04:33:21 +00:00
Celes Renata 23f2134754 fix: stop migrations from resetting ai_agents model_name, default to vllm/AxionML/Qwen3.5-9B-NVFP4
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
2026-04-28 18:58:13 +00:00
Celes Renata 3b22f5e1fc feat: seed default risk_configs with macro and competitive layers enabled
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-3 Pipeline was successful
ci/woodpecker/push/build-2 Pipeline was successful
ci/woodpecker/push/build-1 Pipeline was successful
ci/woodpecker/push/finalize Pipeline was successful
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled
Build and Push / beta-gate (push) Has been cancelled
2026-04-28 16:34:40 +00:00
Celes Renata e360b66c3e fix: beta trading pipeline — max_tokens default, approval re-enqueue, credentials
- Migration 031: change ai_agents/agent_variants max_tokens default
  from 32768 to 4096 (32768 exceeds vLLM context window, causing
  HTTP 400 on every extraction)
- API: re-enqueue approved orders to broker queue — previously
  approved orders sat in DB with nothing to execute them
- values-beta: enable TRADING_ENABLED, update Alpaca paper keys
2026-04-28 14:13:58 +00:00
Celes Renata 4bee7a7874 fix: update vLLM model to AxionML/Qwen3.5-9B-NVFP4 2026-04-23 19:30:22 +00:00
Celes Renata 6909ac5e50 feat: add vLLM config to beta values overlay 2026-04-23 08:19:03 +00:00
Celes Renata 117b693b19 feat: add remote vLLM support with provider abstraction layer
- LLMClient Protocol for provider-agnostic inference
- VLLMClient for OpenAI-compatible /v1/chat/completions API
- LLM client factory with provider routing (ollama/vllm)
- VLLMConfig with VLLM_* environment variable loading
- Updated extractor worker with health check and provider switching
- Updated event classifier to use LLMClient protocol
- Helm values for vLLM configuration
- 18 unit tests + 6 property-based tests
- Full backward compatibility preserved
2026-04-23 08:17:23 +00:00
Celes Renata 5dcbd286e8 fix: update beta Alpaca broker credentials 2026-04-21 16:43:54 +00:00
Celes Renata d407888caa feat: set beta Ollama model to qwen3.6 2026-04-21 09:55:52 +00:00
Celes Renata abdf639925 feat: isolate beta on stonks_beta database, stop sharing stonks_paper 2026-04-21 09:50:22 +00:00
Celes Renata a99aaa29ca feat: point beta Ollama to 192.168.42.254:11434 2026-04-21 09:36:03 +00:00
Celes Renata 08b93ad119 fix: add beta Alpaca broker credentials for separate paper account 2026-04-21 05:32:13 +00:00
Celes Renata 19f39301d3 fix: skip superset/trino ingress when hostname is empty (fixes Kargo beta promotion) 2026-04-21 05:09:08 +00:00
Celes Renata 9275351279 fix: beta stage uses paper DB to resolve permission denied errors
Beta was pointing at stonks_beta DB where tables were owned by postgres
superuser, causing permission denied for the stonks app user. Switch to
sharing stonks_paper DB/user (already has proper grants). DEPLOY_STAGE=beta
still isolates Redis keys and MinIO buckets. Added market data API key
so beta can test ingestion when pipeline is toggled ON.
2026-04-21 04:57:29 +00:00
Celes Renata 5d0635a291 feat: beta deploys all services with pipeline toggle defaulting to OFF
- pipelineEnabled: true in beta so all pods run (Kargo happy)
- PIPELINE_DEFAULT_OFF=true in beta config — scheduler initializes
  the Redis toggle to OFF on first boot
- Shared Ollama (10.1.1.12:2701) between beta and paper
- Flip pipeline ON from the UI when testing, OFF when done
- Optimistic UI update for the toggle button
2026-04-21 03:54:00 +00:00
Celes Renata 6703b20b59 fix: wait for infra pods to exist before checking readiness
kubectl wait fails immediately with 'no matching resources found' if
pods haven't been created yet. Added a poll loop to wait for all 3
infra pods (postgres, redis, minio) to exist before running wait.
2026-04-21 02:42:20 +00:00
Celes Renata c166aafc40 fix: reduce integration test timeout to 5 minutes
Tests complete in ~7s. The 10-minute timeout was causing unnecessary
wait time on failures. Reduced Job activeDeadlineSeconds and kubectl
wait timeout to 300s.
2026-04-21 01:52:11 +00:00
Celes Renata be526ae614 feat: pipeline on/off toggle with per-stage Helm control
- Added pipelineEnabled flag to Helm values (default: true)
- Worker services (scheduler, ingestion, parser, extractor, aggregation,
  recommendation, broker-adapter, lake-publisher) scale to 0 when disabled
- API services always run regardless of toggle
- Redis-based runtime toggle: POST /api/ops/pipeline/toggle
- Scheduler checks the flag before each cycle
- Frontend: green/red Pipeline ON/OFF button on the pipeline page
- Beta defaults to pipelineEnabled: false
- Base values.yaml: blanked external URLs (Ollama, Polygon, Alpaca)
  so stages only connect to what they explicitly configure
2026-04-21 00:21:53 +00:00
Celes Renata a19ed086fe fix: blank external URLs in base values, set only in stage overrides
Base values.yaml now has empty OLLAMA_BASE_URL, MARKET_DATA_BASE_URL,
and BROKER_PROVIDER. Only paper (and eventually live) set the real
URLs. Beta inherits empty defaults so it can't reach external services.
2026-04-21 00:15:22 +00:00
Celes Renata d8fce71178 fix: disable pipeline workers and blank API keys in beta stage
Beta is for API testing only. Scale scheduler, ingestion, parser,
extractor, aggregation, recommendation, broker-adapter, and
lake-publisher to 0 replicas. Blank out Polygon and Alpaca keys.
Infra secrets (postgres, redis, minio) kept so API services work.
2026-04-21 00:14:02 +00:00
Celes Renata 7eab50fda9 fix: disable all external connections and pipeline workers in beta
Beta is for API testing only. Blanked out Polygon/Alpaca/Ollama
credentials, set OLLAMA_BASE_URL to localhost:99999, and scaled
scheduler/ingestion/parser/extractor/aggregation/recommendation/
broker-adapter/lake-publisher to 0 replicas.
2026-04-21 00:13:42 +00:00
Celes Renata 20faa8e20d fix: bake secrets into values-paper.yaml and auto-seed on empty DB
- All paper stage credentials now in values-paper.yaml so ArgoCD
  renders them correctly on every sync (no more empty secrets)
- Added seed-if-empty init container to scheduler: runs the seed
  script if the companies table is empty after migrations
2026-04-20 17:40:41 +00:00
Celes Renata 740ddc1c54 fix: revert extractor to 1 replica (single GPU bottleneck) 2026-04-20 12:16:51 +00:00
Celes Renata f1f0b7e34c fix: scale extractor to 3 replicas in paper stage
The extraction queue had 3000+ SEC filings backed up with a single
extractor pod processing them at 10-115s each. Ollama handles
concurrent requests so multiple extractor pods can share the GPU.
2026-04-20 10:59:05 +00:00
Celes Renata 5289f0f195 fix: use kubectl wait for job completion detection in inttest pipeline
The polling loop checked conditions[0].type which missed the Complete
condition when it wasn't at index 0. Switch to kubectl wait
--for=condition=complete which handles condition matching reliably.
2026-04-20 07:47:10 +00:00
Celes Renata 422326bf83 fix: inttest pipeline timeout and busybox grep compatibility
- Poll job status instead of kubectl wait (catches Failed condition
  immediately instead of waiting 600s for Complete that never comes)
- Replace grep -oP (Perl regex) with POSIX grep -o (BusyBox compat)
2026-04-20 04:23:19 +00:00
Celes Renata f3aac0ac3d fix: superset config uses POSTGRES_DB and REDIS_DB env vars for stage isolation 2026-04-19 23:49:11 +00:00
Celes Renata 0f2f0460a6 fix: dedicated scheduler Dockerfile with psql for migrations, remove Python splitter 2026-04-19 23:35:00 +00:00
Celes Renata 48fed18078 feat: per-stage PostgreSQL users for database isolation (stonks_beta, stonks_paper) 2026-04-19 23:17:22 +00:00
Celes Renata 47f10cd3cf fix: use Python asyncpg migration runner instead of psql, remove postgresql-client from image 2026-04-19 22:54:01 +00:00
Celes Renata 021efba294 feat: auto-run migrations via psql init container on scheduler startup 2026-04-19 22:37:50 +00:00
Celes Renata 5c63264393 feat: stage-isolated infrastructure — separate Postgres DBs, Redis DBs, and MinIO bucket prefixes per stage 2026-04-19 22:20:03 +00:00
Celes Renata 2621b3c5c5 feat: add stage-specific ingress hostnames for beta and paper 2026-04-19 22:00:47 +00:00
Celes Renata 827be709df fix: use Recreate strategy for hive-metastore and superset (RWO PVC) 2026-04-19 20:41:22 +00:00
Celes Renata a9be904afe fix: guard ghcr-secret template against nil ghcrAuth values 2026-04-19 19:51:29 +00:00
Celes Renata 1f69a27e3b fix: replace mktemp with PID-based temp path for BusyBox compat
BusyBox mktemp in alpine/k8s doesn't support .json suffix in template.
The mktemp failure triggered set -e, causing pipeline to report failure
despite all 93 tests passing.
2026-04-19 19:35:02 +00:00
Celes Renata 4df513d096 fix: remove bucket-init job, wait for pods before readiness check
- Remove minio-bucket-init Job entirely (seed_minio.py creates bucket)
- Wait for pods to exist before kubectl wait --for=condition=ready
- Fixes 'no matching resources found' race when pods are still ContainerCreating
2026-04-19 19:25:49 +00:00
Celes Renata b2b8aca7c6 fix: inttest runner crash and minio bucket-init proxy issue
- Remove --profiling-output arg from runner.yaml (plugin uses default path)
- Inline profiling hooks in root conftest.py with graceful fallback
- Replace mc-based bucket-init with Python urllib (no proxy interference)
- Add explicit ProxyHandler({}) to guarantee no proxy usage in bucket-init
2026-04-19 19:15:20 +00:00
Celes Renata 4ebf75134f ci: clear proxy env in minio-bucket-init, capture seed pod logs on failure 2026-04-19 08:55:52 +00:00
Celes Renata 5be3ce2db9 feat: migrate CI/CD from GHCR to local Harbor registry
- Makefile: GHCR -> registry.celestium.life/stonks-oracle
- GitHub Actions: login to Harbor, use HARBOR_PASSWORD secret
- infra/k8s/*.yaml: all image refs -> registry.celestium.life
- inttest pipeline: remove GHCR pull secret (local registry, no auth)
- Steering docs: update registry/git endpoints
2026-04-19 07:34:28 +00:00
Celes Renata c2372ccd1e ci: add NO_PROXY to minio-bucket-init to bypass proxy for internal services 2026-04-19 07:02:27 +00:00
Celes Renata 2d40d70975 ci: remove remaining ghcr-credentials from inttest seed/minio pod overrides 2026-04-19 06:45:46 +00:00
Celes Renata ebafe795c1 fix: bump seed pod timeout to 5m and add debug diagnostics on pipeline failures 2026-04-19 06:34:58 +00:00
Celes Renata 19b63dd369 ci: migrate inttest images from GHCR to local registry, remove ghcr-credentials 2026-04-19 06:22:35 +00:00
Celes Renata e3e1531847 ci: add Docker Hub auth + proxy CA to inttest namespace, fix MinIO pull secret 2026-04-19 06:09:56 +00:00