admin/stonks-oracle
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: pipeline stop now halts all workers and flushes queues
ci/woodpecker/push/test Pipeline was successful
Details
ci/woodpecker/push/build-1 Pipeline was successful
Details
ci/woodpecker/push/build-3 Pipeline was successful
Details
ci/woodpecker/push/build-2 Pipeline was successful
Details
ci/woodpecker/push/finalize Pipeline was successful
Details
Build and Push / lint-and-test (push) Has been cancelled
Details
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Details
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Details
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Details
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Details
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Details
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Details
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Details
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Details
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Details
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Details
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Details
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Details
Build and Push / build-dashboard (push) Has been cancelled
Details
Build and Push / build-superset (push) Has been cancelled
Details
Build and Push / integration-test (push) Has been cancelled
Details
Build and Push / beta-gate (push) Has been cancelled
Details

Browse Source 
Workers (ingestion, parser, extractor, aggregation, recommendation,
broker, lake-publisher) now check the pipeline:enabled Redis flag on
each loop iteration and sleep when disabled.

The toggle endpoint flushes all pipeline queues on disable so queued
jobs don't resume when workers eventually check. Broker/trading queues
are excluded from flush to avoid dropping in-flight orders.
This commit is contained in:
Celes Renata
2026-04-29 07:59:35 +00:00
parent cfcfd655e7
commit 8c3c1aab43
10 changed files with 80 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
services/scheduler/app.py
+2 -2
View File
@@ -20,7 +20,7 @@ from services.shared.config import load_config
from services.shared.db import get_pg_pool, get_redis
from services.shared.logging import setup_logging
from services.shared.redis_keys import (
PREFIX,
PIPELINE_ENABLED_KEY,
QUEUE_EXTRACTION,
QUEUE_INGESTION,
QUEUE_MACRO_CLASSIFICATION,
@@ -501,7 +501,7 @@ async def main() -> None:
rds = get_redis(config)
logger.info("Scheduler started (tick=%ds)", SCHEDULER_TICK)
pipeline_key = f"{PREFIX}:pipeline:enabled"
pipeline_key = PIPELINE_ENABLED_KEY
# If PIPELINE_DEFAULT_OFF is set, initialize the toggle to OFF on first boot
# (only if the key doesn't already exist — preserves manual overrides)