fix: pipeline stop now halts all workers and flushes queues

Workers (ingestion, parser, extractor, aggregation, recommendation,
broker, lake-publisher) now check the pipeline:enabled Redis flag on
each loop iteration and sleep when disabled.

The toggle endpoint flushes all pipeline queues on disable so queued
jobs don't resume when workers eventually check. Broker/trading queues
are excluded from flush to avoid dropping in-flight orders.

services/shared/redis_keys.py

@@ -89,3 +89,18 @@ def trading_cb_key(trigger_type: str) -> str:
 def trading_notification_rate_key(channel: str) -> str:
     """Return the notification rate-limit key for a given channel."""
     return f"{TRADING_NOTIFICATION_RATE}:{channel}"
+
+
+# --- Pipeline toggle ---
+PIPELINE_ENABLED_KEY = f"{PREFIX}:pipeline:enabled"
+
+
+async def is_pipeline_enabled(rds: "redis.asyncio.Redis") -> bool:  # type: ignore[name-defined]  # noqa: F821
+    """Check whether the pipeline is enabled via the Redis toggle.
+
+    Returns True (enabled) when the key is absent or set to anything
+    other than ``"0"``.  Workers should call this at the top of each
+    loop iteration and sleep when it returns False.
+    """
+    val = await rds.get(PIPELINE_ENABLED_KEY)
+    return val != "0"