feat: comprehensive docs, unit tests, docker-compose app services

- Add scheduler and ingestion unit tests (test_scheduler_unit.py, test_ingestion_unit.py)
- Add all 13 app services + dashboard to docker-compose.yml
- Add full documentation suite: API reference, Helm reference, Docker deployment guide,
  3 architecture diagrams (K8s, Docker Compose, data pipeline), AI agent guide,
  backup/restore guide, observability/metrics reference, per-service docs
- Add intelligence pipeline deep-dive docs with Mermaid diagrams
- Update README with documentation index and links
- Add specs for comprehensive-quality-docs, intelligence-pipeline-deep-dive,
  sanitized-pipeline-docs

docs/intelligence-pipeline-deep-dive/index.md

@@ -0,0 +1,40 @@
+# Intelligence Pipeline Deep Dive
+
+This document series provides a narrative walkthrough of the full intelligence-to-decision pipeline in Stonks Oracle. Unlike the existing service reference and API documentation, these pages tell the story of how raw data enters the system, gets processed by AI agents, produces structured signals, accumulates into trend summaries, and ultimately drives autonomous trading decisions.
+
+Each page covers one stage of the pipeline and ends with a transition to the next, so you can read the series end-to-end or jump directly to the stage you need. Diagrams are stored as standalone Mermaid files that can be rendered independently or embedded in other documents.
+
+---
+
+## Table of Contents
+
+1. [Data Ingestion and Preparation](01-data-ingestion-and-preparation.md) — How raw data from Polygon.io, SEC EDGAR, and macro news APIs enters the system, gets deduplicated, stored, parsed, and routed for AI processing.
+2. [AI Agent Processing and Structured Extraction](02-ai-agent-processing-and-extraction.md) — How the Document Intelligence Extractor and Global Event Classifier agents use LLM inference to produce structured JSON intelligence from documents.
+3. [Signal Scoring and the WeightedSignal Abstraction](03-signal-scoring-and-weighted-signals.md) — How raw extraction output is transformed into weighted signals through confidence gating, recency decay, source credibility, novelty bonuses, and market context multipliers.
+4. [Trend Aggregation and Accumulating Signals](04-trend-aggregation-and-accumulating-signals.md) — How the aggregation engine merges weighted signals across five time windows, detects contradictions, ranks evidence, and escalates trend strength as consecutive signals accumulate.
+5. [Recommendation Generation](05-recommendation-generation.md) — How trend summaries pass through data quality suppression, eligibility evaluation, position sizing, thesis generation, and risk classification to produce actionable recommendations.
+6. [Trading Decisions and Execution](06-trading-decisions-and-execution.md) — How the trading engine polls recommendations, runs pre-trade checks, sizes positions, enforces circuit breakers, and submits orders through the broker adapter.
+
+---
+
+## Diagrams
+
+The following Mermaid diagram files can be rendered independently or referenced from the narrative pages:
+
+- [Ingestion to Extraction Flow](diagrams/ingestion-to-extraction-flow.md) — Flowchart from Scheduler through Ingestion, Parser, to Extractor with all queues and storage.
+- [Three-Layer Signal Merging](diagrams/three-layer-signal-merging.md) — Company, Macro, and Competitive signal layers converging into the Aggregation engine.
+- [Weighted Signal Computation](diagrams/weighted-signal-computation.md) — Component breakdown of the composite weight formula.
+- [Trend Accumulation and Escalation](diagrams/trend-accumulation-escalation.md) — How consecutive signals strengthen trends and escalate actions across time windows.
+- [Recommendation Generation Flow](diagrams/recommendation-generation-flow.md) — From TrendSummary through suppression, eligibility, thesis, risk classification, to persistence.
+- [Trading Engine Decision Loop](diagrams/trading-engine-decision-loop.md) — Pre-trade check sequence, position sizing, and order submission flow.
+
+---
+
+## Related Documentation
+
+For reference-level detail on individual services, AI agent configuration, and infrastructure, see the existing documentation:
+
+- [Services Reference](../services.md) — Per-service configuration, database tables, queues, and runtime behaviors.
+- [AI Agents Guide](../ai-agents.md) — AI agent configuration, variants, A/B testing, and the agent management API.
+- [Data Pipeline Architecture](../architecture-data-pipeline.md) — Queue topology, data store summary, and Mermaid flow diagrams for the full data pipeline.
+- [LLM-to-Trade Pipeline](../llm-to-trade-pipeline.md) — End-to-end data flow from model output through signal aggregation to trade execution.