feat: comprehensive docs, unit tests, docker-compose app services

- Add scheduler and ingestion unit tests (test_scheduler_unit.py, test_ingestion_unit.py)
- Add all 13 app services + dashboard to docker-compose.yml
- Add full documentation suite: API reference, Helm reference, Docker deployment guide,
  3 architecture diagrams (K8s, Docker Compose, data pipeline), AI agent guide,
  backup/restore guide, observability/metrics reference, per-service docs
- Add intelligence pipeline deep-dive docs with Mermaid diagrams
- Update README with documentation index and links
- Add specs for comprehensive-quality-docs, intelligence-pipeline-deep-dive,
  sanitized-pipeline-docs

.kiro/steering/development-process.md

@@ -30,18 +30,16 @@
 - Ruff config: `ruff.toml` with `known-first-party = ["services"]` for consistent import sorting
 - Pre-existing test failures (not regressions): `test_extractor_prompts.py`, `test_extractor_schemas.py`, `test_filings_adapter.py`, `test_ollama_client.py`
 
-## CI/CD — GitHub Actions
-- Workflow: `.github/workflows/build.yml`
-- Triggers on push to `main` and PRs
-- Jobs:
-  - `lint-and-test`: ruff lint + pytest + frontend vitest (Node 24)
-  - `build-services`: matrix build of all Python services → GHCR
-  - `build-dashboard`: frontend/Dockerfile → GHCR (TypeScript strict mode — catches unused imports)
-  - `build-superset`: docker/Dockerfile.superset → GHCR
+## CI/CD — Woodpecker CI (Gitea) → GitHub promotion
+- Woodpecker pipelines in `.woodpecker/` — triggered by push to `main` on Gitea
+- Push to Gitea: `git push gitea main`
+- Gitea remote: `http://admin:<password>@10.1.1.12:30300/admin/stonks-oracle.git`
+- Pipeline stages: lint → pytest → frontend vitest → build all service images + dashboard + superset → push to Harbor
+- ArgoCD watches Gitea `main` and auto-syncs beta/paper/live stages
+- **Do NOT push directly to GitHub** — GitHub is the promotion target after CI passes
+- Once Woodpecker builds and tests pass, code is promoted to GitHub (`git push origin main`)
 - CI handles all image builds and pushes — do NOT manually docker push
-- Check CI: `gh run list -L 3`
-- Re-run failed: `gh run rerun <id> --failed`
-- View failure logs: `gh run view <id> --log-failed`
+- Check Woodpecker CI status from the Gitea web UI or Woodpecker dashboard
 
 ## Deploy
 - Full deploy/redeploy: `bash ~/sources/kube/stonks-oracle/runmefirst.sh` (from gremlin-1)
@@ -74,7 +72,9 @@ Ingestion jobs MUST include `source_id`, `source_type`, `ticker`, `company_id`,
 ## Git Conventions
 - Commit after each completed phase task
 - Commit message format: `feat:`, `fix:`, `phase N:` prefix
-- Push to `main` triggers CI
+- Always push to Gitea: `git push gitea main`
+- Do NOT push to GitHub (`origin`) directly — GitHub is the promotion target after CI passes
+- ArgoCD syncs from Gitea automatically
 
 ## Code Style
 - Python 3.12, type hints everywhere

.kiro/steering/project-context.md

@@ -40,14 +40,17 @@ Three-layer signal aggregation engine:
 - Container registry: `registry.celestium.life/stonks-oracle`
 
 ## CI/CD
-- GitHub Actions workflow at `.github/workflows/build.yml`
-- Push to `main` triggers: lint → pytest → frontend vitest → build all service images + dashboard + superset → push to Harbor
+- Woodpecker CI pipelines in `.woodpecker/` — triggered by push to `main` on Gitea
+- Push to Gitea: `git push gitea main` — this is the primary push target
+- ArgoCD watches Gitea `main` and auto-syncs beta/paper/live stages
+- Pipeline stages: lint → pytest → frontend vitest → build all service images + dashboard + superset → push to Harbor
 - Images tagged as `registry.celestium.life/stonks-oracle/<service>:<sha>` and `:latest`
 - Dashboard image: `frontend/Dockerfile` (multi-stage: node:24 → nginx-unprivileged on port 8080)
 - Superset image: `docker/Dockerfile.superset` (apache/superset + trino + psycopg2)
 - Python service images: `docker/Dockerfile` with `SERVICE_CMD` build arg
 - Let CI handle image builds and pushes — do NOT manually `docker build && docker push`
-- Check CI status: `gh run list -L 3`
+- **Do NOT push directly to GitHub** — GitHub (`origin`) is the promotion target after CI builds and tests pass
+- Promotion to GitHub: `git push origin main` (only after Woodpecker CI succeeds)
 
 ## Deployment Scripts
 - `~/sources/kube/stonks-oracle/runmefirst.sh` — full deploy: DB setup, migrations, Helm install, rolling restart (runs from gremlin-1 at 192.168.42.254 where secrets are available)
@@ -76,9 +79,9 @@ When a full reset is needed:
 - Ollama: `ollama.ollama-service.svc.cluster.local:11434` (cluster-internal), also at `http://10.1.1.12:2701` (external), GPU: 4070 Ti Super 16GB
 
 ## Database Migrations
-- Located in `infra/migrations/001_*.sql` through `027_*.sql`
+- Located in `infra/migrations/001_*.sql` through `030_*.sql`
 - Applied automatically by `runmefirst.sh` in sorted order
-- Next migration number: **029**
+- Next migration number: **031**
 - Key migrations:
   - 016: Global news interpolation (global_events, macro_impact_records, exposure_profiles, trend_projections)
   - 017: Competitive intelligence (competitor_relationships, competitive_signal_records)