stonks-oracle/.kiro/specs/local-cicd-pipeline/tasks.md

# Implementation Plan: Local CI/CD Pipeline

## Overview

Migrate the Stonks Oracle CI/CD pipeline from GitHub-dependent infrastructure (ARC + GHCR) to a fully local pipeline using Gitea, Woodpecker CI, and the local Docker registry. Update ArgoCD and Kargo to source from local infrastructure. All pipeline scripts and manifests live in `~/sources/kube/pipelines/` on gremlin-1.

## Tasks

- [x] 1. Tear down ARC infrastructure
  - [x] 1.1 Remove `pipelines/arc/` directory (values.yaml, runner-scaleset.yaml, runner-rbac.yaml)
    - _Requirements: 7.1, 7.2, 7.4_
  - [x] 1.2 Remove `pipelines/pvs/arc-pv.yaml` — ARC NFS PV no longer needed
    - _Requirements: 7.5_
  - [x] 1.3 Update `pipelines/runmefirst.sh` — add ARC teardown section at the beginning: uninstall `arc-runner-set` and `arc` Helm releases, delete `arc/runner-rbac.yaml` ClusterRoleBinding, delete `pipeline-arc-pv` PV, delete `arc-system` namespace. Use `|| true` and `--ignore-not-found` for idempotency.
    - _Requirements: 7.1, 7.2, 7.3, 7.4, 7.5, 11.7_

- [x] 2. Create Woodpecker CI NFS PersistentVolume
  - [x] 2.1 Create `pipelines/pvs/woodpecker-pv.yaml` — NFS PV for Woodpecker server data (5Gi, `nfs://192.168.42.8:/volume1/Kubernetes/pipelines/woodpecker`, `persistentVolumeReclaimPolicy: Retain`, label `app: pipeline-woodpecker`)
    - _Requirements: 24.1, 24.4_

- [x] 3. Create Gitea configuration script
  - [x] 3.1 Create `pipelines/gitea/setup.sh` — Shell script that automates Gitea initial setup via REST API: complete install wizard or create admin user, create OAuth2 application for Woodpecker CI (callback URL `https://stonks-ci.celestium.life/authorize`), create `stonks-oracle` repository. Script outputs OAuth2 client_id and client_secret for use by Woodpecker Helm install. All operations idempotent (check-before-create pattern).
    - _Requirements: 1.1, 1.2, 1.5, 2.3_

- [x] 4. Create Woodpecker CI manifests
  - [x] 4.1 Create `pipelines/woodpecker/values.yaml` — Helm values for `woodpecker/woodpecker` chart in `woodpecker` namespace. Server: Gitea OAuth2 config (`WOODPECKER_GITEA=true`, `WOODPECKER_GITEA_URL`, `WOODPECKER_GITEA_CLIENT`, `WOODPECKER_GITEA_SECRET`), `WOODPECKER_HOST=https://stonks-ci.celestium.life`, `WOODPECKER_ADMIN=admin`, Traefik ingress at `stonks-ci.celestium.life` with TLS via `ca-issuer`, NFS-backed persistent volume (5Gi). Agent: 2 replicas, Kubernetes backend (`WOODPECKER_BACKEND=kubernetes`), `WOODPECKER_BACKEND_K8S_NAMESPACE=woodpecker`, `WOODPECKER_BACKEND_K8S_VOLUME_SIZE=10G`.
    - _Requirements: 2.1, 2.2, 2.3, 2.5, 11.1, 11.5, 28.1_
  - [x] 4.2 Create `pipelines/woodpecker/network-policy.yaml` — NetworkPolicy in `woodpecker` namespace allowing Traefik ingress (from `kube-system` namespace) to Woodpecker server on HTTP port
    - _Requirements: 28.5_
  - [x] 4.3 Create `pipelines/woodpecker/agent-rbac.yaml` — ClusterRoleBinding granting the Woodpecker agent service account `cluster-admin` for integration test steps that create ephemeral namespaces. Mirrors the existing ARC runner RBAC pattern.
    - _Requirements: 26.1, 26.2, 26.3_

- [x] 5. Checkpoint — Verify Woodpecker and Gitea manifests
  - Ensure all YAML manifests are syntactically valid. Verify Woodpecker Helm values include correct Gitea OAuth2 configuration. Verify NFS PV paths are distinct from ArgoCD and Kargo PVs. Ask the user if questions arise.

- [x] 6. Update ArgoCD manifests for Gitea
  - [x] 6.1 Update `pipelines/argocd/repo-secret.yaml` — Change repository URL from `https://github.com/celesrenata/stonks-oracle.git` to the Gitea repository URL (`http://gitea-http.git-server.svc.cluster.local:3000/admin/stonks-oracle.git`). Update credentials to use Gitea admin username/password instead of GitHub token.
    - _Requirements: 25.1, 9.4_
  - [x] 6.2 Update `pipelines/argocd/apps/stonks-beta.yaml` — Change `spec.source.repoURL` from GitHub to Gitea repository URL
    - _Requirements: 9.1, 25.3_
  - [x] 6.3 Update `pipelines/argocd/apps/stonks-paper.yaml` — Change `spec.source.repoURL` from GitHub to Gitea repository URL
    - _Requirements: 9.2, 25.3_
  - [x] 6.4 Update `pipelines/argocd/apps/stonks-live.yaml` — Change `spec.source.repoURL` from GitHub to Gitea repository URL
    - _Requirements: 9.3, 25.3_

- [x] 7. Update Kargo Warehouse for local registry
  - [x] 7.1 Update `pipelines/kargo/warehouse.yaml` — Change `spec.subscriptions[0].image.repoURL` from `ghcr.io/celesrenata/stonks-oracle/query-api` to `registry.celestium.life/stonks-oracle/query-api`
    - _Requirements: 8.1, 8.2_

- [x] 8. Update Helm chart for local registry
  - [x] 8.1 Update `infra/helm/stonks-oracle/values.yaml` — Change `image.registry` from `ghcr.io/celesrenata/stonks-oracle` to `registry.celestium.life/stonks-oracle`. Remove `imagePullSecrets` list (remove `ghcr-credentials` entry). Remove `ghcrAuth` section entirely.
    - _Requirements: 10.1, 10.2, 10.4_

- [x] 9. Checkpoint — Verify ArgoCD, Kargo, and Helm chart updates
  - Ensure all updated YAML is syntactically valid. Verify no references to `ghcr.io` or `github.com` remain in ArgoCD apps, Kargo warehouse, or Helm values. Verify `values-beta.yaml` and `values-paper.yaml` don't need changes (they inherit `image.registry` from base). Ask the user if questions arise.

- [x] 10. Create Woodpecker pipeline file
  - [x] 10.1 Create `.woodpecker.yml` in the repository root — Define all CI pipeline steps in Woodpecker's native YAML format: lint-python (ruff check), test-python (pytest), test-frontend (vitest), build steps for all 12 Python services using `woodpeckerci/plugin-docker-buildx` pushing to `registry.celestium.life/stonks-oracle/<service>:<sha>` and `:latest`, build-dashboard, build-superset, integration-test (run_pipeline.sh), and mirror-github. Use `when` conditions to restrict build/push/mirror steps to `main` branch push events. Use `depends_on` for step ordering. Use `from_secret` for registry and GitHub credentials.
    - _Requirements: 14.1, 14.2, 14.3, 14.4, 14.5, 27.1, 27.2, 27.3, 27.4, 27.5, 3.1, 3.2, 3.3, 3.4, 3.5, 4.1, 4.2, 4.3, 4.4, 4.5, 5.1, 5.2, 5.3, 5.4, 5.5, 6.1, 6.2, 6.3, 6.4, 6.5_

- [x] 11. Update install script (`runmefirst.sh`)
  - [x] 11.1 Rewrite `pipelines/runmefirst.sh` — New install order: (1) ARC teardown, (2) create namespaces (`woodpecker`, `argocd`, `kargo`, `stonks-beta`, `stonks-paper`), (3) apply NFS PVs (argocd, kargo, woodpecker), (4) run Gitea setup script (`gitea/setup.sh`), (5) add Woodpecker Helm repo and install Woodpecker with Gitea OAuth2 credentials injected, (6) apply Woodpecker network policy and agent RBAC, (7) install ArgoCD via Helm with updated values, apply repo secret and Applications, (8) install Kargo via Helm, apply project/warehouse/stages. Use `set -euo pipefail`, idempotent operations throughout.
    - _Requirements: 11.1, 11.2, 11.3, 11.4, 11.5, 11.6, 11.7, 1.1, 1.2, 1.4, 1.5_

- [x] 12. Update teardown script (`runmelast.sh`)
  - [x] 12.1 Rewrite `pipelines/runmelast.sh` — New teardown order: (1) remove Kargo resources + Helm release, (2) remove ArgoCD resources + Helm release, (3) uninstall Woodpecker Helm release, delete Woodpecker agent RBAC and network policy, (4) delete namespaces (`woodpecker`, `argocd`, `kargo`). Preserve: NFS PVs, NFS data, `stonks-oracle` namespace, `stonks-beta`, `stonks-paper`, `git-server` namespace (Gitea + registry). Use `--ignore-not-found` and `|| true` for idempotency.
    - _Requirements: 12.1, 12.2, 12.3, 12.4, 12.5, 13.1, 13.2, 13.3_

- [x] 13. Checkpoint — Verify all scripts and pipeline file
  - Ensure `runmefirst.sh` install order matches design (ARC teardown → Gitea → Woodpecker → ArgoCD → Kargo). Ensure `runmelast.sh` teardown order is reverse (Kargo → ArgoCD → Woodpecker). Verify `.woodpecker.yml` covers all 12 services + dashboard + superset + integration test + GitHub mirror. Verify no references to ARC, `arc-system`, or GHCR remain in any pipeline scripts. Ask the user if questions arise.

- [x] 14. Final review
  - Review all created and modified files for completeness. Verify the full pipeline flow: Gitea push → Woodpecker webhook → lint/test → build/push to local registry → integration test → GitHub mirror → Kargo detects new tag → beta auto-promote → paper/live with market-hours gates. Ensure all 28 requirements are addressed. Ask the user if questions arise.

## Notes

- Pipeline infrastructure scripts (`~/sources/kube/pipelines/`) are on gremlin-1, separate from the stonks-oracle repo
- The `.woodpecker.yml` pipeline file and Helm chart changes are in the stonks-oracle repo
- No property-based tests — this feature is entirely IaC (shell scripts, YAML manifests, Helm values)
- Gitea is already deployed in `git-server` namespace but unconfigured (still on install page)
- Local registry is already deployed and healthy at `registry.celestium.life`
- ArgoCD and Kargo are already deployed — only config updates needed (repo URL, image source)
- ARC is currently deployed and needs to be torn down before Woodpecker install
- The `.github/workflows/build.yml` remains in the repo for reference but won't be used by Woodpecker
- Woodpecker agents use the Kubernetes backend — each pipeline step runs as a standalone Pod
- Image builds use `woodpeckerci/plugin-docker-buildx` with privileged mode
- GitHub mirror is a post-CI step that pushes via SSH key — failure does not block promotion
- Break-glass, audit trail, and Kargo Dashboard features are provided by Kargo out of the box (Requirements 19, 20, 21, 22, 23)
- Market-hours AnalysisTemplate is unchanged from the existing pipeline
- Kargo stages, project, and project-config are unchanged
- `values-beta.yaml` and `values-paper.yaml` inherit `image.registry` from base `values.yaml` — no changes needed
- k3s nodes may need containerd mirror config to trust the local registry for image pulls