Files

31 KiB
Raw Permalink Blame History

Local CI/CD Pipeline — Requirements

Introduction

Fully local CI/CD pipeline for the Stonks Oracle platform that eliminates all dependency on GitHub's API for CI/CD orchestration. Gitea replaces GitHub as the primary Git remote, Woodpecker CI replaces ARC for CI execution and pipeline orchestration, and the local Docker registry at registry.celestium.life replaces GHCR for image storage. Woodpecker CI connects to Gitea via OAuth2 for authentication and receives push/PR webhooks to trigger pipelines. Woodpecker agents execute pipeline steps as Docker containers on the Gremlin Cluster. GitHub becomes a read-only mirror updated only after successful CI runs. The existing ArgoCD and Kargo infrastructure is retained for GitOps deployment and staged promotion, with image sources updated to point at the local registry. All pipeline infrastructure scripts reside in ~/sources/kube/pipelines/ on gremlin-1 and persist state on NFS volumes that survive cluster rebuilds.

Glossary

  • Gitea: A self-hosted Git forge running in the git-server namespace at 10.1.1.x:30300 (web) and :30022 (SSH), providing Git hosting and code review
  • Woodpecker_CI: A CI/CD server that receives webhooks from Gitea and orchestrates pipelines, deployed as a Kubernetes Deployment in the woodpecker namespace, accessible via the Woodpecker_Dashboard at stonks-ci.celestium.life
  • Woodpecker_Agent: The worker component of Woodpecker CI that connects to the Woodpecker_CI server and executes pipeline steps as Docker containers on the Gremlin_Cluster
  • Woodpecker_Dashboard: The Woodpecker CI web UI at stonks-ci.celestium.life for viewing pipeline status, build logs, and managing repository settings
  • Local_Registry: The Docker container registry running in the git-server namespace at registry.celestium.life (HTTPS via Traefik) and :30500 (NodePort), backed by persistent storage at /kubedata-local/registry
  • GitHub_Mirror: A post-CI pipeline step that pushes the repository to GitHub for public visibility after all CI checks pass, using git push --mirror or equivalent
  • ArgoCD: A GitOps continuous delivery controller for Kubernetes that syncs cluster state from Git repositories
  • Kargo: A promotion orchestration layer built on top of ArgoCD providing staged promotion gates, a visual web dashboard, and audit trails
  • Pipeline_Infrastructure: The set of Kubernetes resources (Gitea, Woodpecker_CI, Woodpecker_Agent, Local_Registry, ArgoCD, Kargo) and their supporting manifests, PVs, and scripts that comprise the CI/CD system, deployed from ~/sources/kube/pipelines/
  • Promotion: The act of advancing a specific Image_Tag from one pipeline stage to the next (e.g., beta to paper)
  • Promotion_Blocker: A time-based gate that prevents promotions during US equity market hours (09:3016:00 ET, MondayFriday)
  • Break_Glass: An emergency override mechanism that bypasses the Promotion_Blocker, requiring explicit confirmation and an audit note
  • Stage: One of the deployment environments in the pipeline: CI, Beta, Paper, Live
  • NFS_PV: A Kubernetes PersistentVolume backed by the NFS share at nfs://192.168.42.8:/volume1/Kubernetes/pipelines, used to persist pipeline state across cluster rebuilds
  • Image_Tag: A Docker image tag in the format <sha> (Git commit SHA) used to identify a specific build across all stages
  • Gremlin_Cluster: The 4-node NixOS Kubernetes cluster (gremlin-1 through gremlin-4) at primary address 192.168.42.254
  • Market_Hours: US equity market trading hours, 09:3016:00 Eastern Time, Monday through Friday
  • Kargo_Dashboard: The Kargo web UI providing visual promotion management, stage status, and audit history
  • ReviewBoard: The code review tool running in the docker-reviewboard namespace at cr.celestium.life, used as an optional review gate before merge
  • Integration_Test_Runner: The existing standalone script at infra/inttest/run_pipeline.sh that deploys an ephemeral sandbox, seeds data, runs API tests, and produces inttest-results.json
  • Woodpecker_Pipeline_File: The .woodpecker.yml file in the repository root that defines CI pipeline steps in Woodpecker's YAML format, where each step is a Docker container with commands

Requirements

Requirement 1: Gitea Configuration and Repository Setup

User Story: As a platform operator, I want Gitea configured with an admin user, the stonks-oracle repository, and webhook integration with Woodpecker CI, so that developers can push code to a fully local Git forge that triggers CI pipelines.

Acceptance Criteria

  1. WHEN the operator executes the pipeline install script, THE Pipeline_Infrastructure SHALL configure Gitea with an admin user account and complete the initial setup
  2. WHEN Gitea is configured, THE Pipeline_Infrastructure SHALL create a stonks-oracle repository in Gitea with webhook integration to Woodpecker_CI
  3. THE Gitea SHALL be accessible via the web UI at 10.1.1.x:30300 and via SSH at :30022 for Git operations
  4. WHEN a developer pushes code to the Gitea stonks-oracle repository, THE Gitea SHALL send a webhook event to Woodpecker_CI to trigger the matching pipeline
  5. THE Pipeline_Infrastructure SHALL store Gitea configuration scripts and manifests in ~/sources/kube/pipelines/

Requirement 2: Woodpecker CI Server and Agent Deployment

User Story: As a platform operator, I want Woodpecker CI server and agents deployed on the Gremlin_Cluster and connected to Gitea via OAuth2, so that CI pipelines execute locally on cluster resources without any external dependency.

Acceptance Criteria

  1. WHEN the pipeline install script executes, THE Pipeline_Infrastructure SHALL deploy the Woodpecker_CI server as a Kubernetes Deployment in the woodpecker namespace on the Gremlin_Cluster
  2. WHEN the pipeline install script executes, THE Pipeline_Infrastructure SHALL deploy at least one Woodpecker_Agent as a Kubernetes Deployment in the woodpecker namespace
  3. WHEN Woodpecker_CI is deployed, THE Woodpecker_CI SHALL authenticate with Gitea via OAuth2 and register webhooks for the stonks-oracle repository
  4. WHEN a webhook event is received from Gitea, THE Woodpecker_CI SHALL schedule the pipeline on an available Woodpecker_Agent
  5. THE Woodpecker_Agent SHALL execute each pipeline step as a Docker container on the Gremlin_Cluster
  6. WHEN a pipeline completes, THE Woodpecker_Agent SHALL release cluster resources used by that pipeline's containers

Requirement 3: Woodpecker CI Pipeline — Lint and Test

User Story: As a developer, I want every push to main or pull request to trigger automated linting and testing via Woodpecker CI, so that code quality is validated locally before images are built.

Acceptance Criteria

  1. WHEN a push to the main branch or a pull request is opened in Gitea, THE Woodpecker_CI SHALL trigger the pipeline defined in .woodpecker.yml on a Woodpecker_Agent
  2. WHEN the pipeline runs, THE Woodpecker_CI SHALL execute Python linting using ruff check services/
  3. WHEN the pipeline runs, THE Woodpecker_CI SHALL execute Python unit tests using pytest tests/
  4. WHEN the pipeline runs, THE Woodpecker_CI SHALL install frontend dependencies and execute frontend tests using vitest
  5. IF any lint or test step fails, THEN THE Woodpecker_CI SHALL mark the pipeline as failed and skip image build steps

Requirement 4: Woodpecker CI Pipeline — Image Build and Push to Local Registry

User Story: As a developer, I want Docker images for all services and the dashboard to be built and pushed to the Local_Registry on every successful main branch push, so that new images are available for local deployment without depending on GHCR.

Acceptance Criteria

  1. WHEN lint and tests pass on a push to main, THE Woodpecker_CI SHALL build Docker images for all 12 Python services (scheduler, symbol-registry, ingestion, parser, extractor, aggregation, recommendation, risk, broker-adapter, lake-publisher, query-api, trading-engine) using the woodpeckerci/plugin-docker-buildx plugin
  2. WHEN lint and tests pass on a push to main, THE Woodpecker_CI SHALL build the dashboard Docker image from frontend/Dockerfile
  3. WHEN lint and tests pass on a push to main, THE Woodpecker_CI SHALL build the superset Docker image from docker/Dockerfile.superset
  4. WHEN images are built, THE Woodpecker_CI SHALL push each image to the Local_Registry with tags registry.celestium.life/stonks-oracle/<service>:<git-sha> and registry.celestium.life/stonks-oracle/<service>:latest
  5. WHEN all images are pushed, THE Woodpecker_CI SHALL record the Git SHA as the Image_Tag for downstream stages

Requirement 5: Integration Test Stage

User Story: As a developer, I want the CI pipeline to automatically run integration tests against newly built images, so that functional correctness is validated before promotion to beta.

Acceptance Criteria

  1. WHEN all images are pushed to the Local_Registry for a given Image_Tag, THE Woodpecker_CI SHALL invoke the Integration_Test_Runner with bash infra/inttest/run_pipeline.sh --image-tag <sha>
  2. WHEN the Integration_Test_Runner completes, THE Woodpecker_CI SHALL evaluate the inttest-results.json file for test counts and exit code
  3. IF the Integration_Test_Runner exits with code 0, THEN THE Woodpecker_CI SHALL mark the Image_Tag as eligible for promotion to Beta
  4. IF the Integration_Test_Runner exits with a non-zero code, THEN THE Woodpecker_CI SHALL block promotion to Beta and report the failure details in the Woodpecker_Dashboard
  5. THE Woodpecker_CI SHALL store the inttest-results.json as a pipeline artifact accessible from the Woodpecker_Dashboard

Requirement 6: GitHub Mirror Push After Successful CI

User Story: As a developer, I want the repository automatically mirrored to GitHub after all CI checks pass, so that the codebase remains publicly visible on GitHub without GitHub being in the critical CI/CD path.

Acceptance Criteria

  1. WHEN all pipeline steps (lint, test, image build, integration test) pass on a push to main, THE Woodpecker_CI SHALL push the repository to the GitHub remote at github.com/celesrenata/stonks-oracle
  2. IF any pipeline step fails, THEN THE Woodpecker_CI SHALL skip the GitHub mirror push
  3. THE GitHub_Mirror step SHALL use stored Git credentials (SSH key or token) managed via Woodpecker secrets to authenticate with GitHub
  4. THE GitHub_Mirror step SHALL push all branches and tags to the GitHub remote
  5. THE Pipeline_Infrastructure SHALL have zero dependency on GitHub API availability for CI/CD orchestration — GitHub mirror failure SHALL NOT block image promotion or deployment

Requirement 7: ARC Teardown

User Story: As a platform operator, I want ARC (GitHub Actions Runner Controller) removed from the cluster, so that the deprecated GitHub-dependent CI runner infrastructure is cleaned up.

Acceptance Criteria

  1. WHEN the pipeline install script executes, THE Pipeline_Infrastructure SHALL remove the ARC controller Helm release from the arc-system namespace
  2. WHEN the pipeline install script executes, THE Pipeline_Infrastructure SHALL remove the ARC runner scale set Helm release from the arc-system namespace
  3. WHEN ARC is removed, THE Pipeline_Infrastructure SHALL delete the arc-system namespace
  4. WHEN ARC is removed, THE Pipeline_Infrastructure SHALL remove the ARC runner RBAC ClusterRoleBinding
  5. THE Pipeline_Infrastructure SHALL remove the ARC NFS PersistentVolume (pipeline-arc-pv) since ARC data is no longer needed

Requirement 8: Update Kargo Warehouse to Watch Local Registry

User Story: As a platform operator, I want the Kargo Warehouse to watch the Local_Registry instead of GHCR for new image tags, so that the promotion pipeline sources images from the local infrastructure.

Acceptance Criteria

  1. THE Kargo Warehouse stonks-images SHALL subscribe to image tags at registry.celestium.life/stonks-oracle/query-api instead of ghcr.io/celesrenata/stonks-oracle/query-api
  2. WHEN a new Image_Tag is pushed to the Local_Registry, THE Kargo Warehouse SHALL detect the new tag and create a Freight resource
  3. IF the Local_Registry is temporarily unavailable, THEN THE Kargo Warehouse SHALL retry image discovery and report the error in the Kargo_Dashboard

Requirement 9: Update ArgoCD Applications to Use Local Registry

User Story: As a platform operator, I want ArgoCD Applications to pull images from the Local_Registry instead of GHCR, so that all deployments source images from local infrastructure.

Acceptance Criteria

  1. THE ArgoCD Application stonks-beta SHALL deploy using images from registry.celestium.life/stonks-oracle/ instead of ghcr.io/celesrenata/stonks-oracle/
  2. THE ArgoCD Application stonks-paper SHALL deploy using images from registry.celestium.life/stonks-oracle/ instead of ghcr.io/celesrenata/stonks-oracle/
  3. THE ArgoCD Application stonks-live SHALL deploy using images from registry.celestium.life/stonks-oracle/ instead of ghcr.io/celesrenata/stonks-oracle/
  4. THE ArgoCD Applications SHALL source the Helm chart from the Gitea repository instead of the GitHub repository
  5. WHEN ArgoCD syncs an Application, THE ArgoCD SHALL pull images from the Local_Registry using the Image_Tag set by Kargo during promotion

Requirement 10: Update Helm Chart for Local Registry

User Story: As a platform operator, I want the Helm chart's default image registry updated to the Local_Registry, so that all Kubernetes deployments pull images locally without requiring GHCR credentials.

Acceptance Criteria

  1. THE Helm chart values.yaml SHALL set image.registry to registry.celestium.life/stonks-oracle instead of ghcr.io/celesrenata/stonks-oracle
  2. THE Helm chart SHALL remove the ghcrAuth section and ghcr-credentials imagePullSecret since the Local_Registry does not require authentication
  3. THE Helm chart values-beta.yaml and values-paper.yaml SHALL reference images from the Local_Registry
  4. WHEN a Helm deployment is executed, THE Helm chart SHALL pull all service images from registry.celestium.life/stonks-oracle/<service>:<tag>

Requirement 11: Pipeline Infrastructure Deployment

User Story: As a platform operator, I want a single deployment script that installs all local CI/CD pipeline components (configures Gitea, deploys Woodpecker CI, updates ArgoCD and Kargo) onto the Gremlin_Cluster, so that the pipeline infrastructure can be stood up or rebuilt with one command.

Acceptance Criteria

  1. WHEN the operator executes runmefirst.sh from ~/sources/kube/pipelines/, THE Pipeline_Infrastructure SHALL configure Gitea, deploy Woodpecker_CI server and Woodpecker_Agent, install ArgoCD, and install Kargo into the Gremlin_Cluster in dedicated namespaces
  2. WHEN the operator executes runmefirst.sh, THE Pipeline_Infrastructure SHALL create NFS-backed PersistentVolumes at nfs://192.168.42.8:/volume1/Kubernetes/pipelines for ArgoCD, Kargo, and Woodpecker persistent data
  3. WHEN ArgoCD is deployed, THE Pipeline_Infrastructure SHALL expose the ArgoCD web UI via Traefik ingress with TLS using the ca-issuer ClusterIssuer
  4. WHEN Kargo is deployed, THE Pipeline_Infrastructure SHALL expose the Kargo_Dashboard via Traefik ingress with TLS using the ca-issuer ClusterIssuer
  5. WHEN Woodpecker_CI is deployed, THE Pipeline_Infrastructure SHALL expose the Woodpecker_Dashboard via Traefik ingress with TLS using the ca-issuer ClusterIssuer at stonks-ci.celestium.life
  6. THE Pipeline_Infrastructure SHALL store all deployment manifests and scripts in ~/sources/kube/pipelines/ on gremlin-1
  7. WHEN runmefirst.sh executes, THE Pipeline_Infrastructure SHALL tear down ARC components (controller, runner scale set, namespace, RBAC, PV) before installing local CI components

Requirement 12: Pipeline Infrastructure Teardown

User Story: As a platform operator, I want a teardown script that removes pipeline components without destroying persistent pipeline data, so that pipeline state survives cluster rebuilds.

Acceptance Criteria

  1. WHEN the operator executes runmelast.sh from ~/sources/kube/pipelines/, THE Pipeline_Infrastructure SHALL remove Woodpecker_CI server, Woodpecker_Agent, ArgoCD, and Kargo deployments from the Gremlin_Cluster
  2. WHEN runmelast.sh executes, THE Pipeline_Infrastructure SHALL preserve all NFS_PV resources and the data stored on nfs://192.168.42.8:/volume1/Kubernetes/pipelines
  3. WHEN runmelast.sh executes, THE Pipeline_Infrastructure SHALL leave the stonks-oracle application namespace and all application workloads untouched
  4. WHEN runmelast.sh executes, THE Pipeline_Infrastructure SHALL leave the git-server namespace (Gitea and Local_Registry) untouched since those are managed separately
  5. WHEN the application teardown script ~/sources/kube/stonks-oracle/runmelast.sh executes, THE Pipeline_Infrastructure SHALL remain operational and unaffected

Requirement 13: Pipeline Infrastructure Isolation

User Story: As a platform operator, I want the pipeline infrastructure to be fully isolated from the application infrastructure, so that deploying or tearing down one does not affect the other.

Acceptance Criteria

  1. THE Pipeline_Infrastructure SHALL deploy Woodpecker_CI, ArgoCD, and Kargo in namespaces separate from the stonks-oracle application namespace and the git-server namespace
  2. THE Pipeline_Infrastructure SHALL use independent Helm releases or manifests that share no lifecycle with the stonks-oracle Helm chart
  3. THE Pipeline_Infrastructure SHALL use NFS_PV paths under pipelines/ that are distinct from any application storage paths

Requirement 14: Woodpecker CI Pipeline File

User Story: As a developer, I want the existing CI pipeline translated into Woodpecker's .woodpecker.yml format, so that the pipeline runs natively on Woodpecker CI without GitHub Actions dependencies.

Acceptance Criteria

  1. THE Pipeline_Infrastructure SHALL create a .woodpecker.yml file in the repository root defining all CI pipeline steps in Woodpecker's native YAML format
  2. THE Woodpecker_Pipeline_File SHALL define each pipeline step as a Docker container with explicit image and commands (no GitHub Actions uses: syntax)
  3. THE Woodpecker_Pipeline_File SHALL use when conditions to restrict image build and push steps to pushes on the main branch
  4. THE Woodpecker_Pipeline_File SHALL use the woodpeckerci/plugin-docker-buildx plugin for building and pushing Docker images to the Local_Registry
  5. WHEN the .woodpecker.yml is committed, THE Woodpecker_CI SHALL execute the pipeline for all 12 Python services, the dashboard, and the superset image
  6. THE Woodpecker_Pipeline_File SHALL include a GitHub mirror step that runs only after all other steps succeed on the main branch

Requirement 15: Beta Stage Deployment

User Story: As a developer, I want a beta environment where newly built images are deployed for smoke testing and manual verification before promotion to paper trading, so that regressions are caught early.

Acceptance Criteria

  1. WHEN an Image_Tag passes the integration test stage, THE Beta_Stage SHALL deploy the application with that Image_Tag to the stonks-beta namespace managed by ArgoCD
  2. WHILE the Beta_Stage is active, THE Kargo_Dashboard SHALL display the currently deployed Image_Tag and its promotion status
  3. WHEN a developer requests promotion from Beta to Paper via the Kargo_Dashboard, THE Beta_Stage SHALL verify that the Image_Tag passed integration tests before allowing promotion
  4. THE Beta_Stage SHALL use the same Helm chart (infra/helm/stonks-oracle/) as production, with beta-specific value overrides pulling images from the Local_Registry

Requirement 16: Paper Trading Stage Deployment

User Story: As a trader, I want a paper trading environment that uses the Alpaca paper broker, so that new builds can be validated against simulated market conditions before going live.

Acceptance Criteria

  1. WHEN an Image_Tag is promoted from Beta, THE Paper_Stage SHALL deploy the application with that Image_Tag to the stonks-paper namespace managed by ArgoCD
  2. THE Paper_Stage SHALL configure the broker adapter with BROKER_MODE=paper and BROKER_PROVIDER=alpaca using Alpaca paper trading credentials
  3. WHILE Market_Hours are active (09:3016:00 ET, MondayFriday), THE Paper_Stage SHALL block automatic and manual promotions to the Paper_Stage unless Break_Glass is activated
  4. WHEN a promotion to Paper is attempted outside Market_Hours, THE Paper_Stage SHALL allow the promotion to proceed
  5. THE Paper_Stage SHALL use the same Helm chart (infra/helm/stonks-oracle/) as production, with paper-specific value overrides pulling images from the Local_Registry

Requirement 17: Live Stage Deployment

User Story: As a platform operator, I want production deployments to require explicit manual approval with notes, so that live trading is protected from accidental or untested deployments.

Acceptance Criteria

  1. WHEN an Image_Tag is promoted from Paper, THE Live_Stage SHALL require explicit manual approval with a notes field before deploying to the stonks-oracle production namespace
  2. THE Live_Stage SHALL deploy the application with the approved Image_Tag via ArgoCD syncing the production Helm release
  3. WHILE Market_Hours are active (09:3016:00 ET, MondayFriday), THE Live_Stage SHALL block promotions to the Live_Stage unless Break_Glass is activated
  4. WHEN a promotion to Live is attempted outside Market_Hours with valid approval, THE Live_Stage SHALL allow the promotion to proceed
  5. THE Live_Stage SHALL use the existing stonks-oracle namespace and Helm chart with production values pulling images from the Local_Registry

Requirement 18: Market-Hours Promotion Blocker

User Story: As a risk manager, I want promotions to paper and live environments to be blocked during US market hours, so that deployments do not disrupt active trading sessions.

Acceptance Criteria

  1. WHILE the current time is between 09:30 and 16:00 Eastern Time on a weekday, THE Promotion_Blocker SHALL prevent promotions to the Paper_Stage and Live_Stage
  2. WHEN the current time is outside 09:3016:00 ET or on a weekend, THE Promotion_Blocker SHALL allow promotions to proceed (subject to other gates)
  3. WHEN a promotion is blocked by the Promotion_Blocker, THE Kargo_Dashboard SHALL display a visual indicator showing the block reason and the time until the market closes
  4. THE Promotion_Blocker SHALL evaluate Eastern Time correctly, accounting for US daylight saving time transitions

Requirement 19: Break-Glass Emergency Override

User Story: As a platform operator, I want a break-glass mechanism to bypass market-hours blockers during emergencies, so that critical fixes can be deployed at any time.

Acceptance Criteria

  1. WHEN an operator activates Break_Glass via the Kargo_Dashboard, THE Pipeline_Infrastructure SHALL bypass the Promotion_Blocker for the target Stage
  2. WHEN Break_Glass is activated, THE Kargo_Dashboard SHALL require a confirmation dialog before proceeding
  3. WHEN Break_Glass is activated, THE Pipeline_Infrastructure SHALL require the operator to provide a written justification note
  4. WHEN Break_Glass is used, THE Pipeline_Infrastructure SHALL record the operator identity, timestamp, target Stage, Image_Tag, and justification note in the audit trail
  5. THE Break_Glass mechanism SHALL apply only to the single promotion for which it was activated and SHALL NOT disable the Promotion_Blocker for subsequent promotions

Requirement 20: Per-Stage Enable/Disable Controls

User Story: As a platform operator, I want to independently enable or disable each pipeline stage, so that the pipeline can be configured for different operational modes.

Acceptance Criteria

  1. THE Pipeline_Infrastructure SHALL provide a configuration mechanism to independently enable or disable each of the four stages (CI, Beta, Paper, Live)
  2. WHEN a Stage is disabled, THE Pipeline_Infrastructure SHALL skip that Stage during promotion and advance the Image_Tag to the next enabled Stage
  3. WHEN a Stage is re-enabled, THE Pipeline_Infrastructure SHALL resume gating promotions through that Stage for new Image_Tags

Requirement 21: Revision Tracking

User Story: As a developer, I want to see which Image_Tag (Git SHA) is deployed at each pipeline stage, so that I can track exactly what code is running in each environment.

Acceptance Criteria

  1. THE Kargo_Dashboard SHALL display the currently deployed Image_Tag for each active Stage
  2. WHEN a promotion occurs, THE Kargo_Dashboard SHALL update the displayed Image_Tag for the target Stage within 60 seconds
  3. THE Pipeline_Infrastructure SHALL maintain a mapping of Stage to current Image_Tag that is queryable via the Kargo API or ArgoCD

Requirement 22: Audit Trail

User Story: As a compliance officer, I want a complete audit trail of all promotions including who promoted, when, with what notes, and whether break-glass was used, so that deployment decisions are traceable.

Acceptance Criteria

  1. WHEN a promotion occurs, THE Pipeline_Infrastructure SHALL record the operator identity, timestamp, source Stage, target Stage, Image_Tag, and any notes provided
  2. WHEN Break_Glass is used for a promotion, THE Pipeline_Infrastructure SHALL record the break-glass justification alongside the standard promotion record
  3. THE Kargo_Dashboard SHALL display the promotion history for each Stage, showing all recorded audit fields
  4. THE Pipeline_Infrastructure SHALL persist audit trail data on NFS_PV so that promotion history survives cluster rebuilds

Requirement 23: Kargo Visual Dashboard

User Story: As a platform operator, I want a web dashboard showing all pipeline stages, their current revisions, and promotion controls, so that I can manage deployments visually.

Acceptance Criteria

  1. THE Kargo_Dashboard SHALL display all active Stages with their current deployed Image_Tag and promotion status
  2. THE Kargo_Dashboard SHALL provide a click-to-promote action for advancing an Image_Tag from one Stage to the next
  3. WHEN Market_Hours are active, THE Kargo_Dashboard SHALL display block/allow indicators on the Paper_Stage and Live_Stage
  4. THE Kargo_Dashboard SHALL provide a notes field when promoting or when a promotion is blocked
  5. THE Kargo_Dashboard SHALL provide a Break_Glass button with a confirmation dialog for emergency overrides
  6. THE Kargo_Dashboard SHALL be accessible via Traefik ingress at stonks-kargo.celestium.life with TLS via ca-issuer

Requirement 24: NFS Persistent Storage

User Story: As a platform operator, I want all pipeline state (ArgoCD app configs, Kargo promotion history, Woodpecker build data) to persist on NFS volumes, so that pipeline data survives cluster teardowns and rebuilds.

Acceptance Criteria

  1. THE Pipeline_Infrastructure SHALL create PersistentVolumes backed by the NFS share at nfs://192.168.42.8:/volume1/Kubernetes/pipelines for ArgoCD server data, Kargo data, and Woodpecker_CI data
  2. WHEN runmelast.sh is executed, THE NFS_PV resources and their underlying NFS data SHALL remain intact
  3. WHEN runmefirst.sh is executed after a previous teardown, THE Pipeline_Infrastructure SHALL reattach to the existing NFS data and restore previous pipeline state
  4. THE Pipeline_Infrastructure SHALL use separate NFS subdirectories for ArgoCD, Kargo, and Woodpecker to prevent data conflicts

Requirement 25: ArgoCD GitOps Configuration with Gitea

User Story: As a platform operator, I want ArgoCD to sync Kubernetes manifests from the Gitea repository instead of GitHub, so that the GitOps source of truth is the local Git forge.

Acceptance Criteria

  1. THE ArgoCD SHALL be configured with a repository secret pointing to the Gitea stonks-oracle repository instead of the GitHub repository
  2. WHEN a change is committed to the Helm chart or values files in the Gitea repository, THE ArgoCD SHALL detect the change and sync the updated manifests to the target namespace
  3. THE ArgoCD SHALL support multiple Application resources for beta, paper, and live environments, each with stage-specific value overrides
  4. IF an ArgoCD sync fails, THEN THE ArgoCD SHALL report the failure status in the ArgoCD UI and the Kargo_Dashboard

Requirement 26: Woodpecker Agent RBAC for Integration Tests

User Story: As a platform operator, I want the Woodpecker agent to have sufficient Kubernetes RBAC permissions to run integration tests that create ephemeral namespaces and deploy sandbox infrastructure.

Acceptance Criteria

  1. THE Pipeline_Infrastructure SHALL create a ClusterRoleBinding granting the Woodpecker_Agent service account permissions to create and delete namespaces for integration test sandboxes
  2. THE Woodpecker_Agent RBAC SHALL be scoped to the minimum permissions required for integration test execution
  3. WHEN the Woodpecker_Agent executes integration tests, THE Woodpecker_Agent SHALL have access to create deployments, services, and configmaps in ephemeral test namespaces

Requirement 27: Woodpecker Pipeline File Creation

User Story: As a developer, I want a .woodpecker.yml file created that translates the existing CI pipeline into Woodpecker's native format, targeting the Local_Registry and including a GitHub mirror step.

Acceptance Criteria

  1. THE Woodpecker_Pipeline_File SHALL define lint, test, image build, integration test, and GitHub mirror steps using Woodpecker's container-based step format
  2. THE Woodpecker_Pipeline_File SHALL target all image pushes to registry.celestium.life/stonks-oracle/ instead of ghcr.io
  3. THE Woodpecker_Pipeline_File SHALL use Woodpecker secrets for Local_Registry credentials (if required) and GitHub mirror credentials
  4. THE Woodpecker_Pipeline_File SHALL use when branch and event filters to restrict image builds and mirror pushes to main branch push events
  5. THE Woodpecker_Pipeline_File SHALL support matrix builds or sequential steps for building all 12 Python services, the dashboard, and the superset image

Requirement 28: Woodpecker Dashboard Access via Traefik Ingress

User Story: As a platform operator, I want the Woodpecker CI dashboard accessible via a Traefik ingress with TLS, so that developers can view pipeline status and build logs from a browser.

Acceptance Criteria

  1. THE Pipeline_Infrastructure SHALL expose the Woodpecker_Dashboard via Traefik ingress at stonks-ci.celestium.life with TLS using the ca-issuer ClusterIssuer
  2. THE Woodpecker_Dashboard SHALL display pipeline status, build logs, and step-level output for all stonks-oracle repository builds
  3. WHEN a pipeline fails, THE Woodpecker_Dashboard SHALL display the failing step and its container logs
  4. THE Woodpecker_Dashboard SHALL authenticate users via the Gitea OAuth2 integration so that only Gitea users can access build information
  5. THE Pipeline_Infrastructure SHALL create a Kubernetes NetworkPolicy allowing Traefik ingress traffic to the Woodpecker_CI server on its HTTP port