Stonks Oracle — Project Context

Overview

Stonks Oracle is a Kubernetes-native AI market intelligence and paper-trading platform. Python monorepo with services under services/, infrastructure under infra/, lakehouse schemas under lakehouse/, frontend React dashboard under frontend/, and dashboards under dashboards/.

Local Dev Environment

NixOS dev environment, Python 3.12
Virtual environment at .venv/ — always use it for Python commands
Node.js 24 for frontend (frontend/ directory)
Docker available locally for image builds

Infrastructure

Kubernetes cluster: 4x NixOS nodes (gremlin-1 through gremlin-4), reachable via kubectl, virtctl, ssh root@gremlin-{1,2,3,4}
NixOS configs stored at /etc/nixos on gremlin-1, git-pushed to other hosts
Ingress: Traefik, domain *.celestium.life
Cert-Manager: ca-issuer (local CA) for internal services, celestium-le-production (Let's Encrypt) for public-facing
Container registry: ghcr.io/celesrenata/stonks-oracle

CI/CD

GitHub Actions workflow at .github/workflows/build.yml
Push to main triggers: lint → test → build all service images + dashboard image → push to GHCR
Images tagged as ghcr.io/celesrenata/stonks-oracle/<service>:<sha> and :latest
Dashboard image built from frontend/Dockerfile (multi-stage: node → nginx)
Python service images built from docker/Dockerfile with SERVICE_CMD build arg
Let CI handle image builds and pushes — only build locally for testing or when CI is unavailable

Existing Cluster Services (do NOT redeploy these)

PostgreSQL: postgresql-rw.postgresql-service.svc.cluster.local:5432
Redis: redis-master.redis-service.svc.cluster.local:6379
MinIO: minio.minio-service.svc.cluster.local:80 (API), console at minio-crawler-console.minio-service.svc.cluster.local:9090
Ollama: ollama.ollama-service.svc.cluster.local:11434 (cluster-internal), also at http://10.1.1.12:2701 (external), GPU: 4070 Ti Super 16GB

Key Conventions

All services use services/shared/config.py for configuration via env vars
Redis queues defined in services/shared/redis_keys.py
Pydantic schemas in services/shared/schemas.py
K8s manifests in infra/k8s/, Helm chart in infra/helm/stonks-oracle/, all in stonks-oracle namespace
Lakehouse DDL in lakehouse/schemas/
Crawler patterns inspired by Noctipede (~/sources/splinterstice/noctipede): BeautifulSoup + requests with retry adapters, content hashing, boilerplate stripping, quality scoring

2.5 KiB Raw Blame History