admin/stonks-oracle
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update steering docs and hooks for current project state

Browse Source
This commit is contained in:
Celes Renata
2026-04-11 20:41:57 -07:00
parent 99e17be282
commit 37d5f9b01c
8 changed files with 177 additions and 67 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.kiro/steering/project-context.md
+31 -9
View File
@@ -7,34 +7,56 @@ Python monorepo with services under `services/`, infrastructure under `infra/`,
## Local Dev Environment
- NixOS dev environment, Python 3.12
- Virtual environment at `.venv/` — always use it for Python commands
- For tools not in `.venv/` (like `ruff`, `gh`), use `nix-shell -p <pkg> --run "<cmd>"`
- Node.js 24 for frontend (`frontend/` directory)
- Docker available locally for image builds
- Docker available locally for image builds (but let CI handle pushes)
## Live Endpoints
- Dashboard: `https://stonks.celestium.life`
- Query API: `https://stonks-api.celestium.life`
- Symbol Registry: `https://stonks-registry.celestium.life`
- Superset: `https://stonks-dash.celestium.life`
- Trino: `https://stonks-trino.celestium.life`
## Infrastructure
- Kubernetes cluster: 4x NixOS nodes (gremlin-1 through gremlin-4), reachable via `kubectl`, `virtctl`, `ssh root@gremlin-{1,2,3,4}`
- NixOS configs stored at `/etc/nixos` on gremlin-1, git-pushed to other hosts
- Ingress: Traefik, domain `*.celestium.life`
- Cert-Manager: `ca-issuer` (local CA) for internal services, `celestium-le-production` (Let's Encrypt) for public-facing
- Cert-Manager: `ca-issuer` (local CA) for internal services
- Container registry: `ghcr.io/celesrenata/stonks-oracle`
## CI/CD
- GitHub Actions workflow at `.github/workflows/build.yml`
- Push to `main` triggers: lint → test → build all service images + dashboard image → push to GHCR
- Push to `main` triggers: lint → pytest → frontend vitest → build all service images + dashboard + superset → push to GHCR
- Images tagged as `ghcr.io/celesrenata/stonks-oracle/<service>:<sha>` and `:latest`
- Dashboard image built from `frontend/Dockerfile` (multi-stage: node → nginx)
- Python service images built from `docker/Dockerfile` with `SERVICE_CMD` build arg
- Let CI handle image builds and pushes — only build locally for testing or when CI is unavailable
- Dashboard image: `frontend/Dockerfile` (multi-stage: node:24 → nginx-unprivileged on port 8080)
- Superset image: `docker/Dockerfile.superset` (apache/superset + trino + psycopg2)
- Python service images: `docker/Dockerfile` with `SERVICE_CMD` build arg
- Let CI handle image builds and pushes — do NOT manually `docker build && docker push`
- Check CI status: `nix-shell -p gh --run "gh run list -L 3"`
## Deployment Scripts
- `~/sources/kube/stonks-oracle/runmefirst.sh` — full deploy: DB setup, migrations, Helm install, rolling restart
- `~/sources/kube/stonks-oracle/runmelast.sh` — teardown: Helm uninstall, clean resources (preserves DB/MinIO/Redis)
- After CI builds, deploy with: `helm upgrade --install stonks-oracle infra/helm/stonks-oracle -n stonks-oracle`
- Restart a single service: `kubectl rollout restart deployment/<name> -n stonks-oracle`
## API Secrets
- Stored as files in repo root (gitignored): `polygon.io.key`, `alpaca.key`, `alpaca.secret`, `alpaca.url`
- GitHub token at `/run/secrets/github_token`
- Injected into K8s secrets via `runmefirst.sh` Helm `--set` flags
## Existing Cluster Services (do NOT redeploy these)
- PostgreSQL: `postgresql-rw.postgresql-service.svc.cluster.local:5432`
- Redis: `redis-master.redis-service.svc.cluster.local:6379`
- MinIO: `minio.minio-service.svc.cluster.local:80` (API), console at `minio-crawler-console.minio-service.svc.cluster.local:9090`
- MinIO: `minio.minio-service.svc.cluster.local:80` (API)
- Ollama: `ollama.ollama-service.svc.cluster.local:11434` (cluster-internal), also at `http://10.1.1.12:2701` (external), GPU: 4070 Ti Super 16GB
## Key Conventions
- All services use `services/shared/config.py` for configuration via env vars
- Redis queues defined in `services/shared/redis_keys.py`
- Pydantic schemas in `services/shared/schemas.py`
- K8s manifests in `infra/k8s/`, Helm chart in `infra/helm/stonks-oracle/`, all in `stonks-oracle` namespace
- Helm chart in `infra/helm/stonks-oracle/`, all in `stonks-oracle` namespace
- Lakehouse DDL in `lakehouse/schemas/`
- Crawler patterns inspired by Noctipede (`~/sources/splinterstice/noctipede`): BeautifulSoup + requests with retry adapters, content hashing, boilerplate stripping, quality scoring
- Frontend proxies: `/api/` → query-api:8000, `/registry/` → symbol-registry:8000, `/risk/` → risk:8000
- Network policies: default-deny with explicit allow rules per service