fix: remove bucket-init job, wait for pods before readiness check
ci/woodpecker/push/woodpecker Pipeline failed
Build and Push / lint-and-test (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.adapters.broker_adapter name:broker-adapter]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.aggregation.worker name:aggregation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.extractor.worker name:extractor]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.ingestion.worker name:ingestion]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.lake_publisher.worker name:lake-publisher]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.parser.worker name:parser]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.recommendation.worker name:recommendation]) (push) Has been cancelled
Build and Push / build-services (map[cmd:python -m services.scheduler.app name:scheduler]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.api.app:app --host 0.0.0.0 --port 8000 name:query-api]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.risk.app:app --host 0.0.0.0 --port 8000 name:risk]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.symbol_registry.app:app --host 0.0.0.0 --port 8000 name:symbol-registry]) (push) Has been cancelled
Build and Push / build-services (map[cmd:uvicorn services.trading.app:app --host 0.0.0.0 --port 8000 name:trading-engine]) (push) Has been cancelled
Build and Push / build-dashboard (push) Has been cancelled
Build and Push / build-superset (push) Has been cancelled
Build and Push / integration-test (push) Has been cancelled

- Remove minio-bucket-init Job entirely (seed_minio.py creates bucket)
- Wait for pods to exist before kubectl wait --for=condition=ready
- Fixes 'no matching resources found' race when pods are still ContainerCreating
This commit is contained in:
Celes Renata
2026-04-19 19:25:49 +00:00
parent b2b8aca7c6
commit 4df513d096
2 changed files with 10 additions and 104 deletions
+1 -96
View File
@@ -2,7 +2,7 @@
# Namespace is substituted at runtime via envsubst
# No persistence — uses emptyDir
# Credentials: minioadmin/minioadmin (hardcoded for ephemeral sandbox)
# Includes a Job that waits for MinIO readiness and creates the stonks-normalized bucket
# Bucket creation is handled by seed_minio.py (no separate init job needed)
---
apiVersion: apps/v1
kind: Deployment
@@ -100,98 +100,3 @@ spec:
- port: 9000
targetPort: 9000
protocol: TCP
---
apiVersion: batch/v1
kind: Job
metadata:
name: minio-bucket-init
namespace: ${NAMESPACE}
labels:
app: minio
tier: infra
app.kubernetes.io/part-of: stonks-oracle
spec:
backoffLimit: 4
template:
metadata:
labels:
app: minio-bucket-init
tier: infra
spec:
imagePullSecrets:
- name: dockerhub-credentials
automountServiceAccountToken: false
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
seccompProfile:
type: RuntimeDefault
restartPolicy: OnFailure
containers:
- name: bucket-init
image: registry.celestium.life/dockerhub-cache/library/python:3.12-slim
imagePullPolicy: IfNotPresent
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
resources:
requests:
cpu: 50m
memory: 64Mi
limits:
cpu: 250m
memory: 128Mi
env:
- name: HTTP_PROXY
value: ""
- name: HTTPS_PROXY
value: ""
- name: http_proxy
value: ""
- name: https_proxy
value: ""
- name: NO_PROXY
value: "*"
- name: no_proxy
value: "*"
command: ["python", "-c"]
args:
- |
import urllib.request, urllib.error, time
# Disable all proxy usage in urllib
no_proxy_handler = urllib.request.ProxyHandler({})
opener = urllib.request.build_opener(no_proxy_handler)
urllib.request.install_opener(opener)
endpoint = "http://minio:9000"
max_wait = 60
# Wait for MinIO readiness
print("Waiting for MinIO to be ready...")
start = time.time()
while time.time() - start < max_wait:
try:
r = urllib.request.urlopen(f"{endpoint}/minio/health/ready", timeout=3)
if r.status == 200:
print("MinIO is ready.")
break
except Exception:
pass
time.sleep(2)
else:
raise SystemExit("MinIO did not become ready within 60s")
# Create bucket via S3 PUT request
bucket = "stonks-normalized"
req = urllib.request.Request(f"{endpoint}/{bucket}", method="PUT")
try:
urllib.request.urlopen(req, timeout=5)
print(f"Bucket '{bucket}' created successfully.")
except urllib.error.HTTPError as e:
if e.code == 409:
print(f"Bucket '{bucket}' already exists.")
else:
raise