feat: vLLM in K8s via Harbor mirror, pipelines point to internal svc
- vLLM image mirrored to registry.celestium.life/stonks-oracle/vllm-openai - Deployment uses Harbor image (Docker Hub IPv6 unreachable from cluster) - All 3 pipelines use vllm-external.vllm-service.svc.cluster.local:2701 - K8s manifests at infra/kube-vllm/ and synced to ~/sources/kube/vllm
This commit is contained in:
@@ -0,0 +1,45 @@
|
||||
replicaCount: 1
|
||||
|
||||
image:
|
||||
repository: vllm/vllm-openai
|
||||
tag: latest
|
||||
pullPolicy: Always
|
||||
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 1
|
||||
requests:
|
||||
cpu: "2"
|
||||
memory: "8Gi"
|
||||
|
||||
runtimeClassName: nvidia
|
||||
|
||||
env:
|
||||
- name: HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: vllm-secrets
|
||||
key: HF_TOKEN
|
||||
|
||||
args:
|
||||
- "serve"
|
||||
- "numind/NuExtract3"
|
||||
- "--served-model-name"
|
||||
- "numind/NuExtract3"
|
||||
- "--host"
|
||||
- "0.0.0.0"
|
||||
- "--port"
|
||||
- "8000"
|
||||
- "--gpu-memory-utilization"
|
||||
- "0.45"
|
||||
- "--max-model-len"
|
||||
- "8192"
|
||||
- "--max-num-seqs"
|
||||
- "8"
|
||||
|
||||
service:
|
||||
type: ClusterIP
|
||||
port: 8000
|
||||
|
||||
nodeSelector:
|
||||
kubernetes.io/hostname: gremlin-1
|
||||
Reference in New Issue
Block a user