feat: vLLM serving numind/NuExtract3 on full GPU, ollama disabled

- Ollama scaled to 0 — vLLM gets entire 16GB GPU
- NuExtract3 (9B) running at 80% GPU utilization
- Both OLLAMA_BASE_URL and VLLM_BASE_URL point to vLLM service
- Fixed gremlin-1 missing default route (no internet access)
- Removed GPU resource limit (time-slicing handles allocation)
- Removed init container (direct download works with internet fix)
This commit is contained in:
Celes Renata
2026-07-03 17:57:19 +00:00
parent ecade0dd52
commit a322e00659
6 changed files with 14 additions and 13 deletions
+2 -1
View File
@@ -16,6 +16,7 @@ spec:
app: vllm
spec:
runtimeClassName: nvidia
enableServiceLinks: false
nodeSelector:
kubernetes.io/hostname: gremlin-1
containers:
@@ -32,7 +33,7 @@ spec:
- "--port"
- "8000"
- "--gpu-memory-utilization"
- "0.45"
- "0.80"
- "--max-model-len"
- "8192"
- "--max-num-seqs"