fix: disable thinking mode on vLLM path with chat_template_kwargs
The thesis rewriter uses vLLM (not Ollama) in production. The previous
fix only added think=False to the Ollama payload. For vLLM's
OpenAI-compatible API with Qwen3 models, thinking mode is disabled via
chat_template_kwargs: {enable_thinking: false} in the request body.
This commit is contained in:
@@ -405,6 +405,8 @@ async def _call_vllm_thesis(
|
|||||||
"max_tokens": config.max_tokens,
|
"max_tokens": config.max_tokens,
|
||||||
"temperature": config.temperature,
|
"temperature": config.temperature,
|
||||||
"stream": False,
|
"stream": False,
|
||||||
|
# Disable thinking/reasoning mode for Qwen3 models on vLLM
|
||||||
|
"chat_template_kwargs": {"enable_thinking": False},
|
||||||
}
|
}
|
||||||
|
|
||||||
headers: dict[str, str] = {"Content-Type": "application/json"}
|
headers: dict[str, str] = {"Content-Type": "application/json"}
|
||||||
|
|||||||
Reference in New Issue
Block a user