fix: handle max_tokens for NVIDIA MiniMax M3#9209
Conversation
There was a problem hiding this comment.
Code Review
This pull request renames _apply_provider_specific_extra_body_overrides to _apply_provider_specific_request_overrides to allow modifying request payloads. It introduces a compatibility default of max_tokens = 8192 for NVIDIA's hosted MiniMax M3 model when max_tokens is omitted, resolving an issue where empty choices are returned. Review feedback suggests normalizing the provider name to lowercase and handling potential None values safely to ensure robust, case-insensitive provider checks.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| extra_body: dict[str, Any], | ||
| ) -> None: | ||
| if self.provider_config.get("provider") != "ollama": | ||
| provider = self.provider_config.get("provider") |
There was a problem hiding this comment.
To make the provider check robust and case-insensitive (consistent with how the model parameter is handled on the next line), normalize the provider name to lowercase and handle potential None values safely.
| provider = self.provider_config.get("provider") | |
| provider = str(self.provider_config.get("provider") or "").lower() |
Fixes #9206.
NVIDIA documents
8192as the defaultmax_tokensvalue for MiniMax M3, but the hosted endpoint behavior reported in #9206 returns an emptychoiceslist when the field is omitted. This PR adds a narrow compatibility workaround for that model without changing request limits for other NVIDIA models.Reference: https://docs.api.nvidia.com/nim/reference/minimaxai-minimax-m3-infer
Modifications / 改动点
Apply
max_tokens=8192only when the provider is NVIDIA and the actual request model isminimaxai/minimax-m3.Add the default to the standard request payload rather than
extra_body.Preserve values explicitly supplied through either the standard payload or
custom_extra_body.Keep all other NVIDIA models unchanged because their documented token limits differ.
Add regression tests for the default, both override paths, and an unrelated NVIDIA model.
This is NOT a breaking change. / 这不是一个破坏性变更。
Screenshots or Test Results / 运行截图或测试结果
python -m pytest tests/test_openai_source.py -q: 62 passedruff format --check .: 480 files already formattedruff check .: All checks passedgit diff --check: passedapi.soulter.top; that request is blocked in the local sandbox. GitHub Actions remains the source of truth for the complete suite after workflow approval.Checklist / 检查清单