Skip to content

fix: handle max_tokens for NVIDIA MiniMax M3#9209

Draft
Last-emo-boy wants to merge 1 commit into
AstrBotDevs:masterfrom
Last-emo-boy:fix/9206-nvidia-minimax-max-tokens
Draft

fix: handle max_tokens for NVIDIA MiniMax M3#9209
Last-emo-boy wants to merge 1 commit into
AstrBotDevs:masterfrom
Last-emo-boy:fix/9206-nvidia-minimax-max-tokens

Conversation

@Last-emo-boy

@Last-emo-boy Last-emo-boy commented Jul 11, 2026

Copy link
Copy Markdown

Fixes #9206.

NVIDIA documents 8192 as the default max_tokens value for MiniMax M3, but the hosted endpoint behavior reported in #9206 returns an empty choices list when the field is omitted. This PR adds a narrow compatibility workaround for that model without changing request limits for other NVIDIA models.

Reference: https://docs.api.nvidia.com/nim/reference/minimaxai-minimax-m3-infer

Modifications / 改动点

  • Apply max_tokens=8192 only when the provider is NVIDIA and the actual request model is minimaxai/minimax-m3.

  • Add the default to the standard request payload rather than extra_body.

  • Preserve values explicitly supplied through either the standard payload or custom_extra_body.

  • Keep all other NVIDIA models unchanged because their documented token limits differ.

  • Add regression tests for the default, both override paths, and an unrelated NVIDIA model.

  • This is NOT a breaking change. / 这不是一个破坏性变更。

Screenshots or Test Results / 运行截图或测试结果

  • python -m pytest tests/test_openai_source.py -q: 62 passed
  • ruff format --check .: 480 files already formatted
  • ruff check .: All checks passed
  • git diff --check: passed
  • A full-suite attempt reached an unrelated dashboard plugin test that requires outbound access to api.soulter.top; that request is blocked in the local sandbox. GitHub Actions remains the source of truth for the complete suite after workflow approval.

Checklist / 检查清单

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request renames _apply_provider_specific_extra_body_overrides to _apply_provider_specific_request_overrides to allow modifying request payloads. It introduces a compatibility default of max_tokens = 8192 for NVIDIA's hosted MiniMax M3 model when max_tokens is omitted, resolving an issue where empty choices are returned. Review feedback suggests normalizing the provider name to lowercase and handling potential None values safely to ensure robust, case-insensitive provider checks.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

extra_body: dict[str, Any],
) -> None:
if self.provider_config.get("provider") != "ollama":
provider = self.provider_config.get("provider")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To make the provider check robust and case-insensitive (consistent with how the model parameter is handled on the next line), normalize the provider name to lowercase and handle potential None values safely.

Suggested change
provider = self.provider_config.get("provider")
provider = str(self.provider_config.get("provider") or "").lower()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] NVIDIA Provider 测试连通性报错 "OpenAI completion has no choices" ——max_tokens 未正确写入请求 payload

1 participant