Truly a win-win-win: * NVIDIA is officially the center of open source AI w/ Nemotron and now HF * Hugging Face scores huge + gets the perfect home (no more nagging re business model!) * Open source AI wins! Massive co…

Truly a win-win-win: * NVIDIA is officially the center of open source AI w/ Nemotron and now HF * Hugging Face scores huge + gets the perfect home (no more nagging re business model!) * Open source AI wins! Massive co...

NVIDIA 与 Hugging Face 达成深度合作,将后者纳入其开源 AI 生态版图。此举既让 Hugging Face 摆脱了长期以来的商业模式争议,也让 NVIDIA 通过 Nemotron 模型与 Hugging Face 平台进一步确立开源 AI 基础设施核心地位。

CUDA out of memory

CUDA out of memory

该报错发生在 vLLM 使用 FlashMLA 稀疏 FP8 混合批次解码路径时,稀疏解码内核会分配一个随 --max-num-batched-tokens 线性增长的工作区,且该工作区不受启动时内存预检约束,批次增大后触发运行时 CUDA OOM。优先排查方向是调低 --max-num-batch