标签： LLM

ValueError: LoRA adapter 1 contains fused 3D MoE weights for ‘language_model.model.layers.0.experts’, but the model uses a 2D MoE LoRA wrapper. Start the engine with enable_mixed_moe_lora_form

ValueError: LoRA adapter 1 contains fused 3D MoE weights for 'language_model.model.layers.0.experts', but the model uses a 2D MoE LoRA wrapper. Start the engine with enable_mixed_moe_lora_form

在 vLLM 中加载 Gemma4 MoE 模型并提供 PEFT 专家 LoRA 适配器时，若未启用混合 MoE LoRA 格式，引擎会因 LoRA 适配器含 3D 融合 MoE 权重、而模型侧使用 2D MoE LoRA 包装器而报错。优先排查启动时是否启用了 enable_mixed_moe_l