标签: Python

ValueError: Your model contains `DTensor` parameters, which is incompatible with DDP. Maybe you loaded your model with `device_map=’auto’`? Specify `device_map=’cuda’` or ‘xpu’ or ‘cpu’ instea

ValueError: Your model contains `DTensor` parameters, which is incompatible with DDP. Maybe you loaded your model with `device_map='auto'`? Specify `device_map='cuda'` or 'xpu' or 'cpu' instea

这个报错通常出现在你用 DistributedConfig(fsdp_size=N) 在加载时就把模型 FSDP2 分片,然后把已分片的模型交给 Trainer 训练的场景。此时 accelerate 的 prepare_model 仍按 DDP 分支处理,撞上 DTensor 检查而报错。优先排查