RuntimeError: don’t know how to restore data location of torch.storage.UntypedStorage (tagged with cpu:0)

该报错通常出现在 CPU + 多进程(如 torchrun + gloo)场景下用 Trainer 从 checkpoint 恢复训练时,加载 optimizer state 的过程中失败。优先排查 map_location 是否被传成了带索引的 CPU 设备(如 cpu:0 ),而不是纯 "cpu
![[MPS] Upstream correctness issue in attention when value head dim differs from query](https://www.chat-gpts.plus/wp-content/uploads/2026/09/44554-1770f242-768x403.jpg)







