LlamaTokenizer in v5 overrides tokenizer.json’s ByteLevel pre-tokenizer with Metaspace, silently breaks DeepSeek V3/R1 family

使用 transformers v5 的 AutoTokenizer.from_pretrained() 加载任何 tokenizer_config.json 中设置了 "tokenizer_class": "LlamaTokenizerFast" 或 "LlamaTokenizer" 的模型。已确








