Last Updated on 2024-01-02 by Clay Problem HuggingFace has published an article stating that the current LLM is best trained according to the ChatML format. In normal case, it will be generated according to three different roles of system, user and assistant. The format is as follows: <|im_start|>system …system prompt…<|im_end|> <|im_start|>user …user message…<|im_end|> <|im_start|>assistant … … Continue reading [Solved] Mistral Cannot Generate eos_token `<|im_end|>` After SFTTrainer Fine-tuning