SFTTrainer is a LLM fine-tuning tool provided by HuggingFace team, that can easily adjust many hyper-parameters and config at the fine-tuning task. In the process, response_template is the special string template we need to pass into the tool, any response right by it will be computed the loss.
Paper link: https://arxiv.org/abs/2212.13345
The author of this research work is the renowned figure in the field of deep learning, Geoffrey Hinton, who was originally a researcher at Google Brain when he initially wrote this paper (he left in 2023).
Read More »[Paper Reading] The Forward-Forward Algorithm: Some Preliminary Investigation
Yesterday, I developed a model merging program. This time I have no enough gpu memory to merge the models in only one time, so I tried to merge layer by layer. I found the memory of GPU is easily to release but CPU didn’t.
Read More »[PyTorch] Release GPU / CPU Memory After Delete Model
DPO (Direct Preference Optimization) is a fine-tuning method that want to replace RLHF (Reinforcement Learning from Human Feedback).
Read More »Direct Preference Optimization (DPO) Training Note HuggingFace has published an article stating that the current LLM is best trained according to the ChatML format. In normal case, it will be generated according to three different roles of system, user and assistant. The format is as follows:
Read More »[Solved] Mistral Cannot Generate eos_token `<|im_end|>` After SFTTrainer Fine-tuning
vLLM is a large language model (LLM) acceleration framework that developed by the research team of University of California, Berkeley. It is used PagedAttention to increase the usage rate of GPU VRAM, and this method do not change the model architecture.
Read More »Using vLLM To Accelerate The Decoding Of Large Language Model Last night I tried to improve some code about merge two models. Because my poor device, I cannot merge all layers totally, but need to merge them layer by layer for reducing my memory cost.
Read More »[PyTorch] Delete Model And Free Memory (GPU / CPU) In early 2023, The pytorch 2.0 version added a new function that is torch.compile(), It cloud be accelerate the speed of model training or inference. By using precision training with compiling, it always make my training time reduce to half.
Meta AI has indeed been incredibly powerful recently, seemingly securing its position as a giant in AI research and development in no time at all, and what’s more, it sets the bar high with all its top-tier open-source contributions. From Segment Anything that can segment objects in the image domain, to the public large language model and foundational model, LLaMA (yes, the one causing the llama family appear!), to the recent ImageBind that can transform six modalities and the Massively Multilingual Speech (MMS) project… I must say, for an ordinary person like me, it’s quite an effort to keep up with how to use these technologies, let alone trying to chase their technical prowess.
Read More »ImageBind: A Experience Notes on a Multimodal Vector Transformation Model
Today I trained the model with simpletransformers package, I got an warning message that never seen:
Read More »[Solved] huggingface/tokenizers: The current process just got forked. after parallelism has already been used. Disabling parallelism to avoid deadlocks
