[GitHub] How To Create A Pull Request (PR)
Introduction
Sending a Pull Request (PR) on GitHub to an open-source project is a wonderful yet significant endeavor.
Read More »[GitHub] How To Create A Pull Request (PR)Clay
[PyTorch] Release GPU / CPU Memory After Delete Model
Problem
Yesterday, I developed a model merging program. This time I have no enough gpu memory to merge the models in only one time, so I tried to merge layer by layer. I found the memory of GPU is easily to release but CPU didn't.
Read More »[PyTorch] Release GPU / CPU Memory After Delete Model
[Solved][Linux] /bin/bash: warning: shell level (1000) too high, resetting to 1
Problem
/bin/bash: warning: shell level (1000) too high, resetting to 1
[Python] Creating and Auto-Removing Temporary Directories with Python's `tempfile`
Introduction
Today I was reading the training source code of DreamBooth, and I found the tempfile built-in module; I happened to reconstruct a script for model's layer merging, suddenly thought that the module would make the code more elegant, so I made this note.
Direct Preference Optimization (DPO) Training Note
Introduction
DPO (Direct Preference Optimization) is a fine-tuning method that want to replace RLHF (Reinforcement Learning from Human Feedback).
Read More »Direct Preference Optimization (DPO) Training Note
[Solved] Mistral Cannot Generate eos_token `<|im_end|>` After SFTTrainer Fine-tuning
Problem
HuggingFace has published an article stating that the current LLM is best trained according to the ChatML format. In normal case, it will be generated according to three different roles of system, user and assistant. The format is as follows:
Read More »[Solved] Mistral Cannot Generate eos_token `<|im_end|>` After SFTTrainer Fine-tuning[Solved] fatal error: portaudio.h: No such file or directory 9 | #include "portaudio.h" | ^~~~~~~~~~~~~ compilation terminated
Problem
Today I installed pyaudio on my new Linux laptop (a python package for sound recording), I encountered an error I never met:
Read More »[Solved] fatal error: portaudio.h: No such file or directory 9 | #include "portaudio.h" | ^~~~~~~~~~~~~ compilation terminated[Python] A Line Breaking In The IPython Interface
Introduction
IPython is a system that provided a interactive interface, it could be integrated with shells or graphical interface; for example we can use ipython (if you had been installed it), or you can use some GUI editor/IDE such as VS Code and PyCharm.
LeetCode: 2352-Design a Food Rating System Solution
Problem
Design a food rating system that can do the following:
- Modify the rating of a food item listed in the system.
- Return the highest-rated food item for a type of cuisine in the system.