Python

[PyTorch] Release GPU / CPU Memory After Delete Model

Clay
2024-01-092024-01-15
Python, PyTorch

Problem

Yesterday, I developed a model merging program. This time I have no enough gpu memory to merge the models in only one time, so I tried to merge layer by layer. I found the memory of GPU is easily to release but CPU didn’t.

Clay
2024-01-072024-01-07
Python

Introduction

Today I was reading the training source code of DreamBooth, and I found the tempfile built-in module; I happened to reconstruct a script for model’s layer merging, suddenly thought that the module would make the code more elegant, so I made this note.

Clay
2024-01-022024-01-02
Machine Learning, Python, PyTorch

Introduction

DPO (Direct Preference Optimization) is a fine-tuning method that want to replace RLHF (Reinforcement Learning from Human Feedback).

Clay
2023-12-282023-12-28
Linux, Python

Problem

Today I installed pyaudio on my new Linux laptop (a python package for sound recording), I encountered an error I never met:

Clay
2023-12-182023-12-18
Python

Introduction

IPython is a system that provided a interactive interface, it could be integrated with shells or graphical interface; for example we can use ipython (if you had been installed it), or you can use some GUI editor/IDE such as VS Code and PyCharm.

Clay
2023-12-182023-12-18
C++, LeetCode, Python

LeetCode: 242-Valid Anagram Solution

Clay
2023-12-162023-12-16
C++, LeetCode, Python

Problem

Given two strings s and t, return true if t is an anagram of s, and false otherwise.

Clay
2023-12-152023-12-15
C++, LeetCode, Python

Problem

You are given the array paths, where paths[i] = [cityA_i, cityB_i] means there exists a direct path going from cityA_i to cityB_i. Return the destination city, that is, the city without any path outgoing to another city.

Clay
2023-12-142023-12-14
Machine Learning, Python, PyTorch

Introduction

vLLM is a large language model (LLM) acceleration framework that developed by the research team of University of California, Berkeley. It is used PagedAttention to increase the usage rate of GPU VRAM, and this method do not change the model architecture.

Clay
2023-12-122023-12-12
Python, PyTorch

Problem

Last night I tried to improve some code about merge two models. Because my poor device, I cannot merge all layers totally, but need to merge them layer by layer for reducing my memory cost.

Python

[PyTorch] Release GPU / CPU Memory After Delete Model

Problem

[Python] Creating and Auto-Removing Temporary Directories with Python’s `tempfile`

Introduction

Direct Preference Optimization (DPO) Training Note

Introduction

[Solved] fatal error: portaudio.h: No such file or directory 9 | #include “portaudio.h” | ^~~~~~~~~~~~~ compilation terminated

Problem

[Python] A Line Breaking In The IPython Interface

Introduction

LeetCode: 1913-Maximum Product Difference Between Two Pairs Solution

LeetCode: 242-Valid Anagram Solution

Problem

LeetCode: 1436-Destination City Solution

Problem

Using vLLM To Accelerate The Decoding Of Large Language Model

Introduction

[PyTorch] Delete Model And Free Memory (GPU / CPU)

Problem