In the process of training and fine-tuning deep neural networks, the most important and scarce resource is undoubtedly the GPU’s VRAM. Therefore, making every bit perform at its best is a critical task.
Read More »Differences in Precision Representations in Deep Learning: Float32, Float16, Float8, and BFloat16
Linux has so many useful tools, and I truly want to document every single one of them. To celebrate Linux reaching a usage rate of 4.55% on StatCounter (2024-09-18), I’ve decided to document another tool recommended by a colleague—the fuck command.
z command is something I’ve wanted to write about for a long time! However, I’ve been busy with AI training (company work) and model acceleration (personal interest), so I haven’t had the time. Let’s put it this way, if someone asks me to recommend essential tools for a Linux system, I would undoubtedly place z in my top ten list.
In the FastAPI official documentation, there’s a section about ‘Automatic Interactive API Documentation‘:
Recently, I’ve achieved some good application results by fine-tuning Gemma-2. However, I encountered various errors when deploying it on the client’s equipment, which was quite frustrating. Currently, there isn’t a systematic troubleshooting guide online, so I’m documenting it here.
Read More »Troubleshooting Accelerated Inference of Gemma-2 on V100 GPUs Using vLLM
In Python, the context manager decorator @contextmanager from the contextlib module allows developers to conveniently create our own context manager.
Recently, due to some serendipitous events, I had a chance to modify the architecture of a model slightly. I took this opportunity to explore how to iterate and print the layers of neural networks in PyTorch.
Read More »[PyTorch] Traversing Every Layer of a Neural Network in a Model
Softmax is a commonly used activation function, and it is often employed as the last layer in multi-class classification.
Triton is an open-source GPU programming language compiler released by OpenAI in 2021. Over recent years, it has become increasingly popular among developers for writing and optimizing parallel programs on GPUs. Compared to traditional libraries such as CUDA or OpenCL, Triton offers a Python-like syntax, making it more readable and easier to learn.
Read More »OpenAI Triton Note (1): Vector Addition
My advisor used to tell me, “Don’t just use other people’s libraries; you have to write your own to truly understand.” Back then, I didn’t have much time to implement various technologies I was interested in since I was fully occupied with my dissertation. However, I often recall his earnest advice even now, and it prompted me to finally attempt the implementation of BERT, a classic encoder-only transformer model.
Read More »[PyTorch] BERT Architecture Implementation Note
