[Linux] viddy: An Enhanced Version of the Watch Command
Last Updated on 2024-09-26 by Clay
Introduction
viddy is a tool similar to watch for running a command at regular intervals in a Linux terminal and displaying the output.
[Linux] viddy: An Enhanced Version of the Watch Command
Differences in Precision Representations in Deep Learning: Float32, Float16, Float8, and BFloat16
Last Updated on 2024-09-25 by Clay
In the process of training and fine-tuning deep neural networks, the most important and scarce resource is undoubtedly the GPU's VRAM. Therefore, making every bit perform at its best is a critical task.
Read More »Differences in Precision Representations in Deep Learning: Float32, Float16, Float8, and BFloat16[Linux] Using "The Fuck" Tool to Correct Mistyped Commands with fuck
Last Updated on 2024-09-19 by Clay
Linux has so many useful tools, and I truly want to document every single one of them. To celebrate Linux reaching a usage rate of 4.55% on StatCounter (2024-09-18), I've decided to document another tool recommended by a colleague—the fuck command.
[Linux] Efficient Directory Navigation On Linux: Mastering The `z` Command
Last Updated on 2024-09-17 by Clay
Introduction
z command is something I've wanted to write about for a long time! However, I've been busy with AI training (company work) and model acceleration (personal interest), so I haven't had the time. Let's put it this way, if someone asks me to recommend essential tools for a Linux system, I would undoubtedly place z in my top ten list.
[Python] How to Use FastAPI's Auto-Generated Documentation (Including Exporting Static Files)
Last Updated on 2024-09-15 by Clay
Introduction
In the FastAPI official documentation, there's a section about 'Automatic Interactive API Documentation':
- Robust: Get production-ready code. With automatic interactive documentation.
Troubleshooting Accelerated Inference of Gemma-2 on V100 GPUs Using vLLM
Last Updated on 2024-09-14 by Clay
Problem Description
Recently, I've achieved some good application results by fine-tuning Gemma-2. However, I encountered various errors when deploying it on the client's equipment, which was quite frustrating. Currently, there isn't a systematic troubleshooting guide online, so I'm documenting it here.
Read More »Troubleshooting Accelerated Inference of Gemma-2 on V100 GPUs Using vLLM
[Python] How To Use @contextmanager Decorator
Last Updated on 2024-09-12 by Clay
In Python, the context manager decorator @contextmanager from the contextlib module allows developers to conveniently create our own context manager.
[PyTorch] Traversing Every Layer of a Neural Network in a Model
Last Updated on 2024-09-10 by Clay
Introduction
Recently, due to some serendipitous events, I had a chance to modify the architecture of a model slightly. I took this opportunity to explore how to iterate and print the layers of neural networks in PyTorch.
Read More »[PyTorch] Traversing Every Layer of a Neural Network in a Model
OpenAI Triton Note (2): Fused Softmax
Last Updated on 2024-09-09 by Clay
Introduction
Softmax is a commonly used activation function, and it is often employed as the last layer in multi-class classification.
OpenAI Triton Note (1): Vector Addition
Last Updated on 2024-09-08 by Clay
Introduction
Triton is an open-source GPU programming language compiler released by OpenAI in 2021. Over recent years, it has become increasingly popular among developers for writing and optimizing parallel programs on GPUs. Compared to traditional libraries such as CUDA or OpenCL, Triton offers a Python-like syntax, making it more readable and easier to learn.
Read More »OpenAI Triton Note (1): Vector Addition