This article documents how to check your IP and its location on a Linux system via the terminal. While websites like myip provide this information directly, not all devices have a graphical interface. Moreover, the curl command works on both Windows and Mac OS.
Nginx is a high-performance HTTP server and reverse proxy server often used for web traffic management, load balancing, and HTTP caching. This article focuses on how to configure Nginx to route different API requests to the corresponding services.
Read More »[Linux] Using Nginx to Reverse Proxy and Forward Service Requests to Different Ports on a Single Port
Supervised Fine-Tuning (SFT) is one of the most well-known methods for training Large Language Models (LLM). Essentially, it is similar to traditional language modeling, where the model learns certain knowledge through training data.
Read More »Supervised Fine-tuning Trainer (SFTTrainer) Note
In the fine-tuning tasks of Large Language Models (LLM), several methods such as Supervised Fine-Tuning (SFT), Reinforcement Learning from Human Feedback (RLHF), and Direct Preference Optimization (DPO) are all viable approaches. However, there are some differences among them.
Read More »LLM Fine-tuning Note – Differences Between SFT and DPO
I previously wrote a note introducing the vLLM accelerated inference framework (Using vLLM To Accelerate The Decoding Of Large Language Model), but due to space and time constraints, I couldn’t delve into more detailed features.
Read More »Using vLLM To Accelerate Inference Speed By Continuous Batching
HuggingFace’s Text Generation Inference (TGI) is a framework specifically designed to deploy and accelerate LLM inference services. Below is its architecture diagram:
Read More »Note Of HuggingFace Text Generation Inference (TGI)
Since last year, I have been filled with enthusiasm and curiosity about Multi-Modal AI models. As a staunch advocate of AGI, I believe that AI’s current potential has not yet reached its ceiling. One significant bottleneck and research direction in AI today is naturally the integration of various modalities (text, images, audio…) in model applications.
Read More »Use Text To Retrieve Images: Introduction Of Multi-Modals ColPali
Recently, Meta AI has released various versions of Llama 3.1 (405B, 70B, 8B), with the 405B model being particularly noteworthy. It’s the first time an open-source LLM has caught up with closed-source models like GPT-4 and Claude-3.5. At the same time, Meta AI has also released a smaller model called Prompt-Guard-86M.
Read More »Meta-llama–Prompt-Guard-86M: Open-Source Model for Prompt Protection, Detecting Malicious Attacks
Although PowerPoint (PPT) by Microsoft has always been the go-to for creating presentations, in recent years I’ve found myself preferring Google Slides for making slides.
Read More »[Google Slides] How To Insert Math Formula
In Python programming, we often use the requests module for HTTP requests. However, requests can become a bottleneck when connecting frontend and backend services due to its synchronous request handling. Recently, I experienced Kubernetes probe blockages caused by using requests, which led to the unintended deletion of my service container. In such scenarios, httpx might be a more suitable module for asynchronous request handling.
