I previously wrote a note introducing the vLLM accelerated inference framework (Using vLLM To Accelerate The Decoding Of Large Language Model), but due to space and time constraints, I couldn’t delve into more detailed features.
Read More »Using vLLM To Accelerate Inference Speed By Continuous Batching
HuggingFace’s Text Generation Inference (TGI) is a framework specifically designed to deploy and accelerate LLM inference services. Below is its architecture diagram:
Read More »Note Of HuggingFace Text Generation Inference (TGI)
Since last year, I have been filled with enthusiasm and curiosity about Multi-Modal AI models. As a staunch advocate of AGI, I believe that AI’s current potential has not yet reached its ceiling. One significant bottleneck and research direction in AI today is naturally the integration of various modalities (text, images, audio…) in model applications.
Read More »Use Text To Retrieve Images: Introduction Of Multi-Modals ColPali
Recently, Meta AI has released various versions of Llama 3.1 (405B, 70B, 8B), with the 405B model being particularly noteworthy. It’s the first time an open-source LLM has caught up with closed-source models like GPT-4 and Claude-3.5. At the same time, Meta AI has also released a smaller model called Prompt-Guard-86M.
Read More »Meta-llama–Prompt-Guard-86M: Open-Source Model for Prompt Protection, Detecting Malicious Attacks
Although PowerPoint (PPT) by Microsoft has always been the go-to for creating presentations, in recent years I’ve found myself preferring Google Slides for making slides.
Read More »[Google Slides] How To Insert Math Formula
In Python programming, we often use the requests module for HTTP requests. However, requests can become a bottleneck when connecting frontend and backend services due to its synchronous request handling. Recently, I experienced Kubernetes probe blockages caused by using requests, which led to the unintended deletion of my service container. In such scenarios, httpx might be a more suitable module for asynchronous request handling.
Previously, we finished the configuration of ComfyUI, now we can try to build a basic and simplest workflow. The workflow is the most different point with stable-diffusion-webui. ComfyUI uses a card-based process that makes it easier to understand how the Stable Diffusion model actually performs inference and also makes it easier to customize and achieve more advanced effects.
Read More »Stable Diffusion ComfyUI Note 02 – Build The Basic Workflow
In Linux System, If we want to use USB device for accessing our data, we always need to connect our USB device to the computer, and then open the folder manager, click the USB device that computer detected.
Read More »[Linux] How To Mount An USB Device
Those who play with Stable Diffusion AI-generated images have likely heard of stable-diffusion-webui. It is a visual interface that supports the Stable Diffusion model framework, allowing users to perform inference with AI models without having to write code or deal with complicated command-line operations. ComfyUI, on the other hand, is a slightly more niche front-end interface, but it has quickly garnered a loyal fan base due to its flexibility and customizability. Essentially, it can be seen as a more advanced version of stable-diffusion-webui, though it is less user-friendly.
Read More »Stable Diffusion ComfyUI Note 01 – Download And Installation
HuggingFace Model Hub is now a widely recognized and essential open-source platform for every one. Every day, countless individuals and organizations upload their latest trained models (including those for text, images, speech, and other domains) to this platform. It can be said that anyone working in AI-related fields frequently browses the HuggingFace platform website.
Read More »Use `snapshot_download` To Download The Models Of HuggingFace Hub
