I previously wrote a note introducing the vLLM accelerated inference framework (Using vLLM To Accelerate The Decoding Of Large Language Model), but due to space and time constraints, I couldn't delve into more detailed features.
Read More »Using vLLM To Accelerate Inference Speed By Continuous Batching
HuggingFace's Text Generation Inference (TGI) is a framework specifically designed to deploy and accelerate LLM inference services. Below is its architecture diagram:
Read More »Note Of HuggingFace Text Generation Inference (TGI)
Since last year, I have been filled with enthusiasm and curiosity about Multi-Modal AI models. As a staunch advocate of AGI, I believe that AI's current potential has not yet reached its ceiling. One significant bottleneck and research direction in AI today is naturally the integration of various modalities (text, images, audio...) in model applications.
Read More »Use Text To Retrieve Images: Introduction Of Multi-Modals ColPali
Recently, Meta AI has released various versions of Llama 3.1 (405B, 70B, 8B), with the 405B model being particularly noteworthy. It's the first time an open-source LLM has caught up with closed-source models like GPT-4 and Claude-3.5. At the same time, Meta AI has also released a smaller model called Prompt-Guard-86M.
Read More »Meta-llama--Prompt-Guard-86M: Open-Source Model for Prompt Protection, Detecting Malicious Attacks
Previously, we finished the configuration of ComfyUI, now we can try to build a basic and simplest workflow. The workflow is the most different point with stable-diffusion-webui. ComfyUI uses a card-based process that makes it easier to understand how the Stable Diffusion model actually performs inference and also makes it easier to customize and achieve more advanced effects.
Read More »Stable Diffusion ComfyUI Note 02 - Build The Basic Workflow
Those who play with Stable Diffusion AI-generated images have likely heard of stable-diffusion-webui. It is a visual interface that supports the Stable Diffusion model framework, allowing users to perform inference with AI models without having to write code or deal with complicated command-line operations. ComfyUI, on the other hand, is a slightly more niche front-end interface, but it has quickly garnered a loyal fan base due to its flexibility and customizability. Essentially, it can be seen as a more advanced version of stable-diffusion-webui, though it is less user-friendly.
Read More »Stable Diffusion ComfyUI Note 01 - Download And Installation
