LLM Fine-tuning Note – Differences Between SFT and DPO

Clay
2024-08-022024-08-02
Machine Learning

Last Updated on 2024-08-02 by Clay

Introduction

In the fine-tuning tasks of Large Language Models (LLM), several methods such as Supervised Fine-Tuning (SFT), Reinforcement Learning from Human Feedback (RLHF), and Direct Preference Optimization (DPO) are all viable approaches. However, there are some differences among them.

Clay
2024-07-312024-07-31
AI, Machine Learning

Last Updated on 2024-07-31 by Clay

Introduction

I previously wrote a note introducing the vLLM accelerated inference framework (Using vLLM To Accelerate The Decoding Of Large Language Model), but due to space and time constraints, I couldn’t delve into more detailed features.

Clay
2024-07-312024-07-31
AI, Machine Learning

Last Updated on 2024-07-31 by Clay

Introduction

HuggingFace’s Text Generation Inference (TGI) is a framework specifically designed to deploy and accelerate LLM inference services. Below is its architecture diagram:

Clay
2024-07-302024-07-31
AI, Machine Learning

Last Updated on 2024-07-31 by Clay

Introduction

Since last year, I have been filled with enthusiasm and curiosity about Multi-Modal AI models. As a staunch advocate of AGI, I believe that AI’s current potential has not yet reached its ceiling. One significant bottleneck and research direction in AI today is naturally the integration of various modalities (text, images, audio…) in model applications.

Clay
2024-07-292024-07-29
AI, Machine Learning

Last Updated on 2024-07-29 by Clay

Recently, Meta AI has released various versions of Llama 3.1 (405B, 70B, 8B), with the 405B model being particularly noteworthy. It’s the first time an open-source LLM has caught up with closed-source models like GPT-4 and Claude-3.5. At the same time, Meta AI has also released a smaller model called Prompt-Guard-86M.

Clay
2024-07-292024-07-29
Google

Last Updated on 2024-07-29 by Clay

Although PowerPoint (PPT) by Microsoft has always been the go-to for creating presentations, in recent years I’ve found myself preferring Google Slides for making slides.

Clay
2024-07-282024-07-28
Python

Last Updated on 2024-07-28 by Clay

In Python programming, we often use the requests module for HTTP requests. However, requests can become a bottleneck when connecting frontend and backend services due to its synchronous request handling. Recently, I experienced Kubernetes probe blockages caused by using requests, which led to the unintended deletion of my service container. In such scenarios, httpx might be a more suitable module for asynchronous request handling.

Clay
2024-07-272024-08-12
AI

Last Updated on 2024-08-12 by Clay

Introduction

Previously, we finished the configuration of ComfyUI, now we can try to build a basic and simplest workflow. The workflow is the most different point with stable-diffusion-webui. ComfyUI uses a card-based process that makes it easier to understand how the Stable Diffusion model actually performs inference and also makes it easier to customize and achieve more advanced effects.

Clay
2024-07-252024-07-25
Linux

Last Updated on 2024-07-25 by Clay

Introduction

In Linux System, If we want to use USB device for accessing our data, we always need to connect our USB device to the computer, and then open the folder manager, click the USB device that computer detected.

Clay
2024-07-252024-08-12
AI

Last Updated on 2024-08-12 by Clay

What is ComfyUI？

Those who play with Stable Diffusion AI-generated images have likely heard of stable-diffusion-webui. It is a visual interface that supports the Stable Diffusion model framework, allowing users to perform inference with AI models without having to write code or deal with complicated command-line operations. ComfyUI, on the other hand, is a slightly more niche front-end interface, but it has quickly garnered a loyal fan base due to its flexibility and customizability. Essentially, it can be seen as a more advanced version of stable-diffusion-webui, though it is less user-friendly.

« Previous
1
…
7
8
9
10
11
…
81
Next »

LLM Fine-tuning Note – Differences Between SFT and DPO