Machine Learning

Using CuPy to Accelerate Matrix Operations with GPU

Clay
2024-08-072024-08-07
Python

Introduction

CuPy is an open-source GPU-accelerated numerical computation library designed for deep learning and scientific computing. It shares many of the same methods and functions as the popular NumPy package in Python but extends its capabilities to perform computations on the GPU. In short, tasks that can benefit from parallel computation on the GPU, such as matrix operations, can achieve significant acceleration with CuPy.

Supervised Fine-tuning Trainer (SFTTrainer) Note

Clay
2024-08-022024-08-02
Machine Learning, PyTorch

Introduction

Supervised Fine-Tuning (SFT) is one of the most well-known methods for training Large Language Models (LLM). Essentially, it is similar to traditional language modeling, where the model learns certain knowledge through training data.

LLM Fine-tuning Note – Differences Between SFT and DPO

Clay
2024-08-022024-08-02
Machine Learning

Introduction

In the fine-tuning tasks of Large Language Models (LLM), several methods such as Supervised Fine-Tuning (SFT), Reinforcement Learning from Human Feedback (RLHF), and Direct Preference Optimization (DPO) are all viable approaches. However, there are some differences among them.

Using vLLM To Accelerate Inference Speed By Continuous Batching

Clay
2024-07-312024-07-31
AI, Machine Learning

Introduction

I previously wrote a note introducing the vLLM accelerated inference framework (Using vLLM To Accelerate The Decoding Of Large Language Model), but due to space and time constraints, I couldn’t delve into more detailed features.

Note Of HuggingFace Text Generation Inference (TGI)

Clay
2024-07-312024-07-31
AI, Machine Learning

Introduction

HuggingFace’s Text Generation Inference (TGI) is a framework specifically designed to deploy and accelerate LLM inference services. Below is its architecture diagram:

Use Text To Retrieve Images: Introduction Of Multi-Modals ColPali

Clay
2024-07-302024-07-31
AI, Machine Learning

Introduction

Since last year, I have been filled with enthusiasm and curiosity about Multi-Modal AI models. As a staunch advocate of AGI, I believe that AI’s current potential has not yet reached its ceiling. One significant bottleneck and research direction in AI today is naturally the integration of various modalities (text, images, audio…) in model applications.

Meta-llama–Prompt-Guard-86M: Open-Source Model for Prompt Protection, Detecting Malicious Attacks

Clay
2024-07-292024-07-29
AI, Machine Learning

Recently, Meta AI has released various versions of Llama 3.1 (405B, 70B, 8B), with the 405B model being particularly noteworthy. It’s the first time an open-source LLM has caught up with closed-source models like GPT-4 and Claude-3.5. At the same time, Meta AI has also released a smaller model called Prompt-Guard-86M.

Use `snapshot_download` To Download The Models Of HuggingFace Hub

Clay
2024-07-222024-07-22
Linux, Machine Learning, Python

Introduction

HuggingFace Model Hub is now a widely recognized and essential open-source platform for every one. Every day, countless individuals and organizations upload their latest trained models (including those for text, images, speech, and other domains) to this platform. It can be said that anyone working in AI-related fields frequently browses the HuggingFace platform website.

[Paper Reading] Mistral 7B

Clay
2024-07-212024-07-25
Machine Learning

Introduction

Mistral 7B is a large language model (LLM) proposed on September 27, 2023, trained by the Mistral AI team, which also released its weights as open source. Interestingly, it uses the highly permissive Apache 2.0 license, unlike Llama 2, which has its own Llama license terms. Therefore, Mistral 7B is truly “open source” (Llama’s license requires discussion with Meta AI when the service volume reaches 700 million).

PaddleOCR: A Framework and Model Specialized in Chinese Optical Character Recognition (OCR)

Clay
2024-07-202024-07-20
Machine Learning, Python

Introduction

Recently, I have been exploring models used for Optical Character Recognition (OCR). In the past, OCR was a very popular research field as it was one of the earliest practical applications of computer vision. Today, OCR has become a very mature task, and you can easily find high-performance open-source models online.

« Previous
1
…
3
4
5
6
7
…
15
Next »