Blog

[Paper Reading] Kangaroo: Lossless Self-Speculative Decoding via Double Early Exiting

Clay
2024-06-032024-07-25
Machine Learning, Python

Introduction

The accelerated framework is proposed by Huawei Noah's Ark Lab, it replaces the small model used in the original speculative decoding with the shallow sub-network of the large model. Additionally, it employs an extra-trained adapter and the model’s own decoding head to generate speculative tokens, which are then verified by the large model. The subsequent operations are quite similar to the original speculative decoding process.

Note Of RESTful (With Python FastAPI + CURL Example)

Clay
2024-06-022024-06-02
Linux, Python

Introduction

RESTful design (Representational State Transfer, REST) is an architectural style for designing network applications. It follows principles that make network applications simpler, more scalable, and easier to maintain.

Note Of Universally Unique Identifier (UUID)

Clay
2024-06-012024-06-01
Computer

Introduction

When assigning identifiers to our data, if we want each data to have a unique identifier rather than a simple sequential number, UUID is the most common method we used.

So, what is UUID?

Defense Note Against Prompt Injection Attack

Clay
2024-02-262024-02-26
Machine Learning

What is Prompt Injection Attack?

Prompt injection attacks are a burgeoning security concern, primarily targeting large language models (LLMs) or other AI-related domains.

[Solved] Where Does Loss Calculation Begin When Multiple `response_template` Exist in Training Data Using SFTTrainer?

Clay
2024-02-252024-02-25
Machine Learning

Problem

SFTTrainer is a LLM fine-tuning tool provided by HuggingFace team, that can easily adjust many hyper-parameters and config at the fine-tuning task. In the process, response_template is the special string template we need to pass into the tool, any response right by it will be computed the loss.

[Paper Reading] Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection

Clay
2024-01-222024-07-25
Machine Learning

Introduction

RAG-based LLM is a well-known architecture in current usage of Large Language Models (LLM). It involves "retrieval" to provide the model with prior knowledge that it lacks during training, enabling the model to answer questions in the context of specific information.

[Paper Reading] QLoRA: Efficient Finetuning of Quantized LLMs

Clay
2024-01-212024-07-25
Machine Learning

Introduction

The wave of large models has been unstoppable since the release of ChatGPT in November 2022. Up to now, the scale of open-source Large Language Models (LLMs) continues to increase, such as LLaMA-2-70B and Falcon-180B, to name a few.

an artist s illustration of artificial intelligence ai this image represents how machine learning is inspired by neuroscience and the human brain it was created by novoto studio as par

[Paper Reading] The Forward-Forward Algorithm: Some Preliminary Investigation

Clay
2024-01-182024-07-25
Machine Learning

Introduction

Paper link: https://arxiv.org/abs/2212.13345

The author of this research work is the renowned figure in the field of deep learning, Geoffrey Hinton, who was originally a researcher at Google Brain when he initially wrote this paper (he left in 2023).

[GitHub] How To Create A Pull Request (PR)

Clay
2024-01-152024-01-15
Git, Github

Introduction

Sending a Pull Request (PR) on GitHub to an open-source project is a wonderful yet significant endeavor.

[PyTorch] Release GPU / CPU Memory After Delete Model

Clay
2024-01-092024-01-15
Python, PyTorch

Problem

Yesterday, I developed a model merging program. This time I have no enough gpu memory to merge the models in only one time, so I tried to merge layer by layer. I found the memory of GPU is easily to release but CPU didn't.

« Previous
1
…
5
6
7
8
9
…
77
Next »