使用 vLLM 進行大型語言模型(LLM)的高速推理
介紹
vLLM 是加州柏克萊分校所開發的一種大型語言模型(Large Language Model, LLM)加速推理框架。它主要是利用 PagedAttention 機制提高了 GPU VRAM 的使用率,並且這一方法無須更改模型的架構。
Read More »使用 vLLM 進行大型語言模型(LLM)的高速推理vLLM 是加州柏克萊分校所開發的一種大型語言模型(Large Language Model, LLM)加速推理框架。它主要是利用 PagedAttention 機制提高了 GPU VRAM 的使用率,並且這一方法無須更改模型的架構。
Read More »使用 vLLM 進行大型語言模型(LLM)的高速推理Given the root
of a binary tree, construct a string consisting of parenthesis and integers from a binary tree with the preorder traversal way, and return it.
You are given a string num
, representing a large integer. Return the largest-valued odd integer (as a string) that is a non-empty substring of num
, or an empty string ""
if no odd integer exists.
在 2023 年初,PyTorch 的 2.0 版本新增了一個 torch.compile()
的新功能,讓我們能夠在模型訓練/推理時能夠進一步提昇速度。與混合精度訓練的協同工作,經常能使我的訓練速度提昇一倍左右。
今天我在讀取已經被 torch.compile()
之後儲存起來的模型權重,發現模型權重是使用 OrderedDict 資料結構儲存著,而這種結構本身是有序序列,換言之它的資料內容是需要嚴格遵守排序的。
There are 3n
piles of coins of varying size, you and your friends will take piles of coins as follows:
You are given a 0-indexed array of strings garbage
where garbage[i]
represents the assortment of garbage at the ith
house. garbage[i]
consists only of the characters 'M'
, 'P'
and 'G'
representing one unit of metal, paper and glass garbage respectively. Picking up one unit of any type of garbage takes 1
minute.
You are given four integers sx
, sy
, fx
, fy
, and a non-negative integer t
.
In an infinite 2D grid, you start at the cell (sx, sy)
. Each second, you must move to any of its adjacent cells.
You are playing a video game where you are defending your city from a group of n monsters. You are given a 0-indexed integer array dist of size n, where dist[i] is the initial distance in kilometers of the ith monster from the city.
Read More »LeetCode: 1921-Eliminate Maximum Number of Monsters 解題紀錄變分自動編碼器(Variational AutoEncoder, VAE) 是自動編碼器(AutoEncoder, AE)的進階變體,架構與原本的自動編碼器相似,同樣都是由編碼器(Encoder)和解碼器(Decoder)所組成。
Read More »[Machine Learning] Variational AutoEncoder (VAE) 筆記