vLLM is a large language model (LLM) acceleration framework that developed by the research team of University of California, Berkeley. It is used PagedAttention to increase the usage rate of GPU VRAM, and this method do not change the model architecture.
Read More »Using vLLM To Accelerate The Decoding Of Large Language Model
Last night I tried to improve some code about merge two models. Because my poor device, I cannot merge all layers totally, but need to merge them layer by layer for reducing my memory cost.
Read More »[PyTorch] Delete Model And Free Memory (GPU / CPU)
Although I reach AI and research it all day, but there are too many new architectures, breakout, theories… many things I just know a rough concepts and put it away. The almost deeper understand need to until I must to be adjust the model architecture.
Read More »[Machine Learning] A LLM (Large Language Model) Visualization WebsiteIn early 2023, The pytorch 2.0 version added a new function that is torch.compile(), It cloud be accelerate the speed of model training or inference. By using precision training with compiling, it always make my training time reduce to half.
When using the transformers package, often we receive the following error message when we utilize different task models with various heads like AutoModelForSequenceClassification, AutoModelForSeq2SeqLM…
Read More »[Solved] Some weights of the model checkpoint at distilbert-base-multilingual-cased were not used when initializing DistilBertForSequenceClassification: [‘vocab_projector.bias’, ‘vocab_layer_norm.bias’, ‘vocab_layer_norm.weight’, ‘vocab_transform.weight’, ‘vocab_transform.bias’]
Moderation endpoint is a free tool that developed by OpenAI. It is design for detecting “Not Mode-rational Content” or “Sensitive Content“. The more detailed list you can refer https://openai.com/policies/usage-policies.
Read More »Using OpenAI Moderation Endpoint To Dectect Not Moderation Content
Today when I training the model on my server, I write an GPU parallel training script, and input the newest training data; but in the process of training, I got an error message about “GPU not found“. After I using torch.cuda.is_available() to check my device, I got another error message as following:
Meta AI has indeed been incredibly powerful recently, seemingly securing its position as a giant in AI research and development in no time at all, and what’s more, it sets the bar high with all its top-tier open-source contributions. From Segment Anything that can segment objects in the image domain, to the public large language model and foundational model, LLaMA (yes, the one causing the llama family appear!), to the recent ImageBind that can transform six modalities and the Massively Multilingual Speech (MMS) project… I must say, for an ordinary person like me, it’s quite an effort to keep up with how to use these technologies, let alone trying to chase their technical prowess.
Read More »ImageBind: A Experience Notes on a Multimodal Vector Transformation Model
First, let’s introduce BigCode! BigCode is an open science collaboration project co-led by Hugging Face and ServiceNow, with the goal of jointly code large language models (LLMs) that can be applied to “programming.”
Read More »BigCode & StarCoder: Large Language Model Projects Specialized in ProgrammingIn the current deep learning framework, whether using Tensorflow, PyTorch or others… the input and output of the model, the flow of data, we need to pay great attention to the shape of data. Because if you are not careful, the model will return an error message.
Read More »[Solved] HuggingFace Transformers Model Return “‘ValueError: too many values to unpack (expected 2)’, upon training a Bert binary classification model”
