When applying Large Language Models (LLMs) in real-world scenarios, it's often not just about letting the model generate text freely. We might want the model to return specific structures, such as multiple-choice questions or providing a rating. In such cases, transformers-based models can directly use the outlines tool.
Read More »Structuring Model Outputs Using the Outlines Tool
Generative models are becoming increasingly powerful, and independent researchers are deploying one open-source large language model (LLMs) after another. However, when using LLMs for inference or generating responses, waiting for a longer output can be quite time-consuming.
Read More »Implementing Streamed Output Token Generation Using TextStreamer and TextIteratorStreamer in HuggingFace Transformers
Variational AutoEncoder (VAE) is an advanced variant of the AutoEncoder (AE). The architecture is similar to the original AutoEncoder, consisting of an encoder and a decoder.
Read More »[Machine Learning] Note Of Variational AutoEncoder (VAE)
Currently, LLM services cover a wide range of fields, and Prompt Injection and Jailbreak threats to LLMs are growing by the day. A few months ago, a customer service LLM even provided incorrect information, leading to a loss of customer rights (although that wasn't caused by a prompt attack).
Microsoft's open-source BIPIA (Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models) evaluation method, although tested six months ago without significant updates since, remains a simple and convenient testing method for the tasks I have at hand.
Read More »Evaluating LLM Defense Capabilities Using the Microsoft BIPIA Framework
difflib is a module in the Python standard library used to compare differences between sequences (often text). Back when I was doing my thesis, I implemented this by hand. It's funny and a bit frustrating to realize now in my work that there's such a neat module for this.
In Python class construction, the @property decorator is commonly used and has significant benefits. Its main purpose is to transform a class method into a read-only attribute, allowing users to retrieve computed results via attribute access.
Newton's interpolation is a polynomial interpolation method that constructs a set of polynomial functions using multiple data points. A major advantage is that with the addition of new data, Newton's interpolation method does not require recalculations from scratch but can instead expand on the existing function.
Read More »Note Of Newton Polynomial
On platforms like LeetCode, if one looks at the solutions provided by top coders after solving problems, one often encounters a peculiar piece of code (C++ only):
Read More »[C++] The meaning of ios_base::sync_with_stdio(false) and cin.tie(NULL) in programming competition
ColBERT is an embedding model designed specifically for retrieval tasks, transforming the tokens of Queries and Documents into embeddings and computing the maximum similarity.
Read More »[Paper Reading] ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
Hadolint is a Dockerfile linter that helps you follow best practices and style guidelines when writing Dockerfiles.
Read More »[Linux] How To Use Hadolint To Avoid Terrible Dockerfile
