First, let's introduce BigCode! BigCode is an open science collaboration project co-led by Hugging Face and ServiceNow, with the goal of jointly code large language models (LLMs) that can be applied to "programming."
Read More »BigCode & StarCoder: Large Language Model Projects Specialized in Programming
Today I used transformers package to build an ALBERT model to process a NLP task in the offline environment, and I got an error message when loading the ALBERT tokenizer:
Read More »[Solved] "TypeError: not a string" at Using AutoTokenizer.from_pretrained()
Today I was testing the simpletransformers that is a well-known python deep learning package. But on my notebook, my training was stuck and not continue.
Read More »[Solved] Simpletransformers Stuck at Training And Not Continue
The task of using Python for Natural Language Processing (NLP) is quite common in the field of machine learning and deep learning. Of course, the tools and libraries for text preprocessing are avaiable.
Read More »[Python] How to Convert String Full-width into Half-width Character
aitextgen is a great Python pacakge, it allows user just need to write a few codes to configure the complex AI model. Its architecture is used OpenAI's GPT-2 and EleutherAI's GPT Neo/GPT-3, and it can receive pre-trained model to fine-tune.
Read More »[NLP] Use aitextgen Python Package To Generate Text
Today, when I was processing the English corpus, I used NLTK, a classic natural language processing tool in Python, to segment the text again. But when I use the following program to use the following program to import stopwords:
Read More »[Solved] NLTK ImportError: cannot import name 'stopwords' TF-IDF (Term Frequency - Inverse Document Frequency) is a famous word weighting technology, it can show the importance of words to texts.
Read More »[NLP] The TF-IDF In Text Mining
If you are using PyTorch to processing NLP tasks, you must be familiar with nn.Embedding() in PyTorch.
BLEU (Bilingual Evaluation Understudy) is an algorithm for evaluating text similarity, which is often used to evaluate the similarity between machine translation and human translation.
Read More »[NLP] Use BLEU To Calculate Similarity Of Sentences
GRU (Gate Recurrent Unit), I am not sure whether there is a Chinese translate of its name. Like LSTM, it is a variant of RNN (Recurrent Neural Network), and like LSTM, it aims to solve the gradient problem in RNN.
Read More »Graphical Introduction Note About GRU
