Chinese characters can be roughly divided info "Traditional" and "Simplified". If you need to convert one to another, I think the most convenient tool is OpenCC in Python.
Read More »[NLP][Python] Use OpenCC to convert simplified Chinese to traditional Chinese via opencc-python-reimplementedIf you want to use Python NLP toolkit to analyze Traditional Chinese text, CKIP is your first choice. CKIP is developed by Taiwan Institute of Information Science, Academia Sinica, And won rankings in many competitions.
Read More »[NLP][Python] How to use CKIP to analyzeTraditional ChineseTo segment words from sentence is very important in Chinese. In English you can segment words with space but Chinese cannot.
Let's take an example.
Read More »[NLP][Python] Use "Jieba" package to segment Chinese words
"Word Embedding" is a technology that is often used in natural language processing (NLP), and its concept is convert text into numerical format (numbers).
Read More »[NLP] What is "Word Embedding"
Those who are familiar with natural language processing (NLP) must be familiar with Glove and Python package Gensim.
Glove（Global Vectors for Word Representation）is a paper published by Stanford NLP Group, and it is also an open source pre-trained word embedding model. The Glove that you often see on the Internet now refers to this open source pre-trained model.
Gensim is a Python implementation of the Word2Vec paper proposed by Google in 2013, allowing us to easily train the word vector model using our own corpus via this package.
Read More »[Python] Convert Glove model to a format Gensim can read
word-cloud is a very famous word in natural language processing domain. At first I thought it was just to calculate the frequency of the vocabulary and display the high-frequency words large.
But not only that, the shapes and the styles of characters are all learned not as simple as I thought.
I am a person who likes new things, so I came to study the wordcloud module in Python today and record my experience here.
Read More »Quickly generate word cloud Python module: wordcloudToday when I use Python to process a NLP task, I get a error message:
OSError: [E050] Can't find model 'en'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory.
Cosine Similarity is a common calculation method for calculating text similarity. The basic concept is very simple, it is to calculate the angle between two vectors.
Read More »How to calculate Cosine Similarity (With code)The full text of the NLTK is Nature Language Tool Kit, a package of natural language processing in Python.
Although Chinese can also be processed, but the support for Chinese is not as good as English, so today's examples are all handled by English corpus.
Read More »[NLP][Python] How to use NLTK package to process NLP tasks
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)