Today when I using nn.Embedding in PyTorch, because I used it by mistake, the following error occurred:
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: index out of range: Tried to access index 5 out of table with 4 rows.
at /pytorch/aten/src/TH/generic/THTensorEvenMoreMath.cpp:237It took me a lot of work and it is one of the my most clueless error in the process of learning PyTorch.
The topic is far away, let's take a look at the process of this error.
Problem Description
My problem happened very simple. In short, I was testing the following code:
embedding = nn.Embedding(5, 3, padding_idx=0) inputs = torch.tensor([1, 2, 3, 4, 5]) print(embedding(inputs))
Just when I thought I would get a vector with shape [5, 3], I instead got the following error:
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: index out of range: Tried to access index 5 out of table with 4 rows.
at /pytorch/aten/src/TH/generic/THTensorEvenMoreMath.cpp:237After testing for a long time, I finally found where my problem was.
According to the official document, when entering text, I should convert the "word" to "number" in advance.
This very clear but I got the wrong meaning of num_embeddings.
Today is a nice daySo my input is 5 words. After being converted into a number, it may be:
[1, 2, 3, 4, 5]After entering embedding, there will be an error!
In fact, the problem is really simple: num_embedding does not refer to the length of the string currently entered, but the number of Indexes of all different characters, and it is numbered from 0.
So, if it is the following embedding:
embedding = nn.Embedding(5, 3, padding_idx=0)
The entered Index can only be between [0, 4].
It is strongly recommended that you confirm the dimensions of your dictionary before using nn.Embedding().
References
- https://stackoverflow.com/questions/50747947/embedding-in-pytorch
- https://discuss.pytorch.org/t/embeddings-index-out-of-range-error/12582/5
- https://stackoverflow.com/questions/56010551/pytorch-embedding-index-out-of-range
Read More
- [Solved][PyTorch] IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)
- [Solved][PyTorch] TypeError: not a sequence
- [Solved][PyTorch] RuntimeError: bool value of Tensor with more than one value is ambiguous
- [Solved][PyTorch] ValueError: expected sequence of length 300 at dim 1 (got 3)
- [Solved][PyTorch] AttributeError: 'tuple' object has no attribute 'size'