In the current deep learning framework, whether using Tensorflow, PyTorch or others… the input and output of the model, the flow of data, we need to pay great attention to the shape of data. Because if you are not careful, the model will return an error message.
The following error is exactly this type:
'ValueError: too many values to unpack (expected 2)', upon training a Bert binary classification model
This error message is telling us that the data we passed into the BERT model has too many dimensions. Usually the dimension that we passed to the HuggingFace BERT model should be (batch_size, seq_len), but it may be accidentally extra dimension, which will cause the above error.
Solution
The input data we passed to BERT may be input_ids and attention_mask. We can print their shape out with shape.
If the shape is really different from the input required by the model, then see how to adjust it. For example, when the shape of the input is unexpectedly (1, batch_size, seq_len), we can use:
input_ids = input_ids.squeeze(0)
…to remove the wrong shape. Of course, it needs to be adjusted according to different shapes. It is case by case.
References
- Huggingface Transformers returning ‘ValueError: too many values to unpack (expected 2)’, upon training a Bert binary classification model
- ValueError: too many values to unpack (expected 2) when using BertTokenizer
Read More
- [Solved] huggingface/tokenizers: The current process just got forked. after parallelism has already been used. Disabling parallelism to avoid deadlocks
- [Solved] Simpletransformers Stuck at Training And Not Continue
- [Solved][PyTorch] IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)