[Solved] Some weights of the model checkpoint at distilbert-base-multilingual-cased were not used when initializing DistilBertForSequenceClassification: [‘vocab_projector.bias’, ‘vocab_layer_norm.bias’, ‘vocab_layer_norm.weight’, ‘vocab_transform.weight’, ‘vocab_transform.bias’]
Last Updated on 2023-06-19 by Clay
Problem
When using the transformers package, often we receive the following error message when we utilize different task models with various heads like AutoModelForSequenceClassification, AutoModelForSeq2SeqLM…
Read More »[Solved] Some weights of the model checkpoint at distilbert-base-multilingual-cased were not used when initializing DistilBertForSequenceClassification: [‘vocab_projector.bias’, ‘vocab_layer_norm.bias’, ‘vocab_layer_norm.weight’, ‘vocab_transform.weight’, ‘vocab_transform.bias’]