[Solved] RuntimeError: OrderedDict mutated during iteration - OrderedDict Could Not Changed

Last Updated on 2023-12-02 by Clay

Problem

Today I tried to load the model weights file after torch.compile(), and then I found the weights is stored by OrderedDict data structure. This data structure is an ordered sequence, in other word its content needs to follow the order.

The error message: RuntimeError: OrderedDict mutated during iteration occurred, Because I tried to use for-loop and change the key name from it.

---

Solution

The solution is very easy.

1. Fixed the iteration value: If you want to fixed the OrderedDict key value, you can store the key in a List before you iterate it.
2. Do no change the OrderedDict value: At least, don't do it while iterating.

The following is a simple example. I tried to change the key name:

for key in ordered_dict:
    ordered_dict[key.replace("name1", "name2")] = ordered_dict.pop(key)

It cloud be worked in common Dict but not OrderedDict.

We can try to convert the iteration object from OrderedDict to List. It will fixed the key name we iterate.

for key in list(ordered_dict.keys()):
    ordered_dict[key.replace("name1", "name2")] = ordered_dict.pop(key)

---

References

• Rename a dictionary key
• RuntimeError: OrderedDict mutated during iteration (Python3)

---

Read More

• [Solved] Some weights of the model checkpoint at distilbert-base-multilingual-cased were not used when initializing DistilBertForSequenceClassification: ['vocab_projector.bias', 'vocab_layer_norm.bias', 'vocab_layer_norm.weight', 'vocab_transform.weight', 'vocab_transform.bias']