Skip to content

August 16, 2024

[Machine Learning] Note of Rotary Position Embedding (RoPE)

Last Updated on 2024-08-17 by Clay

Introduction

(Note: Since this article is imported from my personal Hackmd, some symbols and formatting might not display properly in WordPress. I appreciate your understanding, sorry for any inconvenience.)

RoPE is a method for introducing relative position information into the self-attention mechanism through absolute positional encoding.

Read More »[Machine Learning] Note of Rotary Position Embedding (RoPE)