[Machine Learning] Note of Rotary Position Embedding (RoPE)
Last Updated on 2024-08-17 by Clay
Introduction
(Note: Since this article is imported from my personal Hackmd, some symbols and formatting might not display properly in WordPress. I appreciate your understanding, sorry for any inconvenience.)
RoPE is a method for introducing relative position information into the self-attention mechanism through absolute positional encoding.
Read More »[Machine Learning] Note of Rotary Position Embedding (RoPE)