August 19, 2024

[Machine Learning] Note of LayerNorm

Last Updated on 2024-08-19 by Clay

The working principle of LayerNorm is as follows:

$mean = \mu =\frac{\sum_{i=1}^{N}x_i}{N} \newline variance = \sigma^2 = \frac{\sum_{i=1}^{N}(x_{i}-\mu)^2}{N}$ Read More »