Last Updated on 2024-08-18 by Clay Gaussian Error Linear Unit (GELU) is an activation function used in machine learning. While it resembles the classic ReLU (Rectified Linear Unit), there are some key differences. ReLU is a piecewise linear function that outputs 0 for inputs less than 0, and outputs the input itself for inputs greater … Continue reading [Machine Learning] Note of Activation Function GELU