Function gelu
pub fn gelu<const D: usize, B>(tensor: Tensor<B, D>) -> Tensor<B, D>where
B: Backend,
Expand description
Applies the Gaussian Error Linear Units function as described in the paper Gaussian Error Linear Units (GELUs).
$$ \text{GELU}(x) = x \cdot \Phi(x) = x \cdot \frac{1}{2}\left(1 + \text{erf}\left(\frac{x}{\sqrt{2}}\right)\right) $$
where $\Phi(x)$ is the cumulative distribution function for the Gaussian distribution.