activation_gelu

The Gaussian error linear unit (GELU) is defined as:
<code>gelu(x) = x * P(X &lt;= x)</code> where <code>P(X) ~ N(0, 1)</code>,
i.e. <code>gelu(x) = 0.5 * x * (1 + erf(x / sqrt(2)))</code>.
GELU weights inputs by their value, rather than gating
inputs by their sign as in ReLU.

Interface to 'Keras' <https://keras.io>, a high-level neural
networks API. 'Keras' was developed with a focus on enabling fast experimentation,
supports both convolution based networks and recurrent networks (as well as
combinations of the two), and runs seamlessly on both CPU and GPU devices.

Tomasz Kalinowski

keras3

R Interface to 'Keras'

Daniel Falbel

JJ Allaire

François Chollet

Posit Software, PBC 

Google 

Yuan Tang

Wouter Van Der Bijl

Martin Studer

Sigrid Keydana

activation_gelu function

<dl><dt>x</dt>
<dd>Input tensor.</dd>
<dt>approximate</dt>
<dd>A <code>bool</code>, whether to enable approximation.</dd></dl>

Arguments

<ul>
<li><a href="https://arxiv.org/abs/1606.08415">Hendrycks et al., 2016</a></li>
</ul>

Reference

Gaussian error linear unit (GELU) activation function. — activation_gelu

<dl>

<dt>x</dt>
<dd>Input tensor.</dd>


<dt>approximate</dt>
<dd>A <code>bool</code>, whether to enable approximation.</dd>

</dl>

<ul>
<li><a href='https://arxiv.org/abs/1606.08415'>Hendrycks et al., 2016</a></li>
</ul>

activation_gelu: Gaussian error linear unit (GELU) activation function.

Description

Usage

Value

Arguments

Reference

See Also