nnf_layer_norm

Applies Layer Normalization for last certain number of dimensions.

Provides functionality to define and train neural networks similar to
'PyTorch' by Paszke et al (2019) <arXiv:1912.01703> but written entirely in R
using the 'libtorch' library. Also supports low-level tensor operations and
'GPU' acceleration.

Daniel Falbel

torch

Tensors and Neural Networks with 'GPU' Acceleration

Javier Luraschi

Dmitriy Selivanov

Athos Damiani

Christophe Regouby

Krzysztof Joachimiak

Hamada S. Badr

 RStudio

nnf_layer_norm function

<dl><dt>input</dt>
<dd>the input tensor</dd>
<dt>normalized_shape</dt>
<dd>input shape from an expected input of size. If a single
integer is used, it is treated as a singleton list, and this module will normalize
over the last dimension which is expected to be of that specific size.</dd>
<dt>weight</dt>
<dd>the weight tensor</dd>
<dt>bias</dt>
<dd>the bias tensor</dd>
<dt>eps</dt>
<dd>a value added to the denominator for numerical stability. Default: 1e-5</dd></dl>

Arguments

Layer_norm — nnf_layer_norm

<dl>

<dt>input</dt>
<dd>the input tensor</dd>


<dt>normalized_shape</dt>
<dd>input shape from an expected input of size. If a single
integer is used, it is treated as a singleton list, and this module will normalize
over the last dimension which is expected to be of that specific size.</dd>


<dt>weight</dt>
<dd>the weight tensor</dd>


<dt>bias</dt>
<dd>the bias tensor</dd>


<dt>eps</dt>
<dd>a value added to the denominator for numerical stability. Default: 1e-5</dd>

</dl>

nnf_layer_norm: Layer_norm

Description

Usage

Arguments