powered by
The gated linear unit. Computes:
nnf_glu(input, dim = -1)
(Tensor) input tensor
(int) dimension on which to split the input. Default: -1
GLU(a,b)=a⊗σ(b)
where input is split in half along dim to form a and b, σ is the sigmoid function and ⊗ is the element-wise product between matrices.
input
dim
a
b
See Language Modeling with Gated Convolutional Networks.
