darch (version 0.12.0)

maxoutUnit: Maxout / LWTA unit function

Description

The function calculates the activation of the units and returns a list, in which the first entry is the result through the maxout transfer function and the second entry is the derivative of the transfer function.

Usage

maxoutUnit(input, ..., poolSize = getParameter(".darch.maxout.poolSize", 2,
  ...), unitFunc = getParameter(".darch.maxout.unitFunction", linearUnit,
  ...), dropoutMask = vector())

Arguments

input

Input for the activation function.

...

Additional parameters, passed on to inner unit function.

poolSize

The size of each maxout pool.

unitFunc

Inner unit function for maxout.

dropoutMask

Vector containing the dropout mask.

Value

A list with the maxout activation in the first entry and the derivative of the transfer function in the second entry.

Details

Maxout sets the activations of all neurons but the one with the highest activation within a pool to 0. If this is used without maxoutWeightUpdate, it becomes the local-winner-takes-all algorithm, as the only difference between the two is that outgoing weights are shared for maxout.

References

Srivastava, Rupesh Kumar, Jonathan Masci, Sohrob Kazerounian, Faustino Gomez, and Juergen Schmidhuber (2013). "Compete to Compute". In: Advances in Neural Information Processing Systems 26. Ed. by C.J.C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger. Curran Associates, Inc., pp. 2310-2318. URL: http://papers.nips.cc/paper/5059-compete-to-compute.pdf

Goodfellow, Ian J., David Warde-Farley, Mehdi Mirza, Aaron C. Courville, and Yoshua Bengio (2013). "Maxout Networks". In: Proceedings of the 30th International Conference on Machine Learning, ICML 2013, Atlanta, GA, USA, 16-21 June 2013, pp. 1319-1327. URL: http://jmlr.org/proceedings/papers/v28/goodfellow13.html

See Also

Other darch unit functions: exponentialLinearUnit, linearUnit, rectifiedLinearUnit, sigmoidUnit, softmaxUnit, softplusUnit, tanhUnit

Examples

# NOT RUN {
data(iris)
# LWTA:
model <- darch(Species ~ ., iris, c(0, 50, 0),
 darch.unitFunction = c("maxoutUnit", "softmaxUnit"),
 darch.maxout.poolSize = 5, darch.maxout.unitFunction = "sigmoidUnit")
# Maxout:
model <- darch(Species ~ ., iris, c(0, 50, 0),
 darch.unitFunction = c("maxoutUnit", "softmaxUnit"),
 darch.maxout.poolSize = 5, darch.maxout.unitFunction = "sigmoidUnit",
 darch.weightUpdateFunction = c("weightDecayWeightUpdate", "maxoutWeightUpdate"))
# }