torch (version 0.2.1)

optim_lbfgs: LBFGS optimizer

Description

Implements L-BFGS algorithm, heavily inspired by minFunc

Usage

optim_lbfgs(
  params,
  lr = 1,
  max_iter = 20,
  max_eval = NULL,
  tolerance_grad = 1e-07,
  tolerance_change = 1e-09,
  history_size = 100,
  line_search_fn = NULL
)

Arguments

params

(iterable): iterable of parameters to optimize or dicts defining parameter groups

lr

(float): learning rate (default: 1)

max_iter

(int): maximal number of iterations per optimization step (default: 20)

max_eval

(int): maximal number of function evaluations per optimization step (default: max_iter * 1.25).

tolerance_grad

(float): termination tolerance on first order optimality (default: 1e-5).

tolerance_change

(float): termination tolerance on function value/parameter changes (default: 1e-9).

history_size

(int): update history size (default: 100).

line_search_fn

(str): either 'strong_wolfe' or None (default: None).

Warning

This optimizer doesn't support per-parameter options and parameter groups (there can be only one).

Right now all parameters have to be on a single device. This will be improved in the future.