owlqn
Author: Elmar Langholz
owlqn
or Orthant-Wise Limited-memory Quasi-Newton is a minimizer used for training machine learning models. It minimizes functions of the form f(w) = loss(w) + C |w|_1
, where loss
is an arbitrary differentiable convex loss function, and |w|_1
is the L1 norm of the weight (parameter) vector. It is based on the LBFGS Quasi-Newton algorithm, with modifications to deal with the fact that the L1 norm is not differentiable. It was developed and published by Galen Andrew and Jianfeng Gao on Scalable training of L1-regularized log-linear models.
Two examples of machine learning models used are the following:
- L1-regularized least squares models, where OWL-QN finds weights
w
that minimize:sum_i 0.5 |<w, x_i> - y_i|^2_2 + C |w|_1
- L1-regularized logistic regression models, where OWL-QN finds weights
w
that minimize:sum_i log_loss(w | x_i, y_i) + C |w|_1
Installation
npm install owlqn
Usage
// Include dependenciesvar OWLQN = OWLQN;var LogisticRegression = LogisticRegression; // Declare logistic regression problemvar logisticRegressionOptions = features: 1 2 3 4 5 6 7 8 9 10 labels: -1 -1 -1 -1 1 -1 1 -1 1 1 l2Weight: 00;var logisticRegression = logisticRegressionOptions; // Define owl-qn optimizer and minimizevar owlqn = ;var minimizeOptions = differentiableFunction: logisticRegression initializationVector: 0 0 0 0 0 0 0 0 0 0 l1Weight: 01 memoryParameter: 10 convergenceTolerance: 1e-4;;var result = owlqn;