L-BFGS-B

Software for Large-scale Bound-constrained Optimization

L-BFGS-B is a limited-memory quasi-Newton code for bound-constrained optimization, i.e., for problems where the only constraints are of the form l <= x <= u. It is intended for problems in which information on the Hessian matrix is difficult to obtain, or for large dense problems. L-BFGS-B can also be used for unconstrained problems, and in this case performs similarly to its predecessor, algorithm L-BFGS (Harwell routine VA15). The algorithm is implemented in Fortran 77.

Authors

• Ciyou Zhu
• Richard Byrd
• Jorge Nocedal
• Jose Luis Morales

Related Publications

• R. H. Byrd, P. Lu, J. Nocedal and C. Zhu. A Limited Memory Algorithm for Bound Constrained Optimization (1995), SIAM Journal on Scientific and Statistical Computing, Vol. 16, Num. 5, pp. 1190-1208
• C. Zhu, R. H. Byrd and J. Nocedal. L-BFGS-B: Algorithm 778: L-BFGS-B, FORTRAN routines for large scale bound constrained optimization (1997), ACM Transactions on Mathematical Software, Vol. 23, Num. 4, pp. 550-560
• J.L. Morales and J. Nocedal. L-BFGS-B: Remark on Algorithm 778: L-BFGS-B, FORTRAN routines for large scale bound constrained optimization (2011), ACM Transactions on Mathematical Software, Vol. 38, Num. 1
• R. H. Byrd, J. Nocedal and R. B. Schnabel. Representations of quasi-Newton matrices and their use in limited memory methods (1994), Mathematical Programming, Vol. 63, pp. 129-156

Note that the subspace minimization in the LBFGSpp implementation is an exact minimization subject to the bounds, based on the BOXCQP algorithm:

• C. Voglis and I. E. Lagaris, BOXCQP: An Algorithm for Bound Constrained Convex Quadratic Problems (2004), 1st International Conference "From Scientific Computing to Computational Engineering", Athens, Greece

For an eagle-eye overview of L-BFGS-B and the genealogy BFGS->L-BFGS->L-BFGS-B, see Henao's Master's thesis.

Related Software

• wilmerhenao/L-BFGS-B-NS: An L-BFGS-B-NS Optimizer for Non-Smooth Functions
• pcarbo/lbfgsb-matlab: A MATLAB interface for L-BFGS-B
• bgranzow/L-BFGS-B: A pure Matlab implementation of L-BFGS-B (LBFGSB)
• constantino-garcia/lbfgsb_cpp_wrapper: A simple C++ wrapper around the original Fortran L-BGSG-B routine
• yixuan/LBFGSpp: A header-only C++ library for L-BFGS and L-BFGS-B algorithms
• chokkan/liblbfgs: libLBFGS: a library of Limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS)
• mkobos/lbfgsb_wrapper: Java wrapper for the Fortran L-BFGS-B algorithm
• yuhonglin/Lbfgsb.jl: A Julia wrapper of the l-bfgs-b fortran library
• Gnimuc/LBFGSB.jl: Julia wrapper for L-BFGS-B Nonlinear Optimization Code
• afbarnard/go-lbfgsb: L-BFGS-B optimization for Go, C, Fortran 2003
• nepluno/lbfgsb-gpu: An open source library for the GPU-implementation of L-BFGS-B algorithm
• Chris00/L-BFGS-ocaml: OCaml bindings for L-BFGS
• dwicke/L-BFGS-B-Lua: L-BFGS-B lua wrapper around a L-BFGS-B C implementation
• avieira/python_lbfgsb: Pure Python-based L-BFGS-B implementation
• ybyygu/rust-lbfgsb: Ergonomic bindings to L-BFGS-B code for Rust
• rforge/lbfgsb3c: Limited Memory BFGS Minimizer with Bounds on Parameters with optim() 'C' Interface for R
• florafauna/optimParallel-python: A parallel version of ‘scipy.optimize.minimize(method='L-BFGS-B’)`

Notes on this repository

I (J. Schilling) took the freedom to

• put the L-BFGS-B code obtained from the original website up in this repository,
• divide the subroutines and functions into separate files,
• convert parts of the documentation into a format understandable to doxygen and
• replace the included BLAS/LINPACK routines with calls to user-provided BLAS/LAPACK routines. To be precise, the calls to LINPACK's dtrsl were replace with calls to LAPACK's dtrsm and the calls to LINPACK's dpofa were replaces with calls to LAPACK's dpotrf.

Porting to another language

A language-neutral specification of the algorithm – sufficient to implement L-BFGS-B in any language with a BLAS/LAPACK binding, without reading the Fortran source – lives in docs/spec/. Start with `docs/spec/README.md`.

The pack includes:

• 9 foundation documents (algorithm overview, glossary, abstract callback-based API, logical state model, numerical conventions, deviations from the published papers, conformance criteria, F77 -> other-language gotchas, and an optional appendix on the F77 reverse-communication interface for ABI compatibility).
• One per-subroutine spec for each in-scope numerical routine.
• ~110 JSON test vectors mirroring the F77 unit-test cases.
• A Python+NumPy reference implementation (docs/spec/reference_impl/).
• A conformance test runner (docs/spec/runner/conformance.py) with strict (bit-exact) and tolerance (per-subroutine numerical) modes.

Building

A CMake setup is provided for L-BFGS-B in this repository. External modules for BLAS and LAPACK have to be installed on your system. Then, building the shared library liblbfgsb.so and the examples driver*.f and driver*.f90 works as follows:

> mkdir build
> cd build
> cmake ..
> make -j

The resulting shared library and the driver executables can be found in the build directory.

Concluding remarks

The current release is version 3.0. The distribution file was last changed on 02/08/11.

This work was in no way intending to infringe any copyrights or take credit for others' work. Feel free to contact me at any time in case you noticed something against the rules. Above documentation is obtained from the archived version of the original manual.

A PDF version of the documentation is available here: L-BFGS-B.pdf.