Showing preview only (1,602K chars total). Download the full file or copy to clipboard to get everything.
Repository: rjhogan/Adept-2
Branch: master
Commit: d0a7751a0871
Files: 136
Total size: 1.5 MB
Directory structure:
gitextract_yk4ax793/
├── .gitignore
├── .travis.yml
├── AUTHORS
├── COPYING
├── ChangeLog
├── INSTALL
├── Makefile.am
├── NEWS
├── README.md
├── TODO
├── adept/
│ ├── Array.cpp
│ ├── Makefile.am
│ ├── Minimizer.cpp
│ ├── Stack.cpp
│ ├── StackStorageOrig.cpp
│ ├── Storage.cpp
│ ├── cppblas.cpp
│ ├── cpplapack.h
│ ├── index.cpp
│ ├── inv.cpp
│ ├── jacobian.cpp
│ ├── line_search.cpp
│ ├── minimize_conjugate_gradient.cpp
│ ├── minimize_levenberg_marquardt.cpp
│ ├── minimize_limited_memory_bfgs.cpp
│ ├── settings.cpp
│ ├── solve.cpp
│ └── vector_utilities.cpp
├── benchmark/
│ ├── Makefile.am
│ ├── advection_schemes.h
│ ├── advection_schemes_AD.h
│ ├── advection_schemes_K.h
│ ├── animate.cpp
│ ├── autodiff_benchmark.cpp
│ ├── differentiator.h
│ ├── math_benchmark.cpp
│ ├── matrix_benchmark.cpp
│ └── nx.h
├── config_platform_independent.h.in
├── configure.ac
├── doc/
│ ├── COPYING
│ ├── Makefile
│ ├── README
│ ├── adept_documentation.tex
│ └── adept_reference.tex
├── include/
│ ├── Makefile.am
│ ├── Timer.h
│ ├── adept/
│ │ ├── Active.h
│ │ ├── ActiveConstReference.h
│ │ ├── ActiveReference.h
│ │ ├── Allocator.h
│ │ ├── Array.h
│ │ ├── ArrayWrapper.h
│ │ ├── BinaryOperation.h
│ │ ├── Expression.h
│ │ ├── ExpressionSize.h
│ │ ├── FixedArray.h
│ │ ├── GradientIndex.h
│ │ ├── IndexedArray.h
│ │ ├── Minimizer.h
│ │ ├── Optimizable.h
│ │ ├── Packet.h
│ │ ├── RangeIndex.h
│ │ ├── ScratchVector.h
│ │ ├── SpecialMatrix.h
│ │ ├── Stack.h
│ │ ├── StackStorage.h
│ │ ├── StackStorageOrig.h
│ │ ├── StackStorageOrigStl.h
│ │ ├── Statement.h
│ │ ├── Storage.h
│ │ ├── UnaryOperation.h
│ │ ├── array_shortcuts.h
│ │ ├── base.h
│ │ ├── contiguous_matrix.h
│ │ ├── cppblas.h
│ │ ├── eval.h
│ │ ├── exception.h
│ │ ├── interp.h
│ │ ├── inv.h
│ │ ├── matmul.h
│ │ ├── noalias.h
│ │ ├── outer_product.h
│ │ ├── quick_e.h
│ │ ├── reduce.h
│ │ ├── scalar_shortcuts.h
│ │ ├── settings.h
│ │ ├── solve.h
│ │ ├── spread.h
│ │ ├── store_transpose.h
│ │ ├── traits.h
│ │ ├── vector_utilities.h
│ │ └── where.h
│ ├── adept.h
│ ├── adept_arrays.h
│ ├── adept_fortran.h
│ ├── adept_optimize.h
│ └── create_adept_source_header
├── m4/
│ ├── adept.m4
│ ├── ax_blas.m4
│ ├── ax_lapack.m4
│ ├── ltsugar.m4
│ └── lt~obsolete.m4
├── makefile_include.in
└── test/
├── Makefile
├── README
├── algorithm.cpp
├── algorithm.h
├── algorithm_with_and_without_ad.h
├── rosenbrock_banana_function.cpp
├── run_tests.sh
├── simulate_radiances.cpp
├── simulate_radiances.h
├── state.cpp
├── state.h
├── test_adept.cpp
├── test_adept_with_and_without_ad.cpp
├── test_array_derivatives.cpp
├── test_array_speed.cpp
├── test_arrays.cpp
├── test_checkpoint.cpp
├── test_constructors.cpp
├── test_derivatives.cpp
├── test_fastexp.cpp
├── test_fixed_arrays.cpp
├── test_gsl_interface.cpp
├── test_interp.cpp
├── test_minimizer.cpp
├── test_misc.cpp
├── test_no_lib.cpp
├── test_packet_operations.cpp
├── test_radiances.cpp
├── test_radiances_array.cpp
├── test_reduce_active.cpp
├── test_thread_safe.cpp
└── test_thread_safe_arrays.cpp
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
Makefile.in
/aclocal.m4
/config.guess
/config.h.in
/config.log
/config.sub
/config.status
/configure
/depcomp
/install-sh
/ltmain.sh
/missing
/ar-lib
/autom4te.cache
/compile
/libtool
/stamp-*
*.o
*.a
*.so
*.la
*.tar*
doc/adept_*.log
doc/adept_*.toc
doc/adept_*.aux
doc/adept_*.out
.deps
*~
Makefile
!test/Makefile
!doc/Makefile
include/adept_source.h
================================================
FILE: .travis.yml
================================================
language: cpp
os: linux
sudo: required
dist: trusty
compiler:
- gcc
before_install:
- sudo apt-get install gfortran -y
- type gfortran
install: autoreconf -i && ./configure && make -j8
script:
- make check -j8
- cat test/test_results.txt
================================================
FILE: AUTHORS
================================================
Robin Hogan <r.j.hogan@ecmwf.int>
================================================
FILE: COPYING
================================================
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: ChangeLog
================================================
version 2.1.4 (in progress)
- Added support for the copysign function
- Added aArray::set_gradient(Array) function
version 2.1.3 (22 Feb 2024)
- Added interp2d and interp3d interpolation functions
- Added option of nearest-neighbour interpolation
version 2.1.2 (3 Oct 2023)
- Further bug fixes to reduction of active arrays which did not
have addequate space allocated by check_space, including "product"
which requires an additional differential operation per element
- Fixed out-of-bounds access in test_thread_safe_arrays
- Slight change to reduce_dimension to avoid incorrect warning
about ExpressionSize array subscript of -1
- Fixed broken benchmark/autodiff_benchmark to work with ADOL-C
- Changed COMPILE_FLAGS argument order in test/Makefile in case
CPPFLAGS contains Timer.h or other conflicting header file
- Added benchmark/math_benchmark program
version 2.1.1 (10 April 2022)
- interp function can perform 1D interpolation of higher
dimensional Y arrays
- Bug fix in reduction of an "n" dimensional active array to an
"n-1" dimensional array: check_space had been forgotten
- Added Newton-Levenberg[-Marquardt] options to test_minimizer,
which use the exact Hessian of the Rosenbrock banana function
version 2.1 (5 February 2021)
- Removed README in favour of README.md
version 2.0.9 (28 January 2021)
- Fix bug in Array::alignment_offset causing occasional
crashes reduce and assign operations due to unaligned AVX access,
now tested in test_packet_operations
- Added Conjugate-Gradient and L-BFGS minimization methods, both
bounded and unbounded methods
- Disabled vectorization on 32-bit ARM NEON targets as there are
insufficient floating-point intrinsics
- Fixed interp(x,y,xi) function in case x and y have 0 or 1
elements
version 2.0.8 (22 August 2020)
- Added adept_optimize.h header file providing minimization
capability, initially with the constrained and unconstrained
Levenberg-Marquardt minimization algorithm
- Test program test_minimizer tests with the N-dimensional
Rosenbrock function
- The Stack member function "jacobian" can now operate on or
return Adept matrices, rather than solely on raw pointers which
had to point to data in column-major order
- Removed "using namespace internal" from several header files so
that adept namespace is clean
- Fixed C++98 compatibility
version 2.0.7 (23 June 2020)
- Added fast, vectorizable exponential function "fastexp", or can
use as adept::exp if the ADEPT_FAST_EXPONENTIAL preprocessor
variable is defined
- Moved all the vector intrinsic stuff to quick_e.h
- Added ARM-NEON support to quick_e.h
- Adept is now thread safe on Mac OS versions that support the
thread_local keyword
- Fixed bug that caused incorrect differentiation of
Active<double>/int
- Preprocessor option ADEPT_INIT_REAL_SNAN and
ADEPT_INIT_REAL_ZERO initialize real numbers (and complex numbers)
to signaling NaN or zero, useful for debugging
- Fixed bug that caused incorrect result of maxval and minval
applied to active arrays
- Fixed bug that caused incorrect differentiation of "product"
function
- Fixed bug that caused incorrect norm2 for passive vector large
enough to use vectorization
version 2.0.6 (20 February 2020)
- Fixed bug in hand-coded adjoint of Toon advection scheme
(benchmark/advection_schemes_AD.h), as well as other bugs that
would have prevented the Adjoint and hand-coded adjoints from
being correct compared to each other
- Fixed memory leak in Packet.h by ensuring memory is freed in the
case that neither _POSIX_VERSION nor _MSC_VER are defined
- Fixed bug in FixedArray.h that prevented active fixed arrays
from registering themselves with the stack when initialized using
an initializer list
- Fixed missing "template" directives in UnaryOperation.h that
prevented isfinite, isnan and isinf from working correctly on
arrays
- Added Array::resize_contigous functions
- minval and maxval now work correctly with negative and +/-Inf
arguments; previously minval gave incorrect results even for
negative arguments
- Added array_fortran.h to provide the ability to exchange arrays
between C++/Adept and Fortran, for those Fortran compilers that
support the 2018 standard
- Added support for AVX512 vectorization: operations on 16 floats
and 8 doubles at a time;
- Added test_packet_operations to check Intel vector intrinsics
correctly implemented
version 2.0.5 (6 February 2018)
- Use set_array_print_style(x) to set behaviour of <<Array;
available are x=PRINT_STYLE_[PLAIN|CSV|CURLY|MATLAB]
- Fix use of _mm_undefined_ps intrinsic: only use on GCC>=4.9.1
and Clang if appropriate built-in is present; can't guarantee its
presence with other compilers
- Fix writing of active scalar expressions to a stream
- Added missing fmin/fmax(Expr,Scalar)
version 2.0.4 (8 January 2018)
- Packet.h copes with undefined _mm_undefined_ps in GCC<4.9.1
- Fix Packet.h in case SSE2 not enabled
- ADEPT_FAST preprocessor variable enables
ADEPT_NO_DIMENSION_CHECKING, ADEPT_NO_ALIAS_CHECKING and
ADEPT_STACK_THREAD_UNSAFE
- Divide by scalar now only converts to multiply by (1.0/scalar)
if scalar is of floating-point type; this fixes indexing with
"end/2"
- Fix bug in Packet.h (found by valgrind) to ensure new[] followed
by delete[] and posix_memalign followed by free
- Increase initial stack size from 1000 to 1024^2
- Fixed two bugs in IndexedArray.h that broke indexing a matrix
with Matrix(int,intVector)
- Allocated memory in non-OpenMP jacobian_forward is now freed
version 2.0.3 (28 October 2017)
- Replaced template class "cast" with "expr_cast" to avoid clash
with Expression's non-template member function; this enables
compilation with Visual C++.
- Added adept::have_matrix_multiplication() and
adept::have_linear_algebra() to test for BLAS and LAPACK
(respectively) at run-time
version 2.0.2 (21 October 2017)
- Fixed standards-compliance problem with use of Expression in
Curiously Recurring Template Pattern, by removing any "static
const" members that referred to the derived class. This enabled
the same code to work with g++, clang++ and the Intel compiler icc.
version 2.0.1 (18 October 2017)
- Basic passive complex arrays work, tested with
test/test_complex_arrays
- Added ADEPT_NO_DIMENSION_CHECKING option
- Vectorized sqrt, unary-, unary+, max and min
- Removed the option to vectorize with Packet representing a
*pair* of SSE2/AVX packed vector; now a Packet can only represent
a single packed vector. This simplifies maintenance of Packet.h,
and the pair option offered no performance advantage anyway.
- Vectorized reduce operations sum, product etc.
- Many fixes to enable compilation with clang++
- Fixed FixedArray::operator[] for rank>1
version 2.0 (September 2017)
- Finalized version for release
- PDF documentation is no longer installed, so that Git users are
not obliged to have pdflatex
version 1.9.11 (30 September 2017)
- Fixed get_gradient member function of Array and FixedArray
- Added test_array_derivatives test program
- Fixed indexing of FixedArrays of rank>1
- Fixed IndexedArray applied to FixedArrays (before had reference
to temporary dimension object
- Test and benchmarking programs now work with single precision
- Stack functions accept Index passed by value rather than
reference, so that "static const int" passed from FixedArray does
not need to be explicitly instantiated
- Active::add_derivative_dependence and
append_derivative_dependence no longer only accept arguments of
type "Real"
- ADEPT_STORAGE_THREAD_SAFE option to protect Storage reference
counter in multi-threaded environment (C++11 only)
- Added Array::soft_link() as another means to get thread safety
- Added test program test_thread_safe_arrays
- Added adept_reference latex file to doc directory
- Added "dimensions" function for creating ExpressionSize objects
version 1.9.10 (25 September 2017)
- Added link syntax A >>= B
- Added assignment and initialization from initializer_lists for
Array and FixedArray classes
- Implemented Fortran-like "count" reduction function
- Bug fix sending active expression to a stream with "<<"
- Added "spread<dim>(array,n)" to match Fortran spread(array,dim,n)
- Added outer_product(x,y)
- Fixed adept_source.h for non-Unix systems
- Moved mathematical functions from global to adept namespace
- Fixed pausable recording and added test_adept_active_pausable
- Removed unsafe ADEPT_COPY_CONSTRUCTOR_ONLY_ON_RETURN_FROM_FUNCTION
- C++98 and C++11 correctly take cmath functions from :: and std::
respectively
- "make check" now runs test script test/run_tests.sh
- inv and solve now take general expression arguments
- Enabled indexed arrays to be assigned to an initializer list
- BLAS now optional (without it matrix multiplication causes
run-time exception)
- Added test_derivatives to test quality of derivatives for all
mathematical functions
- Enabled SpecialMatrix and IndexedArray to be assigned to an
active scalar expression
- Added fmax and fmin functions (even if C++11 not used)
- Added atan2 support
- C++11 on non-Mac platforms uses thread_local keyword instead of
C++98 compiler extensions
- Matrix multiplication on active special matrices implemented by
copying them to a dense Array<2,Real,true>. Very inefficient, but
it works.
- Matrix multiplication on inactive triangular and "square"
matrices now works by converting to them to a dense Array<2,Real,false>.
- Added alias detection in IndexedArray
- Alias detection in IndexedArray and SpecialMatrix can be
deactivated with ADEPT_NO_ALIAS_CHECKING
- Added "eval" function to evaluate an expression that might be
subject to aliasing
version 1.9.9 (August 2017)
- Put on GitHub as rjhogan/Adept-2
- Added Expression::next_value_contiguous for faster inner loops
in the case that all expressions have a contiguous and increasing
inner dimension
- Preliminary vectorization via Packet class and
Expression::next_packet
- Vectorized forward Jacobian calculation using packets
- Split Expression.h into also UnaryOperation.h and BinaryOperation.h
- Fixed bug in matmul.h that causes failure if matrix in
matrix-vector multiplication is strided in both dimensions
- Added move semantics if C++11 enabled
version 1.9.8 (April 2016):
- Completed FixedArray.h and tested for active arguments
- Added array_shortcuts for FixedArrays: (a)VectorX, (a)MatrixXX
- Added array_shortcuts for Arrays: (a)ArrayXD (for X = 3 to 7)
- interp permits general Expression arguments
version 1.9.7 (April 2016):
- Nearly completed FixedArray.h
version 1.9.6 (March 2016):
- Started FixedArray.h
version 1.9.5 (March 2016):
- Fixed add_derivative_dependence and append_derivative_dependence
when applied to elements of arrays
- Added ADEPT_BOUNDS_CHECKING capability, and fixed IndexedArray
to work with this
- Now call BLAS and LAPACK (Fortran) routines, rather than C-BLAS
and LAPACKE functions
- Added matrix multiplication benchmark program
- Added IndexedArray for dimensions up to 7
- Added Array::data() and Array::const_data() for direct access
- Added Array::subset(); slightly more concise than using "range"
version 1.9.4 (January 2016):
- Completed changes to documentation in doc directory
- Added control/inquiry of settings, e.g. set_max_blas_threads()
and configuration()
version 1.9.3 (December 2015):
- Added "max" and "min" as binary operators (note that "maxval"
and "minval" are reduction operators as in Fortran)
version 1.9.2 (December 2015):
- Added ActiveConstReference type for active constant references
version 1.9.1 (November 2015):
- New matmul.h/matmul.cpp - not yet complete
version 1.9.0 (November 2015):
- SUBSTANTIAL REWRITE TO INCORPORATE ARRAY FUNCTIONALITY
version 1.1 (June 2015):
- Added ./configure script using autotools
- Added support for additional mathematical functions: asinh,
acosh, atanh, expm1, log1p, cbrt, erf, erfc, exp2, log2
- Changed license from GNU General Public License to Apache
License, Version 2.0
- Jacobian calculation uses OpenMP parallelization
- Removed multiscatter example code
- New benchmarking program in benchmark/ that compares to other
automatic differentiation tools if available
- Fixed bug so that gaps in the gradient list now merge properly
- Provided capability to compile code without an external library,
to facilitate porting to Windows
- Added programs in test/ demonstrating checkpointing,
thread-safety and compiling without an external library
version 1.0 (September 2013):
- Very many internal changes and added features
- Detailed documentation in the doc/ directory
- Removed the LIFO requirement on the order with which aReal
objects ought to be created and destroyed
- For users of version 0.9, the main change to the interface is
that the Stack::start() member function is no longer supported;
rather you should call the Stack::new_recording() member function
*after* the independent variables have been initialized but
*before* any mathematical operations are performed using them
version 0.9:
- First public release
================================================
FILE: INSTALL
================================================
Installation Instructions
*************************
Copyright (C) 1994, 1995, 1996, 1999, 2000, 2001, 2002, 2004, 2005,
2006, 2007 Free Software Foundation, Inc.
This file is free documentation; the Free Software Foundation gives
unlimited permission to copy, distribute and modify it.
Basic Installation
==================
Briefly, the shell commands `./configure; make; make install' should
configure, build, and install this package. The following
more-detailed instructions are generic; see the `README' file for
instructions specific to this package.
The `configure' shell script attempts to guess correct values for
various system-dependent variables used during compilation. It uses
those values to create a `Makefile' in each directory of the package.
It may also create one or more `.h' files containing system-dependent
definitions. Finally, it creates a shell script `config.status' that
you can run in the future to recreate the current configuration, and a
file `config.log' containing compiler output (useful mainly for
debugging `configure').
It can also use an optional file (typically called `config.cache'
and enabled with `--cache-file=config.cache' or simply `-C') that saves
the results of its tests to speed up reconfiguring. Caching is
disabled by default to prevent problems with accidental use of stale
cache files.
If you need to do unusual things to compile the package, please try
to figure out how `configure' could check whether to do them, and mail
diffs or instructions to the address given in the `README' so they can
be considered for the next release. If you are using the cache, and at
some point `config.cache' contains results you don't want to keep, you
may remove or edit it.
The file `configure.ac' (or `configure.in') is used to create
`configure' by a program called `autoconf'. You need `configure.ac' if
you want to change it or regenerate `configure' using a newer version
of `autoconf'.
The simplest way to compile this package is:
1. `cd' to the directory containing the package's source code and type
`./configure' to configure the package for your system.
Running `configure' might take a while. While running, it prints
some messages telling which features it is checking for.
2. Type `make' to compile the package.
3. Optionally, type `make check' to run any self-tests that come with
the package.
4. Type `make install' to install the programs and any data files and
documentation.
5. You can remove the program binaries and object files from the
source code directory by typing `make clean'. To also remove the
files that `configure' created (so you can compile the package for
a different kind of computer), type `make distclean'. There is
also a `make maintainer-clean' target, but that is intended mainly
for the package's developers. If you use it, you may have to get
all sorts of other programs in order to regenerate files that came
with the distribution.
6. Often, you can also type `make uninstall' to remove the installed
files again.
Compilers and Options
=====================
Some systems require unusual options for compilation or linking that the
`configure' script does not know about. Run `./configure --help' for
details on some of the pertinent environment variables.
You can give `configure' initial values for configuration parameters
by setting variables in the command line or in the environment. Here
is an example:
./configure CC=c99 CFLAGS=-g LIBS=-lposix
*Note Defining Variables::, for more details.
Compiling For Multiple Architectures
====================================
You can compile the package for more than one kind of computer at the
same time, by placing the object files for each architecture in their
own directory. To do this, you can use GNU `make'. `cd' to the
directory where you want the object files and executables to go and run
the `configure' script. `configure' automatically checks for the
source code in the directory that `configure' is in and in `..'.
With a non-GNU `make', it is safer to compile the package for one
architecture at a time in the source code directory. After you have
installed the package for one architecture, use `make distclean' before
reconfiguring for another architecture.
Installation Names
==================
By default, `make install' installs the package's commands under
`/usr/local/bin', include files under `/usr/local/include', etc. You
can specify an installation prefix other than `/usr/local' by giving
`configure' the option `--prefix=PREFIX'.
You can specify separate installation prefixes for
architecture-specific files and architecture-independent files. If you
pass the option `--exec-prefix=PREFIX' to `configure', the package uses
PREFIX as the prefix for installing programs and libraries.
Documentation and other data files still use the regular prefix.
In addition, if you use an unusual directory layout you can give
options like `--bindir=DIR' to specify different values for particular
kinds of files. Run `configure --help' for a list of the directories
you can set and what kinds of files go in them.
If the package supports it, you can cause programs to be installed
with an extra prefix or suffix on their names by giving `configure' the
option `--program-prefix=PREFIX' or `--program-suffix=SUFFIX'.
Optional Features
=================
Some packages pay attention to `--enable-FEATURE' options to
`configure', where FEATURE indicates an optional part of the package.
They may also pay attention to `--with-PACKAGE' options, where PACKAGE
is something like `gnu-as' or `x' (for the X Window System). The
`README' should mention any `--enable-' and `--with-' options that the
package recognizes.
For packages that use the X Window System, `configure' can usually
find the X include and library files automatically, but if it doesn't,
you can use the `configure' options `--x-includes=DIR' and
`--x-libraries=DIR' to specify their locations.
Specifying the System Type
==========================
There may be some features `configure' cannot figure out automatically,
but needs to determine by the type of machine the package will run on.
Usually, assuming the package is built to be run on the _same_
architectures, `configure' can figure that out, but if it prints a
message saying it cannot guess the machine type, give it the
`--build=TYPE' option. TYPE can either be a short name for the system
type, such as `sun4', or a canonical name which has the form:
CPU-COMPANY-SYSTEM
where SYSTEM can have one of these forms:
OS KERNEL-OS
See the file `config.sub' for the possible values of each field. If
`config.sub' isn't included in this package, then this package doesn't
need to know the machine type.
If you are _building_ compiler tools for cross-compiling, you should
use the option `--target=TYPE' to select the type of system they will
produce code for.
If you want to _use_ a cross compiler, that generates code for a
platform different from the build platform, you should specify the
"host" platform (i.e., that on which the generated programs will
eventually be run) with `--host=TYPE'.
Sharing Defaults
================
If you want to set default values for `configure' scripts to share, you
can create a site shell script called `config.site' that gives default
values for variables like `CC', `cache_file', and `prefix'.
`configure' looks for `PREFIX/share/config.site' if it exists, then
`PREFIX/etc/config.site' if it exists. Or, you can set the
`CONFIG_SITE' environment variable to the location of the site script.
A warning: not all `configure' scripts look for a site script.
Defining Variables
==================
Variables not defined in a site shell script can be set in the
environment passed to `configure'. However, some packages may run
configure again during the build, and the customized values of these
variables may be lost. In order to avoid this problem, you should set
them in the `configure' command line, using `VAR=value'. For example:
./configure CC=/usr/local2/bin/gcc
causes the specified `gcc' to be used as the C compiler (unless it is
overridden in the site shell script).
Unfortunately, this technique does not work for `CONFIG_SHELL' due to
an Autoconf bug. Until the bug is fixed you can use this workaround:
CONFIG_SHELL=/bin/bash /bin/bash ./configure CONFIG_SHELL=/bin/bash
`configure' Invocation
======================
`configure' recognizes the following options to control how it operates.
`--help'
`-h'
Print a summary of the options to `configure', and exit.
`--version'
`-V'
Print the version of Autoconf used to generate the `configure'
script, and exit.
`--cache-file=FILE'
Enable the cache: use and save the results of the tests in FILE,
traditionally `config.cache'. FILE defaults to `/dev/null' to
disable caching.
`--config-cache'
`-C'
Alias for `--cache-file=config.cache'.
`--quiet'
`--silent'
`-q'
Do not print messages saying which checks are being made. To
suppress all normal output, redirect it to `/dev/null' (any error
messages will still be shown).
`--srcdir=DIR'
Look for the package's source code in directory DIR. Usually
`configure' can determine that directory automatically.
`configure' also accepts some other, not widely useful, options. Run
`configure --help' for more details.
================================================
FILE: Makefile.am
================================================
dist_pkgdata_DATA = README.md
pkgdata_DATA = COPYING ChangeLog NEWS AUTHORS
SUBDIRS = adept include benchmark test
# The test/ directory does not use automake so we need to specify the
# files that will be included in the distribution
EXTRA_DIST = test/Makefile test/README test/*.cpp test/*.h test/run_tests.sh \
doc/Makefile doc/README doc/COPYING doc/*.tex
ACLOCAL_AMFLAGS = -I m4
================================================
FILE: NEWS
================================================
version 2.0
- Fixed pausable recording and library-free compilation to provide full backwards compatibility with version 1.1
- C++11 features such as initializer lists
- Automatic vectorization of passive array statements if possible
- Additional mathematical functions: round, trunc, rint, nearbyint, atan2, fmin, fmax
- Additional array operations: spread, outer_product, count, maxval, minval, reshape
- Many more test programs
version 1.9.8 (April 2016)
- First beta release of version 2.0 incorporating array capability up to 7 dimensions
- Matrix multiplication and basic linear from BLAS and LAPACK
- Options for thread-safe accessing of arrays
version 1.1 (June 2015)
- Added ./configure script
- Added support for additional mathematical functions: asinh, acosh, atanh, expm1, log1p, cbrt, erf, erfc, exp2, log2
- License changed to Apache License, Version 2.0
================================================
FILE: README.md
================================================
# Adept 2: Combined array and automatic differentiation library in C++
## Introduction
The Adept version 2.1 software library provides three different
functionalities:
* Its automatic differentiation capability enables algorithms written
in C++ to be differentiated with little code modification, very
useful for a wide range of applications that involve mathematical
optimization. It is backwards compatible with and as fast as Adept
1.1. The name "Adept" refers to "Automatic Differentiation using
Expression Templates".
* Its array capability provides support for vectors, matrices, arrays
of up to 7 dimensions and linear algebra. Adept 2 uses a single
expression-template framework under the hood to enable array
operations to be differentiated with very good computational
performance.
* Its optimization capability provides the various minimization
algorithms (Levenberg, Levenberg-Marquardt, Conjugate Gradient and
Limited Memory BFGS) each of which can be used with or without box
constraints on the state variables. The interface to the
optimization functionality is in terms of Adept vectors and matrices.
If you are not interested in the array or optimization capabilities of
Adept 2 then Adept 1.1 may be more to your liking as a very
lightweight library that has virtually all the
automatic-differentiation capabilities of version 2.
## Documentation and links
* The [Adept web site](http://www.met.reading.ac.uk/clouds/adept/) for formal Adept releases
* The [Adept-2 GitHub page](https://github.com/rjhogan/Adept-2) for the latest snapshot
* The [Adept-1.1 GitHub page](https://github.com/rjhogan/Adept) for the older (scalar) library
* A detailed [User Guide](http://www.met.reading.ac.uk/clouds/adept/adept_documentation.pdf)
* A paper describing the automatic differentiation capability: [Hogan, R. J., 2014: Fast reverse-mode automatic differentiation using expression templates in C++. *ACM Trans. Math. Softw.* **40,** 26:1-26:16](http://www.met.reading.ac.uk/~swrhgnrj/publications/adept.pdf)
* The [Adept Wikipedia page](https://en.wikipedia.org/wiki/Adept_(C++_library))
* Bug fixes, and queries not answered by the documentation, should be addressed to Robin Hogan (r.j.hogan at ecmwf.int)
## Installation
To build Adept from a GitHub snapshot, first do the following to
recreate the configure script (requiring the autotools package):
autoreconf -i
Formal release packages already contain a configure script. The normal
build sequence is then:
./configure
make
make check
make install
Please consult the User Guide for further installation options; in
particular, if you plan to make serious us of matrix multiplication
and linear algebra then you should compile Adept to use an optimized
BLAS library such as OpenBLAS.
## License and copyright
The code in this package has a mix of copyright owners:
Copyright (C) 2012-2015 University of Reading
Copyright (C) 2015- European Centre for Medium-Range Weather Forecasts
Two licenses are used for the code in this package:
* The files that form the Adept library are distributed under the
conditions of the Apache License, Version 2 - see the COPYING file
for details. This is a permissive free-software license but one
that does impose a few conditions if you intend to distribute
derivative works. The files this license applies to are those in
the include/ and adept/ directories, and the subdirectories below
them.
* All code in the test/ and benchmark/ directories is subject to the
terms of the GNU all-permissive license, given at the top of those
files - basically you can do what you like with the code from these
files.
If you use Adept in published scientific work then it is requested
that you cite the Hogan (2014) paper above, but this is not a
condition of the license.
================================================
FILE: TODO
================================================
BUGS
spread<DIM> function does not use the right DIM
DESIRABLE BUT NEEDS NEW STACK
Differentiated BLAS operations on symmetric matrices etc
Implement general OpenMP for forward pass
OPTIMIZATION
Vectorize active expressions
Fix vectorization of spread and outer_product by storing pointer to start of row and not using index
Communicate band diagonals statically to optimize Array = band expression (e.g. 2*TridiagMatrix)
Implement active scalar precomputation
Optimize reciprocal to use 1.0 or 1.0f; vectorize
Optimize storage of data range
SquareMatrix::is_vectorizable = true
FEATURES
long double calls double matmul functions?
std::string configuration function returning options for this compilation unit
Mathematical functions copysign, fdim, hypot, remainder?
Implement user elemental function
Implement user choice of Jacobian array ordering
Clean-up benchmark and test_arrays/test_array_speed code
Check can do Array<*,Active<Real>,false>
Rename ExpressionSize
Enable functions taking ExpressionSize arguments (e.g. resize and array constructor) to take equivalent arguments, e.g. std::vector, initializer lists etc
Fall-back if BLAS not available
Implement pow<int> and sqr
Implement non-member functions merge?, reshape, shape?, size, [un]pack(?), minloc, maxloc
Implement matlab-like tile (generic repmat) plus zeros and ones
Implement iterators
Triangular/symmetric views
Const link does not increment reference counter
Cannot link non-const to const either by construction or explicit link
Should reduce functions take dimensions as template arguments?
reduce operations have a template version with the reduce dimension provided statically
differentiate complex number operations
matmul and solve on complex numbers
complex functions arg, abs, real, imag etc
CHECK
Check Square matmul
All vectorization combinations work, e.g. double/int, aligned/unaligned LHS
Set whole arrays as independent/dependent
Reduce RMS difference in Toon case
CLEAN
References to OpenMP for array operations - remove?
DOCUMENTATION
Document diag_vector non-member function (in reduce.h) and test in test_arrays
OLDER IDEAS
Clarify vector orientation when in matrix multiplication
Vector orientation changed with row(), col()?
Implement move semantics and make copy constructors do deep copy ADEPT_***
Implement OpenMP passive array operations
Implement OpenMP active array operations
Link can only be performed on empty object
If new Expression types are to be added, they should provide the
following interface:
static const int rank_ = 0;
static const int n_scratch_ = 0;
static const int n_active_ = 0;
static const int n_arrays_ = 0;
static const bool is_active_ = false;
static const bool is_vectorizable_ = true;
bool get_dimensions_(ExpressionSize<0>& dim) const;
std::string expression_string_() const;
bool is_aliased_(const Type* mem1, const Type* mem2) const;
Type value_with_len_(const Index& j, const Index& len) const;
template <int MyArrayNum, int NArrays>
void advance_location_(ExpressionSize<NArrays>& loc) const;
template <int MyArrayNum, int NArrays>
Type value_at_location_(const ExpressionSize<NArrays>& loc) const;
template <int MyArrayNum, int NArrays>
Packet<Type>
packet_at_location_(const ExpressionSize<NArrays>& loc) const;
template <int MyArrayNum, int MyScratchNum, int NArrays, int NScratch>
Type value_at_location_store_(const ExpressionSize<NArrays>& loc,
ScratchVector<NScratch>& scratch) const;
template <int MyArrayNum, int MyScratchNum, int NArrays, int NScratch>
Type value_stored_(const ExpressionSize<NArrays>& loc,
const ScratchVector<NScratch>& scratch) const;
template <int MyArrayNum, int MyScratchNum, int NArrays, int NScratch>
void calc_gradient_(Stack& stack,
const ExpressionSize<NArrays>& loc,
const ScratchVector<NScratch>& scratch) const;
template <int MyArrayNum, int MyScratchNum,
int NArrays, int NScratch, typename MyType>
void calc_gradient_(Stack& stack,
const ExpressionSize<NArrays>& loc,
const ScratchVector<NScratch>& scratch,
const MyType& multiplier) const;
template <int MyArrayNum, int Rank, int NArrays>
void set_location_(const ExpressionSize<Rank>& i,
ExpressionSize<NArrays>& index) const;
================================================
FILE: adept/Array.cpp
================================================
/* Array.cpp -- Functions and global variables controlling array behaviour
Copyright (C) 2015-2016 European Centre for Medium-Range Weather Forecasts
Robin Hogan <r.j.hogan@ecmwf.int>
This file is part of the Adept library.
*/
#include <adept/Array.h>
namespace adept {
namespace internal {
bool array_row_major_order = true;
// bool array_print_curly_brackets = true;
// Variables describing how arrays are written to a stream
ArrayPrintStyle array_print_style = PRINT_STYLE_CURLY;
std::string vector_separator = ", ";
std::string vector_print_before = "{";
std::string vector_print_after = "}";
std::string array_opening_bracket = "{";
std::string array_closing_bracket = "}";
std::string array_contiguous_separator = ", ";
std::string array_non_contiguous_separator = ",\n";
std::string array_print_before = "\n{";
std::string array_print_after = "}";
std::string array_print_empty_before = "(empty rank-";
std::string array_print_empty_after = " array)";
bool array_print_indent = true;
bool array_print_empty_rank = true;
}
void set_array_print_style(ArrayPrintStyle ps) {
using namespace internal;
switch (ps) {
case PRINT_STYLE_PLAIN:
vector_separator = " ";
vector_print_before = "";
vector_print_after = "";
array_opening_bracket = "";
array_closing_bracket = "";
array_contiguous_separator = " ";
array_non_contiguous_separator = "\n";
array_print_before = "";
array_print_after = "";
array_print_empty_before = "(empty rank-";
array_print_empty_after = " array)";
array_print_indent = false;
array_print_empty_rank = true;
break;
case PRINT_STYLE_CSV:
vector_separator = ", ";
vector_print_before = "";
vector_print_after = "";
array_opening_bracket = "";
array_closing_bracket = "";
array_contiguous_separator = ", ";
array_non_contiguous_separator = "\n";
array_print_before = "";
array_print_after = "";
array_print_empty_before = "empty";
array_print_empty_after = "";
array_print_indent = false;
array_print_empty_rank = false;
break;
case PRINT_STYLE_MATLAB:
vector_separator = " ";
vector_print_before = "[";
vector_print_after = "]";
array_opening_bracket = "[";
array_closing_bracket = "]";
array_contiguous_separator = " ";
array_non_contiguous_separator = ";\n";
array_print_before = "[";
array_print_after = "]";
array_print_empty_before = "[";
array_print_empty_after = "]";
array_print_indent = true;
array_print_empty_rank = false;
break;
case PRINT_STYLE_CURLY:
vector_separator = ", ";
vector_print_before = "{";
vector_print_after = "}";
array_opening_bracket = "{";
array_closing_bracket = "}";
array_contiguous_separator = ", ";
array_non_contiguous_separator = ",\n";
array_print_before = "\n{";
array_print_after = "}";
array_print_empty_before = "(empty rank-";
array_print_empty_after = " array)";
array_print_indent = true;
array_print_empty_rank = true;
break;
default:
throw invalid_operation("Array print style not understood");
}
array_print_style = ps;
}
}
================================================
FILE: adept/Makefile.am
================================================
lib_LTLIBRARIES = libadept.la
libadept_la_SOURCES = Array.cpp Stack.cpp StackStorageOrig.cpp \
jacobian.cpp Storage.cpp index.cpp settings.cpp \
cppblas.cpp cpplapack.h solve.cpp inv.cpp \
vector_utilities.cpp Minimizer.cpp \
minimize_limited_memory_bfgs.cpp minimize_levenberg_marquardt.cpp \
minimize_conjugate_gradient.cpp line_search.cpp
libadept_la_CPPFLAGS = -I@top_srcdir@/include
================================================
FILE: adept/Minimizer.cpp
================================================
/* Minimizer.h -- class for minimizing the cost function of an optimizable object
Copyright (C) 2020 European Centre for Medium-Range Weather Forecasts
Author: Robin Hogan <r.j.hogan@ecmwf.int>
This file is part of the Adept library.
*/
#include <cctype>
#include <adept/Minimizer.h>
#include <adept/exception.h>
namespace adept {
// List of the names of available minimizer algorithms
static const char* minimizer_algorithm_names_[]
= {"L-BFGS",
"Conjugate-Gradient",
"Conjugate-Gradient-FR",
"Levenberg",
"Levenberg-Marquardt"};
// Lower-case versions of the list above
static const char* minimizer_algorithm_lower_names_[]
= {"l-bfgs",
"conjugate-gradient",
"conjugate-gradient-fr",
"levenberg",
"levenberg-marquardt"};
// Convert to lower case, and convert spaces and underscores to
// hyphens. This function is used to do a case-insensitive
// string-based selection of the minimizer algorithm to use.
static void to_lower_in_place(std::string& str) {
for (std::string::size_type istr = 0; istr < str.size(); ++istr) {
str[istr] = std::tolower(str[istr]);
if (str[istr] == ' ' || str[istr] == '_') {
str[istr] = '-';
}
}
}
// Return a C string describing the minimizer status
const char*
minimizer_status_string(MinimizerStatus status)
{
switch (status) {
case MINIMIZER_STATUS_SUCCESS:
return "Converged";
break;
case MINIMIZER_STATUS_EMPTY_STATE:
return "Empty state vector, no minimization performed";
break;
case MINIMIZER_STATUS_MAX_ITERATIONS_REACHED:
return "Maximum iterations reached";
break;
case MINIMIZER_STATUS_FAILED_TO_CONVERGE:
return "Failed to converge";
break;
case MINIMIZER_STATUS_DIRECTION_UPHILL:
return "Search direction points uphill";
break;
case MINIMIZER_STATUS_BOUND_REACHED:
return "Bound reached"; // Should not be returned from a minimize function
break;
case MINIMIZER_STATUS_INVALID_COST_FUNCTION:
return "Non-finite cost function";
break;
case MINIMIZER_STATUS_INVALID_GRADIENT:
return "Non-finite gradient";
break;
case MINIMIZER_STATUS_INVALID_BOUNDS:
return "Invalid bounds for bounded minimization";
break;
case MINIMIZER_STATUS_NOT_YET_CONVERGED:
return "Minimization still in progress";
break;
default:
return "Status unrecognized";
}
}
// Case-insensitive setting of the miminization algorithm given its
// name
void
Minimizer::set_algorithm(const std::string& algo) {
std::string algo_lower = algo;
to_lower_in_place(algo_lower);
std::cout << "Checking \"" << algo_lower << "\"\n";
for (int ialgo = 0;
ialgo < static_cast<int>(MINIMIZER_ALGORITHM_NUMBER_AVAILABLE);
++ialgo) {
if (algo_lower == minimizer_algorithm_lower_names_[ialgo]) {
set_algorithm(static_cast<MinimizerAlgorithm>(ialgo));
return;
}
}
throw optimization_exception("Algorithm name not understood");
}
std::string
Minimizer::algorithm_name() {
int ialgo = static_cast<MinimizerAlgorithm>(algorithm_);
if (ialgo >= 0 && ialgo < MINIMIZER_ALGORITHM_NUMBER_AVAILABLE) {
return minimizer_algorithm_names_[ialgo];
}
else {
return "Unknown";
}
}
// Unconstrained minimization
MinimizerStatus
Minimizer::minimize(Optimizable& optimizable, Vector x)
{
if (minimizer_algorithm_order(algorithm_) > 1
&& !optimizable.provides_derivative(2)) {
throw optimization_exception("2nd-order minimization algorithm requires optimizable that can provide 2nd derivatives");
}
else if (algorithm_ == MINIMIZER_ALGORITHM_LIMITED_MEMORY_BFGS) {
return minimize_limited_memory_bfgs(optimizable, x);
}
else if (algorithm_ == MINIMIZER_ALGORITHM_CONJUGATE_GRADIENT) {
return minimize_conjugate_gradient(optimizable, x);
}
else if (algorithm_ == MINIMIZER_ALGORITHM_CONJUGATE_GRADIENT_FR) {
return minimize_conjugate_gradient(optimizable, x, true);
}
else if (algorithm_ == MINIMIZER_ALGORITHM_LEVENBERG) {
return minimize_levenberg_marquardt(optimizable, x, true);
}
else if (algorithm_ == MINIMIZER_ALGORITHM_LEVENBERG_MARQUARDT) {
return minimize_levenberg_marquardt(optimizable, x, false);
}
else {
throw optimization_exception("Minimization algorithm not recognized");
}
}
// Constrained minimization
MinimizerStatus
Minimizer::minimize(Optimizable& optimizable, Vector x,
const Vector& x_lower, const Vector& x_upper)
{
if (minimizer_algorithm_order(algorithm_) > 1
&& !optimizable.provides_derivative(2)) {
throw optimization_exception("2nd-order minimization algorithm requires optimizable that can provide 2nd derivatives");
}
if (algorithm_ == MINIMIZER_ALGORITHM_LIMITED_MEMORY_BFGS) {
return minimize_limited_memory_bfgs_bounded(optimizable, x,
x_lower, x_upper);
}
else if (algorithm_ == MINIMIZER_ALGORITHM_CONJUGATE_GRADIENT) {
return minimize_conjugate_gradient_bounded(optimizable, x,
x_lower, x_upper);
}
else if (algorithm_ == MINIMIZER_ALGORITHM_CONJUGATE_GRADIENT_FR) {
return minimize_conjugate_gradient_bounded(optimizable, x,
x_lower, x_upper, true);
}
if (algorithm_ == MINIMIZER_ALGORITHM_LEVENBERG) {
return minimize_levenberg_marquardt_bounded(optimizable, x,
x_lower, x_upper, true);
}
if (algorithm_ == MINIMIZER_ALGORITHM_LEVENBERG_MARQUARDT) {
return minimize_levenberg_marquardt_bounded(optimizable, x,
x_lower, x_upper, false);
}
else {
throw optimization_exception("Constrained minimization algorithm not recognized");
}
}
};
================================================
FILE: adept/Stack.cpp
================================================
/* Stack.cpp -- Stack for storing automatic differentiation information
Copyright (C) 2012-2014 University of Reading
Copyright (C) 2015 European Centre for Medium-Range Weather Forecasts
Author: Robin Hogan <r.j.hogan@ecmwf.int>
This file is part of the Adept library.
*/
#include <iostream>
#include <cstring> // For memcpy
#ifdef _OPENMP
#include <omp.h>
#endif
#include <adept/Stack.h>
namespace adept {
using namespace internal;
// Global pointers to the current thread, the second of which is
// thread safe. The first is only used if ADEPT_STACK_THREAD_UNSAFE
// is defined.
ADEPT_THREAD_LOCAL Stack* _stack_current_thread = 0;
Stack* _stack_current_thread_unsafe = 0;
// MEMBER FUNCTIONS OF THE STACK CLASS
// Destructor: frees dynamically allocated memory (if any)
Stack::~Stack() {
// If this is the currently active stack then set to NULL as
// "this" is shortly to become invalid
if (is_thread_unsafe_) {
if (_stack_current_thread_unsafe == this) {
_stack_current_thread_unsafe = 0;
}
}
else if (_stack_current_thread == this) {
_stack_current_thread = 0;
}
#ifndef ADEPT_STACK_STORAGE_STL
if (gradient_) {
delete[] gradient_;
}
#endif
}
// Make this stack "active" by copying its "this" pointer to a
// global variable; this makes it the stack that aReal objects
// subsequently interact with when being created and participating
// in mathematical expressions
void
Stack::activate()
{
// Check that we don't already have an active stack in this thread
if ((is_thread_unsafe_ && _stack_current_thread_unsafe
&& _stack_current_thread_unsafe != this)
|| ((!is_thread_unsafe_) && _stack_current_thread
&& _stack_current_thread != this)) {
throw(stack_already_active());
}
else {
if (!is_thread_unsafe_) {
_stack_current_thread = this;
}
else {
_stack_current_thread_unsafe = this;
}
}
}
// Set the maximum number of threads to be used in Jacobian
// calculations, if possible. A value of 1 indicates that OpenMP
// will not be used, while a value of 0 indicates that the number
// will match the number of available processors. Returns the
// maximum that will be used, which will be 1 if the Adept library
// was compiled without OpenMP support. Note that a value of 1 will
// disable the use of OpenMP with Adept, so Adept will then use no
// OpenMP directives or function calls. Note that if in your program
// you use OpenMP with each thread performing automatic
// differentiaion with its own independent Adept stack, then
// typically only one OpenMP thread is available for each Jacobian
// calculation, regardless of whether you call this function.
int
Stack::set_max_jacobian_threads(int n)
{
#ifdef _OPENMP
if (have_openmp_) {
if (n == 1) {
openmp_manually_disabled_ = true;
return 1;
}
else if (n < 1) {
openmp_manually_disabled_ = false;
omp_set_num_threads(omp_get_num_procs());
return omp_get_max_threads();
}
else {
openmp_manually_disabled_ = false;
omp_set_num_threads(n);
return omp_get_max_threads();
}
}
#endif
return 1;
}
// Return maximum number of OpenMP threads to be used in Jacobian
// calculation
int
Stack::max_jacobian_threads() const
{
#ifdef _OPENMP
if (have_openmp_) {
if (openmp_manually_disabled_) {
return 1;
}
else {
return omp_get_max_threads();
}
}
#endif
return 1;
}
// Perform to adjoint computation (reverse mode). It is assumed that
// some gradients have been assigned already, otherwise the function
// returns with an error.
void
Stack::compute_adjoint()
{
if (gradients_are_initialized()) {
// Loop backwards through the derivative statements
for (uIndex ist = n_statements_-1; ist > 0; ist--) {
const Statement& statement = statement_[ist];
// We copy the RHS gradient (LHS in the original derivative
// statement but swapped in the adjoint equivalent) to "a" in
// case it appears on the LHS in any of the following statements
Real a = gradient_[statement.index];
gradient_[statement.index] = 0.0;
// By only looping if a is non-zero we gain a significant speed-up
if (a != 0.0) {
// Loop over operations
for (uIndex i = statement_[ist-1].end_plus_one;
i < statement.end_plus_one; i++) {
gradient_[index_[i]] += multiplier_[i]*a;
}
}
}
}
else {
throw(gradients_not_initialized());
}
}
// Perform tangent linear computation (forward mode). It is assumed
// that some gradients have been assigned already, otherwise the
// function returns with an error.
void
Stack::compute_tangent_linear()
{
if (gradients_are_initialized()) {
// Loop forward through the statements
for (uIndex ist = 1; ist < n_statements_; ist++) {
const Statement& statement = statement_[ist];
// We copy the LHS to "a" in case it appears on the RHS in any
// of the following statements
Real a = 0.0;
for (uIndex i = statement_[ist-1].end_plus_one;
i < statement.end_plus_one; i++) {
a += multiplier_[i]*gradient_[index_[i]];
}
gradient_[statement.index] = a;
}
}
else {
throw(gradients_not_initialized());
}
}
// Register n gradients
uIndex
Stack::do_register_gradients(const uIndex& n) {
n_gradients_registered_ += n;
if (!gap_list_.empty()) {
uIndex return_val;
// Insert in a gap, if there is one big enough
for (GapListIterator it = gap_list_.begin();
it != gap_list_.end(); it++) {
uIndex len = it->end + 1 - it->start;
if (len > n) {
// Gap a bit larger than needed: reduce its size
return_val = it->start;
it->start += n;
return return_val;
}
else if (len == n) {
// Gap exactly the size needed: fill it and remove from list
return_val = it->start;
if (most_recent_gap_ == it) {
gap_list_.erase(it);
most_recent_gap_ = gap_list_.end();
}
else {
gap_list_.erase(it);
}
return return_val;
}
}
}
// No suitable gap found; instead add to end of gradient vector
i_gradient_ += n;
if (i_gradient_ > max_gradient_) {
max_gradient_ = i_gradient_;
}
return i_gradient_ - n;
}
// If an aReal object is deleted, its gradient_index is
// unregistered from the stack. If this is at the top of the stack
// then this is easy and is done inline; this is the usual case
// since C++ trys to deallocate automatic objects in the reverse
// order to that in which they were allocated. If it is not at the
// top of the stack then a non-inline function is called to ensure
// that the gap list is adjusted correctly.
void
Stack::unregister_gradient_not_top(const uIndex& gradient_index)
{
enum {
ADDED_AT_BASE,
ADDED_AT_TOP,
NEW_GAP,
NOT_FOUND
} status = NOT_FOUND;
// First try to find if the unregistered element is at the
// start or end of an existing gap
if (!gap_list_.empty() && most_recent_gap_ != gap_list_.end()) {
// We have a "most recent" gap - check whether the gradient
// to be unregistered is here
Gap& current_gap = *most_recent_gap_;
if (gradient_index == current_gap.start - 1) {
current_gap.start--;
status = ADDED_AT_BASE;
}
else if (gradient_index == current_gap.end + 1) {
current_gap.end++;
status = ADDED_AT_TOP;
}
// Should we check for erroneous removal from middle of gap?
}
if (status == NOT_FOUND) {
// Search other gaps
for (GapListIterator it = gap_list_.begin();
it != gap_list_.end(); it++) {
if (gradient_index <= it->end + 1) {
// Gradient to unregister is either within the gap
// referenced by iterator "it", or it is between "it"
// and the previous gap in the list
if (gradient_index == it->start - 1) {
status = ADDED_AT_BASE;
it->start--;
most_recent_gap_ = it;
}
else if (gradient_index == it->end + 1) {
status = ADDED_AT_TOP;
it->end++;
most_recent_gap_ = it;
}
else {
// Insert a new gap of width 1; note that list::insert
// inserts *before* the specified location
most_recent_gap_
= gap_list_.insert(it, Gap(gradient_index));
status = NEW_GAP;
}
break;
}
}
if (status == NOT_FOUND) {
gap_list_.push_back(Gap(gradient_index));
most_recent_gap_ = gap_list_.end();
most_recent_gap_--;
}
}
// Finally check if gaps have merged
if (status == ADDED_AT_BASE
&& most_recent_gap_ != gap_list_.begin()) {
// Check whether the gap has merged with the next one
GapListIterator it = most_recent_gap_;
it--;
if (it->end == most_recent_gap_->start - 1) {
// Merge two gaps
most_recent_gap_->start = it->start;
gap_list_.erase(it);
}
}
else if (status == ADDED_AT_TOP) {
GapListIterator it = most_recent_gap_;
it++;
if (it != gap_list_.end()
&& it->start == most_recent_gap_->end + 1) {
// Merge two gaps
most_recent_gap_->end = it->end;
gap_list_.erase(it);
}
}
}
// Unregister n gradients starting at gradient_index
void
Stack::unregister_gradients(const uIndex& gradient_index,
const uIndex& n)
{
n_gradients_registered_ -= n;
if (gradient_index+n == i_gradient_) {
// Gradient to be unregistered is at the top of the stack
i_gradient_ -= n;
if (!gap_list_.empty()) {
Gap& last_gap = gap_list_.back();
if (i_gradient_ == last_gap.end+1) {
// We have unregistered the elements between the "gap" of
// unregistered element and the top of the stack, so can set
// the variables indicating the presence of the gap to zero
i_gradient_ = last_gap.start;
GapListIterator it = gap_list_.end();
it--;
if (most_recent_gap_ == it) {
most_recent_gap_ = gap_list_.end();
}
gap_list_.pop_back();
}
}
}
else { // Gradients to be unregistered not at top of stack.
enum {
ADDED_AT_BASE,
ADDED_AT_TOP,
NEW_GAP,
NOT_FOUND
} status = NOT_FOUND;
// First try to find if the unregistered element is at the start
// or end of an existing gap
if (!gap_list_.empty() && most_recent_gap_ != gap_list_.end()) {
// We have a "most recent" gap - check whether the gradient
// to be unregistered is here
Gap& current_gap = *most_recent_gap_;
if (gradient_index == current_gap.start - n) {
current_gap.start -= n;
status = ADDED_AT_BASE;
}
else if (gradient_index == current_gap.end + 1) {
current_gap.end += n;
status = ADDED_AT_TOP;
}
/*
else if (gradient_index > current_gap.start - n
&& gradient_index < current_gap.end + 1) {
std::cout << "** Attempt to find " << gradient_index << " in gaps ";
print_gaps();
std::cout << "\n";
throw invalid_operation("Gap list corruption");
}
*/
// Should we check for erroneous removal from middle of gap?
}
if (status == NOT_FOUND) {
// Search other gaps
for (GapListIterator it = gap_list_.begin();
it != gap_list_.end(); it++) {
if (gradient_index <= it->end + 1) {
// Gradient to unregister is either within the gap
// referenced by iterator "it", or it is between "it" and
// the previous gap in the list
if (gradient_index == it->start - n) {
status = ADDED_AT_BASE;
it->start -= n;
most_recent_gap_ = it;
}
else if (gradient_index == it->end + 1) {
status = ADDED_AT_TOP;
it->end += n;
most_recent_gap_ = it;
}
/*
else if (gradient_index > it->start - n) {
std::cout << "*** Attempt to find " << gradient_index << " in gaps ";
print_gaps();
std::cout << "\n";
throw invalid_operation("Gap list corruption");
}
*/
else {
// Insert a new gap; note that list::insert inserts
// *before* the specified location
most_recent_gap_
= gap_list_.insert(it, Gap(gradient_index,
gradient_index+n-1));
status = NEW_GAP;
}
break;
}
}
if (status == NOT_FOUND) {
gap_list_.push_back(Gap(gradient_index,
gradient_index+n-1));
most_recent_gap_ = gap_list_.end();
most_recent_gap_--;
}
}
// Finally check if gaps have merged
if (status == ADDED_AT_BASE
&& most_recent_gap_ != gap_list_.begin()) {
// Check whether the gap has merged with the next one
GapListIterator it = most_recent_gap_;
it--;
if (it->end == most_recent_gap_->start - 1) {
// Merge two gaps
most_recent_gap_->start = it->start;
gap_list_.erase(it);
}
}
else if (status == ADDED_AT_TOP) {
GapListIterator it = most_recent_gap_;
it++;
if (it != gap_list_.end()
&& it->start == most_recent_gap_->end + 1) {
// Merge two gaps
most_recent_gap_->end = it->end;
gap_list_.erase(it);
}
}
}
}
// Print each derivative statement to the specified stream (standard
// output if omitted)
void
Stack::print_statements(std::ostream& os) const
{
for (uIndex ist = 1; ist < n_statements_; ist++) {
const Statement& statement = statement_[ist];
os << ist
<< ": d[" << statement.index
<< "] = ";
if (statement_[ist-1].end_plus_one == statement_[ist].end_plus_one) {
os << "0\n";
}
else {
for (uIndex i = statement_[ist-1].end_plus_one;
i < statement.end_plus_one; i++) {
os << " + " << multiplier_[i] << "*d[" << index_[i] << "]";
}
os << "\n";
}
}
}
// Print the current gradient list to the specified stream (standard
// output if omitted)
bool
Stack::print_gradients(std::ostream& os) const
{
if (gradients_are_initialized()) {
for (uIndex i = 0; i < max_gradient_; i++) {
if (i%10 == 0) {
if (i != 0) {
os << "\n";
}
os << i << ":";
}
os << " " << gradient_[i];
}
os << "\n";
return true;
}
else {
os << "No gradients initialized\n";
return false;
}
}
// Print the list of gaps in the gradient list to the specified
// stream (standard output if omitted)
void
Stack::print_gaps(std::ostream& os) const
{
for (std::list<Gap>::const_iterator it = gap_list_.begin();
it != gap_list_.end(); it++) {
os << it->start << "-" << it->end << " ";
}
}
#ifndef ADEPT_STACK_STORAGE_STL
// Initialize the vector of gradients ready for the adjoint
// calculation
void
Stack::initialize_gradients()
{
if (max_gradient_ > 0) {
if (n_allocated_gradients_ < max_gradient_) {
if (gradient_) {
delete[] gradient_;
}
gradient_ = new Real[max_gradient_];
n_allocated_gradients_ = max_gradient_;
}
for (uIndex i = 0; i < max_gradient_; i++) {
gradient_[i] = 0.0;
}
}
gradients_initialized_ = true;
}
#else
void
Stack::initialize_gradients()
{
gradient_.resize(max_gradient_+10, 0.0);
gradients_initialized_ = true;
}
#endif
// Report information about the stack to the specified stream, or
// standard output if omitted; note that this is synonymous with
// sending the Stack object to a stream using the "<<" operator.
void
Stack::print_status(std::ostream& os) const
{
os << "Automatic Differentiation Stack (address " << this << "):\n";
if ((!is_thread_unsafe_) && _stack_current_thread == this) {
os << " Currently attached - thread safe\n";
}
else if (is_thread_unsafe_ && _stack_current_thread_unsafe == this) {
os << " Currently attached - thread unsafe\n";
}
else {
os << " Currently detached\n";
}
os << " Recording status:\n";
if (is_recording_) {
os << " Recording is ON\n";
}
else {
os << " Recording is PAUSED\n";
}
// Account for the null statement at the start by subtracting one
os << " " << n_statements()-1 << " statements ("
<< n_allocated_statements() << " allocated)";
os << " and " << n_operations() << " operations ("
<< n_allocated_operations() << " allocated)\n";
os << " " << n_gradients_registered() << " gradients currently registered ";
os << "and a total of " << max_gradients() << " needed (current index "
<< i_gradient() << ")\n";
if (gap_list_.empty()) {
os << " Gradient list has no gaps\n";
}
else {
os << " Gradient list has " << gap_list_.size() << " gaps (";
print_gaps(os);
os << ")\n";
}
os << " Computation status:\n";
if (gradients_are_initialized()) {
os << " " << max_gradients() << " gradients assigned ("
<< n_allocated_gradients() << " allocated)\n";
}
else {
os << " 0 gradients assigned (" << n_allocated_gradients()
<< " allocated)\n";
}
os << " Jacobian size: " << n_dependents() << "x" << n_independents() << "\n";
if (n_dependents() <= 10 && n_independents() <= 10) {
os << " Independent indices:";
for (std::size_t i = 0; i < independent_index_.size(); ++i) {
os << " " << independent_index_[i];
}
os << "\n Dependent indices: ";
for (std::size_t i = 0; i < dependent_index_.size(); ++i) {
os << " " << dependent_index_[i];
}
os << "\n";
}
#ifdef _OPENMP
if (have_openmp_) {
if (openmp_manually_disabled_) {
os << " Parallel Jacobian calculation manually disabled\n";
}
else {
os << " Parallel Jacobian calculation can use up to "
<< omp_get_max_threads() << " threads\n";
os << " Each thread treats " << ADEPT_MULTIPASS_SIZE
<< " (in)dependent variables\n";
}
}
else {
#endif
os << " Parallel Jacobian calculation not available\n";
#ifdef _OPENMP
}
#endif
}
} // End namespace adept
================================================
FILE: adept/StackStorageOrig.cpp
================================================
/* StackStorageOrig.cpp -- Original storage of stacks using STL containers
Copyright (C) 2014-2015 University of Reading
Author: Robin Hogan <r.j.hogan@ecmwf.int>
This file is part of the Adept library.
The Stack class inherits from a class providing the storage (and
interface to the storage) for the derivative statements that are
accumulated during the execution of an algorithm. The derivative
statements are held in two stacks described by Hogan (2014): the
"statement stack" and the "operation stack".
This file provides one of the original storage engine, which used
std::vector to hold the two stacks. Note that these stacks are
contiguous in memory, which is not ideal for very large algorithms.
*/
#include <cstring>
#include <adept/StackStorageOrig.h>
namespace adept {
namespace internal {
StackStorageOrig::~StackStorageOrig() {
if (statement_) {
delete[] statement_;
}
if (multiplier_) {
delete[] multiplier_;
}
if (index_) {
delete[] index_;
}
}
// Double the size of the operation stack, or grow it even more if
// the requested minimum number of extra entries (min) is greater
// than this would allow
void
StackStorageOrig::grow_operation_stack(uIndex min)
{
uIndex new_size = 2*n_allocated_operations_;
if (min > 0 && new_size < n_allocated_operations_+min) {
new_size += min;
}
Real* new_multiplier = new Real[new_size];
uIndex* new_index = new uIndex[new_size];
std::memcpy(new_multiplier, multiplier_, n_operations_*sizeof(Real));
std::memcpy(new_index, index_, n_operations_*sizeof(uIndex));
delete[] multiplier_;
delete[] index_;
multiplier_ = new_multiplier;
index_ = new_index;
n_allocated_operations_ = new_size;
}
// ... likewise for the statement stack
void
StackStorageOrig::grow_statement_stack(uIndex min)
{
uIndex new_size = 2*n_allocated_statements_;
if (min > 0 && new_size < n_allocated_statements_+min) {
new_size += min;
}
Statement* new_statement = new Statement[new_size];
std::memcpy(new_statement, statement_,
n_statements_*sizeof(Statement));
delete[] statement_;
statement_ = new_statement;
n_allocated_statements_ = new_size;
}
}
}
================================================
FILE: adept/Storage.cpp
================================================
/* Storage.cpp -- Global variables recording use of Storage objects
Copyright (C) 2015 European Centre for Medium-Range Weather Forecasts
Author: Robin Hogan <r.j.hogan@ecmwf.int>
This file is part of the Adept library.
*/
#include <adept/Storage.h>
namespace adept {
namespace internal {
Index n_storage_objects_created_;
Index n_storage_objects_deleted_;
}
}
================================================
FILE: adept/cppblas.cpp
================================================
/* cppblas.cpp -- C++ interface to BLAS functions
Copyright (C) 2015-2016 European Centre for Medium-Range Weather Forecasts
Author: Robin Hogan <r.j.hogan@ecmwf.int>
This file is part of the Adept library.
This file provides a C++ interface to selected Level-2 and -3 BLAS
functions in which the precision of the arguments (float versus
double) is inferred via overloading
*/
#include <adept/exception.h>
#include <adept/cppblas.h>
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#ifdef HAVE_BLAS
extern "C" {
void sgemm_(const char* TransA, const char* TransB, const int* M,
const int* N, const int* K, const float* alpha,
const float* A, const int* lda, const float* B, const int* ldb,
const float* beta, const float* C, const int* ldc);
void dgemm_(const char* TransA, const char* TransB, const int* M,
const int* N, const int* K, const double* alpha,
const double* A, const int* lda, const double* B, const int* ldb,
const double* beta, const double* C, const int* ldc);
void sgemv_(const char* TransA, const int* M, const int* N, const float* alpha,
const float* A, const int* lda, const float* X, const int* incX,
const float* beta, const float* Y, const int* incY);
void dgemv_(const char* TransA, const int* M, const int* N, const double* alpha,
const double* A, const int* lda, const double* X, const int* incX,
const double* beta, const double* Y, const int* incY);
void ssymm_(const char* side, const char* uplo, const int* M, const int* N,
const float* alpha, const float* A, const int* lda, const float* B,
const int* ldb, const float* beta, float* C, const int* ldc);
void dsymm_(const char* side, const char* uplo, const int* M, const int* N,
const double* alpha, const double* A, const int* lda, const double* B,
const int* ldb, const double* beta, double* C, const int* ldc);
void ssymv_(const char* uplo, const int* N, const float* alpha, const float* A,
const int* lda, const float* X, const int* incX, const float* beta,
const float* Y, const int* incY);
void dsymv_(const char* uplo, const int* N, const double* alpha, const double* A,
const int* lda, const double* X, const int* incX, const double* beta,
const double* Y, const int* incY);
void sgbmv_(const char* TransA, const int* M, const int* N, const int* kl,
const int* ku, const float* alpha, const float* A, const int* lda,
const float* X, const int* incX, const float* beta,
const float* Y, const int* incY);
void dgbmv_(const char* TransA, const int* M, const int* N, const int* kl,
const int* ku, const double* alpha, const double* A, const int* lda,
const double* X, const int* incX, const double* beta,
const double* Y, const int* incY);
}
namespace adept {
namespace internal {
// Matrix-matrix multiplication for general dense matrices
#define ADEPT_DEFINE_GEMM(T, FUNC, FUNC_COMPLEX) \
void cppblas_gemm(BLAS_ORDER Order, \
BLAS_TRANSPOSE TransA, \
BLAS_TRANSPOSE TransB, \
int M, int N, \
int K, T alpha, const T *A, \
int lda, const T *B, int ldb, \
T beta, T *C, int ldc) { \
if (Order == BlasColMajor) { \
FUNC(&TransA, &TransB, &M, &N, &K, &alpha, A, &lda, \
B, &ldb, &beta, C, &ldc); \
} \
else { \
FUNC(&TransB, &TransA, &N, &M, &K, &alpha, B, &ldb, \
A, &lda, &beta, C, &ldc); \
} \
}
ADEPT_DEFINE_GEMM(double, dgemm_, zgemm_)
ADEPT_DEFINE_GEMM(float, sgemm_, cgemm_)
#undef ADEPT_DEFINE_GEMM
// Matrix-vector multiplication for a general dense matrix
#define ADEPT_DEFINE_GEMV(T, FUNC, FUNC_COMPLEX) \
void cppblas_gemv(const BLAS_ORDER Order, \
const BLAS_TRANSPOSE TransA, \
const int M, const int N, \
const T alpha, const T *A, const int lda, \
const T *X, const int incX, const T beta, \
T *Y, const int incY) { \
if (Order == BlasColMajor) { \
FUNC(&TransA, &M, &N, &alpha, A, &lda, X, &incX, \
&beta, Y, &incY); \
} \
else { \
BLAS_TRANSPOSE TransNew \
= TransA == BlasTrans ? BlasNoTrans : BlasTrans; \
FUNC(&TransNew, &N, &M, &alpha, A, &lda, X, &incX, \
&beta, Y, &incY); \
} \
}
ADEPT_DEFINE_GEMV(double, dgemv_, zgemv_)
ADEPT_DEFINE_GEMV(float, sgemv_, cgemv_)
#undef ADEPT_DEFINE_GEMV
// Matrix-matrix multiplication where matrix A is symmetric
// FIX! CHECK ROW MAJOR VERSION IS RIGHT
#define ADEPT_DEFINE_SYMM(T, FUNC, FUNC_COMPLEX) \
void cppblas_symm(const BLAS_ORDER Order, \
const BLAS_SIDE Side, \
const BLAS_UPLO Uplo, \
const int M, const int N, \
const T alpha, const T *A, const int lda, \
const T *B, const int ldb, const T beta, \
T *C, const int ldc) { \
if (Order == BlasColMajor) { \
FUNC(&Side, &Uplo, &M, &N, &alpha, A, &lda, \
B, &ldb, &beta, C, &ldc); \
} \
else { \
BLAS_SIDE SideNew = Side == BlasLeft ? BlasRight : BlasLeft; \
BLAS_UPLO UploNew = Uplo == BlasUpper ? BlasLower : BlasUpper; \
FUNC(&SideNew, &UploNew, &N, &M, &alpha, A, &lda, \
B, &ldb, &beta, C, &ldc); \
} \
}
ADEPT_DEFINE_SYMM(double, dsymm_, zsymm_)
ADEPT_DEFINE_SYMM(float, ssymm_, csymm_)
#undef ADEPT_DEFINE_SYMM
// Matrix-vector multiplication where the matrix is symmetric
#define ADEPT_DEFINE_SYMV(T, FUNC, FUNC_COMPLEX) \
void cppblas_symv(const BLAS_ORDER Order, \
const BLAS_UPLO Uplo, \
const int N, const T alpha, const T *A, \
const int lda, const T *X, const int incX, \
const T beta, T *Y, const int incY) { \
if (Order == BlasColMajor) { \
FUNC(&Uplo, &N, &alpha, A, &lda, X, &incX, &beta, Y, &incY); \
} \
else { \
BLAS_UPLO UploNew = Uplo == BlasUpper ? BlasLower : BlasUpper; \
FUNC(&UploNew, &N, &alpha, A, &lda, X, &incX, &beta, Y, &incY); \
} \
}
ADEPT_DEFINE_SYMV(double, dsymv_, zsymv_)
ADEPT_DEFINE_SYMV(float, ssymv_, csymv_)
#undef ADEPT_DEFINE_SYMV
// Matrix-vector multiplication for a general band matrix
#define ADEPT_DEFINE_GBMV(T, FUNC, FUNC_COMPLEX) \
void cppblas_gbmv(const BLAS_ORDER Order, \
const BLAS_TRANSPOSE TransA, \
const int M, const int N, \
const int KL, const int KU, const T alpha,\
const T *A, const int lda, const T *X, \
const int incX, const T beta, T *Y, \
const int incY) { \
if (Order == BlasColMajor) { \
FUNC(&TransA, &M, &N, &KL, &KU, &alpha, A, &lda, \
X, &incX, &beta, Y, &incY); \
} \
else { \
BLAS_TRANSPOSE TransNew \
= TransA == BlasTrans ? BlasNoTrans : BlasTrans; \
FUNC(&TransNew, &N, &M, &KU, &KL, &alpha, A, &lda, \
X, &incX, &beta, Y, &incY); \
} \
}
ADEPT_DEFINE_GBMV(double, dgbmv_, zgbmv_)
ADEPT_DEFINE_GBMV(float, sgbmv_, cgbmv_)
#undef ADEPT_DEFINE_GBMV
} // End namespace internal
} // End namespace adept
#else // Don't have BLAS
namespace adept {
namespace internal {
// Matrix-matrix multiplication for general dense matrices
#define ADEPT_DEFINE_GEMM(T, FUNC, FUNC_COMPLEX) \
void cppblas_gemm(BLAS_ORDER Order, \
BLAS_TRANSPOSE TransA, \
BLAS_TRANSPOSE TransB, \
int M, int N, \
int K, T alpha, const T *A, \
int lda, const T *B, int ldb, \
T beta, T *C, int ldc) { \
throw feature_not_available("Cannot perform matrix-matrix multiplication because compiled without BLAS"); \
}
ADEPT_DEFINE_GEMM(double, dgemm_, zgemm_)
ADEPT_DEFINE_GEMM(float, sgemm_, cgemm_)
#undef ADEPT_DEFINE_GEMM
// Matrix-vector multiplication for a general dense matrix
#define ADEPT_DEFINE_GEMV(T, FUNC, FUNC_COMPLEX) \
void cppblas_gemv(const BLAS_ORDER Order, \
const BLAS_TRANSPOSE TransA, \
const int M, const int N, \
const T alpha, const T *A, const int lda, \
const T *X, const int incX, const T beta, \
T *Y, const int incY) { \
throw feature_not_available("Cannot perform matrix-vector multiplication because compiled without BLAS"); \
}
ADEPT_DEFINE_GEMV(double, dgemv_, zgemv_)
ADEPT_DEFINE_GEMV(float, sgemv_, cgemv_)
#undef ADEPT_DEFINE_GEMV
// Matrix-matrix multiplication where matrix A is symmetric
// FIX! CHECK ROW MAJOR VERSION IS RIGHT
#define ADEPT_DEFINE_SYMM(T, FUNC, FUNC_COMPLEX) \
void cppblas_symm(const BLAS_ORDER Order, \
const BLAS_SIDE Side, \
const BLAS_UPLO Uplo, \
const int M, const int N, \
const T alpha, const T *A, const int lda, \
const T *B, const int ldb, const T beta, \
T *C, const int ldc) { \
throw feature_not_available("Cannot perform symmetric matrix-matrix multiplication because compiled without BLAS"); \
}
ADEPT_DEFINE_SYMM(double, dsymm_, zsymm_)
ADEPT_DEFINE_SYMM(float, ssymm_, csymm_)
#undef ADEPT_DEFINE_SYMM
// Matrix-vector multiplication where the matrix is symmetric
#define ADEPT_DEFINE_SYMV(T, FUNC, FUNC_COMPLEX) \
void cppblas_symv(const BLAS_ORDER Order, \
const BLAS_UPLO Uplo, \
const int N, const T alpha, const T *A, \
const int lda, const T *X, const int incX, \
const T beta, T *Y, const int incY) { \
throw feature_not_available("Cannot perform symmetric matrix-vector multiplication because compiled without BLAS"); \
}
ADEPT_DEFINE_SYMV(double, dsymv_, zsymv_)
ADEPT_DEFINE_SYMV(float, ssymv_, csymv_)
#undef ADEPT_DEFINE_SYMV
// Matrix-vector multiplication for a general band matrix
#define ADEPT_DEFINE_GBMV(T, FUNC, FUNC_COMPLEX) \
void cppblas_gbmv(const BLAS_ORDER Order, \
const BLAS_TRANSPOSE TransA, \
const int M, const int N, \
const int KL, const int KU, const T alpha,\
const T *A, const int lda, const T *X, \
const int incX, const T beta, T *Y, \
const int incY) { \
throw feature_not_available("Cannot perform band matrix-vector multiplication because compiled without BLAS"); \
}
ADEPT_DEFINE_GBMV(double, dgbmv_, zgbmv_)
ADEPT_DEFINE_GBMV(float, sgbmv_, cgbmv_)
#undef ADEPT_DEFINE_GBMV
}
}
#endif
================================================
FILE: adept/cpplapack.h
================================================
/* cpplapack.h -- C++ interface to LAPACK
Copyright (C) 2015-2016 European Centre for Medium-Range Weather Forecasts
Author: Robin Hogan <r.j.hogan@ecmwf.int>
This file is part of the Adept library.
*/
#ifndef AdeptCppLapack_H
#define AdeptCppLapack_H 1
#include <vector>
#include <cstddef>
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#ifdef HAVE_LAPACK
extern "C" {
// External LAPACK Fortran functions
void sgetrf_(const int* m, const int* n, float* a, const int* lda, int* ipiv, int* info);
void dgetrf_(const int* m, const int* n, double* a, const int* lda, int* ipiv, int* info);
void sgetri_(const int* n, float* a, const int* lda, const int* ipiv,
float* work, const int* lwork, int* info);
void dgetri_(const int* n, double* a, const int* lda, const int* ipiv,
double* work, const int* lwork, int* info);
void ssytrf_(const char* uplo, const int* n, float* a, const int* lda, int* ipiv,
float* work, const int* lwork, int* info);
void dsytrf_(const char* uplo, const int* n, double* a, const int* lda, int* ipiv,
double* work, const int* lwork, int* info);
void ssytri_(const char* uplo, const int* n, float* a, const int* lda,
const int* ipiv, float* work, int* info);
void dsytri_(const char* uplo, const int* n, double* a, const int* lda,
const int* ipiv, double* work, int* info);
void ssysv_(const char* uplo, const int* n, const int* nrhs, float* a, const int* lda,
int* ipiv, float* b, const int* ldb, float* work, const int* lwork, int* info);
void dsysv_(const char* uplo, const int* n, const int* nrhs, double* a, const int* lda,
int* ipiv, double* b, const int* ldb, double* work, const int* lwork, int* info);
void sgesv_(const int* n, const int* nrhs, float* a, const int* lda,
int* ipiv, float* b, const int* ldb, int* info);
void dgesv_(const int* n, const int* nrhs, double* a, const int* lda,
int* ipiv, double* b, const int* ldb, int* info);
}
namespace adept {
// Overloaded functions provide both single &
// double precision versions, and prevents the huge lapacke.h having
// to be included in all user code
namespace internal {
typedef int lapack_int;
// Factorize a general matrix
inline
int cpplapack_getrf(int n, float* a, int lda, int* ipiv) {
int info;
sgetrf_(&n, &n, a, &lda, ipiv, &info);
return info;
}
inline
int cpplapack_getrf(int n, double* a, int lda, int* ipiv) {
int info;
dgetrf_(&n, &n, a, &lda, ipiv, &info);
return info;
}
// Invert a general matrix
inline
int cpplapack_getri(int n, float* a, int lda, const int* ipiv) {
int info;
float work_query;
int lwork = -1;
// Find out how much work memory required
sgetri_(&n, a, &lda, ipiv, &work_query, &lwork, &info);
lwork = static_cast<int>(work_query);
std::vector<float> work(static_cast<std::size_t>(lwork));
// Do full calculation
sgetri_(&n, a, &lda, ipiv, &work[0], &lwork, &info);
return info;
}
inline
int cpplapack_getri(int n, double* a, int lda, const int* ipiv) {
int info;
double work_query;
int lwork = -1;
// Find out how much work memory required
dgetri_(&n, a, &lda, ipiv, &work_query, &lwork, &info);
lwork = static_cast<int>(work_query);
std::vector<double> work(static_cast<std::size_t>(lwork));
// Do full calculation
dgetri_(&n, a, &lda, ipiv, &work[0], &lwork, &info);
return info;
}
// Factorize a symmetric matrix
inline
int cpplapack_sytrf(char uplo, int n, float* a, int lda, int* ipiv) {
int info;
float work_query;
int lwork = -1;
// Find out how much work memory required
ssytrf_(&uplo, &n, a, &lda, ipiv, &work_query, &lwork, &info);
lwork = static_cast<int>(work_query);
std::vector<float> work(static_cast<std::size_t>(lwork));
// Do full calculation
ssytrf_(&uplo, &n, a, &lda, ipiv, &work[0], &lwork, &info);
return info;
}
inline
int cpplapack_sytrf(char uplo, int n, double* a, int lda, int* ipiv) {
int info;
double work_query;
int lwork = -1;
// Find out how much work memory required
dsytrf_(&uplo, &n, a, &lda, ipiv, &work_query, &lwork, &info);
lwork = static_cast<int>(work_query);
std::vector<double> work(static_cast<std::size_t>(lwork));
// Do full calculation
dsytrf_(&uplo, &n, a, &lda, ipiv, &work[0], &lwork, &info);
return info;
}
// Invert a symmetric matrix
inline
int cpplapack_sytri(char uplo, int n, float* a, int lda, const int* ipiv) {
int info;
std::vector<float> work(n);
ssytri_(&uplo, &n, a, &lda, ipiv, &work[0], &info);
return info;
}
inline
int cpplapack_sytri(char uplo, int n, double* a, int lda, const int* ipiv) {
int info;
std::vector<double> work(n);
dsytri_(&uplo, &n, a, &lda, ipiv, &work[0], &info);
return info;
}
// Solve system of linear equations with general matrix
inline
int cpplapack_gesv(int n, int nrhs, float* a, int lda,
int* ipiv, float* b, int ldb) {
int info;
sgesv_(&n, &nrhs, a, &lda, ipiv, b, &lda, &info);
return info;
}
inline
int cpplapack_gesv(int n, int nrhs, double* a, int lda,
int* ipiv, double* b, int ldb) {
int info;
dgesv_(&n, &nrhs, a, &lda, ipiv, b, &lda, &info);
return info;
}
// Solve system of linear equations with symmetric matrix
inline
int cpplapack_sysv(char uplo, int n, int nrhs, float* a, int lda, int* ipiv,
float* b, int ldb) {
int info;
float work_query;
int lwork = -1;
// Find out how much work memory required
ssysv_(&uplo, &n, &nrhs, a, &lda, ipiv, b, &ldb, &work_query, &lwork, &info);
lwork = static_cast<int>(work_query);
std::vector<float> work(static_cast<std::size_t>(lwork));
// Do full calculation
ssysv_(&uplo, &n, &nrhs, a, &lda, ipiv, b, &ldb, &work[0], &lwork, &info);
return info;
}
inline
int cpplapack_sysv(char uplo, int n, int nrhs, double* a, int lda, int* ipiv,
double* b, int ldb) {
int info;
double work_query;
int lwork = -1;
// Find out how much work memory required
dsysv_(&uplo, &n, &nrhs, a, &lda, ipiv, b, &ldb, &work_query, &lwork, &info);
lwork = static_cast<int>(work_query);
std::vector<double> work(static_cast<std::size_t>(lwork));
// Do full calculation
dsysv_(&uplo, &n, &nrhs, a, &lda, ipiv, b, &ldb, &work[0], &lwork, &info);
return info;
}
}
}
#endif
#endif
================================================
FILE: adept/index.cpp
================================================
/* index.cpp -- Definitions of "end" and "__" for array indexing
Copyright (C) 2015 European Centre for Medium-Range Weather Forecasts
Robin Hogan <r.j.hogan@ecmwf.int>
This file is part of the Adept library.
*/
#include <adept/RangeIndex.h>
namespace adept {
::adept::internal::EndIndex end;
::adept::internal::AllIndex __;
}
================================================
FILE: adept/inv.cpp
================================================
/* inv.cpp -- Invert matrices
Copyright (C) 2015-2016 European Centre for Medium-Range Weather Forecasts
Author: Robin Hogan <r.j.hogan@ecmwf.int>
This file is part of the Adept library.
*/
#include <vector>
#include <adept/Array.h>
#include <adept/SpecialMatrix.h>
#ifndef AdeptSource_H
#include "cpplapack.h"
#endif
#ifdef HAVE_LAPACK
namespace adept {
using namespace internal;
// -------------------------------------------------------------------
// Invert general square matrix A
// -------------------------------------------------------------------
template <typename Type>
Array<2,Type,false>
inv(const Array<2,Type,false>& A) {
using internal::cpplapack_getrf;
using internal::cpplapack_getri;
if (A.dimension(0) != A.dimension(1)) {
throw invalid_operation("Only square matrices can be inverted"
ADEPT_EXCEPTION_LOCATION);
}
Array<2,Type,false> A_;
// LAPACKE is more efficient with column-major input
A_.resize_column_major(A.dimensions());
A_ = A;
std::vector<lapack_int> ipiv(A_.dimension(0));
// lapack_int status = LAPACKE_dgetrf(LAPACK_COL_MAJOR, A_.dimension(0), A_.dimension(1),
// A_.data(), A_.offset(1), &ipiv[0]);
lapack_int status = cpplapack_getrf(A_.dimension(0),
A_.data(), A_.offset(1), &ipiv[0]);
if (status != 0) {
std::stringstream s;
s << "Failed to factorize matrix: LAPACK ?getrf returned code " << status;
throw(matrix_ill_conditioned(s.str() ADEPT_EXCEPTION_LOCATION));
}
// status = LAPACKE_dgetri(LAPACK_COL_MAJOR, A_.dimension(0),
// A_.data(), A_.offset(1), &ipiv[0]);
status = cpplapack_getri(A_.dimension(0),
A_.data(), A_.offset(1), &ipiv[0]);
if (status != 0) {
std::stringstream s;
s << "Failed to invert matrix: LAPACK ?getri returned code " << status;
throw(matrix_ill_conditioned(s.str() ADEPT_EXCEPTION_LOCATION));
}
return A_;
}
// -------------------------------------------------------------------
// Invert symmetric matrix A
// -------------------------------------------------------------------
template <typename Type, SymmMatrixOrientation Orient>
SpecialMatrix<Type,SymmEngine<Orient>,false>
inv(const SpecialMatrix<Type,SymmEngine<Orient>,false>& A) {
using internal::cpplapack_sytrf;
using internal::cpplapack_sytri;
SpecialMatrix<Type,SymmEngine<Orient>,false> A_;
A_.resize(A.dimension());
A_ = A;
// Treat symmetric matrix as column-major
char uplo;
if (Orient == ROW_LOWER_COL_UPPER) {
uplo = 'U';
}
else {
uplo = 'L';
}
std::vector<lapack_int> ipiv(A_.dimension(0));
// lapack_int status = LAPACKE_dsytrf(LAPACK_COL_MAJOR, uplo, A_.dimension(),
// A_.data(), A_.offset(), &ipiv[0]);
lapack_int status = cpplapack_sytrf(uplo, A_.dimension(),
A_.data(), A_.offset(), &ipiv[0]);
if (status != 0) {
std::stringstream s;
s << "Failed to factorize symmetric matrix: LAPACK ?sytrf returned code " << status;
throw(matrix_ill_conditioned(s.str() ADEPT_EXCEPTION_LOCATION));
}
// status = LAPACKE_dsytri(LAPACK_COL_MAJOR, uplo, A_.dimension(),
// A_.data(), A_.offset(), &ipiv[0]);
status = cpplapack_sytri(uplo, A_.dimension(),
A_.data(), A_.offset(), &ipiv[0]);
if (status != 0) {
std::stringstream s;
s << "Failed to invert symmetric matrix: LAPACK ?sytri returned code " << status;
throw(matrix_ill_conditioned(s.str() ADEPT_EXCEPTION_LOCATION));
}
return A_;
}
}
#else // LAPACK not available
namespace adept {
using namespace internal;
// -------------------------------------------------------------------
// Invert general square matrix A
// -------------------------------------------------------------------
template <typename Type>
Array<2,Type,false>
inv(const Array<2,Type,false>& A) {
throw feature_not_available("Cannot invert matrix because compiled without LAPACK");
}
// -------------------------------------------------------------------
// Invert symmetric matrix A
// -------------------------------------------------------------------
template <typename Type, SymmMatrixOrientation Orient>
SpecialMatrix<Type,SymmEngine<Orient>,false>
inv(const SpecialMatrix<Type,SymmEngine<Orient>,false>& A) {
throw feature_not_available("Cannot invert matrix because compiled without LAPACK");
}
}
#endif
namespace adept {
// -------------------------------------------------------------------
// Explicit instantiations
// -------------------------------------------------------------------
#define ADEPT_EXPLICIT_INV(TYPE) \
template Array<2,TYPE,false> \
inv(const Array<2,TYPE,false>& A); \
template SpecialMatrix<TYPE,SymmEngine<ROW_LOWER_COL_UPPER>,false> \
inv(const SpecialMatrix<TYPE,SymmEngine<ROW_LOWER_COL_UPPER>,false>&); \
template SpecialMatrix<TYPE,SymmEngine<ROW_UPPER_COL_LOWER>,false> \
inv(const SpecialMatrix<TYPE,SymmEngine<ROW_UPPER_COL_LOWER>,false>&)
ADEPT_EXPLICIT_INV(float);
ADEPT_EXPLICIT_INV(double);
#undef ADEPT_EXPLICIT_INV
}
================================================
FILE: adept/jacobian.cpp
================================================
/* jacobian.cpp -- Computation of Jacobian matrix
Copyright (C) 2012-2014 University of Reading
Copyright (C) 2015-2020 European Centre for Medium-Range Weather Forecasts
Author: Robin Hogan <r.j.hogan@ecmwf.int>
This file is part of the Adept library.
*/
#ifdef _OPENMP
#include <omp.h>
#endif
#include <adept_arrays.h>
namespace adept {
namespace internal {
static const int MULTIPASS_SIZE = ADEPT_REAL_PACKET_SIZE == 1 ? ADEPT_MULTIPASS_SIZE : ADEPT_REAL_PACKET_SIZE;
}
using namespace internal;
template <typename T>
T _check_long_double() {
// The user may have requested Real to be of type "long double" by
// specifying ADEPT_REAL_TYPE_SIZE=16. If the present system can
// only support double then sizeof(long double) will be 8, but
// Adept will not be emitting the best code for this, so it is
// probably better to fail forcing the user to specify
// ADEPT_REAL_TYPE_SIZE=8.
ADEPT_STATIC_ASSERT(ADEPT_REAL_TYPE_SIZE != 16 || ADEPT_REAL_TYPE_SIZE == sizeof(Real),
COMPILER_DOES_NOT_SUPPORT_16_BYTE_LONG_DOUBLE);
return 1;
}
#if ADEPT_REAL_PACKET_SIZE > 1
void
Stack::jacobian_forward_kernel(Real* __restrict gradient_multipass_b) const
{
// Loop forward through the derivative statements
for (uIndex ist = 1; ist < n_statements_; ist++) {
const Statement& statement = statement_[ist];
// We copy the LHS to "a" in case it appears on the RHS in any
// of the following statements
Packet<Real> a; // Zeroed automatically
// Loop through operations
for (uIndex iop = statement_[ist-1].end_plus_one;
iop < statement.end_plus_one; iop++) {
Packet<Real> g(gradient_multipass_b+index_[iop]*MULTIPASS_SIZE);
Packet<Real> m(multiplier_[iop]);
a += m * g;
}
// Copy the results
a.put(gradient_multipass_b+statement.index*MULTIPASS_SIZE);
} // End of loop over statements
}
#else
void
Stack::jacobian_forward_kernel(Real* __restrict gradient_multipass_b) const
{
// Loop forward through the derivative statements
for (uIndex ist = 1; ist < n_statements_; ist++) {
const Statement& statement = statement_[ist];
// We copy the LHS to "a" in case it appears on the RHS in any
// of the following statements
Block<MULTIPASS_SIZE,Real> a; // Zeroed automatically
// Loop through operations
for (uIndex iop = statement_[ist-1].end_plus_one;
iop < statement.end_plus_one; iop++) {
for (uIndex i = 0; i < MULTIPASS_SIZE; i++) {
a[i] += multiplier_[iop]*gradient_multipass_b[index_[iop]*MULTIPASS_SIZE+i];
}
}
// Copy the results
for (uIndex i = 0; i < MULTIPASS_SIZE; i++) {
gradient_multipass_b[statement.index*MULTIPASS_SIZE+i] = a[i];
}
} // End of loop over statements
}
#endif
void
Stack::jacobian_forward_kernel_extra(Real* __restrict gradient_multipass_b,
uIndex n_extra) const
{
// Loop forward through the derivative statements
for (uIndex ist = 1; ist < n_statements_; ist++) {
const Statement& statement = statement_[ist];
// We copy the LHS to "a" in case it appears on the RHS in any
// of the following statements
Block<MULTIPASS_SIZE,Real> a; // Zeroed automatically
// Loop through operations
for (uIndex iop = statement_[ist-1].end_plus_one;
iop < statement.end_plus_one; iop++) {
for (uIndex i = 0; i < n_extra; i++) {
a[i] += multiplier_[iop]*gradient_multipass_b[index_[iop]*MULTIPASS_SIZE+i];
}
}
// Copy the results
for (uIndex i = 0; i < n_extra; i++) {
gradient_multipass_b[statement.index*MULTIPASS_SIZE+i] = a[i];
}
} // End of loop over statements
}
// Compute the Jacobian matrix, parallelized using OpenMP. Normally
// the user would call the jacobian or jacobian_forward functions,
// and the OpenMP version would only be called if OpenMP is
// available and the Jacobian matrix is large enough for
// parallelization to be worthwhile. Note that jacobian_out must be
// allocated to be at least of size m*n, where m is the number of
// dependent variables and n is the number of independents. The
// independents and dependents must have already been identified
// with the functions "independent" and "dependent", otherwise this
// function will fail with FAILURE_XXDEPENDENT_NOT_IDENTIFIED. The
// offsets in memory of the two dimensions are provided by
// dep_offset and indep_offset. This is implemented using a forward
// pass, appropriate for m>=n.
void
Stack::jacobian_forward_openmp(Real* jacobian_out,
Index dep_offset, Index indep_offset) const
{
// Number of blocks to cycle through, including a possible last
// block containing fewer than MULTIPASS_SIZE variables
int n_block = (n_independent() + MULTIPASS_SIZE - 1)
/ MULTIPASS_SIZE;
uIndex n_extra = n_independent() % MULTIPASS_SIZE;
#pragma omp parallel
{
// std::vector<Block<MULTIPASS_SIZE,Real> >
// gradient_multipass_b(max_gradient_);
uIndex gradient_multipass_size = max_gradient_*MULTIPASS_SIZE;
Real* __restrict gradient_multipass_b
= alloc_aligned<Real>(gradient_multipass_size);
#pragma omp for schedule(static)
for (int iblock = 0; iblock < n_block; iblock++) {
// Set the index to the dependent variables for this block
uIndex i_independent = MULTIPASS_SIZE * iblock;
uIndex block_size = MULTIPASS_SIZE;
// If this is the last iteration and the number of extra
// elements is non-zero, then set the block size to the number
// of extra elements. If the number of extra elements is zero,
// then the number of independent variables is exactly divisible
// by MULTIPASS_SIZE, so the last iteration will be the
// same as all the rest.
if (iblock == n_block-1 && n_extra > 0) {
block_size = n_extra;
}
// Set the initial gradients all to zero
for (uIndex i = 0; i < gradient_multipass_size; i++) {
gradient_multipass_b[i] = 0.0;
}
// Each seed vector has one non-zero entry of 1.0
for (uIndex i = 0; i < block_size; i++) {
gradient_multipass_b[independent_index_[i_independent+i]*MULTIPASS_SIZE+i] = 1.0;
}
jacobian_forward_kernel(gradient_multipass_b);
// Copy the gradients corresponding to the dependent variables
// into the Jacobian matrix
if (indep_offset == 1) {
for (uIndex idep = 0; idep < n_dependent(); idep++) {
for (uIndex i = 0; i < block_size; i++) {
jacobian_out[idep*dep_offset+i_independent+i]
= gradient_multipass_b[dependent_index_[idep]*MULTIPASS_SIZE+i];
}
}
}
else {
for (uIndex idep = 0; idep < n_dependent(); idep++) {
for (uIndex i = 0; i < block_size; i++) {
jacobian_out[(i_independent+i)*indep_offset+idep*dep_offset]
= gradient_multipass_b[dependent_index_[idep]*MULTIPASS_SIZE+i];
}
}
}
} // End of loop over blocks
free_aligned(gradient_multipass_b);
} // End of parallel section
} // End of jacobian function
// Compute the Jacobian matrix; note that jacobian_out must be
// allocated to be of size m*n, where m is the number of dependent
// variables and n is the number of independents. The independents
// and dependents must have already been identified with the
// functions "independent" and "dependent", otherwise this function
// will fail with FAILURE_XXDEPENDENT_NOT_IDENTIFIED. This is
// implemented using a forward pass, appropriate for m>=n.
void
Stack::jacobian_forward(Real* jacobian_out,
Index dep_offset, Index indep_offset) const
{
if (independent_index_.empty() || dependent_index_.empty()) {
throw(dependents_or_independents_not_identified());
}
// If either of the offsets are zero, set them to the size of the
// other dimension, which assumes that the full Jacobian matrix is
// contiguous in memory.
if (dep_offset <= 0) {
dep_offset = n_independent();
}
if (indep_offset <= 0) {
indep_offset = n_dependent();
}
#ifdef _OPENMP
if (have_openmp_
&& !openmp_manually_disabled_
&& n_independent() > MULTIPASS_SIZE
&& omp_get_max_threads() > 1) {
// Call the parallel version
jacobian_forward_openmp(jacobian_out, dep_offset, indep_offset);
return;
}
#endif
// For optimization reasons, we process a block of
// MULTIPASS_SIZE columns of the Jacobian at once; calculate
// how many blocks are needed and how many extras will remain
uIndex n_block = n_independent() / MULTIPASS_SIZE;
uIndex n_extra = n_independent() % MULTIPASS_SIZE;
///gradient_multipass_.resize(max_gradient_);
uIndex gradient_multipass_size = max_gradient_*MULTIPASS_SIZE;
Real* __restrict gradient_multipass_b
= alloc_aligned<Real>(gradient_multipass_size);
// Loop over blocks of MULTIPASS_SIZE columns
for (uIndex iblock = 0; iblock < n_block; iblock++) {
// Set the index to the dependent variables for this block
uIndex i_independent = MULTIPASS_SIZE * iblock;
// Set the initial gradients all to zero
///zero_gradient_multipass();
for (uIndex i = 0; i < gradient_multipass_size; i++) {
gradient_multipass_b[i] = 0.0;
}
// Each seed vector has one non-zero entry of 1.0
for (uIndex i = 0; i < MULTIPASS_SIZE; i++) {
gradient_multipass_b[independent_index_[i_independent+i]*MULTIPASS_SIZE+i] = 1.0;
}
jacobian_forward_kernel(gradient_multipass_b);
// Copy the gradients corresponding to the dependent variables
// into the Jacobian matrix
if (indep_offset == 1) {
for (uIndex idep = 0; idep < n_dependent(); idep++) {
for (uIndex i = 0; i < MULTIPASS_SIZE; i++) {
jacobian_out[idep*dep_offset+i_independent+i]
= gradient_multipass_b[dependent_index_[idep]*MULTIPASS_SIZE+i];
}
}
}
else {
for (uIndex idep = 0; idep < n_dependent(); idep++) {
for (uIndex i = 0; i < MULTIPASS_SIZE; i++) {
jacobian_out[(i_independent+i)*indep_offset+idep*dep_offset]
= gradient_multipass_b[dependent_index_[idep]*MULTIPASS_SIZE+i];
}
}
}
} // End of loop over blocks
// Now do the same but for the remaining few columns in the matrix
if (n_extra > 0) {
uIndex i_independent = MULTIPASS_SIZE * n_block;
///zero_gradient_multipass();
for (uIndex i = 0; i < gradient_multipass_size; i++) {
gradient_multipass_b[i] = 0.0;
}
for (uIndex i = 0; i < n_extra; i++) {
gradient_multipass_b[independent_index_[i_independent+i]*MULTIPASS_SIZE+i] = 1.0;
}
jacobian_forward_kernel_extra(gradient_multipass_b, n_extra);
if (indep_offset == 1) {
for (uIndex idep = 0; idep < n_dependent(); idep++) {
for (uIndex i = 0; i < n_extra; i++) {
jacobian_out[idep*dep_offset+i_independent+i]
= gradient_multipass_b[dependent_index_[idep]*MULTIPASS_SIZE+i];
}
}
}
else {
for (uIndex idep = 0; idep < n_dependent(); idep++) {
for (uIndex i = 0; i < n_extra; i++) {
jacobian_out[(i_independent+i)*indep_offset+idep*dep_offset]
= gradient_multipass_b[dependent_index_[idep]*MULTIPASS_SIZE+i];
}
}
}
}
free_aligned(gradient_multipass_b);
}
// Compute the Jacobian matrix, parallelized using OpenMP. Normally
// the user would call the jacobian or jacobian_reverse functions,
// and the OpenMP version would only be called if OpenMP is
// available and the Jacobian matrix is large enough for
// parallelization to be worthwhile. Note that jacobian_out must be
// allocated to be at least of size m*n, where m is the number of
// dependent variables and n is the number of independents. The
// independents and dependents must have already been identified
// with the functions "independent" and "dependent", otherwise this
// function will fail with FAILURE_XXDEPENDENT_NOT_IDENTIFIED. The
// offsets in memory of the two dimensions are provided by
// dep_offset and indep_offset. This is implemented using a reverse
// pass, appropriate for m<n.
void
Stack::jacobian_reverse_openmp(Real* jacobian_out,
Index dep_offset, Index indep_offset) const
{
// Number of blocks to cycle through, including a possible last
// block containing fewer than MULTIPASS_SIZE variables
int n_block = (n_dependent() + MULTIPASS_SIZE - 1)
/ MULTIPASS_SIZE;
uIndex n_extra = n_dependent() % MULTIPASS_SIZE;
// Inside the OpenMP loop, the "this" pointer may be NULL if the
// adept::Stack pointer is declared as thread-local and if the
// OpenMP memory model uses thread-local storage for private
// data. If this is the case then local pointers to or copies of
// the following members of the adept::Stack object may need to be
// made: dependent_index_ n_statements_ statement_ multiplier_
// index_ independent_index_ n_dependent() n_independent().
// Limited testing implies this is OK though.
#pragma omp parallel
{
std::vector<Block<MULTIPASS_SIZE,Real> >
gradient_multipass_b(max_gradient_);
#pragma omp for schedule(static)
for (int iblock = 0; iblock < n_block; iblock++) {
// Set the index to the dependent variables for this block
uIndex i_dependent = MULTIPASS_SIZE * iblock;
uIndex block_size = MULTIPASS_SIZE;
// If this is the last iteration and the number of extra
// elements is non-zero, then set the block size to the number
// of extra elements. If the number of extra elements is zero,
// then the number of independent variables is exactly divisible
// by MULTIPASS_SIZE, so the last iteration will be the
// same as all the rest.
if (iblock == n_block-1 && n_extra > 0) {
block_size = n_extra;
}
// Set the initial gradients all to zero
for (std::size_t i = 0; i < gradient_multipass_b.size(); i++) {
gradient_multipass_b[i].zero();
}
// Each seed vector has one non-zero entry of 1.0
for (uIndex i = 0; i < block_size; i++) {
gradient_multipass_b[dependent_index_[i_dependent+i]][i] = 1.0;
}
// Loop backward through the derivative statements
for (uIndex ist = n_statements_-1; ist > 0; ist--) {
const Statement& statement = statement_[ist];
// We copy the RHS to "a" in case it appears on the LHS in any
// of the following statements
Real a[MULTIPASS_SIZE];
#if MULTIPASS_SIZE > MULTIPASS_SIZE_ZERO_CHECK
// For large blocks, we only process the ones where a[i] is
// non-zero
uIndex i_non_zero[MULTIPASS_SIZE];
#endif
uIndex n_non_zero = 0;
for (uIndex i = 0; i < block_size; i++) {
a[i] = gradient_multipass_b[statement.index][i];
gradient_multipass_b[statement.index][i] = 0.0;
if (a[i] != 0.0) {
#if MULTIPASS_SIZE > MULTIPASS_SIZE_ZERO_CHECK
i_non_zero[n_non_zero++] = i;
#else
n_non_zero = 1;
#endif
}
}
// Only do anything for this statement if any of the a values
// are non-zero
if (n_non_zero) {
// Loop through the operations
for (uIndex iop = statement_[ist-1].end_plus_one;
iop < statement.end_plus_one; iop++) {
// Try to minimize pointer dereferencing by making local
// copies
Real multiplier = multiplier_[iop];
Real* __restrict gradient_multipass
= &(gradient_multipass_b[index_[iop]][0]);
#if MULTIPASS_SIZE > MULTIPASS_SIZE_ZERO_CHECK
// For large blocks, loop over only the indices
// corresponding to non-zero a
for (uIndex i = 0; i < n_non_zero; i++) {
gradient_multipass[i_non_zero[i]] += multiplier*a[i_non_zero[i]];
}
#else
// For small blocks, do all indices
for (uIndex i = 0; i < block_size; i++) {
// for (uIndex i = 0; i < MULTIPASS_SIZE; i++) {
gradient_multipass[i] += multiplier*a[i];
}
#endif
}
}
} // End of loop over statement
// Copy the gradients corresponding to the independent
// variables into the Jacobian matrix
if (dep_offset == 1) {
for (uIndex iindep = 0; iindep < n_independent(); iindep++) {
for (uIndex i = 0; i < block_size; i++) {
jacobian_out[iindep*indep_offset+i_dependent+i]
= gradient_multipass_b[independent_index_[iindep]][i];
}
}
}
else {
for (uIndex iindep = 0; iindep < n_independent(); iindep++) {
for (uIndex i = 0; i < block_size; i++) {
jacobian_out[iindep*indep_offset+(i_dependent+i)*dep_offset]
= gradient_multipass_b[independent_index_[iindep]][i];
}
}
}
} // End of loop over blocks
} // end #pragma omp parallel
} // end jacobian_reverse_openmp
// Compute the Jacobian matrix; note that jacobian_out must be
// allocated to be of size m*n, where m is the number of dependent
// variables and n is the number of independents. The independents
// and dependents must have already been identified with the
// functions "independent" and "dependent", otherwise this function
// will fail with FAILURE_XXDEPENDENT_NOT_IDENTIFIED. This is
// implemented using a reverse pass, appropriate for m<n.
void
Stack::jacobian_reverse(Real* jacobian_out,
Index dep_offset, Index indep_offset) const
{
if (independent_index_.empty() || dependent_index_.empty()) {
throw(dependents_or_independents_not_identified());
}
// If either of the offsets are zero, set them to the size of the
// other dimension, which assumes that the full Jacobian matrix is
// contiguous in memory.
if (dep_offset <= 0) {
dep_offset = n_independent();
}
if (indep_offset <= 0) {
indep_offset = n_dependent();
}
#ifdef _OPENMP
if (have_openmp_
&& !openmp_manually_disabled_
&& n_dependent() > MULTIPASS_SIZE
&& omp_get_max_threads() > 1) {
// Call the parallel version
jacobian_reverse_openmp(jacobian_out,
dep_offset, indep_offset);
return;
}
#endif
// gradient_multipass_.resize(max_gradient_);
std::vector<Block<MULTIPASS_SIZE,Real> >
gradient_multipass_b(max_gradient_);
// For optimization reasons, we process a block of
// MULTIPASS_SIZE rows of the Jacobian at once; calculate
// how many blocks are needed and how many extras will remain
uIndex n_block = n_dependent() / MULTIPASS_SIZE;
uIndex n_extra = n_dependent() % MULTIPASS_SIZE;
uIndex i_dependent = 0; // uIndex of first row in the block we are
// currently computing
// Loop over the of MULTIPASS_SIZE rows
for (uIndex iblock = 0; iblock < n_block; iblock++) {
// Set the initial gradients all to zero
// zero_gradient_multipass();
for (std::size_t i = 0; i < gradient_multipass_b.size(); i++) {
gradient_multipass_b[i].zero();
}
// Each seed vector has one non-zero entry of 1.0
for (uIndex i = 0; i < MULTIPASS_SIZE; i++) {
gradient_multipass_b[dependent_index_[i_dependent+i]][i] = 1.0;
}
// Loop backward through the derivative statements
for (uIndex ist = n_statements_-1; ist > 0; ist--) {
const Statement& statement = statement_[ist];
// We copy the RHS to "a" in case it appears on the LHS in any
// of the following statements
Real a[MULTIPASS_SIZE];
#if MULTIPASS_SIZE > MULTIPASS_SIZE_ZERO_CHECK
// For large blocks, we only process the ones where a[i] is
// non-zero
uIndex i_non_zero[MULTIPASS_SIZE];
#endif
uIndex n_non_zero = 0;
for (uIndex i = 0; i < MULTIPASS_SIZE; i++) {
a[i] = gradient_multipass_b[statement.index][i];
gradient_multipass_b[statement.index][i] = 0.0;
if (a[i] != 0.0) {
#if MULTIPASS_SIZE > MULTIPASS_SIZE_ZERO_CHECK
i_non_zero[n_non_zero++] = i;
#else
n_non_zero = 1;
#endif
}
}
// Only do anything for this statement if any of the a values
// are non-zero
if (n_non_zero) {
// Loop through the operations
for (uIndex iop = statement_[ist-1].end_plus_one;
iop < statement.end_plus_one; iop++) {
// Try to minimize pointer dereferencing by making local
// copies
Real multiplier = multiplier_[iop];
Real* __restrict gradient_multipass
= &(gradient_multipass_b[index_[iop]][0]);
#if MULTIPASS_SIZE > MULTIPASS_SIZE_ZERO_CHECK
// For large blocks, loop over only the indices
// corresponding to non-zero a
for (uIndex i = 0; i < n_non_zero; i++) {
gradient_multipass[i_non_zero[i]] += multiplier*a[i_non_zero[i]];
}
#else
// For small blocks, do all indices
for (uIndex i = 0; i < MULTIPASS_SIZE; i++) {
gradient_multipass[i] += multiplier*a[i];
}
#endif
}
}
} // End of loop over statement
// Copy the gradients corresponding to the independent variables
// into the Jacobian matrix
if (dep_offset == 1) {
for (uIndex iindep = 0; iindep < n_independent(); iindep++) {
for (uIndex i = 0; i < MULTIPASS_SIZE; i++) {
jacobian_out[iindep*indep_offset+i_dependent+i]
= gradient_multipass_b[independent_index_[iindep]][i];
}
}
}
else {
for (uIndex iindep = 0; iindep < n_independent(); iindep++) {
for (uIndex i = 0; i < MULTIPASS_SIZE; i++) {
jacobian_out[iindep*indep_offset+(i_dependent+i)*dep_offset]
= gradient_multipass_b[independent_index_[iindep]][i];
}
}
}
i_dependent += MULTIPASS_SIZE;
} // End of loop over blocks
// Now do the same but for the remaining few rows in the matrix
if (n_extra > 0) {
for (std::size_t i = 0; i < gradient_multipass_b.size(); i++) {
gradient_multipass_b[i].zero();
}
// zero_gradient_multipass();
for (uIndex i = 0; i < n_extra; i++) {
gradient_multipass_b[dependent_index_[i_dependent+i]][i] = 1.0;
}
for (uIndex ist = n_statements_-1; ist > 0; ist--) {
const Statement& statement = statement_[ist];
Real a[MULTIPASS_SIZE];
#if MULTIPASS_SIZE > MULTIPASS_SIZE_ZERO_CHECK
uIndex i_non_zero[MULTIPASS_SIZE];
#endif
uIndex n_non_zero = 0;
for (uIndex i = 0; i < n_extra; i++) {
a[i] = gradient_multipass_b[statement.index][i];
gradient_multipass_b[statement.index][i] = 0.0;
if (a[i] != 0.0) {
#if MULTIPASS_SIZE > MULTIPASS_SIZE_ZERO_CHECK
i_non_zero[n_non_zero++] = i;
#else
n_non_zero = 1;
#endif
}
}
if (n_non_zero) {
for (uIndex iop = statement_[ist-1].end_plus_one;
iop < statement.end_plus_one; iop++) {
Real multiplier = multiplier_[iop];
Real* __restrict gradient_multipass
= &(gradient_multipass_b[index_[iop]][0]);
#if MULTIPASS_SIZE > MULTIPASS_SIZE_ZERO_CHECK
for (uIndex i = 0; i < n_non_zero; i++) {
gradient_multipass[i_non_zero[i]] += multiplier*a[i_non_zero[i]];
}
#else
for (uIndex i = 0; i < n_extra; i++) {
gradient_multipass[i] += multiplier*a[i];
}
#endif
}
}
}
if (dep_offset == 1) {
for (uIndex iindep = 0; iindep < n_independent(); iindep++) {
for (uIndex i = 0; i < n_extra; i++) {
jacobian_out[iindep*indep_offset+i_dependent+i]
= gradient_multipass_b[independent_index_[iindep]][i];
}
}
}
else {
for (uIndex iindep = 0; iindep < n_independent(); iindep++) {
for (uIndex i = 0; i < n_extra; i++) {
jacobian_out[iindep*indep_offset+(i_dependent+i)*dep_offset]
= gradient_multipass_b[independent_index_[iindep]][i];
}
}
}
}
}
// Return the Jacobian matrix in the matrix "jac", using the forward
// or reverse method depending which would be faster
void Stack::jacobian(Array<2,Real,false> jac) const {
if (jac.dimension(0) != n_dependent()
|| jac.dimension(1) != n_independent()) {
throw size_mismatch("Jacobian matrix has wrong size");
}
if (n_independent() <= n_dependent()) {
jacobian_forward(jac.data(), jac.offset(0), jac.offset(1));
}
else {
jacobian_reverse(jac.data(), jac.offset(0), jac.offset(1));
}
}
// Return the Jacobian matrix in the matrix "jac", explicitly
// specifying whether to use the forward or reverse method
void Stack::jacobian_forward(Array<2,Real,false> jac) const {
if (jac.dimension(0) != n_dependent()
|| jac.dimension(1) != n_independent()) {
throw size_mismatch("Jacobian matrix has wrong size");
}
jacobian_forward(jac.data(), jac.offset(0), jac.offset(1));
}
void Stack::jacobian_reverse(Array<2,Real,false> jac) const {
if (jac.dimension(0) != n_dependent()
|| jac.dimension(1) != n_independent()) {
throw size_mismatch("Jacobian matrix has wrong size");
}
jacobian_reverse(jac.data(), jac.offset(0), jac.offset(1));
}
// Return the Jacobian matrix using the forward or reverse method
// depending which would be faster
Array<2,Real,false> Stack::jacobian() const {
Array<2,Real,false> jac(n_dependent(), n_independent());
if (n_independent() <= n_dependent()) {
jacobian_forward(jac.data(), jac.offset(0), jac.offset(1));
}
else {
jacobian_reverse(jac.data(), jac.offset(0), jac.offset(1));
}
return jac;
}
// Return the Jacobian matrix, explicitly specifying whether to use
// the forward or reverse method
Array<2,Real,false> Stack::jacobian_forward() const {
Array<2,Real,false> jac(n_dependent(), n_independent());
jacobian_forward(jac.data(), jac.offset(0), jac.offset(1));
return jac;
}
Array<2,Real,false> Stack::jacobian_reverse() const {
Array<2,Real,false> jac(n_dependent(), n_independent());
jacobian_reverse(jac.data(), jac.offset(0), jac.offset(1));
return jac;
}
} // End namespace adept
================================================
FILE: adept/line_search.cpp
================================================
/* line_search.cpp -- Approximate minimization of function along a line
Copyright (C) 2020 European Centre for Medium-Range Weather Forecasts
Author: Robin Hogan <r.j.hogan@ecmwf.int>
This file is part of the Adept library.
*/
#include <limits>
#include <cmath>
#include <adept/Minimizer.h>
namespace adept {
// Compute the cost function "cf" and gradient vector "gradient",
// along with the scalar gradient "grad" in the search direction
// "direction" (normalized with "dir_scaling"), from the state
// vector "x" plus a step "step_size" in the search direction. If
// the resulting cost function and gradient satisfy the Wolfe
// conditions for sufficient convergence, copy the new state vector
// to "x" and the step size to "final_step_size", and return
// MINIMIZER_STATUS_SUCCESS. Otherwise, return
// MINIMIZER_STATUS_NOT_YET_CONVERGED. Error conditions
// MINIMIZER_STATUS_INVALID_COST_FUNCTION and
// MINIMIZER_STATUS_INVALID_GRADIENT are also possible.
MinimizerStatus
Minimizer::line_search_gradient_check(
Optimizable& optimizable, // Object defining function to be minimized
Vector x, // Initial and returned state vector
const Vector& direction, // Un-normalized search direction
Vector test_x, // Test state vector (working memory)
Real& final_step_size, // Returned step size if converged
Vector gradient, // Gradient vector
int& state_up_to_date, // Is state up-to-date?
Real step_size, // Candidate step size
Real grad0, // Gradient in direction at start of line search
Real dir_scaling, // Scaling of direction vector
Real& cf, // Returned cost function
Real& grad, // Returned gradient in direction
Real curvature_coeff) // Factor by which gradient should reduce (0-1)
{
test_x = x + (step_size * dir_scaling) * direction;
cf = optimizable.calc_cost_function_gradient(test_x, gradient);
++n_samples_;
state_up_to_date = -1;
// Check cost function and gradient are finite
if (!std::isfinite(cf)) {
return MINIMIZER_STATUS_INVALID_COST_FUNCTION;
}
else if (any(!isfinite(gradient))) {
return MINIMIZER_STATUS_INVALID_GRADIENT;
}
// Calculate gradient in search direction
grad = dot_product(direction, gradient) * dir_scaling;
// Check Wolfe conditions
if (cf <= cost_function_ + armijo_coeff_*step_size*grad0 // Armijo condition
&& std::fabs(grad) <= -curvature_coeff*grad0) { // Curvature condition
x = test_x;
final_step_size = step_size;
cost_function_ = cf;
state_up_to_date = 1;
return MINIMIZER_STATUS_SUCCESS;
}
else {
return MINIMIZER_STATUS_NOT_YET_CONVERGED;
}
}
// Perform line search starting at state vector "x" with gradient
// vector "gradient", and initial step "step_size" in un-normalized
// direction "direction". Successful minimization of the function
// (according to Wolfe conditions) will lead to
// MINIMIZER_STATUS_SUCCESS being returned, the new state stored in
// "x", and if state_up_to_date >= 1 then the gradient stored in
// "gradient". Other possible return values are
// MINIMIZER_STATUS_FAILED_TO_CONVERGE and
// MINIMIZER_STATUS_DIRECTION_UPHILL if the initial direction points
// uphill, or MINIMIZER_STATUS_INVALID_COST_FUNCTION,
// MINIMIZER_STATUS_INVALID_GRADIENT or
// MINIMIZER_STATUS_BOUND_REACHED. First the minimum is bracketed,
// then a cubic polynomial is fitted to the values and gradients of
// the function at the two points in order to select the next test
// point.
MinimizerStatus
Minimizer::line_search(
Optimizable& optimizable, // Object defining function to be minimized
Vector x, // Initial and returned state vector
const Vector& direction, // Un-normalized search direction
Vector test_x, // Test state vector (working memory)
Real& step_size, // Initial and final step size
Vector gradient, // Initial and possibly final gradient
int& state_up_to_date, // 1 if gradient up-to-date, -1 otherwise
Real curvature_coeff, // Factor by which gradient should reduce (0-1)
Real bound_step_size) // Maximum step until bound is reached (-1 for no bound)
{
Real dir_scaling = 1.0 / norm2(direction);
// Numerical suffixes to variables indicate different locations
// along the line:
// 0 = initial point of line search, constant within this function
// 1 = point at which gradient has been calculated (initially the same as 0)
// 2 = test point
// 3 = test point
// Step sizes
const Real ss0 = 0.0;
Real ss1 = ss0;
Real ss2 = step_size;
Real ss3;
// Gradients in search direction
Real grad0 = dot_product(direction, gradient) * dir_scaling;
Real grad1 = grad0;
Real grad2, grad3;
// Cost function values
Real cf0 = cost_function_;
Real cf1 = cf0;
Real cf2, cf3;
int iterations_remaining = max_line_search_iterations_;
bool is_bound_step = (bound_step_size > 0.0);
bool at_bound = false;
if (grad0 >= 0.0) {
return MINIMIZER_STATUS_DIRECTION_UPHILL;
}
// Check initial step size is within bounds
if (max_step_size_ > 0.0 && ss2 > max_step_size_) {
ss2 = max_step_size_;
}
if (is_bound_step && ss2 >= bound_step_size) {
ss2 = bound_step_size;
at_bound = true;
}
// First step: bound the minimum
while (iterations_remaining > 0) {
MinimizerStatus status
= line_search_gradient_check(optimizable, x, direction, test_x,
step_size, gradient, state_up_to_date,
ss2, grad0, dir_scaling,
cf2, grad2, curvature_coeff);
if (status == MINIMIZER_STATUS_SUCCESS) {
if (at_bound) {
status = MINIMIZER_STATUS_BOUND_REACHED;
}
return status;
}
else if (status != MINIMIZER_STATUS_NOT_YET_CONVERGED) {
// Cost function or its gradient not finite: revert to
// previous step
step_size = cf1;
if (cf1 > 0.0) {
x += (ss1 * dir_scaling) * direction;
}
state_up_to_date = 0;
return status;
}
if (grad2 > 0.0 || cf2 >= cf1) {
// Positive gradient or cost function increase -> bounded
// between points 1 and 2
break;
}
else if (at_bound) {
// The cost function has been reduced but we are already at
// the maximum step size and the gradient points towards it:
// make this point the solution
x += (ss2 * dir_scaling) * direction;
step_size = ss2;
cost_function_ = cf2;
state_up_to_date = 1;
return MINIMIZER_STATUS_BOUND_REACHED;
}
else {
// Reduced cost function but not yet bounded -> look further
// ahead
Real new_step;
if (cf1 > cf2+grad2*(ss1-ss2)) {
// Positive curvature: fit a quadratic
Real curvature = 2.0*(cf1-cf2-grad2*(ss1-ss2))/((ss1-ss2)*(ss1-ss2));
new_step = ss2-grad2/curvature; // Newton's method
// Bounds on actual step size
new_step = std::max(ss1+1.1*(ss2-ss1), std::min(new_step, ss1+10.0*(ss2-ss1)));
if (max_step_size_ > 0.0 && new_step-ss2 > max_step_size_) {
new_step = ss2 + max_step_size_;
}
}
else {
// Cliff gets steeper... simply jump ahead a lot more
new_step = ss2 + 5.0*(ss2-ss1);
if (max_step_size_ > 0.0 && new_step-ss2 > max_step_size_) {
new_step = ss2 + max_step_size_;
}
}
ss1 = ss2;
cf1 = cf2;
grad1 = grad2;
ss2 = new_step;
if (is_bound_step && ss2 >= bound_step_size) {
ss2 = bound_step_size;
at_bound = true;
}
}
}
// Second step: reduce the bounds until we get sufficiently close
// to the minimum
while (iterations_remaining > 0) {
if (ss2 <= ss1) {
// Two points are identical!
if (cf1 < cf0) {
// Return value at point 1
x += (ss1 * dir_scaling) * direction;
step_size = ss1;
cost_function_ = cf1;
return MINIMIZER_STATUS_SUCCESS;
}
else {
// Cost function did not decrease at all
return MINIMIZER_STATUS_FAILED_TO_CONVERGE;
}
}
// Minimizer of cubic function
Real step_diff = ss2-ss1;
Real theta = (cf1-cf2) * 3.0 / step_diff + grad1 + grad2;
Real max_grad = std::max(std::fabs(theta),
std::max(std::fabs(grad1), std::fabs(grad2)));
Real scaled_theta = theta / max_grad;
Real gamma = max_grad * std::sqrt(scaled_theta*scaled_theta
- (grad1/max_grad) * (grad2/max_grad));
ss3 = ss1 + ((gamma - grad1 + theta) / (2.0*gamma + grad2 - grad1)) * step_diff;
// Bound the step size to be at least 5% away from each end
ss3 = std::max(0.95*ss1+0.05*ss2,
std::min(0.05*ss1+0.95*ss2, ss3));
MinimizerStatus status
= line_search_gradient_check(optimizable, x, direction, test_x,
step_size, gradient, state_up_to_date,
ss3, grad0, dir_scaling,
cf3, grad3, curvature_coeff);
if (status == MINIMIZER_STATUS_SUCCESS) {
return status;
}
else if (status != MINIMIZER_STATUS_NOT_YET_CONVERGED) {
// Cost function or its gradient not finite: revert to
// previous step
step_size = cf1;
if (cf1 > 0.0) {
x += (ss1 * dir_scaling) * direction;
}
state_up_to_date = 0;
return status;
}
if (grad3 > 0.0) {
// Positive gradient -> bounded between points 1 and 3
ss2 = ss3;
cf2 = cf3;
grad2 = grad3;
}
else if (cf3 < cf1) {
// Reduced cost function, negative gradient
ss1 = ss3;
cf1 = cf3;
grad1 = grad3;
}
else {
// Increased cost function, negative gradient
ss2 = ss3;
cf2 = cf3;
grad2 = grad3;
}
--iterations_remaining;
}
// Maximum iterations reached: check if cost function has been
// reduced at all
state_up_to_date = -1;
if (cf2 < cf1) {
// Return value at point 2
x += (ss2 * dir_scaling) * direction;
step_size = ss2;
cost_function_ = cf2;
}
else if (cf1 < cf0) {
// Return value at point 1
x += (ss1 * dir_scaling) * direction;
step_size = ss1;
cost_function_ = cf1;
}
else {
// Cost function did not decrease at all
return MINIMIZER_STATUS_FAILED_TO_CONVERGE;
}
// Cost function decreased
return MINIMIZER_STATUS_SUCCESS;
}
}
================================================
FILE: adept/minimize_conjugate_gradient.cpp
================================================
/* minimize_conjugate_gradient.cpp -- Minimize function using Conjugate Gradient algorithm
Copyright (C) 2020 European Centre for Medium-Range Weather Forecasts
Author: Robin Hogan <r.j.hogan@ecmwf.int>
This file is part of the Adept library.
*/
#include <limits>
#include <cmath>
#include <adept/Minimizer.h>
namespace adept {
// Minimize the cost function embodied in "optimizable" using the
// Conjugate-Gradient algorithm, where "x" is the initial state
// vector and also where the solution is stored. By default the
// Polak-Ribiere method is used to compute the new search direction,
// but Fletcher-Reeves is also available.
MinimizerStatus
Minimizer::minimize_conjugate_gradient(Optimizable& optimizable, Vector x,
bool use_fletcher_reeves)
{
int nx = x.size();
// Initial values
n_iterations_ = 0;
n_samples_ = 0;
status_ = MINIMIZER_STATUS_NOT_YET_CONVERGED;
cost_function_ = std::numeric_limits<Real>::infinity();
// The Conjugate-Gradient method is the most efficient
// gradient-based method in terms of memory usage, requiring a
// working memory of just 4*nx, making it suitable for large state
// vectors.
Vector gradient(nx);
Vector previous_gradient(nx);
Vector direction(nx);
Vector test_x(nx); // Used by the line search only
// Does the last calculation of the cost function in "optimizable"
// match the current contents of the state vector x? -1=no, 0=yes,
// 1=yes and the last calculation included the gradient, 2=yes and
// the last calculation included gradient and Hessian.
int state_up_to_date = -1;
// Initial step size
Real step_size = 1.0;
if (max_step_size_ > 0.0) {
step_size = max_step_size_;
}
// A restart is performed every nx+1 iterations
bool do_restart = true;
int iteration_at_last_restart = n_iterations_;
// Main loop
while (status_ == MINIMIZER_STATUS_NOT_YET_CONVERGED) {
// If the last line search found a minimum along the lines
// satisfying the Wolfe conditions, then the current cost
// function and gradient will be consistent with the current
// state vector. Otherwise we need to compute them.
if (state_up_to_date < 1) {
cost_function_ = optimizable.calc_cost_function_gradient(x, gradient);
state_up_to_date = 1;
++n_samples_;
}
if (n_iterations_ == 0) {
start_cost_function_ = cost_function_;
}
// Check cost function and gradient are finite
if (!std::isfinite(cost_function_)) {
status_ = MINIMIZER_STATUS_INVALID_COST_FUNCTION;
break;
}
else if (any(!isfinite(gradient))) {
status_ = MINIMIZER_STATUS_INVALID_GRADIENT;
break;
}
// Compute L2 norm of gradient to see how "flat" the environment
// is
gradient_norm_ = norm2(gradient);
// Report progress using user-defined function
optimizable.report_progress(n_iterations_, x, cost_function_, gradient_norm_);
// Convergence has been achieved if the L2 norm has been reduced
// to a user-specified threshold
if (gradient_norm_ <= converged_gradient_norm_) {
status_ = MINIMIZER_STATUS_SUCCESS;
break;
}
// Restart every nx+1 iterations
if (n_iterations_ - iteration_at_last_restart > nx) {
do_restart = true;
}
// Find search direction
if (do_restart) {
// Simple gradient descent after a restart
direction = -gradient;
do_restart = false;
iteration_at_last_restart = n_iterations_;
}
else {
// The brains of the Conjugate-Gradient method - note that
// generally the Polak-Ribiere method is believed to be
// superior to Fletcher-Reeves
Real beta;
if (use_fletcher_reeves) {
// Fletcher-Reeves method
beta = dot_product(gradient, gradient)
/ dot_product(previous_gradient, previous_gradient);
}
else {
// Default: Polak-Ribiere method
beta = std::max(sum(gradient * (gradient - previous_gradient))
/ dot_product(previous_gradient, previous_gradient),
0.0);
}
// beta==0 is equivalent to gradient descent (i.e. a restart)
if (beta <= 0) {
iteration_at_last_restart = n_iterations_;
}
// Compute new direction
direction = beta*direction - gradient;
}
// Store gradient for computing beta in next iteration
previous_gradient = gradient;
// Perform line search, storing new state vector in x
MinimizerStatus ls_status
= line_search(optimizable, x, direction,
test_x, step_size, gradient, state_up_to_date,
cg_curvature_coeff_);
if (ls_status == MINIMIZER_STATUS_SUCCESS) {
// Successfully minimized along search direction: continue to
// next iteration
status_ = MINIMIZER_STATUS_NOT_YET_CONVERGED;
}
else if (iteration_at_last_restart != n_iterations_) {
// Line search either made no progress or encountered a
// non-finite cost function or gradient, and this was not a
// restart; try restarting once
do_restart = true;
status_ = MINIMIZER_STATUS_NOT_YET_CONVERGED;
}
else {
// Unrecoverable failure in line-search: return status to
// calling function
status_ = ls_status;
}
// Better convergence if first step size on next line search is
// larger than the actual step size on the last line search
step_size *= 2.0;
++n_iterations_;
if (status_ == MINIMIZER_STATUS_NOT_YET_CONVERGED
&& n_iterations_ >= max_iterations_) {
status_ = MINIMIZER_STATUS_MAX_ITERATIONS_REACHED;
}
// End of main loop: if status_ is anything other than
// MINIMIZER_STATUS_NOT_YET_CONVERGED then no more iterations
// are performed
}
if (state_up_to_date < ensure_updated_state_) {
// The last call to calc_cost_function* was not with the state
// vector returned to the user, and they want it to be.
if (ensure_updated_state_ > 0) {
// User wants at least the first derivative
cost_function_ = optimizable.calc_cost_function_gradient(x, gradient);
}
else {
// User does not need derivatives to have been computed
cost_function_ = optimizable.calc_cost_function(x);
}
}
return status_;
}
// Minimize the cost function embodied in "optimizable" using the
// Conjugate-Gradient algorithm, where "x" is the initial state
// vector and also where the solution is stored, subject to the
// constraint that x lies between min_x and max_x. By default the
// Polak-Ribiere method is used to compute the new search direction,
// but Fletcher-Reeves is also available.
MinimizerStatus
Minimizer::minimize_conjugate_gradient_bounded(Optimizable& optimizable, Vector x,
const Vector& min_x,
const Vector& max_x,
bool use_fletcher_reeves)
{
if (any(min_x >= max_x)
|| min_x.size() != x.size()
|| max_x.size() != x.size()) {
return MINIMIZER_STATUS_INVALID_BOUNDS;
}
int nx = x.size();
// Initial values
n_iterations_ = 0;
n_samples_ = 0;
status_ = MINIMIZER_STATUS_NOT_YET_CONVERGED;
cost_function_ = std::numeric_limits<Real>::infinity();
// The Conjugate-Gradient method is the most efficient
// gradient-based method in terms of memory usage, requiring a
// working memory of just 4*nx, making it suitable for large state
// vectors.
Vector gradient(nx);
Vector previous_gradient(nx);
Vector direction(nx);
Vector test_x(nx); // Used by the line search only
// Which state variables are at the minimum bound (-1), maximum
// bound (1) or free (0)?
intVector bound_status(nx);
bound_status = 0;
// Ensure that initial x lies within the specified bounds
bound_status.where(x >= max_x) = 1;
bound_status.where(x <= min_x) = -1;
x = max(min_x, min(x, max_x));
int nbound = count(bound_status != 0);
int nfree = nx - nbound;
// Floating-point number containing 1.0 if unbound and 0.0 if
// bound
Vector unbound_status(nx);
unbound_status = 1.0-fabs(bound_status);
// Does the last calculation of the cost function in "optimizable"
// match the current contents of the state vector x? -1=no, 0=yes,
// 1=yes and the last calculation included the gradient, 2=yes and
// the last calculation included gradient and Hessian.
int state_up_to_date = -1;
// Initial step size
Real step_size = 1.0;
if (max_step_size_ > 0.0) {
step_size = max_step_size_;
}
// A restart is performed every nx+1 iterations
bool do_restart = true;
int iteration_at_last_restart = n_iterations_;
// Main loop
while (status_ == MINIMIZER_STATUS_NOT_YET_CONVERGED) {
// If the last line search found a minimum along the lines
// satisfying the Wolfe conditions, then the current cost
// function and gradient will be consistent with the current
// state vector. Otherwise we need to compute them.
if (state_up_to_date < 1) {
cost_function_ = optimizable.calc_cost_function_gradient(x, gradient);
state_up_to_date = 1;
++n_samples_;
if (n_iterations_ == 0) {
start_cost_function_ = cost_function_;
}
// Check cost function and gradient are finite
if (!std::isfinite(cost_function_)) {
status_ = MINIMIZER_STATUS_INVALID_COST_FUNCTION;
break;
}
else if (any(!isfinite(gradient))) {
status_ = MINIMIZER_STATUS_INVALID_GRADIENT;
break;
}
}
// Check whether the bound status of each state variable is
// consistent with the gradient if a steepest descent were to be
// taken, and if not flag a restart
if (any(bound_status == -1 && gradient < 0.0)
|| any(bound_status == 1 && gradient > 0.0)) {
bound_status.where(bound_status == -1 && gradient < 0.0) = 0;
bound_status.where(bound_status == 1 && gradient > 0.0) = 0;
unbound_status = 1.0-fabs(bound_status);
do_restart = true;
}
nbound = count(bound_status != 0);
nfree = nx - nbound;
// Set gradient at bound points to zero
gradient.where(bound_status != 0) = 0.0;
// Compute L2 norm of gradient to see how "flat" the environment
// is
if (nfree > 0) {
gradient_norm_ = norm2(gradient);
}
else {
// If no dimensions are in play we are at a corner of the
// bounds and the gradient is pointing into the corner: we
// have reached a minimum in the cost function subject to the
// bounds so have converged
gradient_norm_ = 0.0;
}
// Report progress using user-defined function
optimizable.report_progress(n_iterations_, x, cost_function_, gradient_norm_);
// Convergence has been achieved if the L2 norm has been reduced
// to a user-specified threshold
if (gradient_norm_ <= converged_gradient_norm_) {
status_ = MINIMIZER_STATUS_SUCCESS;
break;
}
// Restart every nx+1 iterations
if (n_iterations_ - iteration_at_last_restart > nx) {
do_restart = true;
}
// Find search direction
if (do_restart) {
// Simple gradient descent after a restart
direction = -gradient;
do_restart = false;
iteration_at_last_restart = n_iterations_;
}
else {
// The brains of the Conjugate-Gradient method - note that
// generally the Polak-Ribiere method is believed to be
// superior to Fletcher-Reeves
Real beta;
if (use_fletcher_reeves) {
// Fletcher-Reeves method
beta = dot_product(gradient, gradient)
/ dot_product(previous_gradient, previous_gradient);
}
else {
// Default: Polak-Ribiere method
beta = std::max(sum(gradient * (gradient - previous_gradient))
/ dot_product(previous_gradient, previous_gradient),
0.0);
}
// beta==0 is equivalent to gradient descent (i.e. a restart)
if (beta <= 0) {
iteration_at_last_restart = n_iterations_;
}
// Compute new direction
direction = beta*direction - gradient;
}
// Store gradient for computing beta in next iteration
previous_gradient = gradient;
// Distance to the nearest bound
Real dir_scaling = norm2(direction);
Real bound_step_size = std::numeric_limits<Real>::max();
int i_nearest_bound = -1;
int i_bound_type = 0;
// Work out the maximum step size along "direction" before a
// bound is met... there must be a faster way to do this
for (int ix = 0; ix < nx; ++ix) {
if (direction(ix) > 0.0 && max_x(ix) < std::numeric_limits<Real>::max()) {
Real local_bound_step_size = dir_scaling*(max_x(ix)-x(ix))/direction(ix);
if (bound_step_size >= local_bound_step_size) {
bound_step_size = local_bound_step_size;
i_nearest_bound = ix;
i_bound_type = 1;
}
}
else if (direction(ix) < 0.0 && min_x(ix) > -std::numeric_limits<Real>::max()) {
Real local_bound_step_size = dir_scaling*(min_x(ix)-x(ix))/direction(ix);
if (bound_step_size >= local_bound_step_size) {
bound_step_size = local_bound_step_size;
i_nearest_bound = ix;
i_bound_type = -1;
}
}
}
MinimizerStatus ls_status; // line-search outcome
if (i_nearest_bound >= 0) {
// Perform line search, storing new state vector in x
ls_status = line_search(optimizable, x, direction,
test_x, step_size, gradient, state_up_to_date,
cg_curvature_coeff_, bound_step_size);
if (ls_status == MINIMIZER_STATUS_BOUND_REACHED) {
bound_status(i_nearest_bound) = i_bound_type;
do_restart = true;
ls_status = MINIMIZER_STATUS_SUCCESS;
}
}
else {
// Perform line search, storing new state vector in x
ls_status = line_search(optimizable, x, direction,
test_x, step_size, gradient, state_up_to_date,
cg_curvature_coeff_);
}
if (ls_status == MINIMIZER_STATUS_SUCCESS) {
// Successfully minimized along search direction: continue to
// next iteration
status_ = MINIMIZER_STATUS_NOT_YET_CONVERGED;
}
else if (iteration_at_last_restart != n_iterations_) {
// Line search either made no progress or encountered a
// non-finite cost function or gradient, and this was not a
// restart; try restarting once
do_restart = true;
status_ = MINIMIZER_STATUS_NOT_YET_CONVERGED;
}
else {
// Unrecoverable failure in line-search: return status to
// calling function
status_ = ls_status;
}
// Better convergence if first step size on next line search is
// larger than the actual step size on the last line search
step_size *= 2.0;
++n_iterations_;
if (status_ == MINIMIZER_STATUS_NOT_YET_CONVERGED
&& n_iterations_ >= max_iterations_) {
status_ = MINIMIZER_STATUS_MAX_ITERATIONS_REACHED;
}
// End of main loop: if status_ is anything other than
// MINIMIZER_STATUS_NOT_YET_CONVERGED then no more iterations
// are performed
}
if (state_up_to_date < ensure_updated_state_) {
// The last call to calc_cost_function* was not with the state
// vector returned to the user, and they want it to be.
if (ensure_updated_state_ > 0) {
// User wants at least the first derivative
cost_function_ = optimizable.calc_cost_function_gradient(x, gradient);
}
else {
// User does not need derivatives to have been computed
cost_function_ = optimizable.calc_cost_function(x);
}
}
return status_;
}
};
================================================
FILE: adept/minimize_levenberg_marquardt.cpp
================================================
/* minimize_levenberg_marquardt.cpp -- Minimize function using Levenberg-Marquardt algorithm
Copyright (C) 2020 European Centre for Medium-Range Weather Forecasts
Author: Robin Hogan <r.j.hogan@ecmwf.int>
This file is part of the Adept library.
*/
#include <limits>
#include <cmath>
#include <adept/Minimizer.h>
namespace adept {
// Minimize the cost function embodied in "optimizable" using the
// Levenberg-Marquardt algorithm, where "x" is the initial state
// vector and also where the solution is stored.
MinimizerStatus
Minimizer::minimize_levenberg_marquardt(Optimizable& optimizable, Vector x,
bool use_additive_damping)
{
int nx = x.size();
// Initial values
n_iterations_ = 0;
n_samples_ = 0;
status_ = MINIMIZER_STATUS_NOT_YET_CONVERGED;
cost_function_ = std::numeric_limits<Real>::infinity();
Real new_cost;
// The main memory storage for the Levenberg family of methods
// consists of the following three vectors...
Vector new_x(nx);
Vector gradient(nx);
Vector dx(nx);
// ...and the Hessian matrix, which is stored explicitly
SymmMatrix hessian(nx);
hessian = 0.0;
Real damping = levenberg_damping_start_;
gradient_norm_ = -1.0;
// Original Levenberg is additive to the diagonal of the Hessian
// so to make the performance insensitive to an arbitrary scaling
// of the cost function, we scale the damping factor by the mean
// of the diagonal of the Hessian
Real diag_scaling;
// Does the last calculation of the cost function in "optimizable"
// match the current contents of the state vector x? -1=no, 0=yes,
// 1=yes and the last calculation included the gradient, 2=yes and
// the last calculation included gradient and Hessian.
int state_up_to_date = -1;
do {
// At this point we have either just started or have just
// reduced the cost function
cost_function_ = optimizable.calc_cost_function_gradient_hessian(x, gradient, hessian);
diag_scaling = mean(hessian.diag_vector());
state_up_to_date = 2;
++n_samples_;
if (n_iterations_ == 0) {
start_cost_function_ = cost_function_;
}
// Check cost function and gradient are finite
if (!std::isfinite(cost_function_)) {
status_ = MINIMIZER_STATUS_INVALID_COST_FUNCTION;
break;
}
else if (any(!isfinite(gradient))) {
status_ = MINIMIZER_STATUS_INVALID_GRADIENT;
break;
}
// Compute L2 norm of gradient to see how "flat" the environment
// is
gradient_norm_ = norm2(gradient);
// Report progress using user-defined function
optimizable.report_progress(n_iterations_, x, cost_function_, gradient_norm_);
// Convergence has been achieved if the L2 norm has been reduced
// to a user-specified threshold
if (gradient_norm_ <= converged_gradient_norm_) {
status_ = MINIMIZER_STATUS_SUCCESS;
break;
}
// Try to minimize cost function
Real previous_diag_scaling = 1.0; // Used in Levenberg-Marquardt version
Real previous_diag_modifier = 0.0; // Used in Levenberg version
while(true) {
if (!use_additive_damping) {
// Levenberg-Marquardt formula: scale the diagonal of the
// Hessian, where the larger the value of "damping", the
// closer the resulting behaviour is to steepest descent
hessian.diag_vector() *= (1.0 + damping)/previous_diag_scaling;
previous_diag_scaling = 1.0 + damping;
}
else {
// Older Levenberg approach: add to the diagonal instead
hessian.diag_vector() += damping*diag_scaling - previous_diag_modifier;
previous_diag_modifier = damping*diag_scaling;
}
dx = -adept::solve(hessian, gradient);
// Limit the maximum step size, if required
if (max_step_size_ > 0.0) {
Real max_dx = maxval(abs(dx));
if (max_dx > max_step_size_) {
dx *= (max_step_size_/max_dx);
}
}
// Compute new cost state vector and cost function, but not
// gradient or Hessian for efficiency
new_x = x+dx;
new_cost = optimizable.calc_cost_function(new_x);
state_up_to_date = -1;
++n_samples_;
// If cost function is not finite it may be possible to
// recover by trying smaller step sizes
bool cost_invalid = !std::isfinite(new_cost);
if (new_cost >= cost_function_ || cost_invalid) {
// We haven't managed to reduce the cost function: increase
// damping value to take smaller steps
if (damping <= 0.0) {
damping = levenberg_damping_restart_;
}
else if (damping < levenberg_damping_max_) {
damping *= levenberg_damping_multiplier_;
}
else {
// The damping value is now larger than the maximum so we
// can get no further
if (cost_invalid) {
status_ = MINIMIZER_STATUS_INVALID_COST_FUNCTION;
}
else {
status_ = MINIMIZER_STATUS_FAILED_TO_CONVERGE;
}
break;
}
}
else {
// Managed to reduce cost function
x = new_x;
n_iterations_++;
// Reduce damping for next iteration
if (damping > levenberg_damping_min_) {
damping /= levenberg_damping_divider_;
}
else {
damping = 0.0;
}
if (n_iterations_ >= max_iterations_) {
status_ = MINIMIZER_STATUS_MAX_ITERATIONS_REACHED;
}
break;
}
} // Inner loop
}
while (status_ == MINIMIZER_STATUS_NOT_YET_CONVERGED);
if (state_up_to_date < ensure_updated_state_) {
// The last call to calc_cost_function* was not with the state
// vector returned to the user, and they want it to be. Note
// that the cost function and gradient norm ought to be
// up-to-date already at this point.
if (ensure_updated_state_ > 0) {
// User wants at least the first derivative, but
// calc_cost_function_gradient() is not guaranteed to be
// present so we call the hessain function
cost_function_ = optimizable.calc_cost_function_gradient_hessian(x, gradient,
hessian);
}
else {
// User does not need derivatives to have been computed
cost_function_ = optimizable.calc_cost_function(x);
}
}
return status_;
}
// Minimize the cost function embodied in "optimizable" using the
// Levenberg-Marquardt algorithm, where "x" is the initial state
// vector and also where the solution is stored, subject to the
// constraint that x lies between min_x and max_x.
MinimizerStatus
Minimizer::minimize_levenberg_marquardt_bounded(Optimizable& optimizable,
Vector x,
const Vector& min_x,
const Vector& max_x,
bool use_additive_damping)
{
if (any(min_x >= max_x)
|| min_x.size() != x.size()
|| max_x.size() != x.size()) {
return MINIMIZER_STATUS_INVALID_BOUNDS;
}
int nx = x.size();
// Initial values
n_iterations_ = 0;
n_samples_ = 0;
status_ = MINIMIZER_STATUS_NOT_YET_CONVERGED;
cost_function_ = std::numeric_limits<Real>::infinity();
Real new_cost;
// The main memory storage for the Levenberg family of methods
// consists of the following three vectors...
Vector new_x(nx);
Vector gradient(nx);
Vector dx(nx);
// ...and the Hessian matrix, which is stored explicitly
SymmMatrix hessian(nx);
SymmMatrix modified_hessian(nx);
SymmMatrix sub_hessian;
Vector sub_gradient;
Vector sub_dx;
hessian = 0.0;
Real damping = levenberg_damping_start_;
// Which state variables are at the minimum bound (-1), maximum
// bound (1) or free (0)?
intVector bound_status(nx);
bound_status = 0;
// Ensure that initial x lies within the specified bounds
bound_status.where(x >= max_x) = 1;
bound_status.where(x <= min_x) = -1;
x = max(min_x, min(x, max_x));
int nbound = count(bound_status != 0);
int nfree = nx - nbound;
gradient_norm_ = -1.0;
// Original Levenberg is additive to the diagonal of the Hessian
// so to make the performance insensitive to an arbitrary scaling
// of the cost function, we scale the damping factor by the mean
// of the diagonal of the Hessian
Real diag_scaling;
// Does the last calculation of the cost function in "optimizable"
// match the current contents of the state vector x? -1=no, 0=yes,
// 1=yes and the last calculation included the gradient, 2=yes and
// the last calculation included gradient and Hessian.
int state_up_to_date = -1;
do {
// At this point we have either just started or have just
// reduced the cost function
cost_function_ = optimizable.calc_cost_function_gradient_hessian(x, gradient, hessian);
diag_scaling = mean(hessian.diag_vector());
state_up_to_date = 2;
++n_samples_;
if (n_iterations_ == 0) {
start_cost_function_ = cost_function_;
}
// Check cost function and gradient are finite
if (!std::isfinite(cost_function_)) {
status_ = MINIMIZER_STATUS_INVALID_COST_FUNCTION;
break;
}
else if (any(!isfinite(gradient))) {
status_ = MINIMIZER_STATUS_INVALID_GRADIENT;
break;
}
// Find which dimensions are in play
if (nbound > 0) {
// We release any dimensions from being at a minimum or
// maximum bound if two conditions are met: (1) the gradient
// in that dimension slopes away from the bound, and (2) the
// Levenberg-Marquardt formula to compute dx using the current
// value of "damping" leads to a point on the valid side of the
// bound
modified_hessian = hessian;
if (!use_additive_damping) {
modified_hessian.diag_vector() *= (1.0 + damping);
}
else {
modified_hessian.diag_vector() += damping*diag_scaling;
}
dx = -adept::solve(modified_hessian, gradient);
// Release points at the minimum bound
bound_status.where(bound_status == -1
&& gradient < 0.0
&& dx > 0.0) = 0;
// Release points at the maximum bound
bound_status.where(bound_status == 1
&& gradient > 0.0
&& dx < 0.0) = 0;
}
nbound = count(bound_status != 0);
nfree = nx - nbound;
// List of indices of free state variables
intVector ifree(nfree);
if (nbound > 0) {
ifree = find(bound_status == 0);
}
else {
ifree = range(0, nx-1);
}
// Compute L2 norm of gradient to see how "flat" the environment
// is, restricting ourselves to the dimensions currently in play
if (nfree > 0) {
gradient_norm_ = norm2(gradient(ifree));
}
else {
// If no dimensions are in play we are at a corner of the
// bounds and the gradient is pointing into the corner: we
// have reached a minimum in the cost function subject to the
// bounds so have converged
gradient_norm_ = 0.0;
}
// Report progress using user-defined function
optimizable.report_progress(n_iterations_, x, cost_function_, gradient_norm_);
// Convergence has been achieved if the L2 norm has been reduced
// to a user-specified threshold
if (gradient_norm_ <= converged_gradient_norm_) {
status_ = MINIMIZER_STATUS_SUCCESS;
break;
}
sub_gradient.clear();
sub_hessian.clear();
if (nbound > 0) {
sub_gradient = gradient(ifree);
sub_hessian = SymmMatrix(Matrix(hessian)(ifree,ifree));
}
else {
sub_gradient >>= gradient;
sub_hessian >>= hessian;
}
// FIX reuse dx if possible below...
// Try to minimize cost function
Real previous_diag_scaling = 1.0; // Used in Levenberg-Marquardt version
Real previous_diag_modifier = 0.0; // Used in Levenberg version
while(true) {
sub_dx.resize(nfree);
if (!use_additive_damping) {
// Levenberg-Marquardt formula: scale the diagonal of the
// Hessian, where the larger the value of "damping", the
// closer the resulting behaviour is to steepest descent
sub_hessian.diag_vector() *= (1.0 + damping)/previous_diag_scaling;
previous_diag_scaling = 1.0 + damping;
}
else {
// Older Levenberg approach: add to the diagonal instead
sub_hessian.diag_vector() += damping*diag_scaling - previous_diag_modifier;
previous_diag_modifier = damping*diag_scaling;
}
sub_dx = -adept::solve(sub_hessian, sub_gradient);
// Limit the maximum step size, if required
if (max_step_size_ > 0.0) {
Real max_dx = maxval(abs(sub_dx));
if (max_dx > max_step_size_) {
sub_dx *= (max_step_size_/max_dx);
}
}
// Check for collision with new bounds
intVector new_min_bounds = find(x(ifree)+sub_dx <= min_x(ifree));
intVector new_max_bounds = find(x(ifree)+sub_dx >= max_x(ifree));
Real mmin_frac = 2.0;
Real mmax_frac = 2.0;
int imin = 0, imax = 0;
if (!new_min_bounds.empty()) {
Vector min_frac = -(x(ifree(new_min_bounds)) - min_x(ifree(new_min_bounds)))
/ sub_dx(new_min_bounds);
mmin_frac = minval(min_frac);
imin = new_min_bounds(minloc(min_frac));
}
if (!new_max_bounds.empty()) {
Vector max_frac = (max_x(ifree(new_max_bounds)) - x(ifree(new_max_bounds)))
/ sub_dx(new_max_bounds);
mmax_frac = minval(max_frac);
imax = new_max_bounds(maxloc(max_frac));
}
Real frac = 1.0;
int bound_type = 0;
int ibound = 0;
if (mmin_frac <= 1.0 || mmax_frac <= 1.0) {
if (mmin_frac < mmax_frac) {
frac = mmin_frac;
ibound = imin;
bound_type = -1;
}
else {
frac = mmax_frac;
ibound = imax;
bound_type = 1;
}
sub_dx *= frac;
}
// Compute new state vector and cost function, but not
// gradient or Hessian for efficiency
new_x = x;
new_x(ifree) += sub_dx;
new_cost = optimizable.calc_cost_function(new_x);
state_up_to_date = -1;
++n_samples_;
// If cost function is not finite it may be possible to
// recover by trying smaller step sizes
bool cost_invalid = !std::isfinite(new_cost);
if (new_cost >= cost_function_ || cost_invalid) {
// We haven't managed to reduce the cost function: increase
// damping value to take smaller steps
if (damping <= 0.0) {
damping = levenberg_damping_restart_;
}
else if (damping < levenberg_damping_max_) {
damping *= levenberg_damping_multiplier_;
}
else {
// The damping value is now larger than the maximum so we
// can get no further
if (cost_invalid) {
status_ = MINIMIZER_STATUS_INVALID_COST_FUNCTION;
}
else {
status_ = MINIMIZER_STATUS_FAILED_TO_CONVERGE;
}
break;
}
}
else {
// Managed to reduce cost function
x = new_x;
n_iterations_++;
if (frac < 1.0) {
// Found a new bound
bound_status(ifree(ibound)) = bound_type;
}
// Reduce damping for next iteration
if (damping > levenberg_damping_min_) {
damping /= levenberg_damping_divider_;
}
else {
damping = 0.0;
}
if (n_iterations_ >= max_iterations_) {
status_ = MINIMIZER_STATUS_MAX_ITERATIONS_REACHED;
}
break;
}
} // Inner loop
}
while (status_ == MINIMIZER_STATUS_NOT_YET_CONVERGED);
if (state_up_to_date < ensure_updated_state_) {
// The last call to calc_cost_function* was not with the state
// vector returned to the user, and they want it to be. Note
// that the cost function and gradient norm ought to be
// up-to-date already at this point.
if (ensure_updated_state_ > 0) {
// User wants at least the first derivative, but
// calc_cost_function_gradient() is not guaranteed to be
// present so we call the hessain function
cost_function_ = optimizable.calc_cost_function_gradient_hessian(x, gradient,
hessian);
}
else {
// User does not need derivatives to have been computed
cost_function_ = optimizable.calc_cost_function(x);
}
}
return status_;
}
};
================================================
FILE: adept/minimize_limited_memory_bfgs.cpp
================================================
/* minimize_limited_memory_bfgs.cpp -- Minimize function using Limited-Memory BFGS algorithm
Copyright (C) 2020 European Centre for Medium-Range Weather Forecasts
Author: Robin Hogan <r.j.hogan@ecmwf.int>
This file is part of the Adept library.
*/
#include <limits>
#include <adept/Minimizer.h>
namespace adept {
// Structure for storing data from previous iterations used by
// L-BFGS minimization algorithm
class LbfgsData {
public:
LbfgsData(int nx, int ni)
: nx_(nx), ni_(ni), iteration_(0) {
x_diff_.resize(ni,nx);
gradient_diff_.resize(ni,nx);
rho_.resize(ni);
alpha_.resize(ni);
gamma_.resize(ni);
}
// Return false if the dot product of x_diff and gradient_diff is
// zero, true otherwise
void store(int iter, const Vector& x_diff, const Vector& gradient_diff) {
int index = (iter-1) % ni_;
x_diff_[index] = x_diff;
gradient_diff_[index] = gradient_diff;
Real dp = dot_product(x_diff, gradient_diff);
if (std::fabs(dp) > 10.0*std::numeric_limits<Real>::min()) {
rho_[index] = 1.0 / dp;
}
else if (dp >= 0.0) {
rho_[index] = 1.0 / std::max(dp, 10.0*std::numeric_limits<Real>::min());
}
else {
rho_[index] = 1.0 / std::min(dp, -10.0*std::numeric_limits<Real>::min());
}
}
// Return read-only vectors containing the differences between
// state vectors and gradients at sequential iterations, by
// slicing off the appropriate row of the matrix
Vector x_diff(int iter) {
return x_diff_[iter % ni_];
};
Vector gradient_diff(int iter) {
return gradient_diff_[iter % ni_];
};
Real& alpha(int iter) { return alpha_[iter % ni_]; }
Real rho(int iter) const { return rho_[iter % ni_]; }
Real gamma(int iter) const { return gamma_[iter % ni_]; }
private:
// Data
int nx_; // Number of state variables
int ni_; // Number of iterations to store
int iteration_; // Current iteration
Matrix x_diff_;
Matrix gradient_diff_;
Vector rho_;
Vector alpha_;
Vector gamma_;
};
// Minimize the cost function embodied in "optimizable" using the
// Limited-Memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS)
// algorithm, where "x" is the initial state vector and also where
// the solution is stored.
MinimizerStatus
Minimizer::minimize_limited_memory_bfgs(Optimizable& optimizable, Vector x)
{
int nx = x.size();
// Initial values
n_iterations_ = 0;
n_samples_ = 0;
status_ = MINIMIZER_STATUS_NOT_YET_CONVERGED;
cost_function_ = std::numeric_limits<Real>::infinity();
Vector previous_x(nx);
Vector gradient(nx);
Vector previous_gradient(nx);
Vector direction(nx);
Vector test_x(nx); // Used by the line search only
// Previous states needed by the L-BFGS algorithm
int n_states = std::min(nx, lbfgs_n_states_);
LbfgsData data(nx, n_states);
// Does the last calculation of the cost function in "optimizable"
// match the current contents of the state vector x? -1=no, 0=yes,
// 1=yes and the last calculation included the gradient, 2=yes and
// the last calculation included gradient and Hessian.
int state_up_to_date = -1;
// Initial step size
Real step_size = 1.0;
if (max_step_size_ > 0.0) {
step_size = max_step_size_;
}
// Main loop
while (status_ == MINIMIZER_STATUS_NOT_YET_CONVERGED) {
// If the last line search found a minimum along the lines
// satisfying the Wolfe conditions, then the current cost
// function and gradient will be consistent with the current
// state vector. Otherwise we need to compute them.
if (state_up_to_date < 1) {
cost_function_ = optimizable.calc_cost_function_gradient(x, gradient);
state_up_to_date = 1;
++n_samples_;
if (n_iterations_ == 0) {
start_cost_function_ = cost_function_;
}
// Check cost function and gradient are finite
if (!std::isfinite(cost_function_)) {
status_ = MINIMIZER_STATUS_INVALID_COST_FUNCTION;
break;
}
else if (any(!isfinite(gradient))) {
status_ = MINIMIZER_STATUS_INVALID_GRADIENT;
break;
}
}
// Check cost function and gradient are finite
if (!std::isfinite(cost_function_)) {
status_ = MINIMIZER_STATUS_INVALID_COST_FUNCTION;
break;
}
else if (any(!isfinite(gradient))) {
status_ = MINIMIZER_STATUS_INVALID_GRADIENT;
break;
}
// Compute L2 norm of gradient to see how "flat" the environment
// is
gradient_norm_ = norm2(gradient);
// Report progress using user-defined function
optimizable.report_progress(n_iterations_, x, cost_function_, gradient_norm_);
// Convergence has been achieved if the L2 norm has been reduced
// to a user-specified threshold
if (gradient_norm_ <= converged_gradient_norm_) {
status_ = MINIMIZER_STATUS_SUCCESS;
break;
}
// Store state and gradient differences
if (n_iterations_ > 0) {
data.store(n_iterations_, x-previous_x, gradient-previous_gradient);
}
// Find search direction: see page 779 of Nocedal (1980):
// Updating quasi-Newton matrices with limited
// storage. Mathematics of Computation, 35, 773-782.
direction = gradient;
if (n_iterations_ > 0) {
for (int ii = n_iterations_-1;
ii >= std::max(0,n_iterations_-n_states);
--ii) {
data.alpha(ii) = data.rho(ii)
* dot_product(data.x_diff(ii), direction);
direction -= data.alpha(ii) * data.gradient_diff(ii);
}
Real gamma = dot_product(x-previous_x, gradient-previous_gradient)
/ std::max(10.0*std::numeric_limits<Real>::min(),
dot_product(gradient-previous_gradient, gradient-previous_gradient));
direction *= gamma;
for (int ii = std::max(0,n_iterations_-n_states);
ii < n_iterations_;
++ii) {
Real beta = data.rho(ii) * dot_product(data.gradient_diff(ii), direction);
direction += data.x_diff(ii) * (data.alpha(ii)-beta);
}
direction = -direction;
}
else {
direction = -gradient * (step_size / norm2(gradient));
}
// Store state and gradient
previous_x = x;
previous_gradient = gradient;
// Perform line search, storing new state vector in x, and
// returning MINIMIZER_STATUS_NOT_YET_CONVERGED on success
Real curvature_coeff = lbfgs_curvature_coeff_;
if (n_iterations_ < n_states) {
// In the early iterations we require the line search to be
// more accurate since the L-BFGS update will have fewer
// states to make a good estimate of the minimum; interpolate
// between the Conjugate Gradient and L-BFGS curvature
// coefficients
curvature_coeff = (cg_curvature_coeff_ * (n_states-n_iterations_)
+ lbfgs_curvature_coeff_ * n_iterations_)
/ n_states;
}
// Direction points to the best estimate of the actual location
// of the minimum, so the step size is the norm of the direction
// vector
step_size = norm2(direction);
MinimizerStatus ls_status
= line_search(optimizable, x, direction,
test_x, step_size, gradient, state_up_to_date,
curvature_coeff);
if (ls_status == MINIMIZER_STATUS_SUCCESS) {
// Successfully minimized along search direction: continue to
// next iteration
status_ = MINIMIZER_STATUS_NOT_YET_CONVERGED;
}
else {
// Unrecoverable failure in line-search: return status to
// calling function
status_ = ls_status;
}
++n_iterations_;
if (status_ == MINIMIZER_STATUS_NOT_YET_CONVERGED
&& n_iterations_ >= max_iterations_) {
status_ = MINIMIZER_STATUS_MAX_ITERATIONS_REACHED;
}
// End of main loop: if status_ is anything other than
// MINIMIZER_STATUS_NOT_YET_CONVERGED then no more iterations
// are performed
}
if (state_up_to_date < ensure_updated_state_) {
// The last call to calc_cost_function* was not with the state
// vector returned to the user, and they want it to be.
if (ensure_updated_state_ > 0) {
// User wants at least the first derivative
cost_function_ = optimizable.calc_cost_function_gradient(x, gradient);
}
else {
// User does not need derivatives to have been computed
cost_function_ = optimizable.calc_cost_function(x);
}
}
return status_;
}
// Minimize the cost function embodied in "optimizable" using the
// Limited-Memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS)
// algorithm, where "x" is the initial state vector and also where
// the solution is stored.
MinimizerStatus
Minimizer::minimize_limited_memory_bfgs_bounded(Optimizable& optimizable, Vector x,
const Vector& min_x,
const Vector& max_x)
{
if (any(min_x >= max_x)
|| min_x.size() != x.size()
|| max_x.size() != x.size()) {
return MINIMIZER_STATUS_INVALID_BOUNDS;
}
int nx = x.size();
// Initial values
n_iterations_ = 0;
n_samples_ = 0;
status_ = MINIMIZER_STATUS_NOT_YET_CONVERGED;
cost_function_ = std::numeric_limits<Real>::infinity();
Vector previous_x(nx);
Vector gradient(nx);
Vector previous_gradient(nx);
Vector direction(nx);
Vector test_x(nx); // Used by the line search only
// Previous states needed by the L-BFGS algorithm
int n_states = std::min(nx, lbfgs_n_states_);
LbfgsData data(nx, n_states);
// Which state variables are at the minimum bound (-1), maximum
// bound (1) or free (0)?
intVector bound_status(nx);
bound_status = 0;
// Ensure that initial x lies within the specified bounds
bound_status.where(x >= max_x) = 1;
bound_status.where(x <= min_x) = -1;
x = max(min_x, min(x, max_x));
int nbound = count(bound_status != 0);
int nfree = nx - nbound;
// Floating-point number containing 1.0 if unbound and 0.0 if
// bound
Vector unbound_status(nx);
unbound_status = 1.0-fabs(bound_status);
// If we reach a bound we need to restart the L-BFGS storage, so
// store the iteration at the last restart
int iteration_last_restart = 0;
// Does the last calculation of the cost function in "optimizable"
// match the current contents of the state vector x? -1=no, 0=yes,
// 1=yes and the last calculation included the gradient, 2=yes and
// the last calculation included gradient and Hessian.
int state_up_to_date = -1;
// Initial step size
Real step_size = 1.0;
if (max_step_size_ > 0.0) {
step_size = max_step_size_;
}
// Main loop
while (status_ == MINIMIZER_STATUS_NOT_YET_CONVERGED) {
// If the last line search found a minimum along the lines
// satisfying the Wolfe conditions, then the current cost
// function and gradient will be consistent with the current
// state vector. Otherwise we need to compute them.
if (state_up_to_date < 1) {
cost_function_ = optimizable.calc_cost_function_gradient(x, gradient);
state_up_to_date = 1;
++n_samples_;
if (n_iterations_ == 0) {
start_cost_function_ = cost_function_;
}
// Check cost function and gradient are finite
if (!std::isfinite(cost_function_)) {
status_ = MINIMIZER_STATUS_INVALID_COST_FUNCTION;
break;
}
else if (any(!isfinite(gradient))) {
status_ = MINIMIZER_STATUS_INVALID_GRADIENT;
break;
}
}
// Check whether the bound status of each state variable is
// consistent with the gradient if a steepest descent were to be
// taken, and if not flag a restart
if (any(bound_status == -1 && gradient < 0.0)
|| any(bound_status == 1 && gradient > 0.0)) {
bound_status.where(bound_status == -1 && gradient < 0.0) = 0;
bound_status.where(bound_status == 1 && gradient > 0.0) = 0;
unbound_status = 1.0-fabs(bound_status);
iteration_last_restart = n_iterations_;
}
nbound = count(bound_status != 0);
nfree = nx - nbound;
// Set gradient at bound points to zero
gradient.where(bound_status != 0) = 0.0;
// Compute L2 norm of gradient to see how "flat" the environment
// is
if (nfree > 0) {
gradient_norm_ = norm2(gradient);
}
else {
// If no dimensions are in play we are at a corner of the
// bounds and the gradient is pointing into the corner: we
// have reached a minimum in the cost function subject to the
// bounds so have converged
gradient_norm_ = 0.0;
}
// Report progress using user-defined function
optimizable.report_progress(n_iterations_, x, cost_function_, gradient_norm_);
// Convergence has been achieved if the L2 norm has been reduced
// to a user-specified threshold
if (gradient_norm_ <= converged_gradient_norm_) {
status_ = MINIMIZER_STATUS_SUCCESS;
break;
}
// Store state and gradient differences
if (n_iterations_ > iteration_last_restart) {
data.store(n_iterations_, x-previous_x, gradient-previous_gradient);
}
// Find search direction: see page 779 of Nocedal (1980):
// Updating quasi-Newton matrices with limited
// storage. Mathematics of Computation, 35, 773-782.
direction = gradient;
if (n_iterations_ > iteration_last_restart) {
for (int ii = n_iterations_-1;
ii >= std::max(iteration_last_restart,n_iterations_-n_states);
--ii) {
data.alpha(ii) = data.rho(ii)
* dot_product(data.x_diff(ii), direction);
direction -= data.alpha(ii) * data.gradient_diff(ii);
}
Real gamma = dot_product(x-previous_x, gradient-previous_gradient)
/ std::max(10.0*std::numeric_limits<Real>::min(),
dot_product(gradient-previous_gradient, gradient-previous_gradient));
direction *= gamma;
for (int ii = std::max(iteration_last_restart,n_iterations_-n_states);
ii < n_iterations_;
++ii) {
Real beta = data.rho(ii) * dot_product(data.gradient_diff(ii), direction);
direction += data.x_diff(ii) * (data.alpha(ii)-beta);
}
direction = -direction;
}
else {
// We are either at the first iteration or have restarted
// having changed the bound dimensions: use steepest descent
direction = -gradient * (step_size / norm2(gradient));
}
// Store state and gradient
previous_x = x;
previous_gradient = gradient;
// Perform line search, storing new state vector in x, and
// returning MINIMIZER_STATUS_NOT_YET_CONVERGED on success
Real curvature_coeff = lbfgs_curvature_coeff_;
int n_stored_iterations = n_iterations_ - iteration_last_restart;
if (n_stored_iterations < n_states) {
// In the early iterations we require the line search to be
// more accurate since the L-BFGS update will have fewer
// states to make a good estimate of the minimum; interpolate
// between the Conjugate Gradient and L-BFGS curvature
// coefficients
curvature_coeff = (cg_curvature_coeff_ * (n_states-n_stored_iterations)
+ lbfgs_curvature_coeff_ * n_stored_iterations)
/ n_states;
}
// Direction points to the best estimate of the actual location
// of the minimum, so the step size is the norm of the direction
// vector
step_size = norm2(direction);
// Distance to the nearest bound
Real dir_scaling = step_size;
Real bound_step_size = std::numeric_limits<Real>::max();
int i_nearest_bound = -1;
int i_bound_type = 0;
// Work out the maximum step size along "direction" before a
// bound is met... there must be a faster way to do this
for (int ix = 0; ix < nx; ++ix) {
if (direction(ix) > 0.0 && max_x(ix) < std::numeric_limits<Real>::max()) {
Real local_bound_step_size = dir_scaling*(max_x(ix)-x(ix))/direction(ix);
if (bound_step_size >= local_bound_step_size) {
bound_step_size = local_bound_step_size;
i_nearest_bound = ix;
i_bound_type = 1;
}
}
else if (direction(ix) < 0.0 && min_x(ix) > -std::numeric_limits<Real>::max()) {
Real local_bound_step_size = dir_scaling*(min_x(ix)-x(ix))/direction(ix);
if (bound_step_size >= local_bound_step_size) {
bound_step_size = local_bound_step_size;
i_nearest_bound = ix;
i_bound_type = -1;
}
}
}
MinimizerStatus ls_status; // line-search outcome
if (i_nearest_bound >= 0) {
// Perform line search, storing new state vector in x
ls_status = line_search(optimizable, x, direction,
test_x, step_size, gradient, state_up_to_date,
curvature_coeff, bound_step_size);
if (ls_status == MINIMIZER_STATUS_BOUND_REACHED) {
bound_status(i_nearest_bound) = i_bound_type;
// Restart the L-BFGS storage
iteration_last_restart = n_iterations_+1;
ls_status = MINIMIZER_STATUS_SUCCESS;
}
}
else {
// Perform line search, storing new state vector in x
ls_status = line_search(optimizable, x, direction,
test_x, step_size, gradient, state_up_to_date,
curvature_coeff);
}
if (ls_status == MINIMIZER_STATUS_SUCCESS) {
// Successfully minimized along search direction: continue to
// next iteration
status_ = MINIMIZER_STATUS_NOT_YET_CONVERGED;
}
else {
// Unrecoverable failure in line-search: return status to
// calling function
status_ = ls_status;
}
++n_iterations_;
if (status_ == MINIMIZER_STATUS_NOT_YET_CONVERGED
&& n_iterations_ >= max_iterations_) {
status_ = MINIMIZER_STATUS_MAX_ITERATIONS_REACHED;
}
// End of main loop: if status_ is anything other than
// MINIMIZER_STATUS_NOT_YET_CONVERGED then no more iterations
// are performed
}
if (state_up_to_date < ensure_updated_state_) {
// The last call to calc_cost_function* was not with the state
// vector returned to the user, and they want it to be.
if (ensure_updated_state_ > 0) {
// User wants at least the first derivative
cost_function_ = optimizable.calc_cost_function_gradient(x, gradient);
}
else {
// User does not need derivatives to have been computed
cost_function_ = optimizable.calc_cost_function(x);
}
}
return status_;
}
};
================================================
FILE: adept/settings.cpp
================================================
/* settings.cpp -- View/change the overall Adept settings
Copyright (C) 2016 European Centre for Medium-Range Weather Forecasts
Author: Robin Hogan <r.j.hogan@ecmwf.int>
This file is part of the Adept library.
*/
#include <sstream>
#include <cstring>
#include <adept/base.h>
#include <adept/settings.h>
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#ifdef HAVE_OPENBLAS_CBLAS_HEADER
#include <cblas.h>
#endif
namespace adept {
// -------------------------------------------------------------------
// Get compile-time settings
// -------------------------------------------------------------------
// Return the version of Adept at compile time
std::string
version()
{
return ADEPT_VERSION_STR;
}
// Return the compiler used to compile the Adept library (e.g. "g++
// [4.3.2]" or "Microsoft Visual C++ [1800]")
std::string
compiler_version()
{
#ifdef CXX
std::string cv = CXX; // Defined in config.h
#elif defined(_MSC_VER)
std::string cv = "Microsoft Visual C++";
#else
std::string cv = "unknown";
#endif
#ifdef __GNUC__
#define STRINGIFY3(A,B,C) STRINGIFY(A) "." STRINGIFY(B) "." STRINGIFY(C)
#define STRINGIFY(A) #A
cv += " [" STRINGIFY3(__GNUC__,__GNUC_MINOR__,__GNUC_PATCHLEVEL__) "]";
#undef STRINGIFY
#undef STRINGIFY3
#elif defined(_MSC_VER)
#define STRINGIFY1(A) STRINGIFY(A)
#define STRINGIFY(A) #A
cv += " [" STRINGIFY1(_MSC_VER) "]";
#undef STRINGIFY
#undef STRINGIFY1
#endif
return cv;
}
// Return the compiler flags used when compiling the Adept library
// (e.g. "-Wall -g -O3")
std::string
compiler_flags()
{
#ifdef CXXFLAGS
return CXXFLAGS; // Defined in config.h
#else
return "unknown";
#endif
}
// Return a multi-line string listing numerous aspects of the way
// Adept has been configured.
std::string
configuration()
{
std::stringstream s;
s << "Adept version " << adept::version() << ":\n";
s << " Compiled with " << adept::compiler_version() << "\n";
s << " Compiler flags \"" << adept::compiler_flags() << "\"\n";
#ifdef BLAS_LIBS
if (std::strlen(BLAS_LIBS) > 2) {
const char* blas_libs = &BLAS_LIBS[2];
s << " BLAS support from " << blas_libs << " library\n";
}
else {
s << " BLAS support from built-in library\n";
}
#endif
#ifdef HAVE_OPENBLAS_CBLAS_HEADER
s << " Number of BLAS threads may be specified up to maximum of "
<< max_blas_threads() << "\n";
#endif
s << " Jacobians processed in blocks of size "
<< ADEPT_MULTIPASS_SIZE << "\n";
return s.str();
}
// -------------------------------------------------------------------
// Get/set number of threads for array operations
// -------------------------------------------------------------------
// Get the maximum number of threads available for BLAS operations
int
max_blas_threads()
{
#ifdef HAVE_OPENBLAS_CBLAS_HEADER
return openblas_get_num_threads();
#else
return 1;
#endif
}
// Set the maximum number of threads available for BLAS operations
// (zero means use the maximum sensible number on the current
// system), and return the number actually set. Note that OpenBLAS
// uses pthreads and the Jacobian calculation uses OpenMP - this can
// lead to inefficient behaviour so if you are computing Jacobians
// then you may get better performance by setting the number of
// array threads to one.
int
set_max_blas_threads(int n)
{
#ifdef HAVE_OPENBLAS_CBLAS_HEADER
openblas_set_num_threads(n);
return openblas_get_num_threads();
#else
return 1;
#endif
}
// Was the library compiled with matrix multiplication support (from
// BLAS)?
bool
have_matrix_multiplication() {
#ifdef HAVE_BLAS
return true;
#else
return false;
#endif
}
// Was the library compiled with linear algebra support (e.g. inv
// and solve from LAPACK)
bool
have_linear_algebra() {
#ifdef HAVE_LAPACK
return true;
#else
return false;
#endif
}
} // End namespace adept
================================================
FILE: adept/solve.cpp
================================================
/* solve.cpp -- Solve systems of linear equations using LAPACK
Copyright (C) 2015-2016 European Centre for Medium-Range Weather Forecasts
Author: Robin Hogan <r.j.hogan@ecmwf.int>
This file is part of the Adept library.
*/
#include <vector>
#include <adept/solve.h>
#include <adept/Array.h>
#include <adept/SpecialMatrix.h>
// If ADEPT_SOURCE_H is defined then we are in a header file generated
// from all the source files, so cpplapack.h will already have been
// included
#ifndef AdeptSource_H
#include "cpplapack.h"
#endif
#ifdef HAVE_LAPACK
namespace adept {
using namespace internal;
// -------------------------------------------------------------------
// Solve Ax = b for general square matrix A
// -------------------------------------------------------------------
template <typename T>
Array<1,T,false>
solve(const Array<2,T,false>& A, const Array<1,T,false>& b) {
Array<2,T,false> A_;
Array<1,T,false> b_;
// LAPACKE is more efficient with column-major input
// if (A.is_row_contiguous()) {
A_.resize_column_major(A.dimensions());
A_ = A;
// }
// else {
// A_.link(A);
// }
// if (b_.offset(0) != 0) {
b_ = b;
// }
// else {
// b_.link(b);
// }
std::vector<lapack_int> ipiv(A_.dimension(0));
// lapack_int status = LAPACKE_dgesv(LAPACK_COL_MAJOR, A_.dimension(0), 1,
// A_.data(), A_.offset(1), &ipiv[0],
// b_.data(), b_.dimension(0));
lapack_int status = cpplapack_gesv(A_.dimension(0), 1,
A_.data(), A_.offset(1), &ipiv[0],
b_.data(), b_.dimension(0));
if (status != 0) {
std::stringstream s;
s << "Failed to solve general system of equations: LAPACK ?gesv returned code " << status;
throw(matrix_ill_conditioned(s.str() ADEPT_EXCEPTION_LOCATION));
}
return b_;
}
// -------------------------------------------------------------------
// Solve AX = B for general square matrix A and rectangular matrix B
// -------------------------------------------------------------------
template <typename T>
Array<2,T,false>
solve(const Array<2,T,false>& A, const Array<2,T,false>& B) {
Array<2,T,false> A_;
Array<2,T,false> B_;
// LAPACKE is more efficient with column-major input
// if (A.is_row_contiguous()) {
A_.resize_column_major(A.dimensions());
A_ = A;
// }
// else {
// A_.link(A);
// }
// if (B.is_row_contiguous()) {
B_.resize_column_major(B.dimensions());
B_ = B;
// }
// else {
// B_.link(B);
// }
std::vector<lapack_int> ipiv(A_.dimension(0));
// lapack_int status = LAPACKE_dgesv(LAPACK_COL_MAJOR, A_.dimension(0), B.dimension(1),
// A_.data(), A_.offset(1), &ipiv[0],
// B_.data(), B_.offset(1));
lapack_int status = cpplapack_gesv(A_.dimension(0), B.dimension(1),
A_.data(), A_.offset(1), &ipiv[0],
B_.data(), B_.offset(1));
if (status != 0) {
std::stringstream s;
s << "Failed to solve general system of equations for matrix RHS: LAPACK ?gesv returned code " << status;
throw(matrix_ill_conditioned(s.str() ADEPT_EXCEPTION_LOCATION));
}
return B_;
}
// -------------------------------------------------------------------
// Solve Ax = b for symmetric square matrix A
// -------------------------------------------------------------------
template <typename T, SymmMatrixOrientation Orient>
Array<1,T,false>
solve(const SpecialMatrix<T,SymmEngine<Orient>,false>& A,
const Array<1,T,false>& b) {
SpecialMatrix<T,SymmEngine<Orient>,false> A_;
Array<1,T,false> b_;
// Not sure why the original code copies A...
A_.resize(A.dimension());
A_ = A;
// A_.link(A);
// if (b.offset(0) != 1) {
b_ = b;
// }
// else {
// b_.link(b);
// }
// Treat symmetric matrix as column-major
char uplo;
if (Orient == ROW_LOWER_COL_UPPER) {
uplo = 'U';
}
else {
uplo = 'L';
}
std::vector<lapack_int> ipiv(A_.dimension());
// lapack_int status = LAPACKE_dsysv(LAPACK_COL_MAJOR, uplo, A_.dimension(0), 1,
// A_.data(), A_.offset(), &ipiv[0],
// b_.data(), b_.dimension(0));
lapack_int status = cpplapack_sysv(uplo, A_.dimension(0), 1,
A_.data(), A_.offset(), &ipiv[0],
b_.data(), b_.dimension(0));
if (status != 0) {
// std::stringstream s;
// s << "Failed to solve symmetric system of equations: LAPACK ?sysv returned code " << status;
// throw(matrix_ill_conditioned(s.str() ADEPT_EXCEPTION_LOCATION));
std::cerr << "Warning: LAPACK solve symmetric system failed (?sysv): trying general (?gesv)\n";
return solve(Array<2,T,false>(A_),b_);
}
return b_;
}
// -------------------------------------------------------------------
// Solve AX = B for symmetric square matrix A
// -------------------------------------------------------------------
template <typename T, SymmMatrixOrientation Orient>
Array<2,T,false>
solve(const SpecialMatrix<T,SymmEngine<Orient>,false>& A,
const Array<2,T,false>& B) {
SpecialMatrix<T,SymmEngine<Orient>,false> A_;
Array<2,T,false> B_;
A_.resize(A.dimension());
A_ = A;
// A_.link(A);
// if (B.is_row_contiguous()) {
B_.resize_column_major(B.dimensions());
B_ = B;
// }
// else {
// B_.link(B);
// }
// Treat symmetric matrix as column-major
char uplo;
if (Orient == ROW_LOWER_COL_UPPER) {
uplo = 'U';
}
else {
uplo = 'L';
}
std::vector<lapack_int> ipiv(A_.dimension());
// lapack_int status = LAPACKE_dsysv(LAPACK_COL_MAJOR, uplo, A_.dimension(0), B.dimension(1),
// A_.data(), A_.offset(), &ipiv[0],
// B_.data(), B_.offset(1));
lapack_int status = cpplapack_sysv(uplo, A_.dimension(0), B.dimension(1),
A_.data(), A_.offset(), &ipiv[0],
B_.data(), B_.offset(1));
if (status != 0) {
std::stringstream s;
s << "Failed to solve symmetric system of equations with matrix RHS: LAPACK ?sysv returned code " << status;
throw(matrix_ill_conditioned(s.str() ADEPT_EXCEPTION_LOCATION));
}
return B_;
}
}
#else
namespace adept {
using namespace internal;
// -------------------------------------------------------------------
// Solve Ax = b for general square matrix A
// -------------------------------------------------------------------
template <typename T>
Array<1,T,false>
solve(const Array<2,T,false>& A, const Array<1,T,false>& b) {
throw feature_not_available("Cannot solve linear equations because compiled without LAPACK");
}
// -------------------------------------------------------------------
// Solve AX = B for general square matrix A and rectangular matrix B
// -------------------------------------------------------------------
template <typename T>
Array<2,T,false>
solve(const Array<2,T,false>& A, const Array<2,T,false>& B) {
throw feature_not_available("Cannot solve linear equations because compiled without LAPACK");
}
// -------------------------------------------------------------------
// Solve Ax = b for symmetric square matrix A
// -------------------------------------------------------------------
template <typename T, SymmMatrixOrientation Orient>
Array<1,T,false>
solve(const SpecialMatrix<T,SymmEngine<Orient>,false>& A,
const Array<1,T,false>& b) {
throw feature_not_available("Cannot solve linear equations because compiled without LAPACK");
}
// -------------------------------------------------------------------
// Solve AX = B for symmetric square matrix A
// -------------------------------------------------------------------
template <typename T, SymmMatrixOrientation Orient>
Array<2,T,false>
solve(const SpecialMatrix<T,SymmEngine<Orient>,false>& A,
const Array<2,T,false>& B) {
throw feature_not_available("Cannot solve linear equations because compiled without LAPACK");
}
}
#endif
namespace adept {
// -------------------------------------------------------------------
// Explicit instantiations
// -------------------------------------------------------------------
#define ADEPT_EXPLICIT_SOLVE(TYPE,RRANK) \
template Array<RRANK,TYPE,false> \
solve(const Array<2,TYPE,false>& A, const Array<RRANK,TYPE,false>& b); \
template Array<RRANK,TYPE,false> \
solve(const SpecialMatrix<TYPE,SymmEngine<ROW_LOWER_COL_UPPER>,false>& A, \
const Array<RRANK,TYPE,false>& b); \
template Array<RRANK,TYPE,false> \
solve(const SpecialMatrix<TYPE,SymmEngine<ROW_UPPER_COL_LOWER>,false>& A, \
const Array<RRANK,TYPE,false>& b);
ADEPT_EXPLICIT_SOLVE(float,1)
ADEPT_EXPLICIT_SOLVE(float,2)
ADEPT_EXPLICIT_SOLVE(double,1)
ADEPT_EXPLICIT_SOLVE(double,2)
#undef ADEPT_EXPLICIT_SOLVE
}
================================================
FILE: adept/vector_utilities.cpp
================================================
/* vector_utilities.cpp -- Vector utility functions
Copyright (C) 2016 European Centre for Medium-Range Weather Forecasts
Author: Robin Hogan <r.j.hogan@ecmwf.int>
This file is part of the Adept library.
*/
#include <adept/vector_utilities.h>
namespace adept {
Array<1,Real,false>
linspace(Real x1, Real x2, Index n) {
Array<1,Real,false> ans(n);
if (n > 1) {
for (Index i = 0; i < n; ++i) {
ans(i) = x1 + (x2-x1)*i / static_cast<Real>(n-1);
}
}
else if (n == 1 && x1 == x2) {
ans(0) = x1;
return ans;
}
else if (n == 1) {
throw(invalid_operation("linspace(x1,x2,n) with n=1 only valid if x1=x2"));
}
return ans;
}
}
================================================
FILE: benchmark/Makefile.am
================================================
check_PROGRAMS = autodiff_benchmark animate matrix_benchmark math_benchmark
autodiff_benchmark_SOURCES = autodiff_benchmark.cpp \
differentiator.h advection_schemes.h \
advection_schemes_AD.h advection_schemes_K.h nx.h
autodiff_benchmark_CPPFLAGS = -I@top_srcdir@/include
autodiff_benchmark_LDFLAGS = -static -no-install -L@top_srcdir@/adept/.libs
autodiff_benchmark_LDADD = -ladept
animate_SOURCES = animate.cpp
animate_CPPFLAGS = -I@top_srcdir@/include
matrix_benchmark_SOURCES = matrix_benchmark.cpp
matrix_benchmark_CPPFLAGS = -I@t
gitextract_yk4ax793/
├── .gitignore
├── .travis.yml
├── AUTHORS
├── COPYING
├── ChangeLog
├── INSTALL
├── Makefile.am
├── NEWS
├── README.md
├── TODO
├── adept/
│ ├── Array.cpp
│ ├── Makefile.am
│ ├── Minimizer.cpp
│ ├── Stack.cpp
│ ├── StackStorageOrig.cpp
│ ├── Storage.cpp
│ ├── cppblas.cpp
│ ├── cpplapack.h
│ ├── index.cpp
│ ├── inv.cpp
│ ├── jacobian.cpp
│ ├── line_search.cpp
│ ├── minimize_conjugate_gradient.cpp
│ ├── minimize_levenberg_marquardt.cpp
│ ├── minimize_limited_memory_bfgs.cpp
│ ├── settings.cpp
│ ├── solve.cpp
│ └── vector_utilities.cpp
├── benchmark/
│ ├── Makefile.am
│ ├── advection_schemes.h
│ ├── advection_schemes_AD.h
│ ├── advection_schemes_K.h
│ ├── animate.cpp
│ ├── autodiff_benchmark.cpp
│ ├── differentiator.h
│ ├── math_benchmark.cpp
│ ├── matrix_benchmark.cpp
│ └── nx.h
├── config_platform_independent.h.in
├── configure.ac
├── doc/
│ ├── COPYING
│ ├── Makefile
│ ├── README
│ ├── adept_documentation.tex
│ └── adept_reference.tex
├── include/
│ ├── Makefile.am
│ ├── Timer.h
│ ├── adept/
│ │ ├── Active.h
│ │ ├── ActiveConstReference.h
│ │ ├── ActiveReference.h
│ │ ├── Allocator.h
│ │ ├── Array.h
│ │ ├── ArrayWrapper.h
│ │ ├── BinaryOperation.h
│ │ ├── Expression.h
│ │ ├── ExpressionSize.h
│ │ ├── FixedArray.h
│ │ ├── GradientIndex.h
│ │ ├── IndexedArray.h
│ │ ├── Minimizer.h
│ │ ├── Optimizable.h
│ │ ├── Packet.h
│ │ ├── RangeIndex.h
│ │ ├── ScratchVector.h
│ │ ├── SpecialMatrix.h
│ │ ├── Stack.h
│ │ ├── StackStorage.h
│ │ ├── StackStorageOrig.h
│ │ ├── StackStorageOrigStl.h
│ │ ├── Statement.h
│ │ ├── Storage.h
│ │ ├── UnaryOperation.h
│ │ ├── array_shortcuts.h
│ │ ├── base.h
│ │ ├── contiguous_matrix.h
│ │ ├── cppblas.h
│ │ ├── eval.h
│ │ ├── exception.h
│ │ ├── interp.h
│ │ ├── inv.h
│ │ ├── matmul.h
│ │ ├── noalias.h
│ │ ├── outer_product.h
│ │ ├── quick_e.h
│ │ ├── reduce.h
│ │ ├── scalar_shortcuts.h
│ │ ├── settings.h
│ │ ├── solve.h
│ │ ├── spread.h
│ │ ├── store_transpose.h
│ │ ├── traits.h
│ │ ├── vector_utilities.h
│ │ └── where.h
│ ├── adept.h
│ ├── adept_arrays.h
│ ├── adept_fortran.h
│ ├── adept_optimize.h
│ └── create_adept_source_header
├── m4/
│ ├── adept.m4
│ ├── ax_blas.m4
│ ├── ax_lapack.m4
│ ├── ltsugar.m4
│ └── lt~obsolete.m4
├── makefile_include.in
└── test/
├── Makefile
├── README
├── algorithm.cpp
├── algorithm.h
├── algorithm_with_and_without_ad.h
├── rosenbrock_banana_function.cpp
├── run_tests.sh
├── simulate_radiances.cpp
├── simulate_radiances.h
├── state.cpp
├── state.h
├── test_adept.cpp
├── test_adept_with_and_without_ad.cpp
├── test_array_derivatives.cpp
├── test_array_speed.cpp
├── test_arrays.cpp
├── test_checkpoint.cpp
├── test_constructors.cpp
├── test_derivatives.cpp
├── test_fastexp.cpp
├── test_fixed_arrays.cpp
├── test_gsl_interface.cpp
├── test_interp.cpp
├── test_minimizer.cpp
├── test_misc.cpp
├── test_no_lib.cpp
├── test_packet_operations.cpp
├── test_radiances.cpp
├── test_radiances_array.cpp
├── test_reduce_active.cpp
├── test_thread_safe.cpp
└── test_thread_safe_arrays.cpp
SYMBOL INDEX (575 symbols across 97 files)
FILE: adept/Array.cpp
type adept (line 13) | namespace adept {
type internal (line 14) | namespace internal {
function set_array_print_style (line 35) | void set_array_print_style(ArrayPrintStyle ps) {
FILE: adept/Minimizer.cpp
type adept (line 16) | namespace adept {
function to_lower_in_place (line 37) | static void to_lower_in_place(std::string& str) {
function MinimizerStatus (line 118) | MinimizerStatus
function MinimizerStatus (line 146) | MinimizerStatus
FILE: adept/Stack.cpp
type adept (line 24) | namespace adept {
function uIndex (line 195) | uIndex
FILE: adept/StackStorageOrig.cpp
type adept (line 25) | namespace adept {
type internal (line 26) | namespace internal {
FILE: adept/Storage.cpp
type adept (line 13) | namespace adept {
type internal (line 14) | namespace internal {
FILE: adept/cppblas.cpp
type adept (line 61) | namespace adept {
type internal (line 63) | namespace internal {
type internal (line 188) | namespace internal {
type adept (line 186) | namespace adept {
type internal (line 63) | namespace internal {
type internal (line 188) | namespace internal {
FILE: adept/cpplapack.h
function namespace (line 48) | namespace adept {
FILE: adept/index.cpp
type adept (line 12) | namespace adept {
FILE: adept/inv.cpp
type adept (line 21) | namespace adept {
function inv (line 29) | Array<2,Type,false>
function inv (line 77) | SpecialMatrix<Type,SymmEngine<Orient>,false>
function inv (line 132) | Array<2,Type,false>
function inv (line 141) | SpecialMatrix<Type,SymmEngine<Orient>,false>
type adept (line 124) | namespace adept {
function inv (line 29) | Array<2,Type,false>
function inv (line 77) | SpecialMatrix<Type,SymmEngine<Orient>,false>
function inv (line 132) | Array<2,Type,false>
function inv (line 141) | SpecialMatrix<Type,SymmEngine<Orient>,false>
type adept (line 150) | namespace adept {
function inv (line 29) | Array<2,Type,false>
function inv (line 77) | SpecialMatrix<Type,SymmEngine<Orient>,false>
function inv (line 132) | Array<2,Type,false>
function inv (line 141) | SpecialMatrix<Type,SymmEngine<Orient>,false>
FILE: adept/jacobian.cpp
type adept (line 18) | namespace adept {
type internal (line 20) | namespace internal {
function T (line 27) | T _check_long_double() {
FILE: adept/line_search.cpp
type adept (line 15) | namespace adept {
function MinimizerStatus (line 28) | MinimizerStatus
function MinimizerStatus (line 89) | MinimizerStatus
FILE: adept/minimize_conjugate_gradient.cpp
type adept (line 15) | namespace adept {
function MinimizerStatus (line 22) | MinimizerStatus
function MinimizerStatus (line 200) | MinimizerStatus
FILE: adept/minimize_levenberg_marquardt.cpp
type adept (line 15) | namespace adept {
function MinimizerStatus (line 20) | MinimizerStatus
function MinimizerStatus (line 195) | MinimizerStatus
FILE: adept/minimize_limited_memory_bfgs.cpp
type adept (line 15) | namespace adept {
class LbfgsData (line 19) | class LbfgsData {
method LbfgsData (line 22) | LbfgsData(int nx, int ni)
method store (line 33) | void store(int iter, const Vector& x_diff, const Vector& gradient_di...
method Vector (line 52) | Vector x_diff(int iter) {
method Vector (line 55) | Vector gradient_diff(int iter) {
method Real (line 59) | Real& alpha(int iter) { return alpha_[iter % ni_]; }
method Real (line 60) | Real rho(int iter) const { return rho_[iter % ni_]; }
method Real (line 61) | Real gamma(int iter) const { return gamma_[iter % ni_]; }
function MinimizerStatus (line 80) | MinimizerStatus
function MinimizerStatus (line 271) | MinimizerStatus
FILE: adept/settings.cpp
type adept (line 25) | namespace adept {
function version (line 32) | std::string
function compiler_version (line 40) | std::string
function compiler_flags (line 73) | std::string
function configuration (line 85) | std::string
function max_blas_threads (line 116) | int
function set_max_blas_threads (line 133) | int
function have_matrix_multiplication (line 146) | bool
function have_linear_algebra (line 157) | bool
FILE: adept/solve.cpp
type adept (line 26) | namespace adept {
function solve (line 34) | Array<1,T,false>
function solve (line 76) | Array<2,T,false>
function solve (line 119) | Array<1,T,false>
function solve (line 170) | Array<2,T,false>
function solve (line 226) | Array<1,T,false>
function solve (line 235) | Array<2,T,false>
function solve (line 244) | Array<1,T,false>
function solve (line 254) | Array<2,T,false>
type adept (line 218) | namespace adept {
function solve (line 34) | Array<1,T,false>
function solve (line 76) | Array<2,T,false>
function solve (line 119) | Array<1,T,false>
function solve (line 170) | Array<2,T,false>
function solve (line 226) | Array<1,T,false>
function solve (line 235) | Array<2,T,false>
function solve (line 244) | Array<1,T,false>
function solve (line 254) | Array<2,T,false>
FILE: adept/vector_utilities.cpp
type adept (line 13) | namespace adept {
function linspace (line 15) | Array<1,Real,false>
FILE: benchmark/advection_schemes.h
function aReal (line 64) | struct is_active<adept::aReal> { static const bool value = true; }
type adept (line 71) | typedef adept::Array<1,Real,::is_active<aReal>::value> my_vector;
type adept (line 93) | typedef adept::Array<1,Real,::is_active<aReal>::value> my_vector;
FILE: benchmark/animate.cpp
function main (line 17) | int
FILE: benchmark/autodiff_benchmark.cpp
function Real (line 22) | static
function usage (line 36) | static
function main (line 64) | int
FILE: benchmark/differentiator.h
type TestAlgorithm (line 45) | enum TestAlgorithm {
function std (line 59) | inline
function class (line 72) | class differentiator_exception : public std::exception {
function virtual (line 76) | virtual const char* what() const throw()
function class (line 84) | class Differentiator {
function virtual (line 118) | virtual bool adjoint(TestAlgorithm test_algorithm,
function reset_timings (line 134) | void reset_timings() {
function virtual (line 141) | virtual std::string name() const = 0; //{ return "GENERIC"; }
function class (line 171) | class HandCodedDifferentiator
function class (line 253) | class AdeptDifferentiator
function virtual (line 285) | virtual bool adjoint(TestAlgorithm test_algorithm,
function virtual (line 365) | virtual void no_openmp() {
function virtual (line 369) | virtual void print() {
function class (line 385) | class AdolcDifferentiator
function class (line 521) | class CppadDifferentiator
function class (line 639) | class SacadoDifferentiator
type AutoDiffTool (line 737) | enum AutoDiffTool {
function std (line 777) | inline
function Differentiator (line 791) | inline
FILE: benchmark/math_benchmark.cpp
function main (line 16) | int main(int argc, const char** argv)
FILE: benchmark/matrix_benchmark.cpp
function time_matmul (line 8) | double
function time_solve (line 50) | double
function main (line 84) | int
FILE: include/Timer.h
function class (line 28) | class Timer {
function print (line 60) | void print() {
function TimerInt (line 74) | TimerInt new_activity(const std::string& name) {
function stop (line 82) | void stop() {
function start (line 90) | void start(TimerInt activity) {
function reset (line 111) | void reset(TimerInt activity) {
function std (line 119) | const std::vector<double>& timings() { return timings_; }
function timing (line 122) | double timing(TimerInt activity) {
function TimerInt (line 132) | TimerInt size() {
type timeval (line 154) | struct timeval
FILE: include/adept/Active.h
type Type (line 59) | typedef Type T;
function get_dimensions_ (line 475) | bool get_dimensions_(ExpressionSize<0>& dim) const { return true; }
function is_aliased_ (line 483) | bool is_aliased_(const Type* mem1, const Type* mem2) const {
function Type (line 487) | Type value_with_len_(const Index& j, const Index& len) const
function get_values (line 584) | void get_values(const Active<Type>* a, Index n, Type* data)
function get_gradients (line 607) | void get_gradients(const Active<Type>* a, Index n, Type* data)
function namespace (line 633) | namespace internal {
FILE: include/adept/ActiveConstReference.h
function namespace (line 25) | namespace adept {
function Type (line 140) | Type get_gradient() const {
function get_dimensions_ (line 243) | bool get_dimensions_(ExpressionSize<0>& dim) const { return true; }
function is_aliased_ (line 251) | bool is_aliased_(const Type* mem1, const Type* mem2) const {
function Type (line 255) | Type value_with_len_(const Index& j, const Index& len) const
function namespace (line 343) | namespace internal {
FILE: include/adept/ActiveReference.h
function namespace (line 25) | namespace adept {
function namespace (line 476) | namespace internal {
FILE: include/adept/Allocator.h
function namespace (line 16) | namespace adept {
FILE: include/adept/Array.h
function namespace (line 40) | namespace adept {
function Array (line 1541) | Array
function Index (line 1835) | Index dimension(int j) const {
function Index (line 1838) | Index size(int j) const {
function Index (line 1843) | Index offset(int j) const {
function Type (line 1868) | Type* data() { return data_; }
function Type (line 1869) | const Type* data() const { return data_; }
function Type (line 1870) | const Type* const_data() const { return data_; }
function Type (line 1873) | Type* data_pointer() { return data_; }
function Type (line 1874) | const Type* data_pointer() const { return data_; }
function Type (line 1875) | const Type* const_data_pointer() const { return data_; }
function Type (line 1879) | Type* data_pointer(Index i) {
function Type (line 1888) | const Type* const_data_pointer(Index i) const {
function clear (line 1904) | void clear() {
function resize (line 1959) | void resize(const ExpressionSize<Rank>& dim) {
function resize_contiguous (line 1964) | void resize_contiguous(const ExpressionSize<Rank>& dim) {
function resize_row_major (line 1969) | void resize_row_major(const ExpressionSize<Rank>& dim) {
function resize_row_major_contiguous (line 1973) | void resize_row_major_contiguous(const ExpressionSize<Rank>& dim) {
function resize_column_major (line 1977) | void resize_column_major(const ExpressionSize<Rank>& dim) {
function Type (line 2104) | Type value_with_len_(const Index& j, const Index& len) const {
FILE: include/adept/ArrayWrapper.h
function namespace (line 16) | namespace adept {
type T (line 134) | typedef const T& __restrict type;
FILE: include/adept/BinaryOperation.h
function namespace (line 17) | namespace adept {
function is_aliased_ (line 115) | bool is_aliased_(const Type* mem1, const Type* mem2) const {
function Type (line 144) | Type value_with_len_(const Index& j, const Index& len) const {
function is_aliased_ (line 436) | bool is_aliased_(const Type* mem1, const Type* mem2) const {
function Type (line 450) | Type value_with_len_(const Index& j, const Index& len) const {
function is_aliased_ (line 634) | bool is_aliased_(const Type* mem1, const Type* mem2) const {
function Type (line 648) | Type value_with_len_(const Index& j, const Index& len) const {
function namespace (line 768) | namespace internal {
type Subtract (line 817) | struct Subtract {
type Multiply (line 860) | struct Multiply {
type Divide (line 906) | struct Divide {
type Pow (line 980) | struct Pow {
type Atan2 (line 1039) | struct Atan2 {
type Max (line 1099) | struct Max {
type Min (line 1180) | struct Min {
type CopySign (line 1260) | struct CopySign {
type typename (line 1424) | typedef typename promote<typename
type typename (line 1443) | typedef typename promote<typename
FILE: include/adept/Expression.h
function namespace (line 22) | namespace adept {
function namespace (line 268) | namespace internal {
FILE: include/adept/ExpressionSize.h
function namespace (line 29) | namespace adept {
function set_all (line 71) | void set_all(Index j) {
function copy (line 78) | void copy(const ExpressionSize& d) {
function copy (line 84) | void copy(const Index* d) {
function Index (line 115) | Index size() const {
function Index (line 168) | const Index& operator[](int i) const { return dim[i]; }
function set_all (line 182) | void set_all(Index) const { }
function const (line 185) | bool operator[](int) const { return 0; }
function namespace (line 204) | namespace internal {
function ExpressionSize (line 248) | inline ExpressionSize<1> expression_size(Index j0)
function ExpressionSize (line 250) | inline ExpressionSize<2> expression_size(Index j0, Index j1)
function ExpressionSize (line 254) | inline ExpressionSize<1> dimensions(Index j0)
function ExpressionSize (line 256) | inline ExpressionSize<2> dimensions(Index j0, Index j1)
function ExpressionSize (line 258) | inline ExpressionSize<3> dimensions(Index j0, Index j1, Index j2)
function ExpressionSize (line 260) | inline ExpressionSize<4> dimensions(Index j0, Index j1, Index j2,
function ExpressionSize (line 263) | inline ExpressionSize<5> dimensions(Index j0, Index j1, Index j2,
function ExpressionSize (line 266) | inline ExpressionSize<6> dimensions(Index j0, Index j1, Index j2,
function ExpressionSize (line 269) | inline ExpressionSize<7> dimensions(Index j0, Index j1, Index j2,
FILE: include/adept/FixedArray.h
function namespace (line 27) | namespace adept {
FILE: include/adept/GradientIndex.h
function namespace (line 8) | namespace adept {
type GradientIndex (line 47) | struct GradientIndex
function set (line 52) | void set(Index val) { }
function clear (line 53) | void clear() { }
FILE: include/adept/IndexedArray.h
function namespace (line 34) | namespace adept {
function Index (line 157) | Index get_value_with_len(const std::vector<T>& ind, const Index& j,
function get_dimensions_ (line 312) | bool get_dimensions_(ExpressionSize<Rank>& dim) const {
function Type (line 370) | Type value_with_len_(const Index& i, const Index& len) const {
function calc_gradient_ (line 424) | void calc_gradient_(Stack& stack,
FILE: include/adept/Minimizer.h
function namespace (line 16) | namespace adept {
function MinimizerAlgorithm (line 133) | MinimizerAlgorithm algorithm() { return algorithm_; }
function max_iterations (line 135) | int max_iterations() { return max_iterations_; }
function Real (line 136) | Real converged_gradient_norm() { return converged_gradient_norm_; }
function set_max_line_search_iterations (line 147) | void set_max_line_search_iterations(int mi) { max_line_search_iterations...
function set_armijo_coeff (line 148) | void set_armijo_coeff(Real ac) {
function set_lbfgs_curvature_coeff (line 156) | void set_lbfgs_curvature_coeff(Real lcc) {
function set_cg_curvature_coeff (line 164) | void set_cg_curvature_coeff(Real cgcc) {
function set_levenberg_damping_limits (line 313) | inline void
function set_levenberg_damping_start (line 324) | inline void
function set_levenberg_damping_restart (line 331) | inline void
function set_levenberg_damping_multiplier (line 338) | inline void
FILE: include/adept/Optimizable.h
function namespace (line 16) | namespace adept {
FILE: include/adept/Packet.h
function namespace (line 65) | namespace internal {
type typename (line 116) | typedef typename quick_e::packet<T,size>::type intrinsic_type;
function data (line 124) | Packet(const Packet& d) : PacketData<T>(d.data) { }
function explicit (line 128) | explicit Packet(const T* d) : PacketData<T>(quick_e::load<intrinsic_type...
function put (line 134) | void put(T* __restrict d) const { quick_e::store(d, data); }
function put_unaligned (line 135) | void put_unaligned(T* __restrict d) const { quick_e::storeu(d, data); }
function operator (line 141) | void operator+=(const Packet& d) { data = quick_e::add(data, d.data); }
function operator (line 142) | void operator-=(const Packet& d) { data = quick_e::sub(data, d.data); }
function operator (line 143) | void operator*=(const Packet& d) { data = quick_e::mul(data, d.data); }
function operator (line 144) | void operator/=(const Packet& d) { data = quick_e::div(data, d.data); }
function exp (line 280) | inline float exp(float x) { return quick_e::exp(x); }
function exp (line 281) | inline double exp(double x) { return quick_e::exp(x); }
function fastexp (line 283) | inline float fastexp(float x) { return quick_e::exp(x); }
function fastexp (line 284) | inline double fastexp(double x) { return quick_e::exp(x); }
function namespace (line 287) | namespace functions {
FILE: include/adept/RangeIndex.h
function namespace (line 42) | namespace adept {
function Index (line 195) | Index size_with_len_(const Index& len) const
function get_dimensions_ (line 198) | bool get_dimensions_(ExpressionSize<1>& dim) const {
function is_aliased_ (line 213) | bool is_aliased_(const Index* mem1, const Index* mem2) const {
function Index (line 223) | Index value_with_len_(const Index&j, const Index& len) const
function Index (line 247) | Index begin(Index len) const
function Index (line 249) | Index end(Index len) const
function Index (line 251) | Index stride(Index len) const
function is_aliased_ (line 286) | bool is_aliased_(const Index* mem1, const Index* mem2) const { return fa...
function Index (line 288) | Index size_with_len_(const Index& len) const
function Index (line 291) | Index value_with_len_(const Index& j, const Index& len) const
function Index (line 294) | Index value_at_location_(const ExpressionSize<1>& loc) const
function Index (line 298) | Index end(Index len) const { return len-1; }
function AllIndex (line 311) | struct is_range<AllIndex> {
FILE: include/adept/ScratchVector.h
function namespace (line 32) | namespace adept {
FILE: include/adept/SpecialMatrix.h
function namespace (line 31) | namespace adept {
type SquareEngine (line 169) | struct SquareEngine
function Index (line 173) | Index pack_offset(Index dim) const { return dim; }
function Index (line 174) | Index index(Index i, Index j, Index offset) const {
function get_row_range (line 181) | void get_row_range(Index i, Index dim, Index offset,
function Index (line 215) | Index upper_offset(Index dim, Index offset, Index offdiag) const {
function Index (line 218) | Index lower_offset(Index dim, Index offset, Index offdiag) const {
type SquareEngine (line 221) | typedef SquareEngine<ROW_MAJOR> transpose_engine;
type BandEngineHelper (line 242) | struct BandEngineHelper
type BandEngineHelper (line 246) | struct BandEngineHelper
type BandEngineHelper (line 250) | struct BandEngineHelper
function Index (line 265) | Index pack_offset(Index dim) const { return diagonals-1; }
function Index (line 266) | Index index(Index i, Index j, Index offset) const {
function get_row_range (line 274) | void get_row_range(Index i, Index dim, Index offset,
type BandEngine (line 282) | typedef BandEngine<COL_MAJOR,UDiags,LDiags> transpose_engine;
function Index (line 336) | Index data_size(Index dim, Index offset) const {
function Index (line 340) | Index upper_offset(Index dim, Index offset, Index offdiag) const {
function Index (line 343) | Index lower_offset(Index dim, Index offset, Index offdiag) const {
function check_upper_diag (line 346) | void check_upper_diag(Index offdiag) const {
function check_lower_diag (line 352) | void check_lower_diag(Index offdiag) const {
function Index (line 400) | Index index(Index i, Index j, Index offset) const {
function get_row_range (line 408) | void get_row_range(Index i, Index dim, Index offset,
type BandEngine (line 416) | typedef BandEngine<ROW_MAJOR,UDiags,LDiags> transpose_engine;
function Index (line 472) | Index upper_offset(Index dim, Index offset, Index offdiag) const {
function Index (line 476) | Index lower_offset(Index dim, Index offset, Index offdiag) const {
function Index (line 511) | Index index(Index i, Index j, Index offset) const {
function get_row_range (line 518) | void get_row_range(Index i, Index dim, Index offset,
type SymmEngine (line 526) | typedef SymmEngine<ROW_LOWER_COL_UPPER> transpose_engine;
function Index (line 557) | Index upper_offset(Index dim, Index offset, Index offdiag) const {
function Index (line 560) | Index lower_offset(Index dim, Index offset, Index offdiag) const {
type SymmEngine (line 579) | struct SymmEngine
function Index (line 585) | Index pack_offset(Index dim) const { return dim; }
function Index (line 586) | Index index(Index i, Index j, Index offset) const {
function get_row_range (line 593) | void get_row_range(Index i, Index dim, Index offset,
type SymmEngine (line 601) | typedef SymmEngine<ROW_UPPER_COL_LOWER> transpose_engine;
function Index (line 602) | Index upper_offset(Index dim, Index offset, Index offdiag) const {
function Index (line 605) | Index lower_offset(Index dim, Index offset, Index offdiag) const {
function check_upper_diag (line 679) | void check_upper_diag(Index offdiag) const {
type UpperEngine (line 758) | typedef UpperEngine<COL_MAJOR> transpose_engine;
function get_row_range (line 759) | void get_row_range(Index i, Index dim, Index offset,
type LowerEngine (line 771) | struct LowerEngine
type UpperEngine (line 775) | typedef UpperEngine<ROW_MAJOR> transpose_engine;
function get_row_range (line 776) | void get_row_range(Index i, Index dim, Index offset,
function check_lower_diag (line 806) | void check_lower_diag(Index offdiag) const {
function get_row_range (line 888) | void get_row_range(Index i, Index dim, Index offset,
type UpperEngine (line 900) | struct UpperEngine
type LowerEngine (line 901) | typedef LowerEngine<ROW_MAJOR> transpose_engine;
function get_row_range (line 905) | void get_row_range(Index i, Index dim, Index offset,
function storage_ (line 953) | SpecialMatrix(Index m0) : storage_(0) { resize(m0); }
function SpecialMatrix (line 1328) | SpecialMatrix
function SpecialMatrix (line 1352) | SpecialMatrix soft_link() {
function get_dimensions_ (line 1424) | bool get_dimensions_(ExpressionSize<2>& dim) const {
function Type (line 1469) | Type* data() { return data_; }
function Type (line 1470) | const Type* data() const { return data_; }
function Type (line 1471) | const Type* const_data() const { return data_; }
function Type (line 1474) | Type* data_pointer() { return data_; }
function Type (line 1475) | const Type* data_pointer() const { return data_; }
function Type (line 1476) | const Type* const_data_pointer() const { return data_; }
function clear (line 1484) | void clear() {
function resize (line 1496) | void resize(Index dim) {
function resize (line 1523) | void resize(Index dim0, Index dim1) {
function is_aliased_ (line 1531) | bool is_aliased_(const Type* mem1, const Type* mem2) const {
function Type (line 1546) | Type value_with_len_(const Index& j, const Index& len) const {
function Index (line 1576) | Index gradient_index() const {
function data_range (line 1641) | void data_range(Type const * &data_begin, Type const * &data_end) const {
FILE: include/adept/Stack.h
function Type (line 72) | const Type& operator[](uIndex i) const { return data[i]; }
function zero (line 74) | void zero() { for (uIndex i = 0; i < Size; i++) data[i] = 0.0; }
type Gap (line 79) | struct Gap {
type std (line 109) | typedef std::list<Gap>::iterator GapListIterator;
function start (line 157) | void start(uIndex n = ADEPT_INITIAL_STACK_LENGTH) {
function update_lhs (line 168) | bool update_lhs(const uIndex& gradient_index) {
function uIndex (line 181) | uIndex register_gradient() {
function uIndex (line 220) | uIndex register_gradients(const uIndex& n) {
function unregister_gradient (line 245) | void unregister_gradient(const uIndex& gradient_index) {
function forward (line 362) | void forward() { return compute_tangent_linear(); }
function reverse (line 368) | void reverse() { return compute_adjoint(); }
function clear_gradients (line 495) | void clear_gradients() {
function clear_independents (line 501) | void clear_independents() {
function clear_dependents (line 507) | void clear_dependents() {
function clear (line 512) | void clear() {
function clear_statements (line 517) | void clear_statements() {
function deactivate (line 531) | void deactivate() {
function new_recording (line 544) | void new_recording() {
function pause_recording (line 575) | bool pause_recording() {
function continue_recording (line 586) | bool continue_recording() {
function uIndex (line 692) | uIndex max_gradient_index() const {
function Real (line 723) | Real fraction_multipliers_equal_to(Real val) {
function preallocate_statements (line 742) | void preallocate_statements(uIndex n) {
function preallocate_operations (line 747) | void preallocate_operations(uIndex n) {
function preallocate_statements (line 845) | inline
function preallocate_operations (line 849) | inline
function Stack (line 855) | inline
function is_thread_unsafe (line 862) | inline bool is_thread_unsafe() { return true; }
function is_thread_unsafe (line 864) | inline bool is_thread_unsafe() { return false; }
FILE: include/adept/StackStorage.h
function namespace (line 28) | namespace adept {
FILE: include/adept/StackStorageOrig.h
function namespace (line 29) | namespace adept {
FILE: include/adept/StackStorageOrigStl.h
function namespace (line 28) | namespace adept {
FILE: include/adept/Statement.h
function namespace (line 16) | namespace adept {
FILE: include/adept/Storage.h
function namespace (line 42) | namespace internal {
function remove_link (line 147) | void remove_link() {
function Type (line 169) | Type*
function Type (line 172) | const Type*
function Index (line 215) | inline Index n_storage_objects()
function Index (line 219) | inline Index n_storage_objects_created()
function Index (line 222) | inline Index n_storage_objects_deleted()
FILE: include/adept/UnaryOperation.h
function namespace (line 17) | namespace adept {
function namespace (line 336) | namespace internal {
FILE: include/adept/array_shortcuts.h
function namespace (line 17) | namespace adept {
FILE: include/adept/base.h
function namespace (line 312) | namespace adept {
FILE: include/adept/contiguous_matrix.h
function namespace (line 15) | namespace adept {
FILE: include/adept/cppblas.h
function namespace (line 18) | namespace adept {
FILE: include/adept/eval.h
function namespace (line 15) | namespace adept {
FILE: include/adept/exception.h
function namespace (line 26) | namespace adept {
function class (line 39) | class feature_not_available : public adept::exception {
function class (line 51) | class autodiff_exception : public adept::exception { }
function class (line 55) | class gradient_out_of_range : public autodiff_exception {
function class (line 62) | class gradients_not_initialized : public autodiff_exception {
function class (line 69) | class stack_already_active : public autodiff_exception {
function class (line 76) | class dependents_or_independents_not_identified : public autodiff_except...
function class (line 83) | class wrong_gradient : public autodiff_exception {
function class (line 90) | class non_finite_gradient : public autodiff_exception {
function class (line 103) | class array_exception : public adept::exception {
function class (line 110) | class size_mismatch : public array_exception {
function class (line 117) | class inner_dimension_mismatch : public array_exception {
function class (line 124) | class empty_array : public array_exception {
function class (line 131) | class invalid_dimension : public array_exception {
function class (line 138) | class index_out_of_bounds : public array_exception {
function class (line 145) | class invalid_operation : public array_exception {
function class (line 152) | class matrix_ill_conditioned : public array_exception {
function class (line 159) | class fortran_interoperability_error : public array_exception {
function class (line 172) | class optimization_exception : public adept::exception {
function namespace (line 191) | namespace internal {
FILE: include/adept/interp.h
function namespace (line 16) | namespace adept {
FILE: include/adept/inv.h
function namespace (line 18) | namespace adept {
FILE: include/adept/matmul.h
function namespace (line 19) | namespace adept {
function namespace (line 617) | namespace internal {
FILE: include/adept/noalias.h
function namespace (line 15) | namespace adept {
FILE: include/adept/outer_product.h
function namespace (line 16) | namespace adept {
FILE: include/adept/quick_e.h
function namespace (line 111) | namespace quick_e {
function V (line 204) | V set0() { return 0.0; }
function fmin (line 218) | inline float fmin(float x, float y) { return std::fmin(x,y); }
function fmin (line 219) | inline double fmin(double x, double y) { return std::fmin(x,y); }
function fmax (line 220) | inline float fmax(float x, float y) { return std::fmax(x,y); }
function fmax (line 221) | inline double fmax(double x, double y) { return std::fmax(x,y); }
function select_gt (line 224) | inline float select_gt(float x1, float x2, float y1, float y2) {
function select_gt (line 227) | inline double select_gt(double x1, double x2, double y1, double y2) {
function all_in_range (line 231) | inline bool all_in_range(float x, float low_bound, float high_bound) {
function all_in_range (line 234) | inline bool all_in_range(double x, double low_bound, double high_bound) {
function __m128 (line 390) | inline __m128 unchecked_round(__m128 x)
function __m128d (line 393) | inline __m128d unchecked_round(__m128d x)
function __m128 (line 399) | inline __m128 unchecked_round(__m128 x)
function __m128d (line 401) | inline __m128d unchecked_round(__m128d x)
function unchecked_round (line 405) | inline float unchecked_round(float x)
function unchecked_round (line 407) | inline double unchecked_round(double x)
function pow2n (line 414) | inline float pow2n(float x)
function pow2n (line 416) | inline double pow2n(double x)
function horiz_and (line 420) | inline bool horiz_and(__m128i a) {
function all_in_range (line 437) | inline bool all_in_range(__m128 x, float low_bound, float high_bound) {
function all_in_range (line 442) | inline bool all_in_range(__m128d x, double low_bound, double high_bound) {
function __m128 (line 449) | inline __m128 select_gt(__m128 x1, __m128 x2,
function __m128d (line 459) | inline __m128d select_gt(__m128d x1, __m128d x2,
function hsum (line 497) | inline float hsum(__m256 x) { return hsum(add(low(x), high(x))); }
function hmul (line 498) | inline float hmul(__m256 x) { return hmul(mul(low(x), high(x))); }
function hmin (line 499) | inline float hmin(__m256 x) { return hmin(fmin(low(x), high(x))); }
function hmax (line 500) | inline float hmax(__m256 x) { return hmax(fmax(low(x), high(x))); }
function hsum (line 501) | inline double hsum(__m256d x) { return hsum(add(low(x), high(x))); }
function hmul (line 502) | inline double hmul(__m256d x) { return hmul(mul(low(x), high(x))); }
function hmin (line 503) | inline double hmin(__m256d x) { return hmin(fmin(low(x), high(x))); }
function hmax (line 504) | inline double hmax(__m256d x) { return hmax(fmax(low(x), high(x))); }
function __m256 (line 515) | inline __m256 unchecked_round(__m256 x)
function __m256d (line 518) | inline __m256d unchecked_round(__m256d x)
function __m256 (line 529) | inline __m256 pow2n(__m256 n) {
function __m256d (line 532) | inline __m256d pow2n(__m256d n) {
function all_in_range (line 543) | inline bool all_in_range(__m256 x, float low_bound, float high_bound) {
function all_in_range (line 549) | inline bool all_in_range(__m256d x, double low_bound, double high_bound) {
function __m256 (line 555) | inline __m256 select_gt(__m256 x1, __m256 x2,
function __m256d (line 559) | inline __m256d select_gt(__m256d x1, __m256d x2,
function __m512 (line 590) | inline __m512 unchecked_round(__m512 x) { return _mm512_roundscale_ps(...
function __m512d (line 591) | inline __m512d unchecked_round(__m512d x) { return _mm512_roundscale_pd(...
function all_in_range (line 601) | inline bool all_in_range(__m512 x, float low_bound, float high_bound) {
function all_in_range (line 607) | inline bool all_in_range(__m512d x, double low_bound, double high_bound) {
function __m512 (line 613) | inline __m512 select_gt(__m512 x1, __m512 x2,
function __m512d (line 617) | inline __m512d select_gt(__m512d x1, __m512d x2,
function float32x4_t (line 628) | inline float32x4_t vzeroq_f32() { return vdupq_n_f32(0.0); }
function float64x2_t (line 629) | inline float64x2_t vzeroq_f64() { return vdupq_n_f64(0.0); }
function vmulvq_f32 (line 631) | inline float vmulvq_f32(float32x4_t x) {
function vmulvq_f64 (line 639) | inline double vmulvq_f64(float64x2_t x) {
function all_in_range (line 668) | inline bool all_in_range(float32x4_t x, double low_bound, double high_bo...
function all_in_range (line 678) | inline bool all_in_range(float64x2_t x, double low_bound, double high_bo...
function float32x4_t (line 688) | inline float32x4_t unchecked_round(float32x4_t x) {
function float64x2_t (line 691) | inline float64x2_t unchecked_round(float64x2_t x) {
function float32x4_t (line 694) | inline float32x4_t select_gt(float32x4_t x1, float32x4_t x2,
function float64x2_t (line 698) | inline float64x2_t select_gt(float64x2_t x1, float64x2_t x2,
function unchecked_round (line 703) | inline float unchecked_round(float x)
function unchecked_round (line 705) | inline double unchecked_round(double x)
function pow2n (line 708) | inline float pow2n(float x) {
function pow2n (line 711) | inline double pow2n(double x) {
function Vec (line 725) | inline
function Vec (line 736) | Vec fastexp_float(Vec const initial_x) {
function Vec (line 812) | Vec fastexp_double(Vec const initial_x) {
function __m128 (line 873) | inline __m128 exp(__m128 x) { return fastexp_float(x); }
function __m128d (line 874) | inline __m128d exp(__m128d x) { return fastexp_double(x); }
function __m256 (line 878) | inline __m256 exp(__m256 x) { return fastexp_float(x); }
function __m256d (line 879) | inline __m256d exp(__m256d x) { return fastexp_double(x); }
function __m512 (line 883) | inline __m512 exp(__m512 x) { return fastexp_float(x); }
function __m512d (line 884) | inline __m512d exp(__m512d x) { return fastexp_double(x); }
function float32x4_t (line 888) | inline float32x4_t exp(float32x4_t x) { return fastexp_float(x); }
function float64x2_t (line 889) | inline float64x2_t exp(float64x2_t x) { return fastexp_double(x); }
function exp (line 894) | inline float exp(float x) { return quick_e::fastexp_float(x); }
function exp (line 895) | inline double exp(double x) { return quick_e::fastexp_double(x); }
function exp (line 899) | inline float exp(float x) { return std::exp(x); }
function exp (line 900) | inline double exp(double x) { return std::exp(x); }
FILE: include/adept/reduce.h
function namespace (line 47) | namespace adept {
function namespace (line 100) | namespace internal {
function finish_active (line 172) | void finish_active(Active<T>& total, const Index& n) {
type T (line 181) | typedef T total_type;
function T (line 186) | T first_value() { return 0; }
type T (line 214) | typedef T total_type;
function T (line 219) | T first_value() { return 1; }
function finish_active (line 242) | void finish_active(Active<T>& total, const Index& n) { }
type T (line 248) | typedef T total_type;
function T (line 254) | T first_value() { return internal::numeric_limits<T>::min_inf(); }
function accumulate (line 256) | void accumulate(T& total, const T& rhs) {
function accumulate (line 266) | void accumulate(T& total, const T& rhs) {
function accumulate (line 276) | void accumulate(Packet<T>& total, const Packet<T>& rhs) { total = fmax(t...
function finish_active (line 299) | void finish_active(Active<T>& total, const Index& n) { }
type T (line 305) | typedef T total_type;
function T (line 310) | T first_value() { return internal::numeric_limits<T>::max_inf(); }
function accumulate (line 312) | void accumulate(T& total, const T& rhs) {
function accumulate_packet2 (line 316) | void accumulate_packet2(T& total, const Packet<T>& ptotal) {
function accumulate (line 321) | void accumulate(T& total, const T& rhs) {
function accumulate_packet2 (line 325) | void accumulate_packet2(T& total, const Packet<T>& ptotal) {
function accumulate (line 330) | void accumulate(Packet<T>& total, const Packet<T>& rhs) { total = fmin(t...
function finish_active (line 351) | void finish_active(Active<T>& total, const Index& n) { }
type T (line 358) | typedef T total_type;
function T (line 363) | T first_value() { return 0; }
function finish_active (line 392) | void finish_active(Active<T>& total, const Index& n) {
function first_value (line 408) | struct All {
function accumulate (line 413) | void accumulate(bool& total, const bool& rhs)
function first_value (line 422) | struct Any {
function accumulate (line 427) | void accumulate(bool& total, const bool& rhs)
function first_value (line 435) | struct Count {
function accumulate (line 440) | void accumulate(Index& total, const bool& rhs)
function reduce_dimension (line 607) | void reduce_dimension(const Expression<Type, E>& rhs, int reduce_dim,
function reduce_active (line 714) | void reduce_active(const Expression<Type, E>& rhs, Active<Type>& total) {
function reduce_dimension (line 772) | void reduce_dimension(const Expression<Type, E>& rhs, int reduce_dim,
function Index (line 995) | Index count(const Expression<bool, E>& rhs)
FILE: include/adept/scalar_shortcuts.h
function namespace (line 20) | namespace adept {
function namespace (line 40) | namespace adept {
FILE: include/adept/settings.h
function namespace (line 16) | namespace adept {
FILE: include/adept/solve.h
function namespace (line 18) | namespace adept {
type typename (line 74) | typedef typename internal::promote<LType,RType>::type PType;
type typename (line 90) | typedef typename internal::promote<LType,RType>::type PType;
FILE: include/adept/spread.h
function namespace (line 15) | namespace adept {
FILE: include/adept/store_transpose.h
function namespace (line 22) | namespace adept {
FILE: include/adept/traits.h
function namespace (line 25) | namespace adept {
type null_type (line 228) | struct null_type { }
function null_type (line 233) | struct is_null_type<null_type>{
type T (line 279) | typedef T type;
type T (line 283) | typedef T type;
type S (line 299) | typedef S type;
type typename (line 302) | typedef typename S::type type;
type typename (line 329) | typedef typename if_then_else<A_float_B_int, A, B>::type float_type;
type typename (line 330) | typedef typename if_then_else<A_bigger_than_B, A, B>::type biggest_type;
type typename (line 331) | typedef typename if_then_else<prefer_float, float_type, biggest_type>::t...
type typename (line 334) | typedef typename promote_primitive<
type typename (line 337) | typedef typename if_then_else<is_complex<L>::value
type T (line 351) | typedef T type;
type T (line 385) | typedef T type;
type T (line 386) | typedef T type;
type T (line 414) | typedef T type;
type typename (line 420) | typedef typename initializer_list_rank<T>::type type;
function float (line 434) | struct matrix_op_defined<float> { static const bool value = true; }
function double (line 437) | struct matrix_op_defined<double> { static const bool value = true; }
function float (line 447) | struct is_floating_point<float> { static const bool value = true; }
function double (line 449) | struct is_floating_point<double> { static const bool value = true; }
type is_floating_point (line 451) | struct is_floating_point
FILE: include/adept/vector_utilities.h
function namespace (line 16) | namespace adept {
FILE: include/adept/where.h
type no (line 60) | typedef struct { char array[2]; } no;
FILE: include/adept_fortran.h
function namespace (line 87) | namespace adept {
FILE: test/algorithm.cpp
function adouble (line 22) | adouble algorithm(const adouble x[2]) {
FILE: test/rosenbrock_banana_function.cpp
function adouble (line 18) | adouble State::calc_function_value(const adouble* x) {
FILE: test/simulate_radiances.cpp
function aReal (line 23) | static
function simulate_radiances (line 51) | void
FILE: test/state.cpp
function my_function_value (line 23) | double my_function_value(const gsl_vector* x, void* params) {
function my_function_gradient (line 29) | void my_function_gradient(const gsl_vector* x, void* params, gsl_vector*...
function my_function_value_and_gradient (line 35) | void my_function_value_and_gradient(const gsl_vector* x, void* params,
FILE: test/state.h
function class (line 15) | class State {
FILE: test/test_adept.cpp
function main (line 19) | int
FILE: test/test_adept_with_and_without_ad.cpp
function main (line 27) | int
FILE: test/test_array_derivatives.cpp
function algorithm (line 20) | void algorithm(const A& x, S& y) {
function main (line 30) | int
FILE: test/test_array_speed.cpp
function main (line 15) | int main()
FILE: test/test_arrays.cpp
function main (line 41) | int
FILE: test/test_checkpoint.cpp
function toon (line 27) | static
function main (line 41) | int
FILE: test/test_constructors.cpp
function Vector (line 23) | Vector square(const Vector& v) {
function square_in_place (line 28) | void square_in_place(Vector& v) {
function Vector (line 33) | Vector square_copy(Vector v) {
function main (line 79) | int
FILE: test/test_derivatives.cpp
function main (line 95) | int
FILE: test/test_fastexp.cpp
function main (line 20) | int main(int argc, const char** argv)
FILE: test/test_fixed_arrays.cpp
function main (line 26) | int
FILE: test/test_gsl_interface.cpp
function main (line 21) | int
FILE: test/test_interp.cpp
function main (line 29) | int
FILE: test/test_minimizer.cpp
class RosenbrockN (line 22) | class RosenbrockN : public Optimizable {
method RosenbrockN (line 25) | RosenbrockN() : ls_iteration_(0), exact_hessian_(false) {}
method calc_y (line 40) | Array<1,Real,IsActive> calc_y(const Array<1,Real,IsActive>& x) {
method calc_exact_hessian (line 51) | void calc_exact_hessian(const adept::Vector& x, SymmMatrix& hessian) {
method report_progress (line 63) | virtual void report_progress(int niter, const adept::Vector& x,
method state_to_stderr (line 70) | void state_to_stderr(const adept::Vector& x, Real cost) {
method final_state_to_stderr (line 81) | void final_state_to_stderr(const adept::Vector& x, Real cost) {
method provides_derivative (line 86) | virtual bool provides_derivative(int order) {
method Real (line 95) | virtual Real calc_cost_function(const Vector& x) {
method Real (line 103) | virtual Real calc_cost_function_gradient(const Vector& x,
method Real (line 117) | virtual Real calc_cost_function_gradient_hessian(const Vector& x,
function main (line 141) | int
FILE: test/test_misc.cpp
function algorithm_ad (line 16) | double algorithm_ad(const double x_val[2], // Input values
function main (line 32) | int main()
FILE: test/test_no_lib.cpp
function algorithm_ad (line 24) | double algorithm_ad(const double x_val[2], // Input values
function main (line 40) | int main()
FILE: test/test_packet_operations.cpp
function p2v (line 21) | Array<1,Type> p2v(internal::Packet<Type> p) {
function test_packet_operations (line 28) | void test_packet_operations() {
function test_vector_operations (line 55) | void test_vector_operations(int N) {
function test_unaligned_reduce (line 78) | void test_unaligned_reduce(int N) {
function test_unaligned_assign (line 89) | void test_unaligned_assign(int N) {
function main (line 106) | int
FILE: test/test_radiances.cpp
function simulate_radiances_wrapper (line 19) | void simulate_radiances_wrapper(int n,
function main (line 70) | int
FILE: test/test_radiances_array.cpp
function simulate_radiances_wrapper (line 23) | void simulate_radiances_wrapper(int n,
function main (line 78) | int
FILE: test/test_reduce_active.cpp
function main (line 39) | int
FILE: test/test_thread_safe.cpp
function toon (line 36) | static
function compute (line 51) | static
function main (line 133) | int
FILE: test/test_thread_safe_arrays.cpp
function main (line 20) | int main(int argc, const char** argv)
Condensed preview — 136 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (1,657K chars).
[
{
"path": ".gitignore",
"chars": 352,
"preview": "Makefile.in\n/aclocal.m4\n/config.guess\n/config.h.in\n/config.log\n/config.sub\n/config.status\n/configure\n/depcomp\n/install-s"
},
{
"path": ".travis.yml",
"chars": 250,
"preview": "language: cpp\nos: linux\nsudo: required\ndist: trusty\ncompiler:\n - gcc\nbefore_install:\n - sudo apt-get install gfortran "
},
{
"path": "AUTHORS",
"chars": 33,
"preview": "Robin Hogan <r.j.hogan@ecmwf.int>"
},
{
"path": "COPYING",
"chars": 11358,
"preview": "\n Apache License\n Version 2.0, January 2004\n "
},
{
"path": "ChangeLog",
"chars": 13154,
"preview": "version 2.1.4 (in progress)\n\t- Added support for the copysign function\n\t- Added aArray::set_gradient(Array) function\n\nve"
},
{
"path": "INSTALL",
"chars": 9512,
"preview": "Installation Instructions\n*************************\n\nCopyright (C) 1994, 1995, 1996, 1999, 2000, 2001, 2002, 2004, 2005,"
},
{
"path": "Makefile.am",
"chars": 386,
"preview": "dist_pkgdata_DATA = README.md\npkgdata_DATA = COPYING ChangeLog NEWS AUTHORS\nSUBDIRS = adept include benchmark test\n# The"
},
{
"path": "NEWS",
"chars": 883,
"preview": "version 2.0\n\t- Fixed pausable recording and library-free compilation to provide full backwards compatibility with versio"
},
{
"path": "README.md",
"chars": 3853,
"preview": "# Adept 2: Combined array and automatic differentiation library in C++\n\n## Introduction\n\nThe Adept version 2.1 software "
},
{
"path": "TODO",
"chars": 4394,
"preview": "BUGS\nspread<DIM> function does not use the right DIM\n\nDESIRABLE BUT NEEDS NEW STACK\nDifferentiated BLAS operations on sy"
},
{
"path": "adept/Array.cpp",
"chars": 3432,
"preview": "/* Array.cpp -- Functions and global variables controlling array behaviour\n\n Copyright (C) 2015-2016 European Centre "
},
{
"path": "adept/Makefile.am",
"chars": 394,
"preview": "lib_LTLIBRARIES = libadept.la\nlibadept_la_SOURCES = Array.cpp Stack.cpp StackStorageOrig.cpp \\\n\tjacobian.cpp Storage.cpp"
},
{
"path": "adept/Minimizer.cpp",
"chars": 5829,
"preview": "/* Minimizer.h -- class for minimizing the cost function of an optimizable object\n\n Copyright (C) 2020 European Centr"
},
{
"path": "adept/Stack.cpp",
"chars": 17862,
"preview": "/* Stack.cpp -- Stack for storing automatic differentiation information\n\n Copyright (C) 2012-2014 University of Read"
},
{
"path": "adept/StackStorageOrig.cpp",
"chars": 2399,
"preview": "/* StackStorageOrig.cpp -- Original storage of stacks using STL containers\n\n Copyright (C) 2014-2015 University of Re"
},
{
"path": "adept/Storage.cpp",
"chars": 391,
"preview": "/* Storage.cpp -- Global variables recording use of Storage objects\n\n Copyright (C) 2015 European Centre for Medium-R"
},
{
"path": "adept/cppblas.cpp",
"chars": 10694,
"preview": "/* cppblas.cpp -- C++ interface to BLAS functions\n\n Copyright (C) 2015-2016 European Centre for Medium-Range Weather "
},
{
"path": "adept/cpplapack.h",
"chars": 6789,
"preview": "/* cpplapack.h -- C++ interface to LAPACK\n\n Copyright (C) 2015-2016 European Centre for Medium-Range Weather Forecast"
},
{
"path": "adept/index.cpp",
"chars": 350,
"preview": "/* index.cpp -- Definitions of \"end\" and \"__\" for array indexing\n\n Copyright (C) 2015 European Centre for Medium-Rang"
},
{
"path": "adept/inv.cpp",
"chars": 5264,
"preview": "/* inv.cpp -- Invert matrices\n\n Copyright (C) 2015-2016 European Centre for Medium-Range Weather Forecasts\n\n Autho"
},
{
"path": "adept/jacobian.cpp",
"chars": 25467,
"preview": "/* jacobian.cpp -- Computation of Jacobian matrix\n\n Copyright (C) 2012-2014 University of Reading\n Copyright (C) 2"
},
{
"path": "adept/line_search.cpp",
"chars": 10087,
"preview": "/* line_search.cpp -- Approximate minimization of function along a line\n\n Copyright (C) 2020 European Centre for Medi"
},
{
"path": "adept/minimize_conjugate_gradient.cpp",
"chars": 15280,
"preview": "/* minimize_conjugate_gradient.cpp -- Minimize function using Conjugate Gradient algorithm\n\n Copyright (C) 2020 Europ"
},
{
"path": "adept/minimize_levenberg_marquardt.cpp",
"chars": 15597,
"preview": "/* minimize_levenberg_marquardt.cpp -- Minimize function using Levenberg-Marquardt algorithm\n\n Copyright (C) 2020 Eur"
},
{
"path": "adept/minimize_limited_memory_bfgs.cpp",
"chars": 18065,
"preview": "/* minimize_limited_memory_bfgs.cpp -- Minimize function using Limited-Memory BFGS algorithm\n\n Copyright (C) 2020 Eur"
},
{
"path": "adept/settings.cpp",
"chars": 4024,
"preview": "/* settings.cpp -- View/change the overall Adept settings\n\n Copyright (C) 2016 European Centre for Medium-Range Weath"
},
{
"path": "adept/solve.cpp",
"chars": 9071,
"preview": "/* solve.cpp -- Solve systems of linear equations using LAPACK\n\n Copyright (C) 2015-2016 European Centre for Medium-R"
},
{
"path": "adept/vector_utilities.cpp",
"chars": 707,
"preview": "/* vector_utilities.cpp -- Vector utility functions\n\n Copyright (C) 2016 European Centre for Medium-Range Weather For"
},
{
"path": "benchmark/Makefile.am",
"chars": 863,
"preview": "check_PROGRAMS = autodiff_benchmark animate matrix_benchmark math_benchmark\nautodiff_benchmark_SOURCES = autodiff_benchm"
},
{
"path": "benchmark/advection_schemes.h",
"chars": 4053,
"preview": "/* advection_schemes.h - Two test advection algorithms from the Adept paper\n\n Copyright (C) 2014 The University of Read"
},
{
"path": "benchmark/advection_schemes_AD.h",
"chars": 3882,
"preview": "/* advection_schemes_AD.h - Header for the hand-coded adjoints\n\n Copyright (C) 2014 The University of Reading\n Copyrig"
},
{
"path": "benchmark/advection_schemes_K.h",
"chars": 3838,
"preview": "/* advection_schemes_K.h - Header for hand-coded Jacobians\n\n Copyright (C) 2014 The University of Reading\n\n Copying an"
},
{
"path": "benchmark/animate.cpp",
"chars": 1757,
"preview": "/* animate.cpp - Visualize the advection\n\n Copyright (C) 2014 The University of Reading\n\n Copying and distribution of "
},
{
"path": "benchmark/autodiff_benchmark.cpp",
"chars": 18385,
"preview": "/* autodiff_benchmark.cpp - Program to benchmark different automatic differentiation tools\n\n Copyright (C) 2014 The Uni"
},
{
"path": "benchmark/differentiator.h",
"chars": 20164,
"preview": "/* differentiator.h\n\n Copyright (C) 2014 The University of Reading\n\n Copying and distribution of this file, with or wi"
},
{
"path": "benchmark/math_benchmark.cpp",
"chars": 2715,
"preview": "/* math_benchmark.cpp - Benchmark mathematical functions\n\n Copyright (C) 2023 ECMWF\n\n Copying and distribution of this"
},
{
"path": "benchmark/matrix_benchmark.cpp",
"chars": 3105,
"preview": "#include <iostream>\n\n#include <adept_arrays.h>\n\n#include \"Timer.h\"\n\ntemplate<bool IsActive>\ndouble\ntime_matmul(int n, in"
},
{
"path": "benchmark/nx.h",
"chars": 33,
"preview": "#ifndef NX\n#define NX 100\n#endif\n"
},
{
"path": "config_platform_independent.h.in",
"chars": 602,
"preview": "/* config_platform_independent.h.in. */\n\n/* Name of package */\n#undef PACKAGE\n\n/* Define to the address where bug report"
},
{
"path": "configure.ac",
"chars": 6935,
"preview": "# Configure autoconf for the Adept library\n\n### GENERAL CONFIGURATION ###\n\nAC_PREREQ([2.61])\nAC_INIT([adept], [2.1.3], ["
},
{
"path": "doc/COPYING",
"chars": 22962,
"preview": "\n GNU Free Documentation License\n Version 1.3, 3 November 2008\n\n\n Copyright (C) 2000, 200"
},
{
"path": "doc/Makefile",
"chars": 475,
"preview": "# If you have pdflatex installed, type \"make\" to create the\n# documentation, \"make clean\" to delete it\n\ndocumentation: a"
},
{
"path": "doc/README",
"chars": 792,
"preview": "This directory contains the LaTeX source files for the Adept User\nGuide and Adept Reference Sheet\n\nType \"make\" to create"
},
{
"path": "doc/adept_documentation.tex",
"chars": 219645,
"preview": "% \n% Adept automatic differentiation library for C++: User guide\n%\n% Type \"pdflatex adept_documentation.tex\" twice to re"
},
{
"path": "doc/adept_reference.tex",
"chars": 23767,
"preview": "\\documentclass[10pt,a4,landscape]{article}\n% Page set up\n\\setlength{\\oddsidemargin}{-1cm} %{0.5cm}\n\\setlength{\\evensidem"
},
{
"path": "include/Makefile.am",
"chars": 1246,
"preview": "include_HEADERS = adept.h adept_arrays.h adept_optimize.h adept_source.h adept_fortran.h\n\npkginclude_HEADERS = adept/Act"
},
{
"path": "include/Timer.h",
"chars": 4365,
"preview": "/* Timer.h - Utility class for timing different parts of a program\n\n Copyright (C) 2012-2014 The University of Reading\n"
},
{
"path": "include/adept/Active.h",
"chars": 22919,
"preview": "/* Active.h -- Active scalar type for automatic differentiation\n\n Copyright (C) 2012-2014 University of Reading\n C"
},
{
"path": "include/adept/ActiveConstReference.h",
"chars": 13409,
"preview": "/* ActiveConstReference.h -- Const reference to an active element of an array\n\n Copyright (C) 2015-2017 European Cent"
},
{
"path": "include/adept/ActiveReference.h",
"chars": 17693,
"preview": "/* ActiveReference.h -- Reference to an active element of an array\n\n Copyright (C) 2015-2018 European Centre for Medi"
},
{
"path": "include/adept/Allocator.h",
"chars": 6088,
"preview": "/* Allocator.h -- Allocates elements to arrays\n\n Copyright (C) 2015 European Centre for Medium-Range Weather Forecast"
},
{
"path": "include/adept/Array.h",
"chars": 127972,
"preview": "/* Array.h -- active or inactive Array of arbitrary rank\n\n Copyright (C) 2014-2021 European Centre for Medium-Range W"
},
{
"path": "include/adept/ArrayWrapper.h",
"chars": 5552,
"preview": "/* ArrayWrapper.h -- Make Arrays work faster in expressions\n\n Copyright (C) 2016-2017 European Centre for Medium-Rang"
},
{
"path": "include/adept/BinaryOperation.h",
"chars": 73570,
"preview": "/* BinaryOperation.h -- Binary operations on Adept expressions\n\n Copyright (C) 2014-2018 European Centre for Medium-R"
},
{
"path": "include/adept/Expression.h",
"chars": 15506,
"preview": "/* Expression.h -- Base class for arrays and active objects\n\n Copyright (C) 2014-2017 European Centre for Medium-Rang"
},
{
"path": "include/adept/ExpressionSize.h",
"chars": 8069,
"preview": "/* ExpressionSize.h -- Class for describing array sizes\n\n Copyright (C) 2014-2017 European Centre for Medium-Range We"
},
{
"path": "include/adept/FixedArray.h",
"chars": 92193,
"preview": "/* FixedArray.h -- active or inactive FixedArray of arbitrary rank\n\n Copyright (C) 2014-2017 European Centre for Medi"
},
{
"path": "include/adept/GradientIndex.h",
"chars": 2055,
"preview": "\n\n#ifndef AdeptGradientIndex_H\n#define AdeptGradientIndex_H 1\n\n#include <adept/Stack.h>\n\nnamespace adept {\n namespace i"
},
{
"path": "include/adept/IndexedArray.h",
"chars": 36752,
"preview": "/* IndexedArray.h -- Support for indexed arrays\n\n Copyright (C) 2015-2018 European Centre for Medium-Range Weather Fo"
},
{
"path": "include/adept/Minimizer.h",
"chars": 13383,
"preview": "/* Minimizer.h -- class for minimizing the cost function of an optimizable object\n\n Copyright (C) 2020 European Centr"
},
{
"path": "include/adept/Optimizable.h",
"chars": 3324,
"preview": "/* Optimizable.h -- abstract base classes representing an optimization problem\n\n Copyright (C) 2020 European Centre f"
},
{
"path": "include/adept/Packet.h",
"chars": 10876,
"preview": "/* Packet.h -- Vectorization support\n\n Copyright (C) 2016-2020 European Centre for Medium-Range Weather Forecasts\n\n "
},
{
"path": "include/adept/RangeIndex.h",
"chars": 14550,
"preview": "/* RangeIndex.h -- Helper classes to enable indexing of arrays\n\n Copyright (C) 2015-2017 European Centre for Medium-R"
},
{
"path": "include/adept/ScratchVector.h",
"chars": 3511,
"preview": "/* ScratchVector.h -- Class for holding temporary real data\n\n Copyright (C) 2015-2017 European Centre for Medium-Rang"
},
{
"path": "include/adept/SpecialMatrix.h",
"chars": 69188,
"preview": "/* SpecialMatrix.h -- Active or inactive symmetric and band-diagonal matrices\n\n Copyright (C) 2015-2017 European Cent"
},
{
"path": "include/adept/Stack.h",
"chars": 31787,
"preview": "/* Stack.h -- Storage of automatic differentiation information\n\n Copyright (C) 2012-2014 University of Reading\n Co"
},
{
"path": "include/adept/StackStorage.h",
"chars": 5951,
"preview": "/* StackStorage.h -- Storage of statement & operation stacks\n\n Copyright (C) 2012-2014 University of Reading\n Copy"
},
{
"path": "include/adept/StackStorageOrig.h",
"chars": 5697,
"preview": "/* StackStorageOrig.h -- Original method to store statement & operation stacks\n\n Copyright (C) 2014-2015 University o"
},
{
"path": "include/adept/StackStorageOrigStl.h",
"chars": 4792,
"preview": "/* StackStorageOrigStl.h -- Original storage of stacks using STL containers\n\n Copyright (C) 2014-2015 University of R"
},
{
"path": "include/adept/Statement.h",
"chars": 864,
"preview": "/* Statement.h -- Original method to store statement & operation stacks\n\n Copyright (C) 2012-2014 University of Readi"
},
{
"path": "include/adept/Storage.h",
"chars": 6967,
"preview": "/* Storage.h -- store array of active or inactive data\n\n Copyright (C) 2012-2014 University of Reading\n Copyright "
},
{
"path": "include/adept/UnaryOperation.h",
"chars": 20355,
"preview": "/* UnaryOperation.h -- Unary operations on Adept expressions\n\n Copyright (C) 2014-2020 European Centre for Medium-Ran"
},
{
"path": "include/adept/array_shortcuts.h",
"chars": 4629,
"preview": "/* array_shortcuts.h -- Definitions of \"shortcut\" typedefs for array types\n\n Copyright (C) 2015-2017 European Centre "
},
{
"path": "include/adept/base.h",
"chars": 14261,
"preview": "/* base.h -- Basic definitions \n\n Copyright (C) 2012-2014 University of Reading\n Copyright (C) 2015-2021 European "
},
{
"path": "include/adept/contiguous_matrix.h",
"chars": 1861,
"preview": "/* contiguous_matrix.h -- Return matrix with contiguous storage\n\n Copyright (C) 2015 European Centre for Medium-Range"
},
{
"path": "include/adept/cppblas.h",
"chars": 3490,
"preview": "/* cppblas.h -- C++ interface to BLAS functions\n\n Copyright (C) 2015-2016 European Centre for Medium-Range Weather Fo"
},
{
"path": "include/adept/eval.h",
"chars": 1138,
"preview": "/* eval.h -- Convert expression to array to avoid aliasing issues\n\n Copyright (C) 2017 European Centre for Medium-Ran"
},
{
"path": "include/adept/exception.h",
"chars": 6445,
"preview": "/* exception.h -- Exceptions thrown by Adept library\n\n Copyright (C) 2012-2014 University of Reading\n Copyright (C"
},
{
"path": "include/adept/interp.h",
"chars": 22871,
"preview": "/* interp.h -- 1D interpolation\n\n Copyright (C) 2015- European Centre for Medium-Range Weather Forecasts\n\n Author:"
},
{
"path": "include/adept/inv.h",
"chars": 1493,
"preview": "/* inv.h -- Invert matrices\n\n Copyright (C) 2015 European Centre for Medium-Range Weather Forecasts\n\n Author: Robi"
},
{
"path": "include/adept/matmul.h",
"chars": 25257,
"preview": "/* matmul.h -- Matrix multiplication capability\n\n Copyright (C) 2015-2017 European Centre for Medium-Range Weather Fo"
},
{
"path": "include/adept/noalias.h",
"chars": 4252,
"preview": "/* noalias.h -- Wrap an expression so that alias checking is not performed\n\n Copyright (C) 2017 European Centre for M"
},
{
"path": "include/adept/outer_product.h",
"chars": 9143,
"preview": "/* outer_product.h -- Compute the outer product of two vectors\n\n Copyright (C) 2017 European Centre for Medium-Range "
},
{
"path": "include/adept/quick_e.h",
"chars": 34299,
"preview": "/* quick_e.h -- Fast exponential function for Intel and ARM intrinsics\n\n Copyright (C) 2020 European Centre for Medium"
},
{
"path": "include/adept/reduce.h",
"chars": 40782,
"preview": "/* reduce.h -- \"Reduce\" functions such as find, all, sum etc.\n\n Copyright (C) 2015-2017 European Centre for Medium-Ra"
},
{
"path": "include/adept/scalar_shortcuts.h",
"chars": 1390,
"preview": "/* shortcuts.h -- Definitions of \"shortcut\" typedefs for scalar types\n\n Copyright (C) 2015 European Centre for Medium"
},
{
"path": "include/adept/settings.h",
"chars": 1995,
"preview": "/* settings.h -- View/change the overall Adept settings\n\n Copyright (C) 2016-2017 European Centre for Medium-Range We"
},
{
"path": "include/adept/solve.h",
"chars": 3921,
"preview": "/* solve.h -- Solve systems of linear equations\n\n Copyright (C) 2015 European Centre for Medium-Range Weather Forecas"
},
{
"path": "include/adept/spread.h",
"chars": 6319,
"preview": "/* spread.h -- Spread an array into an additional dimension\n\n Copyright (C) 2017 European Centre for Medium-Range Wea"
},
{
"path": "include/adept/store_transpose.h",
"chars": 985,
"preview": "/* store_transpose.h -- Store the transpose of a vector of Packets\n\n Copyright (C) 2017 European Centre for Medium-Ra"
},
{
"path": "include/adept/traits.h",
"chars": 16481,
"preview": "/* traits.h -- Traits used to support array/automatic differentiation expressions\n\n Copyright (C) 2012-2014 Universit"
},
{
"path": "include/adept/vector_utilities.h",
"chars": 400,
"preview": "/* vector_utilities.h -- Vector utility functions\n\n Copyright (C) 2016 European Centre for Medium-Range Weather Forec"
},
{
"path": "include/adept/where.h",
"chars": 3857,
"preview": "/* where.h -- Support for Fortran-90-like \"where\" construct\n\n Copyright (C) 2015 European Centre for Medium-Range Wea"
},
{
"path": "include/adept.h",
"chars": 479,
"preview": "/* adept.h -- Header file for basic scalar functionality of Adept automatic differentiation library\n\n Copyright (C) 2"
},
{
"path": "include/adept_arrays.h",
"chars": 716,
"preview": "/* adept_arrays.h -- Header file for array functionality of Adept automatic differentiation library\n\n Copyright (C) 2"
},
{
"path": "include/adept_fortran.h",
"chars": 10307,
"preview": "/* adept_fortran.h -- Interoperability between Adept and Fortran-90 arrays\n\n Copyright (C) 2020 European Centre for M"
},
{
"path": "include/adept_optimize.h",
"chars": 398,
"preview": "/* adept_optimize.h -- Header file for optimization algorithms of Adept library\n\n Copyright (C) 2020 European Centre "
},
{
"path": "include/create_adept_source_header",
"chars": 3390,
"preview": "#!/bin/sh\n# This script creates a header file \"adept_source.h\" containing the\n# ../adept/*.h ../adept/*.cpp source files"
},
{
"path": "m4/adept.m4",
"chars": 3898,
"preview": "# ---------------------------------------------------------------------------\n# FILE : adept.m4\n# COPYRIGHT :"
},
{
"path": "m4/ax_blas.m4",
"chars": 8105,
"preview": "# ===========================================================================\n# http://www.gnu.org/software/aut"
},
{
"path": "m4/ax_lapack.m4",
"chars": 5040,
"preview": "# ===========================================================================\n# http://www.gnu.org/software/auto"
},
{
"path": "m4/ltsugar.m4",
"chars": 4372,
"preview": "# ltsugar.m4 -- libtool m4 base layer. -*-Autoconf-*-\n#\n# Copyright (C) 2004, 2005, 2007, 2008 F"
},
{
"path": "m4/lt~obsolete.m4",
"chars": 6126,
"preview": "# lt~obsolete.m4 -- aclocal satisfying obsolete definitions. -*-Autoconf-*-\n#\n# Copyright (C) 2004, 2005, 2007, 200"
},
{
"path": "makefile_include.in",
"chars": 408,
"preview": "# Template for configure to create makefile_include, which is included\n# by test/Makefile and benchmark/Makefile\n\nAR = @"
},
{
"path": "test/Makefile",
"chars": 7838,
"preview": "# Makefile for example programs that demonstrate different features of\n# the Adept library\n#\n# Note that this Makefile i"
},
{
"path": "test/README",
"chars": 11156,
"preview": "This directory contains examples to demonstrate various features of\nAdept. Type \"make check\" from the directory above to"
},
{
"path": "test/algorithm.cpp",
"chars": 819,
"preview": "/* algorithm.cpp - A simple demonstration algorithm used in Tests 1 & 2 \n\n Copyright (C) 2012-2014 The University of Re"
},
{
"path": "test/algorithm.h",
"chars": 750,
"preview": "/* algorithm.h - Header file for the simple example algorithm function\n\n Copyright (C) 2012-2014 The University of Read"
},
{
"path": "test/algorithm_with_and_without_ad.h",
"chars": 1341,
"preview": "/* algorithm_with_and_without_ad.h\n\n Copyright (C) 2012-2014 The University of Reading\n\n Copying and distribution of t"
},
{
"path": "test/rosenbrock_banana_function.cpp",
"chars": 807,
"preview": "/* rosenbrock_banana_function.cpp - N-dimensional Rosenbrock function\n\n Copyright (C) 2012-2014 The University of Readi"
},
{
"path": "test/run_tests.sh",
"chars": 1068,
"preview": "#!/bin/sh\n\n# Simple script to run all programs provided to it and report whether\n# they succeed or fail\n\nLOG=test_result"
},
{
"path": "test/simulate_radiances.cpp",
"chars": 4859,
"preview": "/* simulate_radiances.cpp - provides a function taking inactive arguments that returns also Jacobian matrices\n\n Copyrig"
},
{
"path": "test/simulate_radiances.h",
"chars": 732,
"preview": "/* simulate_radiances.h - a function taking inactive arguments that returns also Jacobian matrices\n\n Copyright (C) 2012"
},
{
"path": "test/state.cpp",
"chars": 5487,
"preview": "/* state.cpp - An object-oriented interface to an Adept-based minimizer\n\n Copyright (C) 2012-2014 The University of Rea"
},
{
"path": "test/state.h",
"chars": 1501,
"preview": "/* state.h - An object-oriented interface to an Adept-based minimizer\n\n Copyright (C) 2012-2014 The University of Readi"
},
{
"path": "test/test_adept.cpp",
"chars": 5613,
"preview": "/* test_adept.cpp - Demonstration of basic features of Adept\n\n Copyright (C) 2012-2014 The University of Reading\n\n Cop"
},
{
"path": "test/test_adept_with_and_without_ad.cpp",
"chars": 5169,
"preview": "/* test_adept_with_and_without_ad.cpp\n\n Copyright (C) 2012-2014 The University of Reading\n\n Copying and distribution o"
},
{
"path": "test/test_array_derivatives.cpp",
"chars": 5503,
"preview": "/* test_array_derivatives.cpp - Test derivatives of array expressions\n\n Copyright (C) 2017 European Centre for Medium"
},
{
"path": "test/test_array_speed.cpp",
"chars": 4462,
"preview": "#include <iostream>\n#define ADEPT_NO_AUTOMATIC_DIFFERENTIATION\n#define ADEPT_REAL_TYPE_SIZE 4\n#include <adept_arrays.h>\n"
},
{
"path": "test/test_arrays.cpp",
"chars": 37843,
"preview": "/* test_arrays.cpp - Test Adept's array functionality\n\n Copyright (C) 2016-2018 European Centre for Medium-Range Weat"
},
{
"path": "test/test_checkpoint.cpp",
"chars": 7817,
"preview": "/* test_checkpoint.cpp - Test manual checkpointing of a simulation\n\n Copyright (C) 2012-2014 The University of Reading\n"
},
{
"path": "test/test_constructors.cpp",
"chars": 8814,
"preview": "/* test_constructors.cpp - Test Adept's selection of constructors in a range of scenarios\n\n Copyright (C) 2017 Europe"
},
{
"path": "test/test_derivatives.cpp",
"chars": 5830,
"preview": "/* test_derivatives.cpp - Test derivatives of mathematical functions \n\n Copyright (C) 2017 European Centre for Medium"
},
{
"path": "test/test_fastexp.cpp",
"chars": 1887,
"preview": "/* test_fastexp.cpp - Test Adept's fast exponential for correctness \n\n Copyright (C) 2020 European Centre for Medium-Ra"
},
{
"path": "test/test_fixed_arrays.cpp",
"chars": 18673,
"preview": "/* test_arrays.cpp - Test Adept's array functionality\n\n Copyright (C) 2016-2017 European Centre for Medium-Range Weat"
},
{
"path": "test/test_gsl_interface.cpp",
"chars": 1414,
"preview": "/* test_gsl_interface.cpp - \"main\" function for Test 4\n\n Copyright (C) 2012-2014 The University of Reading\n\n Copying a"
},
{
"path": "test/test_interp.cpp",
"chars": 6553,
"preview": "/* test_interp.cpp\n\n Copyright (C) 2024- European Centre for Medium-Range Weather Forecasts\n\n Copying and distribution"
},
{
"path": "test/test_minimizer.cpp",
"chars": 7120,
"preview": "/* test_minimizer.cpp - Test Adept minimizer with N-dimensional Rosenbrock function\n\n Copyright (C) 2020-2022 ECMWF\n\n "
},
{
"path": "test/test_misc.cpp",
"chars": 1822,
"preview": "/* test_misc.cpp\n\n Copyright (C) 2012-2014 The University of Reading\n\n Copying and distribution of this file, with or "
},
{
"path": "test/test_no_lib.cpp",
"chars": 2237,
"preview": "/* test_no_lib.cpp\n\n Copyright (C) 2012-2014 The University of Reading\n\n Copying and distribution of this file, with o"
},
{
"path": "test/test_packet_operations.cpp",
"chars": 4307,
"preview": "/* test_packet_operations.cpp\n\n Copyright (C) 2020 European Centre for Medium-Range Weather Forecasts\n\n Copying and di"
},
{
"path": "test/test_radiances.cpp",
"chars": 5059,
"preview": "/* test_radiances.cpp - \"main\" function for Test 3\n\n Copyright (C) 2012-2014 The University of Reading\n\n Copying and d"
},
{
"path": "test/test_radiances_array.cpp",
"chars": 5236,
"preview": "/* test_radiances.cpp - \"main\" function for Test 3\n\n Copyright (C) 2012-2014 The University of Reading\n Copyright (C) "
},
{
"path": "test/test_reduce_active.cpp",
"chars": 2379,
"preview": "/* test_reduce_active.cpp\n\n Copyright (C) 2020 European Centre for Medium-Range Weather Forecasts\n\n Copying and distri"
},
{
"path": "test/test_thread_safe.cpp",
"chars": 6390,
"preview": "/* test_thread_safe.cpp - Tests that Adept is thread-safe\n\n Copyright (C) 2012-2014 The University of Reading\n\n Copyin"
},
{
"path": "test/test_thread_safe_arrays.cpp",
"chars": 1964,
"preview": "/* test_thread_safe_arrays.cpp - Tests that Adept arrays are thread-safe\n\n Copyright (C) 2017 ECMWF\n\n Copying and dist"
}
]
About this extraction
This page contains the full source code of the rjhogan/Adept-2 GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 136 files (1.5 MB), approximately 428.3k tokens, and a symbol index with 575 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.