# File: README # Author(s): Giacomo Nannicini # International Business Machines Corporation # nannicini@us.ibm.com # Last edit: 06/09/17 # # (C) Copyright Singapore University of Technology and Design 2015. # (C) Copyright International Business Machines Corporation 2016. # You should have received a copy of the license with this code. # Research partially supported by SUTD-MIT International Design Center. This software is released under the Revised BSD License. By using this software, you are implicitly accepting the terms of the license. RBFOpt is a Python library for black-box optimization (also known as derivative-free optimization). It supports Python 2.7 and Python 3. Contents of this directory: LICENSE: Licensing information. VERSION: Version of the library. AUTHORS: Authors of the library. README: This file. requirements.txt: List of dependencies for this project. Contents of the src/ directory: rbfopt_black_box.py: Description of an abstract black-box function. rbfopt_black_box_example.py: Example of an implementation of a simple black-box function. rbfopt_algorithm.py: Main optimization algorithm, both serial and parallel. rbfopt_aux_problems.py: Interface for the auxiliary problems solved during the optimization process. rbfopt_cl_interface.py: Driver for the command-line interface, to run the library on a user-defined black-box function. rbfopt_config.py: (Static) configuration file for the machine. rbfopt_degreeX_models.py: PyOmo models for the auxiliary problems necessary for RBF functions with minimum required polynomial degree X. rbfopt_refinement: Routines for trust-region based refinement phase. rbfopt_settings.py: Global and algorithmic settings. rbfopt_utils.py: Utility routines. Contents of the tests/ directory: rbfopt_test_interface.py: Executable file to test the library on a standard global optimization test set. test_functions.py: Global optimization test functions. test_rbfopt_algorithm.py: Testing module for rbfopt_algorithm.py. test_rbfopt_aux_problems.py: Testing module for rbfopt_aux_problems.py. test_rbfopt_degreeX_models.py: Testing module for rbfopt_degreeX_models.py. test_rbfopt_env.py: Environment variables for testing environment. test_rbfopt_refinement:: Testing module for rbfopt_refinement.py test_rbfopt_settings.py: Testing module for rbfopt_settings.py. test_rbfopt_utils.py Testing module for rbfopt_utils.py. Contents of the doc/ directory: conf.py: Configuration file for Sphinx. Makefile: Makefile (for Linux/Mac) to build the documentation. make.bat: Batch file (for Windows) to build the documentation. *.rst: ReStructured Text files for the documentation. #-------------------------------------------------------------------------- # Installation requirements #-------------------------------------------------------------------------- This package requires the following software: Python version >= 2.7 NumPy version >= 1.11.0 SciPy version >= 0.13.3 Pyomo version >= 4.3.11388 The software has been tested with the versions indicated above. It may work with earlier version and should work with subsequent version, if they are backward compatible. The code is mainly developed keeping Python 3 in mind, but it also runs on Python 2.7. We recommend using Python 3 if possible. The easiest way to install all the dependencies is via the Python module manager pip, using the command: pip install -r requirements.txt The file requirements.txt contains the list of required modules and their latest version that has been tried and found to work correctly. If your want to install them one at a time, you can try: pip install numpy pip install scipy pip install pyomo To build the documentation, you also need numpydoc: pip install numpydoc On Windows systems, we recommend WinPython, available at http://winpython.sourceforge.net/ , which comes with NumPy, SciPy and pip already installed. After installing WinPython, it is typically necessary to update the PATH environment variable. The above command using pip to install missing libraries has been successfully tested on a fresh WinPython installation. RBFOpt requires the solution of convex and nonconvex nonlinear programs (NLPs), as well as nonconvex mixed-integer nonlinear programs (MINLPs) if some of the decision variables (design parameters) are constrained to be integer. Solution of these subproblems is performed through Pyomo, which supports any solver with an AMPL interface (.nl file format). In our tests we employed BonMin and Ipopt, that are open-source and available through the COIN-OR repository. The end-user is responsible for checking that they have the right to use whatever solver they employs. To obtain pre-compiled binaries for BonMin and Ipopt for several platforms, we suggest having a look at: http://ampl.com/products/solvers/open-source/ and http://ampl.com/dl/open/ (for static binaries). In case any of the packages indicated above is missing, some features may be disabled, not function properly, or the software may not run at all. #-------------------------------------------------------------------------- # Installation instructions and getting started #-------------------------------------------------------------------------- 1) Install the required packages NumPy, SciPy, Pyomo, as indicated above. If you use pip, you can verify that they are present with the commands: pip show numpy pip show scipy pip show pyomo 2) Edit rbfopt_config.py to point to the correct location for the NLP and MINLP solvers, using the MINLP_SOLVER_PATH and NLP_SOLVER PATH variables. 3) Enjoy! 4) You can test the installation by running: python tests/rbfopt_test_interface.py branin lp See: python tests/rbfopt_test_interface.py --help for more details on command-line options for the testing utility. Many more test functions, with different characteristics, are implemented in the file test_functions.py. They can all be used for testing. 5) Unit tests for the library can be executed by running: python -m unittest discover -s tests/ from the current (main) directory. This may take some time. If some of the tests fail, the library may or may not work correctly. Some of the test failures are relatively harmless. You are advised to contact the mailing list (see below) if you are unsure about some test failure. 6) To run the library on a user-defined black-box function, have a look at the example in src/rbfopt_black_box_example.py. You can implement a similar class (derived from rbfopt_black_box.BlackBox), say in a file called my_black_box.py, and the alorithm can be executed on it by running: python src/rbfopt_cl_interface.py -m my_black_box.py See: python src/rbfopt_cl_interface.py --help for more details on command-line options. (If no module is specified with the -m option, the algorithm will be applied to rbfopt_black_box_example.py.) #-------------------------------------------------------------------------- # Parallel optimization #-------------------------------------------------------------------------- RBFOpt supports asynchronous parallel optimization using Python's multiprocessing library. This mode is enabled whenever the parameter num_cpus is set to a value greater than 1. Black-box function evaluations as well as some of the heaviest computatations carried out by the algorithm will then be executed in parallel. Since the parallel computations are asynchronous, determinism cannot be guaranteed: in other words, if you execute the parallel optimizer twice in a row, you may (and often will) get different results, even if you provide the same random seed. This is because the order in which the computations will be completed may change, and this may impact the course of the algorithm. The default parameters of the algorithm are optimized for the serial optimization mode. Recommended parameters for the parallel optimization will be detailed in the future. For now, feel free to ask on the mailing list. #-------------------------------------------------------------------------- # Documentation #-------------------------------------------------------------------------- The documentation for the code can be built using Sphinx with the numpydoc extension. numpydoc can be installed with pip (if you followed our instructions above, it should already be installed): pip install numpydoc After that, the directory doc/ contains a Makefile (on Windows, use make.bat) and the Sphinx configuration file conf.py. You can build the HTML documentation (recommended) with: make html The output will be located in _build/html/ and the index can be found in _build/html/index.html . A PDF version of the documentation (much less readable than the HTML version) can be built using the command: make latexpdf An online version of the documentation for the latest master branch of the code, and for the latest stable release, are available on ReadTheDocs at the following URLs: http://rbfopt.readthedocs.org/en/latest/ http://rbfopt.readthedocs.org/en/stable/ #-------------------------------------------------------------------------- # Citing RBFOpt #-------------------------------------------------------------------------- If you use RBFOpt in one of your projects or papers, it would be great if you could cite the following paper: A. Costa and G. Nannicini. RBFOpt: an open-source library for black-box optimization with costly function evaluations. Optimization Online paper 4538. http://www.optimization-online.org/DB_HTML/2014/09/4538.html The paper above describes version 1.0 of RBFOpt. Some of the improvements introduced for version 2.0 are described in the following paper. A. Fokoue, G. Diaz, G. Nannicini, H. Samulowitz. An effective algorithm for hyperparameter optimization of neural networks. IBM Journal of Research and Development, 61(4-5), 2017, to appear. #-------------------------------------------------------------------------- # Support #-------------------------------------------------------------------------- The best place to ask question is the mailing list: rbfopt@list.coin-or.org You can subscribe at: http://list.coin-or.org/mailman/listinfo/rbfopt