GAMS Benchmark

This is a small Python script for benchmarking GAMS - either directly using the command line interface or through JuMP or Pyomo. The results are exported to .trc files that can be analyzed with Paver2.

To start a benchmark, run for example:

python src/benchmark --testset=minlplib --threads=4 --gams=/opt/gams --gamsopt="solver=scip"

Benchmark runs can be interrupted at any time and resumed later on. Simply restart with the same options. If a .trc file can be found for a model, those results will be used. Otherwise the process is (re-)started.

For further help, run:

python src/benchmark -h

Testset Models

Input formats are either GAMS models (.gms files), Pyomo models that define a concrete model, i.e.,

m = ConcreteModel()

in the main namespace, or analogously JuMP models:

m = Model()

There are a few testlibs that can be setup conveniently, for example the MINLPlib. Simply do

cd <repository_root>/testests/minlplib
./update.sh

to download (or update) the model files and

cd <repository_root>/testsets
./convert.sh minlplib pyomo

to convert the models to the Pyomo format.

Options

Benchmark

Option Name	Default	Explanation
`result`	latest	Directory in which the results are stored in
`max_time`	60	Maximum time per job
`kill_time`	30	Additional time to max_time until a job is killed
`max_jobs`	inf	Maximum number of added jobs, if inf then whole testset
`max_total_time`	inf	Maximum time until no jobs are processed anymore
`threads`	1	Number of threads to run jobs in parallel
`output`		Output format, see below

Testset

Option Name	Default	Explanation
`testset`	minlplib	Testset to be used (minlplib, princetonlib, other)
`modelpath`		If testset is other, this specifies the path to the models
`interface`	direct	Modelling interface (direct GAMS call, JuMP, Pyomo)

GAMS Options

Option Name	Default	Explanation
`gams`	/opt/gams	Path to GAMS system directory
`gamsopt`		GAMS solver options, see below