Running all Performance Tools Automatically
The performance tools pprocess.gms runs all Performance
Tools automatically on up to 8 trace files at once. It runs the
routines on all combinations of tracefiles and generates a performance
summary HTML file.
For example, suppose we did runs of the solvers
COPNOPT1, CONOPT2, and CONOPT3 on a set of models
and want performance comparisons of each solver with respect to the
other. The pprocess.gms utility simplifies this task.
Below are descriptions on how to use the utility and its command line options.
Performance Profile Notes:
The performance profile plotting utility can be thought of as depicting
information on solver robustness from the square utility and solver
quality and correctness from the resource time utility. In the
performance result summary, performance profiles are run for different
optimality criteria in terms of optimal solution found:
- for "objective value within 0%" we are considering
best solution as the defining criteria
- for "objective value within Inf%", we are only considering the
fastest resource time as the most important factor
- for other objective value ranges, we consider both
resource time and objective value quality as the
criteria. That is, resource time is important only if the
objective value is within x %
of the best found objective value.
Download Tools:
Download Sample Trace Files:
These are trace files obtained from running an instance of the COPS
NLP test models using CONOPT1, CONOPT2, and CONOPT3, as shown in the
OR/MS CONOPT ad.
- copstest.co1 (trace file
using CONOPT1 on and instance of the COPS models)
- copstest.co2 (trace file
using CONOPT2 on and instance of the COPS models)
- copstest.co3 (trace file
using CONOPT3 on and instance of the COPS models)
Sample Output:
Using pprocess.gms
The solver square comparison utility makes use of the --option to initialize
environment variables from the command line. You must have a GAMS
system 20.1 or later installed.
See the
GAMS 20.1 release notes
for details on this feature.
The general command for square.gms is:
|
>>
gams square --trace1=(input 1 tracefile) --trace2=(input 2 tracefile)
...
--outfile=(output filename) --outdir=(output directory)
|
The options --trace1 and --trace2 are required inputs. Upt to 8
seperate trace files can be included.
All of the other --options are optional, that is if ommitted they are
set to default
options as described in the next section. The resulting performance
summary file will be located in the --outdir directory, which is
created if it does not yet exist.
A sample call is:
|
>>
gams pprocess.gms --trace1=copstest.co1 --trace2=copstest.co2
--trace3=copstest.co3 --outdir=copsconopt
|
This will create an output HTML file called results.htm
in the directory copsconopt/ showing a summary of the
performance test results.
Users can download the sample trace files given above (CONOPT solvers on
the COPS model set) and the Performance Tools. Runnings the command as
given above will produce the following result:
Top
List of pprocess.gms --options and defaults
Required Inputs:
| Option |
Input Type |
Description |
Default |
| --trace1 |
Text |
Trace file of solver 1
|
none (user must specify) |
| --trace2 |
Text |
Trace file of solver 2
|
none (user must specify) |
| --trace(n) |
Text |
Up to 8 trace files can be included
|
none (user must specify) |
Optional Inputs:
See the respective Performance Tools options:
Top
