Performance Profile Sample Scenario
Performance profiles for different objective value tolerances using
two MINLP solvers on all models in
MINLPLib.
The tool is described in detail
at
http://www.gamsworld.org/performance/pprofile.htm.
A solver is considered feasible or optimal, if it has the proper model
and solver return codes and the relative objective value error is within
n % of the best possible solution. If the best found objective
value is 0, then the absolute objective value error is used.
The following objective value tolerances are used:
- No objective value tolerance (i.e. only consider solver efficiency)
- Only best objective value considered
Interpretation:
The first performance profile plot only considers solver efficiency and
not quality of solutions. Thus, a solver "wins" as long as it is faster.
The second plot takes into account the solver objective value; that is,
a solver "wins" only if it has the best objective value.
Therefore, if we are only
interested in solver efficiency in finding a local optima, then solver B
is a better choice (see the first plot at Time Factor=1). If we
are interested in obtaining the best solution, then we would choose
solver A (see the second plot for Time Factor=1).
For reference, also listed is CAN_SOLVE, which shows the percentage
of models that can be solved by any solver.
| Considers only solver efficiency |
 |
|
| Considers quality of solution first
and then solver efficiency. Solver must have best
solution of two solvers. |
 |
|