$ontext ======================================================================= LAST MODIFIED: 2008/10/24 FILENAME: plotprof.gms PURPOSE: Plots the performance profiles computed in pprofile.gms. Assumes GNUPLOT is installed locally and is on the user PATH. USAGE: The same as pprofile.gms with the same inputs PROFILE DESCRIPTION: Suppose that we define a performance ratio: rho(p,s) = resource time for problem p for solver s ------------------------------------------------- min resource time for problem p over all solvers which can be thought of the relative solver time for a problem wrt to the best possible time. Then the performance profile p_s(tau) for a solver s is the probability that a performance ratio rho is within a factor of tau of the best possible ratio. INTERPRETATION OF PERFORMANCE PROFILE: If we are only interested in maximizing the number of "wins" (i.e. maximizing the number of models solved the fastest), then we need only consider p_s(1). If we are interested only in determining if a solver has a high probability of success we are interested in lim p_s(tau) tau_>inf REFERENCE: Dolan, E. and More, J., "Benchmarking optimiation software with performance profiles", Mathematical Programming, Vol 91 (2), pp.201-213, 2002. REQUIRED INPUTS: --trace1: trace file of solver 1 --trace2: trace file of solver 2 etc up to --trace8 OPTIONAL INPUTS: --factor factor for values of tau: 1, factor, factor^2, factor^3 (default 1.25) --figname: name of output figure (default: profile.*, where * is the format ext.) --modelfile: filename containing subset of models to consider. Useful, if only a subset of models from the tracefile are to be used in the solver square comparison. --numpts: number of data points for each solver for performance profile (default: 25) --delta: considered optimal if objective value is within --delta % of best objective value using relative obj. difference. (default: Inf, i.e. ignore) --optonly: Plots only if model is truly optimal (not only feasible). Set to1 if only optimal models. Otherwise considers both feasible and truly optimal models. (default: 0) --otype: output type: windows, gif, ps (default is to windows) --outdir: output directory (default: no directory) --outfile: filename of data to plot in .txt format: generated from pprofile.gms (default: profile.txt) --resmin: minimum resource time threshold. If a solver reports a resource time below --resmin, then it is increased to resmin before being used in ratios. (default 0.05). --topt4: trace option 4. If specified, then assumes tracefile is traceopt=4 (default:none) --total: indicator if total models to be solved by any solver should be displayed. Use --total=0 if to be ignored. (default: 1) REMARKS: Each tracefile can contain information from only one (1) solver for a given set of models. For example, tracefiles containing solver A and solver B will cause error message upon compilation. Users must have a relevant plotting software (GNUPLOT) installed and in the user PATH. $offtext ====================================================================== $if not set figname $set figname profile $if not set otype $set otype windows $if not set delta $set delta Inf $if not set optonly $set optonly 0 $if not set total $set total 1 $set GNUPLOT pgnuplot.exe $if %system.filesys% == UNIX $set GNUPLOT gnuplot $if not set outfile $set outfile %figname% $include pprofile.gms $if "%colselect%"=="res" $set gp_xlabel 'Time Factor' $if not "%colselect%"=="res" $set gp_xlabel '%colselect% Factor' $if not set outdir $goto nooutdir $set outfile %outdir%/%outfile% $set figname %outdir%/%figname% execute 'mkdir %outdir%' execute 'cp style.css %outdir%' $label nooutdir file fgnu / gnu.inc /; put fgnu; $if %otype% == "png" $goto PNG $if %otype% == "ps" $goto POSTSCRIPT $label PNG put "set term png;" / "set output '%figname%.png';" $goto DONE $label POSTSCRIPT put "set term postscript landscape;" / "set output '%figname%.eps';" $goto DONE $label DONE / "set title 'Performance Profile';" / "set xlabel '%gp_xlabel%';" / "set ylabel 'Percent Of Models Solved';" * / "set yrange [0:100];" / "set log x;" / "set grid;" / "set key bottom right;"; parameter i; i=1; loop(solvername, if(i>1, put ",\" / "'%outfile%' using 1:" (i+1):0:0 " title '" solvername.tl:0 "' with linespoints"; else put / "plot '%outfile%' using 1:" (i+1):0:0 " title '" solvername.tl:0 "' with linespoints"; ); i = i+1; ); put ",\" / "'%outfile%' using 1:" (i+1):0:0 " title 'CAN_SOLVE' with linespoints;"; putclose fgnu; execute 'cat gnu.inc | %GNUPLOT%' execute 'rm -f gnu.inc'