How to do a measurement
- 1 Overview
- 1.1 Commandline
- 1.2 ./benchit script (CLI)
- 1.3 Graphical User Interface (GUI)
There are basically three ways to do a measurement:
- use the commandline
- use the ./benchit script (so called Command Line User Interface CLI)
- use the java based Graphical User Interface (GUI)
choose a kernel
You first must choose a kernel to do the measurement with As an example we will choose a matrix-multiplication-kernel (short = matmul). The matmul kernels are located in the subfolder kernel/numerical/matmul/. There you'll have at least the choice between versions written in C, Fortran77 or Java. We'll choose the C-Version because that compiler has to be installed anyway for BenchIT to work and therefor is the safest choice in this example. The next subfolder is named 0 (explanation later) andin there you'll find again a folder named 0 and a few others naming different optimizing libraries such as MKL or ATLAS - we will again choose the default-version named 0. The last subfolder is named double and in there are the sourcefiles of the kernel. The full path is now:
customize the kernel
If you want to customize the kernel, you can do it by changing the values of the PARAMETERS-file with a texteditor of your choice:
For testing - you might decrease the value of the variable:
to a lower value - maybe 100. This variable determines the maximal size of the matrizes - the lower the faster the kernel will finish and the lower values your resultfile will consist of as the kernel iterates over the size of the matrix.
compile the kernel
Now the kernel is set up you should compile it. Therefor you use the COPILE.SH script located in the benchit-root-folder:
./COMPILE.SH [OPTIONS] kernel/numerical/matmul/C/0/0/double/
Because you are running BenchIT for the first time - a script will ask you a bunch of questions about your system unless it can find that information all by itself. All the information is composed into what we call the LOCALDEFS of a machine. You can find and alter these information at any time in the LOCALDEFS/ folder. There you should find three files beginning with the hostname of your machine. The first one of these three contains the important data - the other contain additional infos not nessessary right now. If your kernel compiled successfully, it generates a binary in the bin/ folder which equals the kernelpath (with slashes replaced by dots).
run the kernel
Now you can run your kernel by invoking it with:
./RUN.SH [OPTIONS] bin/numerical.matmul.C.0.0.double.0
While the measurement is running you will see a progress bar that shows how much work is already done. As the execution times of the different problemsizes can vary a lot this does not tell you much on how long the execution of the remaining part will last, but it at least shows that there is some progress. Additionally a file containing the progress information is put into the /progress folder (the location can be changed in the LOCALDEFS). The file is named after the running kernel extended by a timestamp. It is useful if you'r doing a remote measurement where you cannot observe the output on the console.
After finishing, you'll have two new files in your output/ folder. The name depends on your customization in the LOCALDEFS. The first file includes information about the measured system, the used kernel, the environment and finally the measured performance data at the very end. The second file contains only information for the genreation of a eps-graphic with gnuplot - you can alter these settings to alter the graphic.
view the results
You can now take a look at your result with the help of a script located in the tools/ folder:
Congratulations on your first measurement with BenchIT.
for advanced users
It might be necessairy to start multiple measurements with different parameters. To automatize this, you can customize our Iterate-script.
./benchit script (CLI)
This script is fairly selfexplanory. Just keep in mind to perform the steps in the given order and always do the cleanupstep before using any of the other methods to do measurements as this script generates special versions of the PARAMETERS file so that the original PARAMETERS are not altered. The benchit shell script eases running multiple kernels from your shell. It is not intended to be a replacement for the Java GUI. Therefore it has no ability to execute kernels on a remote computer and the handling is by far not as convenient as the GUI. The following sections describe the function of the particular menu entries. Which menu entries are available depends on your progress and your environment. If there e.g. is no GUI available, the "Start GUI" menu item will not show up, and the latter menu items depend on former ones, so not all will be shown at once.
Start the BenchIT GUI. You need to have a window system and Java installed in order to use it.
Select your set of kernels
A file with a list of available kernels will be created. Here you choose which kernels you want to work with in the next steps. If you want to add kernels later on, you have to insert them manually, or you can cleanup (last but one menu entry), then the list will be regenerated, but you will lose all your custom parameter settings.
Set parameters for your kernelset
The parameters from your selected kernels will be copied to one file which you can edit afterwards and adjust the configuration for each kernel.
Compile your kernelset
This starts the compilation of the selected kernels with the parameters you have set in the previous step.
Run your kernelset
After the kernels have been compiled, you can start them using this menu entry. Take a cup of tea or coffee if you have selected a large amount of kernels and have not set a timelimit.
View the most recent resultfile of each selected kernel
This will open the output file from the last measurement of each kernel in the list you have selected at 2.2, provided that you have gnuplot and gv or gview installed as these will be used to generate and show the images. Beware that you might get numerous windows if you have selected many kernels. If there are no output files for your selection available at all, this menu entry will not be available.
Cleanup all generated files
This erases the selection of kernels and your custom parameters.
Quit the CLI.
How to start the GUI
run GUI.sh under UNIX or GUI.bat under Windows in the gui/bin/ directory.
How to run a kernel
- Select the kernel
- Select the tab Kernels at the left of the GUI
- Select the belonging node of the kernel. If the selected node contains others, all of them
are selected. You can select more then one node by holding the <CTRL>-Key
- Choose what to do
Above the editor for the kernel-files a drop-down-box is shown. Here you can select whether to compile or run a kernel, or to do both.
- Start it
Press the RUN-Button
- Choose whether to close the GUI or not
If you want to run a kernel, you will be asked whether to close the GUI or not. Not-closing can lead to lower performance, because of another running process on the machine
How to show a result
- Select the tab Results
- Click the file, you want to see
- By clicking the leaf, which corresponds to the result-file, this one will be plot
- You can edit some settings for the plot at the config-tab. Those will not be stored, when
selecting another result
- To export the plot to a graphic-file select File in the menu and Save as...
How to run a kernel remote under Windows and Unix on a remote unix machine
- Create a remote folder
- Start Setup-->Create a remote folder
- Insert the data
- IP or DNS-Name: the IP or DNS-name (e.g. server.com)
- Username on the remote system
- Foldername on the remote system: A folder with this name will be created within the home directory of user <username> on the remote system
- Select the kernel(s) you want to run
(see how to run a kernel)
- Start the selected kernel(s)
Start Measure --> Execute in remote folder