AVX stands for Advanced Vector Extensions, an extension to the x86 instruction set of Intel and AMD CPUs. With the introduction of these extensions, the SIMD register size was double to 256 bit. The new instruction set allows to execute commands having three operands (e.g, c = a + b) to save copy operations.
The AVX benchmark is intended to help to sort the criteria, which may influence the efficiency of the numerical kernels, in particular in the aspect of cache and memory accesses. For the full utilization of the cache, a lot of things have to be considered, including cache lines, cache associativity, hardware and software prefecthing, vectorization, and so on.
The benchmark “AVX” source code can be found in the /src directory and can be compiled with GNU and Intel compilers with the command:
$cd src
$make avxIt performs arrays addition (like a[i]=b[i]+c[i]) through loops and compares the performance and energy efficient with regards to different data access modes. The execution of the program requires four input arguments:
$./avx config.ini 2 1 /home/data_file_dir The 1 st argument “config.ini” specifies the configuration file, where the program reads associated parameters in. Here is an example of the configuration file contents:
[GLOBAL]
cpu_cores=10;
LENGTH_MIN=300;
LENGTH_MAX=1000000;
num_steps=10;
length_conf = 400 500 1000 2500 6000 12000 22000 30000 45948 1000000;
steps = 15 30 50 70 150 250 300 500 500 15000;
DATA_TYPE =.dat
[COMPONENT_1]
FILE_NAME=seq
[COMPONENT_2]
increment=4;
FILE_NAME=incre
[COMPONENT_3]
FILE_NAME=list
[COMPONENT_4]
FILE_NAME=randidx
[COMPONENT_5]
FILE_NAME=para_pin1
[COMPONENT_6]
FILE_NAME=para_pin2
[COMPONENT_7]
FILE_NAME=para_pin3The 2nd argument is the number representing the type of the benchmark operation, where:
| 2nd argument | Meaning |
|---|---|
| 1 | Sequential array data access. The index of the array is increased by 1 for eachloop. In this case, the addition is vectorized and the loop is unrolled by 8 times. |
| 2 | Fixed increment data access. The index of the array is increased by a fixed number for each loop, while the increment number can be obtained from the configuration file. In this case, no data vectorization is fulfilled. And the loop-unrolling depends on the increment of the index and compiler option (For example by using the gcc 4.9.2 compiler with the option “-funroll-loops”: Loop is unrolled for index increased by 4, however for index increased by 5, there is no loop unrolling). |
| 3 | Array “b” and “c” are changed to linked-list data structure. And for data alignment, each node in the list contains 18 byte address and 38 byte data. Hence, the size of one node is equal to the size of a half cache-line. In this case, neither the data vectorization nor the loop-unrolling is fulfilled. However it can be seen as that the loop is unrolled by 3 times manually for data alignment. |
| 4 | Random array data access. The index of the array is obtained by another array, containing integers in the index range with random order. In this case, no data vectorization is fulfilled. However the loops are unrolled by 8 times. |
| 5 | Parallel processing, fill cores strategy 1: Each thread is statically pinned to a core whose id is the same as the thread id (no hyper-threading is used). |
| 6 | Parallel processing, fill first core strategy 2: Threads are all pinned to the first physical core (no hyper-threading is used). |
| 7 | Parallel processing, fill hw-threads strategy 3: Because of hyper-threading technique, each physical core can be divided into two execution parts sharing the local caches and arithmetic units. Therefore, hyper-threads pinning method pins the first half threads one by one to hw-threads 0-9, and the other half threads are pinned to hw-threads 20-29, while each of them asides physically together with cores 0-9 respectively. |
The 3rd argument is the number of cores that will be used for the computation; and the 4th argument is data file directory, where you want to store the performance measurements.
There is one PBS job script (pbs.sh in /src) provided showing how to execute the benchmark on EXCESS cluster. Users can just submit the job with a command:
$qsub pbs.shBesides the users can also execute the benchmark in an interactive way:
$qsub -I -l nodes=1:node02:ppn=20 -X
$./avx config.ini 1 1 & top -d .5After the execution of the test program, not only the performance data but also associated power and energy data are saved. Performance (execution time) data can be found in the directory as the users specifiy in the 4th argument. Power data will be collected if the users switch on the power measurements by the commands:
$mkdir /nas_home/your_username/.pwm//node0x/ -p
$touch /nas_home/your_username/.pwm//node0x/copy_raw_dataThen after the job is completed, the power data are stored in a directory and can be extracted by:
$tar xfz /nas_home/your_username/pwm//node0x/xxxx.fe.excess-project.eu.tar.gzTo analyze the power data combined with the performance data, we provide a /powercal_tools directory, containing a power calculation tool: power_calculate_avx. The tool needs different input parameters:
| Option | Argument |
|---|---|
| -verbosity | 1 (fixed) |
| -offset_sec | 0.0 (fixed) |
| -relative_time | 0 (fixed) |
| -board_id | 1 (fixed) |
| -config_file | configuration file of the power measurement system |
| -topdir | directory, where the power data are stored |
| -performance_file | performance data file, which is written by the AVX benchmark |
| -power_file | calculated power file, output of the power calculation tool |
| -energy_file | calculated energy file, output of the power calculation tool |
| -joule_per_flop_file | calculated Joule/Flop data file, output of the power calculation tool |
The performance data file is generated by the execution of the AVX benchmark. The file contains the timestamps of the beginning and the end of each test. With the help of these timestamps and the power data, the tool is able to compute the power consumption for each test: The tool generates the set of the data, which contains power, energy and efficiency values for the further analysis and visualization.
There is one script in the directory /powercal_tools, showing how to use the power calculation tool.
This project is realized through EXCESS. EXCESS is funded by the EU 7th Framework Programme (FP7/2013-2016) under grant agreement number 611183.
Fangli Pi, HLRS