High-Performance Computing (HPC) systems capability is increasingly constrained by their power consumption, and this will become worse due to the end of Dennard scaling^{[1, 2]}. However, HPC users want a higher performance to run the more complex model or larger datasets. As such, there is a vital need to find ways to boost the efficiency of HPC applications without increasing power consumption significantly.

Power and performance optimization for HPC systems is certainly not a new research topic. There is considerable work on designing innovative compute architecture for better energy efficiency for HPC systems^{[3, 4, 5, 6, 7, 8, 9, 10, 11]}. Other work utilizes a software-based resource scheduling approach by carefully determining the computation resource settings, such as the number of assigned compute nodes and processor frequency to match the workload to improve application performance under a power constraint^{[12, 13]}. A key advantage of a software-based approach is that it is readily deployable on existing hardware as no hardware modification is required^{[15, 14]}.

The allocation of computing resources, such as computing nodes or Control Processing Unit (CPU) cores, is essential for power and performance optimization. Current resource allocation strategies aimed at maximizing system use, i.e., no additional computing nodes would be allocated unless the processor cores of existing nodes have been fully utilized. Despite these techniques, they do not offer the optimal performance for memory-bound applications by using as few computing nodes as possible to limit the overall power consumption. As we can see in this paper, maximizing processor core usage can lead to serious memory contention, which in turn leads to sub-optimal performance. Because the memory sub-system is now becoming a bottleneck of HPC systems^{[17, 16]}, we need a better resource allocation strategy to effectively tackle the memory contention problem to achieve higher performance.

This work aims at developing a new approach for HPC resource allocation, specifically targeting memory-bound data-parallel applications. Our insight is that instead of maximizing processor utilization, one can allocate additional computing nodes to reduce memory contention. If we can do this, we can then free some processor cores (as well as reduce the number of parallel processes) on a node to reduce memory contention to improve memory-bound workload performance.

However, translating this high-level idea into a practical system is non-trivial. A key challenge is to determine the optimal number of computing nodes to be allocated to an application, for a given number of user-requested processes. If we allocate too few or too many computing nodes, we either will be unable to gain much, or the extra power consumption will outweigh the benefit. Also, our scheduling decision would need to ensure that additional computing nodes will not generate substantially more power consumption as compared to the default approach. To overcome these challenges, we need a new technique to accurately model the application behavior to drive a precise resource allocation strategy.

In this paper, we present a novel HPC resource allocation strategy, which aims to boost the application performance without significantly compromising the power consumption budget for memory-bound workloads. We achieve this by firstly characterizing the application workload behavior based on offline profiling information (Section 3.1). The workload characteristics and profiling information are used to decide how many computing nodes the target application will be run. We then propose a new technique for modeling the subtle interactions between application performance, power consumption, compute nodes, memory bandwidth congestion, and CPU clock frequency. The model is then used as a method to change the number of allocated computing nodes and the CPU frequency to fit the hardware configuration to the workload behavior. Specifically, if the target application is perceived to be a memory-bound workload, we reduce the CPU frequency to lower the system’s power consumption, as the bottleneck is on memory assessment.

We apply our approach to 12 representative Message Passing Interface (MPI) benchmarks from the NAS parallel benchmark and HPC Challenge (HPCC) benchmark suites. Our evaluation platform is an HPC cluster with 20 cores per node. Experimental results show that our approach achieves on average a 12.69% performance improvement over the conventional resource allocation strategy, but uses 7.06% less total power, which translates into 17.77% energy savings.

This paper makes the following technical contributions:

(1) We show that by allocating additional computing nodes with appropriate CPU frequency settings, one can boost application performance without increasing power consumption of the system for memory-bound workloads (Section 2.2).

(2) We present a novel technique for capturing interactions between workload characteristics and computation resource allocation and CPU settings (Section 3.1).

(3) We present a novel algorithm for scheduling and configuring resource allocation at the application level, showing how performance can be enhanced without infringing the overall power consumption constraint (Section 3.2).

Our work tackles the question of resource allocation for distributed HPC workloads. We target typical HPC environments where the user job is submitted to a job queue. As part of the job submission, the user may specify the number of processes required. We are interested in developing a resource scheduler for mapping requested resources (i.e., the number of parallel processes) to compute nodes in this work. A default strategy for the problem would be to allocate the minimal number of compute nodes to run one parallel process on a physical core. However, as we will show in this paper, such an approach often leads to sub-optimal performance for memory-intensive applications. Our goal is to determine the optimal number of compute nodes and the processor clock frequency of each compute node to reduce the running time of memory-bound applications, while at the same time to cap the peak power and energy consumption as the default strategy.

Considering scheduling bt and cg from the NAS MPI parallel benchmark suite on an HPC cluster where each computing node has 20 processor cores (see Section 4.1), in this example, we assume that a user requests to run bt and cg with 169 and 128 processes respectively and both benchmarks run with the class D input.

Our approach is implemented on an HPC cluster as part of the central job scheduler used. This is completely automated, and no modification to the application source code is required. To determine the compute resources to use for a job that is ready to run, our approach takes as input the program binary, input data, and user-supplied requirement the number of parallel processes in our case. It then determines the number of compute nodes to be provided to the job and the CPU’s clock frequency to be used across allocated machines.

To determine the optimal hardware resource allocation and frequency settings, we first profile the application as part of the strategy for resource allocation to capture the memory and CPU characteristics of the target application (Section 3.1). Specifically, VTune is used to collect the memory trace and performance metrics of the target application from a single node, and we use Intel Running Average Power Limit (RAPL) to take power measurements from all computing nodes. Then, under different allocation strategies, the profiling information is used to estimate the performance gain and energy consumption to find the optimum setting (Section 3.1). Finally, we actively analyze the instantaneous power readings to dynamically adjust the CPU frequency to cap the peak power of each computing node (Section 3.2) .

The heart of our approach is a set of functions for modeling how the resource allocation and frequency setting affect the application’s performance and power consumption.

The overall goal of our power-performance model is to improve the program performance without increasing the total power consumption. It achieves this by finding the relationship of energy consumption of allocated computing nodes, CPU frequencies, and parallel execution time. The application execution time, T, and the energy consumption of N allocated computing nodes depend on three parameters: the number of computing nodes, N, the processor frequency, fi, and the application characteristics (memory-bound or not in this paper). With these notations in place, our optimization goal can be formulated as

(1)minT⁢(N,Δ⁢N,fi,BN)s.tΔ⁢P++Δ⁢P-⩽0Δ⁢P+⩾0Δ⁢P-⩽0

where Δ⁢P+ denotes the increased power due to Δ⁢N nodes in addition to the one given by the default resource allocation policy. Specifically, the default policy would assign N computing nodes with all processor cores of a node to be completely used and run at the highest CPU frequency fmax. We use Δ⁢P- to denote the power savings resulted from the scaling down of the CPU frequency.

Table 1 lists all the parameters used by our model and their descriptions. Pcpu⁢(fi) is the energy consumption of a single CPU on one computing node, which depends on the processor frequency, fi. Pmem is the energy consumption of one computing node for the memory sub-system. We observe that Pmem is relatively stable (with a variance of less than 2%) for a given program, regardless of the number of computing nodes to be used. As such, we set Pmem to be a constant value for a given program.

Our resource allocation scheme is described in Algorithm 1. This algorithm obtains nearly optimal values for a given program, Δ⁢N and f*, for parallel performance boost without any increase in total power.

The algorithm’s input parameters include the maximum/minimum CPU frequencies fmax/fmin, physical memory bandwidth B, the threshold for memory traffic to physical memory bandwidth ratio α, the default allocated node number N, the number of processes n, and the number of cores per node c. The outputs are the optimum number of increased computing nodes for the given program Δ⁢N* and uniform processor frequency f*.

Our resource allocation scheme works as follows:

Firstly, we measure all the platform related parameters including all power values in Table 2 under different frequency levels and Pidlecpu. Even if at the same frequency level, the actual power consumption of different applications could be different. Actual power is closely related to the utilization rate of CPUs. The power values listed in Table 2 are the power measured under the nearly maximum processor utilization rate.

Secondly, we run the program and obtain corresponding parameters by performance profile data and power profile data. Performance related profile data for the given program include BN and bound. Power related profile data refer to Pmem and the single-node peak power PN. PN is used to calculate the objective total power. By Eqs. (7) and (9), we can calculate Δ⁢Nperf and Δ⁢Npower. The optimal number of increased nodes Δ⁢N* is obtained by Eq. (11). At the same time, by Eq. (14), Pcpu⁢(f*) is calculated. According to Table 2, the initial optimal frequency f* is decided.

Characterizing the applications via profiling does not restrict our approach, as most scientific computing applications usually run many times. Even if the profile-based approach consumes a great deal of time on data profiling, we can still benefit from the later process.

Finally, we get the target power Ptarget, PN×NN+Δ⁢N*, which is used to control power by RAPL as the given power limit value.

According to this algorithm, to obtain a better optimal frequency setting, we run the program again for real-time power limiting. We use RAPL to ensure that the average power does not surpass Ptarget by automatically adjusting the frequency. In this case, a good starting point could be provided by the initial optimal f* given by our model. In Section 5, we show how our model can effectively support this endeavor.

Hardware. We evaluate our approach with 64 computing nodes on an HPC cluster. Each of the computing nodes has 64 GB DDR4 RAM and two Intel Haswell 10-core E5-2660v3 processors running at 2.6 GHz. The multi-core processor supports Dynamic Voltage and Frequency Scale (DVFS) with 15 states, from 1.2 to 2.6 GHz at a step of 0.1 GHz. Thermal Design Power (TDP) for this processor is 105 W. Each computing node supports RAPL for power measurement. We disable hyperthreading to obtain stable performance.

Benchmarks. We use 12 MPI benchmarks from the NAS parallel benchmark suite and HPCC^[18] suite. Table 3 lists the benchmarks used in our evaluation.

Software. Each computing node runs CentOS 7.4 with Linux kernel 3.10. We rely on the local Operating System (OS) to schedule processes and do not bind tasks to specific cores. All the benchmarks are compiled with gcc 4.8.5 with "-O3" as the compiler option and run with openmpi 4.0.0.

Memory tracing. We use VTune^[19] to collect the memory trace with a sample rate of one second by running the program on the master node of a cluster. We take the weighted average of the memory histogram produced by VTune as the memory traffic value, i.e., BN in Eq. (3). For example, Fig. 5 shows the memory traffic histogram of benchmark sp with BN=51.76⁢GB/s. Later we show that this strategy is effective in capturing the memory characteristics of a program.

Energy measurement. We use powergov^[20] to measure processor power consumption and memory power consumption. Powergov uses RAPL to modelthe power consumption for processor and memory. We use mlc^[21] to measure memory bandwidth and memory access delay under the various memory traffic conditions. We measure the CPU consumption under different frequency settings on a per-core level. Table 4 gives the measured per-core power consumption under different CPU frequencies. For memory power consumption, our experiments on real hardware show that the memory power consumption is constant as the CPU frequency changes. Power consumption of the rest system component, i.e., Pother, is around 25 W, which is obtained through RAPL.

Performance report. We run each model on each input to collect performance profile data and power profile data until the 95% confidence bound per model per input is less than 5%. On average, we run each benchmark three times for each evaluation setting and remove obvious outliers. Then, we report the average performance across multiple runs.

Table 3 shows the performance improvement over the default resource allocation strategy when using our approach. It also gives the number of compute nodes and CPU frequency settings given by our analysis. Here, column "Δ⁢N*" shows the number of additional nodes for each benchmark.

Overall, our approach significantly improves the performance of memory-bound applications. It achieves on average 9.44% improvement for the seven memory-bound benchmarks, but using less 2.17% total power consumption. This translates to an average 10.98% reduction in energy consumption. Our approach improves the performance for most of the memory-bound applications (up to 18.82%) without incurring power consumption increase.

We also evaluated our approach using a larger dataset with a larger number of parallel processes using 30 computing nodes. The results are given in Table 5 where our approach delivers a performance improvement of up to 12.69% for the seven memory-bound benchmarks, with an average total savings of 7.06% for power and 17.77% energy consumption.

Figure 6 shows using more nodes to run programs can reduce the execution time when the CPU frequency is fixed at 2.6 GHz. The performance improvement depends on the degree of memory contention. The higher the memory contention is, the better the performance improvement could be achieved by adding more nodes. However, using more nodes will increase the total power consumption.

Figure 7 shows the details of the increase of power consumption in percentage when all processor frequencies are 2.6 GHz.

We compare the sum of memory traffic on N compute nodes and that on (N+Δ⁢N) compute nodes in Fig. 8. All the memory traffic values in Fig. 8 adopt the weighted memory bandwidth based on VTune memory traffic histogram (Fig. 5). For each group of columns, the left one represents the memory traffic sum on N nodes and the right one shows the memory traffic sum on (N+Δ⁢N*) nodes. Δ⁢N* is determined by model PPC. For memory-bound applications, memory bandwidth congestion on N nodes is usually more serious than that on (N+Δ⁢N*) nodes. So, memory traffic sum we measured on N nodes is usually lower than memory bandwidth the program actually demands. With the increase of nodes, memory bandwidth congestion starts to be relieved. Memory traffic sum we measured on (N+Δ⁢N*) nodes will be closer to memory bandwidth demands, larger than memory traffic sum we measured on N nodes.

After calculating the optimal number of compute nodes Δ⁢N* and the optimal frequency value f* with our approach, we run seven memory-bound applications with the fixed frequency f* and get the experimental results as Fig. 9 and mark Case 0. Regarding the optimal frequency value f* as the initial frequency of power capping, seven memory-bound applications are executed under RAPL power control (where the frequency is dynamically scaled) and the experimental results are recorded as Case 1. The frequency value under the default resource allocation strategy fmax is regarded as the initial frequency of power capping and seven memory-bound programs are executed under RAPL power control. The experimental results are recorded as Case 2.

Figure 9 shows the performance, power, and energy consumption changes of seven applications under three scenarios (Case 0, Case 1, and Case 2) compared with the default resource allocation strategy. The further away from the center, the better performance improvement, power savings, and energy savings an application has. Combining these three graphs, we find that compared with Case 1, Case 2 improves the performance most, but it causes the worst power consumption, which is not in line with the original intention of this paper. The purpose of this paper is to improve the performance of the applications under power constraints. However, to improve the performance of the program at the cost of causing, a lot of power consumption is not what we advocate. Unlike Case 2, Case 1 strictly complies with the power constraints, and the percentage of power consumption reduction tends to zero, which maximizes the performance of the application within the power constraints.

Cases 0 and 1 are the recommended methods. Both of them use the resource allocation strategy proposed in this paper to find the optimal number of compute nodes and frequency. The only difference is that RAPL has been carried on to power capping in Case 1, which achieves higher performance improvement. It is worth emphasizing that both Cases 0 and 1 have applied our approach proposed in this paper, which shows that this algorithm is effective and necessary.

Because our approach adds Δ⁢N* nodes based on the default resource allocation strategy, the number of idle cores increases while the number of processes remains unchanged, which reduces the utilization of processors. Can we further utilize all the processor cores (by increasing the number of processes) to further improve program performance? Experimental results show that for benchmark sp, RandomAccess, DGEMM, and bt, using up all the cores of N+Δ⁢N* nodes leads to 7.77%, 2.98%, 9.65%, and 4.54% performance improvement respectively compared to our approach (Case 1). Power increases by 3.15%, -0.09%, 3.37%, and 4.15%, respectively. Energy reduces by 0.06%, 0.52%, 6.21%, and 0.36%, respectively. But for STREAM, the performance decreases by 21.30% and the energy consumption increases by 16.11%. This result shows that the number of used processor core has a great impact on the performance of memory-bound programs. However, our approach can achieve the performance improvement of memory-bound programs under power constraints.

In this experiment, we enumerate all Δ⁢N values from 1 to 4 and f* values from 1.2 to 2.6 GHz to find the optimal Δ⁢N* and f* for performance improvement and power control. This optimal result by traversal methods is denoted as optimal value in Fig. 10. Figure 10 and Figure 11 compare the performance improvement by our approach and that by optimal setting using 10 and 30 nodes, respectively. Table 6 lists two groups of values for Δ⁢N* and f* with our approach and optimal setting, respectively. We can find the performance improvement by our algorithm is competitive compared to that by optimal setting. The performance improvement difference between them for the whole 12 benchmarks is 1.3% on average in Fig. 10 and 1.5% in Fig. 11, respectively.

The power savings comparison of our approach and optimal setting by traversal methods is shown in Fig. 12 and Fig. 13. The power savings difference between them for 12 benchmarks for 10 nodes is 0.9% on average. By our approach, programs mg and DGEMM get more power savings compared to optimal settings. That is because the processor frequency by our approach is a little lower than that by the optimal setting.

Energy consumption has become one of the most important concerns in computing systems and HPC in particular^[13]. Several researchers have developed techniques and systems to save energy with a slight increase in execution time. Reference [22] used Duty Cycle Modulation technology to save energy in MPI applications and Ref. [23] used DVFS technology to save energy in OpenMP applications. Reference [24] found that combining DVFS and Duty Cycle Modulation can get more energy savings. These approaches saved processor energy consumption by scaling down processor clock frequency with a modest increase in execution time. The increase in execution time depends on the accuracy of processor idle time prediction. References [25,26] used Near Threshold Computing (NTC) to save processor energy consumption. Except for saving processor energy consumption, Refs. [27,28] focused on how to reduce memory energy consumption.

Besides low-power techniques, power-constrained problems for compute nodes are also focused. References [29,30] reasonably allocated power to CPU and memory for performance improvement with the power limits. The main idea is that the power demands of the processor and memory are different for different applications. According to applications’ characteristics, they allocated power to CPU and memory to satisfy their demand for performance and power. Furthermore, Refs. [31,32] focused on a cluster. Firstly, when the power of a cluster is limited, they needed to set the number of active nodes according to an application’s scalability. Secondly, they needed to allocate the power to compute nodes, and also allocate the power to processor and memory in one node ultimately. Finally, they improved the application’s performance with power constraints.

Processor overclocking has been used to improve energy efficiency. For some applications, Ref. [30] found using turbo technology can achieve better performance under power constraints. It changes the clock speed of each socket (include turbo frequency), core use per socket, hyperthreading, the number of sockets in use, and the number of memory controllers in use to improve performance. Reference [33] found F-overclocking technology can achieve greater energy efficiency than DVFS, low voltage technology, and baseline. Reference [34] also found Turbo Boost Technology would enhance the application’s energy efficiency. This prior work is thus complementary to our work and can be used to control the CPU frequency when adding extra nodes.

This paper has presented a novel resource allocation scheme for HPC workloads, specifically targets memory-bound data-parallel applications. Our approach exploits a key observation that to improve performance, by reducing the number of parallel processes on a single host, one can reduce the memory contention. Unlike prior work that aims to maximize the system utilization, our approach judiciously allocates additional computing nodes to run a fewer number of parallel processes on a single node. Furthermore, to cap the total power consumption, our approach automatically determines the best CPU frequency to suit the CPU performance with the memory throughput. We propose a set of analytical models to estimate the profitability of using additional compute nodes based on profiling information. We evaluate our approach by applying it on a high-performance cluster to 12 MPI benchmarks. Experimental findings show that our approach improves the performance of seven memory-bound applications by 12.69% on average, but using 7.06% less overall power consumption, which translates into 17.77% energy savings when compared to the default resource allocation strategy.