2. Benchmarks
Benchmarks play a crucial role in comparing and measuring performance in data-center workloads. They provide standardized tests that enable fair and objective comparisons across different hardware and software configurations. By using benchmarks, we can assess the efficiency and effectiveness of various components and systems within a data center, including network performance, input/output (IO) capabilities, computational power, and memory usage. These task-specific benchmarks help us identify bottlenecks and optimize the overall performance of data-center workloads.
2.1. STREAM
The STREAM benchmark is a widely used tool in high-performance computing to evaluate the memory bandwidth of computer systems. It focuses on assessing how efficiently a system can move data between the processor and memory. The benchmark is particularly relevant in scenarios where data-intensive applications, such as scientific simulations or large-scale data processing, are involved. The primary goal of the STREAM benchmark is to measure sustainable memory bandwidth. Memory bandwidth refers to the rate at which data can be read from or written to the computer’s memory. The benchmark includes simple vector kernels: Copy, Scale, Add and Triad, which are mathematical operations performed on arrays of data. These operations are designed to stress the memory subsystem, providing insights into how well the system can handle large-scale data movements. More information can be found here: stream
2.2. Multiload
Multichase is a pointer chaser benchmark that can measure the performance of different system resources such as CPU, memory, disk, network, and battery. Multichase can perform a pointer chase through an array of different sizes and stride sizes for a certain amount of time on one or more threads. Multiload is a superset of multichase that runs latency, memory bandwidth, and loaded-latency benchmarks. Multiload can run any of the multichase commands in latency-only mode. In bandwidth-only mode, multiload can run a memory bandwidth test using the load argument. In loaded-latency mode, multiload can run one pointer chaser thread on logical CPU0 with multiple memory bandwidth load threads. More information can be found here: multichase
2.3. Sysbench
Sysbench is a powerful and versatile benchmarking tool widely recognized in the realm of performance testing and evaluation. Designed to stress test various aspects of system performance, particularly focused on database systems, Sysbench offers a wide range of workload scenarios to simulate real-world scenarios. Whether it’s testing OLTP (Online Transaction Processing) workloads, file I/O operations, or CPU performance, Sysbench provides an extensive set of tools and metrics to analyze and measure system behavior. Its flexible configuration options allow users to customize test parameters and adjust the workload to mimic specific application requirements. With its ability to generate substantial loads and simulate concurrency, Sysbench enables users to assess system scalability, identify bottlenecks, and optimize system performance. Its comprehensive reporting capabilities offer detailed insights into metrics such as transactions per second (TPS), queries per second (QPS), response times, and more, enabling users to make informed decisions about system configurations and optimizations. In summary, Sysbench stands as an indispensable tool for benchmarking and analyzing system performance, empowering developers and system administrators to finetune and optimize their systems to achieve optimal performance and reliability. More information can be found here: sysbench. Sysbench is a benchmarking tool that supports multiple database systems, making it a versatile choice for performance testing and evaluation across different platforms. Some of the popular databases that Sysbench can work with are MySQL, PostgreSQL, MariaDB, Oracle Database, Amazon Aurora, SQLite. Analyzing the impact of cloud computing, benchmark kits like PerfKit Benchmarker (PKB): PKB are employed to evaluate the performance of workloads both within and outside cloud environments. In this work we will be using PKB with the MySQL database: MySQL.
2.4. Fleetbench
Fleetbench is a benchmarking suite for Google workloads. It is meant for use by chip vendors, compiler researchers, and others interested in making performance optimizations beneficial to workloads similar to Google’s. Fleetbench consists of a curated set of microbenchmarks for hot functions across Google’s fleet. The data set distributions it uses for executing the benchmarks are derived from data collected in production. More information can be found here: fleetbench