One of the things that is learned when studying computer architectures is how processors can take advantage of performance tests in order to manipulate benchmarks. This is something that is not new, moreover, since the beginning of the PC, reports are usually sent where the manufacturer, be it Intel, AMD or any other brand, usually accompanies the information on its architecture with numbers and graphics. However, just as things like financial manipulation also exist, it also happens, oddly enough in the world of processors.
Common cheats that are made to manipulate benchmarks
To know if a processor is better than another we need a cardinal assessment, that is, a number that measures performance and specifies if processor A is better than processor B. Just reading the specifications is not enough for us, since these data they do not help us to compare one CPU with another if they are of different architectures. This is where benchmarks come in and knowing this, processor designers and manufacturers use a number of tricks or traps to manipulate benchmarks.
cheating instructions
A benchmark is nothing more than a program, that is, a series of ordered instructions that are executed in sequence. Well, one of the traps that all hardware architects have been doing for years, if not decades, is to optimize the ones that are most used in benchmarks. What do we mean? For example, make them take less time to run. When they have to make the decision about which instruction has more resources in the design, more importance is always given to the most used ones, and if they are on top, they can give a better result in performance tests.
Clock speed and cache to manipulate benchmarks
Although both are separate elements, apparently they are very related if we talk about manipulating results from the chip itself. Today the clock speed of a processor, the so-called GHz, can fluctuate depending on the level of work. The fact is that depending on where the data is located, if at some level of the cache or in the RAM the consumption is lower. Typically, a program runs in RAM, but there are benchmarks that are microprograms that fit in cache memory.
So when they run they don’t need to access the memory controller, they just do it recursively from the cache. Which not only gives them a very low latency, but also a lower consumption, so this allows to increase the clock speed in the long term. In a conventional environment, such a sustained increase in consumption would not be recommended, but the performance test does not measure that aspect, but the speed at which the program runs.
The trick is none other than to maintain the Turbo or Boost speed for as long as possible. There are even designs that specify a speed in that aspect lower than what they achieve during said tests. But made the law, made the trap.
Multi-threaded testing and heterogeneous configurations
Due to bad marketing, the E-Cores of the last two generations of Intel are confused with cores in the face of energy efficiency. Rather they are optimized for the area. And what sense do they have? When a processor is executing a program, there are sometimes what are called stalls for a cause that let it continue, which generates periods of inactivity that we call bubbles. Well, the idea of multithreading is to give mechanisms to the processor so that it changes context and can take care of other processes.
The problem is that this greatly increases power consumption and is deadly for handheld devices. The solution? Instead of wasting space and transistors on multithreading, simpler processor cores were added to low-power devices. Strategy that Intel has apparently adopted, however, the P-Cores continue to maintain multithreading, so the utility of the E-Cores is not that. Rather they serve so that the most powerful kernels do not waste time processing things that affect the main program. Thus, if applications such as the mail manager are suddenly activated, for example, they should not end up running in the foreground.
In multithreaded benchmarks the score is a sum of the performance of all cores, something that cannot be disabled. This being one of the pitfalls of comparisons between homogeneous and heterogeneous processors. Therefore, the ideal would be to mark how much each type of nucleus adds to the total.
false results
This is the most brazen case, in which, depending on the brand or model, extra points are given. This is possible thanks to the fact that every CPU has an instruction that identifies it, so that the benchmark program can know where it is running and apply a positive or negative condition as the case may be.
