Configuring Linux for High Performance

Modified on Thu, 23 Mar 2023 at 12:53 PM


Introduction 

To ensure optimal and expected performance from a Blackcore server there are a few essential configuration changes that must be made to the Linux environment. This guide will cover the steps needed to make the CPU run at the maximum frequency at all times, as well as reduce latency due to C-state transitions.  This the minimum config required.


Linux Environment 

For this guide we will be using RHEL 8.4 with kernel version 4.18  

 

You must have the intel_pstate driver installed and enabled. This should already be enabled by default, however if you compile your own kernel you might need to add the driver manually. 

 

The steps may differ in other Linux distributions and/or kernels. 


 

Controlling C-States 

Lastly we need to edit the grub configuration to boot with some specific kernel parameters. It is worth noting that Linux has complete control over the CPU and can override some BIOS settings. As such we need to set the following parameters for best performance. 

 

    Intel_idle.max_cstate=0


This tells the intel_pstate driver to stay in the lowest C-state possible, which for the intel_pstate driver is C1.

 

There is some latency that occurs when switching from C1 (idle) to C0 (load), as such we want to keep the CPU in C0 state at all times to remove this latency. 

 

    Idle=poll 

This will ‘spin’ a thread on all cores waiting for an instruction, keeping the CPU in the C0 state, thus removing any latency changing from C1 -> C0 state. 

 

Add the kernel parameters to GRUB "GRUB_CMDLINE_LINUX" option: 

# nano /etc/default/grub 



Regenerate the grub files for both BIOS and UEFI bootloaders 

# grub2-mkconfig -o /boot/grub2/grub.cfg 

# grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg 

Reboot the unit. 

 

Verify the kernel was booted with the correct parameters: 

# cat /proc/cmdline 

 


Finally, we can use turbostat to check that all cores are running at max frequency, and are in the C0 state (“100% BUSY”) 

turbostat 




cpupower utilities 

On some distributions, it may be necesary to configure the CPU scaling governor. We will be using the cpupower set of utilities to manage CPU scaling and governors. It automatically selects the correct modules to load as well as providing a service which we will use to enable our settings at boot. 

 

Install cpupower via: 

# yum install cpupowerutils 

This will install the Kernel-tools package if not already installed. 

 

Check the currently loaded module and governor: 

cpupower frequency-info 

analyzing CPU 0: 

  driver: intel_pstate 

  CPUs which run at the same hardware frequency: 0 

  CPUs which need to have their frequency coordinated by software: 0 

  maximum transition latency: 0.97 ms. 

  hardware limits: 1.20 GHz - 3.80 GHz 

  available cpufreq governors: performance, powersave 

  current policy: frequency should be within 1.20 GHz and 3.80 GHz. 

                  The governor "powersave" may decide which speed to use 

                  within this range. 

  current CPU frequency is 1.20 GHz (asserted by call to hardware). 

  boost state support: 

The correct driver should be intel_pstatehighlighted in green above. The current governor (powersave) is highlighted in blue. 

 

We need to change the CPU governor from powersave to performance: 

cpupower frequency-set -g performance 

Setting cpu: 0 

Setting cpu: 1 

Setting cpu: 2 

[...] 

 


On some systems you may need to configure a service to enable the governor on startup.

 

Create the service: 

systemctl enable cpupower 

Created symlink from /etc/systemd/system/multi-user.target.wants/cpupower.service to /usr/lib/systemd/system/cpupower.service. 

 

Disable the tuned service: 

systemctl disable tuned 

 

Save the changes and reboot the machine. Check the service has started correctly: 

systemctl status cpupower 

  cpupower.service - Configure CPU power related settings 

   Loaded: loaded (/usr/lib/systemd/system/cpupower.service; enabled; vendor preset: disabled) 

   Active: active (exited) since Wed 2016-09-14 16:12:18 BST; 4min 46s ago 

  Process: 3118 ExecStop=/usr/bin/cpupower $CPUPOWER_STOP_OPTS (code=exited, status=0/SUCCESS) 

  Process: 3130 ExecStart=/usr/bin/cpupower $CPUPOWER_START_OPTS (code=exited, status=0/SUCCESS) 

 Main PID: 3130 (code=exited, status=0/SUCCESS) 

 

Sep 14 16:12:18 localhost.localdomain cpupower[3130]: Setting cpu: 15 

Sep 14 16:12:18 localhost.localdomain cpupower[3130]: Setting cpu: 16 

Sep 14 16:12:18 localhost.localdomain cpupower[3130]: Setting cpu: 17 

Sep 14 16:12:18 localhost.localdomain cpupower[3130]: Setting cpu: 18 

Sep 14 16:12:18 localhost.localdomain cpupower[3130]: Setting cpu: 19 

Sep 14 16:12:18 localhost.localdomain systemd[1]: Started Configure CPU power related settings. 

Relevant parts highlighted in green. 

 

Finally, check the current governor is correctly set to performance: 

cpupower frequency-info | grep --color '”performance”' 

                The governor "performance" may decide which speed to use 

This guide only serves as a starting point for optimising Linux for high performance. Advanced configuration such as nohz and isolcpus are beyond the scope of this document. Also note that configuration may need to be tailored to specific software and user requirements.  

Was this article helpful?

That’s Great!

Thank you for your feedback

Sorry! We couldn't be helpful

Thank you for your feedback

Let us know how can we improve this article!

Select atleast one of the reasons

Feedback sent

We appreciate your effort and will try to fix the article