Waiting for an interrupt is much longer than polling a register at the end of a test (CAPI2.0 only)

[bmesnet]

There are 2 ways for an application to know that the action has finished his job.

• 1st option is by polling a MMIO register written by the action.
• 2nd option is by waiting for an interrupt generated at the closure of the action.

When measuring latency on Power8/CAPI1.0, both options gives similar results.
When measuring latency on Power9/CAPI2.0, polling gives good results while using an interrupt can add a very big (and variable) latency (few thousands of us)
The interrupt is sent by the AFU to the PSL (opcode 0x000 "intreq" with information about the source of the interrupt).

The reason for that unexpected latency is the way Power9 processors are differently set than the Power8 cpus. This is due to the power saving mode which if set by default on Power9 cpus.

• Using cpupower idle-set -d 4 can help removing the jitter due to the power sleep mode .
• Using powersave=off command can disable this power saving mode and give you values as good as when using the polling mode. You can either execute it on the commandline of the grub/petitboot or change the /etc/default/grub file inserting the command as follow GRUB_CMDLINE_LINUX="powersave=off", then executing an update-grub and then rebooting the system.

[luyong6]

Although you have closed it, but thank you! Is there a better way to post it? Otherwise it will be hidden in the closed issues and no one will notice that. Do you suggest people to search github issues for possible solutions?

[bmesnet]

Well, it's more to have a way to quickly reference it. My belief is that people who hit a problem looks for old issues and see if they are solved.