intel/intel-cmt-cat

Error Log: Failed to retrieve socket 0 L3 classes! Allocation configuration error!

Biyi-lab opened this issue · 8 comments

Biyi-lab commented

MBA can be used normally.
An error occurs when executing "pqos -e "llc:....". Is this related to the linux kernel version? or need to modify the relevant parameters of CAT?
Thank you！

aleksinx commented

Hi,
Could you provide output from command causing issue with verbose option enabled?

Biyi-lab commented

pqos -e "llc:0=0xff00;llc:1=0x00ff" 

NOTE:  Mixed use of MSR and kernel interfaces to manage
       CAT or CMT & MBM may lead to unexpected behavior.
Failed to retrieve socket 0 L3 classes!
Allocation configuration error!

In fact, when I execute pqos -e, there is no relevant output from L3CA：

NOTE:  Mixed use of MSR and kernel interfaces to manage
       CAT or CMT & MBM may lead to unexpected behavior.
MBA COS definitions for Socket 0:
    MBA COS0 => 100% available
    MBA COS1 => 50% available
    MBA COS2 => 70% available
    MBA COS3 => 100% available
    MBA COS4 => 100% available
    MBA COS5 => 10% available
    MBA COS6 => 100% available
    MBA COS7 => 100% available
MBA COS definitions for Socket 1:
    MBA COS0 => 100% available
    MBA COS1 => 50% available
    MBA COS2 => 70% available
    MBA COS3 => 100% available
    MBA COS4 => 100% available
    MBA COS5 => 10% available
    MBA COS6 => 100% available
    MBA COS7 => 100% available
Core information for socket 0:
    Core 0, L2ID 0, L3ID 0 => COS4
    Core 1, L2ID 1, L3ID 0 => COS4
    Core 2, L2ID 2, L3ID 0 => COS4
    Core 3, L2ID 3, L3ID 0 => COS4
    Core 4, L2ID 4, L3ID 0 => COS4
    Core 5, L2ID 5, L3ID 0 => COS4
.........
aleksinx commented

It seams that L3 CAT is not available on your system. You can check available features using pqos -d command

Biyi-lab commented 
NOTE:  Mixed use of MSR and kernel interfaces to manage
       CAT or CMT & MBM may lead to unexpected behavior.
OS capabilities (Linux kernel 3.10.0-1160.42.2.rt56.1182.el7.x86_64)
    Monitoring
        Cache Monitoring Technology (CMT) events:
            LLC Occupancy (LLC)
        Memory Bandwidth Monitoring (MBM) events:
            Local Memory Bandwidth (LMEM)
            Total Memory Bandwidth (TMEM)
            Remote Memory Bandwidth (RMEM) (calculated)
        PMU events:
            LLC misses
            LLC references
            Instructions/Clock (IPC)
    Allocation
        Memory Bandwidth Allocation (MBA)
            Num COS: 8
            CTRL: disabled

Yes, L3 CAT is not available. Can we fix it by changing the kernel version?

aleksinx commented

Is L3 CAT available in the hardware? You can check is using command pqos --iface=msr -d

Biyi-lab commented 
NOTE:  Mixed use of MSR and kernel interfaces to manage
       CAT or CMT & MBM may lead to unexpected behavior.
WARN: resctl filesystem mounted! Using MSR interface may corrupt resctrl filesystem and cause unexpected behaviour
Hardware capabilities
    Monitoring
        Cache Monitoring Technology (CMT) events:
            LLC Occupancy (LLC)
        Memory Bandwidth Monitoring (MBM) events:
            Total Memory Bandwidth (TMEM)
            Local Memory Bandwidth (LMEM)
            Remote Memory Bandwidth (RMEM) (calculated)
        PMU events:
            Instructions/Clock (IPC)
            LLC misses
            LLC references
            LLC misses - pcie read
            LLC misses - pcie write
            LLC references - pcie read
            LLC references - pcie write
    Allocation
        Cache Allocation Technology (CAT)
            L3 CAT
                CDP: disabled
                Num COS: 16
        Memory Bandwidth Allocation (MBA)
            Num COS: 8
aleksinx commented

PQoS tools supports two interfaces

  1. OS - kernel interface enabled by default
  2. MSR - hardware interface

You can use MSR with --iface=msr option then L3 CAT and will be available for cores only.

On some platforms L3 CAT is disabled in the kernel by default (due to cpu errata). You can enable kernel support using rdt=l3cat cmdline kernel option (For details please refer to https://www.kernel.org/doc/Documentation/x86/resctrl_ui.txt).

Biyi-lab commented

thank you! it worked.