======================================================================== README for Intel(R) RDT Software Package April 2016 ======================================================================== Contents ======== - Overview - Package Content - Hardware Support - Software Compatibility - Legal Disclaimer Overview ======== This software package provides basic support for Intel(R) Resource Director Technology (Intel(R) RDT) including: Cache Monitoring Technology (CMT), Memory Bandwidth Monitoring (MBM), Cache Allocation Technology (CAT) and Code and Data Prioritization (CDP). In principle, the software programs the technologies via Model Specific Registers (MSR) on a per core or hardware thread basis. MSR access is arranged via standard operating system driver: msr on Linux and cpuctl on FreeBSD. Presence of the technologies is detected via CPUID instruction. In some cases CAT is detected via brand string read from CPUID. Software detects presence of Linux perf extensions for the monitoring technologies. In such case it enables library API's for monitoring processes. This software package is maintained, updated and developed on https://github.com/01org/intel-cmt-cat https://github.com/01org/intel-cmt-cat/wiki provides FAQ, usage examples and useful links. Please refer to INSTALL file for package installation instructions. Package Content =============== "lib" directory: Includes software library files providing API's for technology detection, monitoring and allocation. Please refer to the library README for more details (lib/README). "pqos" directory: Includes utility files providing command line access to Intel(R) RDT. The utility links against the library and programs the technologies via its API's. Please refer to the utility README for more details "pqos/README". Manual page of "pqos" utility also provides information about tool usage: $ man pqos "rdtset" directory: Includes files of utility that provides "taskset"-like functionality and CAT configuration. The utility links against the library and programs the technologies via its API's. Please refer to the utility README for more details "rdtset/README". Manual page of "rdtset" utility also provides information about tool usage: $ man rdtset "examples" directory: Includes C and Perl examples of Intel(R) RDT via the library API's. Please refer to README file for more details "examples/README". "srpm" directory: Includes *.src.rpm and *.spec files for the software package. Please refer to README file for more details "srpm/README". "ChangeLog" file: Brief description of changes between releases. "INSTALL" file: Installation instructions. "LICENSE" file: License of the package. Hardware Support ================ Table 1. Intel(R) RDT hardware support +-----------------------------------------------------------+ | | CMT | MBM | CAT | CDP | |---------------------------------+-----+-----+-------+-----| |Intel(R) Xeon(R) processor E5 v3 | Yes | No |Yes (1)| No | |---------------------------------+-----+-----+-------+-----| |Intel(R) Xeon(R) processor D | Yes | Yes |Yes (2)| No | |---------------------------------+-----+-----+-------+-----| |Intel(R) Xeon(R) processor E3 v4 | No | No |Yes (3)| No | |---------------------------------+-----+-----+-------+-----| |Intel(R) Xeon(R) processor E5 v4 | Yes | Yes |Yes (2)| Yes | +-----------------------------------------------------------+ References: (1) Selected SKU's only: Intel(R) Xeon(R) processor E5-2658 v3 Intel(R) Xeon(R) processor E5-2648L v3 Intel(R) Xeon(R) processor E5-2628L v3 Intel(R) Xeon(R) processor E5-2618L v3 Intel(R) Xeon(R) processor E5-2608L v3 Intel(R) Xeon(R) processor E5-2658A v3 Four CAT classes of service (CLOS) and a set of pre-defined classes that should not be changed at run time. CAT CLOS to hardware thread association can be changed at run time. (2) Sixteen CAT classes of service (CLOS). There are no pre-defined classes of service and they can be changed at run time. CAT CLOS to hardware thread association can be changed at run time. (3) Selected SKU's only: Intel(R) Xeon(R) processor E3-1258L v4 Intel(R) Xeon(R) processor E3-1278L v4 Four CAT classes of service (CLOS) and a set of pre-defined classes that should not be changed at run time. CAT CLOS to hardware thread association can be changed at run time. For additional Intel(R) RDT details please refer to the Intel(R) Architecture Software Development Manuals available at: http://www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html Specific information can be found in Chapters 17.16 and 17.17 (revision 058). Software Compatibility ====================== In short, use intel-cmt-cat or Intel(R) PCM software together with Linux perf and cgroup frameworks is not allowed at the moment. As disappointing as it is, use of Linux perf for CMT & MBM and intel-cmt-cat for CAT & CDP is not allowed. This is because Linux perf overrides existing CAT configuration during its operations. There are only two options to choose from in order to make use of CAT: - intel-cmt-cat software for CMT/MBM/CAT and CDP (core granularity only) - patch kernel with an out of tree cgroup patch (CAT) and only use perf for monitoring (CMT kernels 4.1+, MBM kernels 4.6+) Table 2. Software interoperability matrix +---------------------------------------------------------------------------+ | | intel-cmt-cat | Intel(R) PCM | Linux perf | Linux cgroup | |--------------+---------------+--------------+--------------+--------------| |intel-cmt-cat | Yes(1) | Yes(2) | No | No | |--------------+---------------+--------------+--------------+--------------| |Intel(R) PCM | Yes(2) | Yes | No | No | |--------------+---------------+--------------+--------------+--------------| |Linux perf | No | No | Yes | Yes(3) | |--------------+---------------+--------------+--------------+--------------| |Linux cgroup | No | No | Yes | Yes(3) | +---------------------------------------------------------------------------+ References: (1) pqos monitoring from intel-cmt-cat can detect other pqos monitoring processes in the system. rdtset from intel-cmt-cat detects other processes started with rdtset and it will not use their CAT/CDP resources. (2) pqos from intel-cmt-cat can detect that Intel(R) PCM monitors cores and it will not attempt to hijack the cores unless forced. However, if pqos monitoring is started first and then Intel(R) PCM is started then the latter one will hijack monitoring infrastructure from pqos for its use. (3) Linux cgroup is an out of tree patch https://github.com/vshiva1/linux/tree/catv15 Intel(R) PCM is available at https://software.intel.com/en-us/articles/intel-performance-counter-monitor Table 3. Intel(R) RDT software enabling status. +--------------------------------------------------------------+ | | Core | Task | CMT | MBM | CAT | CDP | |--------------+-------+-------+-------+-------+-------+-------| |intel-cmt-cat | Yes | No | Yes | Yes | Yes | Yes | |--------------+-------+-------+-------+-------+-------+-------| |Intel(R) PCM | Yes | No | Yes | Yes | No | No | |--------------+-------+-------+-------+-------+-------+-------| |Linux perf | Yes | Yes | Yes(1)| Yes(2)| No(3) | No(3) | |--------------+-------+-------+-------+-------+-------+-------| |Linux cgroup | No | Yes | No | No | Yes(4)| No | +--------------------------------------------------------------+ Legend: Core - use of technology with core granularity Task - use of technology per task or group of tasks References: (1) Linux kernel version 4.1 and newer (2) Linux kernel version 4.6 and newer (3) Linux perf corrupts CAT and CDP configuration even though it doesn't enable it (4) This is out of tree patch and relies on Linux perf enabling Legal Disclaimer ================ THIS SOFTWARE IS PROVIDED BY INTEL"AS IS". NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS ARE GRANTED THROUGH USE. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE, INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.