bravegag/z9pe-d8_ws-optimization-guide

Optimization guide for the motherboard ASUS Z9PE-D8 WS

ASUS Z9PE-D8_WS motherboard Optimization Guide

This is a Performance Optimization guide for the ASUS Z9PE-D8_WS motherboard. The guide initially offers optimizations geared towards HPC but can be extended for other purposes e.g. gaming performance. The optimization includes performance-related BIOS parameters as well as OS settings. The motivation of this Guide is to also prepare an Experimental Design to find the best of the different parameter combinations.

Why this guide? The Z9PED8 WS is arguably the most powerful Worstation motherboard out there and a very appealing solution for HPC as well as an above average gaming rig.

The main reference for this guid is the official ASUS Users Manual.

BIOS Performance Settings

The following selected BIOS settings (or factors) affect performance in various ways and their values (or factor levels) are discussed:

• AI Tweaker Menu
  • AI Overclock Tuner [Auto]: TODO
  • Clock Spread Spectrum [Disabled]: disabled by default, enhances AI overclocking.
• DRAM Timing Control
  • DRAM latencies WRITE, READ etc.: TODO interesting to try different values maybe to match those of the RAM specifications.
• CPU Configuration
  • Execute Disabled Bit [Enabled –> Disabled]: The Execute Disabled Bit (EDB) prevents malicious buffer overflow attacks when combined with a supporting OS (only enterprise OS versions support this) e.g. Ubuntu does not support it so we can safely disable it. Researching this feature reveals that it may degrade performance and may even cause application/system crashes in non supported OS.
  • Server Class [Custom –> HPC]: options include [Custom, Enterprise or HPC] low level cache prefetching behavior TODO.
  • Intel Virtualization Technology [Enabled –> Disabled]: working in native, there is no need to enable I-VT, researching this parameter turns out that it inhibits automatic overclocking and may ceil performance in native mode.
  • Local APIC Mode [Auto]: APIC (Advanced Programmable Interrupt Controllers) relevant for multi-processor configurations see http://en.wikipedia.org/wiki/Local_APIC and http://en.wikipedia.org/wiki/Local_APIC#Problems
• CPU Power Management
  • Power Technology [Custom]: Controls power management features TODO.
  • Power Technology [Custom] –> Enhanced Intel SpeedStep Technology [Enabled –> Disabled]: may slow things down as CPUs are continuously tuned for power consumption/performance trade-off, may not relevant for HPC machines.
  • Energy Performance [Balanced Performance –> Performance]: optimize power management towards performance.
• Chipset QPI Configuration
  • Isoc [Enabled]: TODO Vudu parameter but sounds important :)
  • QPI Link Speed Mode [Fast]: QPI (Intel QuickPath interconnect) protocol that allows fast direct connection between two CPUs.
  • QPI Link Frequency Select [Auto ?–> 8.0 GT/s]: TODO another Vudu parameter worth further investigating, there is no clear “higher is better” as it seems to depend on other CPU parameters e.g. voltage, overclocking etc.
  • QPI Link0s [Disabled]: TODO Vudu parameter, no explanation further research may be interesting.
  • QPI Link0p [Disabled]: TODO Vudu parameter, no explanation further research may be interesting.
  • QPI Link1 [Enabled]: TODO Vudu parameter, no explanation found but may be interesting researching the effects of this. http://www.ubuntu.com/certification/catalog/component/pci/8086:3c90/ “Unknown” category in Ubuntu, may be worth disabling.
• Memory Configuration
  • Compatibility RID [Enabled]: related to compatibility with Sandybridge BIOS spec, further research may be interesting since compatibility settings tend to affect performance in various ways.
  • Memory Mode [Independent]: explained in detail in this guide http://globalsp.ts.fujitsu.com/dmsp/Publications/public/wp-nehalem-memory-performance-en.pdf from what I understand the Independent mode should lead to performance if the DIMM strips are positioned correctly, we only have one type and all DIMM slots are fully occupied so it should be performance, verify?
  • NUMA [Enabled]: http://en.wikipedia.org/wiki/Non-uniform_memory_access
  • DDR Speed [Auto –> Force DDR 1600Mhz]: fastest supported memory speed already tried increasing to the maximum for the memory model 1866Mhz and the board would no longer boot.
  • Channel Interleaving [Auto]: may be chosen based on the optimal setting of the DIMM memory, research the values and verify that the optimal values are indeed chosen.
  • Rank Interleaving [Auto]: may be chosen based on the optimal setting of the DIMM memory, research the values and verify that the optimal values are indeed chosen.
  • Patrol Scrub [Disabled]: only relevant for ECC memory better disabled for non-ECC memory, may incur in performance overhead. http://en.wikipedia.org/wiki/Memory_scrubbing
  • Demand Scrub [Enabled –> Disabled]: only relevant for ECC memory better disabled for non-ECC memory, may incur in performance overhead. http://en.wikipedia.org/wiki/Memory_scrubbing#Demand_Scrub
  • Data Scrambling [Enabled –> Disabled]: only relevant for ECC memory better disabled for non-ECC memory, may incur in performance overhead. http://en.wikipedia.org/wiki/Data_scrubbing
  • Device Tagging [Disabled ?–> Enabled]: further research needed but appears to offer increasing the amount of main memory cacheable in L2 http://www.webopedia.com/TERM/T/tag_RAM.html
  • Allow 2x Refresh [Enabled]: allow modifying the 2x refreshing function, may have an impact in performance and may be worth further investigating. http://en.wikipedia.org/wiki/Memory_refresh
• CPU IIO Bridge
  • Intel I/OAT [Disabled –> Enabled]: enable I/O acceleration technology, High-Speed Networking acceleration http://www.intel.com/content/www/us/en/wireless-network/accel-technology.html, http://www.intel.com/content/www/us/en/io/i-o-acceleration-technology-paper.html http://en.wikipedia.org/wiki/Direct_memory_access#I.2FOAT
  • DCA Support [Enabled]: Direct Cache Access (DCA) allows a capable I/O device, such as a network controller, to place data directly into CPU cache, reducing cache misses and improving application response times.
• Intel VT-d [Enabled –> Disabled]: hardware assisted VT for directed I/O configuration, not relevant for native configurations.
• PCI Subsystem Settings –> PCI Latency Timer [32 PCI Bus Clocks –> e.g. 64]: TODO good potential finding the best performance parameters for HPC.
• PCI Express Settings –> ASPM Support [Disabled]: Active State Power Management controls power consumption of PCI-E peripherals.
• ACPI Settings –> Enable ACPI Auto Configuration [* –> Disabled]: disable all ACPI settings e.g. disable Hibernation.
• WHEA Configuration –> WHEA Support [Enabled –> Disabled]: Windows Hardware Error Architecture support, disable when using Ubuntu.
• Onboard LAN Configuration –> Intel W82574L OpROM1 [PXE –> iSCSI]: TODO try the setting iSCSI for improved LAN performance (not sure yet whether this will increase the LAN speed to 1000 MB/s) http://www.intel.com/network/connectivity/products/316975.pdf http://en.wikipedia.org/wiki/ISCSI

Operating System Settings

Ubuntu 12.04

• CPU Configuration –> Local APIC Mode [Auto] may slow things down and risk stability issues and often advised to disable while troubleshooting Ubuntu issues GRUB_CMDLINE_LINUX_DEFAULT="quiet splash reboot=bios acpi=off noapic nolapic"
• ACPI Settings –> Enable ACPI Auto Configuration [* –> Disabled]: ACPI (Advanced Configuration and Power Interface) may explain [booting/system stability/segfault/performance] issues and it is often advised to be disabled in Ubuntu. Disable it by editing grub to GRUB_CMDLINE_LINUX_DEFAULT="quiet splash reboot=bios acpi=off noapic nolapic"

Windows 7

Experiments