Intel P cores are not used with --isolation=process

[SauliusZ87]

Description

When running docker job in isolation mode "process" with new Intel processors (with Performance and Efficiency cores) only efficiency cores are being used.
OS: Windows 11
Processor: i9-12900K

In reproduce section I use prime number calculator, but same behaviour can be reproduced using cpu burn

Reproduce

1. Download prime number calculator from github https://github.com/kimwalisch/primesieve/releases/download/v11.1/primesieve-11.1-win-x64.zip
2. Extract into folder C:\prime
3. run docker in standard mode (hyper-v) using command docker run --rm --cpus=24 -v "C:\prime:C:\prime" mcr.microsoft.com/windows/nanoserver:ltsc2022 C:\prime\primesieve 1e12
4. run docker in isolation process mode using command docker run --rm --cpus=24 --isolation=process -v "C:\prime:C:\prime" mcr.microsoft.com/windows/nanoserver:ltsc2022 C:\prime\primesieve 1e12

Expected behavior

I was expecting job to finish in all modes in similar time, but process isolation takes significantly more time. From task manager I can see that initially docker takes all cores, but then immediately drops P cores, and only works on E cores (as can be seen in pictures).

With hyper-v mode job finishes in 15.8 seconds
docker run --rm --cpus=24 -v "C:\prime:C:\prime" mcr.microsoft.com/windows/nanoserver:ltsc2022 C:\prime\primesieve 1e12

Sieve size = 384 KiB
Threads = 24
100%
Seconds: 15.808
Primes: 37607912018

With process isolation mode job finishes in 73 seconds
docker run --rm --cpus=24 --isolation=process -v "C:\prime:C:\prime" mcr.microsoft.com/windows/nanoserver:ltsc2022 C:\prime\primesieve 1e12

Sieve size = 256 KiB
Threads = 24
100%
Seconds: 73.083
Primes: 37

When job is started all cores are used very briefly.

But then work continues only on E cores

607912018

When same command is run localy from command shell (wihout docker) job finishes in 13.7 seconds
C:\prime\primesieve 1e12

Sieve size = 256 KiB
Threads = 24
100%
Seconds: 13.765
Primes: 37607912018

docker version

Client:
 Cloud integration: v1.0.33
 Version:           24.0.2
 API version:       1.43
 Go version:        go1.20.4
 Git commit:        cb74dfc
 Built:             Thu May 25 21:53:15 2023
 OS/Arch:           windows/amd64
 Context:           default

Server: Docker Desktop 4.20.1 (110738)
 Engine:
  Version:          24.0.2
  API version:      1.43 (minimum version 1.24)
  Go version:       go1.20.4
  Git commit:       659604f9
  Built:            Thu May 25 21:52:13 2023
  OS/Arch:          windows/amd64
  Experimental:     false

docker info

Client:
 Version:    24.0.2
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.5
    Path:     C:\Program Files\Docker\cli-plugins\docker-buildx.exe
  compose: Docker Compose (Docker Inc.)
    Version:  v2.18.1
    Path:     C:\Program Files\Docker\cli-plugins\docker-compose.exe
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-dev.exe
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.19
    Path:     C:\Program Files\Docker\cli-plugins\docker-extension.exe
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.4
    Path:     C:\Program Files\Docker\cli-plugins\docker-init.exe
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-sbom.exe
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-scan.exe
  scout: Command line tool for Docker Scout (Docker Inc.)
    Version:  v0.12.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-scout.exe

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 1
 Server Version: 24.0.2
 Storage Driver: windowsfilter
  Windows:
 Logging Driver: json-file
 Plugins:
  Volume: local
  Network: ics internal l2bridge l2tunnel nat null overlay private transparent
  Log: awslogs etwlogs fluentd gcplogs gelf json-file local logentries splunk syslog
 Swarm: inactive
 Default Isolation: hyperv
 Kernel Version: 10.0 22621 (22621.1.amd64fre.ni_release.220506-1250)
 Operating System: Microsoft Windows Version 22H2 (OS Build 22621.1848)
 OSType: windows
 Architecture: x86_64
 CPUs: 24
 Total Memory: 31.75GiB
 Name: DESKTOP-4QGHMCF
 ID: a6cc8889-dc91-466c-a97a-5fba91eb68e1
 Docker Root Dir: C:\ProgramData\Docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

Diagnostics ID

F2C98BA0-E372-4B88-8D67-811B82CF5369/20230622132836

Additional Info

No response

[conioh]

I'd love nothing more than to blame Docker for this, but this times it seems like the problem is on Microsoft's side. See the reproduction using CmDiag.exe in the issues I opened against Microsoft.

[Howard-Haiyang-Hao]

@conioh, Thank you for bringing the issue to my attention. I'll make an effort to reproduce the problem and share my findings with you. In general, processes running inside process isolated containers behave similarly to regular processes on the host, running in a different session. These processes' threads are scheduled by the host kernel, and they shouldn't always be exclusively scheduled on the hyperthread cores. I'll conduct further investigation and provide you with an update shortly.

[conioh]

@Howard-Haiyang-Hao, thanks for the reply. I believe the reproduction instructions I provided in the issues in the Microsoft repositories should be pretty easy to follow. Please note that on Intel CPUs the issue is that the container threads are scheduled only on the E-cores - the non-hyperthreaded cores. I look forward to your update.

[conioh]

@Howard-Haiyang-Hao, have you succeeded in reproducing the issue?