quantumlib/qsim

Detect SIMD architecture from qsimcirq

Closed this issue · 7 comments

95-martin-orion commented

Related to #313. In order for users to have access to the instruction set their machine supports, we need to allow qsimcirq to dynamically choose between the different simulator implementations present in C++. Currently this is handled by the use of simmux.h in the pybind layer, but to enable the use of wheels additional handling may be required.

sergeisakov commented

One can add a template parameter to SimulatorHelper and modify functions like qsim_simulate_expectation_values. That is, for instance,

std::vector<std::complex<double>> qsim_simulate_expectation_values(
    const py::dict &options,
    const std::vector<std::tuple<
                          std::vector<OpString<Cirq::GateCirq<float>>>,
                          unsigned>>& opsums_and_qubit_counts,
    uint64_t input_state) {
  return SimulatorHelper::simulate_expectation_values(
    options, opsums_and_qubit_counts, false, input_state);
}

can be modified as follows

std::vector<std::complex<double>> qsim_simulate_expectation_values(
    const py::dict &options,
    const std::vector<std::tuple<
                          std::vector<OpString<Cirq::GateCirq<float>>>,
                          unsigned>>& opsums_and_qubit_counts,
    uint64_t input_state) {
  auto instruction_set = DetectInstructions();

  switch (instruction_set) {
  case kAVX512:
    return SimulatorHelper<SimulatorAVX512<For>>::simulate_expectation_values(
      options, opsums_and_qubit_counts, false, input_state);
  case kAVX:
    return SimulatorHelper<SimulatorAVX<For>>::simulate_expectation_values(
      options, opsums_and_qubit_counts, false, input_state);
  case kSSE:
    return SimulatorHelper<SimulatorSSE<For>>::simulate_expectation_values(
      options, opsums_and_qubit_counts, false, input_state);
  default:
    return SimulatorHelper<SimulatorBasic<For>>::simulate_expectation_values(
      options, opsums_and_qubit_counts, false, input_state);
  }
}

We need to implement DetectInstructions that detects the SIMD instruction set. Also some additional changes are needed because not every case is covered by SimulatorHelper.

95-martin-orion commented

I think we can fulfill #334 with this by allowing users to pass in instruction_set to these methods (defaulting to DetectInstructions if not specified).

@sergeisakov, do you have the bandwidth to take this item? Otherwise we can check if @laurynasas would be interested in implementing the fix.

laurynasas commented

@95-martin-orion I'd be happy to work on this it's just that it would take me some time to set up and get started, but if that's ok sign me up!

sergeisakov commented

@laurynasas, thank you! I can help you with DetectInstructions.

laurynasas commented

I had some time to look at this, do we want to have something like this?

While this compiles and produces the Python wheel, I get the following error Fatal Python error: Illegal instruction when trying to run qsimcirq_tests on Github Actions VMs with the following CPUs (potentially without AVX512F support?):

For MacOS:
Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz

For Linux:
Intel(R) Xeon(R) CPU E5-2673 v3 @ 2.40GHz

The tests succeed on:

For Linux:
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz

For Windows:
Intel(R) Xeon(R) CPU E5-2673 v3 @ 2.40GHz
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz

I'm not sure why the same tests pass on the same CPU on Win but fails on Linux. Looking at the full stack trace seems like the
Illegal instruction failure occures at the very beginning on import qsimcirq:

Fatal Python error: Illegal instruction

Current thread 0x00007f9c012fa740 (most recent call first):
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 922 in create_module
  File "<frozen importlib._bootstrap>", line 571 in module_from_spec
  File "<frozen importlib._bootstrap>", line 658 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 955 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 971 in _find_and_load
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1023 in _handle_fromlist
  File "/home/runner/.local/lib/python3.6/site-packages/qsimcirq/qsim_circuit.py", line 19 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 678 in exec_module
  File "<frozen importlib._bootstrap>", line 665 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 955 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 971 in _find_and_load
  File "/home/runner/.local/lib/python3.6/site-packages/qsimcirq/__init__.py", line 1 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 678 in exec_module
  File "<frozen importlib._bootstrap>", line 665 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 955 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 971 in _find_and_load
  File "/home/runner/work/qsim/qsim/qsimcirq_tests/qsimcirq_test.py", line 19 in <module>
  File "/home/runner/.local/lib/python3.6/site-packages/_pytest/assertion/rewrite.py", line 170 in exec_module
  File "<frozen importlib._bootstrap>", line 665 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 955 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 971 in _find_and_load
  File "<frozen importlib._bootstrap>", line 994 in _gcd_import
  File "/usr/lib/python3.6/importlib/__init__.py", line 126 in import_module
  File "/home/runner/.local/lib/python3.6/site-packages/_pytest/pathlib.py", line 524 in import_path
  File "/home/runner/.local/lib/python3.6/site-packages/_pytest/python.py", line 578 in _importtestmodule
  File "/home/runner/.local/lib/python3.6/site-packages/_pytest/python.py", line 500 in _getobj
  File "/home/runner/.local/lib/python3.6/site-packages/_pytest/python.py", line 291 in obj
  File "/home/runner/.local/lib/python3.6/site-packages/_pytest/python.py", line 516 in _inject_setup_module_fixture
  File "/home/runner/.local/lib/python3.6/site-packages/_pytest/python.py", line 503 in collect
  File "/home/runner/.local/lib/python3.6/site-packages/_pytest/runner.py", line 341 in <lambda>
  File "/home/runner/.local/lib/python3.6/site-packages/_pytest/runner.py", line 311 in from_call
  File "/home/runner/.local/lib/python3.6/site-packages/_pytest/runner.py", line 341 in pytest_make_collect_report
  File "/home/runner/.local/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/home/runner/.local/lib/python3.6/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/home/runner/.local/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/home/runner/.local/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/home/runner/.local/lib/python3.6/site-packages/_pytest/runner.py", line 458 in collect_one_node
  File "/home/runner/.local/lib/python3.6/site-packages/_pytest/main.py", line 808 in genitems
  File "/home/runner/.local/lib/python3.6/site-packages/_pytest/main.py", line 634 in perform_collect
  File "/home/runner/.local/lib/python3.6/site-packages/_pytest/main.py", line 333 in pytest_collection
  File "/home/runner/.local/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/home/runner/.local/lib/python3.6/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/home/runner/.local/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/home/runner/.local/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/home/runner/.local/lib/python3.6/site-packages/_pytest/main.py", line 322 in _main
  File "/home/runner/.local/lib/python3.6/site-packages/_pytest/main.py", line 269 in wrap_session
  File "/home/runner/.local/lib/python3.6/site-packages/_pytest/main.py", line 316 in pytest_cmdline_main
  File "/home/runner/.local/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/home/runner/.local/lib/python3.6/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/home/runner/.local/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/home/runner/.local/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/home/runner/.local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 163 in main
  File "/home/runner/.local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 185 in console_main
  File "/home/runner/.local/bin/pytest", line 8 in <module>

Could this be due to the flags (-fopenmp -march=knl -mavx512f) that the C++ extension is compiled with?
Or upon the import the entire extension is read and the unsupported by CPU instructions are found (even though not used during runtime) hence the failure?

sergeisakov commented

Why do you use -march=knl? This flag is for the Knights Landing architecture (Intel Xeon Phi processors). Anyway, it seems this approach is not going to work. Sorry about that. The compiler can use AVX512 instructions even for SimulatorBasic if -mavx512f is specified. Probably we need four separate compilation units.

95-martin-orion commented

This is resolved. It may also be interesting to expose the choice of SIMD modes to the user in qsimcirq, but that is a separate concern.