ev-br/mc_lib

windows build

Opened this issue · 18 comments

ev-br commented

a checklist at #14 (comment)

noDGodiaev commented
  1. My previous problems with MinGW backend were due to the combination of native python 32-bit and 32-bit version of MinGW. (If it would be useful, I can test windows build with MinGW x64)
  2. As we discussed, I tried msvc as a compiler. Using it with meson needs some magic passes described here.
  3. Building a simple python app: Successful
  4. Building a simple cython C app:
warning C4115: timeval: определение именованного типа в круглых скобках
simple_cy.pyx.c(1240): warning C4100: def: неиспользованный формальный параметр
simple_cy.pyx.c(1707): warning C4127: условное выражение является константой
...
 Linking target cy.pyd
FAILED: cy.pyd cy.pdb
"link"  /MACHINE:x64 /OUT:cy.pyd cy.pyd.p/meson-generated_.._simple_cy.pyx.c.obj "/nologo" "/release" "/nologo" "/DEBUG"
 "/PDB:cy.pdb" "/DLL" "/IMPLIB:cy.lib" "C:\Users\Dmitry\anaconda3\envs\meson_win\libs\python37.lib"
 "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "shell32.lib" "ole32.lib" "oleaut32.lib"
 "uuid.lib" "comdlg32.lib" "advapi32.lib"
LINK : fatal error LNK1104: не удается открыть файл "python37_d.lib"
ninja: build stopped: subcommand failed.

Python == 3.7
Meson == 0.59
MSVC2019

ev-br commented

No idea TBH. Is it looking for a debug python build (_d )?

noDGodiaev commented

Seems like --buildtype=release solved this. I need some time for testing the installation.
Full command: meson build --buildtype=release --prefix=D:\GitKraken\meson_win\mc_lib\hello_world_dir
Log:

[3/3] Linking target cy.pyd
   Создается библиотека cy.lib и объект cy.exp
Installing cy.pyd to D:\GitKraken\meson_win\mc_lib\hello_world_dir\lib\python3.7\site-packages\hello_world
Installing cy.lib to D:\GitKraken\meson_win\mc_lib\hello_world_dir\lib\python3.7\site-packages\hello_world
File 'cy.pdb' not found, skipping
Installing D:\GitKraken\meson_win\mc_lib\hello_world_dir\simple_py.py to D:\GitKraken\meson_win\mc_lib\hello_world_dir\lib\python3.7\site-packages\hello_world
  • 👍1
noDGodiaev commented

--buildtype=release also works for mc_lib, pytest works.

I still have questions to solve:

  • export on windows

  • PEP517 installation

ev-br commented

If it would be useful, I can test windows build with MinGW x64)

That's quite a rabbit hole. If the MSVC build works, it's good enough, I think.

noDGodiaev commented

set may be the option, but for me:
echo %PYTHONPATH% returns %PYTHONPATH%

I have no great desire to play with system variables. So as the option of testing I suggest cd to installdir and make tests there.

ev-br commented

If cd installdir works, it's OK for now. Please leave a comment somewhere though.

noDGodiaev commented

PEP517 installation with msvc:

Building wheel for mc-lib (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: 'C:\Users\Dmitry\anaconda3\envs\meson_win\python.exe' 'C:\Users\Dmitry\anaconda3\envs\meson_win\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py' build_wheel 'C:\Users\Dmitry\AppData\Local\Temp\tmp1dzkf_k4'
       cwd: C:\Users\Dmitry\AppData\Local\Temp\pip-req-build-axco_vi_
  Complete output (21 lines):
  C:\Users\Dmitry\AppData\Local\Temp\pip-build-env-4lsdrkdd\overlay\Lib\site-packages\mesonpep517\pep425tags.py:66: RuntimeWarning: Config variable 'Py_DEBUG' is unset, Python ABI tag may be incorrect
    if get_flag("Py_DEBUG", hasattr(sys, "gettotalrefcount"), warn=precond):
  meson-python-option-name not specified in the [tool.mesonpep517.metadata] section, assuming `python3`
  Python Traceback (most recent call last):
    File "C:\Users\Dmitry\anaconda3\envs\meson_win\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 349, in <module>
      main()
    File "C:\Users\Dmitry\anaconda3\envs\meson_win\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 331, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "C:\Users\Dmitry\anaconda3\envs\meson_win\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 249, in build_wheel
      metadata_directory)
    File "C:\Users\Dmitry\AppData\Local\Temp\pip-build-env-4lsdrkdd\overlay\Lib\site-packages\mesonpep517\buildapi.py", line 372, in build_wheel
      wheel_directory), config_settings, metadata_directory)
    File "C:\Users\Dmitry\AppData\Local\Temp\pip-build-env-4lsdrkdd\overlay\Lib\site-packages\mesonpep517\buildapi.py", line 340, in build
      abi = get_abi(python)
    File "C:\Users\Dmitry\AppData\Local\Temp\pip-build-env-4lsdrkdd\overlay\Lib\site-packages\mesonpep517\buildapi.py", line 302, in get_abi
      return subprocess.check_output([python, '-c', GET_CHECK]).decode('utf-8').strip('\n')
    File "C:\Users\Dmitry\anaconda3\envs\meson_win\lib\subprocess.py", line 411, in check_output
      **kwargs).stdout
    File "C:\Users\Dmitry\anaconda3\envs\meson_win\lib\subprocess.py", line 512, in run
      output=stdout, stderr=stderr)
  subprocess.CalledProcessError: Command
 '['python3', '-c', '\nfrom mesonpep517 import pep425tags\nprint("{0}{1}-{2}".format(pep425tags.get_abbr_impl(),\n
pep425tags.get_impl_ver(),\n
pep425tags.get_abi_tag())\n)\n']' returned non-zero exit status 9009.
  ----------------------------------------
  ERROR: Failed building wheel for mc-lib
Failed to build mc-lib
ERROR: Could not build wheels for mc-lib which use PEP 517 and cannot be installed directly
noDGodiaev commented

Seems like --buildtype=release solved this. I need some time for testing the installation.

We can put this option inside meson.build:

project(
  'mc_lib',
  ['cpp', 'cython'],
  default_options : ['warning_level=3', 'buildtype=release'],
  version: '0.3',
  license: 'BSD-2',
  meson_version: '>= 0.54',
)
ev-br commented

We can put this option inside meson.build

Unless it messes up other OSes, that is. Which CI will tell us.

noDGodiaev commented

subprocess.CalledProcessError: Command
'['python3', '-c', '\nfrom mesonpep517 import pep425tags\nprint("{0}{1}-{2}".format(pep425tags.get_abbr_impl(),\n
pep425tags.get_impl_ver(),\n
pep425tags.get_abi_tag())\n)\n']' returned non-zero exit status 9009.

The main reason for this error: build system try command with python3, while on windows only python exists.
And, I don't know how to change this(

ev-br commented

There has got to be a way to specify the interpreter and it's name, otherwise it's a royal mess (eg what if I've two interpreters, py3.8 and 3.9).

Edit : not that I have a slightest clue how to tell it that. Something with find_installation("python")?

ev-br commented

Currest status: on CI dev build seems to work with a vanilla MSVC, see gh-62. The current blocker is how to cook up the incantation for the PYTHONPATH in PowerShell -- which GH actions use instead of cmd. With hardcoded paths the build seems to be OK if noisy.

@noDGodyaev @MoskalenkoRomanBorisovich is either of you is up to the challenge? :-).

ev-br commented

OK, fine, #62 is in. Follow-ups:

carlkl commented

@ev-br, you may take a look at cpymad. This project was the first one using the msys2 toolchain for CI building of a standard CPython package. See also: hibtc/cpymad#81

ev-br commented

@carlkl Thanks for the link! Will take a look.
TBH this is somewhat low prio for this project at the moment (if msvc works, it's good enough for now). That said, it's great to have a link to look at when the need arises! And if past experience is any guide, it will at some point :-).

ev-br commented

https://gitlab.com/thiblahute/mesonpep517/-/merge_requests/17 has been merged (April 2022), now waiting for a pypi release : the current version is v0.2, from Feb 2022.