ibmdb/python-ibmdb

build from source version 3.2.1 on MS-windows, ModuleNotFoundError: No module named 'ibm_db'

imavo opened this issue · 12 comments

imavo commented

  • Operating System Name: Win10 22H2 x65

  • db2level output from clidriver if in use: (not using clidriver but local Db2-server v11.5.9.0)

  • Target Db2 Server Version: v11.5

  • Python Version: 3.11 x64

  • ibm_db version: 3.2.1 (from source code python-ibmdb-master.zip)

  • For non-Windows, output of below commands:
    uname
    uname -m

  • Value of below environment variables if set:
    IBM_DB_HOME: is not set
    PATH: has Db2-server on the PATH and on the LIB and INCLUDE environment variables.
    LIB/LD_LIBRARY_PATH/DYLD_LIBRARY_PATH:

  • Test script to reproduce the problem. import ibm_db

  • For installation related issue, complete output of pip install ibm_db command. (Not using pip, because trying to build from source-code)

Steps to Reproduce:

This is a follow-on question for #793 , to take into account changes for python-ibm_db version 3.2.1

I'm building ibm_db from source-code on Microsoft-Windows x64 (win10 22h2)
using visual-studio 2019 , and python 3.11 x64 (as supplied with anaconda 2023.11).

My Db2-LUW v11.5.9.0 server is installed locally on Ms-Windows .

The IBM_DB_HOME environment is NOT set, but Db2 is on the PATH, LIB, INCLUDE environment vars.

Downloaded python-ibmdb-master for version 3.2.1 and unpacked.
python setup.py build --> completes without error

copy C:\Users\user1\Documents\python-ibmdb-master\build\lib.win-amd64-cpython-311\ibm_db.cp311-win_amd64.pyd C:\Users\user1\Documents\python-ibmdb-master\ibm_db64_py311.pyd

python setup.py install

==> completes without error

But when I attempt to use import ibm_db and connect to a local database, I get:
ModuleNotFoundError: No module named 'ibm_db'

My expectation is that if I run python setup.py build followed by python setup.py install that the result should be functionally equivalent to doing pip install ibm_db (with IBM_DB_HOME previously set to the local Db2). Is this a valid expectation?

I notice that there is no file ibm_db.py created, is this relevant?

What do I have to do , in addition to the python setup.py build and python setup.py install to get the resulting built image to import and run as if it had been installed via pip ?

How do I get the build/install process to create the ibm_db.py file automatically on MS-Windows?

bchoudhary6415 commented

@imavo
For installation of ibm_db on windows, firstly you need to set the IBM_DB_HOME and LIB path.
Please refer the link - https://github.com/ibmdb/python-ibmdb?tab=readme-ov-file#envvar
Windows :
set IBM_DB_HOME=C:/Users/userid/clidriver
set LIB=%IBM_DB_HOME%/lib;%LIB%

Then go for installation
Please make sure to set dll path of dependent library of clidriver before importing the module as:
os.add_dll_directory('path to clidriver installation until bin')
import ibm_db

Thank you

imavo commented

Can you please clarify the build instruction, specifically is the direct use of python setup.py build still supported on Microsoft-Windows for ibm_db, or must I used instead python -m build ?

As mentioned in my question, the local Db2-server directories/files are already on the PATH, and INCLUDE, and LIB environment variables. So why would IBM_DB_HOME need to be set when all these environment variables are already set?

bimalkjha commented

@imavo Try pip install ibm_db==3.2.3. It works with Python 3.12. Thanks.

imavo commented

@bimalkjha this request asks for instructions to BUILD FROM SOURCE on MS-Windows. It is not an installation question, so your recent update seems irrelevant.

Please can you add the build-from-source-code instructions to the README, and answer specifically whether we should be using the legacy python setup.py build or python -m build (or whether it matters).

bimalkjha commented

@imavo The manual installation step is documented here: https://github.com/ibmdb/python-ibmdb/blob/master/INSTALL.md#-4-python-ibm_db-installation-on-windows

I observed that after python setup.py build, you are copying file using command copy C:\Users\user1\Documents\python-ibmdb-master\build\lib.win-amd64-cpython-311\ibm_db.cp311-win_amd64.pyd C:\Users\user1\Documents\python-ibmdb-master\ibm_db64_py311.pyd. It is not required.

After success message of python setup.py install, run pip list command and check ibm_db is listed in output or not? If yes, then installation is ok.

After that, run below commands and on windows command prompt and share complete output:

db2level => Note down the install path to be used later with os.add_dll_directory()
python
>>> import os
>>> os.add_dll_directory('Install path as in the output of db2level/bin')
>>> import ibm_db
>>> ibm_db.connect(connstr, '', '')
>>> quit()

For installation of ibm_db from source code, instead of using setup.py file, you can use below commands too:

git clone https://github.com/ibmdb/python-ibmdb.git
cd python-ibm_db
python -m build
pip install dist/ibm_db-3.2.3-cp311-cp311-win_amd64.whl
pip list

C:\pywork\python-ibmdb>echo %IBM_DB_HOME%
%IBM_DB_HOME%

C:\pywork\python-ibmdb>db2level
DB21085I  This instance or install (instance name, where applicable: "*") uses
"64" bits and DB2 code release "SQL11059" with level identifier "060A010F".
Informational tokens are "DB2 v11.5.9000.832", "special_36648",
"DYN2311160803WIN64_36648", and Fix Pack "0".
Product is installed at "C:\nodework\clidriver" with DB2 Copy Name "IBM Data
Server Driver For ODBC and CLI".

C:\pywork\python-ibmdb>python
Python 3.10.10 (tags/v3.10.10:aad5f6a, Feb  7 2023, 17:20:36) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.add_dll_directory('C:\\nodework\\clidriver\\bin')
<AddedDllDirectory('C:\\nodework\\clidriver\\bin')>
>>> import ibm_db
>>> ibm_db.connect('dsn=sample','','')
>>> quit()
C:\pywork\python-ibmdb>

Thanks.

bimalkjha commented

Please can you add the build-from-source-code instructions to the README, and answer specifically whether we should be using the legacy python setup.py build or python -m build (or whether it matters).

You can use both. Either python setup.py build install or python -m build; pip install dist\wheelname.whl. I just tried both commands using latest code and it is working fine. Thanks.

bimalkjha commented

My PATH and LIB system level environment variables settings:

C:\pywork\python-ibmdb>echo %PATH%
C:\nodework\clidriver\bin;C:\nodework\clidriver\lib;C:\Python310\Scripts\;C:\Python310\;C:\Windows\system32;
C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32
\OpenSSH\;C:\ProgramData\chocolatey\bin;C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools
\MSBuild\Current\Bin\amd64;
C:\nodework\nodejs\;C:\Program Files\Git\cmd;C:\Users\bjha\AppData\Local\Microsoft\WindowsApps;
C:\Users\bjha\AppData\Roaming\npm;C:\Users\bjha\AppData\Local\Programs\Microsoft VS Code\bin;

C:\pywork\python-ibmdb>echo %LIB%
C:\nodework\clidriver\bin;C:\nodework\clidriver\lib;

C:\pywork\python-ibmdb>echo %IBM_DB_HOME%
%IBM_DB_HOME%

C:\pywork\python-ibmdb>
bimalkjha commented

@imavo Let me know if you have tried these steps and still facing issue. Thanks.

imavo commented

Why hide the build instructions away in a separate file? Why not have a link to the build instructions directly in the README , so that visitors to the homepage can see at a glance...

Separately, can you explain whether the two methods for building-from-source-code (i.e. python setup.py build etc , and python -m build) will create identical outputs? It seems the former might create an egg, while the latter might create a wheeel (if using current levels of all required components). Or have I missed something?

bimalkjha commented

@imavo Not hiding anything intentionally. Expectation is user will check INSTALL.md for detailed instructions about installation. This link in README.md (https://github.com/ibmdb/python-ibmdb?tab=readme-ov-file#to-install-ibm_db-on-zos-system) also refers the INSTALL.md file. README.md has lot of other info, so detailed installation steps kept in separate file. Yes, we can add a line with link in README.md under installation section.

You are right that python setup.py install creates egg while python -m build creates wheel. python -m build is the new recommended way by Python community to use over setup.py method which may get removed in future version of python.
Agree that we need to update few docs as it it has old info. Thanks.

imavo commented

I would not think to look in INSTALL.md for build, compilation, linking/libraries information and pre-reqs for building. The Installation seems a different activity than developing and building (although of course interconnected). Only a tiny percentage of users will build from source, but nearly all users will perform installation. Different audiences. That is the reason I suggested a more upfront link in the README.

Yes, so many of the docs are in need of some TLC.

bimalkjha commented

Added upfront link in README just before Installation section here: https://github.com/ibmdb/python-ibmdb?tab=readme-ov-file#manual-installation
Closing the issue now. Thanks.