Missing None Check for dbms_name
tkwatkins88 opened this issue · 17 comments
This commit adds logic for determining the reflector class based on the value of dbms_name.
The value of dbms_name is defaulted to None if the value does not exist on the connection. There is currently no handling for the value of None, which results in the below error for some connection URLs.
TypeError: argument of type 'NoneType' is not iterable
I am seeing this when attempting to use the below connection URL.
db2+pyodbc400://me:password1@127.0.0.2:8471/MYDB
The current logic is implemented to select the reflector at run time based on server type after connection is established.
The dbms_name logic helps to determine the server type whether its DB2 ZOS, DB2 For luw, Db2 on Iseries or informix.
Can you provide a bit more details on what kind of connection you are trying.
From the connection string looks like you are trying to use I-series ODBC driver for Iseries connection.
Thanks.
The problem seems to be in base.py, DB2Dialect, initialize(), line 703 and 712.
703: self.dbms_name = getattr(connection.connection, 'dbms_name', None)
...
if(self.dbms_name == 'AS'):
_reflector_cls = ibm_reflection.AS400Reflector
elif(self.dbms_name == "DB2"):
_reflector_cls = ibm_reflection.OS390Reflector
712: elif("DB2/" in self.dbms_name):
_reflector_cls = ibm_reflection.DB2Reflector
elif("IDS/" in self.dbms_name):
_reflector_cls = ibm_reflection.DB2Reflector
The checks for str in self.dbms_name throw an error when self.dbms_name is set to None, which it sometimes is.
Earlier in the code, I saw an example where someone did this:
if dbms_name != None and (dbms_name.find('DB2/') != -1), which protects against doing things against a None value.
HI @odoraf
With respect to the code changes made, the understanding behind the implementation was, when ever any connection is made via the clidriver(db2 clidriver) then dbms_name will return the corresponding values as per the server type
Example :
server Type dbms_name expected
DB2 for LUW DB2/Linuxx8664 -> for linux based instances
DB2/NTX64 -> for windows bases instances
DB2 for ZOS DB2
DB2 for Iseries AS
Informix IDS/xxxxxx -> based on the client platform installed
With this understanding the code changes is made.
Now as per your comment looks like there are cases when dbms_name is NONE. My guess is this is due to the use of the i-series driver which is used.
I have one suggestion to make in this regard, can you please try connection using the dialect as ibm_db_sa do that it uses the clidriver for connection.
And as far as the None check is considered, I will make the code changes into a new branch and notify you, can you please test the code from the branch if that works out for you so that we can take care of the condition in the new release.
Thanks.
I also happen to have the same error when using reflection with ibm_db_sa==0.3.6.
The code i use to connect is
from urllib.parse import quote_plus, unquote_plus
from sqlalchemy import create_engine
from sqlalchemy.ext.automap import automap_base
import urllib
# Allows to solve another problem with the location of unquote_plus
urllib.unquote_plus = unquote_plus
CONNECTION_STRING = "ibm_db_sa+pyodbc:///?odbc_connect={}".format(quote_plus(
"driver={iSeries Access ODBC Driver};"
"system=IPADDR;"
"uid=username;"
"pwd=password;"
))
engine = create_engine(CONNECTION_STRING)
base = automap_base()
base.prepare(engine=engine, reflect=True)
for table_name in base.metadata.tables.keys():
print("FOUND TABLE", table_name)I'm using python 3.9.1 on Windows 64-bit.
SQLAlchemy==1.3.22
pyodbc==4.0.30
ibm-db-sa==0.3.6
When I run the above MWE I obtain None as output from line ibm_db_sa/base.py:707 and then line 712 in the same file errors out because NoneType is not iterable.
How can I debug such an issue? From what I understand it should not happen, right?
Hi,
I will do the code changes for this and deliver the same in a particular branch.
can you please help me test the same in your environment to check if that works.
I will come up with the branch and update.
Thanks.
Hi @Z3r021
Sorry for the delay.
I have handled the None type check and redirected it to the default reflector in case the self.dbms_name returns None.
Can you please test the code from branch new_dev_fixes and let me know your findings.
https://github.com/ibmdb/python-ibmdbsa/tree/new_dev_fixes
If it all goes fine I will take care of this in the new release.
Thanks.
@amukherjee28, I'm trying to test this but am running into issues. I am trying to install the branch into my environment as I have done with other repos / packages, but am getting errors when I issue the below install.
pipenv install -e git+https://github.com/ibmdb/python-ibmdbsa.git@new_dev_fixes#egg=package_name
Installing -e git+https://github.com/ibmdb/python-ibmdbsa.git@new_dev_fixes#egg=package_name...
Error: An error occurred while installing -e git+https://github.com/ibmdb/python-ibmdbsa.git@new_dev_fixes#egg=package_name!
Error text: Obtaining package_name from git+https://github.com/ibmdb/python-ibmdbsa.git@new_dev_fixes#egg=package_name (from -r /var/folders/3r/97_71yjx2px_3kwczdg_pn8m0000gq/T/pipenv-liez9gz7-requirements/pipenv-ftxmu_5r-requirement.txt (line 1))
Updating /Users/user/.local/share/virtualenvs/payswitch-api-ejc-B3da/src/package-name clone (to revision new_dev_fixes)ERROR: Command errored out with exit status 1: command: /Users/user/.local/share/virtualenvs/payswitch-api-ejc-B3da/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/Users/user/.local/share/virtualenvs/payswitch-api-ejc-B3da/src/package-name/setup.py'"'"'; __file__='"'"'/Users/user/.local/share/virtualenvs/payswitch-api-ejc-B3da/src/package-name/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/3r/97_71yjx2px_3kwczdg_pn8m0000gq/T/pip-pip-egg-info-wdwsvdjd cwd: /Users/user/.local/share/virtualenvs/payswitch-api-ejc-B3da/src/package-name/ Complete output (5 lines): Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/local/Cellar/python@3.9/3.9.1_8/Frameworks/Python.framework/Versions/3.9/lib/python3.9/tokenize.py", line 392, in open buffer = _builtin_open(filename, 'rb') FileNotFoundError: [Errno 2] No such file or directory: '/Users/user/.local/share/virtualenvs/payswitch-api-ejc-B3da/src/package-name/setup.py' ----------------------------------------ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
From the error look like some kind of issue while installing the package from git branch.
I will have a look at this with python version 3.9.
Mean while can you clone the code from the branch and use the setup.py to install.
python setup.py install
Can you use this command to install instead of the other approach.
Thanks.
@amukherjee28, I am still running into the same issue and see where the problem is. Because self.dbms_name is None, when we get to the checks on lines 711 and 713 ("elif("DB2/" in self.dbms_name)" and "elif("IDS/" in self.dbms_name)", respectively), we are attempting to see if the value is in None, which leads to the "TypeError: argument of type 'NoneType' is not iterable" error.
Instead of adding an "else" block, if we were to add another "elseif" block before the one on line 711 and have it check to see if self.dbms_name is None, it should work. It would look like the below.
# reflection: these all defer to an BaseDB2Reflector # object which selects between DB2 and AS/400 schemas def initialize(self, connection): self.dbms_ver = getattr(connection.connection, 'dbms_ver', None) self.dbms_name = getattr(connection.connection, 'dbms_name', None) DB2Dialect.serverType = self.dbms_name super(DB2Dialect, self).initialize(connection) #check server type logic here if(self.dbms_name == 'AS'): _reflector_cls = ibm_reflection.AS400Reflector elif(self.dbms_name == "DB2"): _reflector_cls = ibm_reflection.OS390Reflector elif(self.dbms_name is None): _reflector_cls = ibm_reflection.DB2Reflector elif("DB2/" in self.dbms_name): _reflector_cls = ibm_reflection.DB2Reflector elif("IDS/" in self.dbms_name): _reflector_cls = ibm_reflection.DB2Reflector self._reflector = _reflector_cls(self)
Hi,
Thank you for taking the effort and testing it. I really appreciate it.
I have made the changes that you requested. Can you please test it just but doing a git pull and then installing it again by using the same command:
python setup.py install.
Thanks again.
@amukherjee28, same issue because the new elif block on line 715 needs to come before the elif blocks on 711 and 713, as shown in the code I supplied. If self.dbms_name is None, we cannot perform an in comparison, hence us putting the None check before the two in comparisons.
Also, no problem on the testing. Thank you for working to correct the issue!
Done. Hopefully now it should work fine.
That did the trick @amukherjee28! Thanks for your help working on this.
Perfect...great to hear that.
Thanks a lot for your patience and getting it done.
New version is released and this issue had been addressed in the new release 0.3.7.
Please check and let me know if this can be closed.
@amukherjee28, this does seem to fix the issue we had ran into. I think you are safe to close this issue. Thanks for your help.