[BUG] Cannot import BlazingContext when processor type unknown
callofdutyops opened this issue · 2 comments
Describe the bug
Cannot import BlazingContext when processor type unknown.
Steps/Code to reproduce bug
Code and output from ipython (personal info hidden).
In [1]: from blazingsql import BlazingContext
---------------------------------------------------------------------------
FileNotFoundError Traceback (most recent call last)
<ipython-input-7-0b19b5b41f48> in <module>
----> 1 from blazingsql import BlazingContext
~/miniconda3/envs/blazingsql/lib/python3.7/site-packages/blazingsql/__init__.py in <module>
1 from pyblazing.apiv2 import S3EncryptionType
2 from pyblazing.apiv2 import DataType
----> 3 from pyblazing.apiv2.context import BlazingContext
4
5 from cio import getProductDetailsCaller
~/miniconda3/envs/blazingsql/lib/python3.7/site-packages/pyblazing/apiv2/context.py in <module>
105 )
106
--> 107 jpype.startJVM("-ea", convertStrings=False, jvmpath=jvm_path)
108 # jpype.startJVM()
109
~/miniconda3/envs/blazingsql/lib/python3.7/site-packages/jpype/_core.py in startJVM(*args, **kwargs)
225 try:
226 _jpype.startup(jvmpath, tuple(args),
--> 227 ignoreUnrecognized, convertStrings, interrupt)
228 initializeResources()
229 except RuntimeError as ex:
FileNotFoundError: [Errno 2] JVM DLL not found: /home/{my_username}/miniconda3/envs/blazingsql/lib/server/libjvm.so
In [2]: !uname -p
unknown
Expected behavior
Should be imported without any errors.
Environment overview (please complete the following information)
- Environment location: Bare-metal
- Method of BlazingSQL install: conda
- BlazingSQL Version
BlazingSQL version (git hash): 13618d177a37bd34bb20ac832fb8a14f8243ff5c
BlazingSQL branch name: HEAD
BlazingSQL branch tag: v21.08.02
BlazingSQL build id: 0
BlazingSQL compiler version: GNU /usr/local/gcc9/bin/g++ 9.4.0
BlazingSQL cuda flags: -Xcompiler -Wno-parentheses -gencode=arch=compute_60,code=sm_60 -gencode=arch=compute_70,code=sm_70 -gencode=arch=compute_75,code=sm_75 -gencode=arch=compute_75,code=compute_75 --expt-extended-lambda --expt-relaxed-constexpr -Werror=cross-execution-space-call -Xcompiler -Wall,-Wno-error=deprecated-declarations --default-stream=per-thread -DHT_DEFAULT_ALLOCATOR
BlazingSQL Operating system kernel: Linux-5.4.0-1054-aws
BlazingSQL Operating system architecture: x86_64
BlazingSQL Linux Operating system release: NAME=CentOS Linux|VERSION=7 (Core)|ID=centos|ID_LIKE=rhel fedora|VERSION_ID=7|PRETTY_NAME=CentOS Linux 7 (Core)|ANSI_COLOR=031|CPE_NAME=cpe:/o:centos:centos:7|HOME_URL=[https://www.centos.org/|BUG_REPORT_URL=https://bugs.centos.org/||CENTOS_MANTISBT_PROJECT=CentOS-7|CENTOS_MANTISBT_PROJECT_VERSION=7|REDHAT_SUPPORT_PRODUCT=centos|REDHAT_SUPPORT_PRODUCT_VERSION=7|](https://www.centos.org/%7CBUG_REPORT_URL=https://bugs.centos.org/%7C%7CCENTOS_MANTISBT_PROJECT=CentOS-7%7CCENTOS_MANTISBT_PROJECT_VERSION=7%7CREDHAT_SUPPORT_PRODUCT=centos%7CREDHAT_SUPPORT_PRODUCT_VERSION=7%7C)
----For BlazingSQL Developers----
Suspected source of the issue
https://github.com/BlazingDB/blazingsql/blob/branch-21.08/pyblazing/pyblazing/apiv2/context.py#L70
machine_processor = platform.processor()
if machine_processor in ("x86_64", "x64"):
machine_processor = "amd64"
when the uname -p
is unknown, platform.processor()
equals to ''
, thus machine_processor
is empty, which leads to wrong jvm lib path.
I think change machine_processor = platform.processor()
to machine_processor = platform.processor() or platform.machine()
can fix this.
For those who suffer this, tmp solution is monkey patch:
import platform
def x64():
return "x86_64"
platform.processor = x64