"__file__" variable cannot be used when running with frozen program
hahuunguyen opened this issue · 11 comments
After freezing my Python application, I got an error while getting a certificate.
File "/usr/lib/python3.5/importlib/_bootstrap.py", line 816, in exec_module
exec(code, module.__dict__)
File "/usr/local/lib/python3.5/dist-packages/requests/utils.py", line 39, in <module>
DEFAULT_CA_BUNDLE_PATH = certs.where()
File "/usr/local/lib/python3.5/dist-packages/certifi/core.py", line 22, in where
f = os.path.split(__file__)[0]
NameError: name '__file__' is not defined
The reason is the "file" variable in cannot be used in frozen program.
https://stackoverflow.com/questions/21937695/python-cx-freeze-name-file-is-not-defined
Could it be better to handle the frozen program case:
try:
approot = os.path.dirname(os.path.abspath(__file__))
except NameError:
import sys
approot = os.path.dirname(os.path.abspath(sys.argv[0]))
return os.path.join(approot, 'cacert.pem')
It's not clear to me why we need to write custom code to handle application freezing. This fundamentally seems like a problem with cxfreeze, especially as I'm not sure the problem exists anywhere else.
This was also an issue if it's run from a whl. This happend to me when I tried to replace the ubuntu supplied requests whl in venv>/share/python-wheel: I had to replace requests and some other files. In the end I needed certifi as well and simply dropping it into the dir as a *.whl didn't work because the path ended up as /home/js/project/.venv/share/python-wheels/certifi-2017.7.27.1-py2.py3-none-any.whl/certifi/cacert.pem
The workaround was to install certifi directly after the venv creation via pip (=installed not in a whl) and then replace the requests, chardet, idna, and urllib3 whl in <venv>/share/python-wheel/.
Wheels are not intended to be run from, wheels are a distribution format that are installed from.
How are you installing certifi such that it tries to be imported from a whl?
I use ubuntu and ubuntu uses a 'un-vendored' pip and requests. All the wheels are in /usr/share/python-wheels. Pip then imports them from there (https://github.com/pypa/pip/blob/master/src/pip/_vendor/__init__.py#L61).
Currently the version of requests there is old and so does not require certifi, but has probelms with retries (see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=870701), so I tried to replace it as mentioned in the bug report. But this needed certifi then and resulted in the bug above when I added the whl to that directory.
I ran into this because I am attempting to use certifi packaged in a standalone zip. For example, packaged with pex or par. I would be happy to submit a test and a patch that changes where() to detect if it is inside a zip, and if so to extract it to a temporary file, which I think would solve this issue.
However, this is admittedly a rare use case, so I would also accept "sorry we won't do this" as an answer.
I'm running into this same issue trying to use the boto3 and requests libraries in an AWS Glue Spark environment, which runs the user's python library code directly from a zip without extracting first. This may apply to other Spark environments too, I'm not positive.
It looks like requests recently merged a change to handle this in their library, but it seems a shame to make each library author implement that code -- my thought was the same as @evanj , perhaps we could use similar code to what requests added directly in certifi and fix it for all users?
I’d accept a patch that detects the zip case and runs the extraction.
__file__ is not mandated to exist. Custom Python module importers are allowed to not define it if modules aren't backed by files. So strictly speaking certifi and every other Python application relying on the existence of __file__ are buggy. More information including how to work around the problem is available at indygreg/PyOxidizer#69.
Done.