running python.wasm running a test.py script that uses requests package gets stuck in on import requests.

Question

running python.wasm running a test.py script that uses requests package gets stuck in on import requests.

Opened this issue 2 months ago · 9 comments

I am testing the scenario from https://wasmlabs.dev/articles/python-wasm32-wasi/

works with wasmtime and wasmedge. They are able to find requests but fails because ssl is not available which is the correct error.

wasmer run ./bin/python-3.12.0.wasm --mapdir lib:usr/local/lib --mapdir /app:$(PWD) -- /app/test.py

just gets stuck on import requests

I am able to run the repl

wasmer run bin/python-3.12.0.wasm --mapdir lib:usr/local/lib

the repl loads

I run import requests and no output for 20+ min.

I ran repl with verbose and got

import '_socket' # <class '_frozen_importlib.BuiltinImporter'>

trying /selectors.py

trying /selectors.pyc

trying /lib/python312.zip/selectors/init.pyc

trying /lib/python312.zip/selectors/init.py

trying /lib/python312.zip/selectors.pyc

zipimport: zlib available

trying /lib/python312.zip/selectors/init.pyc

trying /lib/python312.zip/selectors/init.py

trying /lib/python312.zip/selectors.pyc

zipimport: zlib available

import 'math' # <class '_frozen_importlib.BuiltinImporter'>
import 'select' # <class '_frozen_importlib.BuiltinImporter'>

This also doesn't work with wasmer run python/python and specifying PYTHONPATH to requests.

wasmer -vV; rustc -vV

wasmer 4.2.7 (??????? 2024-03-19)
binary: wasmer-cli
commit-hash:
commit-date: 2024-03-19
host: x86_64-unknown-linux-gnu
compiler: cranelift
rustc 1.77.1 (7cf61ebde 2024-03-27)
binary: rustc
commit-hash: 7cf61ebde7b22796c69757901dd346d0fe70bd97
commit-date: 2024-03-27
host: x86_64-unknown-linux-gnu
release: 1.77.1
LLVM version: 17.0.6

Answer 1 · 2024-03-29T18:05:13.000Z

RUN-137 running python.wasm running a test.py script that uses requests package gets stuck in on import requests.

Answer 2 · 2024-04-02T08:38:52.000Z

Thanks for opening the issue. We'll investigate. Meanwhile, because socket support in Wasi Preview 1 (the target that Python is compiled to in the link you posted) is very unstable, I don't think it will fully work with them even if SSL is available.

We will debug to see where the halt is coming from, as that's likely an issue in Wasmer.
But meanwhile, can you try to run your program (that uses requests) in Wasmer with the Python package from the Wasmer registry: https://wasmer.io/python/python and let us know if that doesn't work for you?

$ wasmer run python/python

It may be useful to see how to run packages with dependencies based on this example: https://github.com/wasmerio/python-flask-example

Answer 3 · 2024-04-02T12:52:53.000Z

yes I previously ran the one from the register and the same thing occurred. It was stuck in the same place.

Answer 4 · 2024-04-03T15:03:17.000Z

@relcodedev you need to specify wasmer run --net to enable networking.

I don't see that in your posted CLI command.

Answer 5 · 2024-04-03T15:06:55.000Z

I didn't add --net because I wanted to make sure it failed. I want to test that the script did not have any network access. The other runtimes fail and give appropriate message and not get stuck.

Answer 6 · 2024-04-03T15:22:55.000Z

Yeah that may well be a bug.

Until we have time to investigate, you could run with RUST_LOG=wasmer_wasix=trace wasmer run ... to see a log of the syscalls, which would likely tell us where things get stuck.

Answer 7 · 2024-04-03T15:56:10.000Z

Ok, I retested with with wasmer run python/python and it works but not with my python.wasm file. Sorry, I thought I tested properly last time with python/python. I get the proper Not Supported message.

Using my version of the python.wasm, this is the output below.

my script has import sys and import socket. I remove import socket it works.

with import socket, these are the last ~34 lines

zipimport: zlib available

2024-04-03T15:35:31.995069Z TRACE ThreadId(01) fd_write: wasmer_wasix::syscalls::wasi::fd_write: return=Ok(Errno::success) fd=2 nwritten=28
2024-04-03T15:35:31.995104Z TRACE ThreadId(01) fd_write: wasmer_wasix::syscalls::wasi::fd_write: close time.busy=29.5ms time.idle=1.00µs fd=2 nwritten=28
2024-04-03T15:35:31.995402Z TRACE ThreadId(01) fd_fdstat_get: wasmer_wasix::syscalls::wasi::fd_fdstat_get: return=Errno::success fd=5
2024-04-03T15:35:31.995421Z TRACE ThreadId(01) fd_fdstat_get: wasmer_wasix::syscalls::wasi::fd_fdstat_get: close time.busy=21.5µs time.idle=1.60µs fd=5
2024-04-03T15:35:31.995499Z DEBUG ThreadId(01) path_open: wasmer_wasix::syscalls::wasi::path_open: return=Ok(Errno::success) dirfd=5 follow_symlinks=true path="lib/python312.zip" ret_fd=35
2024-04-03T15:35:31.995511Z DEBUG ThreadId(01) path_open: wasmer_wasix::syscalls::wasi::path_open: close time.busy=56.9µs time.idle=800ns dirfd=5 follow_symlinks=true path="lib/python312.zip" ret_fd=35
2024-04-03T15:35:31.995529Z DEBUG ThreadId(01) fd_filestat_get: wasmer_wasix::syscalls::wasi::fd_filestat_get: return=Errno::success fd=35
2024-04-03T15:35:31.995539Z DEBUG ThreadId(01) fd_filestat_get: wasmer_wasix::syscalls::wasi::fd_filestat_get: close time.busy=10.9µs time.idle=600ns fd=35
2024-04-03T15:35:31.995557Z TRACE ThreadId(01) fd_fdstat_get: wasmer_wasix::syscalls::wasi::fd_fdstat_get: return=Errno::success fd=35
2024-04-03T15:35:31.995566Z TRACE ThreadId(01) fd_fdstat_get: wasmer_wasix::syscalls::wasi::fd_fdstat_get: close time.busy=9.60µs time.idle=500ns fd=35
2024-04-03T15:35:31.995594Z TRACE ThreadId(01) fd_seek: wasmer_wasix::syscalls::wasi::fd_seek: new_offset=0 fd=35 offset=0 whence=Whence::Cur
2024-04-03T15:35:31.995604Z TRACE ThreadId(01) fd_seek: wasmer_wasix::syscalls::wasi::fd_seek: return=Ok(Errno::success) fd=35 offset=0 whence=Whence::Cur
2024-04-03T15:35:31.995613Z TRACE ThreadId(01) fd_seek: wasmer_wasix::syscalls::wasi::fd_seek: close time.busy=22.1µs time.idle=900ns fd=35 offset=0 whence=Whence::Cur
2024-04-03T15:35:31.995639Z TRACE ThreadId(01) fd_seek: wasmer_wasix::syscalls::wasi::fd_seek: new_offset=611291 fd=35 offset=611291 whence=Whence::Set
2024-04-03T15:35:31.995649Z TRACE ThreadId(01) fd_seek: wasmer_wasix::syscalls::wasi::fd_seek: return=Ok(Errno::success) fd=35 offset=611291 whence=Whence::Set
2024-04-03T15:35:31.995659Z TRACE ThreadId(01) fd_seek: wasmer_wasix::syscalls::wasi::fd_seek: close time.busy=19.6µs time.idle=600ns fd=35 offset=611291 whence=Whence::Set
2024-04-03T15:35:31.995739Z TRACE ThreadId(01) fd_read: wasmer_wasix::syscalls::wasi::fd_read: return=Ok(Errno::success) fd=35 nread=8192
2024-04-03T15:35:31.995751Z TRACE ThreadId(01) fd_read: wasmer_wasix::syscalls::wasi::fd_read: close time.busy=71.1µs time.idle=700ns fd=35 nread=8192
2024-04-03T15:35:31.995799Z TRACE ThreadId(01) fd_close: wasmer_wasix::fs: weakening file descriptor fd=35 inode=9 ref_cnt=2 pid=1 fd=35
2024-04-03T15:35:31.995812Z DEBUG ThreadId(01) fd_close: wasmer_wasix::syscalls::wasi::fd_close: return=Ok(Errno::success) pid=1 fd=35
2024-04-03T15:35:31.995822Z DEBUG ThreadId(01) fd_close: wasmer_wasix::syscalls::wasi::fd_close: close time.busy=24.0µs time.idle=800ns pid=1 fd=35

zipimport: zlib available

2024-04-03T15:35:31.996029Z TRACE ThreadId(01) fd_write: wasmer_wasix::syscalls::wasi::fd_write: return=Ok(Errno::success) fd=2 nwritten=28
2024-04-03T15:35:31.996043Z TRACE ThreadId(01) fd_write: wasmer_wasix::syscalls::wasi::fd_write: close time.busy=169µs time.idle=1.50µs fd=2 nwritten=28
import 'collections.abc' # <zipimporter object "/lib/python312.zip/collections/">
2024-04-03T15:35:31.996253Z TRACE ThreadId(01) fd_write: wasmer_wasix::syscalls::wasi::fd_write: return=Ok(Errno::success) fd=2 nwritten=82
2024-04-03T15:35:31.996266Z TRACE ThreadId(01) fd_write: wasmer_wasix::syscalls::wasi::fd_write: close time.busy=59.4µs time.idle=1.10µs fd=2 nwritten=82
import 'math' # <class '_frozen_importlib.BuiltinImporter'>
2024-04-03T15:35:31.997213Z TRACE ThreadId(01) fd_write: wasmer_wasix::syscalls::wasi::fd_write: return=Ok(Errno::success) fd=2 nwritten=60
2024-04-03T15:35:31.997228Z TRACE ThreadId(01) fd_write: wasmer_wasix::syscalls::wasi::fd_write: close time.busy=78.0µs time.idle=1.10µs fd=2 nwritten=60
import 'select' # <class '_frozen_importlib.BuiltinImporter'>
2024-04-03T15:35:31.997536Z TRACE ThreadId(01) fd_write: wasmer_wasix::syscalls::wasi::fd_write: return=Ok(Errno::success) fd=2 nwritten=62
2024-04-03T15:35:31.997550Z TRACE ThreadId(01) fd_write: wasmer_wasix::syscalls::wasi::fd_write: close time.busy=56.9µs time.idle=1.20µs fd=2 nwritten=62

Answer 8 · 2024-04-03T16:00:01.000Z

As long as things from the Wasmer registry work, we should be good to go @relcodedev .

The Python package in the registry uses WASIX instead of WASI, and as such requests/asyncio/sockets/... are fully supported.
If you have compiled your python.wasm manually without using WASIX, we can't guarantee the work (since there might be an issue on the way you compiled it).

Is there any reason for using that Python Wasm file instead of the official in the Wasmer repository (python/python)?

Answer 9 · 2024-04-03T16:16:29.000Z

We need to run in a locked down environment. Is there a way to pull down the python/python version to keep locally? Does wasmer store the file locally on run?