python-trio/trio

Trio fails under gevent with NotImplementedError: unsupported platform

pg1671 opened this issue · 16 comments

pg1671 commented

Trio is installed as a dependancy of httpx.

If I run the code:

from gevent.monkey import patch_all; patch_all()
import gevent
import httpx

def worker(n):
     print(n)

if __name__ == '__main__':
    jobs = [gevent.spawn(worker, job_no) for job_no in range(10)]
    gevent.joinall(jobs)

I get:

Traceback (most recent call last):
  File "/tests/gevent_test.py", line 3, in
    import httpx
  File "/work/env3.11/lib/python3.11/site-packages/httpx/init.py", line 2, in
    from ._api import delete, get, head, options, patch, post, put, request, stream
  File "/work/env3.11/lib/python3.11/site-packages/httpx/_api.py", line 6, in
    from ._client import Client
  File "/work/env3.11/lib/python3.11/site-packages/httpx/_client.py", line 32, in
    from ._transports.default import AsyncHTTPTransport, HTTPTransport
  File "/work/env3.11/lib/python3.11/site-packages/httpx/_transports/default.py", line 32, in
    import httpcore
  File "/work/env3.11/lib/python3.11/site-packages/httpcore/init.py", line 1, in
    from ._api import request, stream
  File "/work/env3.11/lib/python3.11/site-packages/httpcore/_api.py", line 5, in
    from ._sync.connection_pool import ConnectionPool
  File "/work/env3.11/lib/python3.11/site-packages/httpcore/_sync/init.py", line 1, in
    from .connection import HTTPConnection
  File "/work/env3.11/lib/python3.11/site-packages/httpcore/_sync/connection.py", line 12, in
    from .._synchronization import Lock
  File "/work/env3.11/lib/python3.11/site-packages/httpcore/_synchronization.py", line 11, in
    import trio
  File "/work/env3.11/lib/python3.11/site-packages/trio/init.py", line 23, in
    from ._core import TASK_STATUS_IGNORED as TASK_STATUS_IGNORED # isort: split
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/env3.11/lib/python3.11/site-packages/trio/_core/init.py", line 21, in
    from ._local import RunVar, RunVarToken
  File "/work/env3.11/lib/python3.11/site-packages/trio/_core/_local.py", line 9, in
    from . import _run
  File "/work/env3.11/lib/python3.11/site-packages/trio/_core/_run.py", line 2840, in
    raise NotImplementedError("unsupported platform")
NotImplementedError: unsupported platform
A5rocks commented

Hi, what trio version are you using? (Just to double check, though I haven't tried to reproduce yet)

pg1671 commented

Hi .. it was the latest 0.25.1.

A5rocks commented

While I'm not sure this will fix it, maybe try pip install git+https://github.com/python-trio/trio? #2928 was merged since v0.25.1 was released

pg1671 commented

Hi .. it doesn't appear to fix it in the example code above. Still getting the same error.

pg1671 commented

I tried the same commands as the top of #2928 and got the error as well.

I am using Python 3.11.

CoolCat467 commented

What operating system are you using?
If you can post the results of

python3.11 -m platform

that would be helpful.

A5rocks commented

We should probably include that in the assert message tbh.

I'm also curious about your platform because after pip install trio httpx gevent in a venv on Windows 11, I get this: (notice: no errors)

>>> from gevent.monkey import patch_all; patch_all()
True
>>> import gevent
>>> import httpx
>>>
>>> def worker(n):
...      print(n)
...
>>> if __name__ == '__main__':
...     jobs = [gevent.spawn(worker, job_no) for job_no in range(10)]
...     gevent.joinall(jobs)
...
0
1
2
3
4
5
6
7
8
9
[<Greenlet at 0x16460a8bce0: _run>, <Greenlet at 0x16460acbe20: _run>, <Greenlet at 0x16460acbec0: _run>, <Greenlet at 0x16460acbba0: _run>, <Greenlet at 0x16460acbf60: _run>, <Greenlet at 0x16460adc040: _run>, <Greenlet at 0x16460adc0e0: _run>, <Greenlet at 0x16460adc180: _run>, <Greenlet at 0x16460adc220: _run>, <Greenlet at 0x16460adc2c0: _run>]

... I assume some sort of Linux? Looking at the code, windows uses a set of primitives always, rather than trying to find something that works.

I think your error totally makes sense and we can probably figure something out (detecting gevent monkey patching?), I'm just surprised this hasn't happened before as far as I know. As a short term fix, when I pip install httpx it doesn't look like it pulls in trio, so I don't think that's a required dependency. Maybe your application will work after uninstalling trio to force httpx to use something else?

CoolCat467 commented

Because of the way the logic works, the system this is failing on is not windows, not linux, not epoll-based, and not kqueue-based. Given the what the select docs say, I predict the target system is Solaris or a Solaris derivative, which uses devpoll for polling.

CoolCat467 commented

Might not actually go anywhere because of issues with getting a solaris runner in CI, but created #3014 for attempting to add solaris support

njsmith commented
pg1671 commented

Hi, the platform is ubuntu 22.04.

Here is the output from python3.11 -m platform:

Linux-6.5.0-1018-aws-x86_64-with-glibc2.35

danpe commented

Same issue here on

trio==0.26.2
trio-websocket==0.11.1
Python 3.12.4
CoolCat467 commented

@danpe Could you share the list of all packages installed with

python -m pip freeze

?

danpe commented

@danpe Could you share the list of all packages installed with

python -m pip freeze

?

aiofiles==24.1.0
aiohappyeyeballs==2.3.5
aiohttp @ file:///private/var/folders/nz/j6p8yfhx1mv_0grj5xl4650h0000gp/T/abs_4dbk4qcpk1/croot/aiohttp_1725527761629/work
aiosignal @ file:///home/conda/feedstock_root/build_artifacts/aiosignal_1667935791922/work
alembic==1.13.2
annotated-types==0.7.0
anthropic==0.34.2
anyio==4.4.0
appdirs==1.4.4
asgiref==3.8.1
asttokens==2.4.1
attrs @ file:///home/conda/feedstock_root/build_artifacts/attrs_1722977137225/work
auth0-python==4.7.2
babel==2.16.0
backoff==2.2.1
bcrypt==4.2.0
beautifulsoup4==4.12.3
bidict==0.23.1
blinker==1.8.2
boto3==1.34.158
botocore==1.34.158
build==1.2.1
CacheControl==0.14.0
cachetools==5.4.0
certifi==2024.7.4
cffi==1.17.1
charset-normalizer==3.3.2
chroma-hnswlib==0.7.3
chromadb==0.4.24
cleo==2.1.0
click @ file:///home/conda/feedstock_root/build_artifacts/click_1692311806742/work
cohere==5.8.0
colorama @ file:///home/conda/feedstock_root/build_artifacts/colorama_1666700638685/work
coloredlogs==15.0.1
contourpy==1.3.0
crashtest==0.4.1
crewai==0.70.1
crewai-tools==0.0.1
cryptography==43.0.1
cycler==0.12.1
dataclasses-json==0.6.7
decorator==5.1.1
defusedxml==0.7.1
Deprecated==1.2.14
deprecation==2.1.0
distlib==0.3.8
distro==1.9.0
dnspython==2.7.0
docker==7.1.0
docstring_parser==0.16
docx==0.2.4
docx2txt==0.8
docxcompose==1.4.0
docxtpl==0.18.0
dulwich==0.21.7
embedchain==0.1.122
eventlet==0.37.0
executing==2.1.0
fastapi==0.112.0
fastavro==1.9.5
fastjsonschema @ file:///home/conda/feedstock_root/build_artifacts/python-fastjsonschema_1718477020893/work/dist
filelock==3.15.4
Flask==3.0.3
Flask-Cors==5.0.0
Flask-SocketIO==5.4.1
flatbuffers==24.3.25
fonttools==4.54.1
frozenlist @ file:///Users/builder/cbouss/perseverance-python-buildout/croot/frozenlist_1699254257028/work
fsspec==2024.6.1
gevent==24.10.3
gevent-websocket==0.10.1
google-ai-generativelanguage==0.6.10
google-api-core==2.19.1
google-api-python-client==2.140.0
google-auth==2.33.0
google-auth-httplib2==0.2.0
google-auth-oauthlib==1.2.1
google-cloud-aiplatform==1.61.0
google-cloud-bigquery==3.25.0
google-cloud-core==2.4.1
google-cloud-monitoring==2.22.2
google-cloud-resource-manager==1.12.5
google-cloud-storage==2.18.2
google-crc32c==1.5.0
google-generativeai==0.8.3
google-resumable-media==2.7.2
googleapis-common-protos==1.63.2
gpt4all==2.8.2
gptcache==0.1.44
greenlet==3.1.1
grpc-google-iam-v1==0.13.1
grpcio==1.65.4
grpcio-status==1.62.3
grpcio-tools==1.62.3
h11==0.14.0
h2==4.1.0
hpack==4.0.0
httpcore==1.0.5
httplib2==0.22.0
httptools==0.6.1
httpx==0.27.0
httpx-sse==0.4.0
huggingface-hub==0.24.5
humanfriendly==10.0
hyperframe==6.0.1
idna @ file:///home/conda/feedstock_root/build_artifacts/idna_1726459485162/work
importlib_metadata==8.0.0
importlib_resources @ file:///home/conda/feedstock_root/build_artifacts/importlib_resources_1725921340658/work
imutils==0.5.4
iniconfig==2.0.0
installer==0.7.0
instructor==1.3.3
ipython==8.28.0
itsdangerous==2.2.0
jaraco.classes==3.4.0
jedi==0.19.1
Jinja2==3.1.4
jiter==0.4.2
jmespath==1.0.1
joblib==1.4.2
json5==0.9.25
json_repair==0.25.3
jsonpatch==1.33
jsonpickle==3.3.0
jsonpointer==3.0.0
jsonref==1.1.0
jsonschema @ file:///home/conda/feedstock_root/build_artifacts/jsonschema_1720529478715/work
jsonschema-specifications @ file:///tmp/tmpkv1z7p57/src
jupyter_core @ file:///home/conda/feedstock_root/build_artifacts/jupyter_core_1727163409502/work
keyring==24.3.1
kiwisolver==1.4.7
kubernetes==30.1.0
lancedb==0.5.7
langchain==0.3.5
langchain-anthropic==0.2.3
langchain-cohere==0.1.9
langchain-community==0.3.3
langchain-core==0.3.13
langchain-experimental==0.0.65
langchain-google-genai==2.0.1
langchain-openai==0.2.2
langchain-text-splitters==0.3.1
langdetect==1.0.9
langfuse==2.51.2
langsmith==0.1.129
litellm==1.48.7
llvmlite==0.43.0
lxml==5.3.0
mailjet-rest==1.3.4
Mako==1.3.5
markdown-it-py==3.0.0
MarkupSafe==2.1.5
marshmallow==3.21.3
matplotlib==3.9.2
matplotlib-inline==0.1.7
mdurl==0.1.2
mem0ai==0.1.17
mmh3==4.1.0
monotonic==1.6
more-itertools==10.5.0
mpmath==1.3.0
msgpack==1.1.0
multidict @ file:///private/var/folders/nz/j6p8yfhx1mv_0grj5xl4650h0000gp/T/abs_10voz9m15i/croot/multidict_1701096890858/work
mypy-extensions==1.0.0
nbformat @ file:///home/conda/feedstock_root/build_artifacts/nbformat_1712238998817/work
neo4j==5.25.0
networkx==3.3
nodeenv==1.9.1
numba==0.60.0
numpy==1.26.4
oauthlib==3.2.2
onnxruntime==1.18.1
openai==1.50.2
opencv-contrib-python==4.10.0.84
opencv-python==4.10.0.84
opentelemetry-api==1.26.0
opentelemetry-exporter-otlp-proto-common==1.26.0
opentelemetry-exporter-otlp-proto-grpc==1.26.0
opentelemetry-exporter-otlp-proto-http==1.26.0
opentelemetry-instrumentation==0.47b0
opentelemetry-instrumentation-asgi==0.47b0
opentelemetry-instrumentation-fastapi==0.47b0
opentelemetry-proto==1.26.0
opentelemetry-sdk==1.26.0
opentelemetry-semantic-conventions==0.47b0
opentelemetry-util-http==0.47b0
orjson==3.10.7
outcome==1.3.0.post0
overrides==7.7.0
packaging==24.1
pandas==2.2.2
parameterized==0.9.0
parso==0.8.4
pdf2image==1.17.0
pexpect==4.9.0
pigar @ file:///home/conda/feedstock_root/build_artifacts/pigar_1721171135631/work
pillow==10.4.0
pkginfo==1.11.1
pkgutil_resolve_name @ file:///home/conda/feedstock_root/build_artifacts/pkgutil-resolve-name_1694617248815/work
platformdirs @ file:///home/conda/feedstock_root/build_artifacts/platformdirs_1726613481435/work
pluggy==1.5.0
poetry==1.8.3
poetry-core==1.9.0
poetry-plugin-export==1.8.0
portalocker==2.10.1
posthog==3.5.0
prompt_toolkit==3.0.48
proto-plus==1.24.0
protobuf==4.25.4
ptyprocess==0.7.0
pulsar-client==3.5.0
pure_eval==0.2.3
py==1.11.0
pyarrow==17.0.0
pyasn1==0.6.0
pyasn1_modules==0.4.0
pycparser==2.22
pydantic==2.8.2
pydantic-settings==2.5.2
pydantic_core==2.20.1
Pygments==2.18.0
PyJWT==2.9.0
pylance==0.9.18
PyMuPDF==1.24.12
pyparsing==3.1.2
pypdf==4.3.1
PyPDF2==3.0.1
PyPika==0.48.9
pyproject_hooks==1.1.0
pyright==1.1.375
pysbd==0.3.4
PySocks==1.7.1
pytesseract==0.3.13
pytest==8.3.2
python-dateutil==2.9.0.post0
python-docx==1.1.2
python-dotenv==1.0.1
python-engineio==4.10.1
python-socketio==5.11.4
pytube==15.0.0
pytz==2024.1
pyvis==0.3.2
PyYAML==6.0.2
qdrant-client==1.10.1
rank-bm25==0.2.2
RapidFuzz==3.10.0
ratelimiter==1.2.0.post0
referencing @ file:///home/conda/feedstock_root/build_artifacts/referencing_1714619483868/work
regex==2024.9.11
requests==2.32.3
requests-oauthlib==2.0.0
requests-toolbelt==1.0.0
retry==0.9.2
rich==13.7.1
rpds-py @ file:///Users/builder/cbouss/perseverance-python-buildout/croot/rpds-py_1699262599461/work
rsa==4.9
s3transfer==0.10.2
safetensors==0.4.5
schema==0.7.7
scikit-learn==1.5.2
scipy==1.14.1
selenium==4.23.1
semver==3.0.2
sentence-transformers==3.1.1
setuptools==72.1.0
shapely==2.0.5
shellingham==1.5.4
simple-websocket==1.1.0
six==1.16.0
sniffio==1.3.1
sortedcontainers==2.4.0
soupsieve==2.5
SQLAlchemy==2.0.32
stack-data==0.6.3
starlette==0.37.2
sympy==1.13.2
tabulate==0.9.0
temporalio==1.8.0
tenacity==8.5.0
threadpoolctl==3.5.0
tiktoken==0.7.0
tokenizers==0.20.0
tomlkit==0.13.2
torch==2.4.1
tqdm==4.66.5
traitlets @ file:///home/conda/feedstock_root/build_artifacts/traitlets_1713535121073/work
transformers==4.45.1
trio==0.27.0
trio-websocket==0.11.1
trove-classifiers==2024.9.12
typer==0.12.3
types-protobuf==5.28.0.20240924
types-requests==2.32.0.20240712
typing-inspect==0.9.0
typing_extensions==4.12.2
tzdata==2024.1
uritemplate==4.1.1
urllib3==2.2.2
uvicorn==0.30.5
uvloop==0.19.0
virtualenv==20.26.6
watchfiles==0.23.0
wcwidth==0.2.13
websocket-client==1.8.0
websockets==12.0
Werkzeug==3.0.3
wheel==0.43.0
wrapt==1.16.0
wsproto==1.2.0
xattr==1.1.0
yarl @ file:///private/var/folders/k1/30mswbxs7r1g6zwn8y4fyt500000gp/T/abs_9et2w8mlgu/croot/yarl_1725976501637/work
zipp @ file:///home/conda/feedstock_root/build_artifacts/zipp_1726248574750/work
zope.event==5.0
zope.interface==7.1.1
A5rocks commented

If you update httpcore to be the recently released version 1.0.6, httpx handles this transparently. This is ultimately because trio and gevent monkey patching are incompatible -- we've decided that error should be raised up front rather than let the user run into it later.

Essentially the only thing that we could do is delay that error with monkeypatching, but we have tried PRs in that direction and it's a game of whack-a-mole. Since httpx handles this and that seems to be the main context where this happens, I'm going to be closing this issue. There's #3087 for improving this error message.

Please reopen this issue if httpcore 1.0.6 doesn't actually fix this.

A5rocks commented

Same issue here on

trio==0.26.2
trio-websocket==0.11.1
Python 3.12.4

@danpe

Cause this isn't exactly using httpx, I'm confused on why you're using trio-websocket in an environment where gevent has monkeypatched things away. Is there a reason you're expecting this to work?