MongoDB 5.0+ requires a CPU with AVX support

Question

MongoDB 5.0+ requires a CPU with AVX support

Closed this issue a year ago · 4 comments

Expected Behavior

I can access the program from the web interface and it works

Actual Behavior

This has the same behavior as the one described in #32 .
Accessing the web interface gives me a fully white page.
Mongo complains about the CPU not having AVX support (see logs)

Steps to reproduce the problem

I am using the following compose.yml

version: '3.9'
services:
    dcef:
        image: 'slada/dcef:main'
        ports:
            - '21011:21011'
        container_name: dcef
        volumes:
            - 'dcef_cache:/dcef/cache'
            - './exports:/dcef/exports'
        restart: unless-stopped

volumes:
    dcef_cache:

Pull the docker image with docker pull slada/dcef:main
Create and start the container with sudo docker compose up -d --force-recreate
Try to access the web interface

Specifications

Environment

Operating system: Manjaro 23.0.2 Uranos with x86_64 Linux 5.15.131-1-MANJARO Kernel
CPU Architecture: x86_64 (PC)

DiscordChatExporter-frontend

Release version or commit hash: This release on docker: https://hub.docker.com/layers/slada/dcef/main/images/sha256-ddcbe0738811274413886e6996deed6db85d6056d4c711f5c9cca79d00f51922 which seems to be v2.7.0
Can the issue be reproduced in the demo?: No

DiscordChatExporter

Version (CLI --version): v2.42.0

How did you export the data?

Export format (CLI --format): Json
Downloaded assets (CLI --media): Yes
Markdown processing (CLI --markdown): No
Other CLI options (optional): --dateformat dd/MM/yyyy HH:mm:ss, --reuse-media

Logs

Is there any relevant information in logs? If so, please provide them:

WARNING: MongoDB 5.0+ requires a CPU with AVX support, and your current system does not appear to have that!
  see https://jira.mongodb.org/browse/SERVER-54407
  see also https://www.mongodb.com/community/forums/t/mongodb-5-0-cpu-intel-g4650-compatibility/116610/2
  see also https://github.com/docker-library/mongo/issues/485#issuecomment-891991814

main_mongo loaded
Connecting to database...
[Preprocess] took 30.1 seconds
Traceback (most recent call last):
  File "/dcef/backend/preprocess/main_mongo.py", line 145, in <module>
    main(input_dir, output_dir)
  File "/dcef/backend/preprocess/main_mongo.py", line 100, in main
    wipe_database(database)
  File "/dcef/backend/preprocess/main_mongo.py", line 32, in wipe_database
    whitelisted_guild_ids = config.find_one({"key": "whitelisted_guild_ids"})
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pymongo/collection.py", line 1459, in find_one
    for result in cursor.limit(-1):
  File "/usr/local/lib/python3.11/dist-packages/pymongo/cursor.py", line 1248, in next
    if len(self.__data) or self._refresh():
                           ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pymongo/cursor.py", line 1139, in _refresh
    self.__session = self.__collection.database.client._ensure_session()
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pymongo/mongo_client.py", line 1740, in _ensure_session
    return self.__start_session(True, causal_consistency=False)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pymongo/mongo_client.py", line 1685, in __start_session
    self._topology._check_implicit_session_support()
  File "/usr/local/lib/python3.11/dist-packages/pymongo/topology.py", line 538, in _check_implicit_session_support
    self._check_session_support()
  File "/usr/local/lib/python3.11/dist-packages/pymongo/topology.py", line 554, in _check_session_support
    self._select_servers_loop(
  File "/usr/local/lib/python3.11/dist-packages/pymongo/topology.py", line 238, in _select_servers_loop
    raise ServerSelectionTimeoutError(
pymongo.errors.ServerSelectionTimeoutError: 127.0.0.1:27017: [Errno 111] Connection refused, Timeout: 30s, Topology Description: <TopologyDescription id: 653e9b253a02030314937ff4, topology_type: Unknown, servers: [<ServerDescription ('127.0.0.1', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('127.0.0.1:27017: [Errno 111] Connection refused')>]>
/dcef/run_container.sh: line 6:    13 Illegal instruction     (core dumped) mongod --dbpath "/dcef/cache/db/" --wiredTigerCacheSizeGB 1.5  (wd: /dcef)
############################################################
# Open http://127.0.0.1:21011/ in your browser to view GUI #
############################################################
2023/10/29 17:49:56 [notice] 18#18: using the "epoll" event method
2023/10/29 17:49:56 [notice] 18#18: nginx/1.18.0 (Ubuntu)
2023/10/29 17:49:56 [notice] 18#18: OS: Linux 5.15.131-1-MANJARO
2023/10/29 17:49:56 [notice] 18#18: getrlimit(RLIMIT_NOFILE): 1073741816:1073741816
2023/10/29 17:49:56 [notice] 18#18: start worker processes
2023/10/29 17:49:56 [notice] 18#18: start worker process 19
INFO:     Started server process [17]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:58000 (Press CTRL+C to quit)
INFO:     [CENSORED]:0 - "GET /guilds HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/uvicorn/protocols/http/httptools_impl.py", line 426, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/fastapi/applications.py", line 270, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.11/dist-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.11/dist-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.11/dist-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.11/dist-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/usr/local/lib/python3.11/dist-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.11/dist-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/usr/local/lib/python3.11/dist-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/dist-packages/starlette/routing.py", line 706, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.11/dist-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/dist-packages/starlette/routing.py", line 66, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/fastapi/routing.py", line 237, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/fastapi/routing.py", line 163, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dcef/backend/fastapi/app.py", line 66, in get_guilds
    whitelisted_guild_ids = get_whitelisted_guild_ids()
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dcef/backend/fastapi/helpers.py", line 18, in get_whitelisted_guild_ids
    whitelisted_guild_ids = collection_config.find_one({"key": "whitelisted_guild_ids"})["value"]
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pymongo/collection.py", line 1459, in find_one
    for result in cursor.limit(-1):
  File "/usr/local/lib/python3.11/dist-packages/pymongo/cursor.py", line 1248, in next
    if len(self.__data) or self._refresh():
                           ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pymongo/cursor.py", line 1139, in _refresh
    self.__session = self.__collection.database.client._ensure_session()
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pymongo/mongo_client.py", line 1740, in _ensure_session
    return self.__start_session(True, causal_consistency=False)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pymongo/mongo_client.py", line 1685, in __start_session
    self._topology._check_implicit_session_support()
  File "/usr/local/lib/python3.11/dist-packages/pymongo/topology.py", line 538, in _check_implicit_session_support
    self._check_session_support()
  File "/usr/local/lib/python3.11/dist-packages/pymongo/topology.py", line 554, in _check_session_support
    self._select_servers_loop(
  File "/usr/local/lib/python3.11/dist-packages/pymongo/topology.py", line 238, in _select_servers_loop
    raise ServerSelectionTimeoutError(
pymongo.errors.ServerSelectionTimeoutError: 127.0.0.1:27017: [Errno 111] Connection refused, Timeout: 30s, Topology Description: <TopologyDescription id: 653e9b4525d89af64a3da447, topology_type: Unknown, servers: [<ServerDescription ('127.0.0.1', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('127.0.0.1:27017: [Errno 111] Connection refused')>]>
2023/10/29 17:53:32 [error] 19#19: *3 open() "/dcef/cache/preprocess/fonts/ggsans-normal-400.woff2" failed (2: No such file or directory), client: [CENSORED], server: localhost, request: "GET /data/fonts/ggsans-normal-400.woff2 HTTP/1.1", host: "[CENSORED]:21011", referrer: "http://[CENSORED]:21011/_app/immutable/assets/_layout-92ac35d3.css"
2023/10/29 17:53:32 [error] 19#19: *4 open() "/dcef/cache/preprocess/fonts/ggsans-normal-700.woff2" failed (2: No such file or directory), client: [CENSORED], server: localhost, request: "GET /data/fonts/ggsans-normal-700.woff2 HTTP/1.1", host: "[CENSORED]:21011", referrer: "http://[CENSORED]:21011/_app/immutable/assets/_layout-92ac35d3.css"

Answer 1 · 2023-10-29T18:09:30.000Z

Also, note that the output of grep flags -m1 /proc/cpuinfo | grep avx is empty, as i indeed don't have avx support.

Answer 2 · 2023-10-29T18:39:33.000Z

Hello, thanks for detailed report.

It looks like only Mongo 4.x supports your CPU. I have considered using older mongodb (latest version is 4.1.4 before they changed the license), but the latest version was considerably faster and it offered official precompiled binaries for windows.

The latest mongodb version without AVX is 4.4.6. There are unofficial workarounds to build mongodb without avx, but nothing official.

If you are running the system under a hypervisor, check if really your processor doesn't support this extended instruction set. Because you would need to have CPU older than 8 years to not have AVX, which is unlikely.

Answer 3 · 2023-10-31T20:37:42.000Z

If you are running the system under a hypervisor, check if really your processor doesn't support this extended instruction set. Because you would need to have CPU older than 8 years to not have AVX, which is unlikely.

A lot of server CPUs for low-consumptions servers also don't have AVX, which is my case.

Will the software work with an older Mongo version?

Answer 4 · 2023-11-05T14:21:03.000Z

You can try to change base docker image in Dockerfile, but I won't support this old mongodb version, sorry.

Often you can ask your hosting provider to move your VPS to another server, where AVX is supported.

Closing as wont-fix.