mamba-org/quetz

`repodata.json` file not found

dhirschfeld opened this issue · 12 comments

dhirschfeld commented

I'm trying to proxy conda-forge but it appears that it can't find the repodata.json file:

❯ mamba create -n test-quetz -c http://localhost:8000/channels/conda-forge python=3.10
Looking for: ['python=3.10']

http://localhost:8000/channels/conda-forge/linux-64          22.0 B @  84.0 B/s 404 failed  0.3s
http://localhost:8000/channels/conda-forge/noarch            22.0 B @  84.0 B/s 404 failed  0.3s
Multi-download failed. Reason: Transfer finalized, status: 404 [http://localhost:8000/channels/conda-forge/noarch/repodata.json] 22 bytes

Quetz logs:

ERROR:    [quetz] User not found
DEBUG:    [quetz] connection closed: NullPool
INFO:     172.17.0.1:50314 - "GET /channels/conda-forge/noarch/repodata.json HTTP/1.1" 404 Not Found
DEBUG:    [quetz] connection opened: NullPool
ERROR:    [quetz] User not found
DEBUG:    [quetz] connection closed: NullPool
INFO:     172.17.0.1:50322 - "GET /channels/conda-forge/linux-64/repodata.json HTTP/1.1" 404 Not Found
dhirschfeld commented

In the container the channels/conda-forge folder exists but it empty:

user@632c78e5ae1d:~/quetz$ ls -la ./channels/conda-forge
total 8
drwxr-xr-x 2 user user 4096 Dec 28 01:38 .
drwxr-xr-x 1 user user 4096 Dec 28 01:56 ..
dhirschfeld commented

If I try to mirror another channel I can see the files are all there in the container:

user@632c78e5ae1d:~/quetz$ tree ./channels/energy-quants/
./channels/energy-quants/
├── channeldata.json
├── channeldata.json.bz2
├── channeldata.json.gz
├── index.html
├── index.html.bz2
├── index.html.gz
├── linux-64
│   ├── index.html
│   ├── index.html.bz2
│   ├── index.html.gz
│   ├── quantdev-2022.7.11-hb0f4dca_0.tar.bz2
│   ├── repodata.json
│   ├── repodata.json.bz2
│   └── repodata.json.gz
└── noarch
    ├── index.html
    ├── index.html.bz2
    ├── index.html.gz
    ├── repodata.json
    ├── repodata.json.bz2
    └── repodata.json.gz

2 directories, 19 files

...but I still get the 404 for the repodata.json file:

❯ mamba create -n test-quetz -c http://localhost:8000/channel/energy-quants quantdev

                  __    __    __    __
                 /  \  /  \  /  \  /  \
                /    \/    \/    \/    \
███████████████/  /██/  /██/  /██/  /████████████████████████
              /  / \   / \   / \   / \  \____
             /  /   \_/   \_/   \_/   \    o \__,
            / _/                       \_____/  `
            |/
        ███╗   ███╗ █████╗ ███╗   ███╗██████╗  █████╗
        ████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗
        ██╔████╔██║███████║██╔████╔██║██████╔╝███████║
        ██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║
        ██║ ╚═╝ ██║██║  ██║██║ ╚═╝ ██║██████╔╝██║  ██║
        ╚═╝     ╚═╝╚═╝  ╚═╝╚═╝     ╚═╝╚═════╝ ╚═╝  ╚═╝

        mamba (1.1.0) supported by @QuantStack

        GitHub:  https://github.com/mamba-org/mamba
        Twitter: https://twitter.com/QuantStack

█████████████████████████████████████████████████████████████


Looking for: ['quantdev']

http://localhost:8000/channel/energy-quants/linu..  22.0 B @ 202.0 B/s 404 failed  0.1s
http://localhost:8000/channel/energy-quants/noarch  22.0 B @ 202.0 B/s 404 failed  0.1s
Multi-download failed. Reason: Transfer finalized, status: 404 [http://localhost:8000/channel/energy-quants/noarch/repodata.json] 22 bytes

ERROR:    [quetz] User not found
DEBUG:    [quetz] connection closed: NullPool
INFO:     172.17.0.1:50784 - "GET /channel/energy-quants/linux-64/repodata.json HTTP/1.1" 404 Not Found
DEBUG:    [quetz] connection opened: NullPool
ERROR:    [quetz] User not found
DEBUG:    [quetz] connection closed: NullPool
INFO:     172.17.0.1:50768 - "GET /channel/energy-quants/noarch/repodata.json HTTP/1.1" 404 Not Found
dhirschfeld commented

I tested with the frontend implementation in this repo and am seeing the same issue with not being able to access the repodata.json file

dhirschfeld commented

I just tested with a package I uploaded to a new channel (i.e. not a proxy channel) and got the same error.

dhirschfeld commented

My current working assumption is that the issue (#583 (comment)) with the quetz-client creating a null user might be stuffing up the channel access as well:

image

dhirschfeld commented

In a new instance with no null user:

quetz.sqlite> select * from users
+------------------------------------+-------------------+-------+
| id                                 | username          | role  |
+------------------------------------+-------------------+-------+
| 0xd4e3705a759741919a6ed680cec74593 | dhirschfeld       | owner |
+------------------------------------+-------------------+-------+
1 row in set
Time: 0.009s

I'm still seeing the User not found error before the server returns 404:

ERROR:    [quetz] User not found
ERROR:    [quetz] User not found
INFO:     10.244.2.35:48158 - "GET /channels/seau-dev/noarch/repodata.json HTTP/1.1" 404 Not Found
INFO:     10.244.2.35:48154 - "GET /channels/seau-dev/linux-64/repodata.json HTTP/1.1" 404 Not Found

...so, I guess I'm stumped 😞

dhirschfeld commented

I guess it might be useful if the error message [quetz] User not found had a bit more context to aid debugging. Mentioning which user was not found might be helpful.

dhirschfeld commented

I guess it's one of these two failing:

quetz/quetz/dao.py

Lines 194 to 198 in 6cdf96e

def get_profile(self, user_id):
try:
return self.db.query(Profile).filter(Profile.user_id == user_id).one()
except exc.NoResultFound:
logger.error("User not found")

quetz/quetz/dao.py

Lines 200 to 204 in 6cdf96e

def get_user(self, user_id):
try:
return self.db.query(User).filter(User.id == user_id).one()
except exc.NoResultFound:
logger.error("User not found")

...but I'm in there so I'm not sure where it's getting it's user_id from:

quetz.sqlite> select * from users
+------------------------------------+-------------------+-------+
| id                                 | username          | role  |
+------------------------------------+-------------------+-------+
| 0xd4e3705a759741919a6ed680cec74593 | dhirschfeld       | owner |
+------------------------------------+-------------------+-------+
1 row in set
Time: 0.004s
quetz.sqlite> select * from profiles
+-----------------+-------------------------------------------------------+------------------------------------+
| name            | avatar_url                                            | user_id                            |
+-----------------+-------------------------------------------------------+------------------------------------+
| Dave Hirschfeld | https://avatars.githubusercontent.com/u/107382673?v=4 | 0xd4e3705a759741919a6ed680cec74593 |
+-----------------+-------------------------------------------------------+------------------------------------+
1 row in set
Time: 0.004s
jonas-w commented

After 3 hours of puzzling with the documentation I read the README and because I used dev mode, I needed to use "quetz/get/channel-name" instead of "quetz/channels/channel-name".

dhirschfeld commented

Thanks, I'll definitely give that a go! I've been meaning to pick this back up...

jonas-w commented

Also looking at the commits it seems like the documentation is 2-3 years old, so maybe this changed since then and wasn't documented :/

mbestipa commented

I had the same error as was posted here.
Two things I did before this worked:

  1. I made sure the config had the following set
[local_store]
redirect_enabled = false
  1. I reindexed the channel
curl -X POST "${QUETZ_BASE_URL}/api/channels" \
    -H  "accept: application/json" \
    -H  "Content-Type: application/json" \
    -H  "X-API-Key: ${QUETZ_API_KEY}" \
    -d '{"name":"<channel>",
         "private":false,
         "actions": ["reindex"]}'

I made the following call to install the package
mamba install -c http://localhost:8000/get/<channel> <package>