Can not setup workspace in mounted directory
Closed this issue · 31 comments
Describe the bug
Can not setup workspace in mounted directory
Steps To Reproduce
exegol start mainhack full -w "/home/user/myhacksetup"
Exegol Wrapper Version
[*] Exegol is currently in version v4.3.1
[*] Exegol Discord serv.: https://discord.gg/cXThyp7D6P
[*] Exegol documentation: https://exegol.rtfd.io/
[+] We thank Capgemini for supporting the project (helping with dev) 🙏
[+] We thank HackTheBox for sponsoring the multi-arch support 💚
[D] Pip installation: On ✔
[D] Git source installation: Off 🪓
[D] Host OS: Linux (Kernel)
[D] Arch: amd64
[D] Raw arch: x86_64
[D] Docker desktop: Off 🪓
[D] Shell type: Linux
[D] Last wrapper update check: 01/02/2024
Host OS
Linux
Configuration of the concerned container
No response
Execution logs in debug mode
...
Exception
No response
Anything else?
No response
Thank you for raising the issue
Please provide debug logs, stack trace that shows what failing so that we can look for the bug in the code if any
[*] Exegol is currently in version v4.3.1
[*] Exegol Discord serv.: https://discord.gg/cXThyp7D6P
[*] Exegol documentation: https://exegol.rtfd.io/
[+] We thank Capgemini for supporting the project (helping with dev) 🙏
[+] We thank HackTheBox for sponsoring the multi-arch support 💚
[D] Pip installation: On ✔
[D] Git source installation: Off 🪓
[D] Host OS: Linux (Kernel)
[D] Arch: amd64
[D] Raw arch: x86_64
[D] Docker desktop: Off 🪓
[D] Shell type: Linux
[D] Last wrapper update check: 01/02/2024
[*] Starting exegol
[*] Arguments supplied with the command, skipping interactive mode
[D] Attribute not found in parameters: multicontainertag
[V] Configuring new exegol container
[D] Attribute not found in parameters: multiimagetag
[D] └── full → (remote) sha256:a87696f3b27523be0dc5b915d7efcd6ef09bbd8f31f0ab61e8048b1f17c659e0
[D] Auto-load remote version for the specific image 'full'
[V] Config: Enabling display sharing
[V] Config: Enabling host timezones
[V] Volume was successfully added for /etc/timezone
[V] Volume was successfully added for /etc/localtime
[V] Config: Enabling my-resources volume
[V] Updating the permissions of /home/alex/.exegol/my-resources (and sub-folders) to allow file sharing between the container and the host user
[D] Adding setgid permission recursively on directories from /home/alex/.exegol/my-resources
[D] Loading git at /home/alex/.exegol/exegol-resources
[D] Repo path: /home/alex/.exegol/exegol-resources/.git
[D] Git repository successfully loaded
[V] Config: Enabling exegol resources volume
[V] Config: Sharing workspace directory /home/alex/test
⭐ Container summary
┌──────────────────┬───────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Name │ demo │
│ Image │ full - v.3.1.2 (Up to date) (amd64) │
├──────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Credentials │ root : svGGLUTpNRCMr6ENBrLSH6QJDlTNAI │
│ Desktop │ Off 🪓 │
│ X11 │ On ✔ │
│ Network │ host │
│ Timezone │ On ✔ │
│ Exegol resources │ On ✔ (/opt/resources) │
│ My resources │ On ✔ (/opt/my-resources) │
│ Shell logging │ Off 🪓 │
│ Privileged │ Off ✔ │
│ Workspace │ /home/alex/test (/workspace) │
│ Envs │ DISPLAY=:0 │
│ │ _JAVA_AWT_WM_NONREPARENTING=1 │
│ │ QT_X11_NO_MITSHM=1 │
│ Volumes │ (RO) /home/alex/.local/lib/python3.10/site-packages/exegol/utils/imgsync/spawn.sh ➡ /.exegol/spawn.sh │
│ │ (RW) /tmp/.X11-unix ➡ /tmp/.X11-unix │
│ │ (RO) /etc/timezone ➡ /etc/timezone │
│ │ (RO) /etc/localtime ➡ /etc/localtime │
│ │ (RW) /home/alex/.exegol/my-resources ➡ /opt/my-resources │
│ │ (RW) /home/alex/.exegol/exegol-resources ➡ /opt/resources │
└──────────────────┴───────────────────────────────────────────────────────────────────────────────────────────────────────┘
[*] Creating new exegol container
[!] The file sharing permissions between the container and the host will not be applied automatically by Exegol. (Use the --update-fs option to enable the feature)
[D] demo - full
Privileged: False
Capabilities: []
Sysctls: {}
X: True
TTY: True
Network host: host
Ports: {}
Share timezone: True
Common resources: True
Envs (3): {'DISPLAY': ':0', '_JAVA_AWT_WM_NONREPARENTING': '1', 'QT_X11_NO_MITSHM': '1'}
Labels (0): {}
Shares (7): [{'Target': '/.exegol/spawn.sh', 'Source': '/home/alex/.local/lib/python3.10/site-packages/exegol/utils/imgsync/spawn.sh', 'Type': 'bind', 'ReadOnly': True}, {'Target': '/tmp/.X11-unix',
'Source': '/tmp/.X11-unix', 'Type': 'bind', 'ReadOnly': False}, {'Target': '/etc/timezone', 'Source': '/etc/timezone', 'Type': 'bind', 'ReadOnly': True}, {'Target': '/etc/localtime', 'Source':
'/etc/localtime', 'Type': 'bind', 'ReadOnly': True}, {'Target': '/opt/my-resources', 'Source': '/home/alex/.exegol/my-resources', 'Type': 'bind', 'ReadOnly': False}, {'Target': '/opt/resources', 'Source':
'/home/alex/.exegol/exegol-resources', 'Type': 'bind', 'ReadOnly': False}, {'Target': '/workspace', 'Source': '/home/alex/test', 'Type': 'bind', 'ReadOnly': False}]
Devices (0): []
VPN: N/A
[D] Entrypoint: ['/bin/bash', '/.exegol/entrypoint.sh']
[D] Cmd: ['load_setups', 'endless']
[-] invalid mount config for type "bind": stat /home/alex/test: permission denied
[D] 400 Client Error for http+docker://localhost/v1.43/containers/create?name=exegol-demo: Bad Request ("invalid mount config for type "bind": stat /home/alex/test: permission denied")
[!] Error while creating exegol container. Exiting.
can you run whoami && ls -al ~/test?
alex total 8 drwxrwxr-x 2 alex alex 4096 feb 2 01:42 . drwxr-x---+ 69 alex alex 4096 feb 2 01:44 ..
Even giving the rights with chmod 755 test or chown -R alex:alex /home/alex/test changes nothing. I believe that the problem is probably linked to the docker permission which certainly does not have access to the mount folder, do you know how to resolve this problem?
weird, @Dramelac @QU35T-code any idea on this? I don't have much time to think on it rn
Hello @Lucstay11 are you using rootless docker ? If so, it's not fully supported by exegol there is a lot a limitation (as you can see), try to use 'standard' docker and follow the exegol doc (either install exegol as root to run it with sudo OR add yourself to the docker group to use a user-installed exegol).
Hello @Lucstay11 are you using rootless docker ? If so, it's not fully supported by exegol there is a lot a limitation (as you can see), try to use 'standard' docker and follow the exegol doc (either install exegol as root to run it with sudo OR add yourself to the docker group to use a user-installed exegol).
Do we have an easy way of knowing if docker's install is rootless? Would be nice to catch it in the wrapper imo
Bonjour @Lucstay11utilisez-vous rootless docker ? Si c'est le cas, il n'est pas entièrement pris en charge par exegol il y a beaucoup de limitations (comme vous pouvez le voir), essayez d'utiliser 'standard' docker et suivez le doc d'exégol (soit installer exégol en tant que root pour l'exécuter avec sudo OU ajouter vous-même au groupe docker pour utiliser un exégol installé par l'utilisateur).
To be honest I have never used docker so I would not answer you nevertheless especially since it is a docker modified in a wrapper to make exegol work, I installed exegol as mentioned in the doc and that's it. The reason for the problem is that exegol cannot access a mount workspace!
The problem you have is not an Exegol error.
This error is from the docker daemon when exegol tried to start your container:
[D] 400 Client Error for http+docker://localhost/v1.43/containers/create?name=exegol-demo: Bad Request ("invalid mount config for type "bind": stat /home/alex/test: permission denied")
What do you mean by it is a docker modified ?
Maybe try to reinstall docker on your host, you can follow the install doc of docker on the exegol documentation or directly from the official docker website.
I installed docker as mentioned above and gave rights to the docker group but I got the same error:
exegol start cryptsetup -w /home/alex/exegolspace
[] Exegol is currently in version v4.3.1
[] Exegol Discord serv.: https://discord.gg/cXThyp7D6P
[] Exegol documentation: https://exegol.rtfd.io/
[+] We thank Capgemini for supporting the project (helping with dev) 🙏
[+] We thank HackTheBox for sponsoring the multi-arch support 💚
[] Starting exegol
[*] Arguments supplied with the command, skipping interactive mode
🛸 Available images
┌───────────┬─────────┬──────────────────────┐
│ Image tag │ Size │ Status │
├───────────┼─────────┼──────────────────────┤
│ full │ 50.7GB │ Up to date (v.3.1.2) │
│ web │ ~23.5GB │ Not installed │
│ osint │ ~13.3GB │ Not installed │
│ light │ ~14.2GB │ Not installed │
│ ad │ ~40.4GB │ Not installed │
│ nightly │ ~55.2GB │ Not installed │
└───────────┴─────────┴──────────────────────┘
[*] You can use a name that does not already exist to build a new image from local sources
[?] Select an image by its name (full): full
⭐ Container summary
┌──────────────────┬───────────────────────────────────────┐
│ Name │ cryptsetup │
│ Image │ full - v.3.1.2 (Up to date) │
├──────────────────┼───────────────────────────────────────┤
│ Credentials │ root : G5NHik0TfMe7Jt2KGJxGAaynejpVUc │
│ Desktop │ Off 🪓 │
│ X11 │ On ✔ │
│ Network │ host │
│ Timezone │ On ✔ │
│ Exegol resources │ On ✔ (/opt/resources) │
│ My resources │ On ✔ (/opt/my-resources) │
│ Shell logging │ Off 🪓 │
│ Privileged │ Off ✔ │
│ Workspace │ /home/alex/exegolspace (/workspace) │
└──────────────────┴───────────────────────────────────────┘
[*] Creating new exegol container
[!] The file sharing permissions between the container and the host will not be applied automatically by Exegol. (Use the --update-fs option to enable the feature)
[-] invalid mount config for type "bind": stat /home/alex/exegolspace: permission denied
[!] Error while creating exegol container. Exiting.`
Can you try to create a simple docker container ? Maybe we will have more information on the docker error:
docker run --rm -it --mount type=bind,source=/home/alex/exegolspace,destination=/workspace debian /bin/bash -c 'ls -lha /workspace'
If you still have the error, try with sudo to see if you have the same outcome.
Can you try to create a simple docker container ? Maybe we will have more information on the docker error:
docker run --rm -it --mount type=bind,source=/home/alex/exegolspace,destination=/workspace debian /bin/bash -c 'ls -lha /workspace'If you still have the error, try with sudo to see if you have the same outcome.
docker run --rm -it --mount type=bind,source=/home/alex/exegolspace,destination=/workspace debian /bin/bash -c 'ls -lha /workspace' Unable to find image 'debian:latest' locally latest: Pulling from library/debian 7bb465c29149: Pull complete Digest: sha256:4482958b4461ff7d9fabc24b3a9ab1e9a2c85ece07b2db1840c7cbc01d053e90 Status: Downloaded newer image for debian:latest total 8.0K drwxrwxr-x 2 1000 1000 4.0K Feb 7 14:03 . drwxr-xr-x 1 root root 4.0K Feb 27 16:50 ..
I will explain in detail what I wanted to do. I would like to create a gocrypt encrypter container which would serve as a mount folder for the exegol workspace. But when I want to create a workspace in a folder mounted docker prevents me...
Hello
I see. It's in the exegol roadmap to have encrypted workspace.
What was your mount setup when you tried the docker run command ?
I did not mount the folder during the previous test... When I mounted it even if I execute the command with sudo docker does not have permission to access the mount folder. The problem really comes from docker, how to give it the rights??
sudo docker run --rm -it --mount type=bind,source=/home/alex/exegolspace,destination=/workspace debian /bin/bash -c 'ls -lha /workspace'
docker: Error response from daemon: invalid mount config for type "bind": stat /home/alex/exegolspace: permission denied.
See 'docker run --help'.
Indeed it is a docker error, it should work if your mount directory is accessible on your host ...
I suggest you can create an issue on the docker repository (maybe this one https://github.com/docker/cli) and detail how to reproduce your error by creating the same mount setup.
Indeed it is a docker error, it should work if your mount directory is accessible on your host ... I suggest you can create an issue on the docker repository (maybe this one https://github.com/docker/cli) and detail how to reproduce your error by creating the same mount setup.
Yes I would see but can you also try to create an exegol container in a mounter folder and show me what you get back?
Indeed it is a docker error, it should work if your mount directory is accessible on your host ... I suggest you can create an issue on the docker repository (maybe this one https://github.com/docker/cli) and detail how to reproduce your error by creating the same mount setup.
ok I understood the error because my folder is already mounted... Do you know how to do so that docker can mount the exegol workspace in a folder that is already mounted?
To supplied a custom workspace directory to exegol, you can use -w /path/to/dir parameter. Docker should be able to use any folder event if it's mounted from a usb drive for exemple.
In your case, i don't know how you have mounted your folder, if you can describe how to setup this environment to reproduce maybe i can help.
But your error is a docker limitation for the moment and Exegol doesn't support encrypted workspace for the moment, it's in our roadmap.
@Lucstay11 any update ?
@Dramelac
I encrypted my worskspace (.exegol/workspaces/myhackworkspace) with gocryptfs, you can try to reproduce my situation:
gocryptfs -init crypt_workspace myhackworkspace
but when I mount my myhackworkspace folder
gocryptfs crypt_workspace myhackworkspace
and I launch my exegol session on this mount folder, exegol cannot access it.
I'm sure it's a simple permissions problem but I can't seem to solve it...
Try it on your side!
@Lucstay11 did you try with --cap SYS_ADMIN ? Or directly with --privileged ?
@Lucstay11 did you try with
--cap SYS_ADMIN? Or directly with--privileged?
where should I place these parameters?
@Lucstay11 did you try with
--cap SYS_ADMIN? Or directly with--privileged?where should I place these parameters?
Something like
# exegol start [OPTIONS] <container> <image>
exegol start --cap SYS_ADMIN somecontainer
exegol start --privileged someothercontainerNot work is the same issue...
Can you try your hand and find a solution?
@Dramelac @ShutdownRepo Can you try it on your side and find any solution please?
Did you create a new container with --privileged option and you still have the same problem ?
Because from what i saw in your previous message (using directly docker run commands) if the privileged mode doesn't work it's a docker issue and you should open an issue in their repo
Regarding gocryptfs, i'm not familiar with. I only used cryptsetup and luks in the past for encrypted volume.
Official support of encrypted volume are in the exegol roadmap but not yet here unfortunately. We can support you in best-effort but until their is an official feature i cannot guarantee you that docker itself support what you are trying to do :/
I reproduce locally and indeed docker cannot mount volume of type fuse.gocryptfs even in privileged mode..
It's probably a security restriction from fuse.gocryptfs (or docker do not support this mount type).
Feel free to create an issue in the project https://github.com/rfjakob/gocryptfs asking why / if it's intended that gocryptfs cannot be mounted inside a docker container. Maybe they can add support for it or give more explanation on retrictions.
Otherwise, in the current situation, if you wan to use gocryptfs with exegol you can, but you have to mount it from exegol, not from the host. The following setup works but give extra-permission to the exegol container (so be very carefull with that, these are dangerous permissions):
- Create a privileged exegol:
exegol start newcontainer full -V ./crypt_workspace:/crypt --privileged - Setup inside exegol:
apt update && apt install -y gocryptfs && \
gocryptfs /crypt /workspace && \
cd /workspace
If you want, you can automate the installation of gocryptfs with my-resources.
I'm now closing this issue, if you found more information on restriction and how we could integrate this further, don't hesitate to re-open it.
Hello @Lucstay11
I have found a solution, it was as expected a restriction from gocryptfs.
- First you need to edit this file on your host
/etc/fuse.confand uncomment the lineuser_allow_other. - Then, you can mount on your host your private volume with the otpion
-allow_otherlike
gocryptfs -allow_other ./crypt_workspace/ ./workspace/ - And then create an exegol container with a custom workspace:
exegol start newcontainer full -w ./workspace
@Dramelac he work fine at the begining,but when at reboot my machine and i decrypt the workspace and I start the exegol container i have this ERROR
[] Exegol is currently in version v4.3.2
[] Exegol Discord serv.: https://discord.gg/cXThyp7D6P
[] Exegol documentation: https://exegol.rtfd.io/
[] Starting exegol
[] Arguments supplied with the command, skipping interactive mode
[] Location of the exegol workspace on the host :
/home/alex/exegolspace/testhack
[+] Opening shell in Exegol 'newcontainer'
OCI runtime exec failed: exec failed: unable to start container process: chdir to cwd ("/workspace") set in config.json failed: transport endpoint is not connected: unknown
I don't have this error not matter what i try to do wrong. Even after reboot.
Dont forget the -allow_other parameter when you mount your workspace before starting your exegol container after rebooting. And always use the same mounting directory.