SolidFS is a FUSE driver for Solid.
It is very limited in what it can support but is able to interact with Solid Pods.
Some of the limitations are:
- shortcomings in this code
- related to server implementations
- related to the current Solid specification
There is no plan to resolve any one of the specific shortcomings as at 2024-04-11.
There is no warranty with SolidFS and it may break your content on your machine, in your Pod or both. At this time you should only use SolidFS if you can cope with the consequences of that.
Tested using id.inrupt.com auth or no auth.
This may sound obvious, but this only covers the tests you can see.
| Solid Server | Technology | Tested |
|---|---|---|
| PodSpaces | ESS | ✅ |
| inrupt.net | NSS | ✅ |
| solidcommunity.net | NSS | ✅ |
| Solid Lab Playground | CSS | ✅ |
It's currently very hard to run this from source. You may find the Docker approach the easiest if you're unsure of what to do.
In the .env file there are a number of settings.
This is the most important setting.
You can get a Pod from many places.
The value should be something like this:
| Solid Server | Example SOLIDFS_BASE_URL |
|---|---|
| PodSpaces | https://storage.inrupt.com/ecce92aa-5fa3-4333-8883-4917c405d4d5 |
| inrupt.net | https://example.inrupt.net |
| solidcommunity.net | https://example.solidcommunity.net |
| redpencil.io | https://solid.redpencil.io/example |
Interestingly you can also mount a Container deeper within a Pod. This is useful if you only have access to a small part of a Pod. In that case you'd have a value such as this:
https://storage.inrupt.com/ecce92aa-5fa3-4333-8883-4917c405d4d5/some/part/for/me
Most access will be authenticated, but if you don't provide these SolidFS will run without authentication.
SolidFS uses client credentials. If you are using PodSpaces you can generate them in the OpenID Broker Registration page. Of course you should keep them secret because they allow something with those credentials to do anything you can do.
This is the endpoint where the credentials are exchanged for an access token. If you're using PodSpaces then just use https://login.inrupt.com/token
There is a very rudimentary caching mechanism in SolidFS. This turns it on with a 1 or off with anything else, for example 0 or if you don't set it.
This was changed on 2024-04-16 to use HTTP-based caching so you will see content changes. Note that fuselib has some caching so this may stop you seeing the changes immediately.
Directory listings are cached in all cases at the moment.
Experimental: The resulting notifications aren't used for anything yet.
Enable with 1 to have websockets created to monitor changes in the Resources. Any other value will avoid websockets being created.
Experimental: httpx is currently slower and fails some tests. The problems are very likely a result of how it's being used by this code.
Use httpx to enable httpx. Any other value will use requests.
You can specify options to fuselib but in most case you'll want to use the defaults.
Be thoughtful about how this might impact your machine.
First install the dependencies:
python3 -m pip --trusted-host pypi.org install -r requirements.txtNow copy .env.sample to .env and put your settings in there.
Now you can mount your Pod. Be thoughtful about where you mount it. For example if you mount it in your home and you have a backup process covering all of home it might copy your Pod too.
python3 src/solidfs.py -d ~/pod/Note the -d turns on debug for fuselib which is generally useful whilst SolidFS is still hardening.
You can mount the FUSE device inside the container and therefore use Docker to mount your Pod. Although the code works in a rootless podman container, it doesn't appear in the host OS which is a problem in most use-cases but may be OK for your use-case.
cd to the place you want to mount your Pod, or alter the $(pwd) part of the following command:
docker run \
--rm \
--cap-add=SYS_ADMIN \
--device=/dev/fuse \
--name solidfs \
-v $(pwd)/:/data/:rshared \
--env-file ${HOME}/.env \
myforest/solidfs \
-fd /data/In the very likely case that you want to access the Pod mount as a normal user you will want to add the -o allow_other option. Of course this means other people on your machine can see your Pod so be thoughtful about that.
cd to the place you want to mount your Pod, or alter the $(pwd) part of the following command:
docker run \
--rm \
--cap-add=SYS_ADMIN \
--device=/dev/fuse \
--name solidfs \
-v $(pwd)/:/data/:rshared \
--env-file ${HOME}/.env \
myforest/solidfs \
-fd -o allow_other /data/fuselib has hard-coded limits of 128KiB but it's still better than the default of 4KiB pages of data, especially in something like SolidFS where there is an overhead on each call.
The maximum permissable read and write sizes are already set in the code.
If you'd like to build the container locally use a command such as this:
docker build -t myforest/solidfs .In these examples we assume your working directory is the place you mounted your Pod.
Start off gently:
ls -lhYou may see some output like this:
total 0
drwx------. 2 root root 0 Jul 20 2022 bookmarks
drwx------. 2 root root 0 Jul 23 2022 contacts
drwx------. 2 root root 0 Jul 20 2022 inbox
drwx------. 2 root root 0 Jul 20 2022 policies
drwx------. 2 root root 0 Jul 20 2022 private
drwx------. 2 root root 0 Apr 9 18:40 profile
drwx------. 2 root root 0 Jul 20 2022 settings
Being more adventerous:
find test/bob/In my Pod this returns:
/data/test/bob/
/data/test/bob/b2o🐝
/data/test/bob/b2o🐝/🦖
/data/test/bob/b2o🐝/🦖/🦢
/data/test/bob/b2o🐝/🦖/🦢/test.html
/data/test/bob/b2o🐝/🦖/🦢/date2.txt
/data/test/bob/b2o🐝/🦖/🦢/date3.txt
/data/test/bob/b2o🐝/🦖/🦢/date4.txt
/data/test/bob/b2o🐝/🦖/🦢/card.ttl
/data/test/bob/b2o🐝/🦖/🦢/🌳
Here I'm using an existing tool to push some content up to my Pod without writing any new code.
Unfortunately Solid does not offer server-side moves so we suppress the temporary file creation using --inplace which updates the Resources directly:
rsync -av --partial --inplace weather/2023/2023-04 solid/weather/sending incremental file list
2023-04/
2023-04/2023-04-01.txt
2023-04/2023-04-02.txt
2023-04/2023-04-03.txt
...
2023-04/2023-04-29.txt
2023-04/2023-04-30.txt
sent 606,817 bytes received 590 bytes 3,480.84 bytes/sec
total size is 604,760 speedup is 1.00
Cleaning up:
rm -rf solid/weather/Here I'm looking for something in a file:
$ grep name solid/profile/card
vcard:organization-name "MyForest" ;
foaf:name "MyForest 🦆" .Here is a graphical tool using the file system mount:
Using the extended file attribute support of the file system allows us to report additional properties for Resources. Here we are including the content type which can be seen using the extended attribute tool exa which shows the extended attributes as a list under each file:
$ exa -@l
drwx------@ - root 22 Dec 2023 getting-started
└── user.mime_type: "text/turtle"
.rwx------@ 3.2M root 13 Apr 00:24 IMGP2841.JPG
└── user.mime_type: "image/jpeg"
drwx------@ - root 12 Apr 23:04 profile
└── user.mime_type: "text/turtle"
drwx------@ - root 13 Apr 02:44 test
└── user.mime_type: "text/turtle"
.rwx------@ 925 root 13 Apr 02:23 test.jpg
└── user.mime_type: "image/jpeg"
.rwx------@ 1 root 13 Apr 02:24 test.png
└── user.mime_type: "image/png"
Another example, this time when multiple Pods are mounted and we want to investigate the server type:
pods/$ exa --tree --level 2 --only-dirs --list-dirs --extended | grep -v ".link." | grep -iv "http" | grep -v ".allow." | grep -v "security.selinux" | grep -v "user.mime_type" | grep -iB2 "x-powered-by"
.
├── myforest.inrupt.net
│ ├── user.header.x-powered-by: "solid-server/5.7.10"
--
│ └── settings
├── myforest.solidcommunity.net
│ ├── user.header.x-powered-by: "solid-server/5.7.10"
--
├── solidweb.me
│ └── myforest
│ ├── user.header.x-powered-by: "Community Solid Server"
There is not much guidance on this yet.
- Only bind structlog context to values that are safe, not things like byte arrays.
- Don't include security-sensitive information in logs, such as access tokens or headers that contain them.
With a Pod mounted at /data/ run:
pytest -n auto test/*.py========================================== test session starts ==========================================
platform linux -- Python 3.12.2, pytest-8.1.1, pluggy-1.4.0
rootdir: /workspaces/SolidFS
configfile: pytest.ini
plugins: xdist-3.5.0
16 workers [17 items]
................. [100%]
========================================== 17 passed in 12.37s ==========================================
Jesse Wright has had a solidFS extension for VSCode for a while. We agreed there won't be confusion because SolidFS will never be a VSCode extension. Despite the similar names David Bowen opted for "SolidFS" to align with other FUSE driver naming.
There are a number of Solid apps, for example Jeff Zucker's solid-file-client can be used to do many of the things SolidFS would do, but solid-file-client has Solid-focused parts such as specifying the content type for files which can't be done using normal file interactions via FUSE.
SolidLab have a bashlib tool which uses TypeScript to implement the file management functions. By using custom commands it provides more options on each command than you get with SolidFS.