/solidos

The operating system for Solid

Primary LanguageShellMIT LicenseMIT

🤗 Welcome to the repo of SolidOS

[Solid logo]

MIT license SolidOS issues Matrix

If you made it here you must already have heard about Solid. This space is home to the SolidOS code. Keep reading if you want to know:

For experimenting with SolidOS implementations, you can:

If you are looking for something else, let us try and guide you:

Further links:

🤔 What is SolidOS?

🌟🌟🌟 SolidOS is an Operating System for Solid. 🌟🌟🌟

Solid is developing into a booming ecosystem which involves specifications 📃, tech stack 🛠, servers 💻, and apps 🕹. We, the SolidOS team, believe that this ecosystem also needs an Operating System.

When you get a new phone, PC, or tablet, they usually come with an operating system that provides some basic functionality to get started and be productive. More importantly, you can personalize your OS to your needs, by installing apps, managing content, and much more.

Solid is not shipped with a piece of hardware (who knows, maybe in the future it will...). For now, you get into the ecosystem once you create a WebID and provision your own personal data store (often called a "Pod") (see getting started with Solid - get a pod). Immediately after getting your new Solid WebID and Pod space, SolidOS is helping you to navigate the Solid ecosystem.

SolidOS is much more. SolidOS showcases the possibility of Solid for the future, by which we mean —

  • true data ownership — management of personal data & authorization control
  • avoidance of vendor lock-in to services — easy moving to a different Pod or WebID provider
  • data reuse between applications — with help of data interoperability and data discoverability

Watch a SolidOS explanation video as part of the Solid World event series.

What you can do today with SolidOS

Take a look at an example: SolidOS project Pod. SolidOS implemented features:

SolidOS vision, mission and roadmap

Read more about the current SolidOS 🌟 vision, goals 🎯, and roadmap 🚗 on the SolidOS project Pod.

Note: SolidOS is also known by names like Data Browser (default) or Databrowser, and at times as mashlib. Which name is used depends on which flavour of SolidOS you are referring to:

👩🏽‍💻 SolidOS technical intro

The SolidOS stack contains —

It also makes use of —

Let's take a look at an architecture diagram of SolidOS: SolidOS architectural overview

A colorful dependency tree can be seen here: SolidOS dependencies

As you can see, SolidOS is composed of several repositories:

  • rdflib.js — Javascript RDF library for browsers and Node.js
  • solid-logic — core business logic of SolidOS
  • mashlib — a solid-compatible code library of application-level functionality for the world of Solid
  • solid-panes — a set of core solid-compatible panes based on solid-ui
  • solid-ui — User Interface widgets and utilities for Solid. Building blocks for solid-based apps

In the above diagram, SolidOS is deployed on the Node Solid Server (NSS), but it can also be set up to run on the Community Solid Server (CSS) or on ANY Solid-compliant server. When you download and compile the SolidOS code, an NSS is also installed locally, to have everything ready to develop.

SolidOS deeper technical topics

For further details about each GitHub repository, please visit them via the links above for Documentation.

We collect SolidOS code good practices and know how in SolidOS documentation pages.

SolidOS FAQs part of the SolidOS developer guide also contains some Q&A and technical troubleshooting infos.

👯 How the SolidOS team works

First and foremost who are the contributors of SolidOS?

The SolidOS codebase has a long history and there have been a lot of contributors over the years (see: GitHub contributors). The most active team members are mentioned in the SolidOS Team on the SolidOS Pod Contacts.

SolidOS team meetings

The SolidOS team meets every week for a 1h touchdown. We discuss what was done over the past week, what needs to be done next, and delegation of tasks. Find the meeting time and link on the SolidOS project Pod.

We take minutes on our meetings. You can find them on the SolidOS pod.

SolidOS team instant chat

In between team meetings, we avidly communicate over at the gitter SolidOS channel. Drop by to chat with us, ask questions, or simply say "Hi".

SolidOS team discussions

Sometimes some ideas need an incubation period and further discussion. We make use of GitHub discussions for that.

SolidOS tasks

For daily tasks, we have a Project Board with different views.

For a longer-term roadmap, we use a Solid task manager, and plan the next milestones on Kanban.

Additional useful information

🙋🏽‍ How you can contribute and help SolidOS thrive

The SolidOS team is always looking for volunteers to help improve SolidOS. Pull Requests (PRs) and edits are always welcome from code, to text, to style. We are looking for UX designers, technical writers, frontend developers, backend developers, DevOps. Don't let these titles intimidate you; they are just some examples. You can find your own place no matter the level of knowledge you are at.

To check for tasks you might help with immediately, look at the Newcomer View in the Project Board. You are welcome to visit us at a weekly team meeting or on the ongoing Gitter-based chat to say 'Hi' or let us know about any blocker you might have encountered.

For anyone up to writing some code

We keep track of stuff to do in Git issues of each repo. An overview you can find on the project board.

Writing tests as a way to understand the code is always a good idea. Tests, in each repo, should be found in the test folder. One can start from there or/and add tests.

Note: Please get familiar with coding and testing guidelines.

For anyone who likes builds or GitHub CI or releases or deployments

There is an existing process and codebase in place to help with SolidOS releases. However, we would like to get better and automate as much as possible. Open issues can be found on the Project Board under the CI category.

Note: Please get familiar with release guidelines.

Builds

SolidOS contains different repositories (mashlib, solid-logic, solid-ui, solid-panes, ...). Each repository contains a package.json with scripts. Most repos contain an npm run build which is used to build the project.

GitHub CI

When you push or PR a change to a repo, a git CI is activated and runs every time. An example is the solid-panes workflow. This CI YML can contain instructions to test and build the repo on different Node versions. If upon push or PR, an instruction fails, one should take care to fix it and keep the branches green.

Testing & releasing a new SolidOS version

In SolidOS, you will find a bash scripts under scripts which is related to releasing a new SolidOS stack. The release script is also used to update dependencies in each repo.

Following best practices, we deploy the new version on the testserver as mentioned here.

Deployment on solidcommunity.net server

Before you start, make sure you have access to all the GitHub repos and all the npm packages. Using Ubuntu or other Unix-like OS, ssh into the server as root.

tmux new
adduser --shell /bin/bash --home /home/build --ingroup sudo build
su - build
whoami
sudo whoami

Then:

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.0/install.sh | bash
export NVM_DIR="$([ -z "${XDG_CONFIG_HOME-}" ] && printf %s "${HOME}/.nvm" || printf %s "${XDG_CONFIG_HOME}/nvm")"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm
nvm install 16
nvm use 16

ssh-keygen -t ed25519 -C "michiel+build@unhosted.org"
git config --global user.name "Solid OS Build (Michiel)"
git config --global user.email "michiel+build@unhosted.org"
cat .ssh/id_ed25519.pub
npm login

Log in to npm with your npm account and add the SSH public key to your GitHub account. Then continue:

git clone https://github.com/solidos/solidos
cd solidos
npm run install-nvm
npm run release

More information can be also found over at the server, solidcommunity.net, repo.

For anyone who likes writing text

SolidOS has quite some documentation around that needs constant improvement. Places to start:

We are open to suggestions to improve these resources from structure, translation, UI to content in general.

For anyone with an eye for design

Solid-ui does the heavy lifting to all things UI for SolidOS. Currently, we use Storybook to help develop components independent of other panes. Make sure to visit the solid-ui readme for information on how to set it up and get started. There is a second option to run solid-ui on its own. Read about it at Debugging solid-ui using Solid Pane Tester.

You can also find the current issues over at the solid-ui issues. And some more information over at the SolidOS Wiki.

SolidOS needs a lot of improvements on UI, including UX and style-guides. Maybe you are the one who can help out? Visit the Project Board and look for UX and UI categories.

🆕 Getting started with the SolidOS code

Before you start coding, please review our guidelines:

SolidOS first time setup of code

Make sure you have the needed environment: nvm for SolidOS, npm, Node. If you have problems with node versions on the Apple M1 chip, in the Troubleshooting SolidOS you can find a solution.

git clone https://github.com/solidos/solidos
cd solidos
npm run setup

Note: It might prompt you to install a specific node version, something like nvm install xxx # version missing mentioned in console log.

Note: In case of errors, try to follow what the output messages (errors) suggest in the console to fix the problems, and let us know on the SolidOS team chat.

Run the above lines in a terminal of your choice to setup your SolidOS project folder. By default, some dependent repos are also set up for you:

  • rdflib.js: Javascript RDF library for browsers and Node.js
  • solid-logic: core business logic of SolidOS
  • pane-registry: an index to hold all loaded solid panes
  • mashlib: a solid-compatible code library of application-level functionality for the world of Solid
  • solid-panes: a set of core solid-compatible panes based on solid-ui
  • solid-ui: User Interface widgets and utilities for Solid. Building blocks for solid-based apps
  • node-solid-server: the server that allows you to test your changes

You can start your server and test out your code with:

npm start

If you get into problems check out SolidOS FAQs or ask us directly at SolidOS team chat.

Note: The NPM scripts are using bash scripts. These might not work if you're developing on a Windows machine. Let us know, over at SolidOS team chat if you want support for this.

How to use SolidOS on localhost

Once you managed to get SolidOS running locally (npm start) you can see it over at https://localhost:8443/. If you encounter any problems make sure to check the Troubleshooting SolidOS page.

To work on localhost, first you need to register a local user, so hit register on https://localhost:8443/. After you have created your user, you can navigate to your new pod over at https://username.localhost:8443/. Whenever you need to login again, remember to put https://localhost:8443/ in the Enter the URL of your identity provider: input field. Otherwise you will be logged in with a different provider and redirected out of the localhost environment.

How to make changes in repos

As a newcomer, you do not have direct access to the repos, but you can still contribute through Pull Requests (PRs). First, navigate to the repo you want to work on, and create a fork. Make your changes on your fork, and then create a PR. We will be notified, and you will receive feedback on your changes. For more details on how to do this, visit the GitHub documentation, which explains it much better than we ever could.

If you do have direct access to the repos, it is usual to create a branch for your changes and then a PR. A PR helps you receive feedback and lets us know easily about any changes to the code. Read more about Pull Requests over at the GitHub documentation.

Make sure to read more about working with branches and missing repos over at the SolidOS Wiki.

Developing SolidOS code

Very likely you will want to make changes in the dependent packages/repos of SolidOS (mashlib, solid-logic, solid-ui, solid-panes...).

You have two choices:

Work directly in SolidOS

The npm start script contains a lerna command: npx lerna bootstrap --force-local which makes sure that packages are bootstrapped/taken from your local machine even if versions don't match.

If you need to bootstrap any packages again (e.g. you've run npm install in any of the projects) and don't want to stop the server, you can do npx lerna bootstrap --force-local only. You do not need to stop the server and start it again (npm start).

Another option is to start SolidOS with the npm run watch script. This triggers the watch-script for mashlib, solid-ui, and solid-panes. If you want to run watch-script for rdflib or any of the panes, you'll have to open another terminal window, navigate to the respective project and start its watch-script doing npm run watch.

The output for the watch-script can be a bit difficult to interpret since all output for mashlib, solid-ui, and solid-panes are presented in the same window. You might also consider having each watch scripts running in a separate terminal window. The downside of using this approach is that at its worst, you'll have five separate watch-scripts running (in addition to the terminal window where you started the server) when working on a pane that needs to pick up a change in rdflib. If you find this unwieldy for your setup, or require too many resources, you should consider to work in the according dependent package.

Work in the according dependent package

Any changes you do in a project need to be committed to their original repos and eventually be pushed to NPM manually (this is the part of Lerna that we do not use for this project).

Some projects require you to build a package before you can see changes, so check the various package.json files to see which scripts are available. You can usually do npm run build, and some also support npm run watch which builds a new version each time you do a local change.

Be aware, the packages depend on one another. Here's a simplified view of the dependencies:

node-solid-server --> rdflib
node-solid-server --> mashlib --> rdflib
node-solid-server --> mashlib --> solid-panes --> rdflib
node-solid-server --> mashlib --> solid-panes --> solid-ui --> rdflib
node-solid-server --> mashlib --> solid-panes --> [pane project] --> solid-ui --> rdflib

This means that if you do a change in solid-panes and want to see the result on your local NSS, you need to make sure that mashlib compiles the changes as well. Similarly, if you do changes to solid-ui, and some pane relies on those changes, you need to make sure that the pane compiles those changes, that solid-panes compiles the changes from the pane, and finally that mashlib compiles the changes from solid-panes. This quickly becomes hard to track, so we've devised a couple of ways to mitigate this.

Read about in detail how each pane can be debugged over at the SolidOS Wiki.

Testing SolidOS code

Most of the modules in SolidOS have a test script which can be called with npm run test. In some cases the tests run an ESLint command eslint 'src/**/*.ts' or a jest test or both.

Jest can also offer information related to test coverage by simply runnig npm run coverage.

You can find a repo's tests usually in a dedicated folder called test.

SolidOS build and release

The SolidOS code stack build and release are described above.

📜 License

The SolidOS code is available under the MIT License.

🎤 Feedback and questions

Don't hesitate to chat with us on gitter or report a bug.