Create, share, fetch and model Atomic Data! AtomicServer is a lightweight, yet powerful CMS / Graph Database. Demo on atomicdata.dev
This repo also includes:
- Atomic Data Browser, the React front-end for Atomic-Server.
@tomic/lib
JS NPM library.@tomic/react
React NPM library.atomic_lib
Rust library.atomic-cli
terminal client.
Status: alpha. Breaking changes are expected until 1.0.
- 🚀 Fast (less than 1ms median response time on my laptop), powered by actix-web and sled
- 🪶 Lightweight (8MB download, no runtime dependencies)
- 💻 Runs everywhere (linux, windows, mac, arm)
- 🔧 Custom data models: create your own classes and forms. All verified and sharable using Atomic Schema
- 📄 Documents, collaborative, rich text, similar to Google Docs / Notion.
- ⚙️ Restful API, with JSON-AD responses.
- 🗄️ Tables, with strict schema validation, keyboard support, copy / paste support. Similar to Airtable.
- 💬 Group chat,
- 💾 Event-sourced versioning / history powered by Atomic Commits
- 🔄 Synchronization using websockets: communicates state changes with a client.
- 🌐 Embedded server with support for HTTP / HTTPS / HTTP2.0 and Built-in LetsEncrypt handshake.
- 🧰 Many serialization options: to JSON, JSON-AD, and various Linked Data / RDF formats (RDF/XML, N-Triples / Turtle / JSON-LD).
- 🔎 Full-text search with fuzzy search and various operators, often <3ms responses. Powered by tantivy.
- 📖 Pagination, sorting and filtering queries using Atomic Collections.
- 🔐 Authorization (read / write permissions) and Hierarchical structures powered by Atomic Hierarchy
- 📲 Invite and sharing system with Atomic Invites
- 📂 File management: Upload, download and preview attachments.
- 🖥️ Desktop app: Easy desktop installation, with status bar icon, powered by tauri.
- 📚 Libraries: Javascript / Typescript, React, Svelte
Powered by Rust, atomic-lib and more.
Atomic.Data.Screencap.mp4
- Table of contents
- When should you use this
- When not to use this
- Installation & getting started
- Initial setup and configuration
- Usage
- FAQ & Troubleshooting
- How do I make a website using this CMS?
- Can / should I create backups?
- I lost the key / secret to my Root Agent, and the
/setup
invite is no longer usable! What now? - How do I migrate my data to a new domain?
- How do I reset my database?
- How do I make my data private, yet available online?
- Items are missing in my Collections / Search results
- I get a
failed to retrieve
error when opening - Can I embed AtomicServer in another application?
- I want to use my own authorization. How do I do that?
- Where is my data stored on my machine?
- Also in this Repo
- Also check out
- Contribute
- You want a powerful, lightweight, fast and easy to use CMS or database with live updates, editors, modelling capabilities and an intuitive API
- You want to build a webapplication, and like working with using React or Svelte.
- You want to make (high-value) datasets as easily accessible as possible
- You want to specify and share a common vocabulary / ontology / schema for some specific domain or dataset. Example classes here.
- You want to use and share linked data, but don't want to deal with most of the complexities of RDF, SPARQL, Triple Stores, Named Graphs and Blank Nodes.
- You are interested in re-decentralizing the web or want want to work with tech that improves data ownership and interoperability.
- High-throughput numerical data / numerical analysis. AtomicServer does not have aggregate queries.
- If you need high stability, look further (for now). This is beta sofware and can change.
- You're dealing with very sensitive / private data. The built-in authorization mechanisms are relatively new and not rigorously tested. The database itself is not encrypted.
- Complex query requirements. We have queries with filters and features for path traversal, but it may fall short. Check out NEO4j, Apache Jena or maybe TerminusDB.
You can run AtomicServer in five ways:
- Using docker (probably the quickest):
docker run -p 80:80 -p 443:443 -v atomic-storage:/atomic-storage joepmeneer/atomic-server
- Install a desktop build (macOS only as of now)
- From a published binary
- Using Cargo from crates.io:
cargo install atomic-server
- Manually from source
When you're running AtomicServer, go to Initial setup and configuration. If you want to run this locally as a developer / contributor, check out the Contributors guide.
- Run:
docker run -p 80:80 -p 443:443 -v atomic-storage:/atomic-storage joepmeneer/atomic-server
Thedockerfile
is located in the project root, above thisserver
folder. - If you want to make changes (e.g. to the port), make sure to pass the relevant CLI options (e.g.
--port 9883
). - If you want to update, run
docker pull joepmeneer/atomic-server
and docker should fetch the latest version.
We automatically build .dmg
installers for MacOS. You can download them from the releases page.
Get the binaries from the releases page and copy them to your bin
folder.
# Make sure pnpm is installed and available in path! https://pnpm.io/
pnpm --version
# Install from source using cargo, and add it to your path
# If things go wrong, check out `Troubleshooting compiling from source:` below
cargo install atomic-server --locked
# Check the available options and commands
atomic-server --help
# Run it!
atomic-server
# make sure pnpm is installed and available in path! https://pnpm.io/
pnpm --version
git clone git@github.com:atomicdata-dev/atomic-server.git
cd atomic-server/server
cargo run
If things go wrong while compiling from source:
# If cc-linker, pkg-config or libssl-dev is not installed, make sure to install them
sudo apt-get install -y build-essential pkg-config libssl-dev --fix-missing
- You can configure the server by passing arguments (see
atomic-server --help
), or by setting ENV variables. - The server loads the
.env
from the current path by default. Create a.env
file from the default template in your current directory withatomic-server generate-dotenv
- After running the server, check the logs and take note of the
Agent Subject
andPrivate key
. You should use these in theatomic-cli
and atomic-data-browser clients for authorization. - A directory is made:
~/.config/atomic
, which stores your newly created Agent keys, the HTTPS certificates other configuration. Depending on your OS, the actual data is stored in different locations. See use theshow-config
command to find out where, if you need the files. - Visit
http://localhost:9883/setup
to register your first (admin) user. You can use an existing Agent, or create a new one. Note that if you create alocalhost
agent, it cannot be used on the web (since, well, it's local).
If you want to make your -server available on the web, but don't want (or cannot) deal with setting up port-forwarding and DNS, you can use a tunneling service. It's the easiest way to get your server to run on the web, yet still have full control over your server.
- Create an account on some tunneling service, such as tunnelto.dev (which we will use here). Make sure to reserve a subdomain, you want it to remain stable.
tunnelto --port 9883 --subdomain joepio --key YOUR_API_KEY
atomic-server --domain joepio.tunnelto.dev --custom-server-url 'https://joepio.tunnelto.dev' --initialize
You'll probably want to make your Atomic Data available through HTTPS on some server. You can use the embedded HTTPS / TLS setup powered by LetsEncrypt, acme_lib and rustls.
You can do this by passing these flags:
Run the server: atomic-server --https --email some@example.com --domain example.com
.
You can also set these things using a .env
or by setting them some other way.
Make sure the server is accessible at ATOMIC_DOMAIN
at port 80, because Let's Encrypt will send an HTTP request to this server's /.well-known
directory to check the keys.
The default Ports are 9883
for HTTP, and 9884
for HTTPS.
If you're running the server publicly, set these to 80
and 433
: atomic-server --https --port 80 --port-https 433
.
It will now initialize the certificate.
Read the logs, watch for errors.
HTTPS certificates are automatically renewed when the server is restarted, and the certs are 4 weeks or older.
They are stored in your .config/atomic/
dir.
Atomic-server has built-in HTTPS support using letsencrypt, but there are usecases for using external TLS source (e.g. Traeffik / Nginx / Ingress).
To do this, users need to set these ENVS:
ATOMIC_DOMAIN=example.com
# We'll use this regular HTTP port, not the HTTPS one
ATOMIC_PORT=80
# Disable built-in letsencrypt
ATOMIC_HTTPS=false
# Since Atomic-server is no longer aware of the existence of the external HTTPS service, we need to set the full URL here:
ATOMIC_SERVER_URL=https://example.com
In Linux operating systems, you can use systemd
to manage running processes.
You can configure it to restart automatically, and collect logs with journalctl
.
Create a service:
nano /etc/systemd/system/atomic.service
Add this to its contents, make changes if needed:
[Unit]
Description=Atomic-Server
#After=network.targetdd
StartLimitIntervalSec=0[Service]
[Service]
Type=simple
Restart=always
RestartSec=1
User=root
ExecStart=/root/atomic-server
WorkingDirectory=/root/
EnvironmentFil=/root/.env
[Install]
WantedBy=multi-user.target
# start / status / restart commands:
systemctl start atomic
systemctl status atomic
systemctl restart atomic
# show recent logs, follow them on screen
journalctl -u atomic.service --since "1 hour ago" -f
There are three ways to interact with this server:
- GUI: Use the
atomic-data-browser
JS frontend by visitinglocalhost:9883
. - API: We have a subset of the API documented using Swagger / OpenAPI. Check out ./_requests.http for various HTTP requests to the server. Also, read the Atomic Data Docs. You can also try the react boilerplate to build your own front-end app using @tomic/lib and @tomic/react.
- CLI: The
atomic-cli
terminal app
Open your server in your browser.
By default, that's http://localhost:9883
.
Fun fact: ⚛
is HTML entity code for the Atom icon: ⚛.
The first screen should show you your Drive. You can think of this as your root folder. It is the resource hosted at the root URL, effectively being the home page of your server.
There's an instruction on the screen about the /setup
page.
Click this, and you'll get a screen showing an Invite.
Normally, you could Accept as new user
, but since you're running on localhost
, you won't be able to use the newly created Agent on non-local Atomic-Servers.
Therefore, it may be best to create an Agent on some other running server, such as the demo Invite on AtomicData.dev.
And after that, copy the Secret from the User settings
panel from AtomicData.dev, go back to your localhost
version, and press sign in
.
Paste the Secret, and voila! You're signed in.
Now, again go to /setup
. This time, you can Accept as {user}
.
After clicking, your Agent has gotten write
rights for the Drive!
You can verify this by hovering over the description field, clicking the edit icon, and making a few changes.
You can also press the menu button (three dots, top left) and press Data view
to see your agent after the write
field.
Note that you can now edit every field.
You can also fetch your data now as various formats.
Try checking out the other features in the menu bar, and check out the collections
.
atomic-cli
is a useful terminal tool for interacting with atomic-server
.
It makes it easy to query and edit Atomic Data from the command line.
Check it out.
You can fetch individual items by sending a GET request to their URL.
# Fetch as JSON-AD (de facto standard for Atomic Data)
curl -i -H "Accept: application/ad+json" https://atomicdata.dev/properties/shortname
# Fetch as JSON-LD
curl -i -H "Accept: application/ld+json" https://atomicdata.dev/properties/shortname
# Fetch as JSON
curl -i -H "Accept: application/json" https://atomicdata.dev/properties/shortname
# Fetch as Turtle / N3
curl -i -H "Accept: text/turtle" https://atomicdata.dev/properties/shortname
Check out ./example_requests.http for more things that you can do. We have a subset of the API documented using Swagger / OpenAPI. Also, read the Atomic Data Docs to learn more about Collections, Commits, JSON-AD and other concepts used here.
Check out atomic-sveltekit-demo
You should.
Run atomic-server export
to create a JSON-AD backup in your ~/.config/atomic/backups
folder.
Import them using atomic-server import -p ~/.config/atomic/backups/${date}.json
.'
You could also copy all folders atomic-server
uses. To see what these are, see atomic-server show-config
.
You can run atomic-server --initialize
to recreate the /setup
invite. It will be reset to 1
usage.
There are no helper functions for this, but you could atomic-server export
your JSON-AD, and find + replace your old domain with the new one.
This could especially be helpful if you're running at localhost:9883
and want to move to a live server.
atomic-server reset
You can press the menu icon (the three dots in the navigation bar), go to sharing, and uncheck the public read
right.
See the Hierarchy chapter in the docs on more info of the authorization model.
You might have a problem with your indexes.
Try rebuilding the indexes using atomic-server --rebuild-index
.
Also, if you can, recreate and describe the indexing issue in the issue tracker, so we can fix it.
Try re-initializing atomic server atomic-server --initialize
.
Yes. This is what I'm doing with the Tauri desktop distribution of AtomicServer.
Check out the desktop
code for an example!
You can disable all authorization using --public-mode
.
Make sure AtomicServer is not publicly accessible, because this will allow anyone to read any data.
It depends on your operating system, because some data is temporary, others are configuration files, and so forth. Run atomic-server show-config
to see the used paths. You can overwrite these if you want, see --help
.
Atomic.Data.Screencap.mp4
A simple Command Line Interface tool to fetch, create and query Atomic Data. Especially useful for interacting with an AtomicServer.
A Rust library to serialize, parse, store, convert, validate, edit, fetch and store Atomic Data.
Powers both atomic-cli
and atomic-server
.
Front-end for Atomic-Server, built with React.
Library with Store
, Commit
, JSON-AD
parsing, and more.
React library with many useful hooks for rendering and editing Atomic Data.
- Atomic-Data-Browser, an in-browser app for viewing and editing atomic data. Also contains a typescript / react front-end library. Will replace most of the html templating in this project.
- The Docs, a book containing detailed documentation of Atomic Data.
- RayCast extension for searching stuff
- Newsletter
- Discord
Issues and PR's are welcome! And join our Discord! Read more in the Contributors guide.