Graphistry is the most scalable graph-based visual analysis and investigation automation platform. It supports both cloud and on-prem deployment options. Big graphs are tons of fun!
The documentation here covers system administration
See bottom of page for table of contents and additional resources.
Pick an appropriate hardware/software configuration:
- Graphistry Hub: Graphistry manages Hub for its users
- AWS/Azure Marketplace: See below instructions
- Docker (self-hosted): See enterprise release portal
- Experimental - Kubernetes Helm chart: See graphistry/graphistry-helm repository
Email, Zoom, Slack, phone, tickets -- we encourage using the Graphistry support channel that works best for you. We want you and your users to succeed!
See also release guides and user and developer documentation
AWS and Azure marketplaces: The fastest way to start using Graphistry is to quick launch a preconfigured private Graphistry instance on AWS and Azure marketplaces
It runs in your private cloud provider account and is configured to autostart. See AWS launch walkthrough tutorial & videos and linked guides for optional post-launch configuration and maintenance.
Out-of-the-box configurations include GPU drivers, Docker with Nvidia runtime, multi-GPU support, Graphistry installation, auto-start on instance stop/restart, and more
Requirements: Download Graphistry and verify docker-compose is setup for Nvidia runtimes
1. Install if not already available from the folder with containers.tar.gz
, and likely using sudo
:
docker load -i containers.tar.gz
Note: In previous versions (< v2.35
), the file was containers.tar
2. Launch from the folder with docker-compose.yml
if not already up, and likely using sudo
:
docker-compose up -d
Note: Takes 1-3 min, and around 5 min, docker ps
should report all services as healthy
3. Test: Go to
http://localhost
-
Create an account, and then try running a prebuilt Jupyter Notebook from the dashboard!
- The first account gets an admin role, upon which account self-registration closes. Admins can then invite users or open self-registration. See User Creation for more information.
-
Try a visualization like http://localhost/graph/graph.html?dataset=Facebook&play=5000&splashAfter=false
- Warning: First viz load may be slow (1 min) as RAPIDS generates just-in-time code for each GPU worker upon first encounter, and/or require a page refresh
Graphistry supports advanced command-line administration via standard docker-compose
, .yml
/ .env
files, and caddy
reverse-proxy configuration.
Image | Command |
---|---|
AWS | ssh -i [your_key].pem ubuntu@[your_public_ip] |
Azure | ssh -i [your_key].pem [your_user]@[your_public_ip] ssh [your_user]@[your_public_ip] (pwd-based) |
gcloud compute [your_instance] ssh |
|
On-prem / BYOL | Contact your admin |
All likely require sudo
. Run from where your docker-compose.yml
file is located: /home/ubuntu/graphistry
(AWS), /var/graphistry
(Azure), or /var/graphistry/<releases>/<version>
(recommended on-prem).
TASK | COMMAND | NOTES |
---|---|---|
Install | docker load -i containers.tar.gz |
Install the containers.tar.gz Graphistry release from the current folder. You may need to first run tar -xvvf my-graphistry-release.tar.gz . |
Start interactive |
docker-compose up |
Starts Graphistry, close with ctrl-c |
Start daemon |
docker-compose up -d |
Starts Graphistry as background process |
Start namespaced (concurrent) |
docker-compose -p my_unique_namespace up |
Starts Graphistry in a specific namespace. Enables running multiple independent instances of Graphistry. NOTE: Must modify Caddy service in docker-compose.yml to use non-conflicting public ports, and likewise change global volumes to be independent. |
Stop | docker-compose stop |
Stops Graphistry |
Restart (soft) | docker restart <CONTAINER> |
Soft restart. May also need to restart service nginx . |
Restart (hard) | docker up -d --force-recreate --no-deps <CONTAINER> |
Restart with fresh state. May also need to restart service nginx . |
Reset | docker-compose down -v && docker-compose up -d |
Stop Graphistry, remove all internal state (including the user account database!), and start fresh . |
Status | docker-compose ps , docker ps , and docker status |
Status: Uptime, healthchecks, ... |
GPU Status | nvidia-smi |
See GPU processes, compute/memory consumption, and driver. Ex: watch -n 1.5 nvidia-smi . Also, docker run --rm -it nvidia/cuda:11.5.0-base-ubuntu20.04 nvidia-smi for in-container test. |
1.0 API Key | docker-compose exec streamgl-vgraph-etl curl "http://0.0.0.0:8080/api/internal/provision?text=MYUSERNAME" | Generates API key for a developer or notebook user (1.0 API is deprecated) |
Logs | docker-compose logs <CONTAINER> |
Ex: Watch all logs, starting with the 20 most recent lines: docker-compose logs -f -t --tail=20 forge-etl-python . You likely need to switch Docker to use the local json logging driver by deleting the two default managed Splunk log driver options in /etc/docker/daemon.json and then restarting the docker daemon (see below). |
Create Users | Use Admin Panel (see Create Users) or etc/scripts/rest |
|
Restart Docker Daemon | sudo service docker restart |
Use when changing /etc/docker/daemon.json , ... |
Jupyter shell | docker exec -it -u root graphistry_notebook_1 bash then source activate rapids |
Use for admin tasks like global package installs |
NOTE: Managed Graphistry instances do not require any of these steps.
The Graphistry environnment depends soley on Nvidia RAPIDS and Nvidia Docker via Docker Compose 3
, and ships with all other dependencies built in.
You can test your GPU environment via Graphistry's base RAPIDS Docker image on DockerHub:
docker run --rm -it --entrypoint=/bin/bash graphistry/graphistry-forge-base:latest -c "source activate rapids && python3 -c \"import cudf; print(cudf.DataFrame({'x': [0,1,2]})['x'].sum())\""
=>
3
See GPU testing to identify individual issues.
See sample Ubuntu 18.04 TLS and RHEL 7.6 environment setup scripts for Nvidia drivers, Docker, nvidia-docker runtime, and docker-compose. See Testing an Install for environment testing.
Please contact Graphistry staff for environment automation options.
Download the latest enterprise distribution from the enterprise release list on the support site. Please contact your Graphistry support staff for access if not available.
If nvidia
is already your docker info | grep Default
runtime:
############ Install & Launch
wget -O release.tar.gz "https://..."
tar -xvvf release.tar.gz
docker load -i containers.tar.gz
docker-compose up -d
-
Where are the docs? See this GitHub repository for admin docs and Graphistry Hub docs (or http://your_graphistry/docs) for analyst and developer docs
-
Where do I get help? Whether community chat, email, tickets, a call, or even a training, pick the most convienent option
-
Can Graphistry run in the cloud? Yes - privately both via a preconfigured AWS/Azure marketplace and as a self-managed docker binary. Contact our team for upcoming managed Graphistry Hub cloud tiers.
-
Can Graphistry run privately?
- On-prem, including air-gapped, as a team backend server or a Linux-based analyst workstation, via docker image
- Cloud, via prebuilt marketplace instance
- Cloud, via docker image
-
Can Graphistry run in ...
- A VM: Yes, including VMWare vSphere, Nutanix AHV, and anywhere else Nvidia RAPIDS.ai runs. Just set
RMM_ALLOCATOR=default
in yourdata/config/custom.env
to avoid relying on CUDA Unified Memory, which vGPUs do not support. - Ubuntu / Red Hat / ... : Yes, just ensure the Nvidia Docker runtime is set as the default for docker-compose. We can assist with reference environment bootstrap scripts.
- Podman: Maybe! We have confirmed our core containers run on RHEL 8.3 with Podman, Nvidia container runtime, and docker-compose cli. Please contact our staff for the possibility of an alternate podman-compatible tarball.
- A VM: Yes, including VMWare vSphere, Nutanix AHV, and anywhere else Nvidia RAPIDS.ai runs. Just set
-
How do I do license management?
- Graphistry does not require software-managed license registration, we can work with your procurement team on self-reported use
-
Do I need a GPU on the client? No, clients do not need a GPU. They do need WebGL enabled, such as Chrome's non-GPU software emulation mode. If some of your users are on extremely limited environments, e.g., worse than a modern phone, or you have extremely powerful GPUs you would like to share, users report great experiences with GPU VDI technologies.
-
Do I need a GPU on the server? Yes, the server requires an Nvidia GPU that is Pascal or later (T4, P100, V100, A100, RTX, ...).
-
Can Graphistry use multiple GPUs and multiple servers?
- Graphistry visualizations take advantage of multiple GPUs & CPUs on the same server to handle more users
- Graphistry-managed Jupyter notebooks enable users to run custom GPU code, where each user may run multi-GPU tasks (e.g., via dask-cudf and dask-sql)
- For high availability configuration and operation, contact staff for additional guidance
- For many-node deployment and multi-GPU visualization acceleration, contact staff for roadmap
-
Can I run multiple instances of Graphistry? Yes, see the command section for running in an isolated namespace. This is primarily for testing and in-place upgrading. If your goal is for Graphistry to use multiple CPUs and GPUs, it already does so automatically, so you can launch as usual.
-
Can I use Graphistry from OS X / Windows? Yes, analysts can use any modern browser on any modern OS such as Chrome on Windows and Firefox on OS X, and even on small devices like Android phones and Apple tablets. The server requires Linux (Ubuntu, RHEL, ...) with a GPU. A self-contained analyst workstation would be Linux based.
-
How do I try it out?
- Notebook/API users can get a free account on Graphistry Hub
- Interact with pregenerated live visualizations on the PyGraphistry gallery
- If you have a private sample CSV/XLS/etc., you can spin up a private server in your AWS/Azure account and turn it off when done, and our team is happy to help
-
How can I test if my GPU supports Graphistry and my GPU environment is setup properly? Graphistry only requires a RAPIDS-compatible Docker environment, so you can use the community resources for that. In addition, see Testing an Install.
-
The server is slow to start, is it broken?
- The server may take 1-3min to start; check the health status of each service with
sudo docker ps
- By default, Graphistry has 4 RAPIDS workers (service
etl-server-python
) that perform just-in-time GPU compilation, meaning the first load on each is slow. - ... System start and the first visualization load per process might be sped up by ensuring Docker is using a native diff driver (see performance tuning)
- ... Subsequent use of those workers are fast for new datasets (code is already compiled), and subsequent reloads of recent datasets are extra fast (cached)
- The server may take 1-3min to start; check the health status of each service with
-
Can I add extra security layers? Yes -- see the hardening section for configuring areas like TLS, and contact the team for assistance with more custom/experimental layers like SSO
-
Can I run on another port? Yes -- modify
docker-compose.yml
's servicecaddy:
, such as for 80 to instead be 8888:
ports:
- 8888:80
expose:
- "8888"
-
Plan a deployment
- Architecture: Deployment architecture planning guide
- Capacity: Hardware/software requirements
- Security: Recommended hardening, threat model, authentication
-
Install
- Release portal for enterprise admins to download the latest Docker images
- Manual Graphistry Installation for non-marketplace / non-hosted:
AWS BYOL, Azure BYOL, On-Prem (RHEL/Ubuntu), & Air-Gapped - Nvidia vGPU virtualization support
-
Configure
- System settings:
TLS/SSL/HTTPS, performance, logging, backups to disk, multiple proxy layers, and more - Content:
- Investigations connectors: Splunk, Neo4j, and more
- Investigation templates: Save, reuse, link, and embed workflows
- Custom pivots: Streamline common investigation steps with simplified UIs
- Ontology: Add new types and customize colors, icons, sizes, and more
- The Graphistry Data Bridge: Go between cloud <> on-prem
- PyGraphistry and notebooks
- System settings:
-
Maintain
- Roadmap and release notes
- On unconfigured instance reboots, you may need to first run
sudo systemctl start docker
- Update, backup, and migrate
-
Use and develop
- Main documentation or the same URL adjusted for your local server's address
-
Test: