This repo contains the React/Gatsby application that powers the EDB Docs website. The site pulls Markdown content from several repos in a process called "sourcing", and then renders it all into high-performance markup. You can install the application on your local computer for easy editing, viewing, and eventually publishing to the GitHub repo.
We've been checking PDF files into Git. That's not a good plan, so we've stopped doing that. The next step is to remove these files from Git history and move other large files to LFS. Among many other good things, that ought to reduce the time to clone this repository substantially.
But it comes at a cost. If there are any local repositories that were cloned before the change, we risk introducing dirty history back into the repository. So we're asking that everyone who has a local repository they cloned before (or on) August 13, 2021 to delete those repositories. Unfortunately, we'll need to reject any pull requests that introduce PDF files back into Git history. (If you need any help with this, please contact jon.ericson@enterprisedb.com.)
We recommend using MacOS to work with the EDB Docs application.
-
Install Homebrew, if it's not already installed. (Use
brew -v
to check.) -
Install Git as well as Git-LFS using Homebrew with
brew install git git-lfs
, if they're not already installed. (Usegit --version
andgit-lfs --version
to check.) -
Set up an SSH key in GitHub, if you haven't done so already. (Go to GitHub's SSH Keys page to check.) If you don't have an SSH Key set up yet, you'll need to set one up to authenticate you to GitHub. See GitHub's SSH docs for more information.
-
Clone the EDB Docs GitHub repo to your local machine using GitHub Desktop, or via the Terminal with
git clone https://github.com/EnterpriseDB/docs.git
-
Navigate to the cloned repo directory in your Terminal, if you haven't already done so.
-
Create a
.env.development
file:cp env.development.example .env.development
.
At this point you have a couple options.
-
This is the preferred set up method. Choose this option if you just want to make updates to documentation, and don't want to worry about installing and managing the correct version of Node.
-
Set up a full development environment
Choose this option if you are an advanced user and need to make more in depth changes to the docs application, such as new functionality.
-
Install Docker on your mac. Follow the direction below.
-
If you're a member of the EnterpriseDB Github Org, follow the instructions below to enable icons in the docs application.
-
Navigate to the cloned repo directory in your terminal
-
Run
npm run start
. The application will start in the background and take a few minutes to load. -
You can view logs and monitor the startup process by running
npm run logs
. Once it's finished it can be accessed athttp://localhost:8000/
.
-
npm run stop
— Stop the dev server -
npm run logs
— View logs from the server, to exit the logs pressctrl
+c
-
npm run shell
— open up a shell for the dev container -
npm run docker:rebuild
— rebuild the images used for the dev container -
To run the server on a different port, change the
PORT
config in.env.development
and restart the dev server
If you find that the container crashes frequently or see that your container has exited with code 137, increasing the Docker memory should help. Allocating at least 4GB is recommended.
-
open Docker Desktop
-
Go to Preferences (gear icon in the top right corner)
-
Click on Resources in the left nav menu
-
In the Memory section, adjust the slider to 4.00 GB
-
Click Apply & Restart button
If you are still experiencing errors or other issues with the site, try the following in the project folder:
-
npm run stop
thennpm run start
to restart the docker container -
npm run docker:rebuild
thennpm run start
to rebuild and restart the container
You will need to follow these instructions if you want to build PDFs locally, or get started quickly with Docker.
brew install --cask docker
If you get a message saying that you already have Docker installed, check which version is installed using these commands:
brew ls --formula docker
brew ls --cask docker
If the first command yields results, enter the following command to uninstall the formula version and to install the cask version:
brew uninstall -f docker && brew install --cask docker
These instructions are for members of the EnterpriseDB Github Org only. The icons we are using are not free, and so cannot be distributed as part of the open source docs repository. You'll want to follow these steps so that icons are pulled to the docs application instead of placeholder icons.
-
Create a Github token which can pull private packages. You can learn how to do that here.
-
The minimum scope the token requires is
read:packages
-
Make sure to enable SSO for your token or it may not work correctly
-
-
Once you have that token, update your
.env.development
file with this line:NPM_TOKEN=your-token-here
. That will be utilized in the NPM_TOKEN variable in the .npmrc file.
-
Install Node.js version 14 LTS. We recommend using Node version 14 LTS (the Long Term Support release) as version 15 is not compatible with some of our dependencies at this time.
-
If you already have Node installed, you can verify your version by running
node -v
in the cloned repo directory. -
If you already have a different version of Node installed, you may want to consider using Node Version Manager (NVM) for a simpler way to manage multiple versions of Node.js. Follow the directions to install NVM, then run
nvm install
in the cloned repo directory, followed bynvm use
which will auto-detect the correct version of Node.js to use (currently 14 LTS).
-
-
Install Python 3 with
brew install python3
, if it's not already installed. (Usepython3 -V
to check that you have version 3.8 or higher.) Python is not needed for the core Gatsby system, but is required by several source scripts. -
NPM 7 is the package manager we're using for this project.
-
run
npm -v
to ensure you are using the correct version of npm. If you do not have version 7, you can runnpm install -g npm@7
to install it. -
NPM may fail with a permissions related issue. To fix that, ensure that your user account owns the required directory:
sudo chown -R $(whoami) /usr/local/lib/node_modules
-
-
Install Gatsby with
npm i -g gatsby-cli
. Gatsby is the software that powers the EDB Docs site. -
If you're a member of the EnterpriseDB Github Org, follow the instructions above to enable icons in the docs application.
-
In order to ensure environmental variables are properly set, install direnv and run
direnv allow .
-
Install all required packages by running
npm install
-
And finally, you can start up the site locally with
npm run develop
, which should make it live athttp://localhost:8000/
. Huzzah!
If you experience errors or other issues with the site, try the following in the project folder:
rm -rf node_modules
to clean out installed JavaScript packagesnpm install
to reinstall JavaScript packagesnpm run clean
to clean up Gatsby cachenpm run develop
to start the development environment again. Keep in mind this will take longer than usual as Gatsby will need to rebuild everything.
To build PDFs locally, you'll need to use a Docker container.
-
Install Docker on your mac. Follow the direction above.
-
Start the Docker app. You can tell whether Docker has started or not by looking at your menu bar icons, you should see a whale with containers on its back:
-
Create the Docker image (optional):
npm run pdf:rebuild-docker-image
-
Run the following command inside the docs project to create a PDF:
npm run pdf:build product_docs/docs/<product_folder>/<version>
For example, to build a PDF for the EPAS 13 documentation:
npm run pdf:build product_docs/docs/epas/13
If you need to run parts of the RST to MDX conversion pipeline, you'll need to install pandoc
, a general purpose document conversion tool. This can also be installed with homebrew - brew install pandoc
.
If you are a Windows user, you can work with Docs without installing it locally by using a Docker container and VSCode. See Working on Docs in a Docker container using VSCode
-
Advocacy (
/advocacy_docs
, always loaded) -
EDB Product Docs (
/product_docs
)- For a list of these sources, see product_docs/docs
By default, all document sources will be loaded into the app during development. It's possible to set up a configuration file, dev-sources.json
, to only load specific sources, but this is not required.
Run npm run config-sources
to setup your dev-sources.json
file. This file tells Gatsby which sources to load. The script is interactive!
Alternatively, you can setup your dev-sources.json
file manually by copying dev-sources.sample
to dev-sources.json
, and editing as desired. The sample file will source everything by default.
Advocacy Docs are tutorial content, getting-started material, and anything that is about a subject matter area, but not explicitly tied to a product version.
Product Docs are versioned documentation for products. They follow a slightly stricter file structure to allow for version switching and other features.
More details can be found on the Adding New Sources page.
See Adding New Sources for a guide to choosing a source type, adding the files, and other configuration.
All changes should have a pull request opened against the default branch, develop
. To generate [#draft-deployments](Draft deployments) for the branch, add the deploy
label to the pull request: a new deployment at a unique URL will be produced every time changes are pushed to the branch. Note: GitHub must be able to merge the branch cleanly in order for this to work; if there are conflicts shown on the pull request, resolve them in order to obtain a new draft deployment.
When a PR is merged into the develop
branch, the result will be deployed to the staging environment.
To deploy to production, create a pull request merging develop
into main
. When that PR is merged, main
will automatically build and deploy to the production site.
Deployments of the site use the build-sources.json
file to determine which sources need to be loaded. All environments are continuously deployed - new commits to relevant branches will trigger a build of the associated environment. The builds are done using Github Actions, so you can view deployment progress by clicking the "Actions" tab.
Staging is hosted on Netlify, and is built from the develop
branch. The build and deployment process is handled by the deploy-develop.yml
GitHub workflow.
Staging environment URL: https://edb-docs-staging.netlify.app/docs/
Production is hosted on Netlify, and is built from the main
branch. The build and deployment process is handled by the deploy-main.yml
GitHub workflow. The production deployment process will update the search index on Algolia.
Production environment URL: https://www.enterprisedb.com/docs
Review builds are automatically created for pull requests when the deploy
tag is added. The build and deployment process is handled by the deploy-draft.yml
GitHub workflow. Draft builds are a Netlify feature - each new draft has a unique URL (based on the Staging URL) that will persist even when later revisions are deployed.
The app is concerned with two different types of redirects that can be defined in frontmatter.
For specific examples of when to use redirects, see: How to avoid breaking links when reorganizing, consolidating or deprecating content.
The redirects
frontmatter is to be used for redirects internal to Docs. For example, if you had a file great_file.mdx
with this following frontmatter...
redirects:
- "/old_path"
- "/another_old_path"
both /old_path
and /another_old_path
would redirect to great_file.mdx
's current path. This is perfect for setting up redirects when moving a file around within Docs. Redirects created with redirects
are permanent (301).
These paths can be absolute (starting with the root of the site) or relative (to the file in which they are contained). For a /file/at/path/
,
- Absolute:
/path/to/file/
- redirects requests for /path/to/file to /file/at/path/ - Relative:
former_child/
- redirects requests for /file/at/path/former_child/ to /file/at/path/ - Relative:
../sibling/
- redirects requests for /file/at/sibling/ to /file/at/path/
Netlify is the hosting service we use for Docs. Netlify-specific redirects can be found in /static/_redirects
. These are generally used for large-scale redirects, such as when renaming or removing an entire product version.
This app builds a list of nginx style redirects that are loaded into a separate server. These redirects direct users from links to the old docs site, to the appropriate page on the new docs site.
This frontmatter is an automatically generated list of redirects for Docs 1.0 to Docs 2.0 (this repo). These redirects are built by scripts/legacy_redirects/add_legecy_redirects.py
, and should not be manually edited.
If you need to setup a redirect from Docs 1.0 to Docs 2.0 manually, this is the place to do it. If the legacyRedirectsGenerated
frontmatter does not include the redirect you need, you should add it here.
Documentation must be formatted as an MDX file with the .mdx
extension. MDX is a superset of Markdown.
Each document requires a YAML frontmatter section at the top with a title:
---
title: Title of page
---
If the title contains special characters, it will need to be quoted. There also needs to be a space after title:
In addition to title
, frontmatter may optionally include navTitle
and description
:
---
title: An exhaustive guide to all things wonderful about Postgres
navTitle: Postgres guide
description: Everything you need to know about Postgres
---
The navTitle
is used for the left navigation so it can take up less space. It is also used in "cards".
The description
is used in cards as well.
All of these files use Markdown for styling. The options for what can be done can be seen here
If you need to draw attention to information, consider using an admonition:
!!! Note Optional title
This is text you'd like the reader to notice.
Admonitions begin with the !!!
signifier. Next comes a (case-insensitive) type which is one of:
- important
- tip
- note
- caution
- warning
There are several aliases:
- info => important
- success => tip
- secondary => note
- danger => warning
- seealso => note
- hint => tip
Titles are optional. If you don't include one, the admonition will default to the type name ("Important", "Tip", etc.).
The body of the admonition is indented 4 spaces. It should line up with the first letter of the admonition type. Alternatively, fenced admonitions that begin and end with !!!
lines are supported:
!!! Tip
Use fenced admonitions to avoid space counting.
!!!
For examples of what you can do with admonitions, see this demo.
The items in the left nav are sorted alphabetically by file name. This can be done with a numerical prefix. The titles of each page are used for the names in the left nav.
Content is indexed for search when the production site builds.
To contribute content to this site submit as a pull request (PR). There are two options for this:
Option 1: locally
- Clone this repository.
- Make a new branch.
- Add commits to branch and push to GitHub.
- Create a new PR on GitHub.
Option 2: on GitHub
- Edit a file on GitHub.
- Submit changes as a PR on a new branch.