This is the main monorepo of Unearth. This repository contains the code for the backend server, frontend client, and various scripts consumed by cron jobs.
Reddit is a goldmine of information—with posts and comments ranging in emotion from funny and lighthearted, to straight-to-the-point and motivational, to everything in-between and beyond. It's not uncommon for Redditors to save these things, finding value in them and wishing to have a reference to it later on.
But saving things leads to a couple of problems. For starters, the number of saves allowed on Reddit is limited and there is the chance of previous saves being overwritten when that limit is reached. The other issue is many Redditors save things but forget to ever go back and view said things. Read more about why Reddit's save functionality is subpar.
The current tools provided by Reddit are insufficient at helping Redditors manage and rediscover their saved things.
Unearth aims to be the best service to help Redditors rediscover the things they have saved through the use of active reminders. By leveraging content resurfacing to create personalized emails with users' saved things, we will be able to help more Redditors take control of their saved content and evoke a sense of nostalgia as they are reminded of the posts and comments they have saved during their time on Reddit.
Unearth is currently still in development but the minimum viable product (MVP) is completed. Feel free to try Unearth today.
If you found a technical bug on Unearth or have ideas for features we should implement, the issue tracker is the best place to share your ideas. (click here to open a new issue)
If you find a bug on Unearth and open a PR that fixes it we'll review it as soon as possible.
If you want to implement a new feature, open an issue first to discuss what it'd look like to ensure it fits in our roadmap.
Want to fix a bug or implement an agreed-upon feature? Great, jump to the local setup instructions!
With the ground rules out of the way, let's talk about the core architecture of this monorepo:
-
Full-stack JavaScript: we use Node.js to power our servers, and React to power our frontend apps. Almost all of the code you'll touch in this codebase will be JavaScript.
-
Background Jobs: background jobs are the bread-and-butter of Unearth—they are what send the emails at the correct frequency. The jobs themselves execute scripts which can be found in the
jobs/directory.
Here is a list of all the big technologies used in this project:
-
React/Gatsby: frontend client app
-
Express.js: RESTful API server
-
SocketIO: WebSocket endpoint for bi-directional communication between client and server
-
PostgreSQL: data storage
-
Heroku/Netlify: platform hosting
unearth/
├── .github # GitHub-specific files for managing this repo
├── api # RESTful API endpoints
├── client # DEPRECATED - frontend SPA
├── client-v2 # Gatsby dynamic web app
├── database # config file, migrations, and seeds for postgres db
├── docs # coming soon!
├── emails # Pug.js email templates
├── jobs # executed using cron
├── mailers # business logic for sending emails
├── services # business logic for connecting to Reddit
├── tests # mix of functional and unit tests for the backend app
└── utils # shared functionality used throughout the backend app
The first step to running Unearth locally is downloading the code by cloning the repository:
$ git clone https://github.com/cedricium/unearth.gitUnearth uses a variety of technologies that need to be installed on your local machine before development can begin. The general installation steps are as follows:
- Install PostgreSQL: see the PostgreSQL documentation for instructions on installing it with your OS
- Install yarn: we use yarn to handle our JavaScript dependencies
- Install the dependencies: finally, install the dependencies for the back-
and front-end projects (
$ yarn installin both the root andclient/directories)
When you first download the code and want to run it locally you have to migrate the database and seed it with test data. First, ensure postgres is up and running on your machine.
Then in a new terminal tab or window, run these commands:
$ yarn db:migrate
$ yarn db:seedThere's also a shortcut for dropping, migrating and seeding the database:
$ yarn db:resetNote: the easiest way to get real test data into the database is to go through the onboarding process using your own Reddit account.
In order to get all components of the app working, there are several secrets needing to be configured. To get those set up, copy the provided example secrets file to the real location:
$ cp .env.example .envNote: this will need to be done in both the root and the
client-v2/directories
Now you're ready to run the app locally and start development!
Whenever you want to run Unearth locally it is advised you have postgres running in the background. First start postgres then make sure you have migrated and seeded the database.
No matter what you're trying to work on, you'll want to have the API running, so start that in a background tab:
$ yarn devThe current Unearth website is built using GatsbyJS and is hybrid static/dynamic web application.
Look over the GatsbyJS documentation for more information:
- Gatsby Docs: https://www.gatsbyjs.org/docs/
To get started making contributions to the frontend client:
$ cd client-v2then start the app:
$ yarn devAt the time of this writing, all of the email templates have been written using Pug.js, so some familiarity with that would be beneficial.
When developing locally, emails are not actually sent; rather, they go through Ethereal, a fake SMTP service that displays the rendered emails in the browser. This is helpful when making changes to the email templates and you want to preview said changes.
Visit the Nodemailer and Ethereal documentation for more information:
-
Nodemailer: https://nodemailer.com/
-
Ethereal: https://ethereal.email/
Note: if something did not work or you ran into any issues during the setup process, please submit PRs to improve this doc and keep it up to date!
MIT, see the LICENSE file.