OpenCompany Digest Service

Background

The single most important ingredient in the recipe for success is transparency because transparency builds trust.

-- Denise Morrison

Teams struggle to keep everyone on the same page. People are hyper-connected in the moment with chat and email, but it gets noisy as teams grow, and people miss key information. Everyone needs clear and consistent leadership, and the solution is surprisingly simple and effective - great leadership updates that build transparency and alignment.

With that in mind we designed Carrot, a software-as-a-service application powered by the open source OpenCompany platform and a source-available web UI.

With Carrot, important company updates, announcements, stories, and strategic plans create focused, topic-based conversations that keep everyone aligned without interruptions. When information is shared transparently, it inspires trust, new ideas and new levels of stakeholder engagement. Carrot makes it easy for leaders to engage with employees, investors, and customers, creating alignment for everyone.

Transparency expectations are changing. Organizations need to change as well if they are going to attract and retain savvy teams, investors and customers. Just as open source changed the way we build software, transparency changes how we build successful companies with information that is open, interactive, and always accessible. Carrot turns transparency into a competitive advantage.

To get started, head to: Carrot

Overview

Daily Slack and Email digests.

Local Setup

Prospective users of Carrot should get started by going to Carrot.io. The following local setup is for developers wanting to work on the OpenCompany Digest Service.

Most of the dependencies are internal, meaning Leiningen will handle getting them for you. There are a few exceptions:

Java - a Java 8+ JRE is needed to run Clojure
Leiningen v2.9.1+ - Clojure's build and dependency management tool
RethinkDB v2.3.6+ - a multi-modal (document, key/value, relational) open source NoSQL database

Java

Your system may already have Java 8+ installed. You can verify this with:

java -version

If you do not have Java 8+ download it and follow the installation instructions.

An option we recommend is OpenJDK. There are instructions for Linux and Homebrew can be used to install OpenJDK on a Mac with:

brew update && brew cask install adoptopenjdk8

Leiningen

Leiningen is easy to install:

Download the latest lein script from the stable branch.
Place it somewhere that's on your $PATH (env | grep PATH). /usr/local/bin is a good choice if it is on your PATH.
Set it to be executable. chmod 755 /usr/local/bin/lein
Run it: lein This will finish the installation.

Then let Leiningen install the rest of the dependencies:

git clone https://github.com/open-company/open-company-digest.git
cd open-company-digest
lein deps

RethinkDB

RethinkDB is easy to install with official and community supported packages for most operating systems.

RethinkDB for Mac OS X via Brew

Assuming you are running Mac OS X and are a Homebrew user, use brew to install RethinkDB:

brew update && brew install rethinkdb

If you already have RethinkDB installed via brew, check the version:

rethinkdb -v

If it's older, then upgrade it with:

brew update && brew upgrade rethinkdb && brew services restart rethinkdb

Follow the instructions provided by brew to run RethinkDB every time at login:

ln -sfv /usr/local/opt/rethinkdb/*.plist ~/Library/LaunchAgents

And to run RethinkDB now:

launchctl load ~/Library/LaunchAgents/homebrew.mxcl.rethinkdb.plist

Verify you can access the RethinkDB admin console:

open http://localhost:8080/

After installing with brew:

Your RethinkDB binary will be at /usr/local/bin/rethinkdb
Your RethinkDB data directory will be at /usr/local/var/rethinkdb
Your RethinkDB log will be at /usr/local/var/log/rethinkdb/rethinkdb.log
Your RethinkDB launchd file will be at ~/Library/LaunchAgents/homebrew.mxcl.rethinkdb.plist

RethinkDB for Mac OS X (Binary Package)

If you don't use brew, there is a binary installer package available for Mac OS X from the Mac download page.

After downloading the disk image, mounting it (double click) and running the rethinkdb.pkg installer, you need to manually create the data directory:

sudo mkdir -p /Library/RethinkDB
sudo chown <your-own-user-id> /Library/RethinkDB
mkdir /Library/RethinkDB/data

And you will need to manually create the launchd config file to run RethinkDB every time at login. From within this repo run:

cp ./opt/com.rethinkdb.server.plist ~/Library/LaunchAgents/com.rethinkdb.server.plist

And to run RethinkDB now:

launchctl load ~/Library/LaunchAgents/com.rethinkdb.server.plist

Verify you can access the RethinkDB admin console:

open http://localhost:8080/

After installing with the binary package:

Your RethinkDB binary will be at /usr/local/bin/rethinkdb
Your RethinkDB data directory will be at /Library/RethinkDB/data
Your RethinkDB log will be at /var/log/rethinkdb.log
Your RethinkDB launchd file will be at ~/Library/LaunchAgents/com.rethinkdb.server.plist

RethinkDB for Linux

If you run Linux on your development environment (good for you, hardcore!) you can get a package for you distribution or compile from source. Details are on the installation page.

RethinkDB for Windows

RethinkDB isn't supported on Windows directly. If you are stuck on Windows, you can run Linux in a virtualized environment to host RethinkDB.

Required Secrets

A secret is shared between the Storage service and the Digest service for creating and validating JSON Web Tokens.

Make sure you update the CHANGE-ME items in the section of the project.clj that looks like this to contain your actual JWT, Slack, and AWS secrets:

;; Dev environment and dependencies
:dev [:qa {
  :env ^:replace {
    :db-name "open_company_auth_dev"
    :open-company-auth-passphrase "this_is_a_dev_secret" ; JWT secret
    :hot-reload "true" ; reload code when changed on the file system
    :aws-access-key-id "CHANGE-ME"
    :aws-secret-access-key "CHANGE-ME"
    :log-level "debug"
  }

You can also override these settings with environmental variables in the form of OPEN_COMPANY_AUTH_PASSPHRASE and AWS_ACCESS_KEY_ID, etc. Use environmental variables to provide production secrets when running in production.

Usage

Prospective users of Carrot should get started by going to Carrot.io. The following usage is for developers wanting to work on the OpenCompany Digest Service.

Make sure you've updated project.clj as described above.

To start a production instance:

lein start!

Or to start a development instance:

lein start

Open your browser to http://localhost:3008/ping and check that it's working.

To clean all compiled files:

lein clean

To create a production build run:

lein build

To initiate a test digest run (nothing will be sent out):

lein dry-run

To initiate a digest run:

lein wet-run

To trigger a digest request, you can also use the following urls:

http://localhost:3008/_/email/run
http://localhost:3008/_/slack/run

Obviously you can replace the domain to run it on other environments.

Technical Design

The Digest Service is composed of 2 main responsibilities:

Send a daily email digests to users in a time appropriate for their timezone
Send a daily Slack digests to users in a time appropriate for their timezone

The Digest Service runs on an hourly schedule so that users get the daily digest in their own timezone, or a digest run can be invoked manually. It uses read-only access to the RethinkDB database of the Authorization Service to enumerate users. For each of those users, a digest request is generated for each organization the user has access to (most users just have 1 organization) by generating a JSON Web Token for that user, using the JWToken to request a date range of activity for the particular user and the particular org from the REST API of Storage Service, and then generating a JSON digest request, sent via SQS, to either the Email Service or Bot Service.

Testing

Tests are run in continuous integration of the master and mainline branches on Travis CI:

To run the tests locally:

lein test!

Participation

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.

License

Distributed under the GNU Affero General Public License Version 3.

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.