connectordb: A Go repository from migsc

A repository for all of your quantified-self data, and a control center for your IoT devices.

There already exist many apps and fitness trackers that gather and attempt to make sense of your data. Most of these services are isolated - your phone's fitness tracking software knows nothing about your browser's time-tracking extension. Furthermore, each app and service has its own method for downloading data (if they offer raw data at all!), which makes an all-encompassing analysis of life extremely tedious. ConnectorDB offers a self-hosted open-source alternative to these services. It allows every device you have to synchronize with one central database, which allows creating an in-depth picture of your life.

Please visit the website for more information.

Gather Data

This repository is just the backend server - to automatically gather data, you'll want to run the apps on your devices, which will automatically sync to ConnectorDB:

ConnectorDB Desktop - an app that gathers data about your computer usage (such as keypresses and active application). It is included by default in the desktop version of ConnectorDB.
Android App - Gathers several metrics from your android phone in the background, including step count, location, and current activity.
Chrome Extension - Gathers your web browsing history, allowing you to keep track of how much time you spend on various websites.
Python API - gives full access to ConnectorDB, and allows you to write your own data-gathering apps (such as integrating custom sensors and devices), as well as data-analysis or IoT apps, which make use of your data (make your thermostat turn on in response to your phone's location).

Rate And Plot Your Life

ConnectorDB also has built-in support for manual data input - the server's frontend allows inputting any type of data, and has special support for star ratings - such as ratings of mood or productivity. The frontend also automatically plots various types of data.

Installing

Installation instructions, and precompiled binaries are are available on the website.

Development builds are available here. These are usually direct builds of master, and as such might be less stable.

Building

To perform a full build of ConnectorDB you will need a linux machine, preferably arch or a recent version of ubuntu, and corresponding cross-compilers.

While technically builds can be performed on windows/OSX/Raspberry Pi, they are not officially supported. OSX/Raspberry Pi builds should "just work", with the following caveats:

Raspbian has an old version of Redis, so you will need to download the source of Redis >3.0 and compile, putting redis-server and redis-cli in bin/dep folder. Look at the arm portion of makerelease for specific instructions.
Windows has issues setting up NPM. If building on Windows, it is recommended that you use a precompiled version of the frontend from here, put in bin/app.

Compile

You must have redis and postgres installed (and not running). ConnectorDB also requires golang 1.7.

To initialize the subrepositories and install all necessary dependencies, run:

make deps

To build ConnectorDB, you need to add the connectordb directory to your gopath. After doing so, the following will create all necessary binaries:

make

At that point, binaries are located in bin. Using the binaries in this folder, you can continue from the setup tutorial.

Note: On ubuntu your build might fail on npm step. This is because node is installed as nodejs. sudo ln -s /usr/bin/nodejs /usr/bin/node should fix the issue.

Update

A git pull is not enough to update ConnectorDB, since submodules are used. You will need to re-run make deps to update the necessary dependencies.

make clean
git pull
make deps
make

To make sure that the newest version of the frontend is used, you will need to log out and log back in (in the browser).

Quick Recompile

When testing modifications to the server, you can recompile just the connectordb binary by running:

make bin/connectordb

Test

This will run all tests, spawning the necessary servers in the process (make sure you don't have any running connectordb instances):

make test

Note that this must be run after build is completed.

Debug

Sometimes you might want to see exactly what is going on with the server. To do so, start ConnectorDB in extremely verbose mode:

connectordb start mydatabase -l=DEBUG --vvv

The --vvv command will log the request and response content for each query to the server. When posting server logs in bug reports, please make sure your logs are in extremely verbose mode, so we know exactly what is going on.

Note: Do not run ConnectorDB in extremely verbose mode by default - it is meant only for debugging.

migsc/connectordb