benshaw/github-sync-demo

Demo code that syncs data from the GitHub API

Github sync

Monitor stargazers of an org in GitHub

• Find all repos for an org and then all stargazers of those repos
• syncs all repos and stargazers locally if not already
• uses notificiations (via webhooks)
• utilize streams where possible

Requirements to run locally

• Postgres used for local storage

docker run --rm --name my_postgres -e POSTGRES_PASSWORD=password -v my_dbdata:/var/lib/postgresql/data -p 5432:5432 postgres:11 -c log_statement=all ```

• For GitHub web hooks to work the machine your running from need to be public, I use ngrok for this

ngrok http 8080

Setup

• Setup the Db using the SQL found in table.sql

Running the app

with env vars (GH_TOKEN, GH_URL, LOG_APP, LOG_CLI, HOST)

i.e.

 GH_TOKEN={GitHubToken} LOG_CLI=true LOG_APP=true HOST=https://483f0adf.ngrok.io sbt ~reStart

This will start the app on localhost with port 8080

Interact with the app

GET /org/{orgName}/starred

i.e.

 curl -v "localhost:8080/org/shawsolutions/starred"

Env Vars

• GH_TOKEN
  • The authentication token used when sending requests to GitHub
• GH_URL
  • The GitHub URL
• LOG_APP
  • Enable Application(requests to and from the application) logging (disabled by default)
• LOG_CLI
  • Enable Client (from the app to github) logging (disabled by default)
• HOST
  • Callback URL for webhooks

Tests

Unit Tests

sbt test

Design

• Http4s giter template
• Cats and Cats-Effect
• Doobie for Db and FS2 for streams
• Tagless Final pattern for core domain
• Inspiration from the scala petstore
• Higher kinded types
• Error handling with classy optics inspired by Gabriel Volpe

License

This project is licensed under the MIT License - see the LICENSE.md file for details