GitHub Multi-Repo Management Microservices

This is sample code for a OSCON 2019 tutorial: https://conferences.oreilly.com/oscon/oscon-or/public/schedule/detail/76039

Please find the corresponding slides here

Pre-requisites

If installing using a thumb drive

Install the binaries in the appropriate folder osx, windows, linux

Docker images are also included in the docker-images folder so you don't need to download them. Move them to your local machine and then install them like so.

cd DIRECTORY_WHERE_YOU_COPIED_THE_DOCKER_IMAGES
docker load -i python.tar
docker load -i nginx.tar

If installing from the Internet

Virtual Box
Docker
- OS X
- Windows
- Linux x86
MiniKube
- OS X
- Linux
- Windows
Kompose
- OS X
- Linux AMD
- Linux ARM
- Windows
Clone GitHub Microservices

Installation

Update .env_example.
Update examples/config.example.

mv ./.env_example ./.env
mv ./examples/config.example ./examples/config.py
source .env
docker-machine create -d virtualbox github-manager-microservices
eval "$(docker-machine env github-manager-microservices)"
export GITHUB_MANAGER_MICROSERVICES_IP="$(docker-machine ip github-manager-microservices)"
docker-compose -f docker-compose.yml up -d --build

Installation (PowerShell)

mv ./.env_example ./.env
mv ./examples/config.example ./examples/config.py

# Load the environment variables in .env
foreach ($line in (Get-Content ./.env -ErrorAction Stop)) {
    if ($line.StartsWith("#")) { continue }
    if ($line.Trim()) {
        $formatted = $line.Replace("'","").Replace("export ","")
        $kvp = $formatted -split "=",2
        [Environment]::SetEnvironmentVariable($kvp[0], $kvp[1])
    }
}

docker-machine create -d virtualbox github-manager-microservices
& docker-machine env github-manager-microservices | Invoke-Expression
$env:GITHUB_MANAGER_MICROSERVICES_IP = docker-machine ip github-manager-microservices
docker-compose -f docker-compose.yml up -d --build

Quickstart

Go to http://<$GITHUB_MANAGER_MICROSERVICES_IP> in your browser. If you see "There are a total...", you're good to go.

python3 -m venv venv
source ./venv/bin/activate
pip install -r services/web/requirements.txt
source .env
export GITHUB_MANAGER_MICROSERVICES_IP="$(docker-machine ip github-manager-microservices)"
python examples/get_all_issues.py
python examples/get_all_prs.py
python examples/send_email.py
python examples/send_sms.py

Local Kubernetes Setup

Update kubernetes/secret.example.

minikube start
mv ./kubernetes/secret.example ./kubernetes/secret.yml
kubectl apply -f ./kubernetes/secret.yml
minikube dashboard // Verify secrets are deployed
minikube addons enable ingress
docker login
docker build -t <YOUR DOCKER HUB USERNAME>/github ./services/github
docker push <YOUR DOCKER HUB USERNAME>/github
docker build -t <YOUR DOCKER HUB USERNAME>/sms ./services/sms
docker push <YOUR DOCKER HUB USERNAME>/sms
docker build -t <YOUR DOCKER HUB USERNAME>/email ./services/email
docker push <YOUR DOCKER HUB USERNAME>/email
docker build -t <YOUR DOCKER HUB USERNAME>/web ./services/web
docker push <YOUR DOCKER HUB USERNAME>/web
kubectl create -f ./kubernetes/github-deploy.yml
kubectl create -f ./kubernetes/github-service.yml
kubectl create -f ./kubernetes/sms-deploy.yml
kubectl create -f ./kubernetes/sms-service.yml
kubectl create -f ./kubernetes/email-deploy.yml
kubectl create -f ./kubernetes/email-service.yml
kubectl create -f ./kubernetes/web-deploy.yml
kubectl create -f ./kubernetes/web-service.yml
kubectl apply -f ./kubernetes/minikube-ingress.yml
echo "$(minikube ip) hello.world" | sudo tee -a /etc/hosts // Add minikube ip to /etc/hosts
export GITHUB_MANAGER_MICROSERVICES_IP="hello.world"
python examples/get_all_issues.py
python examples/get_all_prs.py
python examples/send_email.py
python examples/send_sms.py

Troubleshooting

kubectl

When installing kubectl using brew you might run into schema errors. You can fix this by relinking.

rm /usr/local/bin/kubectl
brew link --overwrite kubernetes-cli

And also optionally:

brew link --overwrite --dry-run kubernetes-cli.

IP Connection

If you are unable to connect to http://<$GITHUB_MANAGER_MICROSERVICES_IP>, try regenerating the certs:

docker-machine regenerate-certs -f github-manager-microservices

thinkingserious/github-microservices