BOSH Google CPI release

This is a BOSH release for the external BOSH Google CPI.

Disclaimer

This is NOT presently a production ready BOSH Google CPI BOSH release. This is a work in progress. It is suitable for experimentation and may not become supported in the future.

Usage

I am assuming you are familiar with BOSH and its terminology. If not, please take a look at the BOSH documentation before running this procedure.

Setup the Google Cloud Platform environment

Sign up and activate Google Compute Engine, if you haven't already.
Download and Install the Google Cloud SDK command line tool.

Initialize the gcloud CLI if you haven't done so previously. Choose one of the following:

Use the CLI-guided walkthrough:
```
$ gcloud init
```

Manually initialize the CLI:

Create a service account and securely store the downloaded JSON Service Key.
Set the credentials:

# Set ENV:
export GOOGLE_APPLICATION_CREDENTIALS=<path/to/SERVICE_KEY.json>

# Or, use the CLI:
$ gcloud auth activate-service-account --key-file <path/to/SERVICE_KEY.json>

Set the project configuration (see Regions and Zones):

$ gcloud config set project <PROJECT_ID>
$ gcloud config set compute/region <REGION> # e.g., us-east1
$ gcloud config set compute/zone <ZONE>     # e.g., us-east1-b

Reserve a new static external IP address:
```
$ gcloud compute addresses create bosh
```
Create a new network with auto-created subnetwork ranges:
```
$ gcloud compute networks create cf --mode auto
```

Create the following firewalls and set the appropriate rules:

$ gcloud compute firewall-rules create cf-internal \
    --description "Cloud Foundry Internal traffic" \
    --network cf \
    --source-tags cf-internal \
    --target-tags cf-internal \
    --allow tcp,udp,icmp

$ gcloud compute firewall-rules create cf-bosh \
    --description "Cloud Foundry BOSH External traffic" \
    --network cf \
    --target-tags cf-bosh \
    --allow tcp:22,tcp:443,tcp:4222,tcp:6868,tcp:25250,tcp:25555,tcp:25777,udp:53

Create a password-less SSH key if you haven't already.

Install the bosh-init CLI

Install the bosh-init tool.

Create a deployment directory

Create a deployment directory to store all bosh-init artifacts:

$ mkdir google-bosh-deployment
$ cd google-bosh-deployment

Create a deployment manifest

Create a google-bosh-manifest.yml deployment manifest file inside the previously created deployment directory with the following content and update it with your properties:

---
name: bosh

releases:
  - name: bosh
    url: https://bosh.io/d/github.com/cloudfoundry/bosh?v=236
    sha1: 88dd60313dbd7dd832faa44c90493ffa6cd85448
  - name: bosh-google-cpi
    url: https://storage.googleapis.com/bosh-stemcells/bosh-google-cpi-12.tgz
    sha1: d022f5d3f95b5456ff2a9558721760c5a3f0c772

resource_pools:
  - name: vms
    network: private
    stemcell:
      url: https://storage.googleapis.com/bosh-stemcells/light-bosh-stemcell-3202-google-kvm-ubuntu-trusty-go_agent.tgz
      sha1: b59b6f51cd34b25a55a189d2b800ebb7608f9cc6
    cloud_properties:
      machine_type: n1-standard-4
      root_disk_size_gb: 40
      root_disk_type: pd-standard
      service_scopes:
        - compute
        - devstorage.full_control

disk_pools:
  - name: disks
    disk_size: 32_768
    cloud_properties:
      type: pd-standard

networks:
  - name: private
    type: dynamic
    cloud_properties:
      network_name: cf
      tags:
        - cf-internal
        - cf-bosh
  - name: public
    type: vip

jobs:
  - name: bosh
    instances: 1

    templates:
      - name: nats
        release: bosh
      - name: redis
        release: bosh
      - name: postgres
        release: bosh
      - name: powerdns
        release: bosh
      - name: blobstore
        release: bosh
      - name: director
        release: bosh
      - name: health_monitor
        release: bosh
      - name: registry
        release: bosh
      - name: google_cpi
        release: bosh-google-cpi

    resource_pool: vms
    persistent_disk_pool: disks

    networks:
      - name: private
        default:
          - dns
          - gateway
      - name: public
        static_ips:
          - __STATIC_IP__ # <--- Replace with the static IP

    properties:
      nats:
        address: 127.0.0.1
        user: nats
        password: nats-password

      redis:
        listen_address: 127.0.0.1
        address: 127.0.0.1
        password: redis-password

      postgres: &db
        listen_address: 127.0.0.1
        host: 127.0.0.1
        user: postgres
        password: postgres-password
        database: bosh
        adapter: postgres

      dns:
        address: __STATIC_IP__ # <--- Replace with the static IP
        domain_name: microbosh
        db: *db
        recursor: 8.8.8.8

      registry:
        address: __STATIC_IP__ # <--- Replace with the static IP
        host: __STATIC_IP__ # <--- Replace with the static IP
        db: *db
        http:
          user: registry
          password: registry-password
          port: 25777
        username: registry
        password: registry-password
        port: 25777

      blobstore:
        address: __STATIC_IP__ # <--- Replace with the static IP
        port: 25250
        provider: dav
        director:
          user: director
          password: director-password
        agent:
          user: agent
          password: agent-password

      director:
        address: 127.0.0.1
        name: micro-google
        db: *db
        cpi_job: google_cpi
        user_management:
          provider: local
          local:
            users:
              - name: admin
                password: admin
              - name: hm
                password: hm-password
      hm:
        director_account:
          user: hm
          password: hm-password
        resurrector_enabled: true

      google: &google_properties
        project: __GCE_PROJECT__ # <--- Replace with your GCE project
        default_zone: __GCE_DEFAULT_ZONE__ # <--- Replace with the GCE zone to use by default

      agent:
        mbus: nats://nats:nats-password@__STATIC_IP__:4222 # <--- Replace with the static IP
        ntp: *ntp
        blobstore:
           options:
             endpoint: http://__STATIC_IP__:25250 # <--- Replace with the static IP
             user: agent
             password: agent-password

      ntp: &ntp
        - 169.254.169.254

cloud_provider:
  template:
    name: google_cpi
    release: bosh-google-cpi

  ssh_tunnel:
    host: __STATIC_IP__ # <--- Replace with the static IP
    port: 22
    user: __SSH_USER__ # <--- Replace with the user corresponding to your private SSH key
    private_key: __PRIVATE_KEY_PATH__ # <--- Replace with the location of your google_compute_engine SSH private key

  mbus: https://mbus:mbus-password@__STATIC_IP__:6868 # <--- Replace with the static IP

  properties:
    google: *google_properties
    agent:
      mbus: https://mbus:mbus-password@0.0.0.0:6868
      blobstore:
        provider: local
        options:
          blobstore_path: /var/vcap/micro_bosh/data/cache
      ntp: *ntp

Deploy

Using the previously created deployment manifest, now we can deploy it:

$ bosh-init deploy google-bosh-manifest.yml

Install the BOSH CLI

Install the BOSH CLI tool in your workstation.

Then target your BOSH environment:

$ bosh target <YOUR BOSH IP ADDRESS>

Your username is admin and password is admin.

Deploy other software

Contributing

In the spirit of free software, everyone is encouraged to help improve this project.

Here are some ways you can contribute:

by using alpha, beta, and prerelease versions
by reporting bugs
by suggesting new features
by writing or editing documentation
by writing specifications
by writing code (no patch is too small: fix typos, add comments, clean up inconsistent whitespace)
by refactoring code
by closing issues
by reviewing patches

Submitting an Issue

We use the GitHub issue tracker to track bugs and features. Before submitting a bug report or feature request, check to make sure it hasn't already been submitted. You can indicate support for an existing issue by voting it up. When submitting a bug report, please include a Gist that includes a stack trace and any details that may be necessary to reproduce the bug, including your gem version, Ruby version, and operating system. Ideally, a bug report should include a pull request with failing specs.

Submitting a Pull Request

Fork the project.
Create a topic branch.
Implement your feature or bug fix.
Commit and push your changes.
Submit a pull request.

Create new release

Creating a final release

If you need to create a new final release, you will need to get read/write API credentials to the @cloudfoundry-community s3 account.

Please email Dr Nic Williams and he will create unique API credentials for you.

Create a config/private.yml file with the following contents:

---
blobstore:
  s3:
    access_key_id:     ACCESS
    secret_access_key: PRIVATE

You can now create final releases for everyone to enjoy!

bosh create release
# test this dev release
git commit -m "updated BOSH Google CPI release"
bosh create release --final
git commit -m "creating vXYZ release"
git tag vXYZ
git push origin master --tags

ljfranklin/bosh-google-cpi-boshrelease