Issue while bosh-init on AWS

[Senthilannaswamy]

I'm trying to bosh-init, by following below reference.

http://bosh.io/docs/init-aws.html

While running "bosh-init deploy ./bosh.yml", getting error like, (please note, have tried Uncommenting "properties:nats:password", still same issue). Please suggest.

Command 'deploy' failed:
  Installing CPI:
    Rendering and uploading Jobs:
      Rendering job templates for installation:
        Rendering templates for job 'aws_cpi/2a34fab07ff9a36e43ede22e227c02918a5ccdef':
          Rendering template src: cpi.json.erb, dst: config/cpi.json:
            Rendering template src: /home/ec2-user/.bosh_init/installations/e10f34eb-3e4d-4377-5320-132e9ec3054a/tmp/bosh-init-release942259122/extracted_jobs/aws_cpi/templates/cpi.json.erb, dst: /home/ec2-user/.bosh_init/installations/e10f34eb-3e4d-4377-5320-132e9ec3054a/tmp/rendered-jobs471613702/config/cpi.json:
              Running ruby to render templates:
                Running command: 'ruby /home/ec2-user/.bosh_init/installations/e10f34eb-3e4d-4377-5320-132e9ec3054a/tmp/erb-renderer232054824/erb-render.rb /home/ec2-user/.bosh_init/installations/e10f34eb-3e4d-4377-5320-132e9ec3054a/tmp/erb-renderer232054824/erb-context.json /home/ec2-user/.bosh_init/installations/e10f34eb-3e4d-4377-5320-132e9ec3054a/tmp/bosh-init-release942259122/extracted_jobs/aws_cpi/templates/cpi.json.erb /home/ec2-user/.bosh_init/installations/e10f34eb-3e4d-4377-5320-132e9ec3054a/tmp/rendered-jobs471613702/config/cpi.json', stdout: '', stderr: '/home/ec2-user/.bosh_init/installations/e10f34eb-3e4d-4377-5320-132e9ec3054a/tmp/erb-renderer232054824/erb-render.rb:189:in `rescue in render': Error filling in template '/home/ec2-user/.bosh_init/installations/e10f34eb-3e4d-4377-5320-132e9ec3054a/tmp/bosh-init-release942259122/extracted_jobs/aws_cpi/templates/cpi.json.erb' for aws_cpi/0 (line 92: #<TemplateEvaluationContext::UnknownProperty: Can't find property 'nats.password'>) (RuntimeError)
        from /home/ec2-user/.bosh_init/installations/e10f34eb-3e4d-4377-5320-132e9ec3054a/tmp/erb-renderer232054824/erb-render.rb:175:in `render'
        from /home/ec2-user/.bosh_init/installations/e10f34eb-3e4d-4377-5320-132e9ec3054a/tmp/erb-renderer232054824/erb-render.rb:200:in `<main>'

Deployment manifest file was changed only for following properties,

1. ELASTIC-IP,
2. SUBNET-ID,
3. REGION (e.g. us-east-1),
4. AVAILABILITY-ZONE (e.g. us-east-1a),
5. ACCESS-KEY-ID,
6. SECRET-ACCESS-KEY .

---
name: bosh

releases:
- name: bosh
  url: https://bosh.io/d/github.com/cloudfoundry/bosh?v=261.2
  sha1: d4635b4b82b0dc5fd083b83eb7e7405832f6654b
- name: bosh-aws-cpi
  url: https://bosh.io/d/github.com/cloudfoundry-incubator/bosh-aws-cpi-release?v=62
  sha1: f36967927ceae09e5663a41fdda199edfe649dc6

resource_pools:
- name: vms
  network: private
  stemcell:
    url: https://bosh.io/d/stemcells/bosh-aws-xen-hvm-ubuntu-trusty-go_agent?v=3363.9
    sha1: c5a61384178efaade80210de7ebf994d70e6ca25
  cloud_properties:
    instance_type: m3.xlarge
    ephemeral_disk: {size: 25_000, type: gp2}
    availability_zone: us-east-1b # <--- Replace with Availability Zone

disk_pools:
- name: disks
  disk_size: 20_000
  cloud_properties: {type: gp2}

networks:
- name: private
  type: manual
  subnets:
  - range: 10.11.0.0/24
    gateway: 10.11.0.1
    dns: [10.11.0.2]
    cloud_properties: {subnet: subnet-f07640dd} # <--- Replace with Subnet ID
- name: public
  type: vip

jobs:
- name: bosh
  instances: 1

  templates:
  - {name: nats, release: bosh}
  - {name: postgres, release: bosh}
  - {name: blobstore, release: bosh}
  - {name: director, release: bosh}
  - {name: health_monitor, release: bosh}
  - {name: registry, release: bosh}
  - {name: aws_cpi, release: bosh-aws-cpi}

  resource_pool: vms
  persistent_disk_pool: disks

  networks:
  - name: private
    static_ips: [10.11.0.6]
    default: [dns, gateway]
  - name: public
    static_ips: [34.192.209.204] # <--- Replace with Elastic IP

  properties:
    nats:
      address: 127.0.0.1
      user: nats
      # password: nats-password # <--- Uncomment & change

    postgres: &db
      listen_address: 127.0.0.1
      host: 127.0.0.1
      user: postgres
      # password: postgres-password # <--- Uncomment & change
      database: bosh
      adapter: postgres

    registry:
      address: 10.11.0.6
      host: 10.11.0.6
      db: *db
      http:
        user: admin
        # password: admin # <--- Uncomment & change
        port: 25777
      username: admin
      # password: admin # <--- Uncomment & change
      port: 25777

    blobstore:
      address: 10.11.0.6
      port: 25250
      provider: dav
      director:
        user: director
        # password: director-password # <--- Uncomment & change
      agent:
        user: agent
        # password: agent-password # <--- Uncomment & change

    director:
      address: 127.0.0.1
      name: my-bosh
      db: *db
      cpi_job: aws_cpi
      max_threads: 10
      user_management:
        provider: local
        local:
          users:
          # - {name: admin, password: admin} # <--- Uncomment & change
          # - {name: hm, password: hm-password} # <--- Uncomment & change

    hm:
      director_account:
        user: hm
        # password: hm-password # <--- Uncomment & change
      resurrector_enabled: true

    aws: &aws
      access_key_id: {{redacted}} # <--- Replace with AWS Access Key ID
      secret_access_key: {{redacted}} # <--- Replace with AWS Secret Key
      default_key_name: bosh
      default_security_groups: [offshore-bosh-sg]
      region: us-east-1  # <--- Replace with Region

    # agent: {mbus: "nats://nats:nats-password@10.0.0.6:4222"} # <--- Uncomment & change

    ntp: &ntp [0.pool.ntp.org, 1.pool.ntp.org]

cloud_provider:
  template: {name: aws_cpi, release: bosh-aws-cpi}

  ssh_tunnel:
    host: 34.192.209.204 # <--- Replace with your Elastic IP address
    port: 22
    user: vcap
    private_key: ./bosh.pem # Path relative to this manifest file

  # mbus: "https://mbus:mbus-password@34.192.209.204:6868" # <--- Uncomment & change

  properties:
    aws: *aws
    # agent: {mbus: "https://mbus:mbus-password@0.0.0.0:6868"} # <--- Uncomment & change
    blobstore: {provider: local, path: /var/vcap/micro_bosh/data/cache}
    ntp: *ntp

[dpb587-pivotal]

Please take note of the "Uncomment & change" sections and make sure you have uncommented them and provided non-default passwords for them - they're required for deployment.

Also, please rotate your access key and secret access key ending in LIQA - it should have been redacted and has now been emailed out to watchers of this repo.

[jacko1118]

dear all.

I got the same error and stuck there even I have uncomment and put a password.

Could you pls help.
Thanks.

Command 'deploy' failed:
Installing CPI:
Rendering and uploading Jobs:
Rendering job templates for installation:
Rendering templates for job 'vsphere_cpi/677eb08a995a5bfde2ad0ed9c4ec8c0914272c47':
Rendering template src: cpi.json.erb, dst: config/cpi.json:
Rendering template src: /root/.bosh_init/installations/afc81326-c887-4809-5b11-dd90070b0422/tmp/bosh-init-release722782197/extracted_jobs/vsphere_cpi/templates/cpi.json.erb, dst: /root/.bosh_init/installations/afc81326-c887-4809-5b11-dd90070b0422/tmp/rendered-jobs717219588/config/cpi.json:
Running ruby to render templates:
Running command: 'ruby /root/.bosh_init/installations/afc81326-c887-4809-5b11-dd90070b0422/tmp/erb-renderer631514685/erb-render.rb /root/.bosh_init/installations/afc81326-c887-4809-5b11-dd90070b0422/tmp/erb-renderer631514685/erb-context.json /root/.bosh_init/installations/afc81326-c887-4809-5b11-dd90070b0422/tmp/bosh-init-release722782197/extracted_jobs/vsphere_cpi/templates/cpi.json.erb /root/.bosh_init/installations/afc81326-c887-4809-5b11-dd90070b0422/tmp/rendered-jobs717219588/config/cpi.json', stdout: '', stderr: '/root/.bosh_init/installations/afc81326-c887-4809-5b11-dd90070b0422/tmp/erb-renderer631514685/erb-render.rb:189:in rescue in render': Error filling in template '/root/.bosh_init/installations/afc81326-c887-4809-5b11-dd90070b0422/tmp/bosh-init-release722782197/extracted_jobs/vsphere_cpi/templates/cpi.json.erb' for vsphere_cpi/0 (line 112: #<TemplateEvaluationContext::UnknownProperty: Can't find property 'nats.password'>) (RuntimeError) from /root/.bosh_init/installations/afc81326-c887-4809-5b11-dd90070b0422/tmp/erb-renderer631514685/erb-render.rb:175:in render'
from /root/.bosh_init/installations/afc81326-c887-4809-5b11-dd90070b0422/tmp/erb-renderer631514685/erb-render.rb:200:in `

'
':
exit status 1
root@ubuntu6:~/my-bosh#

[Senthilannaswamy]

HI Jacko1118,

Can you share the bosh.yml?

Regards
Senthil.

[jacko1118]

---

name: bosh

releases:

• name: bosh
  url: https://bosh.io/d/github.com/cloudfoundry/bosh?v=261.4
  sha1: 4da9cedbcc8fbf11378ef439fb89de08300ad091
• name: bosh-vsphere-cpi
  url: https://bosh.io/d/github.com/cloudfoundry-incubator/bosh-vsphere-cpi-release?v=39
  sha1: 2b68e45b553c3867fc4f7021a723eae7ec3c23fd

resource_pools:

• name: vms
  network: private
  stemcell:
  url: https://bosh.io/d/stemcells/bosh-vsphere-esxi-ubuntu-trusty-go_agent?v=3363.15
  sha1: d5571cd8e13d1daca99dc821e4fb751f4cdd42f8
  cloud_properties:
  cpu: 2
  ram: 4_096
  disk: 20_000
  env:
  bosh:
  # c1oudc0w is a default password for vcap user
  password: "$6$4gDD3aV0rdqlrKC$2axHCxGKIObs6tAmMTqYCspcdvQXh3JJcvWOY2WGb4SrdXtnCyNaWlrf3WEqvYR2MYizEGp3kMmbpwBC6jsHt0"

disk_pools:

• name: disks
  disk_size: 20_000

networks:

• name: private
  type: manual
  subnets:
  • range: 10.1.1.140/24
    gateway: 10.1.1.100
    dns: [10.1.1.100]
    cloud_properties: {name: VM-Network } # <--- Replace with Network name

jobs:

• name: bosh
  instances: 1

  templates:

  • {name: nats, release: bosh}
  • {name: postgres, release: bosh}
  • {name: blobstore, release: bosh}
  • {name: director, release: bosh}
  • {name: health_monitor, release: bosh}
  • {name: vsphere_cpi, release: bosh-vsphere-cpi}

resource_pool: vms
persistent_disk_pool: disks

networks:

• {name: private, static_ips: [10.1.1.140]}

properties:
nats:
address: 10.1.1.141
user: nats
password: "Test1111q" # <--- Uncomment & change

postgres: &db
  listen_address: 10.1.1.142
  host: 127.0.0.1
  user: PoStGrEs
   password: pOsTgReS  # <--- Uncomment & change
  database: bosh
  adapter: postgres

blobstore:
  address: 10.1.1.145
  port: 25250
  provider: dav
  director:
    user: DiReCtOr
     password: dIrEcToR # <--- Uncomment & change
  agent:
    user: AgEnT
     password: aGeNt # <--- Uncomment & change

director:
  address: 10.1.1.144
  name: my-bosh
  db: *db
  cpi_job: vsphere_cpi
  user_management:
    provider: local
    local:
      users:
       - {name: admin, password: admin} # <--- Uncomment & change
       - {name: AdMiN, password: pAsSwOrD} # <--- Uncomment & change

hm:
  director_account:
    user: AdMiN
    password: pAsSwOrD # <--- Uncomment & change
  resurrector_enabled: true
  intervals:
    agent_timeout: 180

vcenter: &vcenter # <--- Replace values below
address: 10.1.1.120
user: administrator@vsphere.local
password: Hugo@8888
datacenters:
- name: sodec_cf
vm_folder: vm_folder
template_folder: template_folder_bosh
datastore_pattern: datastore1
persistent_datastore_pattern: datastore1
disk_path: my-bosh-disks
clusters: [test_cluster]

 agent: {mbus: "nats://nats:test11@10.0.0.6:4222"} # <--- Uncomment & change

ntp: &ntp [0.pool.ntp.org, 1.pool.ntp.org]

cloud_provider:
template: {name: vsphere_cpi, release: bosh-vsphere-cpi}

mbus: "https://mbus:mbus-password@10.0.0.6:6868" # <--- Uncomment & change

properties:
vcenter: *vcenter
agent: {mbus: "https://mbus:mbus-password@0.0.0.0:6868"} # <--- Uncomment & change
blobstore: {provider: local, path: /var/vcap/micro_bosh/data/cache}
ntp: *ntp

[jacko1118]

dear Senthil,

I got the same error if let the password field uncomment.

If I change the field , I got below error:

root@ubuntu6:~/my-bosh# bosh-init deploy ./bosh.yml
Deployment manifest: '/root/my-bosh/bosh.yml'
Deployment state: '/root/my-bosh/bosh-state.json'

Started validating
Failed validating (00:00:00)

Command 'deploy' failed:
Parsing release set manifest '/root/my-bosh/bosh.yml':
Unmarshalling release set manifest:
yaml: line 61: mapping values are not allowed in this context
root@ubuntu6:~/my-bosh#

[jacko1118]

templates:

• {name: nats, release: bosh}
• {name: postgres, release: bosh}
• {name: blobstore, release: bosh}
• {name: director, release: bosh}
• {name: health_monitor, release: bosh}
• {name: vsphere_cpi, release: bosh-vsphere-cpi}

resource_pool: vms
persistent_disk_pool: disks

networks:

• {name: private, static_ips: [10.1.1.140]}

properties:
nats:
address: 10.1.1.141
user: natsjacko
password: Test1111q # <--- Uncomment & change

                                                                                61,1          35%

Command 'deploy' failed:
Parsing release set manifest '/root/my-bosh/bosh.yml':
Unmarshalling release set manifest:
yaml: line 61: mapping values are not allowed in this context

you can see the log detect my error in line 61 which is the line for nats user.
I am stucking here. could you pls help?

[Senthilannaswamy]

HI jacko1118,

Its looks like intent issue, can you attach bosh.yml as file (you need to change the extension of the file).
-Senthil

[jacko1118]

hi Senthil,
finally I success to deploy the bosh director with below bosh.yml.
the format of the bosh.yml is very strictly to control, even more than one space I would get the error

thanks.

[drnic]

If it helps, here's a blog post on deploying bosh to AWS using bosh create-env + new bosh-deployment repo (which are replacing bosh-init) https://www.starkandwayne.com/blog/bootstrap-bosh-2-0-on-aws/

[dpb587-pivotal]

Senthil, thanks for helping out others here! Were you able to resolve your original issue that this can be closed?

[Senthilannaswamy]

HI There,

Yes, you can consider as closed.

Senthil.