Replicaset recipe fails with "undefined method '>' for nil:NilClass"

[kplimack]

Recipe: mongodb::replicaset

* ruby_block[config_replicaset] action create

Error executing action create on resource 'ruby_block[config_replicaset]'

NoMethodError

undefined method `>' for nil:NilClass

Cookbook Trace:

/var/chef/cache/cookbooks/mongodb/libraries/mongodb.rb:62:in block in configure_replicaset' /var/chef/cache/cookbooks/mongodb/libraries/mongodb.rb:55:ineach_index'
/var/chef/cache/cookbooks/mongodb/libraries/mongodb.rb:55:in configure_replicaset' /var/chef/cache/cookbooks/mongodb/definitions/mongodb.rb:221:inblock (3 levels) in from_file'

Resource Declaration:

In /var/chef/cache/cookbooks/mongodb/definitions/mongodb.rb

219: ruby_block 'config_replicaset' do
220: block do
221: MongoDB.configure_replicaset(new_resource.replicaset, replicaset_name, rs_nodes) unless new_resource.replicaset.nil?
222: end
223: action :nothing
224: end
225:

Compiled Resource:

Declared in /var/chef/cache/cookbooks/mongodb/definitions/mongodb.rb:219:in `block in from_file'

ruby_block("config_replicaset") do
params {:mongodb_type=>"mongod", :action=>[:enable, :start], :logpath=>nil, :dbpath=>"/data/mongodb", :configservers=>[], :replicaset=>node[use1d-mongo101.sm], :notifies=>[], :port=>27017, :enable_rest=>nil, :smallfiles=>nil, :name=>"mongodb"}
action [:nothing]
retries 0
retry_delay 2
guard_interpreter :default
block_name "config_replicaset"
cookbook_name "mongodb"
recipe_name "replicaset"
block #Proc:0x0000000449ada0@/var/chef/cache/cookbooks/mongodb/definitions/mongodb.rb:220
end

Running handlers:
[2014-09-09T20:03:51+00:00] ERROR: Running exception handlers
Running handlers complete

[2014-09-09T20:03:51+00:00] ERROR: Exception handlers complete
[2014-09-09T20:03:51+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
Chef Client failed. 10 resources updated in 20.815626112 seconds
[2014-09-09T20:03:51+00:00] ERROR: ruby_block[config_replicaset](mongodb::replicaset line 219) had an error: NoMethodError: undefined method `>' for nil:NilClass
[2014-09-09T20:03:51+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

---

newresource when config_replicaset is called:
OpenStruct name="mongodb", dbpath="/data/mongodb", logpath=nil, replicaset=node[use1d-mongo101.sm], service_action=[:enable, :start], service_notifies=[], auto_configure_replicaset=true, auto_configure_sharding=false, bind_ip="0.0.0.0", cluster_name="clusterface1", config={"port"=>27017, "bind_ip"=>"0.0.0.0", "dbpath"=>"/data/mongodb"}, dbconfig_file="/etc/mongodb.conf", dbconfig_file_template="mongodb.conf.erb", init_dir="/etc/init.d", init_script_template="debian-mongodb.init.erb", is_replicaset=true, is_shard=nil, is_configserver=nil, is_mongos=nil, mongodb_group=nil, mongodb_user=nil, replicaset_name=nil, port=27017, root_group=nil, shard_name="defaultShard", sharded_collections=nil, sysconfig_file="/etc/default/mongodb", sysconfig_file_template="mongodb.sysconfig.erb", sysconfig_vars={"DAEMON"=>"/usr/bin/$NAME", "DAEMON_USER"=>nil, "DAEMON_OPTS"=>"--config /etc/mongodb.conf", "CONFIGFILE"=>"/etc/mongodb.conf", "ENABLE_MONGODB"=>"yes", "DAEMONUSER"=>nil, "ENABLE_MONGOD"=>"yes", "ENABLE_MONGO"=>"yes"}, template_cookbook=nil, ulimit={"fsize"=>"unlimited", "cpu"=>"unlimited", "as"=>"unlimited", "nofile"=>64000, "rss"=>"unlimited", "nproc"=>32000}, reload_action=nil, init_file="/etc/init.d/mongodb">
nr.replicaset: node[use1d-mongo101.sm]

[cjhubert]

Are you using chef-solo or chef-server?

[srathbun]

I ran into the same problem using chef-server. It seems to be an issue with how I'm starting all of my nodes. The sharding+replication wiki page leaves something to be desired. I've configured each node with the roles and options needed, but how do I stand them up together?

I have just been kicking off a mass chef-client command on all of them together, but that fails now that I have more nodes.

I can start the config server nodes by themselves, but then I get the above error when I attempt to start the mongos and the shards+replicas.

[cjhubert]

The reason I asked chef-solo or chef-server is because it appears to be error'ing out on this line:

rs_options[host]['slaveDelay'] = slave_delay if slave_delay > 0

Which sounds like slave_delay is currently nil. That should be being read from the search() of node variables. On chef-server I think this would work without issue but on chef-solo you have to use data_bags for search() to work. I think I ran into this exact issue when I first started using data_bags, they can be pretty difficult to work with.

@srathbun, I might not be able to help with chef-server issues, but what errors are you getting?

[srathbun]

@ceejh It varies between the one up above, and undefined method [] for nil:NilClass when attempting to run the config_sharding block. It seems that the search is running prematurely, and not populating the shard_nodes variable with any contents, since the shards do not exist yet or something.

Since I started using chef yesterday, it may be an issue with how I'm kicking off the build across the cluster. As far as I know, I should just be able to create the roles listed on the sharding+replication wiki page.

Once I've got the roles, and my nodes have been assigned, I run knife ssh 'cluster_name:foo' 'chef-client' to kick them all off. Is this not the chef way to provision a set of servers? Or is there some chef docs I missed that would explain it?

[srathbun]

Ok, so it is my fault. To whom it may concern, it's important to start from a pristine set of nodes. Configuring a cluster to use one configserver and then trying to add two more, does not work. The mongos refuses to start, and then the ruby code throws errors, which were red herrings.

Make sure that your nodes do not have mongo running, or old database files laying about. Then, make sure to start your config servers, followed by the shards, followed by the mongos. Starting them all together fails, because the mongos quit when the config servers are unreachable, and that throws off the shards.

[cjhubert]

Thanks for posting your findings, glad you were able to get it sorted out, and sorry I wasn't any help (definitely didn't have any ideas myself). Yea, adding config servers seems to be difficult from what I've google'd/read. Handling it automatically through the cookbook might be a pretty tall order.