Debian installation on mco > 2.8 removes plugins path from libdir
luksi1 opened this issue · 5 comments
luksi1 commented
Mcollective 1.11.1
Puppet 3.8
OS: Ubuntu 16.04
According to the docs, the way I understand them, the plugins directory needs to be retained. The core lib needs can be taken away after 2.8. The problem with the following code is that you remove the plugins directory and only retain the core lib directory.
libdir = /usr/local/share/mcollective
when it should be
libdir = /usr/share/mcollective/plugins
if versioncmp($::puppetversion, '4') < 0 {
$confdir = '/etc/mcollective'
$_core_libdir = $::osfamily ? {
'Debian' => '/usr/share/mcollective/plugins',
'OpenBSD' => '/usr/local/libexec/mcollective',
default => '/usr/libexec/mcollective',
}
# Where this module will sync file-managed plugins to.
# These paths may need revisiting by someone who understands FHS and
# distribution standards for site-specific application-specific
# library paths.
$site_libdir = $::osfamily ? {
'Debian' => '/usr/local/share/mcollective',
'OpenBSD' => regsubst($::rubyversion, '^(\d+)\.(\d+)\.(\d+)$', '/usr/local/lib/ruby/vendor_ruby/\1.\2/mcollective'),
default => '/usr/local/libexec/mcollective',
}
} else {
$confdir = '/etc/puppetlabs/mcollective'
$_core_libdir = '/opt/puppetlabs/mcollective/plugins'
$site_libdir = '/opt/puppetlabs/mcollective'
}
# Since mcollective version 2.8, there is no core libdir
# https://docs.puppetlabs.com/mcollective/releasenotes.html#libdirloadpath-changes-and-core-plugins
$mco_assumed_version = '2.8.5'
$_mco_version = pick_default($::mco_version, $mco_assumed_version)
if versioncmp($_mco_version, '2.8') >= 0 {
$core_libdir = undef
} else {
$core_libdir = $_core_libdir
}
I think the code should be something like this.
$_mco_version = pick_default($::mco_version, $mco_assumed_version)
if versioncmp($_mco_version, '2.8') >= 0 {
$site_libdir = undef
}
luksi1 commented
Here is how mcollective-puppet-agent looks on Ubuntu 16.04
# dpkg -L mcollective-puppet-agent
/.
/usr
/usr/share
/usr/share/doc
/usr/share/doc/mcollective-puppet-agent
/usr/share/doc/mcollective-puppet-agent/copyright
/usr/share/doc/mcollective-puppet-agent/changelog.Debian.gz
/usr/share/mcollective
/usr/share/mcollective/plugins
/usr/share/mcollective/plugins/mcollective
/usr/share/mcollective/plugins/mcollective/agent
/usr/share/mcollective/plugins/mcollective/agent/puppet.rb
/usr/local/share/mcollective though is gone from mcollective-common
# dpkg -L mcollective-common
/.
/usr
/usr/lib
/usr/lib/ruby
/usr/lib/ruby/vendor_ruby
/usr/lib/ruby/vendor_ruby/mcollective
/usr/lib/ruby/vendor_ruby/mcollective/rpc.rb
/usr/lib/ruby/vendor_ruby/mcollective/monkey_patches.rb
/usr/lib/ruby/vendor_ruby/mcollective/connector.rb
/usr/lib/ruby/vendor_ruby/mcollective/pluginmanager.rb
/usr/lib/ruby/vendor_ruby/mcollective/client.rb
/usr/lib/ruby/vendor_ruby/mcollective/rpc
/usr/lib/ruby/vendor_ruby/mcollective/rpc/helpers.rb
/usr/lib/ruby/vendor_ruby/mcollective/rpc/client.rb
/usr/lib/ruby/vendor_ruby/mcollective/rpc/progress.rb
/usr/lib/ruby/vendor_ruby/mcollective/rpc/reply.rb
/usr/lib/ruby/vendor_ruby/mcollective/rpc/audit.rb
/usr/lib/ruby/vendor_ruby/mcollective/rpc/result.rb
/usr/lib/ruby/vendor_ruby/mcollective/rpc/agent.rb
/usr/lib/ruby/vendor_ruby/mcollective/rpc/request.rb
/usr/lib/ruby/vendor_ruby/mcollective/rpc/stats.rb
/usr/lib/ruby/vendor_ruby/mcollective/rpc/actionrunner.rb
/usr/lib/ruby/vendor_ruby/mcollective/facts
/usr/lib/ruby/vendor_ruby/mcollective/facts/yaml_facts.rb
/usr/lib/ruby/vendor_ruby/mcollective/facts/base.rb
/usr/lib/ruby/vendor_ruby/mcollective/security.rb
/usr/lib/ruby/vendor_ruby/mcollective/log.rb
/usr/lib/ruby/vendor_ruby/mcollective/matcher.rb
/usr/lib/ruby/vendor_ruby/mcollective/unix_daemon.rb
/usr/lib/ruby/vendor_ruby/mcollective/application
/usr/lib/ruby/vendor_ruby/mcollective/application/rpc.rb
/usr/lib/ruby/vendor_ruby/mcollective/application/plugin.rb
/usr/lib/ruby/vendor_ruby/mcollective/application/ping.rb
/usr/lib/ruby/vendor_ruby/mcollective/application/completion.rb
/usr/lib/ruby/vendor_ruby/mcollective/application/describe_filter.rb
/usr/lib/ruby/vendor_ruby/mcollective/application/facts.rb
/usr/lib/ruby/vendor_ruby/mcollective/application/help.rb
/usr/lib/ruby/vendor_ruby/mcollective/application/inventory.rb
/usr/lib/ruby/vendor_ruby/mcollective/application/find.rb
/usr/lib/ruby/vendor_ruby/mcollective/validator
/usr/lib/ruby/vendor_ruby/mcollective/validator/regex_validator.ddl
/usr/lib/ruby/vendor_ruby/mcollective/validator/typecheck_validator.ddl
/usr/lib/ruby/vendor_ruby/mcollective/validator/shellsafe_validator.ddl
/usr/lib/ruby/vendor_ruby/mcollective/validator/ipv6address_validator.ddl
/usr/lib/ruby/vendor_ruby/mcollective/validator/length_validator.ddl
/usr/lib/ruby/vendor_ruby/mcollective/validator/shellsafe_validator.rb
/usr/lib/ruby/vendor_ruby/mcollective/validator/ipv4address_validator.ddl
/usr/lib/ruby/vendor_ruby/mcollective/validator/ipv4address_validator.rb
/usr/lib/ruby/vendor_ruby/mcollective/validator/array_validator.ddl
/usr/lib/ruby/vendor_ruby/mcollective/validator/length_validator.rb
/usr/lib/ruby/vendor_ruby/mcollective/validator/regex_validator.rb
/usr/lib/ruby/vendor_ruby/mcollective/validator/ipv6address_validator.rb
/usr/lib/ruby/vendor_ruby/mcollective/validator/array_validator.rb
/usr/lib/ruby/vendor_ruby/mcollective/validator/typecheck_validator.rb
/usr/lib/ruby/vendor_ruby/mcollective/windows_daemon.rb
/usr/lib/ruby/vendor_ruby/mcollective/aggregate
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/sum.ddl
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/average.ddl
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/summary.ddl
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/result
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/result/collection_result.rb
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/result/numeric_result.rb
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/result/base.rb
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/sum.rb
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/result.rb
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/average.rb
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/summary.rb
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/base.rb
/usr/lib/ruby/vendor_ruby/mcollective/runner.rb
/usr/lib/ruby/vendor_ruby/mcollective/optionparser.rb
/usr/lib/ruby/vendor_ruby/mcollective/ssl.rb
/usr/lib/ruby/vendor_ruby/mcollective/runnerstats.rb
/usr/lib/ruby/vendor_ruby/mcollective/shell.rb
/usr/lib/ruby/vendor_ruby/mcollective/validator.rb
/usr/lib/ruby/vendor_ruby/mcollective/vendor
/usr/lib/ruby/vendor_ruby/mcollective/vendor/require_vendored.rb
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/lib
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/lib/systemu.rb
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/samples
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/samples/b.rb
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/samples/a.rb
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/samples/d.rb
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/samples/e.rb
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/samples/f.rb
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/samples/c.rb
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/systemu.gemspec
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/Rakefile
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/README
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/LICENSE
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/README.erb
/usr/lib/ruby/vendor_ruby/mcollective/vendor/load_systemu.rb
/usr/lib/ruby/vendor_ruby/mcollective/matcher
/usr/lib/ruby/vendor_ruby/mcollective/matcher/parser.rb
/usr/lib/ruby/vendor_ruby/mcollective/matcher/scanner.rb
/usr/lib/ruby/vendor_ruby/mcollective/audit
/usr/lib/ruby/vendor_ruby/mcollective/audit/logfile.rb
/usr/lib/ruby/vendor_ruby/mcollective/config.rb
/usr/lib/ruby/vendor_ruby/mcollective/message.rb
/usr/lib/ruby/vendor_ruby/mcollective/application.rb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager.rb
/usr/lib/ruby/vendor_ruby/mcollective/facts.rb
/usr/lib/ruby/vendor_ruby/mcollective/aggregate.rb
/usr/lib/ruby/vendor_ruby/mcollective/cache.rb
/usr/lib/ruby/vendor_ruby/mcollective/exceptions.rb
/usr/lib/ruby/vendor_ruby/mcollective/registration
/usr/lib/ruby/vendor_ruby/mcollective/registration/agentlist.rb
/usr/lib/ruby/vendor_ruby/mcollective/registration/base.rb
/usr/lib/ruby/vendor_ruby/mcollective/discovery
/usr/lib/ruby/vendor_ruby/mcollective/discovery/mc.rb
/usr/lib/ruby/vendor_ruby/mcollective/discovery/mc.ddl
/usr/lib/ruby/vendor_ruby/mcollective/discovery/flatfile.ddl
/usr/lib/ruby/vendor_ruby/mcollective/discovery/stdin.ddl
/usr/lib/ruby/vendor_ruby/mcollective/discovery/stdin.rb
/usr/lib/ruby/vendor_ruby/mcollective/discovery/flatfile.rb
/usr/lib/ruby/vendor_ruby/mcollective/applications.rb
/usr/lib/ruby/vendor_ruby/mcollective/generators.rb
/usr/lib/ruby/vendor_ruby/mcollective/data
/usr/lib/ruby/vendor_ruby/mcollective/data/fact_data.ddl
/usr/lib/ruby/vendor_ruby/mcollective/data/agent_data.rb
/usr/lib/ruby/vendor_ruby/mcollective/data/collective_data.rb
/usr/lib/ruby/vendor_ruby/mcollective/data/agent_data.ddl
/usr/lib/ruby/vendor_ruby/mcollective/data/result.rb
/usr/lib/ruby/vendor_ruby/mcollective/data/collective_data.ddl
/usr/lib/ruby/vendor_ruby/mcollective/data/fact_data.rb
/usr/lib/ruby/vendor_ruby/mcollective/data/fstat_data.ddl
/usr/lib/ruby/vendor_ruby/mcollective/data/fstat_data.rb
/usr/lib/ruby/vendor_ruby/mcollective/data/base.rb
/usr/lib/ruby/vendor_ruby/mcollective/ddl
/usr/lib/ruby/vendor_ruby/mcollective/ddl/dataddl.rb
/usr/lib/ruby/vendor_ruby/mcollective/ddl/discoveryddl.rb
/usr/lib/ruby/vendor_ruby/mcollective/ddl/validatorddl.rb
/usr/lib/ruby/vendor_ruby/mcollective/ddl/agentddl.rb
/usr/lib/ruby/vendor_ruby/mcollective/ddl/base.rb
/usr/lib/ruby/vendor_ruby/mcollective/data.rb
/usr/lib/ruby/vendor_ruby/mcollective/logger
/usr/lib/ruby/vendor_ruby/mcollective/logger/file_logger.rb
/usr/lib/ruby/vendor_ruby/mcollective/logger/console_logger.rb
/usr/lib/ruby/vendor_ruby/mcollective/logger/syslog_logger.rb
/usr/lib/ruby/vendor_ruby/mcollective/logger/base.rb
/usr/lib/ruby/vendor_ruby/mcollective/registration.rb
/usr/lib/ruby/vendor_ruby/mcollective/discovery.rb
/usr/lib/ruby/vendor_ruby/mcollective/util.rb
/usr/lib/ruby/vendor_ruby/mcollective/logger.rb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/agent_definition.rb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/debian
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/debian/compat.erb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/debian/control.erb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/debian/rules.erb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/debian/changelog.erb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/debian/Makefile.erb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/debian/copyright.erb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/module
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/module/README.md.erb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/module/_manifest.pp.erb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/module/Modulefile.erb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/redhat
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/redhat/rpm_spec.erb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/standard_definition.rb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/rpmpackage_packager.rb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/debpackage_packager.rb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/ospackage_packager.rb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/modulepackage_packager.rb
/usr/lib/ruby/vendor_ruby/mcollective/agent.rb
/usr/lib/ruby/vendor_ruby/mcollective/agents.rb
/usr/lib/ruby/vendor_ruby/mcollective/agent
/usr/lib/ruby/vendor_ruby/mcollective/agent/discovery.rb
/usr/lib/ruby/vendor_ruby/mcollective/agent/rpcutil.rb
/usr/lib/ruby/vendor_ruby/mcollective/agent/rpcutil.ddl
/usr/lib/ruby/vendor_ruby/mcollective/ddl.rb
/usr/lib/ruby/vendor_ruby/mcollective/connector
/usr/lib/ruby/vendor_ruby/mcollective/connector/activemq.rb
/usr/lib/ruby/vendor_ruby/mcollective/connector/rabbitmq.rb
/usr/lib/ruby/vendor_ruby/mcollective/connector/rabbitmq.ddl
/usr/lib/ruby/vendor_ruby/mcollective/connector/activemq.ddl
/usr/lib/ruby/vendor_ruby/mcollective/connector/base.rb
/usr/lib/ruby/vendor_ruby/mcollective/generators
/usr/lib/ruby/vendor_ruby/mcollective/generators/agent_generator.rb
/usr/lib/ruby/vendor_ruby/mcollective/generators/templates
/usr/lib/ruby/vendor_ruby/mcollective/generators/templates/action_snippet.erb
/usr/lib/ruby/vendor_ruby/mcollective/generators/templates/ddl.erb
/usr/lib/ruby/vendor_ruby/mcollective/generators/templates/data_input_snippet.erb
/usr/lib/ruby/vendor_ruby/mcollective/generators/templates/plugin.erb
/usr/lib/ruby/vendor_ruby/mcollective/generators/data_generator.rb
/usr/lib/ruby/vendor_ruby/mcollective/generators/base.rb
/usr/lib/ruby/vendor_ruby/mcollective/vendor.rb
/usr/lib/ruby/vendor_ruby/mcollective/security
/usr/lib/ruby/vendor_ruby/mcollective/security/ssl.rb
/usr/lib/ruby/vendor_ruby/mcollective/security/aes_security.rb
/usr/lib/ruby/vendor_ruby/mcollective/security/psk.rb
/usr/lib/ruby/vendor_ruby/mcollective/security/base.rb
/usr/lib/ruby/vendor_ruby/mcollective.rb
/usr/share
/usr/share/doc
/usr/share/doc/mcollective-common
/usr/share/doc/mcollective-common/copyright
/usr/share/doc/mcollective-common/changelog.Debian.gz
/etc
/etc/mcollective
/etc/mcollective/metadata-help.erb
/etc/mcollective/data-help.erb
/etc/mcollective/rpc-help.erb
/etc/mcollective/discovery-help.erb
Not sure if you can simply switch core_libdir and site_libdir without affecting older versions, which is what I am testing our infrastructure (12.04, 14.04, 16.04). I can though say that the current configuration will not work on 16.04 unless you're possibly installing from the source packages or something as then the source would get copied to your libdir and all would be well.
Does a clean libdir and a fresh install of mcollective-puppet-agent really work on your test server with mco > 2.8?
luksi1 commented
And I think this issue is along the same lines:
mcollective::plugin { 'puppet':
package => true,
}
This will install mcollective-puppet-agent and then the next run will filebucket it! As the package will be installed under the site plugins directory.
antaflos commented
As I've commented in issue #325 you should be using the Puppetlabs-provided MCollective packages on 12.04 and 14.04. If you then set core_libdir like this it should work nicely:
class { '::mcollective':
...
core_libdir => '/usr/share/mcollective/plugins',
...
}That's what we currently do on 12.04 and 14.04, using this module and MCollective 2.9 (from the Puppetlabs packages).
Unfortunately Puppetlabs do not provide MCollective packages on 16.04, so you can either try to make it work with whatever Ubuntu provides, or can you read through https://tickets.puppetlabs.com/browse/MCO-722 for other ideas.
As you can see the whole situation is a mess and I am not impressed by how Puppetlabs seem to treat MCollective more and more like a second-class citizen.
Semi-related question: how do you get Puppet 3.8 to work on 16.04? The default Ruby version on 16.04 is 2.3 which Puppet 3.8 does not support. Do you install an older Ruby version, perhaps from the Brightbox PPAs?
luksi1 commented
This works nicely. I did not catch the need to set core_libdir as a necessity for Ubuntu in the docs; maybe this should be updated in the README file?
This is the output from a 16.04 LTS server we have.
myserver:~# ps -ef | grep puppet
root 442 32208 0 17:21 pts/0 00:00:00 grep --color=auto puppet
root 2916 1 0 aug24 ? 00:01:30 /usr/bin/ruby /usr/bin/puppet agent
myserver:~# ruby --version
ruby 2.3.1p112 (2016-04-26) [x86_64-linux-gnu]
myserver:~# dpkg -s puppet
Package: puppet
Status: install ok installed
Priority: optional
Section: admin
Installed-Size: 59
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Architecture: all
Version: 3.8.5-2
Depends: init-system-helpers (>= 1.18~), puppet-common (= 3.8.5-2), ruby | ruby-interpreter
Suggests: puppet-el, vim-puppet, etckeeper
Conffiles:
/etc/init.d/puppet 9eb2754a651af8a0ad2091fbba775bcc
Description: configuration management system, agent
Puppet is a server automation tool. It is composed of a declarative
language for expressing system configuration, a client and server for
distributing it, and a library for realizing the configuration.
.
The primary design goal of Puppet is to have an expressive enough
language backed by a powerful enough library that you can write your
own server automation applications in just a few lines of code.
Puppet’s deep extensibility and open source license lets you add
functionality as needed and share your innovations with others.
.
This package contains "puppet" service and init script used to run
the puppet agent, and depend on the "puppet-common" package which
contains the software.
Original-Maintainer: Puppet Package Maintainers <pkg-puppet-devel@lists.alioth.debian.org>
Homepage: http://projects.puppetlabs.com/projects/puppet
antaflos commented
I see, thank you. So you use the Puppet 3.8 packages provided by Ubuntu, not by Puppetlabs. I assume the Ubuntu package maintainers patched Puppet and its dependencies so that they work properly with Ruby 2.3.x. Which makes sense, since Puppetlabs don't even provide Puppet 3.8 packages for Ubuntu 16.04 any more (only PC1/AIO for Puppet 4). Good to know!