berks upload fails with gem error on 0.10.0
janskarvall opened this issue · 15 comments
I am stuck with the error reported here: chef-boneyard/chef-dk#603. The error was not reported by me, but I have made a comment just stating that I get the same (or similar) error.
I was hoping that issue 603 was going to be fixed, but that issue is now closed without a fix.
So, I need advice on how to work around it.
I am experiencing the same issue, with a fresh instalation of chefdk 0.10.0-01.
How did you install knife & knife-solo? I suspect a fix would be to stick to only using chefdk and installing knife-solo via chef gem install knife-solo
Have you tried that approach?
Or if you're using a Gemfile locking it to berkshelf-4.0.1 & httpclient-2.7.0 might help.
Just to be sure, I made a fresh install of chef and knife-solo in a VirtualBox machine running an also fresh install of Ubuntu 14.04 LTS Desktop, 64-bit. I downloaded and installed the latest Chef 0.10.0-1 package (using Ubuntu Software Center which starts automatically after download). I also entered the 'eval "$(chef shell-init bash)"' line, not in ~/.bash_profile as the install guide says, as it is not run when starting a graphical terminal, but in ~/.profile), and checked that the path to ruby is set to /opt/chefdk/embedded/bin/ruby. Then I installed knife-solo with 'chef gem install knife-solo'.
I also have an old installation that I have been using, and that I have the original trouble with. I believe that the installations of chef and knife-sole are the same as on the VirtualBox, but I'm not 100 percent sure. One thing that differs is that I have also installed the Ruby package on that machine, but the path to ruby is the same, i.e. /opt/chefdk/embedded/bin/ruby. That installation has been working fine until, I think, I upgraded chef to 0.10.0. Then the problem occurred. I'm not sure if it is connected to the upgrade. If I remember right, I also made a 'berks update'.
In this old installation I have a lot of site-cookbooks that I rely on. I copied the whole cook-book structure to the VirtualBox installation (using rsync), and I get the same problem in the VirtualBox installation. I made 'berks update' on the VirtualBox machine too.
In both cases I target another, physical machine, the same in both cases.
I'm not using a Gemfile - I wouldn't know how to do that.
Thanks. I'm getting a little lost in all the permutations so I'll plan to
try with a fresh VM as well and send a vagrant file with results.
If you have specific commands you're using to get the issue let me know.
On Saturday, 6 February 2016, janskarvall notifications@github.com wrote:
Just to be sure, I made a fresh install of chef and knife-solo in a
VirtualBox machine running an also fresh install of Ubuntu 14.04 LTS
Desktop, 64-bit. I downloaded and installed the latest Chef 0.10.0-1
package (using Ubuntu Software Center which starts automatically after
download). I also entered the 'eval "$(chef shell-init bash)"' line, not in
~/.bash_profile as the install guide says, as it is not run when starting a
graphical terminal, but in ~/.profile), and checked that the path to ruby
is set to /opt/chefdk/embedded/bin/ruby. Then I installed knife-solo with
'chef gem install knife-solo'.I also have an old installation that I have been using, and that I have
the original trouble with. I believe that the installations of chef and
knife-sole are the same as on the VirtualBox, but I'm not 100 percent sure.
One thing that differs is that I have also installed the Ruby package on
that machine, but the path to ruby is the same, i.e.
/opt/chefdk/embedded/bin/ruby. That installation has been working fine
until, I think, I upgraded chef to 0.10.0. Then the problem occurred. I'm
not sure if it is connected to the upgrade. If I remember right, I also
made a 'berks update'.In this old installation I have a lot of site-cookbooks that I rely on. I
copied the whole cook-book structure to the VirtualBox installation (using
rsync), and I get the same problem in the VirtualBox installation. I made
'berks update' on the VirtualBox machine too.In both cases I target another, physical machine, the same in both cases.
I'm not using a Gemfile - I wouldn't know how to do that.
—
Reply to this email directly or view it on GitHub
#476 (comment)
.
-Mat
matschaffer.com
I have never used vagrant, so I do not know what to get out of a vagrant file.
In order to assure that the problem is not connected to my cookbook structure, I made a very simple structure and tested:
janne@janne-VirtualBox:~/workspace/test$ cat nodes/192.168.0.182.json
{
"run_list":
[
"recipe[test]"
]
}
janne@janne-VirtualBox:~/workspace/test$ cat site-cookbooks/test/recipes/default.rb
file '/tmp/test' do
content 'test'
end
janne@janne-VirtualBox:~/workspace/test$ cat site-cookbooks/test/metadata.rb
name 'test'
maintainer 'YOUR_COMPANY_NAME'
maintainer_email 'YOUR_EMAIL'
license 'All rights reserved'
description 'Installs/Configures apptome-server'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version '0.1.0'
janne@janne-VirtualBox:~/workspace/test$ cat Berksfile
source "https://supermarket.chef.io"
# metadata
janne@janne-VirtualBox:~/workspace/test$ knife solo cook 192.168.0.182
Running Chef on 192.168.0.182...
Checking Chef version...
janne@192.168.0.182's password:
janne@192.168.0.182's password:
Enter the password for janne@192.168.0.182:
WARNING: #<Gem::ConflictError: Unable to activate berkshelf-4.0.1, because httpclient-2.7.0 conflicts with httpclient (~> 2.6.0)>
WARNING: Berkshelf could not be loaded
WARNING: Please add the berkshelf gem to your Gemfile or install it manually with `gem install berkshelf`
Uploading the kitchen...
janne@192.168.0.182's password:
Generating solo config...
Running Chef: sudo chef-solo -c ~/chef-solo/solo.rb -j ~/chef-solo/dna.json
Starting Chef Client, version 12.5.1
Compiling Cookbooks...
Converging 1 resources
Recipe: test::default
* file[/tmp/test] action create
- create new file /tmp/test
- update content in file /tmp/test from none to 9f86d0
--- /tmp/test 2016-02-06 10:59:27.217874435 +0100
+++ /tmp/.test20160206-19959-1rl5yyv 2016-02-06 10:59:27.217874435 +0100
@@ -1 +1,2 @@
+test
Running handlers:
Running handlers complete
Chef Client finished, 1/1 resources updated in 02 seconds
janne@janne-VirtualBox:~/workspace/test$
So, the warning that worries me is still there, and I don't know how to interpret it, but it sounds worrying.
I got exact same error.
I do not run in a VM. I am on a fresh Ubuntu 15.10 install (ruby is not installed) with a fresh chefdk install. Knife solo have been installed via:
chef gem install knife-solo
What is strange is that I am experiencing the same problem on an older installation which used to work, and, AFAIR, no update of chefdk or knife-solo has been made since then.
These warnings result in problems with finding my recipes.
I'm not sure how Berkshelf should work, but I have problems getting berks update to work properly.
janne@janne-VirtualBox:~/workspace/chef-repo$ berks update
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/httpclient-2.6.0.1/lib/httpclient/webagent-cookie.rb:458: warning: already initialized constant HTTPClient::CookieManager
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/httpclient-2.6.0.1/lib/httpclient/cookie.rb:8: warning: previous definition of CookieManager was here
Resolving cookbook dependencies...
(lines removed)
Using firewall (2.4.0)
(lines removed)
janne@janne-VirtualBox:~/workspace/chef-repo$
The warnings are discussed in Berks install warnings after ChefDK 0.10.
So, firewall is resolved to version 2.4.0. However, looking at the downloaded firewall metadata, I get:
janne@janne-VirtualBox:~/workspace/chef-repo$ head cookbooks/firewall/metadata.json
{"name":"firewall","version":"1.6.1", (text deleted)
So berks update fails to update the cookbook. Compare with firewall/metadata.rb.
I don't know if berks update also should delete a cookbook from the cookbooks directory if you delete that cookbook from Berksfile. Anyway, it does not.
I suspect that berks update does update the cookbooks in ~/.berkshelf/cookbooks/ rather than in the expected place, i.e. in my case ~/workspace/chef-repo/cookbooks. I have not found out why yet.
I'm totally lost. I think I got the ~/workspace/chef-repo/.chef/knife.rb file by cloning chef/chef-repo a long time ago. Having a look there, that repository now seems to be deprecated, and one refers to chef generate repo command. Looking at chef (executable) I don't find any documentation about that command. However, I can run it:
janne@janne-VirtualBox:~/workspace$ chef generate repo x
Compiling Cookbooks...
Recipe: code_generator::repo
* directory[/home/janne/workspace/x] action create
- create new directory /home/janne/workspace/x
* template[/home/janne/workspace/x/LICENSE] action create_if_missing
- create new file /home/janne/workspace/x/LICENSE
- update content in file /home/janne/workspace/x/LICENSE from none to f750c0
(diff output suppressed by config)
* cookbook_file[/home/janne/workspace/x/.chef-repo.txt] action create_if_missing
- create new file /home/janne/workspace/x/.chef-repo.txt
- update content in file /home/janne/workspace/x/.chef-repo.txt from none to 2bed28
(diff output suppressed by config)
* cookbook_file[/home/janne/workspace/x/README.md] action create_if_missing
- create new file /home/janne/workspace/x/README.md
- update content in file /home/janne/workspace/x/README.md from none to 1b81aa
(diff output suppressed by config)
* cookbook_file[/home/janne/workspace/x/chefignore] action create_if_missing
- create new file /home/janne/workspace/x/chefignore
- update content in file /home/janne/workspace/x/chefignore from none to 51b09a
(diff output suppressed by config)
* remote_directory[/home/janne/workspace/x/cookbooks] action create_if_missing
- create new directory /home/janne/workspace/x/cookbooks
Recipe: <Dynamically Defined Resource>
* directory[/home/janne/workspace/x/cookbooks/example] action create
- create new directory /home/janne/workspace/x/cookbooks/example
- change mode from '' to '0775'
- change owner from '' to 'janne'
- change group from '' to 'janne'
* cookbook_file[/home/janne/workspace/x/cookbooks/example/README.md] action create_if_missing
- create new file /home/janne/workspace/x/cookbooks/example/README.md
- update content in file /home/janne/workspace/x/cookbooks/example/README.md from none to d6634d
(diff output suppressed by config)
- change mode from '' to '0644'
* cookbook_file[/home/janne/workspace/x/cookbooks/example/metadata.rb] action create_if_missing
- create new file /home/janne/workspace/x/cookbooks/example/metadata.rb
- update content in file /home/janne/workspace/x/cookbooks/example/metadata.rb from none to f0778c
(diff output suppressed by config)
- change mode from '' to '0644'
* directory[/home/janne/workspace/x/cookbooks/example/recipes] action create
- create new directory /home/janne/workspace/x/cookbooks/example/recipes
- change mode from '' to '0775'
- change owner from '' to 'janne'
- change group from '' to 'janne'
* cookbook_file[/home/janne/workspace/x/cookbooks/example/recipes/default.rb] action create_if_missing
- create new file /home/janne/workspace/x/cookbooks/example/recipes/default.rb
- update content in file /home/janne/workspace/x/cookbooks/example/recipes/default.rb from none to fec476
(diff output suppressed by config)
- change mode from '' to '0644'
* directory[/home/janne/workspace/x/cookbooks/example/attributes] action create
- create new directory /home/janne/workspace/x/cookbooks/example/attributes
- change mode from '' to '0775'
- change owner from '' to 'janne'
- change group from '' to 'janne'
* cookbook_file[/home/janne/workspace/x/cookbooks/example/attributes/default.rb] action create_if_missing
- create new file /home/janne/workspace/x/cookbooks/example/attributes/default.rb
- update content in file /home/janne/workspace/x/cookbooks/example/attributes/default.rb from none to 16a397
(diff output suppressed by config)
- change mode from '' to '0644'
Recipe: code_generator::repo
* remote_directory[/home/janne/workspace/x/data_bags] action create_if_missing
- create new directory /home/janne/workspace/x/data_bags
Recipe: <Dynamically Defined Resource>
* cookbook_file[/home/janne/workspace/x/data_bags/README.md] action create_if_missing
- create new file /home/janne/workspace/x/data_bags/README.md
- update content in file /home/janne/workspace/x/data_bags/README.md from none to 3167c5
(diff output suppressed by config)
- change mode from '' to '0644'
* directory[/home/janne/workspace/x/data_bags/example] action create
- create new directory /home/janne/workspace/x/data_bags/example
- change mode from '' to '0775'
- change owner from '' to 'janne'
- change group from '' to 'janne'
* cookbook_file[/home/janne/workspace/x/data_bags/example/example_item.json] action create_if_missing
- create new file /home/janne/workspace/x/data_bags/example/example_item.json
- update content in file /home/janne/workspace/x/data_bags/example/example_item.json from none to 1048ec
(diff output suppressed by config)
- change mode from '' to '0644'
Recipe: code_generator::repo
* remote_directory[/home/janne/workspace/x/roles] action create_if_missing
- create new directory /home/janne/workspace/x/roles
Recipe: <Dynamically Defined Resource>
* cookbook_file[/home/janne/workspace/x/roles/README.md] action create_if_missing
- create new file /home/janne/workspace/x/roles/README.md
- update content in file /home/janne/workspace/x/roles/README.md from none to 7a28c9
(diff output suppressed by config)
- change mode from '' to '0644'
* cookbook_file[/home/janne/workspace/x/roles/example.json] action create_if_missing
- create new file /home/janne/workspace/x/roles/example.json
- update content in file /home/janne/workspace/x/roles/example.json from none to 297ea8
(diff output suppressed by config)
- change mode from '' to '0644'
Recipe: code_generator::repo
* remote_directory[/home/janne/workspace/x/environments] action create_if_missing
- create new directory /home/janne/workspace/x/environments
Recipe: <Dynamically Defined Resource>
* cookbook_file[/home/janne/workspace/x/environments/README.md] action create_if_missing
- create new file /home/janne/workspace/x/environments/README.md
- update content in file /home/janne/workspace/x/environments/README.md from none to c9ccdf
(diff output suppressed by config)
- change mode from '' to '0644'
* cookbook_file[/home/janne/workspace/x/environments/example.json] action create_if_missing
- create new file /home/janne/workspace/x/environments/example.json
- update content in file /home/janne/workspace/x/environments/example.json from none to 59a963
(diff output suppressed by config)
- change mode from '' to '0644'
Recipe: code_generator::repo
* cookbook_file[/home/janne/workspace/x/cookbooks/README.md] action create_if_missing
- create new file /home/janne/workspace/x/cookbooks/README.md
- update content in file /home/janne/workspace/x/cookbooks/README.md from none to 86e9ef
(diff output suppressed by config)
* execute[initialize-git] action run
- execute git init .
* template[/home/janne/workspace/x/.gitignore] action create_if_missing
- create new file /home/janne/workspace/x/.gitignore
- update content in file /home/janne/workspace/x/.gitignore from none to 3523c4
(diff output suppressed by config)
janne@janne-VirtualBox:~/workspace$ cd x
janne@janne-VirtualBox:~/workspace/x$ ls -a
. chefignore cookbooks environments .gitignore README.md
.. .chef-repo.txt data_bags .git LICENSE roles
janne@janne-VirtualBox:~/workspace/x$
But there is neither ~/workspace/x/.chef/knife.rb file or a ~/workspace/x/.chef/config.rb file generated. Also, the Chef documentation about those files don't mention the knife[:berkshelf_path] = "cookbooks" configuration that I have in my ~/workspace/chef-repo/.chef/knife.rb file.
So, I can not understand how berks get to know where to put the downloaded cookbooks.
@janskarvall I tried this out this morning, here's the result:
https://gist.github.com/matschaffer/b386abcc1d88863dcc5a
Looks like knife-solo's berkshelf integration is no longer chefdk-compatible as of 0.10.0.
I'll file another ticket referencing this one. In the mean time the best resolutions I can offer is downgrading to chefdk 0.9.0, or using ruby directly and locking gem versions via bundler.
I have tried to find out if it is time to upgrade Chef DK.
I am currently using chefdk 0.9.0-1 together with knife-solo 0.5.1 on my Ubuntu 14.04 LTS client, targeting a Ununtu 16.04 LTS server without problems.
On the client (DK) side is seems that Chef is not yet promising that it runs om Ubuntu 16.04, so maybe I have to wait before upgrading the client (DK) machine to Ubuntu 16.04, but I wonder if it is possible to upgrade Chef-DK and knife-solo on that machine, and if so, to what versions.
Chef DK's latest version seems to be 0.15.16 right now.
Should chef 0.15.16 work together with knife-solo 0.6.0 (which seems to require chef >= 10.20) on my Ubuntu 14.04 machine?
Pretty sure all the latest releases should work together, and ubuntu 14.04 is recent enough so I'd recommend giving it a try. Note that the chef requirement in knife-solo is to https://rubygems.org/gems/chef (currently 12.12.15), not Chef DK.
If you hit a particular error let me know and I can try to test out your scenario on VMs.
I successfully cooked my Ununtu 16.04 LTS server using Chef DK 0.15.16 and knife-solo 0.6.0 from my Ubuntu 14.04 client.
When running chef gem list chef I however get chef (12.11.18).
Should I worry about that?
I wouldn't. :) good to hear it's working.
That's close enough to the latest that you shouldn't see any functionality
difference.
Could be the chef omnibus packages (what knife-solo will use on Ubuntu by
default) haven't been updated yet.
On Wednesday, 20 July 2016, janskarvall notifications@github.com wrote:
I successfully cooked my Ununtu 16.04 LTS server using Chef DK 0.15.16 and
knife-solo 0.6.0 from my Ubuntu 14.04 client.When running chef gem list chef I however get chef (12.11.18).
Should I worry about that?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
#476 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAACsirdoqweZLrCBQbg8rwZh4Bc8yJkks5qXf7mgaJpZM4HUPyO
.
-Mat
matschaffer.com
I downloaded Chef-DK 0.15.16 using this link and installed it using Ubuntu Software Center on my Ubuntu 14.04 LTS Desktop 64 bit client.