PierreRambaud/gemirro

Network error while fetching

Closed this issue · 14 comments

questers commented

Hi,

I try to use gemirro as gem mirror, but I have problems with it.

My Gemfile:

source 'http://173.17.41.6:2000'

gem 'rake'
gem 'asciidoctor', '1.5.0'
gem 'asciidoctor-pdf', '1.5.0.alpha.8'
gem 'coderay'
gem 'thread_safe'

Then I get following error:

C:\Data\mytest\scm_server>bundle install
Fetching gem metadata from http://173.17.41.6:2000/.Retrying dependency api due
to error (2/4): Bundler::HTTPError Network error while fetching http://173.17.41.6:2000/api/v1/dependencies?gems=rake,asciidoctor,asciidoctor-pdf,coderay,thread
_safe
Retrying dependency api due to error (3/4): Bundler::HTTPError Network error whi
le fetching http://173.17.41.6:2000/api/v1/dependencies?gems=rake,asciidoctor,as
ciidoctor-pdf,coderay,thread_safe
Retrying dependency api due to error (4/4): Bundler::HTTPError Network error whi
le fetching http://173.17.41.6:2000/api/v1/dependencies?gems=rake,asciidoctor,as
ciidoctor-pdf,coderay,thread_safe


Network error while fetching
http://173.17.41.6:2000/quick/Marshal.4.8/asciidoctor-pdf-1.5.0.alpha.8.gemspec.
rz

C:\Data\mytest\scm_server>

After getting the error, gemirro take 100% of CPU on the server for a long time (15 minutes and longer).

Its possible to solve the problem?

Best regards,

PierreRambaud commented

Hi,

Is gemirro running under Unix system?
How did you start the server?
Have you tried to go to http://173.17.41.6:2000 with your browser?
What error logs file contains?

Regards

questers commented

running at Ubuntu Server 14.04.2 and Ruby 2.1.6.

The server was started with

sudo gemirro server --start

If gemirro not access full CPU then I can access the web interface. It shows a part of my needed gems.

in accesslog I found following:

I, [2015-07-29T16:48:43.826350 #2277]  INFO -- : Try to download coderay with version 0.9.8.pre
I, [2015-07-29T16:48:43.827161 #2277]  INFO -- : Generating indexes
I, [2015-07-29T16:48:44.930176 #2277]  INFO -- : Try to download coderay with version 0.9.7.pre
I, [2015-07-29T16:48:44.930613 #2277]  INFO -- : Generating indexes
I, [2015-07-29T16:48:58.882428 #2277]  INFO -- : Try to download asciidoctor-pdf with version 1.5.0.alpha.1
E, [2015-07-29T16:48:58.882600 #2277] ERROR -- : Need a Gem::Version: nil
I, [2015-07-29T16:48:58.882688 #2277]  INFO -- : Try to download asciidoctor-pdf with version 1.5.0.alpha.2
E, [2015-07-29T16:48:58.882757 #2277] ERROR -- : Need a Gem::Version: nil
I, [2015-07-29T16:48:58.882837 #2277]  INFO -- : Try to download asciidoctor-pdf with version 1.5.0.alpha.3
E, [2015-07-29T16:48:58.882903 #2277] ERROR -- : Need a Gem::Version: nil
I, [2015-07-29T16:48:58.882982 #2277]  INFO -- : Try to download asciidoctor-pdf with version 1.5.0.alpha.4
E, [2015-07-29T16:48:58.883046 #2277] ERROR -- : Need a Gem::Version: nil
I, [2015-07-29T16:48:58.883123 #2277]  INFO -- : Try to download asciidoctor-pdf with version 1.5.0.alpha.5
E, [2015-07-29T16:48:58.883186 #2277] ERROR -- : Need a Gem::Version: nil
I, [2015-07-29T16:48:58.883263 #2277]  INFO -- : Try to download asciidoctor-pdf with version 1.5.0.alpha.6
E, [2015-07-29T16:48:58.883327 #2277] ERROR -- : Need a Gem::Version: nil
I, [2015-07-29T16:48:58.883403 #2277]  INFO -- : Try to download asciidoctor-pdf with version 1.5.0.alpha.7
E, [2015-07-29T16:48:58.883466 #2277] ERROR -- : Need a Gem::Version: nil
I, [2015-07-29T16:48:58.883540 #2277]  INFO -- : Try to download asciidoctor-pdf with version 1.5.0.alpha.8
E, [2015-07-29T16:48:58.883603 #2277] ERROR -- : Need a Gem::Version: nil
I, [2015-07-29T16:49:09.638873 #2277]  INFO -- : Try to download coderay with version 0.9.6.pre
I, [2015-07-29T16:49:09.639474 #2277]  INFO -- : Generating indexes
I, [2015-07-29T16:49:15.848356 #2277]  INFO -- : Try to download coderay with version 0.9.6.pre2
I, [2015-07-29T16:49:15.848812 #2277]  INFO -- : Generating indexes
I, [2015-07-29T16:49:20.498287 #2277]  INFO -- : Try to download coderay with version 0.9.6.pre2
I, [2015-07-29T16:49:20.498929 #2277]  INFO -- : Generating indexes
I, [2015-07-29T16:49:22.549719 #2277]  INFO -- : Try to download coderay with version 1.0.0.788.pre
I, [2015-07-29T16:49:22.550517 #2277]  INFO -- : Generating indexes
I, [2015-07-29T16:49:24.304202 #2277]  INFO -- : Try to download coderay with version 1.0.0.738.pre
I, [2015-07-29T16:49:24.304677 #2277]  INFO -- : Generating indexes
I, [2015-07-29T16:49:26.301096 #2277]  INFO -- : Try to download coderay with version 0.9.7.pre
I, [2015-07-29T16:49:26.301936 #2277]  INFO -- : Generating indexes
I, [2015-07-29T16:49:26.406461 #2277]  INFO -- : Try to download coderay with version 1.0.0.598.pre
I, [2015-07-29T16:49:26.407034 #2277]  INFO -- : Generating indexes
I, [2015-07-29T16:49:26.725595 #2277]  INFO -- : Try to download coderay with version 0.9.8.pre
I, [2015-07-29T16:49:26.726384 #2277]  INFO -- : Generating indexes
PierreRambaud commented

Can you use the master branch instead of latest release on rubygems?

questers commented

I'm not a ruby developer, but If you give me the steps then I can try it.

PierreRambaud commented

It should be something like this.

git clone https://github.com/PierreRambaud/gemirro
cd gemirro
bundle install
bundle exec rake install
questers commented

The last command can't finish:

user@vm-projects:~/gemirrotest/gemirro$ sudo bundle exec rake install
gemirro 0.10.5 built to pkg/gemirro-0.10.5.gem.
rake aborted!
Couldn't install gem, run `gem install /home/user/gemirrotest/gemirro/pkg/gemirro-0.10.5.gem' for more detailed output
/var/lib/gems/1.9.1/gems/bundler-1.6.5/lib/bundler/gem_helper.rb:69:in `install_gem'
/var/lib/gems/1.9.1/gems/bundler-1.6.5/lib/bundler/gem_helper.rb:44:in `block in install'
Tasks: TOP => install
(See full trace by running task with --trace)
user@vm-projects:~/gemirrotest/gemirro$
questers commented

The last installproblem I have solved by updating bundler on server to 1.10.6. With master branch I get following error:

C:\Data\mytest\scm_server>bundle install
Fetching gem metadata from http://173.17.41.6:2000/.Retrying dependency api due
to error (2/4): Bundler::HTTPError Network error while fetching http://173.17.41.6:2000/api/v1/dependencies?gems=rake,asciidoctor,asciidoctor-pdf,coderay,thread
_safe
Retrying dependency api due to error (3/4): Bundler::HTTPError Network error whi
le fetching http://173.17.41.6:2000/api/v1/dependencies?gems=rake,asciidoctor,as
ciidoctor-pdf,coderay,thread_safe
Retrying dependency api due to error (4/4): Bundler::HTTPError Network error whi
le fetching http://173.17.41.6:2000/api/v1/dependencies?gems=rake,asciidoctor,as
ciidoctor-pdf,coderay,thread_safe


Net::HTTPNotFound: <!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <meta name="description" content="Ruby gems mirror">
    <meta name="author" content="Pierre Rambaud">

    <title>Gemirro - a simple ruby gems mirror</title>

<link rel="stylesheet" href="/dist/css/bootstrap.min.css" type="text/css"
media="screen">
<link rel="stylesheet" href="/dist/css/gemirro.css" type="text/css"
media="screen">
  </head>
  <body>
    <div class="container-fluid">
      <div class="row">
        <div class="col-lg-12 text-center">
          <h1>Gemirro</h1>
        </div>
      </div>
      <div class="row">
        <div class="col-lg-12">
          <div class="center-block text-center">
  <h1>Page not found.</h1>
<a class="btn btn-default" href="http://173.17.41.6:2000/">Back to
http://173.17.41.6:2000/ →</a>
</div>

        </div>
      </div>
    </div>
    <script type="text/javascript" src="/dist/js/jquery.min.js"></script>
    <script type="text/javascript" src="/dist/js/bootstrap.min.js"></script>
  </body>
</html>

C:\Data\mytest\scm_server>

The log folder is empty and access to web interface is possbile. But after the first run only rack is shown in web interface. In the version from rubygems of gemirro after the first run a part of neede gems was shown in web interface.

dgoradia commented

I'm also getting the same issue. Using tag 0.10.5 but also tried with master.

git clone git@github.com:PierreRambaud/gemirro.git
cd gemirro
git checkout tags/0.10.5
bundle install
bundle exec rake install

Gemfile:

source 'http://10.25.100.21:2000'

ruby '2.1.5'

gem 'rake',                    '10.4.2'
gem 'chef',                    '12.3.0'
gem 'berkshelf',               '3.3.0'
gem 'chef-vault',              '2.5.0'
gem 'chef-vault-testfixtures', '0.5.0'
gem 'chefspec',                '4.3.0'
gem 'foodcritic',              '4.0.0'
gem 'test-kitchen',            '1.4.0'
gem 'kitchen-vagrant',         '0.18.0'
gem 'nokogiri',                '1.6.3.1'
gem 'net-ping',                '1.7.7'

group :development do
    gem 'knife-spork',         '1.5.1'
end

Error:

dgoradia@Drus-MacBook-Pro:~/chef-repo/cookbooks/kvm$ bundle update --verbose
HTTP GET http://10.25.100.21:2000/api/v1/dependencies
HTTP 200 OK
Fetching gem metadata from http://10.25.100.21:2000/
Query List: ["rake", "chef", "berkshelf", "chef-vault-testfixtures", "chefspec", "foodcritic", "test-kitchen", "kitchen-vagrant", "nokogiri", "net-ping", "insight-ctl", "knife-spork"]
Query Gemcutter Dependency Endpoint API: rake,chef,berkshelf,chef-vault-testfixtures,chefspec,foodcritic,test-kitchen,kitchen-vagrant,nokogiri,net-ping,insight-ctl,knife-spork
HTTP GET http://10.25.100.21:2000/api/v1/dependencies?gems=rake,chef,berkshelf,chef-vault-testfixtures,chefspec,foodcritic,test-kitchen,kitchen-vagrant,nokogiri,net-ping,insight-ctl,knife-spork
Net::HTTP::Persistent::Error: too many connection resets (due to Net::ReadTimeout - Net::ReadTimeout) after 1 requests on 70347981504980, last used 20.156306 seconds ago
/opt/chefdk/embedded/lib/ruby/2.1.0/net/protocol.rb:158:in `rescue in rbuf_fill'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/protocol.rb:152:in `rbuf_fill'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/protocol.rb:134:in `readuntil'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/protocol.rb:144:in `readline'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http/response.rb:39:in `read_status_line'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http/response.rb:28:in `read_new'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http.rb:1408:in `block in transport_request'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http.rb:1405:in `catch'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http.rb:1405:in `transport_request'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http.rb:1378:in `request'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/vendor/net/http/persistent.rb:999:in `request'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/fetcher.rb:287:in `request'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/fetcher.rb:257:in `fetch'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/fetcher.rb:304:in `fetch_dependency_remote_specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/fetcher.rb:213:in `block in fetch_remote_specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/retry.rb:38:in `call'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/retry.rb:38:in `run'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/retry.rb:28:in `attempt'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/fetcher.rb:212:in `fetch_remote_specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/fetcher.rb:166:in `specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/source/rubygems.rb:301:in `block (2 levels) in remote_specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/source/rubygems.rb:299:in `each'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/source/rubygems.rb:299:in `block in remote_specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/index.rb:9:in `build'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/source/rubygems.rb:284:in `remote_specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/source/rubygems.rb:72:in `specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:204:in `block (2 levels) in index'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:202:in `each'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:202:in `block in index'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/index.rb:9:in `build'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:199:in `index'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:193:in `resolve'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:132:in `specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:121:in `resolve_remotely!'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/installer.rb:79:in `run'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/installer.rb:18:in `install'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/cli/update.rb:51:in `run'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/cli.rb:167:in `update'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/vendor/thor/command.rb:27:in `run'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/vendor/thor/invocation.rb:121:in `invoke_command'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/vendor/thor.rb:363:in `dispatch'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/vendor/thor/base.rb:440:in `start'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/cli.rb:9:in `start'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/bin/bundle:20:in `block in <top (required)>'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/friendly_errors.rb:5:in `with_friendly_errors'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/bin/bundle:18:in `<top (required)>'
/opt/chefdk/embedded/bin/bundle:23:in `load'
/opt/chefdk/embedded/bin/bundle:23:in `<main>'
Retrying dependency api due to error (2/3): Bundler::HTTPError Network error while fetching http://10.25.100.21:2000/api/v1/dependencies?gems=rake,chef,berkshelf,chef-vault-testfixtures,chefspec,foodcritic,test-kitchen,kitchen-vagrant,nokogiri,net-ping,insight-ctl,knife-spork
Query Gemcutter Dependency Endpoint API: rake,chef,berkshelf,chef-vault-testfixtures,chefspec,foodcritic,test-kitchen,kitchen-vagrant,nokogiri,net-ping,insight-ctl,knife-spork
HTTP GET http://10.25.100.21:2000/api/v1/dependencies?gems=rake,chef,berkshelf,chef-vault-testfixtures,chefspec,foodcritic,test-kitchen,kitchen-vagrant,nokogiri,net-ping,insight-ctl,knife-spork
Net::HTTP::Persistent::Error: too many connection resets (due to Net::ReadTimeout - Net::ReadTimeout) after 0 requests on 70347981504980, last used 20.490323 seconds ago
/opt/chefdk/embedded/lib/ruby/2.1.0/net/protocol.rb:158:in `rescue in rbuf_fill'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/protocol.rb:152:in `rbuf_fill'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/protocol.rb:134:in `readuntil'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/protocol.rb:144:in `readline'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http/response.rb:39:in `read_status_line'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http/response.rb:28:in `read_new'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http.rb:1408:in `block in transport_request'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http.rb:1405:in `catch'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http.rb:1405:in `transport_request'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http.rb:1378:in `request'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/vendor/net/http/persistent.rb:999:in `request'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/fetcher.rb:287:in `request'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/fetcher.rb:257:in `fetch'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/fetcher.rb:304:in `fetch_dependency_remote_specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/fetcher.rb:213:in `block in fetch_remote_specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/retry.rb:38:in `call'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/retry.rb:38:in `run'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/retry.rb:28:in `attempt'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/fetcher.rb:212:in `fetch_remote_specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/fetcher.rb:166:in `specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/source/rubygems.rb:301:in `block (2 levels) in remote_specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/source/rubygems.rb:299:in `each'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/source/rubygems.rb:299:in `block in remote_specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/index.rb:9:in `build'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/source/rubygems.rb:284:in `remote_specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/source/rubygems.rb:72:in `specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:204:in `block (2 levels) in index'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:202:in `each'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:202:in `block in index'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/index.rb:9:in `build'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:199:in `index'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:193:in `resolve'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:132:in `specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:121:in `resolve_remotely!'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/installer.rb:79:in `run'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/installer.rb:18:in `install'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/cli/update.rb:51:in `run'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/cli.rb:167:in `update'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/vendor/thor/command.rb:27:in `run'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/vendor/thor/invocation.rb:121:in `invoke_command'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/vendor/thor.rb:363:in `dispatch'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/vendor/thor/base.rb:440:in `start'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/cli.rb:9:in `start'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/bin/bundle:20:in `block in <top (required)>'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/friendly_errors.rb:5:in `with_friendly_errors'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/bin/bundle:18:in `<top (required)>'
/opt/chefdk/embedded/bin/bundle:23:in `load'
/opt/chefdk/embedded/bin/bundle:23:in `<main>'
Retrying dependency api due to error (3/3): Bundler::HTTPError Network error while fetching http://10.25.100.21:2000/api/v1/dependencies?gems=rake,chef,berkshelf,chef-vault-testfixtures,chefspec,foodcritic,test-kitchen,kitchen-vagrant,nokogiri,net-ping,insight-ctl,knife-spork
Query Gemcutter Dependency Endpoint API: rake,chef,berkshelf,chef-vault-testfixtures,chefspec,foodcritic,test-kitchen,kitchen-vagrant,nokogiri,net-ping,insight-ctl,knife-spork
HTTP GET http://10.25.100.21:2000/api/v1/dependencies?gems=rake,chef,berkshelf,chef-vault-testfixtures,chefspec,foodcritic,test-kitchen,kitchen-vagrant,nokogiri,net-ping,insight-ctl,knife-spork
Net::HTTP::Persistent::Error: too many connection resets (due to Net::ReadTimeout - Net::ReadTimeout) after 0 requests on 70347981504980, last used 20.500208 seconds ago
/opt/chefdk/embedded/lib/ruby/2.1.0/net/protocol.rb:158:in `rescue in rbuf_fill'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/protocol.rb:152:in `rbuf_fill'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/protocol.rb:134:in `readuntil'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/protocol.rb:144:in `readline'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http/response.rb:39:in `read_status_line'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http/response.rb:28:in `read_new'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http.rb:1408:in `block in transport_request'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http.rb:1405:in `catch'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http.rb:1405:in `transport_request'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http.rb:1378:in `request'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/vendor/net/http/persistent.rb:999:in `request'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/fetcher.rb:287:in `request'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/fetcher.rb:257:in `fetch'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/fetcher.rb:304:in `fetch_dependency_remote_specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/fetcher.rb:213:in `block in fetch_remote_specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/retry.rb:38:in `call'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/retry.rb:38:in `run'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/retry.rb:28:in `attempt'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/fetcher.rb:212:in `fetch_remote_specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/fetcher.rb:166:in `specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/source/rubygems.rb:301:in `block (2 levels) in remote_specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/source/rubygems.rb:299:in `each'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/source/rubygems.rb:299:in `block in remote_specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/index.rb:9:in `build'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/source/rubygems.rb:284:in `remote_specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/source/rubygems.rb:72:in `specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:204:in `block (2 levels) in index'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:202:in `each'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:202:in `block in index'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/index.rb:9:in `build'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:199:in `index'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:193:in `resolve'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:132:in `specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:121:in `resolve_remotely!'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/installer.rb:79:in `run'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/installer.rb:18:in `install'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/cli/update.rb:51:in `run'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/cli.rb:167:in `update'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/vendor/thor/command.rb:27:in `run'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/vendor/thor/invocation.rb:121:in `invoke_command'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/vendor/thor.rb:363:in `dispatch'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/vendor/thor/base.rb:440:in `start'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/cli.rb:9:in `start'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/bin/bundle:20:in `block in <top (required)>'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/friendly_errors.rb:5:in `with_friendly_errors'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/bin/bundle:18:in `<top (required)>'
/opt/chefdk/embedded/bin/bundle:23:in `load'
/opt/chefdk/embedded/bin/bundle:23:in `<main>'
could not fetch from the dependency API, trying the full index
Resolving dependencies...HTTP GET http://10.25.100.21:2000/quick/Marshal.4.8/chef-12.3.0-x86-mingw32.gemspec.rz
Net::HTTP::Persistent::Error: too many connection resets (due to Net::ReadTimeout - Net::ReadTimeout) after 0 requests on 70347981504980, last used 1438223698.4910588 seconds ago
/opt/chefdk/embedded/lib/ruby/2.1.0/net/protocol.rb:158:in `rescue in rbuf_fill'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/protocol.rb:152:in `rbuf_fill'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/protocol.rb:134:in `readuntil'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/protocol.rb:144:in `readline'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http/response.rb:39:in `read_status_line'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http/response.rb:28:in `read_new'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http.rb:1408:in `block in transport_request'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http.rb:1405:in `catch'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http.rb:1405:in `transport_request'
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http.rb:1378:in `request'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/vendor/net/http/persistent.rb:999:in `request'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/fetcher.rb:287:in `request'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/fetcher.rb:257:in `fetch'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/fetcher.rb:146:in `fetch_spec'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/remote_specification.rb:46:in `_remote_specification'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/remote_specification.rb:51:in `method_missing'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/resolver.rb:102:in `block in __dependencies'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/resolver.rb:99:in `each'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/resolver.rb:99:in `__dependencies'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/resolver.rb:69:in `activate_platform'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/resolver.rb:219:in `activate_gem'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/resolver.rb:382:in `resolve'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/resolver.rb:166:in `start'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/resolver.rb:129:in `resolve'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:193:in `resolve'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:132:in `specs'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/definition.rb:121:in `resolve_remotely!'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/installer.rb:79:in `run'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/installer.rb:18:in `install'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/cli/update.rb:51:in `run'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/cli.rb:167:in `update'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/vendor/thor/command.rb:27:in `run'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/vendor/thor/invocation.rb:121:in `invoke_command'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/vendor/thor.rb:363:in `dispatch'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/vendor/thor/base.rb:440:in `start'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/cli.rb:9:in `start'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/bin/bundle:20:in `block in <top (required)>'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/lib/bundler/friendly_errors.rb:5:in `with_friendly_errors'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.7.12/bin/bundle:18:in `<top (required)>'
/opt/chefdk/embedded/bin/bundle:23:in `load'
/opt/chefdk/embedded/bin/bundle:23:in `<main>'

Network error while fetching http://10.25.100.21:2000/quick/Marshal.4.8/chef-12.3.0-x86-mingw32.gemspec.rz
PierreRambaud commented

Thanks, I will try to reproduce today.

PierreRambaud commented

I can reproduce the bug, but don't really know how to optimize downloads :/

When bundle request the api, it calls all gemspecs at the same time with all version, and some gems such as rake have really many many versions.

For example: /api/v1/dependencies?gems=rake,rack-test,rspec,simplecov,rubocop,fakefs,slop,httpclient,confstruct,builder,sinatra,thin request more than 600 files to be downloaded.

To fix this issue with bundle, you can add gems in config.rb file and run gemirro update and gemirro index --update:

gem 'rake',                    '10.4.2'
gem 'chef',                    '12.3.0'
gem 'berkshelf',               '3.3.0'
gem 'chef-vault',              '2.5.0'
gem 'chef-vault-testfixtures', '0.5.0'
gem 'chefspec',                '4.3.0'
gem 'foodcritic',              '4.0.0'
gem 'test-kitchen',            '1.4.0'
gem 'kitchen-vagrant',         '0.18.0'
gem 'nokogiri',                '1.6.3.1'
gem 'net-ping',                '1.7.7'

Really sorry :/

questers commented

How can I uninstall the version builded from master branch?

I have tested your tip, but get following error on master branch version:

user@vm-projects:/var/lib/gemirro$ sudo gemirro update
I, [2015-07-31T10:49:15.339755 #4608]  INFO -- : Updating rubygems (http://rubygems.org)
I, [2015-07-31T10:49:20.656759 #4608]  INFO -- : Fetching asciidoctor-1.5.0.gem
/usr/lib/ruby/2.1.0/rubygems/requirement.rb:225:in `satisfied_by?': Need a Gem::Version: nil (ArgumentError)
        from /var/lib/gems/2.1.0/gems/gemirro-0.10.5/lib/gemirro/gems_fetcher.rb:31:in `block (2 levels) in fetch'
        from /var/lib/gems/2.1.0/gems/gemirro-0.10.5/lib/gemirro/gems_fetcher.rb:29:in `each'
        from /var/lib/gems/2.1.0/gems/gemirro-0.10.5/lib/gemirro/gems_fetcher.rb:29:in `block in fetch'
        from /var/lib/gems/2.1.0/gems/gemirro-0.10.5/lib/gemirro/gems_fetcher.rb:28:in `each'
        from /var/lib/gems/2.1.0/gems/gemirro-0.10.5/lib/gemirro/gems_fetcher.rb:28:in `fetch'
        from /var/lib/gems/2.1.0/gems/gemirro-0.10.5/lib/gemirro/cli/update.rb:16:in `block (2 levels) in <top (required)>'
        from /var/lib/gems/2.1.0/gems/slop-3.6.0/lib/slop.rb:260:in `call'
        from /var/lib/gems/2.1.0/gems/slop-3.6.0/lib/slop.rb:260:in `parse!'
        from /var/lib/gems/2.1.0/gems/slop-3.6.0/lib/slop.rb:235:in `parse!'
        from /var/lib/gems/2.1.0/gems/slop-3.6.0/lib/slop.rb:211:in `parse'
        from /var/lib/gems/2.1.0/gems/gemirro-0.10.5/bin/gemirro:7:in `<top (required)>'
        from /usr/local/bin/gemirro:23:in `load'
        from /usr/local/bin/gemirro:23:in `<main>'
user@vm-projects:/var/lib/gemirro$

The config output:

# -*- coding: utf-8 -*-
# This is the main configuration file for your RubyGems mirror. Here you can
# change settings such as the location to store Gem files in and what source
# and Gems you'd like to mirror at start.
Gemirro.configuration.configure do
  # Define sinatra environment
  environment :production

  # The directory to store indexing information as well as the Gem files in.
  destination File.expand_path('../public', __FILE__)

  # If you're in development mode your probably want to switch to a debug
  # logging level.
  logger.level = Logger::INFO

  # If you want to run your server on a specific host and port, you must
  # change the following parameters (server_host and server_port).
  #
  server.host '173.17.41.6'
  server.port '2000'
  server.access_log File.expand_path('../logs/access.log', __FILE__)
  server.error_log File.expand_path('../logs/error.log', __FILE__)

  # You must define a source wich where gems will be downloaded.
  # All gem in the block will be downloaded with the update command.
  # Other gems will be downloaded with the server.
  define_source 'rubygems', 'http://rubygems.org' do
    gem 'rack', '>= 1.0.0'
    gem 'asciidoctor', '>= 1.5.0'
    gem 'asciidoctor-pdf', '>= 1.5.0.alpha.8'
    gem 'coderay', '>= 1.1.0'
    gem 'thread_safe', '>= 0.3.4'
  end
end
PierreRambaud commented

Woot, it's a bug (never test with alpha or beta version name :/), prerelease versions aren't loaded, I fix it.
If you want to uninstall gem, you can run gem uninstall GEM_NAME, it will display all available versions on the system and you can select which one you want to uninstall.

Regards

questers commented

thanks for you investigation. I wait for 0.11.0 :)

PierreRambaud commented

Released! Don't hesitate to open a new issue if there is a problem :) Thanks again