You cannot specify the same gem twice coming from different sources...

[gshilin]

I need to use gem paperclip. The thoughtbot's version doesn't work properly in windows, while benben's does. So I need something like this as I develop on windows, but deploy on linux:

gem 'paperclip', :git => 'http://github.com/thoughtbot/paperclip.git', :platforms => :mri
gem 'paperclip', :git => 'http://github.com/benben/paperclip.git', :platforms => :mswin

But bundle complains:
You cannot specify the same gem twice coming from different sources.

Any advise?

[indirect]

Well, that's correct -- you can only have one paperclip gem in your Gemfile. If Bundler allowed more than one paperclip gem, it could no longer ensure that your gems would all work together. I suggest you talk to benben, or thoughtbot, or possibly put together your own gem that works on both Windows and Linux? Bundler can't fix broken gems for you, sorry. :(

[myronmarston]

@gshilin--bundler is a great tool but don't forget that you can always manage the load path yourself. Something like this should work:

• Vendor both versions of of paperclip, to directories like vendor/windows/paperclip and vendor/posix/paperclip. You could use git submodules, or clone the repos there and check all the code into your repo.
• Wherever you call Bundler.setup, add the appropriate load path (either vendor/windows/paperclip/lib or vendor/posix/paperclip/lib) based on the platform you are running on.

Definitely not as elegant as using bundler, but it locks you to a version in much the same way that bundler's Gemfile.lock does.

[samoli]

It would be great if this worked.

group :development do
  platform :ruby_18 do
    gem 'mongrel'
    gem 'ruby-debug'
  end

  platform :ruby_19 do
    gem 'mongrel', '1.2.0.pre2'
    gem 'ruby-debug19'
  end
end

In this example, some of our developers are testing with ruby_19 while the rest are working on ruby_18. The ruby_19 guys don't want to hassle the ruby_18 guys with their pre-release gems etc.

[indirect]

samoli, Bundler exists to guarantee that your gems work together. That means it is never going to allow you to declare two of the same gem. It sounds like you want to do something else -- have two separate sets of gems, one for ruby 1.8 and one for ruby 1.9. Testing your app with ruby 1.9 is a great example of a time to use a git branch, so that one set of developers can make changes without interfering with the main development process.

[myronmarston]

I agree with @indirect. I'd also point out that you can use ruby-debug on 1.8 and ruby-debug19 on 1.9 just fine. Even though they are essentially different version of the same gem, they have different names, and bundler treats them as separate gems.

You won't be able to use different versions of mongrel like your example shows, though.

[samoli]

Yes ruby-debug19 works fine, although having the ruby version in the gem name for whatever reason is a bit ugly.

@indirect a branch would be overkill for our use case, as the only difference would be a few lines in the Gemfile

The gems would need to be locked for each platform somehow so it's probably not realistic. It's not that important really, the work arounds are ok.. It's just a "would be nice" if there was a simple way to do it.

[indirect]

How is a git branch overkill? It only stores the diff, which as you just said is only a few bytes. If you want to test and modify your app until it works on ruby 1.9, do it in a branch. The Gemfile is not the place to try to solve your problem.

[samoli]

A branch would be useful in the scenario you are describing, but that's not what we need it for. The app already works fine in 1.9 - the point is we want the same branch to work in both ruby 1.8 and 1.9 without modification of the Gemfile. I think that's reasonable. The problem is really caused by the incompatible gems of course, but this isn't always easily solvable.

As I said it's not a big deal really, but it does seem like the obvious application for the :platform option, otherwise why does it exist?

[indirect]

Just like in the original post, when the Paperclip gem forked into different versions that each only work on one platform, the incompatible gems are really an issue with the gems themselves. Unfortunately, that means they are outside the realm of what Bundler can fix.

The platform option exists so you can include gems that only work on one platform. It doesn't exist so you can declare the same gem more than one time. As I said before, the entire reason for Bundler's existence is to guarantee a single cohesive set of gems (plus or minus specific gems per platform). If we allow Gemfiles to declare the same gem more than once, that entire idea breaks down, and Bundler just becomes a shortcut for gem install. If that's all you want, please just write a rake task that calls gem install. I hope that helps clear things up. :)

[nirvdrum]

I'm confused. Why is it bundler can delineate when different gems are specified with platform sections, but not when the same gem name is specified? Couldn't it just segment each into the different platforms and only activate the relevant section when run on the same platform?

I wholly disagree with the git branch solution. I want to ensure I can run on REE, YARV, and JRuby. Different parts of the app should be able to run in different ruby versions. The git branch approach only works if you're moving wholly to one or the other. And obviously keeping three different branches in sync and deploying three different branches makes minimal sense.

[nirvdrum]

Also, I'm confused by the original argument. Bundler makes no effort to ensure my gems all work together. It merely resolves dependency chains, but those gems are certainly able to crap on each other.

[indirect]

@nirvdrum, some gems are badly-behaved and declared themselves to work on any platform when in reality they only work on one platform. Bundler's platform functionality exists so that you can conditionally disable those gems that only work on a single platform. Bundler does not and will not ever allow multiple versions of the same gem to be switched between depending on... well, anything, really. Because that would destroy the entire reason Bundler exists, which is to provide as strong a guarantee as possible that the code running in production is the same code running in development and everywhere else.

[nirvdrum]

I guess I'm just being really dense. I fail to see how if I'm running on different platforms in production, testing on different platforms in a CI server, and run different platforms locally that this voids the contract you're implying. Likewise, it's not all that difficult to maintain two different dependency graphs and activate one over the other depending on platform being run. I'd argue that it's much better for an app to be able to run on as many different ruby versions as possible -- that shouldn't be seen as a negative.

Perhaps it wouldn't be so painful if merging Gemfile.lock's didn't result in a guaranteed conflict on almost any modification . . .

But just so I'm fully educated, this isn't a technical argument, it's a conscientious design decision?

[indirect]

You can run on REE, YARV, and JRuby. You just need gems that are either 1) compatible with all three rubies or 2) compatible with only one ruby, so they can be disabled on the other rubies. If you have three versions of the same gem, one for each ruby, that is a problem with the gem, and it should be fixed.

If you want multiple dependency graphs, create multiple Gemfile.locks. Merging Gemfile.locks via diff is generally a bad idea, do this instead:

git checkout HEAD -- Gemfile.lock
bundle install
git add Gemfile.lock

Bundler's entire architecture from the beginning has revolved around the idea that it would guarantee you got the same code (or a subset of the same code) everywhere. Multiple gem versions in a single Gemfile violates that guarantee, so it's not something Bundler can allow while staying true to its core purpose.

[nirvdrum]

If that's true, then bundler shouldn't have :platform at all since you're distinctly violating that guarantee that development matches production. I've already screwed with the dependency graph at that point.

No one's arguing that the gem isn't broken. What I'm trying to do is fix the gem. So, I forked a gem that only works in REE, I have it working in JRuby. I'm trying to merge the work. It's a tad foolish I can't work on my app until I make that gem work with both.

For now I'll just give the gem a different name, continue to use the git repo, and then change the :require line, trivially circumventing the limitation. But if that's all it takes, and this supposedly supports bundler's contract, then I don't really understand bundler's contract. Or more to the point, I fail to see how allowing replacement of a subcomponent of the dependency graph breaks anything that you're stating is bundler's contract. I'm pretty sure that's provably not the case.

[nirvdrum]

Sorry, I don't mean to come off as combative. I'm really just thinking about the problem from both the CS sense (thus the provable portion re: graph theory) and from a realistic application.

[indirect]

The platform option stretches the guarantee to "or a subset of". In a perfect world, Gemfile platforms wouldn't exist. A subset of the code is as far as the Bundler team is willing to go, which is why it has platforms but doesn't allow multiple versions of the same gem.

Multiple versions of the same gems opens such a pandora's box of things that could go wrong that it simply isn't possible for us (the unpaid completely volunteer Bundler team) to support such a thing. In a similar vein, we want to use Bundler to encourage better gem behaviour without completely blocking workarounds. It is of course entirely possible to circumvent via forks and gem names, but that is the only workaround that we are can to put time and effort into supporting.

[bigtunacan]

I understand bundler is to make sure we have the correct version and only one version of a gem. However I anticipated the following would work.

group :production do
     gem "foo", :git => "https://github.com/me/my-production-version-gem.git"
end

group :test, :development do
     gem "foo", :git => "https://github.com/me/my-development-version-gem.git"
end

I would expect this to work since only 1 version of the gem will be installed at any given time; with the version installed being dependent on the current group.

[indirect]

That won't work. Sorry!

On Mon, Aug 5, 2013 at 8:26 AM, bigtunacan notifications@github.com
wrote:

bundler is to make sure we have the correct version and only one version of a gem. However I anticipated the following would work.

group :production do
     gem "foo", :git => "https://github.com/me/my-production-version-gem.git"
end
group :test, :development do
     gem "foo", :git => "https://github.com/me/my-development-version-gem.git"
end
---
Reply to this email directly or view it on GitHub:
https://github.com/bundler/bundler/issues/751#issuecomment-22113199

[bigtunacan]

Yeah; what I just ended up doing something like this instead.

foo = "https://github.com/me/my-development-version-gem.git"
group :production do
    foo = "https://github.com/me/my-production-version-gem.git"
end
gem "foo", :git => foo

This is an effective workaround for anyone trying it. It is still just ruby after all ;)

[TimMoore]

@bigtunacan I don't think that will do what you want. The blocks defining each group are evaluated regardless of whether the group is being installed:

$ cat Gemfile
# A sample Gemfile
source "https://rubygems.org"
group :production do
  puts "HI"
end

# gem "rails"
(devmac-12) (16:29:41) ~/projects/tmp 1.8.7
$ bundle install --without production
HI
The Gemfile specifies no dependencies
Your bundle is complete!
Gems in the group production were not installed.
Use `bundle show [gemname]` to see where a bundled gem is installed.

[timboisvert]

This needs to be reopened, for no other reason than the fact that there are a number of people who WANT this functionality. I understand some of you responding that it doesn't work that way, but clearly there are legitimate reasons why it COULD or SHOULD support this option.

[xaviershay]

No one on the core team is going to do anything on this.

If you want to further the debate, working code would be the best avenue.

[bigtunacan]

@TimMoore, thanks for pointing that out. Good thing I had it set so the failure was in dev rather than when going to deploy to prod...

@timboisvert I agree there is a very valid need for this. The common case I run into, and I imagine others, is that I have internal gems that are in flux for projects. While in development I want my development nightly gem builds so I am constantly testing out the gem via real world use; when I release to production I want the stable build of my gem.

However, this is OSS; if the users (us in this case) have a need that the maintainers don't then the expectation is not that they will suddenly jump to it and add these features for us, and free of charge to boot, that they don't need.

Then as @xaviershay said, producing a working code for the feature is the best approach. I do however have concern with working on an issue such as this, issuing a pull request only to have it rejected by the core team. I would want to know up front if the core team is open to this functionality being added to bundler, and if so, if they have any direction/expectation of how it would best integrate with the project today.

[radar]

@bigtunacan It sounds like what you're asking for could be accomplished using Bundler local overrides. See "Local Git Repos" on this page.

[radar]

The main issue that I see with adding new features to an OSS project is that the maintainers of that project, not the people who are requesting the features, are stuck with maintaining that feature for its life. Is it worth having the feature in the project to support these few edge cases of typical Bundler usage? We've deemed the answer to that question to be "no".

I am stunned that @timboisvert would write such a vitriolic reply (deleted, but still arrived in the inboxes of the Bundler team) when he himself has not contributed anything to the project other than armchair criticism.

This is Ruby; we are better than that.

[timboisvert]

@bigtunacan No one expects them to just do it for us even if they have no need for it. Based on the communication from "them" in this thread, though, I don't have much confidence that they'd be willing to accept it even if those of us who did need it were willing to contribute it. Therefore it'd be a waste of time to even lay down even a single line of code aimed at fixing this problem. And regardless of what people like @xaviershay say, this is indeed a valid problem. You pointed out the same use case that my team and I have -- It's a royal PITA to test some gems (say, for example, a Rails gem) in a real-world example without pointing to a local path. We have a line in a Gemfile that gets commented in and out no fewer than 50 times a day as a result. Here's the process:

1. Set host app's Gemfile to use the local path for the gem.
2. Bundle update.
3. Make changes to the gem and test within the host app.
4. Commit the gem to Github.
5. Switch host app's Gemfile to use the Github path for the gem.
6. Bundle update.
7. Commit the new Gemfile and Gemfile.lock.
8. Deploy to Heroku.
9. CRAP, forgot an edge case! Set host app's Gemfile to use the local path for the gem.
10. Bundle update.
11. Make changes to the gem and test within the host app.
12. Commit the gem to github.
    etc., etc.

Try that 50 times a day. That's reality. Now, if only the same gem (with different source paths or repos) could be declared in different environments within the Gemfile, this entire cycle could be avoided. As far as ensuring that the set of gems is consistent between environments is concerned, forcing bundler to "update" whenever it's called from a different environment than the previous bundle would keep the set of gems sane.

I'm grateful for the work done by open-source communities like this one. There seems to be an undercurrent in this thread, though, that suggests that my experience with Bundler in this particular scenario isn't a valid reason to want the tool changed. I'd throw code at solving the problem but based on this thread the supposed core team doesn't think this is a problem worth solving.

[radar]

@timboisvert I believe what you're saying can be solved also with the "Local Git Repos" section that I linked to earlier. It's really simple.

By doing it this way you only need to have the path to GitHub in your Gemfile. For example, on Spree I have this:

gem 'spree', :github => 'spree/spree', :branch => 'master'

To use my local copy, all I did was:

bundle config local.spree ~/Projects/gems/spree

So I develop Spree locally, make a couple of commits and push them to GitHub. To deploy my test application, I'd need to run bundle update spree to make sure it has the correct ref in Gemfile.lock, and then I commit the Gemfile.lock to GitHub. Deploying the app to the server will be different from local, because the server isn't set up to use a local copy of Spree, and so it will attempt to clone the repo from GitHub. Assuming that I pushed the commits I needed to Spree and I updated the Gemfile.lock correctly, everything will go smooth.

If other people pull down my app and run bundle install, it will install Spree from GitHub also, unless they too have set Bundler up to use a local copy.

There's no need to do all that changing of your Gemfile when you want to switch between local and remote any more. I think this was a feature introduced in 1.3, although don't quote me on that.

[timboisvert]

@radar Vitriol wasn't my aim, so I deleted. Surely you've thought better of a reply at times after sending it. My apologies for a response that clearly came out wrong. @bigtunacan's response gave me an opportunity to state my issue in a more reasoned tone.

And I hear your point regarding maintenance. No one wants to leave a burden behind long after they've moved on to other needs. I'm an OSS supporter to the core (my employment history validates that), but even I know that community requires thoughtful and open communication. The responses to a valid question by OP were met with snark and disinterest from the beginning. You (the general you) simply can't respond to users of your tool the way some people respond here and expect new community members to actually want to contribute to that project.

Bundler's a great tool, I just feel dissatisfied with the tone suggesting that my need (and the need described by OP and others in the thread) wasn't a real one.

[timboisvert]

@radar Awesome! I'll try that first thing in the morning. Thank you!

[bigtunacan]

@radar Thanks for the link; I was unfamiliar with that feature. That should do the trick for us currently.

Now back to writing more rules in Prolog; yes for realz...