Collapse Key Dependencies via Sprockets?
wied03 opened this issue · 21 comments
Not if this has been thought of or is feasible but most of the time, when people are developing with Rails, etc., they may not need each Opal (or for that matter RSpec) file supplied to their browser as an individual file. That creates a lot of requests, which even with caching/304 responses, slows down the browser during development (production is not a problem because everything gets precompiled).
Might it be possible to tweak Sprockets to make this happen? It would be kind of an asset precompile on the fly but only for certain assets.
I know it's easy to programmatically compile using the Sprockets::Manifest class but that's the easy part. The hard part is changing the behavior as assets are required/served. It can't be done in Sprockets::Server because that's too late in the cycle.
I suppose this would need to be done in Opal::Processor under sprockets by overriding the evaluate method and, if the filename includes 'opal.rb', spin up a separate sprockets context that precompiles that file, then returns that.
Is that a good approach? Is this a problem worth solving?
That's surely a valuable proposal, especially for speeding up requests in (sprockets) debug mode by cutting down the number of requests.
I see two ways to do this, the first is adding some kind of option to the processor that will render opal as a single file with no dependencies, that would go directly into opal.
The other is to add something like opal-collapsed.js.erb to opal-rails implemented as follows:
//= depend_on opal
<%= Rails.application.assets['opal'].to_s %>@wied03 @adambeynon any preference?
@elia, I suppose I didn't think about how easy it is (using the erb approach you described) to get sprockets to dump the entire thing to a string.
That said, it seems like both of these approaches would be needed. Otherwise, how would you hook into Sprockets when it's processing all of the app source that includes require 'opal' ? You would still need to do that using some approach that tweaks the Processor, right?
It also might not be just Opal (I could think of doing this for opal-rspec right out of the gate as well).
This sounds much alike the framework blackboxing feature in Chrome, we could have a list of assets for which the collapsing is on:
Opal::Config.collapsed_paths = %w[opal opal-rspec]About the processor approach I'd have to try first but I think the context environment can be used to replicate basically the same thing sprockets does (pseudocode):
if Opal::Config.collapsed_paths.include? logical_path
dependencies_code = dependencies.map {|d| context.environment[d.logical_path].to_s}
result = "#{dependencies_code}#{result}"
endOn the other hand the soft .js.erb approach can be extended at will to any asset
@elia - Yeah, Rails.application.assets is a Sprockets::Environment object so that would make sense.
Remaining questions:
- Should this go in core opal? Seems like the answer is yes since that's where the Sprockets code is
- OK to go ahead with a pull request for core opal?
- It could be broken out into 2 settings: collapsed_paths and a enable_collapse. Separate from that, what should the default be? I would imagine that not collapsing by default is more sensible than forcing it.
I'd really like to hear @adambeynon thoughts on this, I started experimenting with the erb solution in a project (using exactly the code above) and works great.
The more I think about it the more comes to mind that should be a sprockets (or sprockets-rails) feature instead. I'll open an issue there and see what they think.
In the meantime if you need this kind of feature on some project you can go with the .js.erb solution until this is sorted out.
That would be good. In trying to do this in the meantime (with opal-rspec under opal-rails, I'll do the actual Rails app run later), I've also noticed a lot of the opal requires are unpacked before Opal::Processor is ever called in this scenario but I can't figure out exactly how (I know OpalSpecController creates the high level temp file using Opal::Rails::SpecBuilder but I can't track down what converts that to JS. If I set breakpoints in Opal::Processor.call or Opal::TiltTemplate I never see that temp file come in). The temp file looks fine (example below) I just can't figure out what is exploding the first require 'opal'. Any idea here?
temp file contents (tmp/opal_spec/opal_spec_runner_a884f369a73a02d07b78915eceaed573d85032af.js.rb)
require "opal"
require "opal-rspec"
require "components/base/html_ajax_spec"
require "components/base/option_spec"
require "components/base/select_label_spec"
require "components/base/select_spec"
require "components/form/space_spec"
require "components/mixins/collection_upfront_spec"
require "components/mixins/html_spec"
Opal::RSpec::Runner.autorun
Ouch
Yes, that part is quite messy, let me see if I can fix it tonight
Ideally I'd require the spec_helper to "require" Opal-rspec. That way you're free to require it as opal-rspec-collapsed.
@elia I also sent you an email about an opal-rspec PR (opal/opal-rspec#19). I don't think that's relevant here but wondered what the thoughts were on that
@elia If things are changing towards a combined asset anyways, then maybe this problem mainly sticks around on the testing side. One possible option is to do something like Karma does where you keep a 'server' process and browser (phantom) sitting there running and can re-run tests quickly.
I don't know exactly how that works (phantom constantly polling or what not). I guess the Rails server would need to stay up and then the Rake task. I would say maybe Spring can help here but I'm not sure where the delay is exactly.
I've been using the collapsed (via erb) version for a while now and the difference is quite noticeable. I'll try to bake the collapsing feature right into the processor so that any lib can be processed that way.
How do you do use that exactly? Do you go and replace all the require 'opal' statements in your code with require 'opal-collapsed'?
Yes, but I tipically have just one require for opal
We were discussing this on the gitter site, and I had the thought (not sure if its relevant or practical:) Could the configuration default be to detect which "mode" (i.e. debug, or production) based on the location of the gem files. If the gem files were in the normal gem directory then everything gets compressed, and there are no debug source maps. If the the gem is not in the "normal" directory (i.e. you are referencing the gem on your local machine) then it uses the rails development/production setting.
I'm just not sure if you can detect where the gem source files are, and to evaluate from the location if you are doing something like gem 'opal-react', path: '../..' but this would be what I think most people would want / expect.
@catmando - That might be kind of difficult trying to guess that. I think detecting what environment you're in or DEBUG or not is the least of the problems. The bigger problem is what to do when you already know you should collapse (or not). It seems like, given the discussion above, it might make more sense to let the Sprockets dust settle w/ collapse by default + source maps before investing much here.
@elia - What about just going the direction sprockets is with source maps? I set config.assets.debug=true and that rolls everything up to 1, non-minified file. The source map if in opal-rails' engine.rb would need to be modified to ignore that debug flag. That aside, source maps didn't start working. I did some tinkering with that a couple months ago but now I forget what I did to make source maps work.
@wied03 remembering?
I did some tinkering with that a couple months ago but now I forget what I did to make source maps work.
Forgot Sprockets - best solution )))
Yeah, when I'm running opal-rspec tests, which is the main case I want source maps for, I started using webpack. On the runtime side, it's still sprockets.
Seriously!
can Opal team drop sprockets dependency?