[Question] Loading external dependencies

[DannyBen]

Hi,

I hope it is ok to ask a question here. It's just that all the search queries I tried here and on StackOverflow brought me to mostly rails related pages, which is not what I am after.

I have a non-rails project, with many external dependencies. Usually I am doing this in my boot.rb file:

require 'bundler'
Bundler.require :default

This adds about 500ms to my boot time, which I wish to avoid.

Currently I am trying this:

# lazyload dependencies
autoload :Sting, 'sting'
autoload :Sequel, 'sequel'
autoload :RSS, 'rss'
autoload :ActiveCabinet, 'active_cabinet'
# ... more dependencies here

but I wonder if I can simply do this with zeitwerk.

I tried something like this (without "pushing" any directory), in hopes it will surprise me and work out of the box, but it didn't.

require 'zeitwerk'
loader = Zeitwerk::Loader.new
loader.setup # ready!

Is there any way to use zeitwerk like this?

[fxn]

Not really, the way gems load their own code and how are they structured internally is out of your control.

[DannyBen]

I see... didn't realize zeitwerk needs to know anything like that. Doesn't it use Ruby's autoload wherever possible?

In other words, I thought if zeitwek already captures "module not found" errors, it can just require or autoload its downcase snake case:

ActiveCabinet => require 'active_cabinet'
# or
ActiveCabinet => autoload :ActiveCabinet, 'active_cabinet'

This is my best idea on the subject so far:

class String
  def underscore
    gsub(/([A-Z]+)([A-Z][a-z])/,'\1_\2').gsub(/([a-z\d])([A-Z])/,'\1_\2').downcase
  end
end

list = %w[
  RSS
  ActiveCabinet
  Sequel
]

list.each do |gem|
  autoload gem.to_sym, gem.underscore
end

# works
p ActiveCabinet, Sequel, RSS

[fxn]

Projects have to deliberately be thought to use Zeitwerk. They need to conform to the project structure conventions, they need to say what cannot be eager loaded, configure inflections, ....

The way a gem loads its code is private and client code cannot mess with that. You document, in order to use this gem require the entrypoint and use this or this API, for example. What does that do internally? None of the business of client code.

The use case for Zeitwerk is that you as a gem author prefer not to use explicit requires internally, in the gem itself.

Your approach with top-level autoloads should work for what you want, because client code does decide when to load the entrypoint.

[DannyBen]

Thanks for the quick response, I appreciate it.
I will try to cook my own auto-autoloader then :)

[fxn]

Sure thing!

Of course, some gems may have restrictions about when they should be loaded.

For example, in a Rails application it would be generally incorrect to load an engine after the application has booted.

So, client code decides, but, if there are constraints, it is also responsible for respecting them.

Chances are most, if not all, of your dependencies do no have such constraints.

[DannyBen]

Thanks. I do not have these problems. I sort of just wanted to replace all requires with autoloads, and thought there was a way for me to avoid typing that list manually.

That said - I just tested this initial concept, and it seems like Ruby's autoload is not giving me any significant performance boost over plain ol' require. It still takes time, even when the autoloaded classes are not called or used, which was susprising to me.

[fxn]

Just running Bundler.require needs some time:

% time ruby -rbundler -e1
ruby -rbundler -e1  0.08s user 0.03s system 67% cpu 0.165 total
% time ruby -rbundler -e 'Bundler.require'
ruby -rbundler -e 'Bundler.require'  0.20s user 0.06s system 83% cpu 0.313 total

EDIT: Updated the execution of Bundler.require to run with a Gemfile that has only one dependency with require: false.