Loading shadowed files (QUESTION)
catmando opened this issue · 10 comments
I have a use case where it is needed to allow shadow files to be loaded. For details on why see below.
To do this I am using the loader's on_load
callback to detect when the primary file is loaded, then checking to see if there is a shadowed file in the appropriate directory, and then calling require
on the shadowed file.
This seems to work fine (and I have been doing a similar thing in classic mode) but I am wondering if there are any negative implications within Zeitwerk.
BTW: It was very nice implementing this in Zeitwerk as it required no monkey patches (as I had to do in classic mode.). Seems like a great step forward, thanks!
WHY AM I DOING THIS (IN CASE YOU CARE)
The Hyperstack Framework (a client-server isomorphic extension to Rails) allows for shadowed Ruby files.
This allows a model definition to have a server-only part and a client-and-server part. This is generally needed in some models
and services for efficiency and security.
The Hyperstack directory structure (within a Rails app) looks like this:
--app
|- hyperstack
|- components
|- models
- some_model.rb
....
|- ...
|- models
- some_model.rb
....
In this case, the definitions in app/hyperstack/models/some_model.rb
will be available on both the server and client (i.e. browser) while the definitions in app/models/some_model.rb
will only be seen (per standard rails) on the server.
For details see: Hyper Model Documentation
Interesting!
Couple of questions come to mind.
Do you really need app/hyperstack/models
in the autoload paths? Would it make sense to configure the autoloader to ignore that directory?
I am wondering because if that was possible, the main file would be better defined as the only one managed by Zeitwerk. Otherwise it depends on the order of configuration of the root directories.
Also, could be intentional to ignore..., but depends on usage.
Another question would be related to idempotence. If you require
the shadowed file, on reload that require
won't interpret the file again.
Let me share these first thoughts and we can iterate from there :).
Thank you so much for you attention to this.
1). Both directories need to be autoloaded.
Perhaps a simple but dumb example will make it clear why:
# app/hyperstack/models/sample.rb
class Sample < ApplicationRecord
end
# app/models/sample.rb
class Sample < ApplicationRecord
def self.super_secret_server_side_method
true
end
end
Here the Sample
model will be accessible on the server as normal, but will also be compiled (via Opal) for the client. So all the normal Rails ActiveRecord methods will be visible on both server and client. For example you might say Sample.create
on the client to create a new record.
But for whatever reason we want super_secret_server_side_method
to only be accessible on the server. This is typically either because the method(s) have complex computations that are best done on the server, or for security reasons, the code should not be exposed or executed on the client.
Thus we can optionally split definitions across two files. This by the way also applies to other directories.
2). Reloading
Yes this was what I was afraid of. However when a reload occurs does the on_load
callback get called again? If so then all should still work, as that is how I know to look for the shadowed files. If on_load
is not called, is there another hook that could be used to tell us that file is getting reloaded?
But, in your example, app/hyperstack/models/sample.rb
is not autoloaded by Zeitwerk, it is manually autoloaded by your callback. That is the key observation. Question would be, do you have anything in app/hyperstack/models
that is not also present in app/models
and needs to be autoloaded by Zeitwerk regularly?
Yes, on_load
is called when the class is loaded again after a reload too.
Hmm, after reading again the first message, perhaps I got it backwards. Which of the two sample.rb
is considered to be the primary file?
the app/hyperstack/ directory is first, so it is "shadowing" any other directories in app.
So the order that things are loaded in the example are:
app/hyperstack/models/sample.rb
when this loaded the on_load
handler then discovers that
app/models/sample.rb
also exists, and so does a require of that file
and btw if on_load is called then everything should work (I think)
OK, then the actual question is: do you need app/models
to be in the autoload paths?
and btw if on_load is called then everything should work (I think)
In principle, Kernel#load
would be the way to go.
Hey, do you need more help with this?
@fxn apologies, I was out hiking! I think its all set, but I didn't understand you comment about Kernel#load
. Are you saying use that instead of require
inside the on_load handler? I am using require
now, and all is well but I will try load and see if there is any difference.
Thanks again for all the help, and making such a nice robust API.
FYI for reference here is the handler, in case it helps someone else, or you spot a problem I might have missed:
Note this works for Rails >= 5.0 / classic or zeitwerk.
# require "hyperstack/server_side_auto_require.rb" in your hyperstack initializer
# to autoload shadowed server side files that match files
# in the hyperstack directory
if Rails.configuration.try(:autoloader) == :zeitwerk
Rails.autoloaders.each do |loader|
loader.on_load do |_cpath, _value, abspath|
ActiveSupport::Dependencies.add_server_side_dependency(abspath) do |load_path|
loader.send(:log, "Hyperstack loading server side shadowed file: #{load_path}") if loader&.logger
require("#{load_path}.rb")
end
end
end
end
module ActiveSupport
module Dependencies
HYPERSTACK_DIR = "hyperstack"
class << self
alias original_require_or_load require_or_load
# before requiring_or_loading a file, first check if
# we have the same file in the server side directory
# and add that as a dependency
def require_or_load(file_name, const_path = nil)
add_server_side_dependency(file_name) { |load_path| require_dependency load_path }
original_require_or_load(file_name, const_path)
end
# search the filename path from the end towards the beginning
# for the HYPERSTACK_DIR directory. If found, remove it from
# the filename, and if a ruby file exists at that location then
# add it as a dependency
def add_server_side_dependency(file_name, loader = nil)
path = File.expand_path(file_name.chomp(".rb"))
.split(File::SEPARATOR).reverse
hs_index = path.find_index(HYPERSTACK_DIR)
return unless hs_index # no hyperstack directory here
new_path = (path[0..hs_index - 1] + path[hs_index + 1..-1]).reverse
load_path = new_path.join(File::SEPARATOR)
return unless File.exist? "#{load_path}.rb"
yield load_path
end
end
end
end
@catmando awesome re hiking! :)
Sure, let me explain. Let's consider the Sample
model.
- The model is somehow loaded for the first time.
- The callback notices it has to decorate it.
- Evaluates the companion file with
Kernel#require
.
Cool, now we reload. The original Sample
constant is removed from Object
.
Sample
is referenced an loaded again.- The callback notices it has to decorate it.
- Evaluates the companion file with
Kernel#require
.
Now, (3) is no longer able to evaluate the file because Kernel#require
is idempotent, so Sample
is not extended anymore on reloads.
On the other hand, if (3) is done with Kernel#load
, it always executes the file.
(Note that require_or_load
is private interface, and only used by classic
.)