Unload plugin does not reset ActionController callbacks (e.g. after_action)

[joshbuker]

Please complete all sections; bug reports that cannot be reproduced will likely
be closed.

Describe the bug
When using unload_plugin to remove a plugin that has ActionController callbacks, the callbacks are not properly reset.

To Reproduce
Steps to reproduce the behavior:

1. Inside of ApplicationController:

authenticates_with_sorcery! do |config|
  config.load_plugin(:activity_logging)
end

2. Inside of a child controller (e.g. AdminSessionsController < ApplicationController)

authenticates_with_sorcery! do |config|
  config.unload_plugin(:activity_logging)
end

Expected behavior
Plugin callbacks do not get ran in controllers that unload the plugin.