acquia/blt

Resolve "Could not scan for classes inside" notices

Closed this issue ยท 36 comments

This is a follow-up to #3120: in BLT 10.0.x. these notices are generated on composer installs. See that issue for more information.

You can see an example of these errors in our tests: https://travis-ci.org/acquia/blt/jobs/438701163#L1411

source:build:composer
[Exec] Running export COMPOSER_EXIT_ON_PATCH_FAILURE=1; composer install --ansi --no-interaction in /tmp/blt-sandbox-instance
Could not scan for classes inside "/tmp/blt-sandbox-instance/vendor/drupal/core/lib/Drupal.php" which does not appear to be a file nor a folder
(etc)

Pasting comment from #3120:

One issue I haven't resolved: adding drupal/core (or anything that requires it) to BLT's own composer.json causes these warnings during Composer operations:

Could not scan for classes inside "/tmp/blt-sandbox-instance/vendor/drupal/core/lib/Drupal.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/tmp/blt-sandbox-instance/vendor/drupal/core/lib/Drupal/Component/Utility/Timer.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/tmp/blt-sandbox-instance/vendor/drupal/core/lib/Drupal/Component/Utility/Unicode.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/tmp/blt-sandbox-instance/vendor/drupal/core/lib/Drupal/Core/Database/Database.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/tmp/blt-sandbox-instance/vendor/drupal/core/lib/Drupal/Core/DrupalKernel.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/tmp/blt-sandbox-instance/vendor/drupal/core/lib/Drupal/Core/DrupalKernelInterface.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/tmp/blt-sandbox-instance/vendor/drupal/core/lib/Drupal/Core/Site/Settings.php" which does not appear to be a file nor a folder

All tests still pass and it seems to have no functional impact, but it's very annoying and I cannot figure out how to resolve it.

So yes, it's EXACTLY the same missing classes we're running into with BLT 9.2.0. I need to mention it's happening both locally and with the Pipelines build.

EDIT: can we label this as a bug for 9.2.0 as well? Otherwise I'm afraid we might not consider this more urgent than it actually is.

Just noting that I am seeing this too.

Just want to mention that the reason for this happening in 9.2.x very well may be different than the reason in 10.0.x.

There aren't many Composer related differences between 9.1.x and 9.2.x, see:
https://github.com/acquia/blt/compare/9.0.x...9.2.x?expand=1#diff-b5d0ee8c97c7abd7e3fa29b9a27d1780

I'd consider the following debugging steps:

  • Remove dependency on webflo/drupal-core-require-dev
  • Downgrade to drupal/core 8.5.x
composer require drupal/core:~8.5.0 --no-update
rm -rf docroot/core vendor composer.lock && composer update
composer dump-autoload

Fixes the issue.

The issue reappears after:

composer require drupal/core:^8.6.2 --no-update
rm -rf docroot/core vendor composer.lock && composer update
composer dump-autoload

A comparison of the changes to Drupal core's composer.json 8.5.x ... 8.6.x:

drupal/core@8.6.x...8.5.x#diff-b5d0ee8c97c7abd7e3fa29b9a27d1780

The merge plugin changes look a bit suspicious.

I wonder if moving the "classmap" values from Drupal core's composer.json into the "child" composer.json files would fix it. E.g., add 'Timer.php' to 'classmap' in "core/lib/Drupal/Component/Utility/composer.json".

Note that the above debugging steps only work on 9.2.x.

The issue does not seem to be caused by BLT nor Drupal itself.
I've been using BLT 9.2.0 and Drupal Core 8.6.2 and I start getting the warnings only after I update the following packages:

composer update
Gathering patches for root package.
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 0 installs, 2 updates, 1 removal
  - Removing symfony/polyfill-php72 (v1.10.0)
Gathering patches for root package.
Gathering patches for dependencies. This might take a minute.
  - Updating symfony/var-dumper (v4.1.6 => v3.4.17): Loading from cache
  - Updating drush/drush (9.4.0 => 9.5.2): Loading from cache
Writing lock file
Generating autoload files

I'm not sure if that information can help to debug.

I do think this is caused by the change to Drupal core. But, it only shows up after a set of strange conditions are met. I think it might require the autoload class map being dumped multiple times.

Began seeing this issue when performing composer install, update and dump-autoload operations. The project was using:

acquia/blt:9.1.8
composer/installers:1.6.0
drupal/core:8.5.8
wikimedia/composer-merge-plugin:1.4.1

It is indeed related to the drupal/core classmap and an interaction between acquia/blt and/or wikimedia/composer-merge-plugin not respecting the custom install locations defined for the composer/installers plugin.

I found the warnings shown twice, once by BLT and once by the merge plugin. I tried running composer without plugins, which would often reduce duplication of warnings, but they still remained.

composer update --no-scripts
composer update --no-plugins
composer update --no-plugins --no-scripts 

To solve this issue we have taken these steps:

  1. Moved any missing dependencies or configurations from blt/composer.required.json into project root composer.json
  2. Removed extras.merge-plugin.require from root composer.json
  3. Created a custom patch to acquia/blt that removes the wikimedia/composer-merge-plugin dependency
  4. Deleted the composer lock file and installed dependencies git clean -Xdff -e '!/.idea/' -e '!/.idea/*' -e '!/.vagrant/' -e '!Vagrantfile'
  5. Updated everything from scratch composer update

We now have no more warnings.

Update: looks like the merge plugin is still there, but it's weirdness is curtailed since we made the changes above.

# composer why wikimedia/composer-merge-plugin
acquia/blt  9.1.8  requires  wikimedia/composer-merge-plugin (^1.4.1)

I wonder why this occurs on the 10.x branch given that it does not use the merge plugin.

Well, being that this has come back now for us, ugh, I have a theory. You see it was all fine, until we went to install another composer dependency, then it came back again.

theory

Not sure how to test this theory just yet, but this is it:

BLT (and maybe one other plug-in) contains some composer event listeners, that fire during update, install operations. It's as if there is something in there that triggers the traversal of the file-system, looking for the Drupal core classmap files. This is happening though, before the composer-installers plugin has had a chance to alter the file-system locations for drupal/core based on the project root composer.json config.

how to test?

I'll try using debug/verbose composer output.
Something else to try will be putting in breakpoints and then looking at the call-stack to find out where this problem surfaces.

Iโ€™ve actually tried removing Plugin.php entirely and I find thay the notice still appears.

I've tried running composer commands, with the --no-plugins option. What I find is that the warnings still appear, but they will sometimes go from two or three warning blocks, down to one.

The classmap is stored in the vendor/composer/installed.json and the paths used there account for the first block of warnings. After that, when a plugin is fired, another warning block is shown for me. If I hand-change the paths in the vendor/composer/installed.json the first block of warnings goes away. So, it's an odd one.

Dropping a note here that I only started seeing these after upgrading from drush 9.4.0 to 9.5.x and greater, as did @rumyanar in the comment above. I can consistently reproduce downgrading to drush 9.4.0 solving the issue.

The same error was in drupal-vm project.
The solution was:

  • composer self-update
  • composer clearcache
  • rm -Rf docroot/core
  • rm -Rf vendor
  • composer install

geerlingguy/drupal-vm#1439

After updating Composer to 1.8.4 I haven't see these errors during composer install and composer update. Strangely, though, I now see them when I run composer --version and composer clear-cache.

@danepowell I'm still seeing these on 1.8.4. What version of drush are you on?

these messages are due to inconsistencies in Composer cache clearing and autoload dumping which is often exacerbated by the user running composer on both the host and within a vm, and having global composer packages installed on either.

If you run your offending composer command with -vvv you will see the exact point and file path where this is occurring - it is usually cached --dist repo archives in ~/.composer which have downloaded core via composer install where a package that has a hard dependency in the root project directory's composer.json of drupal/core where the composer merge plugin is also being used. There are a few ways to resolve the specific autoloading and cache config/dumping so if anyone on this thread having issues could post the output of their composer command with -vvv, it will help inform possible solutions and if this is something that can even be addressed in BLT.

Well, here's my output:

$ composer -V -vvv
Reading ./composer.json
Loading config file /Users/travis.carden/.composer/config.json
Loading config file /Users/travis.carden/.composer/auth.json
Loading config file ./composer.json
Checked CA file /private/etc/ssl/cert.pem: valid
Executing command (/Users/travis.carden/Projects/Work/orca-build): git branch --no-color --no-abbrev -v
Executing command (/Users/travis.carden/Projects/Work/orca-build): git describe --exact-match --tags
Reading /Users/travis.carden/.composer/composer.json
Loading config file /Users/travis.carden/.composer/config.json
Loading config file /Users/travis.carden/.composer/auth.json
Loading config file /Users/travis.carden/.composer/composer.json
Reading /Users/travis.carden/Projects/Work/orca-build/vendor/composer/installed.json
Reading /Users/travis.carden/.composer/vendor/composer/installed.json
Could not scan for classes inside "/Users/travis.carden/Projects/Work/orca-build/vendor/drupal/core/lib/Drupal.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/Users/travis.carden/Projects/Work/orca-build/vendor/drupal/core/lib/Drupal/Component/Utility/Timer.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/Users/travis.carden/Projects/Work/orca-build/vendor/drupal/core/lib/Drupal/Component/Utility/Unicode.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/Users/travis.carden/Projects/Work/orca-build/vendor/drupal/core/lib/Drupal/Core/Database/Database.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/Users/travis.carden/Projects/Work/orca-build/vendor/drupal/core/lib/Drupal/Core/DrupalKernel.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/Users/travis.carden/Projects/Work/orca-build/vendor/drupal/core/lib/Drupal/Core/DrupalKernelInterface.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/Users/travis.carden/Projects/Work/orca-build/vendor/drupal/core/lib/Drupal/Core/Site/Settings.php" which does not appear to be a file nor a folder
Loading plugin Acquia\Blt\Composer\Plugin
Loading plugin Composer\Installers\Plugin
Loading plugin cweagans\Composer\Patches
Loading plugin Dealerdirect\Composer\Plugin\Installers\PHPCodeSniffer\Plugin
Loading plugin DrupalComposer\DrupalScaffold\Plugin
Loading plugin grasmash\DrupalSecurityWarning\Composer\Plugin
Loading plugin OomphInc\ComposerInstallersExtender\Plugin
Loading plugin zaporylie\ComposerDrupalOptimizations\Plugin
Loading plugin Hirak\Prestissimo\Plugin
Loading plugin Localheinz\Composer\Normalize\NormalizePlugin
Running 1.8.4 (2019-02-11 10:52:10) with PHP 7.1.25 on Darwin / 18.2.0
Composer version 1.8.4 2019-02-11 10:52:10

Adding in mine as well:

vagrant@local:/var/www/mis$ composer -V -vvv
Reading ./composer.json
Loading config file ./composer.json
Checked CA file /etc/ssl/certs/ca-certificates.crt: valid
Executing command (/var/www/mis): git branch --no-color --no-abbrev -v
Reading /home/vagrant/.composer/composer.json
Loading config file /home/vagrant/.composer/composer.json
Reading /var/www/mis/vendor/composer/installed.json
Reading /home/vagrant/.composer/vendor/composer/installed.json
Could not scan for classes inside "/var/www/mis/vendor/drupal/core/lib/Drupal.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/var/www/mis/vendor/drupal/core/lib/Drupal/Component/Utility/Timer.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/var/www/mis/vendor/drupal/core/lib/Drupal/Component/Utility/Unicode.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/var/www/mis/vendor/drupal/core/lib/Drupal/Core/Database/Database.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/var/www/mis/vendor/drupal/core/lib/Drupal/Core/DrupalKernel.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/var/www/mis/vendor/drupal/core/lib/Drupal/Core/DrupalKernelInterface.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/var/www/mis/vendor/drupal/core/lib/Drupal/Core/Site/Settings.php" which does not appear to be a file nor a folder
Loading plugin Acquia\Blt\Composer\Plugin
Loading plugin Composer\Installers\Plugin
Loading plugin cweagans\Composer\Patches
Loading plugin Dealerdirect\Composer\Plugin\Installers\PHPCodeSniffer\Plugin
Loading plugin DrupalComposer\DrupalScaffold\Plugin
Loading plugin grasmash\DrupalSecurityWarning\Composer\Plugin
Loading plugin OomphInc\ComposerInstallersExtender\Plugin
Loading plugin Wikimedia\Composer\MergePlugin
Loading plugin zaporylie\ComposerDrupalOptimizations\Plugin
Loading plugin Hirak\Prestissimo\Plugin
  [merge-plugin] Loading blt/composer.required.json...
  [merge-plugin] Prepending composer repository
  [merge-plugin] Merging drupal/core
  [merge-plugin] Ignoring duplicate drupal/config_split
  [merge-plugin] Merging php-http/guzzle6-adapter
  [merge-plugin] Loading docroot/modules/custom/mis_git/composer.json...
  [merge-plugin] Merging cpliakas/git-wrapper
Running 1.8.4 (2019-02-11 10:52:10) with PHP 7.1.26-1+ubuntu16.04.1+deb.sury.org+1 on Linux / 4.4.0-116-generic
Composer version 1.8.4 2019-02-11 10:52:10

I wonder how the composer cache clearing and autoload dumping may relate to drush, and why I only starting seeing this messages on drush 9.5.x+. Here is my output:

$ composer -V -vvv
Reading ./composer.json
Loading config file ./composer.json
Checked CA file /private/etc/ssl/cert.pem: valid
Executing command (/Users/malik.kotob/Sites/project): git branch --no-color --no-abbrev -v
Reading /Users/malik.kotob/.composer/composer.json
Loading config file /Users/malik.kotob/.composer/composer.json
Reading /Users/malik.kotob/Sites/project/vendor/composer/installed.json
Reading /Users/malik.kotob/.composer/vendor/composer/installed.json
Could not scan for classes inside "/Users/malik.kotob/Sites/project/vendor/drupal/core/lib/Drupal.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/Users/malik.kotob/Sites/project/vendor/drupal/core/lib/Drupal/Component/Utility/Timer.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/Users/malik.kotob/Sites/project/vendor/drupal/core/lib/Drupal/Component/Utility/Unicode.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/Users/malik.kotob/Sites/project/vendor/drupal/core/lib/Drupal/Core/Database/Database.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/Users/malik.kotob/Sites/project/vendor/drupal/core/lib/Drupal/Core/DrupalKernel.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/Users/malik.kotob/Sites/project/vendor/drupal/core/lib/Drupal/Core/DrupalKernelInterface.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/Users/malik.kotob/Sites/project/vendor/drupal/core/lib/Drupal/Core/Site/Settings.php" which does not appear to be a file nor a folder
Loading plugin Acquia\Blt\Composer\Plugin
Loading plugin Composer\Installers\Plugin
Loading plugin cweagans\Composer\Patches
Loading plugin Dealerdirect\Composer\Plugin\Installers\PHPCodeSniffer\Plugin
Loading plugin DrupalComposer\DrupalScaffold\Plugin
Loading plugin grasmash\DrupalSecurityWarning\Composer\Plugin
Loading plugin OomphInc\ComposerInstallersExtender\Plugin
Loading plugin Wikimedia\Composer\MergePlugin
Loading plugin Hirak\Prestissimo\Plugin
  [merge-plugin] Loading blt/composer.required.json...
  [merge-plugin] Prepending composer repository
  [merge-plugin] Merging drupal/core
  [merge-plugin] Merging drupal/config_split
  [merge-plugin] Loading blt/composer.suggested.json...
  [merge-plugin] Prepending composer repository
  [merge-plugin] Ignoring duplicate acquia/lightning
  [merge-plugin] Merging acquia/drupal-spec-tool
  [merge-plugin] Merging drupal/acquia_connector
  [merge-plugin] Merging drupal/acquia_purge
  [merge-plugin] Merging drupal/cog
  [merge-plugin] Merging drupal/devel
  [merge-plugin] Merging drupal/qa_accounts
  [merge-plugin] Merging drupal/memcache
  [merge-plugin] Merging drupal/seckit
  [merge-plugin] Merging drupal/security_review
  [merge-plugin] Merging drupal/shield
  [merge-plugin] Merging drupal/features
Running 1.8.4 (2019-02-11 10:52:10) with PHP 7.1.17 on Darwin / 18.0.0
Composer version 1.8.4 2019-02-11 10:52:10

This has nothing to do with a VM for me. It's super easy to reproduce from a brand new project (with no VM) using Composer 1.8.4 (latest): composer create-project acquia/blt-project:^10.0.0-alpha1 --no-interaction blt3148, then cd blt3148

composer -vvv --version
Reading ./composer.json
Loading config file ./composer.json
Checked CA file /etc/ssl/certs/ca-certificates.crt: valid
Executing command (/home/dane/sites/blt3148): git branch --no-color --no-abbrev -v
Reading /home/dane/.config/composer/composer.json
Loading config file /home/dane/.config/composer/composer.json
Reading /home/dane/sites/blt3148/vendor/composer/installed.json
Reading /home/dane/.config/composer/vendor/composer/installed.json
Could not scan for classes inside "/home/dane/sites/blt3148/vendor/drupal/core/lib/Drupal.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/home/dane/sites/blt3148/vendor/drupal/core/lib/Drupal/Component/Utility/Timer.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/home/dane/sites/blt3148/vendor/drupal/core/lib/Drupal/Component/Utility/Unicode.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/home/dane/sites/blt3148/vendor/drupal/core/lib/Drupal/Core/Database/Database.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/home/dane/sites/blt3148/vendor/drupal/core/lib/Drupal/Core/DrupalKernel.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/home/dane/sites/blt3148/vendor/drupal/core/lib/Drupal/Core/DrupalKernelInterface.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/home/dane/sites/blt3148/vendor/drupal/core/lib/Drupal/Core/Site/Settings.php" which does not appear to be a file nor a folder
Loading plugin Acquia\Blt\Composer\Plugin
Loading plugin Composer\Installers\Plugin
Loading plugin cweagans\Composer\Patches
Loading plugin Dealerdirect\Composer\Plugin\Installers\PHPCodeSniffer\Plugin
Loading plugin DrupalComposer\DrupalScaffold\Plugin
Loading plugin grasmash\DrupalSecurityWarning\Composer\Plugin
Loading plugin OomphInc\ComposerInstallersExtender\Plugin
Loading plugin zaporylie\ComposerDrupalOptimizations\Plugin
Loading plugin Hirak\Prestissimo\Plugin
Running 1.8.4 (2019-02-11 10:52:10) with PHP 7.2.15-0ubuntu0.18.10.1 on Linux / 4.18.0-15-generic
Composer version 1.8.4 2019-02-11 10:52:10

I think is actually the issue I reported with zaporylie/composer-drupal-optimizations but my BLT and upstream PR to drupal project were closed when the upstream bug was reported fixed. I'll reopen #3336 to see if that can give us more information into the exact cause.

Just confirming that I too have hit the same issue trying to install BLT with latest Drupal.

Could not scan for classes inside "/app/vendor/drupal/core/lib/Drupal/Component/Utility/Timer.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/app/vendor/drupal/core/lib/Drupal/Component/Utility/Unicode.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/app/vendor/drupal/core/lib/Drupal/Core/Database/Database.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/app/vendor/drupal/core/lib/Drupal/Core/DrupalKernel.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/app/vendor/drupal/core/lib/Drupal/Core/DrupalKernelInterface.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/app/vendor/drupal/core/lib/Drupal/Core/Site/Settings.php" which does not appear to be a file nor a folder

Removing BLT from my project allows Drupal to install and work. Hoping the BLT project can get this fixed; not sure how to get it working at this point yet. Was trying with acquia/blt:^10.0@beta

@bdanin Just to be clear, nothing should be "not working" due to this error. As far as we know it's purely cosmetic. If you are having trouble installing BLT please open a new issue.

Dropping a note here that I only started seeing these after upgrading from drush 9.4.0 to 9.5.x and greater, as did @rumyanar in the comment above. I can consistently reproduce downgrading to drush 9.4.0 solving the issue.

I just tried downgrading to Drush 9.4 on BLT 10 and it didn't change anything. So I think there's more going on than that.

I think this is some sort of negative resonance between BLT's own Composer plugin and composer/installers, which also provides a Composer plugin.

If I disable BLT's Composer plugin, the error goes away. If I remove composer/installers, the error goes away. But with both in place I get the error.

Obviously neither of these is really a workaround so now we need to figure out why they conflict.

Perhaps relevant? wikimedia/composer-merge-plugin#139

I created a minimum failing example that reproduces this issue from almost nothing (with basically no parts of BLT involved), which leads me to strongly suspect that composer installers is the culprit: composer/installers#427

I also pursued another avenue of investigation that eventually led to a dead end, trying to isolate when this problem first appeared. It looks like it started between these two cron runs of Travis:

That's super interesting because of course BLT did not change anything between those cron runs, so it must be due to a dependency. Drush upgraded itself from 9.4.0 to 9.5.2 in that run. Unfortunately, that doesn't help much in troubleshooting because rolling Drush 9.4.0 nowadays doesn't fix this. And we didn't use composer.lock back then so there's no way to exactly reproduce that environment and upgrade path to isolate the cause.

I think I have a good understanding of what's going on now, and it's most likely a problem with how Composer loads plugins: composer/composer#8065

I don't think it's something that BLT is doing wrong, or can solve on its own, unless we rename the package to zzzacquia/blt (seriously, that fixes it! ๐Ÿ˜„)

I've positively identified this as a problem with Composer, and unfortunately it's not something BLT can work around. But the good news is a PR is open and it's a simple fix (although it took me nearly a week to find!): composer/composer#8070

New upstream PR awaiting review: composer/composer#8085

I am facing this same issue can please some one help me..
Thanks

[RuntimeException]
Could not scan for classes inside "app/http/controllers/Admin" which does not appear to be a file nor a folder

Delete the vendor folder.
Then run, composer install --prefer-dist