Contrib tests can not be executed

[mikran]

I would want to use drupal-core-development-project for contrib development too but currently I can not do that. Tests for contrib are failing. Core tests work ok.

Running

phpunit web/modules/contrib/devel

Gives error

Fatal error: Uncaught Error: Class "Drupal\Tests\devel_generate\Functional\DevelGenerateBrowserTestBase" not found in /var/www/web/modules/contrib/devel/devel_generate/tests/src/Functional/DevelGenerateBrowserTest.php:14

[joachim-n]

Urgh, that's because Drupal's core/tests/bootstrap.php does this:

  $loader = require __DIR__ . '/../../autoload.php';

and so is loading repos/drupal/autoload.php. That works because of the workaround in this project template which symlinks the vendor/ folder. But elsewhere in core/tests/bootstrap.php, in drupal_phpunit_contrib_extension_directory_roots() it registers namespaces from contrib modules with the autoloader, and obviously, doesn't find any contrib modules because they're not in /repos/drupal.

Not sure how to tackle this. We're including core/tests/bootstrap.php from phpunit.xml, which is handwritten and so knows the path to the file. But once inside that file, it can't know that it's symlinked in.

[joachim-n]

I think this can only be fixed in core as part of https://www.drupal.org/project/drupal/issues/1792310.

As a workaround, you could hack the file to change all instances of DIR to the path with the symlink, that is, web/core/tests rather than repos/drupal/core/tests.

[joachim-n]

I can see how we could do this before that core issue is fixed, but it's fiddly.

Method 1 - quick but dirty, doable by users:

1. Copy the version of core/tests/bootstrap.php
   from the MR for the core issue -- https://git.drupalcode.org/project/drupal/-/blob/2fb261b10d31a23eccfd52d68fbd61e34e999d92/core/tests/bootstrap.php -- and place it somewhere like the project root
2. Make a second phpunit.xml file in the project root, called, say, phpunit-contrib.xml
3. Edit that file to define the location variables PROJECT_ROOT and DRUPAL_APP_ROOT, as demonstrated in the MR: https://git.drupalcode.org/project/drupal/-/blob/2fb261b10d31a23eccfd52d68fbd61e34e999d92/core/phpunit.xml.dist
4. Edit that file so that its bootstrap is the bootstrap.php file you've just made
5. To run contrib tests, use the -c flag to load the phpunit-contrib.xml file

Method 2 - longer and also dirty, fixed in this template:

1. We add a patch for core/tests/bootstrap.php from that MR into this template
2. Add a step to the installation process for this template which:
   a. copies core/tests/bootstrap.php to the project root
   b. applies the patch to it
3. Add details to the README for adding the location environment variables to phpunit.xml

[joachim-n]

I'm experimenting with this, and WEIRD STUFF is happening.

I put a phpunit-contrib.xml file in the project root, pointing to a custom test bootstrap.php file at tests/bootstrap.php.

A contrib kernel test fails with:

PHPUnit\Framework\Exception: Fatal error: Uncaught PHPUnit\Util\Xml\Exception: Could not read "phpunit-contrib.xml". in /Users/joachim/Sites/drupal-core-composer/vendor/phpunit/phpunit/src/Util/Xml/Loader.php:41

This appears to be because at some point the working dir is changed to 'web'. I assume it's this in KernelTestBase:

    // Change the current dir to DRUPAL_ROOT.
    chdir(static::getDrupalRoot());

So, rather than go digging into why that is done, I symlinked phpunit-contrib.xml into 'web'.

The test then fails with:

PHPUnit\Framework\Exception: Warning: require_once(./tests/bootstrap.php): Failed to open stream: No such file or directory in Standard input code on line 119

Presumably that is the dir change causing that again.

I then replaced the bootstrap file location in phpunit-contrib.xml with an absolute path.

(Though what confuses me is how this all works when phpunit is run without the -c option and it just finds the phpunit.xml file in the current working directory. What happens after KernelTestBase does a chdir() -- how does PHPUnit find that file afterwards?)

I got this:

PHPUnit\Framework\Exception: Warning: Class "\Drupal\Tests\DocumentElement" not found in /Users/joachim/Sites/drupal-core-composer/tests/bootstrap.php on line 172

Which I've seen before in other circumstances -- it's the autoloader not being set up with the correct path. So again, it's to do with being in the Drupal root.

[joachim-n]

(Though what confuses me is how this all works when phpunit is run without the -c option and it just finds the phpunit.xml file in the current working directory. What happens after KernelTestBase does a chdir() -- how does PHPUnit find that file afterwards?)

In that circumstance, PHPUnit\Util\Xml\Loader::loadFile() gets an absolute path. With the -c option it gets a relative path.

[markfullmer]

This may be obvious to folks coming here (thanks so much, @joachim-n for providing this really useful tool for core contribution!!), but a simple workaround could be to add a symlink for the contrib modules, e.g.,

ln -s ../../../web/modules/contrib repos/drupal/modules/contrib

[joachim-n]

Might be an idea to revisit this now that we're on a new major version of PHPUnit -- the behaviour of PHPUnit\Util\Xml\Loader::loadFile() might have changed.