ampproject/amp-wp

PHPUnit tests fail in development environment

westonruter opened this issue · 0 comments

Bug Description

When running npm run test:php (see also #7589), there are test failures and warnings:

$ npm run test:php
> amp-wp@ test:php .../amp
> wp-env run tests-cli --env-cwd=/var/www/html/wp-content/plugins/amp/ env WORDPRESS_TABLE_PREFIX=wptests_ vendor/bin/phpunit

ℹ Starting 'env WORDPRESS_TABLE_PREFIX=wptests_ vendor/bin/phpunit' on the tests-cli container. 

[17-Jul-2023 21:02:29 UTC] PHP Warning:  Private methods cannot be final as they are never overridden by other classes in /var/www/html/wp-content/plugins/amp/vendor/phpunit/phpunit/src/Util/Configuration.php on line 176
Installing...
Running as single site... To run multisite, use -c tests/phpunit/multisite.xml
Not running ajax tests. To execute these, use --group ajax.
Not running ms-files tests. To execute these, use --group ms-files.
Not running external-http tests. To execute these, use --group external-http.
PHPUnit 6.5.14 by Sebastian Bergmann and contributors.

.............................................................   61 / 2959 (  2%)
........................FF.....FFFFFFF...FFFFFFF.............  122 / 2959 (  4%)
.............................................................  183 / 2959 (  6%)
.............................................................  244 / 2959 (  8%)
.............................................................  305 / 2959 ( 10%)
.............................................................  366 / 2959 ( 12%)
.............................................................  427 / 2959 ( 14%)
.............................................................  488 / 2959 ( 16%)
.............................................................  549 / 2959 ( 18%)
..........................................F..................  610 / 2959 ( 20%)
.............................................................  671 / 2959 ( 22%)
.............................................................  732 / 2959 ( 24%)
.............................................................  793 / 2959 ( 26%)
.............................................................  854 / 2959 ( 28%)
.............................................................  915 / 2959 ( 30%)
.............................................................  976 / 2959 ( 32%)
............................................................. 1037 / 2959 ( 35%)
............................................................. 1098 / 2959 ( 37%)
............................................................. 1159 / 2959 ( 39%)
............................................................. 1220 / 2959 ( 41%)
............................................................. 1281 / 2959 ( 43%)
............................................................. 1342 / 2959 ( 45%)
............................................................. 1403 / 2959 ( 47%)
............................................................. 1464 / 2959 ( 49%)
............................................................. 1525 / 2959 ( 51%)
............................................................. 1586 / 2959 ( 53%)
............................................................. 1647 / 2959 ( 55%)
............................................................. 1708 / 2959 ( 57%)
............................................................. 1769 / 2959 ( 59%)
............................................................. 1830 / 2959 ( 61%)
............................................................. 1891 / 2959 ( 63%)
............................................................. 1952 / 2959 ( 65%)
............................................................. 2013 / 2959 ( 68%)
............................................................. 2074 / 2959 ( 70%)
............................................................. 2135 / 2959 ( 72%)
............................................................. 2196 / 2959 ( 74%)
............................................................. 2257 / 2959 ( 76%)
............................................................. 2318 / 2959 ( 78%)
.........................................................[17-Jul-2023 21:03:25 UTC] PHP Warning:  fseek(): Stream does not support seeking in /var/www/html/wp-content/plugins/amp/src/Support/SupportData.php on line 378
.
Warning: fseek(): Stream does not support seeking in /var/www/html/wp-content/plugins/amp/src/Support/SupportData.php on line 378
... 2379 / 2959 ( 80%)
............................................................. 2440 / 2959 ( 82%)
............................................................. 2501 / 2959 ( 84%)
...S......................................................... 2562 / 2959 ( 86%)
............................................................. 2623 / 2959 ( 88%)
............................................................. 2684 / 2959 ( 90%)
............................................................. 2745 / 2959 ( 92%)
............................................................. 2806 / 2959 ( 94%)
............................................................. 2867 / 2959 ( 96%)
............................................................. 2928 / 2959 ( 98%)
...............................                               2959 / 2959 (100%)

Time: 1.14 minutes, Memory: 155.00MB

There were 17 failures:

1) AMP_Form_Sanitizer_Test::test_converter with data set "form_with_get_method_http_action_and_no_target" ('<form method="get" action="ht.../form>', '<form method="get" action="//.../form>')
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
-    1 => '<form action="//localhost:888..._top">'
-    3 => '</form>'

/var/www/html/wp-content/plugins/amp/tests/php/src/Helpers/MarkupComparison.php:119
/var/www/html/wp-content/plugins/amp/tests/php/test-amp-form-sanitizer.php:299

2) AMP_Form_Sanitizer_Test::test_converter with data set "form_with_get_method_and_action_xhr" ('<form method="get" action="//.../form>', '<form method="get" action="//.../form>')
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
-    1 => '<form action-xhr="//localhost...")]}">'
-    3 => '</form>'

/var/www/html/wp-content/plugins/amp/tests/php/src/Helpers/MarkupComparison.php:119
/var/www/html/wp-content/plugins/amp/tests/php/test-amp-form-sanitizer.php:299

3) AMP_Form_Sanitizer_Test::test_converter with data set "form_with_post_method_http_action_and_no_target" ('<form method="post" action="h.../form>', '<form method="post" action-xh.../form>')
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
-    1 => '<form action-xhr="//localhost..._top">'
-    3 => '<div class="amp-wp-default-fo...error>'
-    5 => '<template type="amp-mustache">'
-    7 => '...'
-    9 => '</template>'
-    11 => '</div>'
-    13 => '<div class="amp-wp-default-fo...ccess>'
-    15 => '<template type="amp-mustache">'
-    17 => '...'
-    19 => '</template>'
-    21 => '</div>'
-    23 => '<div class="amp-wp-default-fo...tting>'
-    25 => '<template type="amp-mustache">'
-    27 => '...'
-    29 => '</template>'
-    31 => '</div>'
-    33 => '</form>'

/var/www/html/wp-content/plugins/amp/tests/php/src/Helpers/MarkupComparison.php:119
/var/www/html/wp-content/plugins/amp/tests/php/test-amp-form-sanitizer.php:299

4) AMP_Form_Sanitizer_Test::test_converter with data set "form_with_exiting_mustache_templates_left_intact" ('\n                                <form method="post" act...\n                            ', '\n                          <form method="post" act...\n                            ')
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
-    1 => '<form action-xhr="//localhost..._top">'
-    3 => '<div class="amp-wp-default-fo...error>'
-    5 => '<template type="amp-mustache">'
-    7 => '...'
-    9 => '</template>'
-    11 => '</div>'
-    13 => '<div class="amp-wp-default-fo...ccess>'
-    15 => '<template type="amp-mustache">'
-    17 => '...'
-    19 => '</template>'
-    21 => '</div>'
-    23 => '<div class="amp-wp-default-fo...tting>'
-    25 => '<template type="amp-mustache">'
-    27 => '...'
-    29 => '</template>'
-    31 => '</div>'
-    33 => '</form>'

/var/www/html/wp-content/plugins/amp/tests/php/src/Helpers/MarkupComparison.php:119
/var/www/html/wp-content/plugins/amp/tests/php/test-amp-form-sanitizer.php:299

5) AMP_Form_Sanitizer_Test::test_converter with data set "form_with_exiting_mustache_plain_text_scripts_left_intact" ('\n                               <form method="post" act...\n                            ', '\n                          <form method="post" act...\n                          ')
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
-    1 => '<form action-xhr="//localhost..._top">'
-    3 => '<div class="amp-wp-default-fo...error>'
-    5 => '<script template="amp-mustach...lain">'
-    7 => '...'
-    9 => '</script>'
-    11 => '</div>'
-    13 => '<div class="amp-wp-default-fo...ccess>'
-    15 => '<script template="amp-mustach...lain">'
-    17 => '...'
-    19 => '</script>'
-    21 => '</div>'
-    23 => '<div class="amp-wp-default-fo...tting>'
-    25 => '<script template="amp-mustach...lain">'
-    27 => '...'
-    29 => '</script>'
-    31 => '</div>'
-    33 => '</form>'

/var/www/html/wp-content/plugins/amp/tests/php/src/Helpers/MarkupComparison.php:119
/var/www/html/wp-content/plugins/amp/tests/php/test-amp-form-sanitizer.php:299

6) AMP_Form_Sanitizer_Test::test_converter with data set "form_with_not_all_templates_templates_supplied" ('\n                          <form method="post" act...\n                            ', '\n                          <form method="post" act...\n                            ')
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
-    1 => '<form action-xhr="//localhost..._top">'
-    3 => '<div class="amp-wp-default-fo...ccess>'
-    5 => '<script template="amp-mustach...lain">'
-    7 => '...'
-    9 => '</script>'
-    11 => '</div>'
-    13 => '<div class="amp-wp-default-fo...error>'
-    15 => '<template type="amp-mustache">'
-    17 => '...'
-    19 => '</template>'
-    21 => '</div>'
-    23 => '<div class="amp-wp-default-fo...tting>'
-    25 => '<template type="amp-mustache">'
-    27 => '...'
-    29 => '</template>'
-    31 => '</div>'
-    33 => '</form>'

/var/www/html/wp-content/plugins/amp/tests/php/src/Helpers/MarkupComparison.php:119
/var/www/html/wp-content/plugins/amp/tests/php/test-amp-form-sanitizer.php:299

7) AMP_Form_Sanitizer_Test::test_converter with data set "form_with_post_method_http_action_and_blank_target" ('<form method="post" action-xh.../form>', '<form method="post" action-xh.../form>')
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
-    1 => '<form action-xhr="//localhost...lank">'
-    3 => '</form>'

/var/www/html/wp-content/plugins/amp/tests/php/src/Helpers/MarkupComparison.php:119
/var/www/html/wp-content/plugins/amp/tests/php/test-amp-form-sanitizer.php:299

8) AMP_Form_Sanitizer_Test::test_converter with data set "form_with_post_method_http_action_and_self_target" ('<form method="get" action="ht.../form>', '<form method="get" action="ht.../form>')
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
-    1 => '<form action="https://localho..._top">'
-    3 => '</form>'

/var/www/html/wp-content/plugins/amp/tests/php/src/Helpers/MarkupComparison.php:119
/var/www/html/wp-content/plugins/amp/tests/php/test-amp-form-sanitizer.php:299

9) AMP_Form_Sanitizer_Test::test_converter with data set "form_with_post_method_https_action_and_custom_target" ('<form method="post" action="h.../form>', '<form method="post" target="_.../form>')
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
-    1 => '<form action-xhr="https://loc...lank">'
-    3 => '<div class="amp-wp-default-fo...error>'
-    5 => '<template type="amp-mustache">'
-    7 => '...'
-    9 => '</template>'
-    11 => '</div>'
-    13 => '<div class="amp-wp-default-fo...ccess>'
-    15 => '<template type="amp-mustache">'
-    17 => '...'
-    19 => '</template>'
-    21 => '</div>'
-    23 => '<div class="amp-wp-default-fo...tting>'
-    25 => '<template type="amp-mustache">'
-    27 => '...'
-    29 => '</template>'
-    31 => '</div>'
-    33 => '</form>'

/var/www/html/wp-content/plugins/amp/tests/php/src/Helpers/MarkupComparison.php:119
/var/www/html/wp-content/plugins/amp/tests/php/test-amp-form-sanitizer.php:299

10) AMP_Form_Sanitizer_Test::test_converter with data set "form_with_relative_action_url" ('<form method="post" action="/.../form>', '<form method="post" action-xh.../form>')
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
-    1 => '<form action-xhr="//localhost:8889:8889/login/?_wp_amp_action_xhr_converted=1" method="post" target="_top">'
+    1 => '<form action-xhr="//localhost:8889/login/?_wp_amp_action_xhr_converted=1" method="post" target="_top">'

/var/www/html/wp-content/plugins/amp/tests/php/src/Helpers/MarkupComparison.php:119
/var/www/html/wp-content/plugins/amp/tests/php/test-amp-form-sanitizer.php:299

11) AMP_Form_Sanitizer_Test::test_converter with data set "form_with_relative_path_action_url" ('<form method="post" action="..../form>', '<form method="post" action-xh.../form>')
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
-    1 => '<form action-xhr="//localhost:8889:8889/current-page/../?_wp_amp_action_xhr_converted=1" method="post" target="_top">'
+    1 => '<form action-xhr="//localhost:8889/current-page/../?_wp_amp_action_xhr_converted=1" method="post" target="_top">'

/var/www/html/wp-content/plugins/amp/tests/php/src/Helpers/MarkupComparison.php:119
/var/www/html/wp-content/plugins/amp/tests/php/test-amp-form-sanitizer.php:299

12) AMP_Form_Sanitizer_Test::test_converter with data set "form_with_another_relative_path_action_url" ('<form method="post" action="f.../form>', '<form method="post" action-xh.../form>')
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
-    1 => '<form action-xhr="//localhost:8889:8889/current-page/foo/?_wp_amp_action_xhr_converted=1" method="post" target="_top">'
+    1 => '<form action-xhr="//localhost:8889/current-page/foo/?_wp_amp_action_xhr_converted=1" method="post" target="_top">'

/var/www/html/wp-content/plugins/amp/tests/php/src/Helpers/MarkupComparison.php:119
/var/www/html/wp-content/plugins/amp/tests/php/test-amp-form-sanitizer.php:299

13) AMP_Form_Sanitizer_Test::test_converter with data set "form_with_relative_query_action_url" ('<form method="post" action="?.../form>', '<form method="post" action-xh.../form>')
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
-    1 => '<form action-xhr="//localhost:8889:8889/current-page/?foo=bar&amp;_wp_amp_action_xhr_converted=1" method="post" target="_top">'
+    1 => '<form action-xhr="//localhost:8889/current-page/?foo=bar&amp;_wp_amp_action_xhr_converted=1" method="post" target="_top">'

/var/www/html/wp-content/plugins/amp/tests/php/src/Helpers/MarkupComparison.php:119
/var/www/html/wp-content/plugins/amp/tests/php/test-amp-form-sanitizer.php:299

14) AMP_Form_Sanitizer_Test::test_converter with data set "form_with_multiple_relative_queries_action_url" ('<form method="post" action="?.../form>', '<form method="post" action-xh.../form>')
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
-    1 => '<form action-xhr="//localhost:8889:8889/current-page/?foo=bar&amp;baz=buzz&amp;_wp_amp_action_xhr_converted=1" method="post" target="_top">'
+    1 => '<form action-xhr="//localhost:8889/current-page/?foo=bar&amp;baz=buzz&amp;_wp_amp_action_xhr_converted=1" method="post" target="_top">'

/var/www/html/wp-content/plugins/amp/tests/php/src/Helpers/MarkupComparison.php:119
/var/www/html/wp-content/plugins/amp/tests/php/test-amp-form-sanitizer.php:299

15) AMP_Form_Sanitizer_Test::test_converter with data set "form_with_relative_fragment_action_url" ('<form method="post" action="#.../form>', '<form method="post" action-xh.../form>')
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
-    1 => '<form action-xhr="//localhost:8889:8889/current-page/?_wp_amp_action_xhr_converted=1#foo" method="post" target="_top">'
+    1 => '<form action-xhr="//localhost:8889/current-page/?_wp_amp_action_xhr_converted=1#foo" method="post" target="_top">'

/var/www/html/wp-content/plugins/amp/tests/php/src/Helpers/MarkupComparison.php:119
/var/www/html/wp-content/plugins/amp/tests/php/test-amp-form-sanitizer.php:299

16) AMP_Form_Sanitizer_Test::test_converter with data set "form_with_relative_query_and_fragment_action_url" ('<form method="post" action="?.../form>', '<form method="post" action-xh.../form>')
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
-    1 => '<form action-xhr="//localhost:8889:8889/current-page/?foo=bar&amp;_wp_amp_action_xhr_converted=1#baz" method="post" target="_top">'
+    1 => '<form action-xhr="//localhost:8889/current-page/?foo=bar&amp;_wp_amp_action_xhr_converted=1#baz" method="post" target="_top">'

/var/www/html/wp-content/plugins/amp/tests/php/src/Helpers/MarkupComparison.php:119
/var/www/html/wp-content/plugins/amp/tests/php/test-amp-form-sanitizer.php:299

17) AMP_Style_Sanitizer_Test::test_attribute_selector with data set "amp_replacement_elements_with_id" ('\n                                     <img id="my-img" src="...\n                             ', array(true, true, true, true, true, true, true, true, true, true, true, true, true, true))
add_noscript_fallback is false
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
     0 => '#my-audio'
     1 => '#my-iframe'
     2 => '#my-img'
     3 => '#my-o2'
-    4 => '#my-object'
-    5 => '#my-playbuzz'
-    6 => '#my-video'
-    7 => 'amp-audio#my-audio'
-    8 => 'amp-google-document-embed#my-object'
-    9 => 'amp-iframe#my-iframe'
-    10 => 'amp-img#my-img'
-    11 => 'amp-o2-player#my-o2'
-    12 => 'amp-playbuzz#my-playbuzz'
-    13 => 'amp-video#my-video'
+    4 => '#my-playbuzz'
+    5 => '#my-video'
+    6 => 'amp-audio#my-audio'
+    7 => 'amp-google-document-embed#my-object'
+    8 => 'amp-iframe#my-iframe'
+    9 => 'amp-img#my-img'
+    10 => 'amp-o2-player#my-o2'
+    11 => 'amp-playbuzz#my-playbuzz'
+    12 => 'amp-video#my-video'

/wordpress-phpunit/includes/abstract-testcase.php:990
/var/www/html/wp-content/plugins/amp/tests/php/test-amp-style-sanitizer.php:1730

FAILURES!
Tests: 2959, Assertions: 13011, Failures: 17, Skipped: 1.
✖ Command failed with exit code 1

Some of these seem to be due to the environment running with a home URL that includes a port number.

Expected Behaviour

Tests should run without failures or warnings.

Screenshots

No response

PHP Version

8.0

Plugin Version

2.4.2-RC1

AMP plugin template mode

Standard, Transitional, Reader

WordPress Version

No response

Site Health

No response

Gutenberg Version

No response

OS(s) Affected

No response

Browser(s) Affected

No response

Device(s) Affected

No response

Acceptance Criteria

No response

Implementation Brief

No response

QA Testing Instructions

No response

Demo

No response

Changelog Entry

No response