New testing library

Question

New testing library

thekid opened this issue 2 years ago · 18 comments

Scope of Change

This RFC suggests to create a new testing library for the XP Framework. It will only work with baseless, single-instance test classes.

Rationale

Drop support for extends TestCase, supporting of which costs lots of code
Drop support for seldomly used features such as test timeouts, exotic Values variations or external provider methods
Simplify test actions, these should have not use exceptions for flow control
Integrate Assert::that() extended assertions to simplify commonly used test code

Functionality

In a nutshell, tests reside inside a class and are annotated with the Test attribute. Except for the changed namespace (see below) and the changed default output, this is basically identical to the current usage of the unittest library.

use test\{Assert, Test};

class CalculatorTest {

  #[Test]
  public function addition() {
    Assert::equals(2, (new Calculator())->add(1, 1));
  }
}

To run this test, use the test subcommand:

$ xp test CalculatorTest.class.php
> [PASS] CalculatorTest
  ✓ addition

Tests:       1 succeeded, 0 skipped, 0 failed
Memory used: 1556.36 kB (1610.49 kB peak)
Time taken:  0.001 seconds

Naming

The library will be called test, because it can not only run unittests. The top-level package used follows this, meaning the Assert class' fully qualified name is test.Assert.

Assert DSL

The following shorthand methods exist on the Assert class:

equals(mixed $expected, mixed $actual) - check two values are equal. Uses the util.Objects::equal() method internally, which allows overwriting object comparison.
notEquals(mixed $expected, mixed $actual) - opposite of above
true(mixed $actual) - check a given value is equal to the true boolean
false(mixed $actual) - check a given value is equal to the false boolean
null(mixed $actual) - check a given value is null
instance(string|lang.Type $expected, mixed $actual) - check a given value is an instance of the given type.

Extended assertions

The Assert::that() method starts an assertion chain:

Fluent assertions

Each of the following may be chained to Assert::that():

is(unittest.assert.Condition $condition) - Asserts a given condition matches
isNot(unittest.assert.Condition $condition) - Asserts a given condition does not match
isEqualTo(var $compare) - Asserts the value is equal to a given comparison
isNotEqualTo(var $compare) - Asserts the value is not equal to a given comparison
isNull() - Asserts the value is null
isTrue() - Asserts the value is true
isFalse() - Asserts the value is false
isInstanceOf(string|lang.Type $type) - Asserts the value is of a given type

Transformation

Transforming the value before comparison can make it easier to create the value to compare against. This can be achieved by chaining map() to Assert::that():

$records= $db->open('select ...');
$expected= ['one', 'two'];

// Before
$actual= [];
foreach ($records as $record) {
  $actual[]= $r['name'];
}
Assert::equals($expected, $actual);

// After
Assert::that($records)->mappedBy(fn($r) => $r['name'])->isEqualTo($expected);

Values and providers

#[Values] is a basic provider implementation. It can either return a static list of values as follows:

use test\{Assert, Test, Values};

class CalculatorTest {

  #[Test, Values([[0, 0], [1, 1], [-1, 1]])]
  public function addition($a, $b) {
    Assert::equals($a + $b, (new Calculator())->add($a, $b));
  }
}

...or invoke a method, as seen in this example:

use test\{Assert, Test, Values};

class CalculatorTest {

  private function operands(): iterable {
    yield [0, 0];
    yield [1, 1];
    yield [-1, 1];
  }

  #[Test, Values(from: 'operands')]
  public function addition($a, $b) {
    Assert::equals($a + $b, (new Calculator())->add($a, $b));
  }
}

Provider implementation example

use test\Provider;
use lang\reflection\Type;

class StartServer implements Provider {
  private $connection;

  /** Starts a new server */
  public function __construct(string $bind, ?int $port= null) {
    $port??= rand(1024, 65535);
    $this->connection= "Socket({$bind}:{$port})"; // TODO: Actual implementation ;)
  }

  /**
   * Returns values
   *
   * @param  Type $type
   * @param  ?object $instance
   * @return iterable
   */
  public function values($type, $instance= null) {
    return [$this->connection];
  }
}

Provider values are passed as method argument, just like #[Values], the previous only implementation.

use test\{Assert, Test};

class ServerTest {

  #[Test, StartServer('0.0.0.0', 8080)]
  public function connect($connection) {
    Assert::equals('Socket(0.0.0.0:8080)', $connection);
  }
}

Provider values are passed to the constructor and the connection can be used for all test cases. Note: The provider's values() method is invoked with $instance= null!

use test\{Assert, Test};

#[StartServer('0.0.0.0', 8080)]
class ServerTest {

  public function __construct(private $connection) { }

  #[Test]
  public function connect() {
    Assert::equals('Socket(0.0.0.0:8080)', $this->connection);
  }
}

Test prerequisites

Prerequisites can exist on a test class or a test method. Unlike in the old library, they do not require the Action annotation, but stand alone.

use test\{Assert, Test};
use test\verify\{Runtime, Condition};

#[Condition('class_exists(Calculator::class)')]
class CalculatorTest {

  #[Test, Runtime(php: '^8.1')]
  public function addition() {
    Assert::equals(2, (new Calculator())->add(1, 1));
  }
}

Security considerations

n/a

Speed impact

Same

Dependencies

https://github.com/xp-framework/reflection

Related documents

Answer 1 · 2023-01-05T20:40:15.000Z

First POC @ https://github.com/xp-framework/test

Answer 2 · 2023-01-10T16:14:43.000Z

Migration script

Implementation: https://gist.github.com/thekid/ad6f908f4badbc3460a74a43e9544a86
xp-framework/test#2

Answer 3 · 2023-01-15T12:17:25.000Z

Group (class) and case (method) prerequisites @ xp-framework/test#1

Answer 4 · 2023-01-15T15:43:23.000Z

Test prerequisites analysis

Usage of test actions:

address/.../RecordOfTest.class.php:73:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")'), Expect(class: Error::class, withMessage: '/Too few arguments .+/')]
address/.../RecordOfTest.class.php:78:  #[Test, Action(eval: 'new RuntimeVersion("<7.1")'), Expect(IllegalArgumentException::class)]
address/.../UsingNextTest.class.php:40:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
address/.../ValueOfTest.class.php:22:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
compiler/.../emit/EnumTest.class.php:8:#[Action(eval: 'new VerifyThat(fn() => function_exists("enum_exists"))')]
compiler/.../emit/IntersectionTypesTest.class.php:50:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1.0-dev")')]
compiler/.../emit/IntersectionTypesTest.class.php:62:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1.0-dev")')]
compiler/.../emit/IntersectionTypesTest.class.php:74:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1.0-dev")')]
compiler/.../emit/InvocationTest.class.php:142:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
compiler/.../emit/InvocationTest.class.php:158:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
compiler/.../emit/LambdasTest.class.php:51:  #[Test, Action(eval: 'new VerifyThat(fn() => property_exists(LambdaExpression::class, "static"))')]
compiler/.../emit/LambdasTest.class.php:62:  #[Test, Action(eval: 'new VerifyThat(fn() => property_exists(ClosureExpression::class, "static"))')]
compiler/.../emit/ParameterTest.class.php:76:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
compiler/.../emit/ReflectionTest.class.php:43:  #[Test, Action(eval: 'new RuntimeVersion("<8.1")')]
compiler/.../emit/ReflectionTest.class.php:61:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
compiler/.../emit/TraitsTest.class.php:66:  #[Test, Action(eval: 'new RuntimeVersion(">=8.2.0-dev")')]
compiler/.../emit/UnionTypesTest.class.php:74:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
compiler/.../emit/UnionTypesTest.class.php:86:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
compiler/.../emit/UnionTypesTest.class.php:98:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
compiler/.../emit/UnionTypesTest.class.php:110:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
compiler/.../loader/CompilingClassLoaderTest.class.php:141:  #[Test, Action(eval: 'new RuntimeVersion(">=7.3")'), Expect(class: ClassFormatException::class, withMessage: '/Compiler error: Class .+ not found/')]
compression/.../CompressionTest.class.php:54:  #[Test, Values(['gzip', 'GZIP', '.gz', '.GZ']), Action(eval: 'new ExtensionAvailable("zlib")')]
core/.../annotations/BrokenAnnotationTest.class.php:31:  #[Test, Action(eval: 'new RuntimeVersion("<8.0")'), Expect(['class' => ClassFormatException::class, 'withMessage' => '/Unterminated annotation/'])]
core/.../core/EnumTest.class.php:59:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:162:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:170:  #[Test, Expect(IllegalArgumentException::class), Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:217:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:344:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/ErrorsTest.class.php:115:  #[Test, Expect(IllegalArgumentException::class), Action(eval: 'new RuntimeVersion("<7.1.0-dev")')]
core/.../core/ErrorsTest.class.php:121:  #[Test, Expect(Error::class), Action(eval: 'new RuntimeVersion(">=7.1.0")')]
core/.../core/ErrorsTest.class.php:127:  #[Test, Expect(ClassCastException::class), Action(eval: 'new RuntimeVersion("<7.4.0-dev")')]
core/.../core/ErrorsTest.class.php:133:  #[Test, Expect(Error::class), Action(eval: 'new RuntimeVersion(">=7.4.0")')]
core/.../core/ExceptionsTest.class.php:140:  #[Test, Action(eval: 'new RuntimeVersion(">=7.0.0")')]
core/.../core/FunctionTypeTest.class.php:275:  #[Test, Action(eval: 'new VerifyThat(fn() => !extension_loaded("xdebug"))')]
core/.../core/NewInstanceTest.class.php:202:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:224:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:246:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:258:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:270:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:282:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:390:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:405:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:420:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:437:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:453:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:468:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:483:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:498:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.2")]')]
core/.../core/NewInstanceTest.class.php:513:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.1")]')]
core/.../core/NewInstanceTest.class.php:528:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.2")]')]
core/.../core/NewInstanceTest.class.php:543:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.1")]')]
core/.../core/NewInstanceTest.class.php:559:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:574:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:589:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/ProcessResolveTest.class.php:41:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:46:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:51:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:58:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:65:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:72:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:77:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:87:  #[Test, Action(eval: 'new IsPlatform("WIN")'), Expect(IOException::class)]
core/.../core/ProcessResolveTest.class.php:107:  #[Test, Action(eval: 'new IsPlatform("ANDROID")')]
core/.../core/ProcessResolveTest.class.php:113:  #[Test, Action(eval: 'new IsPlatform("!(WIN|ANDROID)")')]
core/.../core/ProcessResolveTest.class.php:118:  #[Test, Values(['"ls"', "'ls'"]), Action(eval: 'new IsPlatform("!(WIN|ANDROID)")')]
core/.../core/ProcessResolveTest.class.php:123:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:128:  #[Test, Action(eval: 'new IsPlatform("!(WIN|ANDROID)")')]
core/.../core/TypeIntersectionTest.class.php:120:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1.0-dev")')]
core/.../core/TypeIntersectionTest.class.php:127:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1.0-dev")')]
core/.../core/TypeIntersectionTest.class.php:134:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1.0-dev")')]
core/.../core/TypeOfTest.class.php:99:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
core/.../core/TypeOfTest.class.php:104:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
core/.../core/TypeOfTest.class.php:109:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeOfTest.class.php:114:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:192:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:201:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:210:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:219:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:228:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:234:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:240:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:246:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../io/PathTest.class.php:190:  #[Test, Action(eval: 'new IsPlatform("!^Win")')]
core/.../io/PathTest.class.php:256:  #[Test, Action(eval: 'new IsPlatform("^Win")')]
core/.../io/PathTest.class.php:266:  #[Test, Values(['C:', 'C:/', 'c:', 'C:/']), Action(eval: 'new IsPlatform("^Win")')]
core/.../io/PathTest.class.php:347:  #[Test, Action(eval: 'new IsPlatform("^Win")'), Values([['\\\\remote\\file.txt', true], ['\\\\remote', true]])]
core/.../io/PathTest.class.php:352:  #[Test, Action(eval: 'new IsPlatform("^Win")')]
core/.../io/PathTest.class.php:357:  #[Test, Action(eval: 'new IsPlatform("^Win")')]
core/.../reflection/FieldAccessTest.class.php:75:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
core/.../reflection/FieldAccessTest.class.php:85:  #[Test, Expect(IllegalAccessException::class), Action(eval: 'new RuntimeVersion(">=8.1")')]
core/.../reflection/FieldAccessTest.class.php:91:  #[Test, Expect(IllegalAccessException::class), Action(eval: 'new RuntimeVersion(">=8.1")')]
core/.../reflection/FieldModifiersTest.class.php:28:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
core/.../reflection/FieldTypeTest.class.php:64:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
core/.../reflection/FieldTypeTest.class.php:77:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
core/.../reflection/FieldTypeTest.class.php:89:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
core/.../reflection/MethodInvocationTest.class.php:46:  #[Test, Expect(TargetInvocationException::class), Action(eval: 'new RuntimeVersion(">=7.0")')]
core/.../reflection/MethodInvocationTest.class.php:52:  #[Test, Expect(TargetInvocationException::class), Action(eval: 'new RuntimeVersion(">=7.0")')]
core/.../reflection/MethodParametersTest.class.php:97:  #[Test, Action(eval: 'new RuntimeVersion(">=7.0")'), Values([['string'], ['int'], ['bool'], ['float']])]
core/.../reflection/MethodParametersTest.class.php:105:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
core/.../reflection/MethodParametersTest.class.php:114:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")'), Values([['string|int'], ['string|false']])]
core/.../reflection/MethodParametersTest.class.php:270:  #[Test, Action(eval: 'new RuntimeVersion("<8.0")')]
core/.../reflection/MethodParametersTest.class.php:275:  #[Test, Action(eval: 'new RuntimeVersion("<8.0")')]
core/.../reflection/MethodParametersTest.class.php:280:  #[Test, Action(eval: 'new RuntimeVersion("<8.0")')]
core/.../reflection/MethodParametersTest.class.php:285:  #[Test, Action(eval: 'new RuntimeVersion("<8.0")')]
core/.../reflection/MethodParametersTest.class.php:290:  #[Test, Action(eval: 'new RuntimeVersion("<8.0")')]
core/.../reflection/MethodReturnTypesTest.class.php:87:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
core/.../reflection/MethodReturnTypesTest.class.php:93:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
core/.../reflection/MethodReturnTypesTest.class.php:99:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
core/.../reflection/MethodReturnTypesTest.class.php:105:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")'), Values([['string|int'], ['string|false']])]
core/.../reflection/MethodReturnTypesTest.class.php:169:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
core/.../reflection/MethodReturnTypesTest.class.php:190:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
core/.../reflection/RuntimeClassDefinitionTest.class.php:148:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
core/.../reflection/TypeSyntaxTest.class.php:32:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
core/.../reflection/TypeSyntaxTest.class.php:53:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
core/.../reflection/TypeSyntaxTest.class.php:60:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
core/.../reflection/TypeSyntaxTest.class.php:67:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
core/.../reflection/TypeSyntaxTest.class.php:74:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
core/.../util/MoneyTest.class.php:8:#[Action(eval: 'new ExtensionAvailable("bcmath")')]
core/.../util/OpenSSLSecretTest.class.php:10:#[Action(eval: 'new ExtensionAvailable("openssl")')]
core/.../util/PropertyExpansionTest.class.php:31:  #[Test, Action(eval: 'new VerifyThat(fn() => !extension_loaded("xdebug"))')]
core/.../util/RandomTest.class.php:50:  #[Test, Action(eval: 'new VerifyThat(function() { return (function_exists("random_bytes") || function_exists("openssl_random_pseudo_bytes"));})')]
core/.../util/RandomTest.class.php:55:  #[Test, Action(eval: 'new ExtensionAvailable("openssl")')]
core/.../util/RandomTest.class.php:60:  #[Test, Action(eval: 'new VerifyThat(function() { return is_readable("/dev/urandom"); })')]
core/.../util/RandomTest.class.php:93:  #[Test, Action(eval: 'new ExtensionAvailable("openssl")')]
core/.../util/RandomTest.class.php:99:  #[Test, Action(eval: 'new VerifyThat(function() { return is_readable("/dev/urandom"); })')]
core/.../util/RandomTest.class.php:116:  #[Test, Expect(IllegalArgumentException::class), Action(eval: 'new VerifyThat(function() { return 0x7FFFFFFF === PHP_INT_MAX; })')]
core/.../util/RandomTest.class.php:121:  #[Test, Expect(IllegalArgumentException::class), Action(eval: 'new VerifyThat(function() { return 0x7FFFFFFF === PHP_INT_MAX; })')]
core/.../util/SodiumSecretTest.class.php:10:#[Action(eval: 'new ExtensionAvailable("sodium")')]
coverage/.../coverage/tests/RecordCoverageTest.class.php:7:#[Action(eval: 'new VerifyThat(function() { return interface_exists(\unittest\Listener::class); })')]
ftp/.../IntegrationTest.class.php:15:#[Action(eval: 'new StartServer("peer.ftp.unittest.TestingServer", "connected", "shutdown")')]
imaging/.../ExifDataTest.class.php:14:#[Action(eval: 'new ExtensionAvailable("exif")')]
imaging/.../GifImageReaderTest.class.php:15:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("GIF")]')]
imaging/.../GifImageWriterTest.class.php:13:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("GIF")]')]
imaging/.../ImageReaderTest.class.php:16:#[Action(eval: 'new ExtensionAvailable("gd")')]
imaging/.../JpegImageReaderTest.class.php:15:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("JPEG")]')]
imaging/.../JpegImageWriterTest.class.php:13:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("JPEG")]')]
imaging/.../PngImageReaderTest.class.php:15:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("PNG")]')]
imaging/.../PngImageWriterTest.class.php:13:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("PNG")]')]
imaging/.../WebpImageReaderTest.class.php:14:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("WEBP")]')]
imaging/.../WebpImageWriterTest.class.php:13:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("WEBP")]')]
mail/.../ImapStoreTest.class.php:13:#[Action(eval: 'new ExtensionAvailable("imap")')]
networking/.../sockets/BSDSocketTest.class.php:15:#[Action(eval: '[new ExtensionAvailable("sockets"), new StartServer(TestingServer::class, "connected", "shutdown")]')]
networking/.../sockets/SocketTest.class.php:12:#[Action(eval: 'new StartServer(TestingServer::class, "connected", "shutdown")')]
rdbms/.../ConfiguredConnectionManagerTest.class.php:14:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../CriteriaTest.class.php:15:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../DataSetTest.class.php:16:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../DBTest.class.php:10:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../FinderTest.class.php:15:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../ProjectionTest.class.php:19:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../QueryTest.class.php:14:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../QueuedConnectionManagerTest.class.php:14:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../RegisteredConnectionManagerTest.class.php:14:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../sqlite3/SQLite3ConnectionTest.class.php:14:#[Action(eval: 'new ExtensionAvailable("sqlite3")')]
rdbms/.../sqlite3/SQLite3CreationTest.class.php:16:#[Action(eval: 'new ExtensionAvailable("sqlite3")')]
rdbms/.../StatementTest.class.php:13:#[Action(eval: 'new RegisterMockConnection()')]
reflection/.../ConstantsTest.class.php:30:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
reflection/.../ConstantsTest.class.php:56:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
reflection/.../ConstantsTest.class.php:62:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
reflection/.../ConstantsTest.class.php:68:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
reflection/.../InstantiationTest.class.php:89:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../MethodsTest.class.php:89:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
reflection/.../MethodsTest.class.php:95:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../MethodsTest.class.php:101:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
reflection/.../MethodsTest.class.php:107:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
reflection/.../MethodsTest.class.php:113:  #[Test, Action(eval: 'new RuntimeVersion(">=8.2")'), Values(['true', 'false'])]
reflection/.../MethodsTest.class.php:125:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
reflection/.../MethodsTest.class.php:228:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../MethodsTest.class.php:237:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../MethodsTest.class.php:246:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
reflection/.../MethodsTest.class.php:285:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../MethodsTest.class.php:294:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
reflection/.../ParametersTest.class.php:143:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../PropertiesTest.class.php:106:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")'), Expect(AccessingFailed::class)]
reflection/.../PropertiesTest.class.php:148:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
reflection/.../PropertiesTest.class.php:157:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
reflection/.../PropertiesTest.class.php:166:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
reflection/.../PropertiesTest.class.php:175:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../PropertiesTest.class.php:184:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
reflection/.../PropertiesTest.class.php:193:  #[Test, Action(eval: 'new RuntimeVersion(">=8.2")'), Values(['true', 'false'])]
reflection/.../PropertiesTest.class.php:220:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
reflection/.../PropertiesTest.class.php:229:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../PropertiesTest.class.php:238:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
reflection/.../TypeTest.class.php:81:  #[Test, Action(eval: 'new RuntimeVersion(">=8.2")')]
reflection/.../TypeTest.class.php:111:  #[Test, Action(eval: 'new VerifyThat(fn() => !self::$ENUMS)')]
reflection/.../TypeTest.class.php:117:  #[Test, Action(eval: 'new VerifyThat(fn() => self::$ENUMS)')]
reflection/.../TypeTest.class.php:204:  #[Test, Action(eval: 'new VerifyThat(fn() => self::$ENUMS)')]
reflection/.../TypeTest.class.php:210:  #[Test, Action(eval: 'new VerifyThat(fn() => self::$ENUMS)')]
rest-client/.../CompressionHandlingTest.class.php:43:  #[Test, Action(eval: 'new ExtensionAvailable("zlib")')]
rest-client/.../CompressionHandlingTest.class.php:48:  #[Test, Action(eval: 'new ExtensionAvailable("brotli")')]
rest-client/.../CompressionHandlingTest.class.php:74:  #[Test, Values([1, 6, 9]), Action(eval: 'new ExtensionAvailable("zlib")')]
rest-client/.../CompressionHandlingTest.class.php:83:  #[Test, Values([0, 6, 11]), Action(eval: 'new ExtensionAvailable("brotli")')]
sequence/.../EachTest.class.php:85:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1.0")')]
sequence/.../PeekTest.class.php:76:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1.0")')]
text-encode/.../Base64InputStreamTest.class.php:13:#[Action(eval: 'new VerifyThat(fn() => in_array("convert.*", stream_get_filters()))')]
text-encode/.../Base64OutputStreamTest.class.php:13:#[Action(eval: 'new VerifyThat(fn() => in_array("convert.*", stream_get_filters()))')]
webtest/.../web/tests/FormsTest.class.php:19:    $this->assertEquals($action, $form->getAction());
xml/.../XslCallbackTest.class.php:16:#[Action(eval: '[new ExtensionAvailable("dom"), new ExtensionAvailable("xsl")]')]
zip/.../AbstractZipFileTest.class.php:13:#[Action(eval: 'new ExtensionAvailable("zlib")')]
zip/.../vendors/SevenZipFileTest.class.php:36:  #[Test, Action(eval: 'new ExtensionAvailable("zlib")')]
zip/.../vendors/SevenZipFileTest.class.php:41:  #[Test, Action(eval: 'new ExtensionAvailable("bz2")')]

Usage of VerifyThat:

compiler/.../emit/EnumTest.class.php:8:#[Action(eval: 'new VerifyThat(fn() => function_exists("enum_exists"))')]
compiler/.../emit/LambdasTest.class.php:51:  #[Test, Action(eval: 'new VerifyThat(fn() => property_exists(LambdaExpression::class, "static"))')]
compiler/.../emit/LambdasTest.class.php:62:  #[Test, Action(eval: 'new VerifyThat(fn() => property_exists(ClosureExpression::class, "static"))')]
core/.../core/EnumTest.class.php:59:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:162:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:170:  #[Test, Expect(IllegalArgumentException::class), Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:217:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:344:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/FunctionTypeTest.class.php:275:  #[Test, Action(eval: 'new VerifyThat(fn() => !extension_loaded("xdebug"))')]
core/.../core/NewInstanceTest.class.php:202:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:224:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:246:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:258:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:270:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:282:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:390:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:405:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:420:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:437:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:453:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:468:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:483:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:498:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.2")]')]
core/.../core/NewInstanceTest.class.php:513:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.1")]')]
core/.../core/NewInstanceTest.class.php:528:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.2")]')]
core/.../core/NewInstanceTest.class.php:543:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.1")]')]
core/.../core/NewInstanceTest.class.php:559:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:574:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:589:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../util/PropertyExpansionTest.class.php:31:  #[Test, Action(eval: 'new VerifyThat(fn() => !extension_loaded("xdebug"))')]
core/.../util/RandomTest.class.php:50:  #[Test, Action(eval: 'new VerifyThat(function() { return (function_exists("random_bytes") || function_exists("openssl_random_pseudo_bytes"));})')]
core/.../util/RandomTest.class.php:60:  #[Test, Action(eval: 'new VerifyThat(function() { return is_readable("/dev/urandom"); })')]
core/.../util/RandomTest.class.php:99:  #[Test, Action(eval: 'new VerifyThat(function() { return is_readable("/dev/urandom"); })')]
core/.../util/RandomTest.class.php:116:  #[Test, Expect(IllegalArgumentException::class), Action(eval: 'new VerifyThat(function() { return 0x7FFFFFFF === PHP_INT_MAX; })')]
core/.../util/RandomTest.class.php:121:  #[Test, Expect(IllegalArgumentException::class), Action(eval: 'new VerifyThat(function() { return 0x7FFFFFFF === PHP_INT_MAX; })')]
coverage/.../RecordCoverageTest.class.php:7:#[Action(eval: 'new VerifyThat(function() { return interface_exists(\unittest\Listener::class); })')]
reflection/.../TypeTest.class.php:111:  #[Test, Action(eval: 'new VerifyThat(fn() => !self::$ENUMS)')]
reflection/.../TypeTest.class.php:117:  #[Test, Action(eval: 'new VerifyThat(fn() => self::$ENUMS)')]
reflection/.../TypeTest.class.php:204:  #[Test, Action(eval: 'new VerifyThat(fn() => self::$ENUMS)')]
reflection/.../TypeTest.class.php:210:  #[Test, Action(eval: 'new VerifyThat(fn() => self::$ENUMS)')]
text-encode/.../Base64InputStreamTest.class.php:13:#[Action(eval: 'new VerifyThat(fn() => in_array("convert.*", stream_get_filters()))')]
text-encode/.../Base64OutputStreamTest.class.php:13:#[Action(eval: 'new VerifyThat(fn() => in_array("convert.*", stream_get_filters()))')]

Answer 5 · 2023-01-16T18:15:21.000Z

Transform values before comparison @ xp-framework/test#3

Answer 6 · 2023-01-16T18:30:19.000Z

https://github.com/xp-forge/assert has been set to public archive.

Answer 7 · 2023-01-16T18:52:17.000Z

Submitted to Packagist -> https://packagist.org/packages/xp-framework/test

Answer 8 · 2023-01-21T10:07:23.000Z

Generalized provider concept @ xp-framework/test#4

Answer 9 · 2023-01-29T19:35:04.000Z

First migration for Dialog done partially by migration script. The integration test needed to be migrated manually, as constructor arguments are not passed by default - see xp-framework/test#7

Answer 10 · 2023-01-29T21:33:53.000Z

Current library is less than 1000 LOC, compared to 2974 for xp-framework/unittest:

$ cloc-1.82.pl src/main/php/
      39 text files.
      39 unique files.
       9 files ignored.

github.com/AlDanial/cloc v 1.82  T=0.10 s (404.0 files/s, 15444.7 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
PHP                             39            221            368            902
-------------------------------------------------------------------------------
SUM:                            39            221            368            902
-------------------------------------------------------------------------------

Answer 11 · 2023-02-09T19:42:04.000Z

First release @ https://github.com/xp-framework/test/releases/tag/v1.0.0 🎉

Answer 12 · 2023-02-10T21:00:32.000Z

Early adapters:

Answer 13 · 2023-02-12T12:31:32.000Z

Project created for tracking progress: https://github.com/orgs/xp-framework/projects/3/views/1?layout=board

Answer 14 · 2023-02-12T16:39:47.000Z

All of xp-lang is now migrated, see here

Answer 15 · 2023-03-10T10:16:44.000Z

All of xp-forge is now migrated, see here

Answer 16 · 2023-06-25T16:41:10.000Z

Finally, XP Core is migrated 🎉

Answer 17 · 2023-06-25T16:54:15.000Z

The repository https://github.com/xp-framework/unittest is now archived

Answer 18 · 2024-03-10T10:34:34.000Z

Remaining libraries:

$ grep 'xp-framework/unittest' */composer.json | grep -v ^unittest
caching/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0"
coverage/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.7",
ftp/composer.json:    "xp-framework/unittest": "^11.0 | ^10.1"
imaging/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0"
io-collections/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.0 | ^8.0 | ^7.0 | ^6.5"
ldap/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.0 | ^8.0 | ^7.0"
mail/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.0 | ^8.0 | ^7.0"
measure/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.0 | ^8.0 | ^7.0"
meilisearch/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0"
mocks/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.0 | ^8.0 |^7.0",
patterns/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.0 | ^8.0 | ^7.0"
sql-parser/composer.json:    "xp-framework/unittest": "^11.1"
text-encode/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0"
webtest/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.0 | ^8.0",