Awesome Package but Slow for my use-case.
rotexdegba opened this issue · 6 comments
rotexdegba commented
I am using this package to compare classes between two branches of a github repo.
Here is what I have in my composer.json
"roave/better-reflection": "^6.25"
Here is the output from
composer show -p
composer 2.7.1 Composer package
composer-plugin-api 2.6.0 The Composer Plugin API
composer-runtime-api 2.2.2 The Composer Runtime API
ext-apcu 5.1.21 The apcu PHP extension
ext-bcmath 8.1.27 The bcmath PHP extension
ext-bz2 8.1.27 The bz2 PHP extension
ext-calendar 8.1.27 The calendar PHP extension
ext-ctype 8.1.27 The ctype PHP extension
ext-curl 8.1.27 The curl PHP extension
ext-date 8.1.27 The date PHP extension
ext-dom 20031129 The dom PHP extension
ext-exif 8.1.27 The exif PHP extension
ext-fileinfo 8.1.27 The fileinfo PHP extension
ext-filter 8.1.27 The filter PHP extension
ext-ftp 8.1.27 The ftp PHP extension
ext-gd 8.1.27 The gd PHP extension
ext-gettext 8.1.27 The gettext PHP extension
ext-hash 8.1.27 The hash PHP extension
ext-iconv 8.1.27 The iconv PHP extension
ext-intl 8.1.27 The intl PHP extension
ext-json 8.1.27 The json PHP extension
ext-ldap 8.1.27 The ldap PHP extension
ext-libxml 8.1.27 The libxml PHP extension
ext-mbstring 8.1.27 The mbstring PHP extension
ext-mysqli 8.1.27 The mysqli PHP extension
ext-mysqlnd 0 The mysqlnd PHP extension (actual version: mysqlnd 8.1.27)
ext-openssl 8.1.27 The openssl PHP extension
ext-pcntl 8.1.27 The pcntl PHP extension
ext-pcre 8.1.27 The pcre PHP extension
ext-pdo 8.1.27 The PDO PHP extension
ext-pdo_mysql 8.1.27 The pdo_mysql PHP extension
ext-pdo_pgsql 8.1.27 The pdo_pgsql PHP extension
ext-pdo_sqlite 8.1.27 The pdo_sqlite PHP extension
ext-pgsql 8.1.27 The pgsql PHP extension
ext-phar 8.1.27 The Phar PHP extension
ext-posix 8.1.27 The posix PHP extension
ext-readline 8.1.27 The readline PHP extension
ext-reflection 8.1.27 The Reflection PHP extension
ext-session 8.1.27 The session PHP extension
ext-shmop 8.1.27 The shmop PHP extension
ext-simplexml 8.1.27 The SimpleXML PHP extension
ext-sockets 8.1.27 The sockets PHP extension
ext-spl 8.1.27 The SPL PHP extension
ext-sqlite3 8.1.27 The sqlite3 PHP extension
ext-sysvmsg 8.1.27 The sysvmsg PHP extension
ext-sysvsem 8.1.27 The sysvsem PHP extension
ext-sysvshm 8.1.27 The sysvshm PHP extension
ext-tokenizer 8.1.27 The tokenizer PHP extension
ext-xdebug 3.1.4 The xdebug PHP extension
ext-xml 8.1.27 The xml PHP extension
ext-xmlreader 8.1.27 The xmlreader PHP extension
ext-xmlwriter 8.1.27 The xmlwriter PHP extension
ext-xsl 8.1.27 The xsl PHP extension
ext-zend-opcache 8.1.27 The Zend OPcache PHP extension
ext-zip 1.20.1 The zip PHP extension
ext-zlib 8.1.27 The zlib PHP extension
lib-bz2 1.0.8 The bz2 library
lib-curl 7.76.1 The curl library
lib-curl-libssh 0.10.4 curl libssh version
lib-curl-openssl 3.0.7 curl OpenSSL version (3.0.7)
lib-curl-zlib 1.2.11 curl zlib version
lib-date-timelib 2021.19 date timelib version
lib-fileinfo-libmagic 540 fileinfo libmagic version
lib-gd 2.3.2 The gd library
lib-iconv 2.34 The iconv library
lib-icu 67.1 The ICU unicode and globalization support library
lib-icu-cldr 37 ICU CLDR project version
lib-icu-unicode 13.0.0 ICU unicode version
lib-icu-zoneinfo 2019.3 zoneinfo ("Olson") database for icu
lib-ldap-openldap 2.6.3 OpenLDAP version of ldap
lib-libxml 2.9.13 libxml library version
lib-libxslt 1.1.34 The libxslt library
lib-libxslt-libxml 2.9.13 libxml version libxslt is compiled against
lib-mbstring-libmbfl 1.3.2 mbstring libmbfl version
lib-mbstring-oniguruma 6.9.6 mbstring oniguruma version
lib-openssl 3.0.7 OpenSSL 3.0.7 1 Nov 2022
lib-pcre 10.40 The pcre library
lib-pcre-unicode 14.0.0 PCRE Unicode version support
lib-pdo_pgsql-libpq 16.2 libpq for pdo_pgsql
lib-pdo_sqlite-sqlite 3.34.1 The pdo_sqlite-sqlite library
lib-pgsql-libpq 16.2 libpq for pgsql
lib-sqlite3-sqlite 3.34.1 The sqlite3-sqlite library
lib-zip-libzip 1.7.3 The zip-libzip library
lib-zlib 1.2.11 The zlib library
php 8.1.27 The PHP interpreter
php-64bit 8.1.27 The PHP interpreter, 64bit
php-ipv6 8.1.27 The PHP interpreter, with IPv6 support
Below is the script I wrote:
<?php
include './vendor/autoload.php';
use Roave\BetterReflection\BetterReflection;
use Roave\BetterReflection\Reflector\DefaultReflector;
use Roave\BetterReflection\SourceLocator\SourceStubber\ReflectionSourceStubber;
use Roave\BetterReflection\SourceLocator\Type\AggregateSourceLocator;
use Roave\BetterReflection\SourceLocator\Type\PhpInternalSourceLocator;
use Roave\BetterReflection\SourceLocator\Type\DirectoriesSourceLocator;
ini_set('display_errors', '1' );
ini_set('error_reporting', E_ALL);
ini_set('memory_limit', '2048M');
function readableElapsedTime($microtime, $format = null, $round = 3) {
if (is_null($format)) {
$format = '%.3f%s';
}
if ($microtime >= 3600) {
$unit = ' hour(s)';
$time = round(($microtime / 3600), $round);
} elseif ($microtime >= 60) {
$unit = ' minute(s)';
$time = round(($microtime / 60), $round);
} elseif ($microtime >= 1) {
$unit = ' second(s)';
$time = round($microtime, $round);
} else {
$unit = 'ms';
$time = round($microtime*1000);
$format = preg_replace('/(%.[\d]+f)/', '%d', $format);
}
return sprintf($format, $time, $unit);
}
$tmp_dir = sys_get_temp_dir() . DIRECTORY_SEPARATOR;
if(is_writable($tmp_dir)) {
gc_enable();
$start_time = microtime(true);
$classes_to_compare = [
'\\LeanOrm\\Model',
'\\LeanOrm\\DBConnector',
'\\LeanOrm\\CachingModel',
'\\LeanOrm\\Model\\Collection',
'\\LeanOrm\\Model\\Record',
'\\LeanOrm\\Model\\ReadOnlyRecord',
];
$repo_url = 'https://github.com/rotexsoft/leanorm.git';
$repo_name = 'leanorm';
$left_branch = 'master';
$right_branch = '4.x';
$code_left_branch_path = "{$tmp_dir}{$repo_name}-{$left_branch}";
$code_right_branch_path = "{$tmp_dir}{$repo_name}-{$right_branch}";
$clone_left_branch = "git clone -b {$left_branch} {$repo_url} {$code_left_branch_path}";
$clone_right_branch = "git clone -b {$right_branch} {$repo_url} {$code_right_branch_path}";
(file_exists($code_left_branch_path)) || system($clone_left_branch); // if already cloned, file_exists($code_left_branch_path), don't clone again
(file_exists($code_right_branch_path)) || system($clone_right_branch); // if already cloned, file_exists($code_right_branch_path), don't clone again
$current_directory = __DIR__;
system("cd $code_left_branch_path && composer update"); // install composer dependencies
system("cd $code_right_branch_path && composer update"); // install composer dependencies
system ("cd {$current_directory}");
////////////////////////////////////////////////////////////////////////////
echo PHP_EOL ."Processing all Class Files...." . PHP_EOL;
$astLocator = (new BetterReflection())->astLocator();
$directoriesSourceLocator = new DirectoriesSourceLocator(
[
$code_left_branch_path . DIRECTORY_SEPARATOR . 'src',
$code_left_branch_path . DIRECTORY_SEPARATOR . 'vendor',
],
$astLocator
);
$reflector = new DefaultReflector(
new AggregateSourceLocator(
[
$directoriesSourceLocator,
new PhpInternalSourceLocator($astLocator, new ReflectionSourceStubber())
]
)
);
////////////////////////////////////////////////////////
$astLocator2 = (new BetterReflection())->astLocator();
$directoriesSourceLocator2 = new DirectoriesSourceLocator(
[
$code_right_branch_path . DIRECTORY_SEPARATOR . 'src',
$code_right_branch_path . DIRECTORY_SEPARATOR . 'vendor',
],
$astLocator2
);
$reflector2 = new DefaultReflector(
new AggregateSourceLocator(
[
$directoriesSourceLocator2,
new PhpInternalSourceLocator($astLocator, new ReflectionSourceStubber())
]
)
);
foreach($classes_to_compare as $class_to_compare) {
echo PHP_EOL ."Processing `{$class_to_compare}`...." . PHP_EOL;
////////////////////////////////////////////////////////
$reflectionClass = $reflector->reflectClass($class_to_compare);
$method_objs1 = $reflectionClass->getMethods();
$method_names1 = array_keys($method_objs1);
sort($method_names1);
//var_dump($method_names1);
////////////////////////////////////////////////////////
$reflectionClass2 = $reflector2->reflectClass($class_to_compare);
$method_objs2 = $reflectionClass2->getMethods();
$method_names2 = array_keys($method_objs2);
sort($method_names2);
//var_dump($method_names2);
$methods_in_left_not_in_right_branch = array_diff($method_names1, $method_names2);
$methods_in_right_branch_not_in_left = array_diff($method_names2, $method_names1);
echo PHP_EOL . "`{$class_to_compare}'s` Methods in `{$left_branch}` branch not in `{$right_branch}` branch:" . PHP_EOL;
echo implode(PHP_EOL, $methods_in_left_not_in_right_branch) . PHP_EOL;
echo PHP_EOL . "`{$class_to_compare}'s` Methods in `{$right_branch}` branch not in `{$left_branch}` branch:" . PHP_EOL;
echo implode(PHP_EOL, $methods_in_right_branch_not_in_left) . PHP_EOL;
echo PHP_EOL ."DONE: Processing `{$class_to_compare}`...." . PHP_EOL;
gc_collect_cycles();
}
$end_time = microtime(true);
$elapsed = $end_time - $start_time;
echo PHP_EOL . 'Time taken: ' . readableElapsedTime($elapsed). PHP_EOL. PHP_EOL;
} else {
echo PHP_EOL . "`{$tmp_dir}` is not writable. Exiting..." . PHP_EOL;
} // if(is_writable($tmp_dir)){...} else {...}Can someone give me any tips to make it run faster.
Thanks!!!!
Ocramius commented
The DirectoriesSourceLocator will do tons of sequential path scans: check if you can use a PSR locator or such instead.
rotexdegba commented
The
DirectoriesSourceLocatorwill do tons of sequential path scans: check if you can use a PSR locator or such instead.
Thanks for that tip
rotexdegba commented
@Ocramius I changed the follwoing lines in my script from:
system("cd $code_left_branch_path && composer update"); // install composer dependencies
system("cd $code_right_branch_path && composer update"); // install composer dependenciesto
system("cd $code_left_branch_path && composer install --no-dev"); // install composer dependencies
system("cd $code_right_branch_path && composer install --no-dev"); // install composer dependenciesThat has drastically improved the performance since huge dev dependencies like rector, psalm & phpunit are not being pulled in and scanned. It went from running in like close to 3 hours to about 3 minutes.
rotexdegba commented
Here's the more performant script that makes use of the composer install without dev dependencies and swaps out the DirectoriesSourceLocator with (new MakeLocatorForComposerJsonAndInstalledJson)->__invoke(string $installationPath, Locator $astLocator): SourceLocator . Runs under 2 minutes.
<?php
include './vendor/autoload.php';
use Roave\BetterReflection\BetterReflection;
use Roave\BetterReflection\Reflector\DefaultReflector;
use Roave\BetterReflection\SourceLocator\SourceStubber\ReflectionSourceStubber;
use Roave\BetterReflection\SourceLocator\Type\AggregateSourceLocator;
use Roave\BetterReflection\SourceLocator\Type\PhpInternalSourceLocator;
use Roave\BetterReflection\SourceLocator\Type\DirectoriesSourceLocator;
use Roave\BetterReflection\SourceLocator\Type\Composer\Factory\MakeLocatorForComposerJsonAndInstalledJson;
ini_set('display_errors', '1' );
ini_set('error_reporting', E_ALL);
ini_set('memory_limit', '2048M');
function readableElapsedTime($microtime, $format = null, $round = 3) {
if (is_null($format)) {
$format = '%.3f%s';
}
if ($microtime >= 3600) {
$unit = ' hour(s)';
$time = round(($microtime / 3600), $round);
} elseif ($microtime >= 60) {
$unit = ' minute(s)';
$time = round(($microtime / 60), $round);
} elseif ($microtime >= 1) {
$unit = ' second(s)';
$time = round($microtime, $round);
} else {
$unit = 'ms';
$time = round($microtime*1000);
$format = preg_replace('/(%.[\d]+f)/', '%d', $format);
}
return sprintf($format, $time, $unit);
}
$tmp_dir = sys_get_temp_dir() . DIRECTORY_SEPARATOR;
if(is_writable($tmp_dir)) {
gc_enable();
$start_time = microtime(true);
$classes_to_compare = [
'\\LeanOrm\\Model',
'\\LeanOrm\\DBConnector',
'\\LeanOrm\\CachingModel',
'\\LeanOrm\\Model\\Collection',
'\\LeanOrm\\Model\\Record',
'\\LeanOrm\\Model\\ReadOnlyRecord',
];
$repo_url = 'https://github.com/rotexsoft/leanorm.git';
$repo_name = 'leanorm';
$left_branch = 'master';
$right_branch = '4.x';
$code_left_branch_path = "{$tmp_dir}{$repo_name}-{$left_branch}";
$code_right_branch_path = "{$tmp_dir}{$repo_name}-{$right_branch}";
$clone_left_branch = "git clone -b {$left_branch} {$repo_url} {$code_left_branch_path}";
$clone_right_branch = "git clone -b {$right_branch} {$repo_url} {$code_right_branch_path}";
(file_exists($code_left_branch_path)) || system($clone_left_branch); // if already cloned, file_exists($code_left_branch_path), don't clone again
(file_exists($code_right_branch_path)) || system($clone_right_branch); // if already cloned, file_exists($code_right_branch_path), don't clone again
$current_directory = __DIR__;
system("cd $code_left_branch_path && composer install --no-dev"); // install composer dependencies
system("cd $code_right_branch_path && composer install --no-dev"); // install composer dependencies
system ("cd {$current_directory}");
////////////////////////////////////////////////////////////////////////////
echo PHP_EOL ."Processing all Class Files...." . PHP_EOL;
$astLocator = (new BetterReflection())->astLocator();
// $directoriesSourceLocator = new DirectoriesSourceLocator(
// [
// $code_left_branch_path . DIRECTORY_SEPARATOR . 'src',
// $code_left_branch_path . DIRECTORY_SEPARATOR . 'vendor',
// ],
// $astLocator
// );
$directoriesSourceLocator = (new MakeLocatorForComposerJsonAndInstalledJson)
(
$code_left_branch_path,
$astLocator
);
$reflector = new DefaultReflector(
new AggregateSourceLocator(
[
$directoriesSourceLocator,
new PhpInternalSourceLocator($astLocator, new ReflectionSourceStubber())
]
)
);
////////////////////////////////////////////////////////
$astLocator2 = (new BetterReflection())->astLocator();
// $directoriesSourceLocator2 = new DirectoriesSourceLocator(
// [
// $code_right_branch_path . DIRECTORY_SEPARATOR . 'src',
// $code_right_branch_path . DIRECTORY_SEPARATOR . 'vendor',
// ],
// $astLocator2
// );
$directoriesSourceLocator2 = (new MakeLocatorForComposerJsonAndInstalledJson)
(
$code_right_branch_path,
$astLocator2
);
$reflector2 = new DefaultReflector(
new AggregateSourceLocator(
[
$directoriesSourceLocator2,
new PhpInternalSourceLocator($astLocator, new ReflectionSourceStubber())
]
)
);
foreach($classes_to_compare as $class_to_compare) {
echo PHP_EOL ."Processing `{$class_to_compare}`...." . PHP_EOL;
////////////////////////////////////////////////////////
$reflectionClass = $reflector->reflectClass($class_to_compare);
$method_objs1 = $reflectionClass->getMethods();
$method_names1 = array_keys($method_objs1);
sort($method_names1);
//var_dump($method_names1);
////////////////////////////////////////////////////////
$reflectionClass2 = $reflector2->reflectClass($class_to_compare);
$method_objs2 = $reflectionClass2->getMethods();
$method_names2 = array_keys($method_objs2);
sort($method_names2);
//var_dump($method_names2);
$methods_in_left_not_in_right_branch = array_diff($method_names1, $method_names2);
$methods_in_right_branch_not_in_left = array_diff($method_names2, $method_names1);
echo PHP_EOL . "`{$class_to_compare}'s` Methods in `{$left_branch}` branch not in `{$right_branch}` branch:" . PHP_EOL;
echo implode(PHP_EOL, $methods_in_left_not_in_right_branch) . PHP_EOL;
echo PHP_EOL . "`{$class_to_compare}'s` Methods in `{$right_branch}` branch not in `{$left_branch}` branch:" . PHP_EOL;
echo implode(PHP_EOL, $methods_in_right_branch_not_in_left) . PHP_EOL;
echo PHP_EOL ."DONE: Processing `{$class_to_compare}`...." . PHP_EOL;
gc_collect_cycles();
}
$end_time = microtime(true);
$elapsed = $end_time - $start_time;
echo PHP_EOL . 'Time taken: ' . readableElapsedTime($elapsed). PHP_EOL. PHP_EOL;
} else {
echo PHP_EOL . "`{$tmp_dir}` is not writable. Exiting..." . PHP_EOL;
} // if(is_writable($tmp_dir)){...} else {...}asgrim commented
@rotexdegba btw I don't know if it fits your use case exactly, but we also have a tool - roave/backward-compatibility-check; it does more or less what you're doing there I think :)
That said, glad you are sorted! Closing this 🤘
rotexdegba commented
Thanks @asgrim