lavary/crunz

When running more than one task at the same time, seems to only do the first one

jamminjames opened this issue · 17 comments

jamminjames commented

Crunz version: 2.3.0

PHP version: 7.4

Operating system type and version: Linux, Nginx server

Description
I have several jobs I'm trying to run at the same time, using:

    ->daily()
    ->at("$continuetime")

The $continuetime variable is retrieved from the db, and seems to work fine, but only on the first task that uses it. The others don't seem to fire. I'm verifying this by using ->sendOutputTo("logfile.log") on each task, with different log files, and only the first one gets written to.

Also, I tried switching one of them to ->everyMinute(), and it ran, so this also makes me suspect the ->at() method is not working for multiple files set for the same time of day.

Is this expected? Do I need to use different times if using the ->at() method?

PabloKowalczyk commented

Hello, did you enable preventOverlapping?
Could post whole task file and output of command vendor/bin/crunz s:r -vvv --force?

jamminjames commented

Would preventOverlapping be set in crunz.yml? It's not in there. Should it be enabled or disabled?

jamminjames commented

Post the whole task file in the CLI? Not sure how to do that.

PabloKowalczyk commented

preventOverlapping is set per event/task, like $event->preventOverlapping().

Post the whole task file in the CLI? Not sure how to do that.

I mean post here, on GitHub :)

jamminjames commented

Not using ->preventOverlapping(), should I? Not sure how it works. From the documentation, it seems it would prevent a file from running if another is running at the same time, right? So if I don't use it, they should work okay? Or should I use it for each task that runs at the same time, and then they'll run sequentially?

PabloKowalczyk commented

it seems it would prevent a file from running if another is running at the same time, right? So if I don't use it, they should work okay?

Right.

Please post your task file here.

jamminjames commented

Okay, it's very involved though. To simplify, should I post only the tasks running at the same time? Some of them use functions, with variables from db, passed with use ().

PabloKowalczyk commented

Only PHP code of Crunz tasks.

jamminjames commented

Thanks for looking. Here it is:

use Crunz\Schedule;

$schedule->run('/usr/bin/php renewalemail.php')
    ->daily()
    ->at("$continuetime")
	//->everyMinute()
	->sendOutputTo('log/cron_finishemails.log')
	->description('Continue emails daily until done, limited by Renewal Notice Criteria setting in renewalemail.php.')
	->when(function() use ($lastrenewid,&$cronrunning,$apcuAvailable,$nextqueue,$db){
		$thisjob = 1;
		if ($cronrunning==$thisjob) {
			return true;
		} elseif ($lastrenewid==0 && $cronrunning==0 && $nextqueue==$thisjob) {
			$stmt = $db->prepare('update control set cronrunning=?');
			$stmt->bind_param('i', $thisjob);
			$stmt->execute();
			$stmt->close();

			$cronrunning=$thisjob;
			if ($apcuAvailable) {
				apcu_store('cronrunning', $cronrunning);
		    }
			return true;
		} else { return false; }
	});

$schedule->run('/usr/bin/php bt_billing_notice.php')
    ->daily()
    ->at("$continuetime")
	//->everyMinute()
	->sendOutputTo('log/cron_finish_bill_notice.log')
	->description('Continue recurring billing notice emails daily until done, limited by setting in bt-bill-reminder.php.')
	->when(function() use ($lastrenewid,&$cronrunning,$nextqueue,$recurdate,$apcuAvailable,$db){
		$thisjob = 2;
		if ($recurdate==0) {
			return false;
		} elseif ($cronrunning==$thisjob) {
			return true;
		} elseif ($lastrenewid==0 && $cronrunning==0 && $nextqueue==$thisjob) {
			$stmt = $db->prepare('update control set cronrunning=?');
			$stmt->bind_param('i', $thisjob);
			$stmt->execute();
			$stmt->close();

			$cronrunning=$thisjob;
			if ($apcuAvailable) {
				apcu_store('cronrunning', $cronrunning);
		    }
			return true;
		} else { return false; }
	});

$schedule->run('/usr/bin/php pdfissueemail.php')
    ->daily()
    ->at("$continuetime")
	//->everyMinute()
	->sendOutputTo('log/cron_PDFemails.log')
	->description('Continue PDF emails daily until done, limited by Renewal Notice Criteria setting in renewalcriteria.php.')
	->when(function() use ($lastrenewid,&$cronrunning,$nextqueue,$apcuAvailable,$db){
		$thisjob = 3;
		if ($cronrunning==$thisjob) {
			return true;
		} elseif ($lastrenewid==0 && $cronrunning==0 && $nextqueue==$thisjob) {
			$stmt = $db->prepare('update control set cronrunning=?');
			$stmt->bind_param('i', $thisjob);
			$stmt->execute();
			$stmt->close();

			$cronrunning=$thisjob;
			if ($apcuAvailable) {
				apcu_store('cronrunning', $cronrunning);
		    }
			return true;
		} else { return false; }
	});

$schedule->run('/usr/bin/php customemail.php')
    ->daily()
    ->at("$continuetime")
	//->everyMinute()
	->sendOutputTo('log/cron_customemails.log')
	->description('Continue custom emails daily until done, limited by Renewal Notice Criteria setting in renewalcriteria.php.')
	->when(function() use ($lastrenewid,&$cronrunning,$nextqueue,$apcuAvailable,$db){
		$thisjob = 4;
		if ($cronrunning==$thisjob) {
			return true;
		} elseif ($lastrenewid==0 && $cronrunning==0 && $nextqueue==$thisjob) {
			$stmt = $db->prepare('update control set cronrunning=?');
			$stmt->bind_param('i', $thisjob);
			$stmt->execute();
			$stmt->close();

			$cronrunning=$thisjob;
			if ($apcuAvailable) {
				apcu_store('cronrunning', $cronrunning);
		    }
			return true;
		} else { return false; }
	});

$schedule->run('/usr/bin/php incompleteemail.php')
	->daily()
	->at("$continuetime")
	->sendOutputTo('log/cron_incompleteemail.log')
	->description('Pending sub reminder email.')
	->when(function() use ($pendingcronjob){
		if ($pendingcronjob) {	return true; }
	});

return $schedule;
PabloKowalczyk commented

Now, could you run vendor/bin/crunz s:r -vvv --force and paste output here?

jamminjames commented

Here you go:

Using config file /var/www/subs.humortimes/crunz.yml.
Task source path '/var/www/subs.humortimes/tasks'
Task finder suffix: 'Tasks.php'
Realpath for '/var/www/subs.humortimes/tasks' is '/var/www/subs.humortimes/tasks'
Found 1 task(s) at path '/var/www/subs.humortimes/tasks'
Doing HumorTimesTasks.php.<br>apcuAvailable? 1<br>NOT using apcu<br>db crontab: 05 03 11 * *<BR>
db lastrenewid: 8824<BR>
db cronjob: 1<BR>
db cronqueue: <BR>
db cronrunning: 1<BR>
PHP Notice:  Undefined variable: nextqueue in /var/www/subs.humortimes/tasks/HumorTimesTasks.php on line 105
db nextqueue: <BR>
continuetime: 03:04<br>
added to apcu, fetch crontab: 05 03 11 * *<br>added to apcu, fetch continuetime: 03:04<br>before Finish Emails<BR>
before Fetchmail<BR>
Finished HumorTimesTasks.php.<br>Timezone from config: 'America/Los_Angeles'.
Timezone for comparisons: 'America/Los_Angeles'.
Invoke Schedule's ping before
There is no ping before url.
Class for 'logger_factory': 'Crunz\Infrastructure\Psr\Logger\PsrStreamLoggerFactory'.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping after.
There is no ping after url.

In PsrStreamLogger.php line 116:

  [Crunz\Exception\CrunzException]
  Stream path cannot be empty.


Exception trace:
  at /var/www/subs.humortimes/vendor/lavary/crunz/src/Infrastructure/Psr/Logger/PsrStreamLogger.php:116
 Crunz\Infrastructure\Psr\Logger\PsrStreamLogger->initializeHandler() at /var/www/subs.humortimes/vendor/lavary/crunz/src/Infrastructure/Psr/Logger/PsrStreamLogger.php:96
 Crunz\Infrastructure\Psr\Logger\PsrStreamLogger->createInfoHandler() at /var/www/subs.humortimes/vendor/lavary/crunz/src/Infrastructure/Psr/Logger/PsrStreamLogger.php:67
 Crunz\Infrastructure\Psr\Logger\PsrStreamLogger->log() at /var/www/subs.humortimes/vendor/lavary/crunz/src/Infrastructure/Psr/Logger/EnabledLoggerDecorator.php:47
 Crunz\Infrastructure\Psr\Logger\EnabledLoggerDecorator->log() at /var/www/subs.humortimes/vendor/lavary/crunz/src/Logger/Logger.php:37
 Crunz\Logger\Logger->log() at /var/www/subs.humortimes/vendor/lavary/crunz/src/Logger/Logger.php:23
 Crunz\Logger\Logger->info() at /var/www/subs.humortimes/vendor/lavary/crunz/src/EventRunner.php:201
 Crunz\EventRunner->handleOutput() at /var/www/subs.humortimes/vendor/lavary/crunz/src/EventRunner.php:138
 Crunz\EventRunner->manageStartedEvents() at /var/www/subs.humortimes/vendor/lavary/crunz/src/EventRunner.php:74
 Crunz\EventRunner->handle() at /var/www/subs.humortimes/vendor/lavary/crunz/src/Console/Command/ScheduleRunCommand.php:152
 Crunz\Console\Command\ScheduleRunCommand->execute() at /var/www/subs.humortimes/vendor/symfony/console/Command/Command.php:256
 Symfony\Component\Console\Command\Command->run() at /var/www/subs.humortimes/vendor/symfony/console/Application.php:971
 Symfony\Component\Console\Application->doRunCommand() at /var/www/subs.humortimes/vendor/symfony/console/Application.php:290
 Symfony\Component\Console\Application->doRun() at /var/www/subs.humortimes/vendor/symfony/console/Application.php:166
 Symfony\Component\Console\Application->run() at /var/www/subs.humortimes/vendor/lavary/crunz/src/Application.php:98
 Crunz\Application->run() at /var/www/subs.humortimes/vendor/lavary/crunz/crunz:71

schedule:run [-f|--force] [-t|--task TASK] [--] [<source>]
PabloKowalczyk commented

As you can see there are two issues, first:

PHP Notice: Undefined variable: nextqueue in /var/www/subs.humortimes/tasks/HumorTimesTasks.php on line 105

seems like error in task file, not in Crunz.

Second is:

In PsrStreamLogger.php line 116:

[Crunz\Exception\CrunzException]
Stream path cannot be empty.

and this can be Crunz error or wrong configuration.

Please post your configuration here.

jamminjames commented

I assume you mean the crunz.yml file. Here it is:

# Crunz Configuration Settings

# This option defines where the task files and
# directories reside.
# The path is relative to this config file.
# Trailing slashes will be ignored.
source: tasks

# The suffix is meant to target the task files inside the ":source" directory.
# Please note if you change this value, you need
# to make sure all the existing tasks files are renamed accordingly.
suffix: Tasks.php

# Timezone is used to calculate task run time
# This option is very important and not setting it is deprecated
# and will result in exception in 2.0 version.
timezone: America/Los_Angeles

# By default the errors are not logged by Crunz
# You may set the value to true for logging the errors
log_errors: false

# This is the absolute path to the errors' log file
# You need to make sure you have the required permission to write to this file though.
errors_log_file: /var/www/subs.humortimes/log/cron_errors.log

# By default the output is not logged as they are redirected to the
# null output.
# Set this to true if you want to keep the outputs
log_output: true

# This is the absolute path to the global output log file
# The events which have dedicated log files (defined with them), won't be
# logged to this file though.
output_log_file: ~

# By default line breaks in logs aren't allowed.
# Set the value to true to allow them.
log_allow_line_breaks: false

# This option determines whether the output should be emailed or not.
email_output: true

# This option determines whether the error messages should be emailed or not.
email_errors: true

# Global Swift Mailer settings
mailer:
    # Possible values: smtp, mail, and sendmail
    transport: smtp
    recipients:
    sender_name:
    sender_email:


# SMTP settings
smtp:
    host: ~
    port: ~
    username: ~
    password: ~
    encryption: ~
PabloKowalczyk commented

I assume you mean the crunz.yml file. Here it is:

Yep, the crunz.yml.
Looks like log_output is enabled, but there are null path in output_log_file. Try specifying log file and run Crunz again.

jamminjames commented

Here you go:

Using config file /var/www/subs.humortimes/crunz.yml.
Task source path '/var/www/subs.humortimes/tasks'
Task finder suffix: 'Tasks.php'
Realpath for '/var/www/subs.humortimes/tasks' is '/var/www/subs.humortimes/tasks'
Found 1 task(s) at path '/var/www/subs.humortimes/tasks'
Doing HumorTimesTasks.php.<br>apcuAvailable? 1<br>NOT using apcu<br>db crontab: 05 03 11 * *<BR>
db lastrenewid: 0<BR>
db cronjob: 1<BR>
db cronqueue: 4<BR>
db cronrunning: 3<BR>
db nextqueue: 4<BR>
continuetime: 03:04<br>
added to apcu, fetch crontab: 05 03 11 * *<br>added to apcu, fetch continuetime: 03:04<br>before Finish Emails<BR>
before Fetchmail<BR>
Finished HumorTimesTasks.php.<br>Timezone from config: 'America/Los_Angeles'.
Timezone for comparisons: 'America/Los_Angeles'.
Invoke Schedule's ping before
There is no ping before url.
Class for 'logger_factory': 'Crunz\Infrastructure\Psr\Logger\PsrStreamLoggerFactory'.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping before.
There is no ping before url.
Invoke Event's ping after.
There is no ping after url.
Task TEST crontab variable. status: success.
Invoke Event's ping after.
There is no ping after url.
Task Start a monthly renewal email, if cron enabled. status: success.
Invoke Event's ping after.
There is no ping after url.
Task Continue emails daily until done, limited by Renewal Notice Criteria setting in renewalemail.php. status: success.
Invoke Event's ping after.
There is no ping after url.
Task Start recurring billing notice emails. status: success.
Invoke Event's ping after.
There is no ping after url.
Task Continue recurring billing notice emails daily until done, limited by setting in bt-bill-reminder.php. status: success.
Invoke Event's ping after.
There is no ping after url.
Task Continue custom emails daily until done, limited by Renewal Notice Criteria setting in renewalcriteria.php. status: success.
Invoke Event's ping after.
There is no ping after url.
Task Pending & recurring accounts change notice. status: success.
Invoke Event's ping after.
There is no ping after url.
Task Run bad-email-addr.php 15 minutes after fetchmail to trigger admin report. status: success.
Invoke Event's ping after.
There is no ping after url.
Task refresh variables in apcu status: success.
Invoke Event's ping after.
There is no ping after url.
Task Pending sub reminder email. status: success.
Task Fetchmail polls Omsoft email folder looking for remove requests and returned emails, and uses removemail.php. status: fail.
Invoke Event's ping after.
There is no ping after url.
Task Continue PDF emails daily until done, limited by Renewal Notice Criteria setting in renewalcriteria.php. status: success.
Invoke Schedule's ping after.
There is no ping after url.
PabloKowalczyk commented

Looks like one task failed: Task Fetchmail polls Omsoft email folder looking for remove requests and returned emails, and uses removemail.php. status: fail., i don't know why but it, for 99,99%, is not Crunz bug.

jamminjames commented

Okay, looks like all the other ones ran. Thanks!!!