TaskFinder::getTasks() should not include Example tasks
Closed this issue · 6 comments
Not really performance impacting, but it adds a lot of clutter in the query task and make database logs heavier and difficult to read.
I would expect that task names ending with "Example" are not included in available tasks.
Maybe a continue; in the foreach (or array_filter after it) can be added with some check like strelen($name) >= 7 && 0 === substr_compare($name, "Example", -7, 7);
Not sure opening a PR because I don't know if it will break tests expecting these tasks to be present. If it's needed maybe there's a way to skip only when not running tests? Or an option in the configuration array that one can set when in production env?
SELECT
(
IFNULL(
TIMESTAMPDIFF(
SECOND, "2022-02-28 16:07:57", notbefore
),
0
)
) AS `age`,
`QueuedJobs`.`id` AS `QueuedJobs__id`,
`QueuedJobs`.`job_task` AS `QueuedJobs__job_task`,
`QueuedJobs`.`data` AS `QueuedJobs__data`,
`QueuedJobs`.`job_group` AS `QueuedJobs__job_group`,
`QueuedJobs`.`reference` AS `QueuedJobs__reference`,
`QueuedJobs`.`created` AS `QueuedJobs__created`,
`QueuedJobs`.`notbefore` AS `QueuedJobs__notbefore`,
`QueuedJobs`.`fetched` AS `QueuedJobs__fetched`,
`QueuedJobs`.`completed` AS `QueuedJobs__completed`,
`QueuedJobs`.`progress` AS `QueuedJobs__progress`,
`QueuedJobs`.`failed` AS `QueuedJobs__failed`,
`QueuedJobs`.`failure_message` AS `QueuedJobs__failure_message`,
`QueuedJobs`.`workerkey` AS `QueuedJobs__workerkey`,
`QueuedJobs`.`status` AS `QueuedJobs__status`,
`QueuedJobs`.`priority` AS `QueuedJobs__priority`
FROM
`queued_jobs` `QueuedJobs`
WHERE
(
(`completed`) IS NULL
AND (
(
`job_task` = 'GeneratePreview'
AND (
(
`notbefore` < '2022-02-28 16:07:57'
OR (`notbefore`) IS NULL
)
AND (
`fetched` < '2022-02-28 16:06:57'
OR (`fetched`) IS NULL
)
)
AND `failed` < 5
AND UNIX_TIMESTAMP() >= '1646060866'
)
OR (
`job_task` = 'Queue.UniqueExample'
AND (
(
`notbefore` < '2022-02-28 16:07:57'
OR (`notbefore`) IS NULL
)
AND (
`fetched` < '2022-02-28 16:06:57'
OR (`fetched`) IS NULL
)
)
AND `failed` < 5
)
OR (
`job_task` = 'Queue.ExceptionExample'
AND (
(
`notbefore` < '2022-02-28 16:07:57'
OR (`notbefore`) IS NULL
)
AND (
`fetched` < '2022-02-28 16:07:47'
OR (`fetched`) IS NULL
)
)
AND `failed` < 5
)
OR (
`job_task` = 'Queue.CostsExample'
AND (
(
`notbefore` < '2022-02-28 16:07:57'
OR (`notbefore`) IS NULL
)
AND (
`fetched` < '2022-02-28 16:06:57'
OR (`fetched`) IS NULL
)
)
AND `failed` < 5
)
OR (
`job_task` = 'Queue.Email'
AND (
(
`notbefore` < '2022-02-28 16:07:57'
OR (`notbefore`) IS NULL
)
AND (
`fetched` < '2022-02-28 16:05:57'
OR (`fetched`) IS NULL
)
)
AND `failed` < 5
)
OR (
`job_task` = 'Queue.MonitorExample'
AND (
(
`notbefore` < '2022-02-28 16:07:57'
OR (`notbefore`) IS NULL
)
AND (
`fetched` < '2022-02-28 16:07:47'
OR (`fetched`) IS NULL
)
)
AND `failed` < 5
)
OR (
`job_task` = 'Queue.ProgressExample'
AND (
(
`notbefore` < '2022-02-28 16:07:57'
OR (`notbefore`) IS NULL
)
AND (
`fetched` < '2022-02-28 16:05:57'
OR (`fetched`) IS NULL
)
)
AND `failed` < 5
)
OR (
`job_task` = 'Queue.SuperExample'
AND (
(
`notbefore` < '2022-02-28 16:07:57'
OR (`notbefore`) IS NULL
)
AND (
`fetched` < '2022-02-28 16:07:47'
OR (`fetched`) IS NULL
)
)
AND `failed` < 5
)
OR (
`job_task` = 'Queue.RetryExample'
AND (
(
`notbefore` < '2022-02-28 16:07:57'
OR (`notbefore`) IS NULL
)
AND (
`fetched` < '2022-02-28 16:07:47'
OR (`fetched`) IS NULL
)
)
AND `failed` < 5
)
OR (
`job_task` = 'Queue.Execute'
AND (
(
`notbefore` < '2022-02-28 16:07:57'
OR (`notbefore`) IS NULL
)
AND (
`fetched` < '2022-02-28 16:06:57'
OR (`fetched`) IS NULL
)
)
AND `failed` < 5
)
OR (
`job_task` = 'Queue.Example'
AND (
(
`notbefore` < '2022-02-28 16:07:57'
OR (`notbefore`) IS NULL
)
AND (
`fetched` < '2022-02-28 16:07:47'
OR (`fetched`) IS NULL
)
)
AND `failed` < 5
)
)
)
ORDER BY
`priority` ASC,
`age` ASC,
`id` ASC
LIMIT
1 FOR
UPDATE
;
Well, removing them would mean that you could not run them anymore, either locally or on prod
If you want to test it there.
Instead, maybe we could provide a blacklist of sorts?
protected function getTaskConf(): array {
if (!is_array($this->taskConf)) {
$tasks = (new TaskFinder())->all();
$this->taskConf = Config::taskConfig($tasks);
}
return $this->taskConf;
}So this all() method would just need to throw out the ones you added to that list to be ignored.
E.g. a config option ignoredTasks or alike?
I don't want to delete the tasks, just not include them when the queue is running on the prod app.
I think I'm missing your point of providing an explicit blacklist. Why it's needed if the tasks you want to filter out share the ending "Example" in the name?
Is there some method/variable we could use to determine that the test is running?
Or maybe if the test includes the task directly and not through the TaskFinder->all() method, Examples could be moved to a different namespace?
Well, how else would you want to have a different env on prod vs local.
I guess we could make it based on debug mode, but then people want to run Example ones on prod would lose that part.
I think you misunderstand Examples:
They are not for tests (phpunit), they are for demo running and testing your live queue.
So they are for some people very much needed outside of tests.
See also https://sandbox.dereuromark.de/sandbox/queue-examples
Those would not run if we excluded them by default for prod.
I much rather would have a backlist to exclude them for those that are sure they dont need or want them there.
Imagine other plugins that ship with 3-5 queue tasks, but you only need 1 of those each.
So the others could be blacklisted, and we have a generic solution that could fit everyone's need.
I think you misunderstand Examples:
Oh yes, I didn't consider that. The blacklist sounds fine then. Were you thinking about a config array or a callable?
I think a simple array of string names or class names should suffice.
Feel free to make a PR proposal.