techfromsage/tripod-php

Discover impacted subjects job should handle large amounts of subjects

Opened this issue · 0 comments

When a discover impacted subject job runs, it finds all subjects grouped by queue and adds them to an apply operation job.

https://github.com/talis/tripod-php/blob/d441fdf5184298afb9e741711e43d4694edfdbea/src/mongo/jobs/DiscoverImpactedSubjects.class.php#L129

In some instances, the $subjects param can contain a lot of data and when the apply operation job tried to turn the subjects into an array, it can easily run out of memory:

https://github.com/talis/tripod-php/blob/d441fdf5184298afb9e741711e43d4694edfdbea/src/mongo/jobs/ApplyOperation.class.php#L82

We should think about we can queue up an apply operation job without being completed bound by how much memory is allocated to a PHP process.