Allowed memory size exhausted when try to search

[jsarola]

Hello,

I have an installation with a mailbox with 1500000 conversations, the system works correctly but when I try to do a search I get a PHP error:

"PHP message: PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 33554440 bytes) in /opt/freescout/overrides/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php on line 262" while reading response header from upstream, client: 10.1.1.1.1, server: freescout.my.lan, request: "GET /search?f%5Bmailbox%5D=13 HTTP/2.0", upstream: "fastcgi://unix:/var/run/php/php-fpm.sock:", host: "freescout.my.lan", referrer: "https://freescout.my.lan/mailbox/13"

I think the problem is in the fetchAll, which fills the memory. I don't know if it's a bug, or if it can be solved in another way (not doing a fetch all but doing a query of fewer records)

I don't know if the module melisearch can fix this problem.

Regards,

PHP version: 8.3.9
FreeScout version: 1.8.150
Database: MySQL (8.0.37)
Are you using CloudFlare: No
Are you using non-official modules: No

[freescout-help]

FreeScout fetches only conversations visible on the search page and uses pagination. Try to find and send full error message with the stack trace.

[mywebhero]

Have you tried to increase you PHP memory allocation and see if that solve the problem?

[jsarola]

Hi, the only error founded is this:

[2024-08-26 09:20:12] production.ERROR: Allowed memory size of 536870912 bytes exhausted (tried to allocate 33554440 bytes) {"userId":2,"email":"myuser@my.lan","exception":"[object] (Symfony\\Component\\Debug\\Exception\\FatalErrorException(code: 1): Allowed memory size of 536870912 bytes exhausted (tried to allocate 33554440 bytes) at /opt/freescout/overrides/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php:262) [stacktrace] #0 {main} "}

And it occurs when I select the button search without any search string. Can I do more trace with some .env configuration? It's possible to send errors to sentry?

Second comment:
I doubled memory to php and now it works (1024M), but... is this the solution? This mailbox it's now with 1.500.000 conversations, but I'm migrating a helpscout mailbox with 10.000.000 conversations, I have to change php memory allocation to 10GB?

[mywebhero]

The memory issue you're encountering is mainly due to large email threads. To address this, you might want to try the Faster Search module, Faster Search Module

For a more fundamental solution, the code needs to implement pagination. This involves breaking down the search results into smaller chunks, which is how databases and PHP handle large datasets more efficiently.

[jsarola]

The memory issue you're encountering is mainly due to large email threads. To address this, you might want to try the Faster Search module, Faster Search Module

For a more fundamental solution, the code needs to implement pagination. This involves breaking down the search results into smaller chunks, which is how databases and PHP handle large datasets more efficiently.

Thanks, I'll test the module and pass on information on how it went.