Requested shared memory is always less than captured
maksm90 opened this issue · 4 comments
Starting from commit postgres/postgres@cdbc0ca48ca shared preload libraries are become to initialize before MaxBackends global variable setup. This might lead to invalid calculation of requested shared memory size if this calculation is based on max number of possible backend processes in postgres instance. For pg_wait_sampling module the size calculation of queryId shared memory follows this rule, i.e. it uses MaxBackends value. As result, for this shared memory fragment we request zero size (UPD. not zero but NUM_AUXILIARY_PROCS + max_prepared_xacts size) but later we assign non-zero memory (with already initialized MaxBackends value) for this purpose. Such behavior might incur future occurrences of out of shared memory errors.
The hackers thread have to help in finding of possible solution.
FTR I noticed this problem a few days ago, which is mostly why I sent this email. I was waiting for an agreement on a new hook on the mailing list to send a PR here.
Note that even with the new hook the memory requested during _PG_init will still be wrong on pre pg15 version, so I can send a patch for that already if you want (which would be computing what should be MaxBackends, even if it can be underestimated if later modules changes some GUC, as I'm already doing in e.g. https://github.com/powa-team/pg_stat_kcache/blob/master/pg_stat_kcache.c#L672-L673)
Hi @rjuju !
FTR I noticed this problem a few days ago, which is mostly why I sent this email. I was waiting for an agreement on a new hook on the mailing list to send a PR here.
Thanks for working on this.
Note that even with the new hook the memory requested during _PG_init will still be wrong on pre pg15 version, so I can send a patch for that already if you want (which would be computing what should be MaxBackends, even if it can be underestimated if later modules changes some GUC, as I'm already doing in e.g. https://github.com/powa-team/pg_stat_kcache/blob/master/pg_stat_kcache.c#L672-L673)
Yes, you can if this doesn't bother you.
Sure!
Note that we're now two people thinking that the current GetMaxBackends() approach doesn't provide a clean basis to fix this problem so I don't think that there will be something available in pg15 for entirely fix the problem.