http request from js_body_filter njs script

[kalungedamaji]

we have a requirement to capture the whole request and response from the upstream 1 . We did this by using the js_body_filter njs script.
Need to send the original response from the upstream 1
Now facing challenges to send the captured request and response body to the API/SQS/Webhook due to only synchronous code is supported in the js_body_filter njs script
Need help to achieve above requirement.

Detailed configuration please refer https://gist.github.com/kalungedamaji/251da280f5ecf6e694a70dd28625faf3

[jo-carter]

Hello @kalungedamaji

It'd be better to use js_content with a subrequest rather than js_body_filter for this use-case. It:
A) performs full buffering of the response up to size set by subrequest_output_buffer_size (and more efficiently than manually creating your own buffer)
B) allows calls to asynchronous functions, such as ones needed for outbound requests such as fetch api (or subrequest).

Since js_content is a content handler, and you cannot have two content handlers in the same location context, a common pattern is to put js_content in the existing location (/hello in your example), and perform subrequest to another (internal) location to perform proxy_pass to backend.

You can then return result to the client, and then send response to external capture endpoint. A very simple (non generic) example:

...
location /hello {
    set $orig_uri $uri;
    js_content capture.handler;
}

location /_hello {
    internal;
    rewrite ^ $orig_uri break;
    ...
    proxy_pass ...
}

location =/_send-capture {
    internal;
    proxy_pass ...
}

#js
async function handler (r) {
    var proxyRes = await r.subrequest("/_hello", {body: r.requestBuffer, args: r.variables.args, method: r.method});
    Object.keys(proxyRes.headersOut).forEach(x => r.headersOut[x] = proxyRes.headersOut[x]);

    await r.subrequest("/_send-capture", {body: proxyRes.responseBuffer, method: "POST", detached: true});
    return r.return (proxyRes.status, proxyRes.responseBuffer);
}
export default {handler}

Finally, if you still must use js_body_filter for some reason - you may also be able to leverage an undocumented nginx directive post_action <location> which will process the specified location after regular content phase (js_content can be run there to send previously collected body buffer) - I'd advise against that approach unless there are no other options as it's quite the hack (and due it being undocumented).