Request: mixed sync/async transports

[FoxxMD]

When using multistream or specifying any transports pino always runs all transports on separate threads (async). The usual advice to users who want pino to be synchronous is to use the destination arg for pino init and use (or specify) a sync-capable transport. destination only works for a single transport, though.

However the wording used in most places seems to imply that transports can be run either/or because of program design but not because it's actually impossible to run both.

There are more closed issues than those mentioned above that follow this common scenario:

• Author wants synchronous output to stdout/stderr IE console/docker output because immediacy and order is important for debugging or display to a real-time user
• Author also wants async output to file or another transport which is not important for real-time

I've tried to dig through pino's source to determine what reasoning is for not supporting this scenario is but it's not clear to me. Can you please clarify and, if possible, consider:

Does pino not support mixed transport sync/async due to a design decision/philosophy or due to a technical limitation? If its not a technical issue could it be implemented and what would it take (at a high level?)

Thanks

[mcollina]

Yes it's possible but not well documented. You can run thread-stream in sync mode, blocking the main thread until all data has been delivered to the transports.

However in your case I would use a pino.multistream(), with two pino.destination(), one sync and one async.

[FoxxMD]

Thanks for the guidance. Using multistream and specifying sync when using destination() does work! For anyone else finding this here's an example that meets the requirements I laid out (typescript):

import {destination, StreamEntry, Logger, pino} from "pino";
import path from "path";
import pRoll from "pino-roll";
//import pinoElastic from "pino-elasticsearch";

export const getPinoLogger = async (options?: any /* for later use */): Promise<Logger> => {

    // SYNCHRONOUS logging to stdout (console)
    const streams: StreamEntry[] = [
        {
            level: 'debug',
            stream: destination({dest: 1, sync: true})
        }
    ];

    // async file logging using plain pino.destination
    streams.push({
        level: 'warn',
        stream: destination({
            dest: path.resolve('logs', 'app-single.log'),
            mkdir: true,
            sync: false,
        })
    });

    // async file logging using pino-roll (log rotation)
    streams.push({
        level: 'warn',
        stream: await pRoll({
            file: path.resolve('logs', 'app'),
            frequency: 'hourly',
            extension: '.log',
            mkdir: true,
            sync: false,
        })
    });

    // async(?) logging to elasticsearch
    // streams.push({
    //     level: 'debug',
    //     stream: await pinoElastic({
    //         index: 'an-index',
    //         node: 'http://localhost:9200',
    //         esVersion: 7,
    //         flushBytes: 1000
    //     })
    // });

    return pino({}, pino.multistream(streams));
}

This comment is what led me astray during my research:

Note that multistream can only be asynchronous (by its nature).

Was the author mistaken or did functionality change between time of posting and now?

[mcollina]

That sentence is incorrect, would you like to amend it?

[FoxxMD]

Oh I'm sorry I didn't realize you were the comment author 😅 Had some tunnel vision there...didn't mean for my comment to come off as passive aggressive. Yes please! A correction on the comment would be helpful for future users and I hope this issue also lends some clarification.

Thanks for the help, I think my problem has been adequately resolved here. If I have time I'll see if I can contribute to the docs in meaningful way to clarify these points and provide an example.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.