An odd idea for a transformer... a progress meter

[kdebisschop]

It is not a normal ETL construct by any means, but what do you think of the idea of creating a transformer that periodically outputs some sort of progress indicator (and does nothing else)?

[Feeling a bit annoyed with myself for nit having a better way than counting rows in a database to measure progress :-( ]

[ecourtial]

The idea is interesting yes. In one of our project we had to do it, using the Symfony progress bar. However I am not sure if we should rely on any other component.

[kdebisschop]

Hmmm - I was even thinking just printing a line every N rows. But progress bar would be nice if it did not add (much) dependency. I'll see what I can work up.

[devfrey]

I had a use case for this and hooked up a Symfony ProgressBar to the internal Pipeline:

final class ProgressAwarePipeline extends Pipeline
{
    /** @var \Symfony\Component\Console\Helper\ProgressBar */
    private $progressBar;

    /**
     * @param  \Symfony\Component\Console\Helper\ProgressBar  $progressBar
     */
    public function __construct(ProgressBar $progressBar)
    {
        $this->progressBar = $progressBar;
    }

    public function rewind(): void
    {
        $this->progressBar->setProgress(0);

        parent::rewind();
    }

    public function next(): void
    {
        parent::next();

        $this->progressBar->setProgress($this->key);
    }

    protected function finalize(): void
    {
        parent::finalize();

        $this->progressBar->finish();
    }
}

Then new Etl(new ProgressAwarePipeline($progressBar)) will do the trick 👌

Not sure if it's the best way – but it works.

[ecourtial]

Nice example @devfrey
We could add it to the documentation :)

[ecourtial]

Hi @devfrey
We would like to add a examples section in the documentation. This could be a nice example with your integration of a progress bar.

• Documentation main level
• • ..
• • Examples
• • • Overriding the Pipeline: integration of a progress bar

Would you mind to create a PR ?

[ecourtial]

Added it into the documentation.