brianc/node-pg-copy-streams

Strange output for on('data', ...) event

Closed this issue · 1 comments

The code looks like this

const copyFrom = require('pg-copy-streams').from;


const columns = [
        'col0',
        'col1', 
        'col2',
        'col3',
        'col4',
        'col5',
        'col6',
        'col7',
        'col8',
        'col9'
];

let sql = `COPY my_table (${columns.join(',')}) FROM STDIN (DELIMITER '|', NULL '')`;
const client = new pg.Client(...);
await client.connect();
const fromStream = client.query(copyFrom(sql));
fromStream.on('data', function (data) {
                console.log(' ~~~ DATA ~~~', data.toString());
            });

fromStream.write('XXXXXX001|123456|alpha|1991||49|25|13145253.38|13145253.38|100');
fromStream.write('XXXXXX002|123456|alpha|1991||49|25|13145253.38|13145253.38|100');
fromStream.end();

Event on('data'), however is printing additional lines.


~~~ DATA ~~~~ dB
~~~ DATA ~~~~ XXXXXX001|123456|alpha|1991||49|25|13145253.38|13145253.38|100
~~~ DATA ~~~~ dB
~~~ DATA ~~~~ XXXXXX002|123456|alpha|1991||49|25|13145253.38|13145253.38|100
~~~ DATA ~~~~ c

What am I missing?
It is not inserting data into the table so while debugging I found this so asking if that is the bug or expected.

Hello,
thanks for the report. I don't have time to reproduce your output right now. Until then here are a few points :

  • do you have data in the database when you remove the on('data'). I always try to avoid on('data') because of the side-effects it has on node streams
  • you should have end-of-line separators at the end of your "csv" stdin input
  • fromStream is a writable ; it is only for internal reasons that it may look like a readable so this is a double warning for on('data') having unsuspected effects
  • make sure the table is already created and have the correct schema

keep me posted on your tests.