can someone help me with this? sftp.createReadStream

[Manoharvuradi]

     await sftp.connect(sfgCredentials)
        .then(async () => {
            console.log("Connected to the FTP server...");
            if (recentFile) {
                const remoteFilePath = `/Inbox/${recentFile.name}`;
                const options = {
                    flags: 'r',
                    encoding: 'base64',  // Set encoding to 'base64'
                    autoClose: true,
                    start: 0,
                    end: 100
                };
                const readStream = sftp.createReadStream(remoteFilePath, options);
                let base64Data = '';
                 readStream.on('ready', () => {
                    console.log('Stream ready.');
                });

                readStream.on('data', (chunk) => {
                    console.log('Received base64 chunk:', chunk);
                    base64Data += chunk;
                });

            } else {
                console.error("No files named Employee_290MAC.001 found in /inbox");
            }
        }).then(() => {
            sftp.end();
        });

//In console
Connected to the FTP server...
node:events:495
throw er; // Unhandled 'error' event
^

Error: No response from server
at cleanupRequests (C:\Users\mahesh\Desktop\JS\node_modules\ssh2\lib\protocol\SFTP.js:2730:15)
at SFTP.push (C:\Users\mahesh\Desktop\JS\node_modules\ssh2\lib\protocol\SFTP.js:191:7)
at onCHANNEL_CLOSE (C:\Users\mahesh\Desktop\JS\node_modules\ssh2\lib\utils.js:50:13)
at ChannelManager.cleanup (C:\Users\mahesh\Desktop\JS\node_modules\ssh2\lib\utils.js:200:7)
at Socket. (C:\Users\mahesh\Desktop\JS\node_modules\ssh2\lib\client.js:831:21)
at Socket.emit (node:events:517:28)
at TCP. (node:net:350:12)
Emitted 'error' event on ReadStream instance at:
at Object.cb (C:\Users\mahesh\Desktop\JS\node_modules\ssh2\lib\protocol\SFTP.js:3710:12)
at cleanupRequests (C:\Users\mahesh\Desktop\JS\node_modules\ssh2\lib\protocol\SFTP.js:2734:11)
at SFTP.push (C:\Users\mahesh\Desktop\JS\node_modules\ssh2\lib\protocol\SFTP.js:191:7)
[... lines matching original stack trace ...]
at TCP. (node:net:350:12)

[theophilusx]

It isn't clear whay your trying to do. However - Don't mix Promise chains e.g. method().then()... and async/await - use one or the other. What you are doing shows some confusion regarding how promises and asynchronous JS works. - Why are you using createReadStream and not just using get()? In the ssh2-sftp-client github repository, there is a directory called 'example' which contains a number of exmaple scripts. You might find them useful.

[Manoharvuradi]

@theophilusx
I'm using createReadStream because, I'm trying to get the file in base64 format.

Thanks

[theophilusx]

You don't need to use createReadStream. The destination (2nd) argument to get() can be a write stream. You can setup a pipeline where a base64 decoder is piped into a file write stream and then pass the encoder as the destination argument. The file will be transferred, decoded and saved. I frequently use this technique to upload files and save them in gzip format and download a file in gzip format and save it in its unzipped format. This is definitely one of the powers of streams.

…

On Mon, 24 Jun 2024 at 19:23, Manohar Reddy ***@***.***> wrote: @theophilusx <https://github.com/theophilusx> I'm using createReadStream because, I'm trying to get the file in base64 format. Thanks — Reply to this email directly, view it on GitHub <#541 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMFIKNBO7ROEP3V7JUQGK3ZI7QQRAVCNFSM6AAAAABJZFZ2RSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBWGAZDIMRYGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- regards, Tim

-- Tim Cross

[Manoharvuradi]

Thanks @theophilusx

[Manoharvuradi]

Hello @theophilusx
using get() method can we give destination path as S3 bucket url S3://<ingest-bucket>/<file-basename>/<filename>

something like this
sftp.get(remoteFilePath, S3://<ingest-bucket>/<file-basename>/<filename>)

[theophilusx]

No. The destination can only be a string (a file path), a buffer or a stream. To support what you are asking for would require additional libs to create a remote write stream within S3. That would be too platform specific for this library which is focused just on sftp. Howeber, I don't think it would be too hard to add that abstraction to your own code.

[theophilusx]

I don't think there is anything more required here. Plan to close this job if I don't hear otherwise.