Server crashes in case of multiple blobService.listBlobsSegmentedWithPrefix calls

[sunil-jhamnani]

For latest features support, please switch to Azure Storage JavaScript SDK V10.

Which service(blob, file, queue, table) does this issue concern?

blob

Which version of the SDK was used?

azure-storage: "2.10.3"

What's the Node.js/Browser version?

Node: v10.15

What problem was encountered

Express server crashes in case of multiple blobService.listBlobsSegmentedWithPrefix calls

Steps to reproduce the issue?

function listAllFilesInPath(req, res, folderPath, appName, continuationToken = null, options) {
  const { blobService, container } = getBlobService(req, appName)
  return new Promise((resolve, reject) => {
    blobService.listBlobsSegmentedWithPrefix(container, folderPath, continuationToken === 'undefined' ? null : continuationToken, options, (err, result) => {
      if (err) {
        reject(err)
      } else {
        resolve(result)
      }
    });
  });
}

function getAllFilesInPath(req, res, folderPath, appName, continuationToken, options) {
  return new Promise((resolve, reject) => { 
    this.listAllFilesInPath(req, res, folderPath, appName, continuationToken, options)
    .then(result => {
      const files = []
      result.entries.forEach(entry => {
        const fileName = entry.name.replace(`${folderPath}`, ''); 
        const creationTime = entry.creationTime;
        const lastModified = entry.lastModified;
        const size = entry.contentLength;
        files.push({fileName, lastModified, creationTime, size, fileNameWithPath : entry.name});
        });
        resolve({status: 'SUCCESS', files, 'continuationToken' : result.continuationToken});
    }).catch(err => {
        reject({status: 'ERROR', message: err});
    })});
}

let daywiseFiles = await Promise.all(daysToFetch.map(async(day, index) => {
        let folderPath = format(baseFolderPath, {seller_id, month, year, day});
        const resultsForDay = await azureStorage.getAllFilesInPath(req, res, folderPath, appName);
      }))

This the code I have right now to list all the blobs. The issue is for the daywiseFiles variable. When daysToFetch is lets say 30, the function gets called 30 times which is crashing my express server with this error:

[2020-08-31T13:49:46.278 +0530] UncaughtException Error: read ECONNRESET
    at TLSWrap.onStreamRead (internal/stream_base_commons.js:111:27)

Have you found a mitigation/solution?

No

My feature is not call blobService.listBlobsSegmentedWithPrefix multiple time recursively to get all the blobs but the server is crashing due to that. Any help would be much much appreciated.

[jiacfan]

Hi, @sunil-jhamnani

Thanks for reaching us.

The issue looks like due to 30 calls to service side in parallel, which cause client side network usage burst, and cause ECONNRESET.

For workaround, please consider to:

1. Limit the parallel requests going to server side, e.g. 5 each times vs 28+ each times now.
2. Might worth try to use global http agent with customized configurations, e.g. http keep alive = false

this.blobService = azureStorage.createBlobService();

//use global http agent, and tune global http agent's parameter
this.blobService.enableGlobalHttpAgent = true;

At same time, for JS SDK, the latest v12 SDK is maintained in this repro(link), welcome to have a try.

Thanks,
Jiachen