contentful/contentful-export

Export hitting rate limit with very few entries exported

hartmagj opened this issue · 0 comments

My team and I are trying to migrate the content from one contentful environment to another. Running our migration script on a subset of the content works just fine, but when we run our script on a larger set of contentful entries, we receive rate limit errors that look like this (though not necessarily in this order):
Oh no! Some errors occurred! Error [ContentfulMultiError]: Errors occured
11:20:68 - Rate limit error occurred. Waiting for 1624 ms before retrying...
11:20:45 - Connection error occurred. Waiting for 1978 ms before retrying...

After these errors only one or two entries actually get migrated, the rest of the changes seem to be ignored.

For reference this is a relevant piece of the migration script:

const exportOptions = expOpts;
exportOptions.queryEntries = ['sys.id=0000'];
exportOptions.queryAssets = [sys.id=${id}];
contentfulExport(exportOptions)
.then((result) => {
contentfulImport({
content: result,
...impOpts,
});
})
.catch((err) => {
console.log('Oh no! Some errors occurred!', err);
});

This is being run on node 12.19.1 and has been tested on node 16 as well with no improvement