ericblade/mws-advanced

extremely large reports fail somehow or other

ericblade opened this issue · 2 comments

ericblade commented

this is all i get along with a partial dump of a massive report that a user was attempting to download.

   at MwsAdvanced.doRequest (Z:\www\random\pos\server\node_modules\mws-advanced\lib\index.js:140:25)
[0]     at processTicksAndRejections (internal/process/task_queues.js:86:5)
ericblade commented

Current investigation seems to indicate that it was not inside this library it was blowing up, but rather code that uses this library was blowing up because Amazon changed a bunch of field names in txt files when they introduced csv reports for ones that were previously only text.

But I'm going to keep this open for a day or two to make sure

ericblade commented

... so, it turns out what is actually failing is getReportListAll. Either it does not throttle properly, or Amazon is not following the throttle rules correctly.

* processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * processRequest text/xml
[0] * request error { ServerError: Service Unavailable
[0]     at Request.postRequest [as _callback] (Z:\www\random\pos\server\node_modules\@ericblade\mws-simple\lib\makeRequest.js:24:23)
[0]     at Request.self.callback (Z:\www\random\pos\server\node_modules\@ericblade\mws-simple\node_modules\request\request.js:185:22)
[0]     at Request.emit (events.js:193:13)
[0]     at Request.<anonymous> (Z:\www\random\pos\server\node_modules\@ericblade\mws-simple\node_modules\request\request.js:1161:10)
[0]     at Request.emit (events.js:193:13)
[0]     at IncomingMessage.<anonymous> (Z:\www\random\pos\server\node_modules\@ericblade\mws-simple\node_modules\request\request.js:1083:12)
[0]     at Object.onceWrapper (events.js:281:20)
[0]     at IncomingMessage.emit (events.js:198:15)
[0]     at endReadableNT (_stream_readable.js:1139:12)
[0]     at processTicksAndRejections (internal/process/task_queues.js:81:17)
[0]   code: 503,
[0]   body:
[0]    '<?xml version="1.0"?>\n<ErrorResponse xmlns="http://mws.amazonaws.com/doc/2009-01-01/">\n  <Error>\n    <Type></Type>\n    <Code>RequestThrottled</Code>\n    <Message>Request is throttled</Message>\n  </Error>\n  <RequestID>c900e45f-872c-4ef1-8c22-561aa13b4bf1</RequestID>\n</ErrorResponse>\n' }

... after 40-41 results picked up by getReportListAll, it throws the Request Throttled error 503. mws-advanced should not be throwing in response to that, it should be throttling.