client.Jobs.ImportUsersAsync returns exception of `Cannot access a closed Stream.`
CasperCBroeren opened this issue · 7 comments
Describe the problem
When uploading a record set of 2451 items of 499834 bytes to client.Jobs.ImportUsersAsync there is an exception in reading the response. Cannot access a closed Stream.
It might be valueable to know that we insert more than 1 job in a short time. We have around 7000 records and we need them in Auth0. Our process splits them in batches to not exceed the upload max.
I've also tried lowering the bytes send greatly but it also fails. Also we delay 2 seconds after each call
What was the expected behavior?
Return an processable error and not an exception
Stacktrace
The only thing I can share is this;
System.Net.Http.HttpRequestException: Error while copying content to a stream.
---> System.ObjectDisposedException: Cannot access a closed Stream.
at System.IO.MemoryStream.CopyToAsync(Stream destination, Int32 bufferSize, CancellationToken cancellationToken)
at System.IO.Stream.CopyToAsync(Stream destination, CancellationToken cancellationToken)
at System.Net.Http.StreamToStreamCopy.CopyAsync(Stream source, Stream destination, Int32 bufferSize, Boolean disposeSource, CancellationToken cancellationToken)
--- End of stack trace from previous location ---
at System.Net.Http.HttpContent.<CopyToAsync>g__WaitAsync|56_0(ValueTask copyTask)
--- End of inner exception stack trace ---
at System.Net.Http.HttpContent.<CopyToAsync>g__WaitAsync|56_0(ValueTask copyTask)
at System.Net.Http.MultipartContent.SerializeToStreamAsyncCore(Stream stream, TransportContext context, CancellationToken cancellationToken)
at System.Net.Http.HttpContent.<CopyToAsync>g__WaitAsync|56_0(ValueTask copyTask)
at System.Net.Http.HttpConnection.SendRequestContentAsync(HttpRequestMessage request, HttpContentWriteStream stream, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
at Auth0.ManagementApi.HttpClientManagementConnection.SendRequest[T](HttpRequestMessage request, JsonConverter[] converters, CancellationToken cancellationToken)
at Auth0.ManagementApi.HttpClientManagementConnection.SendAsyncInternal[T](HttpMethod method, Uri uri, Object body, IDictionary`2 headers, IList`1 files, JsonConverter[] converters, CancellationToken cancellationToken)
at Auth0.ManagementApi.HttpClientManagementConnection.<>c__DisplayClass14_0`1.<<SendAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at Auth0.ManagementApi.HttpClientManagementConnection.Retry[TResult](Func`1 retryable)
at Auth0.ManagementApi.HttpClientManagementConnection.SendAsync[T](HttpMethod method, Uri uri, Object body, IDictionary`2 headers, IList`1 files, JsonConverter[] converters, CancellationToken cancellationToken)
Update from my side; I've quit using this library and made my own request with RestSharp. I also coulnd't get it to work in regular System.Net.Http
Thanks for reporting this. Ths has been reported before but haven't been able to reproduce this, see #443. Would any of the information mentioned there help you in any way?
If you would be able to provide the code you use to call our SDK, that could be helpful for us to try and reproduce this and see what's causing this.
Actually, this might be related to the fact that you can't run more than 2 import user jobs at the same time, else the Auth0 API returns HTTP 429. It might be the case that this is incorrectly handled and causing issues.
I will try and reproduce this and get back to you! In the meantime, if u have any code to share, that would still be very helpful!
It's indeed true that when I try to process my 7500 records to be inserted, I split them in batches of 2000 (max upload of 500kb), So when I come to batch 3, I get this error.
Also with RestSharp I can better post multipart forms and get the real answer back (something like too many requests).
@frederikprijck I am also facing the exact same issue. I am trying to migrate approx. 3600 users. I have generated multiple json files for 480Kb approx each.
When I put the debugger and wait for 2-3 secs it processes all files, Otherwise it breaks in between and throw the below exception.
InnerException = {"Cannot access a closed Stream."}
Message = "Error while copying content to a stream."
at System.Net.Http.HttpContent.<<CopyToAsync>g__WaitAsync|56_0>d.MoveNext() at System.Net.Http.MultipartContent.<SerializeToStreamAsyncCore>d__24.MoveNext() at System.Net.Http.HttpContent.<<CopyToAsync>g__WaitAsync|56_0>d.MoveNext() at System.Net.Http.HttpConnection.<SendRequestContentAsync>d__61.MoveNext() at System.Net.Http.HttpConnection.<SendAsync>d__57.MoveNext() at System.Net.Http.HttpConnection.<SendAsync>d__57.MoveNext() at System.Net.Http.HttpConnectionPool.<SendWithVersionDetectionAndRetryAsync>d__89.MoveNext() at System.Threading.Tasks.ValueTask1.get_Result()
at System.Net.Http.RedirectHandler.d__4.MoveNext()
at System.Net.Http.HttpClient.<g__Core|83_0>d.MoveNext()
at Auth0.ManagementApi.HttpClientManagementConnection.d__191.MoveNext() at Auth0.ManagementApi.HttpClientManagementConnection.<SendAsyncInternal>d__161.MoveNext()
at Auth0.ManagementApi.HttpClientManagementConnection.<>c__DisplayClass14_01.<<SendAsync>b__0>d.MoveNext() at Auth0.ManagementApi.HttpClientManagementConnection.<Retry>d__261.MoveNext()
at Auth0.ManagementApi.HttpClientManagementConnection.d__141.MoveNext() at Program.<>c__DisplayClass0_0.<<<Main>$>g__ExecuteImportJobAsync|4>d.MoveNext() in C:\POC\NativeEmailConfirmation\FileSize\Program.cs:line 122 at Program.<>c__DisplayClass0_0.<<<Main>$>g__ImportUsersFromJsonFileAsync|3>d.MoveNext() in C:\POC\NativeEmailConfirmation\FileSize\Program.cs:line 102
It's indeed true that when I try to process my 7500 records to be inserted, I split them in batches of 2000 (max upload of 500kb), So when I come to batch 3, I get this error. Also with RestSharp I can better post multipart forms and get the real answer back (something like too many requests).
Can you please share you code snippet so that we can try your way as well.
Actually, this might be related to the fact that you can't run more than 2 import user jobs at the same time, else the Auth0 API returns HTTP 429. It might be the case that this is incorrectly handled and causing issues.
I will try and reproduce this and get back to you! In the meantime, if u have any code to share, that would still be very helpful!
No, running this synchronously 1 by 1 also results in same error.