HttpComponentsClientHttpRequestFactory OutOfMemory during Post large files

[MasterEmit]

If I try to upload a file via Post, I get an OutOfMemoryError. I just saw, that this problem was solved at the "normal" Spring Framework, but sadly not for Android. I need to use the HttpComponentsClientHttpRequestFactory.

SimpleClientHttpRequestFactory has setBufferRequestBody, which makes it possible to stream the content and not loadling everything into memory.

FileSize ist around 18MB.

Does anybody know a workaround?

    java.lang.OutOfMemoryError
            at java.io.ByteArrayOutputStream.expand(ByteArrayOutputStream.java:91)
            at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:201)
            at org.springframework.util.FileCopyUtils.copy(FileCopyUtils.java:113)
            at org.springframework.http.converter.ResourceHttpMessageConverter.writeInternal(ResourceHttpMessageConverter.java:73)
            at org.springframework.http.converter.ResourceHttpMessageConverter.writeInternal(ResourceHttpMessageConverter.java:39)
            at org.springframework.http.converter.AbstractHttpMessageConverter.write(AbstractHttpMessageConverter.java:173)
            at org.springframework.http.converter.FormHttpMessageConverter.writePart(FormHttpMessageConverter.java:310)
            at org.springframework.http.converter.FormHttpMessageConverter.writeParts(FormHttpMessageConverter.java:276)
            at org.springframework.http.converter.FormHttpMessageConverter.writeMultipart(FormHttpMessageConverter.java:266)
            at org.springframework.http.converter.FormHttpMessageConverter.write(FormHttpMessageConverter.java:209)
            at org.springframework.http.converter.FormHttpMessageConverter.write(FormHttpMessageConverter.java:91)
            at org.springframework.web.client.RestTemplate$HttpEntityRequestCallback.doWithRequest(RestTemplate.java:624)
            at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:474)
            at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:447)
            at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:422)

I use HttpComponentsClientHttpRequestFactory
Function has url as String, file as File and filename and contentType as String params

                MultiValueMap<String, Object> multiValueMap = new LinkedMultiValueMap<String, Object>();
                FileSystemResource fileSystemResource = new FileSystemResource(file.getAbsoluteFile()) {

                    @Override
                    public String getFilename() {
                        String filename = fFilename;

                        if (fContentType.toLowerCase().equals("image/jpeg")) {
                            if (!filename.toLowerCase().endsWith(".jpeg")) {
                                filename += ".jpeg";
                            }
                        }

                        return filename;
                    }
                };


                MediaType mediaType = MediaType.valueOf(contentType);
                HttpHeaders headers = new HttpHeaders();
                headers.setContentType(mediaType);
                HttpEntity httpEntity = new HttpEntity(fileSystemResource, headers);

                multiValueMap.add("file", httpEntity);

                additionalHeaders.put("Accept", "");

                jsonAttachmentCreated = restClient.postAttachment(url, multiValueMap);

[royclarkson]

Thanks for reporting! Do you have a JIRA link for the Spring Framework fix or did you inspect the code?

[royclarkson]

One issue is that HttpComponents on Android is an older version from Spring Framework. Http Client 4.3 is now available for android. We can investigate adding support for it through a new HttpRequestFactory. See also ANDROID-90

[MasterEmit]

I just inspected the code. I used Googlke before to find a solution and quite many were talking, that since version x its solved. And I found out, that HttpComponentsClientHttpRequestFactory has a flagsetBufferRequestBody, but not the Android one.

I also tried to write a new ResourceHttpMessageConverter and overwrite writeInternal. I just made a copy of StreamUtils from Spring and used the copy function. It helped on some devices, but not all.

So at the moment the "ugly" largeHeap flag would be the only solution, which would be easy to implement. But of course, I would be happy, if I could avoid that

[royclarkson]

I'm currently thinking of deprecating the existing (native) 4.0 Http Client in favor of supporting the newer 4.3 HttpClient android port and merging in the latest changes from Spring Framework for the HttpComponentsClientHttpRequestFactory. That should resolve this, if I understand correctly.

[MasterEmit]

Sounds like a plan and I would be happy to test it :-)

[royclarkson]

Yes, that will be very helpful. Unfortunately, we haven't had a release in quite some time, but I'm working on addressing some issues and hopefully will release something soon.

[royclarkson]

HttpComponentsClientHttpRequestFactory has been updated to use the Android port of HttpClient 4.3. It requires the external dependency available in Maven central. The updated request factory is based on Spring Framework 4.1.

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient-android</artifactId>
    <version>4.3.5</version>
</dependency>

Support for the native HttpClient 4.0.x has been deprecated and renamed to HttpComponentsAndroidClientHttpRequestFactory. It will remain available indefinitely since there is no way to remove the Android provided HttpClient.