500 Internal Server Error when trying to upload a map to the Map Vault
Closed this issue · 6 comments
Details:
https://www.screencast.com/t/wig6l7xEO6
Map could not be uploaded. Server gave the following errors: 500 Internal Server Error
org.springframework.web.client.HttpServerErrorException$InternalServerError: 500 Internal Server Error
at org.springframework.web.client.HttpServerErrorException.create(HttpServerErrorException.java:79)
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:99)
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:79)
at com.faforever.client.api.JsonApiErrorHandler.handleError(JsonApiErrorHandler.java:31)
at org.springframework.security.oauth2.client.http.OAuth2ErrorHandler.handleError(OAuth2ErrorHandler.java:84)
at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63)
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:777)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:735)
at org.springframework.security.oauth2.client.OAuth2RestTemplate.doExecute(OAuth2RestTemplate.java:128)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:669)
at org.springframework.web.client.RestTemplate.postForEntity(RestTemplate.java:444)
at com.faforever.client.api.FafApiAccessorImpl.post(FafApiAccessorImpl.java:544)
at com.faforever.client.api.FafApiAccessorImpl.uploadMap(FafApiAccessorImpl.java:290)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:206)
at com.sun.proxy.$Proxy67.uploadMap(Unknown Source)
at com.faforever.client.map.MapUploadTask.call(MapUploadTask.java:86)
at com.faforever.client.map.MapUploadTask.call(MapUploadTask.java:30)
at javafx.graphics/javafx.concurrent.Task$TaskCallable.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Link to the map in question: https://drive.google.com/open?id=1iulQ-rRvL6vi0Q7PRzdDWEgrCky2korg
I just found out that FAFMapEditor v0.605 also has an upload function, so gave it a try. It also returned a 500 server error, however this time with more details:
{"requestId":"6b7432f7-14c7-4ed3-a0f2-6d8e58ae26d0","errors":[{"status":"500","title":"com.faforever.commons.io.ZipBombException","detail":"Zip bomb detected. Aborting unzip process"}]}
Exception is:
500 (Internal Server Error)
For some reason the server thinks that my map has a zip bomb in it. Which is strange since the zip was created both by FAFMapEditor and DownlordsFAFClient themselves just before upload. What gives? Likely a false positive, right? Could I somehow sidestep this issue and upload?
Also, two more people seem to be having the same issue in the "Maps & Mods FAF" Discord (https://discord.gg/Z5pVWSx) in the #faf-editor channel.
p.s. I've moved this issue over from FAForever/server#483 after seeing Askaholic's comment, I hope this is the better place for it.
You have come to the right place.
The zip bomb detection is supposed to trigger if the unzipping exceeds factor 100 on compressed to uncompressed files. E.g. if a 1MB map zip file would produce 100MB unzipped. This is not the case here, it is only factor ~10. I need to investigate why this happens.
Yes, my guess was also that the compression ratio was checked, and was thinking that even 1:10 is too much in my file for the server. And yes, 1:100 sounds more sensible. Looking forward to what your investigation uncovers. If I can help you somehow let me know.
I just now tried to upload a different map (here it is, it's 40km x 40km this time instead of 81km x 81km) with a padding file (a large picture of Hubble deep field). I was trying to see if lower compression ratios play a role. Here are the results:
| map size | padding size | unzipped size | zipped size | compression ratio | |
|---|---|---|---|---|---|
| Without padding | 55 MB | 0 MB | 55 MB | 3.8 MB | 14.5 : 1 |
| With padding | 55 MB | 61 MB | 116 MB | 65.0 MB | 1.9 : 1 |
The verdict is the same; even with about 2:1 compression ratio, the same zip bomb error is triggered.
Could it be that absolute size matters here? E.g. anything that is over 50MB unzipped is rejected? I am wondering this because some days ago I was able to upload 10km x 5km and 20km x 10km maps.
Also, while this is sorted out, could these two maps be added to the vault in some other way? I'd like to play them with friends and others. E.g. could you copy them in by hand? Or please tell me if there is some uploading interface (e.g. a regular website) that I am unaware of so I can try to use that as well.
Edit: A curl command is also fine for me (e.g. curl -d "param1=value1¶m2=value2" -X POST http://localhost:3000/data) though looking at this https://api.faforever.com/swagger-ui.html it seems that OAuth and cookies might make that a bit more complicated, i.e. minimum 2 commands with a 'cookie jar'. This way I could create a zip with 1:1 compression ratio even without any padding and see if that overcomes the issue.
Ok this is fixed now in the base library :) But I need to fix #325 as well, otherwise we still break stuff.
Great!
I just gave it a try again, and the upload still fails with the same zip bomb error though. Perhaps the fix is not deployed yet?
At any rate, my map names don't contain any special characters, so I wouldn't be hindered by that other issue afaiu.
Yes it is not deployed yet on purpose until #325 is also fixed