lets-blade/blade

Exception passing string containing =

sd205 opened this issue · 18 comments

sd205 commented

I've created a simple POST service using Blade. In my request class I have a String field. When I send a request containing an equals character, "=", I get the following exception:

io.netty.handler.codec.http.multipart.HttpPostRequestDecoder$ErrorDataDecoderException: java.io.IOException: The filename, directory name, or volume label syntax is incorrect
at io.netty.handler.codec.http.multipart.HttpPostStandardRequestDecoder.parseBodyAttributes(HttpPostStandardRequestDecoder.java:615)
at io.netty.handler.codec.http.multipart.HttpPostStandardRequestDecoder.parseBody(HttpPostStandardRequestDecoder.java:366)
at io.netty.handler.codec.http.multipart.HttpPostStandardRequestDecoder.offer(HttpPostStandardRequestDecoder.java:295)
at io.netty.handler.codec.http.multipart.HttpPostStandardRequestDecoder.offer(HttpPostStandardRequestDecoder.java:47)
at io.netty.handler.codec.http.multipart.HttpPostRequestDecoder.offer(HttpPostRequestDecoder.java:223)
at com.blade.mvc.http.HttpRequest.init(HttpRequest.java:376)
at com.blade.server.netty.HttpServerHandler.buildWebContext(HttpServerHandler.java:96)
at com.blade.server.netty.HttpServerHandler.lambda$channelRead0$0(HttpServerHandler.java:85)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:497)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: The filename, directory name, or volume label syntax is incorrect
at java.io.WinNTFileSystem.createFileExclusively(Native Method)
at java.io.File.createTempFile(File.java:2024)
at java.io.File.createTempFile(File.java:2070)
at io.netty.handler.codec.http.multipart.AbstractDiskHttpData.tempFile(AbstractDiskHttpData.java:90)
at io.netty.handler.codec.http.multipart.AbstractDiskHttpData.addContent(AbstractDiskHttpData.java:162)
at io.netty.handler.codec.http.multipart.DiskAttribute.addContent(DiskAttribute.java:98)
at io.netty.handler.codec.http.multipart.HttpPostStandardRequestDecoder.setFinalBuffer(HttpPostStandardRequestDecoder.java:624)
at io.netty.handler.codec.http.multipart.HttpPostStandardRequestDecoder.parseBodyAttributes(HttpPostStandardRequestDecoder.java:595)

Here is the full request payload, only the formatString is material. Any string containing an equals sign results in the exception, other strings are ok:

{"formatString":"=","delimiters":null,"params":["","","","","","","","","",""]}

It's not reaching my code, as it is failing in Netty while parsing the message body.

wanglunhui2012 commented

can you show the core code that can reappear this error.

sd205 commented

What do you mean by core code? As I say, the request isn't reaching my code, so it's not material. I've raised an issue on netty, as noted above. Apart from that all I have is the stack trace, which I have shared.

wanglunhui2012 commented

In my code, i can't reappear this error . blade-mvc version is 2.0.15.RELEASE.

public static void main(String[] args) {
    Blade.of().listen(8080).post("/body", ctx -> {
        System.out.println("body string is:" + ctx.bodyToString());
    }).start();
}
curl -X POST http://localhost:8080/body -d '{"formatString=":"=","delimiters=":null,"params":["","","","","","","","","",""]}'

the console print:

body string is:{"formatString=":"=","delimiters=":null,"params":["","","","","","","","","",""]}
sd205 commented

test-case.zip
See the attached test case:
mvn package
java -jar target\blade-test-case.jar
GET http://localhost:9000/static/index.html
POST http://localhost:9000/test {"str":"="}

wanglunhui2012 commented

how do you send the post request? i try the code, no problem, no error.

sd205 commented

Interesting! I have tested from a react app using Axios and from Postman. In both cases it works fine until I add an = to the string, i.e. {"str":"a"} works but {"str":"a="} fails. Both add various headers, so I'm trying now with CURL (new build attached):

curl -i http://localhost:8999/static/index.html

curl -X POST http://localhost:8999/test -d {"str":"abc"} // this works

curl -X POST http://localhost:8999/test -d {"str":"ab="} // this fails with

2022/01/13 10:30:30 ERROR [ worker@thread-5 ] c.b.m.h.DefaultExceptionHandler : com.blade.kit.json.ParseException: {str:ab= (7):Expected a ',' or '}'
at com.blade.kit.json.SampleJsonSerializer.nextValue(SampleJsonSerializer.java:118)
at com.blade.kit.json.SampleJsonSerializer.deserialize(SampleJsonSerializer.java:84)
at com.blade.kit.json.DefaultJsonSupport.formJson(DefaultJsonSupport.java:22)
at com.blade.kit.JsonKit.formJson(JsonKit.java:52)
at com.blade.mvc.handler.RouteActionArguments.getBodyParam(RouteActionArguments.java:152)
at com.blade.mvc.handler.RouteActionArguments.getAnnotationParam(RouteActionArguments.java:120)
at com.blade.mvc.handler.RouteActionArguments.getRouteActionParameters(RouteActionArguments.java:46)
at com.blade.mvc.RouteContext.injectParameters(RouteContext.java:585)
at com.blade.server.netty.RouteMethodHandler.handle(RouteMethodHandler.java:79)
at com.blade.server.netty.HttpServerHandler.executeLogic(HttpServerHandler.java:159)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:497)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at java.lang.Thread.run(Thread.java:748)

curl -X POST http://localhost:8999/test -d '{"str":"ab="}' // fails with:

2022/01/13 10:37:30 ERROR [ worker@thread-10 ] c.b.m.h.DefaultExceptionHandler : java.lang.ClassCastException: java.lang.String cannot be cast to test.TestRouter$StringBean
at test.TestRouterMethodAccess.invoke(Unknown Source)
at com.blade.reflectasm.MethodAccess.invoke(MethodAccess.java:43)
at com.blade.server.netty.RouteMethodHandler.routeHandle(RouteMethodHandler.java:256)
at com.blade.server.netty.RouteMethodHandler.handle(RouteMethodHandler.java:87)
at com.blade.server.netty.HttpServerHandler.executeLogic(HttpServerHandler.java:159)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:497)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at java.lang.Thread.run(Thread.java:748)

I tried with --data-urlencode, didn't help. I tried escaping the equals with a backslash, that just hung.

test-case.zip

fredericBregier commented

@sd205 Is it possible that you missused POST request?

POST request shall be either:

  1. URL encoded form (as in GET format but in body)

    Content-Type: application/x-www-form-urlencoded
Content-length=nn

field1={"str":"ab="}

so through but with data-encoding only for left part of 'field1={"str":"ab="}' (not 100% sure curl command)

   curl -X POST http://localhost:8999/test -F field1='{"str":"ab="}'

  1. Through multipart

    Content-Type: multipart/form-data;boundary="boundary"

--boundary
Content-Disposition: form-data; name="field1"

{"str":"ab="}
--boundary--

so with curl (not 100% sure curl command)

  curl -X PUT URL \
     --header 'Content-Type: multipart/form-data; boundary=---------BOUNDARY' \
     --data-binary @file

where file is the file containing {"str":"ab="}

  1. native (text or binary)

    Content-Type: text/plain
    Content-length=nn

    {"str":"ab="}

as you did but then there is no decoding at all (no data, either url form or multipart form), just binary.
Therefore, when it tries to decode the content, it first search for the first 'equal' sign, since not multipart, and find it within "value" since there is no field declared first.

sd205 commented

Thanks both for your help!

I've also tried with Content-Type: application/json...

This works fine:
curl -X POST http://localhost:8999/test -H "Content-Type: application/json" -d {"str":"abc"}

But this fails with the ParseException:
curl -X POST http://localhost:8999/test -H "Content-Type: application/json" -d {"str":"ab="}

If I add Content-Length it just hangs, e.g.:
curl -X POST http://localhost:8999/test -H "Content-Type: application/json" -H "Content-Length: 13" -d {"str":"abc"}

If I add Content-Length and get the JSON from a file, it works if the file contains {"str":"abc"}:
curl -X POST http://localhost:8999/test -H "Content-Type: application/json" -H "Content-Length: 13" -d @test.json

But it hangs when the file contains {"str":"ab="}

Any other suggestions of things to try?

fredericBregier commented

You still send a "binary" content (whatever the content-type to application/json).

Unde the wood: the decoder tries to find out "varname=value" while you provide only "value".
Therefore, when "value" contains '=', it believes this is the delimiter of the varname, but then your varname is {"str":"ab which is incorrect from HTTP point of vue (only letter and number, with at least one letter at first pos, probably including some others like '_' or '-' but in ASCII).
As the implementation behind is using a default Factory which uses disk storage, the varname is used as part of the filename. But some characters are not allowed for filename, so the error.

The main point is: you don't send a form, but a binary content. The decoder should not be used in this case (as if you send a PNG file without any form context).
If you want to send a form, you have to change your input, either using the Url encoded form or the multipart form (point 1 or 2 in my previous answear).

sd205 commented

Thanks for explanation - though I'm still not clear where the problem is, whether someone is going to fix something or whether I can send the request (from react/axios not just curl) in a different way to get the required behaviour.

Again, I am sending a valid JSON object as the request body and specifying application/json. There's no reason for the server-side code to start parsing this like a form and tripping over the = signs.

Until now I found Blade a very neat solution to my needs, but this is quite problematic for me as sending an = in the JSON body is a fairly common scenario in my application.

fredericBregier commented

@sd205 I don't know much Blade, but I think you missused it.
According to the stack traces, Blade is expecting a Form request (either var=value&var2=value2 with url encoding for value, either multipart form with delimiter-definition of var-content value-delimiter-definition of var2-content value2-end delimiter).
You are sending a simple Json object, but this is not a Form (no variable defined).

Meybe a missused of Blade ? @wanglunhui2012

sd205 commented

It appears to be a documented capability, see: https://github.com/lets-blade/blade#body-parameters "Body Parameter To Model"

However, I note that their examples wrap the JSON in single quotes, which gives me a ClassCastException, as noted above:
curl -X POST http://localhost:8999/test -H "Content-Type: application/json" -d '{"str":"abc"}'

fredericBregier commented

In case of simple body, not a form, so not when Content-Type: application/x-www-form-urlencoded or Content-Type: multipart/form-data;boundary="xxx" I guess that Blade should not used the default HttpPostRequestDecoder but a simple Body to String content like HttpObjectAggregator to get the body content and then uses it as needed.

My 2 cents

sd205 commented

@wanglunhui2012 anything to suggest?

sd205 commented

@wanglunhui2012 it's gone quiet on this issue, what are the next steps please?

sd205 commented

Anyone?

fredericBregier commented

@wanglunhui2012 I could suggest the following:

In your code, check when the request arrived (first HttpRequest) if the request is one of Content-Type: application/x-www-form-urlencoded or Content-Type: multipart/form-data;boundary="xxx".

  • If yes : use standard HttpPostRequestDecoder which handles one or the other
  • If no : use an HttpObjectAggregator to return BODY object as is, since it is not a request

In the example given by @sd205 , this is not a request but a simple JSON, so a simple body.

  • 👍1
  • ❤️1