help request: ext-plugin(java) error with multipart/form-data?
channer99 opened this issue · 7 comments
channer99 commented
Description
apisix-java-plugin-runner version is 0.4.0
request Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryedcKdnGDWD6rJDcu
payload is
------WebKitFormBoundaryedcKdnGDWD6rJDcu
Content-Disposition: form-data; name="file"; filename="test.xlsx"
Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
------WebKitFormBoundaryedcKdnGDWD6rJDcu
Content-Disposition: form-data; name="age"
20
------WebKitFormBoundaryedcKdnGDWD6rJDcu--
but ext 503 error occured..
error.log is below.. is that library issue?
pls check this log
2023/12/22 09:46:12 [warn] 52461#52461: *63 [lua] init.lua:956: 2023-12-22 09:46:12.590 ERROR 52473 --- [tLoopGroup-34-4] o.a.a.p.r.h.RpcCallHandler : handle request error:
java.lang.IndexOutOfBoundsException: null
at java.nio.Buffer.checkIndex(Buffer.java:687) ~[?:?]
at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:269) ~[?:?]
at io.github.api7.A6.ExtraInfo.Resp.result(Resp.java:18) ~[A6-0.6.0-RELEASE.jar!/:?]
at org.apache.apisix.plugin.runner.ExtraInfoResponse.getResult(ExtraInfoResponse.java:41) ~[apisix-runner-plugin-sdk-0.4.0.jar!/:0.4.0]
at org.apache.apisix.plugin.runner.handler.RpcCallHandler.handleExtraInfo(RpcCallHandler.java:218) ~[apisix-runner-core-0.4.0.jar!/:0.4.0]
at org.apache.apisix.plugin.runner.handler.RpcCallHandler.channelRead0(RpcCallHandler.java:92) ~[apisix-runner-core-0.4.0.jar!/:0.4.0]
at org.apache.apisix.plugin.runner.handler.RpcCallHandler.channelRead0(RpcCallHandler.java:54) ~[apisix-runner-core-0.4.0.jar!/:0.4.0]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
at org.apache.apisix.plugin.runner.handler.PrepareConfHandler.channelRead0(PrepareConfHandler.java:57) ~[apisix-runner-core-0.4.0.jar!/:0.4.0]
at org.apache.apisix.plugin.runner.handler.PrepareConfHandler.channelRead0(PrepareConfHandler.java:45) ~[apisix-runner-core-0.4.0.jar!/:0.4.0]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
2023/12/22 09:46:12 [warn] 52461#52461: *63 [lua] init.lua:956: y.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:280) ~[netty-handler-4.1.78.Final.jar!/:4.1.78.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800) ~[netty-transport-classes-epoll-4.1.78.Final.jar!/:4.1.78.Final]
at io.netty.channel.epoll.EpollDomainSocketChannel$EpollDomainUnsafe.epollInReady(EpollDomainSocketChannel.java:138) ~[netty-transport-classes-epoll-4.1.78.Final.jar!/:4.1.78.Final]
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:487) ~[netty-transport-classes-epoll-4.1.78.Final.jar!/:4.1.78.Final]
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:385) ~[netty-transport-classes-epoll-4.1.78.Final.jar!/:4.1.78.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[netty-common-4.1.78.Final.jar!/:4.1.78.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.78.Final.jar!/:4.1.78.Final]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.78.Final.jar!/:4.1.78.Final]
at java.lang.Thread.run(Thread.java:829) ~[?:?]
, context: ngx.timer
2023/12/22 09:46:12 [warn] 52461#52461: *63 [lua] init.lua:956: 2023-12-22 09:46:12.597 WARN 52473 --- [tLoopGroup-34-4] o.a.a.p.r.h.PayloadDecoder : receive unsupported type: -4
, context: ngx.timer
2023/12/22 09:46:12 [warn] 52461#52461: *63 [lua] init.lua:956: 2023-12-22 09:46:12.605 WARN 52473 --- [tLoopGroup-34-4] o.a.a.p.r.h.PayloadDecoder : receive unsupported type: -72
, context: ngx.timer
2023/12/22 09:46:12 [error] 52455#52455: *6011569 [lua] init.lua:884: phase_func(): failed to receive RPC_HTTP_REQ_CALL: service unavailable, client:
2023/12/22 09:46:12 [warn] 52455#52455: *6011569 [lua] plugin.lua:1102: run_plugin(): ext-plugin-pre-req exits with http status code 503,
Environment
- APISIX version (run
apisix version): 3.4.1 - Operating system (run
uname -a): - OpenResty / Nginx version (run
openresty -Vornginx -V): - etcd version, if relevant (run
curl http://127.0.0.1:9090/v1/server_info): - APISIX Dashboard version, if relevant:
- Plugin runner version, for issues related to plugin runners:
- LuaRocks version, for installation issues (run
luarocks --version):
sheharyaar commented
Hi, @channer99 , can you pleasee send the following in correct format :
- Routes, services and other APISIX configs used
- The
completesteps to reproduce the issue.
Please format the configs and logs using backticks(`)
channer99 commented
route config is
{
"uri": "/test*",
"name": "test",
"methods": [
"GET",
"POST",
"PUT",
"DELETE",
"PATCH",
"OPTIONS"
],
"plugins": {
"ext-plugin-pre-req": {
"_meta": {
"disable": false
},
"allow_degradation": false,
"conf": [
{
"name": "Test_filter",
"value": "{\"rejected_body2\":\"failed\",\"rejected_code2\":\"403\"}"
}
]
},
},
"upstream_id": "473424305712006806",
"status": 1
}`
apisix config.yaml
apisix:
node_listen:
- port: 80
ssl:
enable: false
deployment:
role: traditional
role_traditional:
config_provider: etcd
admin:
allow_admin:
- all
admin_listen:
port: 9180
admin_key:
- name: admin
key: edd1c9f034335f136f87ad84b625c8f1 # using fixed API token has security risk, please update it when you deploy to production environment
role: admin
etcd:
host:
- "http://xxx.xx.xxx.xxx:2379"
nginx_config:
enable_cpu_affinity: |
true;
http_configuration_snippet: |
client_body_buffer_size 10m;
proxy_max_temp_file_size 0;
http_server_configuration_snippet: |
merge_slashes off;
proxy_ignore_client_abort on;
ext-plugin:
cmd: ['java', '-jar', '-Xmx1g', '-Xms1g', '/usr/local/apisix/test_filter.jar']
channer99 commented
attached test.xlsx is 30MB excel.
my request code(java spring) is
String filePath = "/Users/aaaa/Desktop/test.xlsx";
Path path = Paths.get(filePath);
byte[] fileData = null;
try {
fileData = Files.readAllBytes(path);
} catch (IOException e) {
e.printStackTrace();
}
String url2 = "/test";
HttpHeaders headers2 = new HttpHeaders();
headers2.setContentType(MediaType.MULTIPART_FORM_DATA);
headers2.set("boundary", "----WebKitFormBoundarykMBT8G8fZJ6YOU6X");
MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
body.add("file", new HttpEntity<>(fileData, getMultipartFileHeaders(path.getFileName().toString())));
HttpEntity<MultiValueMap<String, Object>> entity2 = new HttpEntity<>(body, headers2);
RestTemplate restTemplate2 = new RestTemplate();
restTemplate2.setRequestFactory(new HttpComponentsClientHttpRequestFactory());
ResponseEntity<String> response2 = restTemplate2.exchange("http://{gw}" + url2, HttpMethod.POST,
entity2, String.class);
If you refer to the above, you will be able to reproduce it easily.
503 occurs before ext filter logic is performed.
pls check @sheharyaar
sheharyaar commented
Ok, I will check and update you soon. I cannot find the attached xls, I will create a dummy xls for this.
channer99 commented
Ok, I will check and update you soon. I cannot find the attached xls, I will create a dummy xls for this.
@sheharyaar
It seems to be a file size issue. When I reduced the size of the same xlsx file to 10mb, the call was successful. Is it not possible to call large files larger than 10MB in apisix?
sheharyaar commented
Ca you check the timeout for this error?? Maybe try setting larger timeout : https://apisix.apache.org/docs/apisix/admin-api/
channer99 commented
increased all timeout(connect, read, send), but it doesn't work