Sitemap not responding | Bad Request
byposeidon opened this issue · 1 comments
Hello, when trying to crawl a website with many links, sitemap doesn't work.
Response: {"timestamp":"2022-05-27T22:00:38.345+0000","status":400,"error":"Bad Request","message":"URLDecoder: Incomplete trailing escape (%) pattern","path":"/burp/target/sitemap"}%
The endpoint is responding "Bad Request" because it's receiving a wrong encoded string (see https://stackoverflow.com/questions/10590080/java-urldecoder-throws-exception-when-used-with-a-string-containing-a )
You are probably sending a wrong request similar to this
curl -v -X GET "http://127.0.0.1:8090/burp/target/sitemap?urlPrefix=https://www.site.com%" -H "accept: */*"
In addition to that, the right header to use is
-H "accept: */*"
not
-H "accept: /"