Spider doesn't walk all the URL's in the context
Closed this issue · 1 comments
My problem I thing stems from having a login URL that is different than the URL I need to scan.
In the GUI application of ZAP I can execute a spider or a scan against all the URLs in the context.
I start ZAP in daemon
/zap.sh -config api.key=12345 -port 8090 -daemon &
import my context
zap-cli -p 8090 --api-key 12345 context import /path/to/my.context
Then I attempt to spider
Say I had the following two URLs in my context
https://my.login.url/login
https://my.scan.url/whatsup/*
The GUI app would login for me (after I setup the user) at the login URL and then walk all the pages in "https://my.scan.url/whatsup/*" using the context as a guide.
With zap-cli I can do the following successfully, but it only goes after the login URL and does not proceed onto the rest of the context:
zap-cli -p 8090 --api-key 12345 spider -c my_awesome_context -u my.user@someserver.com https://my.login.url/login
when I try to do:
zap-cli -p 8090 --api-key 12345 spider -c my_awesome_context -u my.user@someserver.com https://my.scan.url/whatsup/
or
zap-cli -p 8090 --api-key 12345 spider -c my_awesome_context -u my.user@someserver.com https://my.scan.url/whatsup/*
I get
[INFO] Running spider...
1062599 [ZAP-SpiderInitThread-1] INFO org.zaproxy.zap.extension.spider.SpiderThread - Starting spidering scan on Context: my_awesome_context at Wed May 16 16:37:23 EDT 2018
1062600 [ZAP-SpiderInitThread-1] INFO org.zaproxy.zap.spider.Spider - Spider initializing...
1062611 [ZAP-SpiderInitThread-1] INFO org.zaproxy.zap.spider.Spider - Starting spider...
1062611 [ZAP-SpiderInitThread-1] INFO org.zaproxy.zap.spider.Spider - Scan will be performed from the point of view of User: my.user@someserver.com
1064919 [ZAP-SpiderThreadPool-1-thread-2] ERROR io.swagger.parser.SwaggerCompatConverter - failed to read resource listing
com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'var': was expecting ('true', 'false' or 'null')
at [Source: /tmp/openapi12718607691477304505.defn; line: 1, column: 5]
at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1702)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:558)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._reportInvalidToken(UTF8StreamJsonParser.java:3528)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._handleUnexpectedValue(UTF8StreamJsonParser.java:2686)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._nextTokenNotInObject(UTF8StreamJsonParser.java:878)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:772)
at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:3850)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3799)
at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:2447)
at io.swagger.parser.SwaggerCompatConverter.readResourceListing(SwaggerCompatConverter.java:189)
at io.swagger.parser.SwaggerCompatConverter.read(SwaggerCompatConverter.java:116)
at io.swagger.parser.SwaggerCompatConverter.read(SwaggerCompatConverter.java:107)
at org.zaproxy.zap.extension.openapi.converter.swagger.SwaggerConverter.readOpenAPISpec(SwaggerConverter.java:99)
at org.zaproxy.zap.extension.openapi.converter.swagger.SwaggerConverter.getRequestModels(SwaggerConverter.java:74)
at org.zaproxy.zap.extension.openapi.OpenApiSpider.parseResource(OpenApiSpider.java:55)
at org.zaproxy.zap.spider.SpiderTask.processResource(SpiderTask.java:393)
at org.zaproxy.zap.spider.SpiderTask.runImpl(SpiderTask.java:259)
at org.zaproxy.zap.spider.SpiderTask.run(SpiderTask.java:187)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:844)
1065169 [ZAP-SpiderThreadPool-1-thread-1] INFO org.zaproxy.zap.spider.Spider - Spidering process is complete. Shutting down...
1065170 [ZAP-SpiderShutdownThread-1] INFO org.zaproxy.zap.extension.spider.SpiderThread - Spider scanning complete: true
It is not readily apparent how I can initiate a scan or a spider that uses all URLs in the context provided and maintain a login as I do with the GUI application.
How can I initiate a scan or spider against the whole context?
closed - it was working the whole time I just didn't see it.
Sorry for the chaff in the issues