sitemap not getting populated with urls through config file

[hardikrana437]

Describe the bug
sitemap not getting populated with urls through config file

To Reproduce

Steps to reproduce the behavior:

1. Create a config.json file

json is as follows:
{"config": {
"targetSitemap": "http://localhost:5432",
"scope": [
"http://localhost:5432/auth",
"http://localhost:5432/users",
"http://localhost:5432/users/1",
"http://localhost:5432/users?search=asd",
"http://localhost:5432/bar/foo"
]
} }

2. started the burp with burp-rest-api.bat --headless.mode=false --config-file=config.json

Burp got started but sitemap was not populated

I have also tried with below config.xml file
<?xml version="1.0" encoding="UTF-8"?> <config> <reportType>HTML</reportType> <targetSitemap><![CDATA[http://localhost:5432]]></targetSitemap> <scope> <url><![CDATA[http://localhost:5432/auth]]></url> <url><![CDATA[http://localhost:5432/users]]></url> <url><![CDATA[http://localhost:5432/users/1]]></url> <url><![CDATA[http://localhost:5432/users?search=asd]]></url> <url><![CDATA[http://localhost:5432/bar/foo]]></url> </scope> </config>

It came with json file not supported for config files type error.

@ikkisoft @adracea any inputs?

[adracea]

I don't think it's possible to do so via config file(unless config files have changed significantly and I'm not aware as I'm not up to date with latest burp versions) , rather a combination of session + config .
Also scope should be defined differently , the way you have your scope defined http://localhost:5432/bar/bar would be considered out of scope as well as http://localhost:5432/ would be out of scope . I recommend setting the scope via config file to the root of the application , in your case that would be http://localhost:5432/ and then manually browse to populate a session file , export it , and re-use it .

This is more general burp usage rather than something specific to this rest-api project.

[adracea]

Example options file :

{
    "target":{
        "scope":{
            "advanced_mode":false,
            "exclude":[],
            "include":[
                {
                    "enabled":true,
                    "prefix":"https://accounts.google.com/"
                }
            ]
        }
    }
}

[hardikrana437]

@adracea

In my below example, I guess the values in the scope are the urls of the target host which we have to audit, not the target scope.

{"config": {
"targetSitemap": "http://localhost:5432",
"scope": [
"http://localhost:5432/auth",
"http://localhost:5432/users",
"http://localhost:5432/users/1",
"http://localhost:5432/users?search=asd",
"http://localhost:5432/bar/foo"
]
} }

[adracea]

That still isn't a valid Burp Suite configuration file. You can use the one below :

{
    "target":{
        "scope":{
            "advanced_mode":false,
            "exclude":[],
            "include":[
                {
                    "enabled":true,
                    "prefix":"http://localhost:5432/auth"
                },
                {
                    "enabled":true,
                    "prefix":"http://localhost:5432/users"
                },
                {
                    "enabled":true,
                    "prefix":"http://localhost:5432/bar/foo"
                }
            ]
        }
    }
}

In order to build the sitemap , you'd have to browse or spider each of these URLs , save the session file ( session.burp ) , and then use --project-file=session.burp when starting up .

[hardikrana437]

Thanks a lot @adracea , will try this approach. I think instead of saving to file I can try:

1. run with your config data
   2.then spider the urls
   3.include the url in scope and then run scan.

Sounds fine?

[hardikrana437]

@adracea I have ran with the config file you mentioned but the urls are not getting included in scope

I tried burp-rest-api.bat --config.file=config.json

with below config file
{ "target":{ "scope":{ "advanced_mode":false, "exclude":[], "include":[ { "enabled":true, "prefix":"http://localhost:5432/auth" }, { "enabled":true, "prefix":"http://localhost:5432/users" }, { "enabled":true, "prefix":"http://localhost:5432/bar/foo" } ] } } }

[adracea]

@hardikrana437 I think it's --config-file=config.json . You might need to do the config in the UI and export the project configuration but this should work .

[hardikrana437]

Ahhh!! my bad!!
thanks!!it worked..
Before closing this, I have 1 question. For testing the rest-api can we send the api urls to scan without passing the http method as well as its payload @adracea ?

[adracea]

When you start the scan , it's started based off of the URL you give so , if you add to scope just these 3 URLs , it will only scan these 3 but you'll have to make separate requests for all 3 . If you scan the base url http://localhost:5432/ it might not start a scan because URL is not in scope , but , if you have the 3 urls in sitemap , have the base url in scope , and start a scan on it , it will scan all of the urls in sitemap with a single call .

[hardikrana437]

@adracea
Yes I got that, but will the burp take care of the http method and payload of the urls?

[adracea]

You mean , get/post/put ? Well it will use the ones present in the forms when you spider or the ones you used in the session file , otherwise it will just run blind gets requests at these .

[hardikrana437]

I have 2 options now

1. To populate the sitemap through my rest api framework. In this case http methods and payload is available.
2. To poulate the sitemap by send my rest urls to spider. It will just run blind gets requests at these.

Which one is better @adracea as 1st one is extracting more vulnerabilities for me.

Note: I am referring to rest vulnerability testing, not UI one. So there are no forms as we have in UI.

[adracea]

I have 2 options now

1. To populate the sitemap through my rest api framework. In this case http methods and payload is available.
2. To poulate the sitemap by send my rest urls to spider. It will just run blind gets requests at these.

Which one is better as 1st one is extracting more vulnerabilities for me.

The 2nd option will spider the pages and look for the forms and the form action and generate http requests based off of these . Using your own testing framework to push traffic through burp is much better than just using the spider :)

[hardikrana437]

@adracea I am referring to rest api testing, not UI one. So there is no form option to track the req as we have in UI.

[adracea]

Ah , then , no , the spider is not a good option . Best you use a testing framework like newman(headless Postman) or similar with a proxy option set to burp and that's it .

[hardikrana437]

Yes, in my opinion too :), but that is causing me having duplicate urls as in my framework I was trying diff inputs with the rest call.

[adracea]

You can look at some more advanced scope management rules , as burp now allows for it . I think you can also filter items that will be scanned based off of parameters like HTTP method , etc. Do look at burp's UI and try to play around with the settings before moving on to using the rest api :D

[hardikrana437]

Thanks a lot :D closing the thread.
😆 👍