This program is designed for the web application asset discovery and tracking. It was originally developed to to cover the gaps of a similar commercial product. Over the time it grows to be a more capable and complete replacement (IMHO).
The latest release is Beta version 1.5.x as of fall 2014. Please refer to the CHANGELOG.md for the program’s history information.
This program is designed and developed by Yang Li. You can reach him by Email: <yang.li@owasp.org>
To take full power of this program, you would need an *nix flavor machine with direct Internet access. I have installed it successfully on both Mac and Linux machines. You’ll also need the Ruby environment being setup properly. The easiest way to install OWASP Web Mapper is by using Ruby Gems. Download the latest gem ‘wmap-x.x.x.gem’ into the local file system. Then install it from command line there:
gem install wmap-x.x.x.gem --no-rdoc
You need the Ruby 1.9.2 or above in order to use this program. In my test environment, I was able to set it up with RVM. Please refer to this page for more installation information: www.ruby-lang.org/en/downloads/
In addition, the following Ruby GEM dependency are needed by different components of this software:
require "digest/md5"
require "dnsruby"
require "geoip"
require "minitest/autorun"
require "net/http"
require "net/ping"
require "netaddr"
require "nokogiri"
require "open_uri_redirections"
require "openssl"
require "open-uri"
require "parallel"
require "resolv"
require "singleton"
require "uri"
require "whois"
require 'httpclient'
require 'nokogiri'
require 'open-uri'
To install "uri" gem for example, use the command below:
$ gem install uri
This software depends on a patched version of Ruby gem ruby-whois-2.7.0 (www.ruby-whois.org/) for the domain whois lookup feature. For better result, you could manually add the patches into your local whois gem installation directory as shown below:
$ cp whois_patches/* [Your_ruby_whois_gem_path]/whois/lib/whois/record/parser/
Or you can directly download the branched whois gem from this repository - github.com/yangsec888/whois
You need to define a scope for the program to run successful. The scope includes both your legitimate Internet domain, and your public network block in the CIDR format.
To add your Internet domain into the scope, use the build-in shell command below:
$ trust XYZ.COM
To add your public network block into the scope:
$ trust x.x.x.x/x
$ wmap <seed file | target host | target url | target IP or network cidr>
The above utility is intelligent enough to take argument as either a seed file, or a string such as a host, an IP, a network block, or a URL. The new discoveries will be automatically tracked in the data file ‘lib/wmap/data/target_sites’.
Note: seed file - mix of url, cidr and domain seed, one entry per line.
url seed - known URL(s) for further discovery via the web crawler. cidr seed - list of known network blocks, for discovering web service via port scanning; it is also used to validate if the web service has a known IP (internal hosted). domain seed - validated internet domain to be used for DNS record brute-forcing; it is also used to validate the ownership of found web service.
You can dump out the program output by using the build-in utility ‘wdump’ as shown below:
$ wdump [output file name from you]
The above utility will dump out the discovery database into a single file as program output. Currently, the supported file format is Comma-separated Value (.csv) and Extensible Markup Language (.xml)
There are more examples under the ‘demos’ folder of this package. The examples show how to use the ‘wmap’ API to get your job done easily. Please check out the code - they should be easy and straightforward to be understood.
The software comes with the Ruby doc during your installation as shown above. For your convenience, the Ruby doc is also distributed with this software. You can navigate to the ‘doc’ folder of your local installation, and click the ‘index.html’ to open the start page in your favorite browser. You can also download the wmap-x.x.x.rdoc.zip documentation package alone from GitHub, unzip and open the doc/index.html in your browser.
If you need additional documentation / information other than this README file and the Ruby document package, please be patient - as I’m still working on it :)
Contact the author Yang Li directly at email ‘yang.li@owasp.org’.
This software is provided strictly ‘as-if’ without any implied warranty. You’re free to copy or modify the codes anyway you want - a reference back to this software will be appreciated. Please refer to the ‘LICENSE.txt’ file for more information.