A tool to scan a particular website and extract useful information out of it
Following information is obtained
- Nmap port scan
- IP Address
- Whois detail
- robots.txt
Input is expected as the name of project directory and the full Url of the website
TODO: Check for existence of robots.txt. Many websites allow full scraping access.