/web-recon

All About Web Recon & OSINT

pr0xh4ck © 2022

demo

Table of Contents
  1. IP Check
  2. DNS
  3. Internet Search Engine Discovery
  4. Sudomain Enumeration
  5. DNS Bruteforce
  6. OSINT
  7. HTTP Probing
  8. Subdomain Takeover
  9. Web screenshot
  10. CMS Enumeration
  11. Automation
  12. Cloud Enumeration
  13. Github & Secrets
  14. Email Hunting
  15. Data Breach
  16. Web Wayback
  17. Ports Scannig
  18. WAF
  19. Directory Search
  20. Hidden File or Directory
  21. Hidden Parameter Find
  22. Bypass Forbidden Direcory
  23. Wordlists & Payloads
  24. Miscellaneous
  25. Social Engineering
  26. One Line Scripts
  27. API kay
  28. Code review
  29. Log File Analyze
  30. Public programs
  31. Burp Suite Extension
  32. DOS
  33. Websocket

Start

shodan

ssl.cert.subject.CN:”*.target com”+200
http.favicon.hash:


org:"Target" http.title:"GitLab"
Username: root & pass: 5iveL!fe
Username: admin & Pass: 5iveL!fe

ip-test

Virtual Host Finding

dns

apt-get update
apt-get install dnsutils
  • domaineye
  • anslookup
  • dns
  • DNSStager
  • singularity - A DNS rebinding attack framework.
  • whonow - A "malicious" DNS server for executing DNS Rebinding attacks on the fly (public instance running on rebind.network:53)
  • dns-rebind-toolkit - A front-end JavaScript toolkit for creating DNS rebinding attacks.
  • dref - DNS Rebinding Exploitation Framework
  • rbndr - Simple DNS Rebinding Service
  • httprebind - Automatic tool for DNS rebinding-based SSRF attacks
  • dnsFookup - DNS rebinding toolkit

DNS public name server

internet-search-engine-discovery

subdomain-enumeration

curl 'https://crt.sh/?q=%.example.com&output=json' | jq '.name_value' | sed 's/\"//g' | sed 's/\*\.//g' | sort -u

Exception(web) subdomain enumeration

curl -s https://dns.bufferover.run/dns?q=DOMAIN.com |jq -r .FDNS_A[]|cut -d',' -f2|sort -u

Find subdomain on GitHub

Find subdomain from Official DoD(Depart of Defence) website

dns-bruteforce

osint

  • DarkScrape - OSINT Tool For Scraping Dark Websites
  • virustotal - Analyze suspicious files and URLs to detect types of malware, automatically share them with the security community
  • RED_HAWK - All in one tool for Information Gathering, Vulnerability Scanning and Crawling. A must have tool for all penetration testers
  • siteindices - siteindices
  • udork.sh
  • fav-up
  • testssl - Testing TLS/SSL encryption anywhere on any port
  • bbtz
  • sonar search
  • notify - Notify is a Go-based assistance package that enables you to stream the output of several tools (or read from a file) and publish it to a variety of supported platforms.
  • email finder
  • analytics relationships
  • mapcidr
  • ppfuzz
  • cloud-detect
  • interactsh
  • bbrf
  • spiderfoot - SpiderFoot automates OSINT for threat intelligence and mapping your attack surface.
  • visualsitemapper - free service that can quickly show an interactive visual map of your site.
  • jwt - JWT.IO allows you to decode, verify and generate JWT. Gain control over your JWTs
  • bgp.he - Internet Backbone and Colocation Provider
  • spyse - Find any Internet asset by digital fingerprints
  • whoxy - whois database

http-probing

subdomain-takeover

host -t CNAME input.com
  • subjack - Subdomain Takeover tool written in Go
  • SubOver - A Powerful Subdomain Takeover Tool
  • autoSubTakeover - A tool used to check if a CNAME resolves to the scope address. If the CNAME resolves to a non-scope address it might be worth checking out if subdomain takeover is possible.
  • NSBrute - Python utility to takeover domains vulnerable to AWS NS Takeover
  • can-i-take-over-xyz - "Can I take over XYZ?" — a list of services and how to claim (sub)domains with dangling DNS records.
  • Can-I-take-over-xyz-v2 - V2
  • cnames - take a list of resolved subdomains and output any corresponding CNAMES en masse.
  • subHijack - Hijacking forgotten & misconfigured subdomains
  • tko-subs - A tool that can help detect and takeover subdomains with dead DNS records
  • HostileSubBruteforcer - This app will bruteforce for exisiting subdomains and provide information if the 3rd party host has been properly setup.
  • second-order - Second-order subdomain takeover scanner
  • takeover - A tool for testing subdomain takeover possibilities at a mass scale.

web-screenshot

  • EyeWitness - EyeWitness is designed to take screenshots of websites, provide some server header info, and identify default credentials if possible.
  • aquatone - Aquatone is a tool for visual inspection of websites across a large amount of hosts and is convenient for quickly gaining an overview of HTTP-based attack surface.
  • screenshoteer - Make website screenshots and mobile emulations from the command line.
  • gowitness - gowitness - a golang, web screenshot utility using Chrome Headless
  • WitnessMe - Web Inventory tool, takes screenshots of webpages using Pyppeteer (headless Chrome/Chromium) and provides some extra bells & whistles to make life easier.
  • eyeballer - Convolutional neural network for analyzing pentest screenshots
  • scrying - A tool for collecting RDP, web and VNC screenshots all in one place
  • Depix - Recovers passwords from pixelized screenshots
  • httpscreenshot - HTTPScreenshot is a tool for grabbing screenshots and HTML of large numbers of websites.

cms-enumeration

  • ObserverWard - Cross platform community web fingerprint identification tool

AEM - aem-hacker

  • cmseek - CMS Detection and Exploitation suite - Scan WordPress, Joomla, Drupal and over 180 other CMSs
  • webanlyze - Port of Wappalyzer (uncovers technologies used on websites) to automate mass scanning.
  • whatweb - Next generation web scanner
  • wappalyzer - wappalyzer website
  • wappalyzer cli - Identify technology on websites.
  • build with
  • build with cli - BuiltWith API client
  • backlinkwatch - Website for backlink finding
  • retirejs -scanner detecting the use of JavaScript libraries with known vulnerabilities

automation

  • inventory - Asset inventory on public bug bounty programs.
  • bugradar - Advanced external automation on bug bounty programs by running the best set of tools to perform scanning and finding out vulnerabilities.
  • wapiti-scanner - Scan your website
  • nuclei - Nuclei is a fast tool for configurable targeted scanning based on templates offering massive extensibility and ease of use.
  • scant3r - ScanT3r - Module based Bug Bounty Automation Tool
  • Sn1per - Automated pentest framework for offensive security experts
  • metasploit-framework - Metasploit Framework
  • nikto - Nikto web server scanner
  • arachni - Web Application Security Scanner Framework
  • jaeles - The Swiss Army knife for automated Web Application Testing
  • retire.js - scanner detecting the use of JavaScript libraries with known vulnerabilities
  • Osmedeus - Fully automated offensive security framework for reconnaissance and vulnerability scanning
  • getsploit - Command line utility for searching and downloading exploits
  • flan - A pretty sweet vulnerability scanner
  • Findsploit - Find exploits in local and online databases instantly
  • BlackWidow - A Python based web application scanner to gather OSINT and fuzz for OWASP vulnerabilities on a target website.
  • backslash-powered-scanner - Finds unknown classes of injection vulnerabilities
  • Eagle - Multithreaded Plugin based vulnerability scanner for mass detection of web-based applications vulnerabilities
  • cariddi - Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more...
  • kenzer - automated web assets enumeration & scanning
  • ReScue - An automated tool for the detection of regexes' slow-matching vulnerabilities.

file upload scanner

  • fuxploider - File upload vulnerability scanner and exploitation tool.

Network Scanner

  • openvas - Free software implementation of the popular Nessus vulnerability assessment system.
  • vuls - Agentless vulnerability scanner for GNU/Linux and FreeBSD, written in Go.
  • nexpose - Commercial vulnerability and risk management assessment engine that integrates with Metasploit, sold by Rapid7.
  • nessus - Commercial vulnerability management, configuration, and compliance assessment platform, sold by Tenable.

Vulnerable Pattern Search

wordpress

joomla

drupal

  • droopescan - A plugin-based scanner that aids security researchers in identifying issues with several CMSs, mainly Drupal & Silverstripe.

cloud-enumeration

Buckets

  • S3Scanner - Scan for open AWS S3 buckets and dump the contents
  • AWSBucketDump - Security Tool to Look For Interesting Files in S3 Buckets
  • CloudScraper - CloudScraper: Tool to enumerate targets in search of cloud resources. S3 Buckets, Azure Blobs, Digital Ocean Storage Space.
  • s3viewer - Publicly Open Amazon AWS S3 Bucket Viewer
  • festin - FestIn - S3 Bucket Weakness Discovery
  • s3reverse - The format of various s3 buckets is convert in one format. for bugbounty and security testing.
  • mass-s3-bucket-tester - This tests a list of s3 buckets to see if they have dir listings enabled or if they are uploadable
  • S3BucketList - Firefox plugin that lists Amazon S3 Buckets found in requests
  • dirlstr - Finds Directory Listings or open S3 buckets from a list of URLs
  • Burp-AnonymousCloud - Burp extension that performs a passive scan to identify cloud buckets and then test them for publicly accessible vulnerabilities
  • kicks3 - S3 bucket finder from html,js and bucket misconfiguration testing tool
  • 2tearsinabucket - Enumerate s3 buckets for a specific target.
  • s3_objects_check - Whitebox evaluation of effective S3 object permissions, to identify publicly accessible files.
  • s3tk - A security toolkit for Amazon S3
  • CloudBrute - Awesome cloud enumerator
  • s3cario - This tool will get the CNAME first if it's a valid Amazon s3 bucket and if it's not, it will try to check if the domain is a bucket name.
  • S3Cruze - All-in-one AWS S3 bucket tool for pentesters.

github-secrets

  • githacker
  • git-hound
  • gh-dork - Github dorking tool
  • gitdorker - A Python program to scrape secrets from GitHub through usage of a large repository of dorks.
  • github-endpoints
  • git-secrets - Prevents you from committing secrets and credentials into git repositories
  • gitleaks - Scan git repos (or files) for secrets using regex and entropy
  • truffleHog - Searches through git repositories for high entropy strings and secrets, digging deep into commit history
  • gitGraber - gitGraber: monitor GitHub to search and find sensitive data in real time for different online services
  • talisman - By hooking into the pre-push hook provided by Git, Talisman validates the outgoing changeset for things that look suspicious - such as authorization tokens and private keys.
  • GitGot - Semi-automated, feedback-driven tool to rapidly search through troves of public data on GitHub for sensitive secrets.
  • git-all-secrets - A tool to capture all the git secrets by leveraging multiple open source git searching tools
  • github-search - Tools to perform basic search on GitHub.
  • git-vuln-finder - Finding potential software vulnerabilities from git commit messages
  • commit-stream - #OSINT tool for finding Github repositories by extracting commit logs in real time from the Github event API
  • gitrob - Reconnaissance tool for GitHub organizations
  • repo-supervisor - Scan your code for security misconfiguration, search for passwords and secrets.
  • GitMiner - Tool for advanced mining for content on Github
  • shhgit - Ah shhgit! Find GitHub secrets in real time
  • detect-secrets - An enterprise friendly way of detecting and preventing secrets in code.
  • rusty-hog - A suite of secret scanners built in Rust for performance. Based on TruffleHog
  • whispers - Identify hardcoded secrets and dangerous behaviours
  • yar - Yar is a tool for plunderin' organizations, users and/or repositories.
  • dufflebag - Search exposed EBS volumes for secrets
  • secret-bridge - Monitors Github for leaked secrets
  • earlybird - EarlyBird is a sensitive data detection tool capable of scanning source code repositories for clear text password violations, PII, outdated cryptography methods, key files and more.

GitHub dork wordlist

Git

  • GitTools - A repository with 3 tools for pwn'ing websites with .git repositories available
  • gitjacker - Leak git repositories from misconfigured websites
  • git-dumper - A tool to dump a git repository from a website
  • GitHunter - A tool for searching a Git repository for interesting content
  • dvcs-ripper - Rip web accessible (distributed) version control systems: SVN/GIT/HG...

email-hunting

data-breach

web-wayback

  • waymore - Find way more from the Wayback Machine!
  • sigurlfind3r - A passive reconnaissance tool for known URLs discovery - it gathers a list of URLs passively using various online sources
  • waybackurls - Fetch all the URLs that the Wayback Machine knows about for a domain
  • gau - Fetch known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, and Common Crawl.
  • gauplus - A modified version of gau
  • waybackpy - Wayback Machine API Python interfaces and CLI tool.
  • chronos - Extract pieces of info from a web page's Wayback Machine history

Replace parameter value

  • bhedak - A replacement of "qsreplace", accepts URLs as standard input, replaces all query string values with user-supplied values and stdout.

Find reflected params

  • gxss - A tool to check a bunch of URLs that contain reflecting params.
  • freq - This is go CLI tool for send fast Multiple get HTTP request.
  • bxss - A Blind XSS Injector tool

Find js file from waybackurls.txt

Automatic put parameter value

Declutters url lists

ports-scanning

  • masscan - TCP port scanner, spews SYN packets asynchronously, scanning entire Internet in under 5 minutes.
  • RustScan - The Modern Port Scanner
  • naabu - A fast port scanner written in go with focus on reliability and simplicity.
  • nmap - Nmap - the Network Mapper. Github mirror of official SVN repository.
  • sandmap - Nmap on steroids. Simple CLI with the ability to run pure Nmap engine, 31 modules with 459 scan profiles.
  • ScanCannon - Combines the speed of masscan with the reliability and detailed enumeration of nmap
  • unimap

Brute-Forcing from Nmap output

waf

  • wafw00f
  • cf-check
  • w3af - w3af: web application attack and audit framework, the open source web vulnerability scanner.

Waf bypass

  • bypass-firewalls-by-DNS-history - Firewall bypass script based on DNS history records. This script will search for DNS A history records and check if the server replies for that domain. Handy for bugbounty hunters.
  • CloudFail - Utilize misconfigured DNS and old database records to find hidden IP's behind the CloudFlare network

directory-search

  • gobuster - Directory/File, DNS and VHost busting tool written in Go
  • recursebuster - rapid content discovery tool for recursively querying webservers, handy in pentesting and web application assessments
  • feroxbuster - A fast, simple, recursive content discovery tool written in Rust.
  • dirsearch - Web path scanner
  • dirsearch - A Go implementation of dirsearch.
  • filebuster - An extremely fast and flexible web fuzzer
  • dirstalk - Modern alternative to dirbuster/dirb
  • dirbuster-ng - dirbuster-ng is C CLI implementation of the Java dirbuster tool
  • gospider - Gospider - Fast web spider written in Go
  • hakrawler - Simple, fast web crawler designed for easy, quick discovery of endpoints and assets within a web application

Fuzzing

  • ffuf - Fast web fuzzer written in Go
  • wfuzz - Web application fuzzer
  • fuzzdb - Dictionary of attack patterns and primitives for black-box application fault injection and resource discovery.
  • IntruderPayloads - A collection of Burpsuite Intruder payloads, BurpBounty payloads, fuzz lists, malicious file uploads and web pentesting methodologies and checklists.
  • fuzz.txt - Potentially dangerous files
  • fuzzilli - A JavaScript Engine Fuzzer
  • fuzzapi - Fuzzapi is a tool used for REST API pentesting and uses API_Fuzzer gem
  • qsfuzz - qsfuzz (Query String Fuzz) allows you to build your own rules to fuzz query strings and easily identify vulnerabilities.

hidden-file-or-directory

18-03-22

JS

  • linx - Reveals invisible links within JavaScript files
  • diffJs - Tool for monitoring changes in javascript files on WebApps for reconnaissance.
  • scripthunter - Tool to find JavaScript files on Websites

Metadata

  • exiftool - ExifTool meta information reader/writer

  • earlybird - EarlyBird is a sensitive data detection tool capable of scanning source code repositories for clear text password violations, PII, outdated cryptography methods, key files and more.

  • DumpsterDiver - Tool to search secrets in various filetypes.

  • ChopChop - ChopChop is a CLI to help developers scanning endpoints and identifying exposition of sensitive services/files/folders.

  • gospider - Fast web spider written in Go

  • gobuster - Directory/File, DNS and VHost busting tool written in Go

  • janusec

  • source leak hacker

  • favfreak

  • jwsxploiter - A tool to test security of json web token

  • bfac - BFAC (Backup File Artifacts Checker): An automated tool that checks for backup artifacts that may disclose the web-application's source code.

  • jsearch

  • linkfinder - A python script that finds endpoints in JavaScript files

  • secretfinder - A python script for find sensitive data (apikeys, accesstoken,jwt,..) and search anything on javascript files

  • jsa

  • JSParser - A python 2.7 script using Tornado and JSBeautifier to parse relative URLs from JavaScript files. Useful for easily discovering AJAX requests when performing security research or bug bounty hunting.

Broken link

parameter-finder

  • paramspider - Mining parameters from dark corners of Web Archives
  • parameth - This tool can be used to brute discover GET and POST parameters
  • param-miner - This extension identifies hidden, unlinked parameters. It's particularly useful for finding web cache poisoning vulnerabilities.
  • ParamPamPam - This tool for brute discover GET and POST parameters.
  • Arjun - HTTP parameter discovery suite.

Dlelte Duplicate from waybacks

  • dpfilter - BugBounty , sort and delete duplicates param value without missing original value

bypass-forbidder-directory

  • dirdar - DirDar is a tool that searches for (403-Forbidden) directories to break it and get dir listing on it
  • 4-ZERO-3 - 403/401 Bypass Methods
  • byp4xx - Pyhton script for HTTP 40X responses bypassing. Features: Verb tampering, headers, #bugbountytips tricks and 2454 User-Agents.
  • 403bypasser - 403bypasser automates techniques used to bypass access control restrictions on target pages. This tool will continue to be developed, contributions are welcome.

wordlists-payloads

  • bruteforce-lists - Some files for bruteforcing certain things.

  • CheatSheetSeries - The OWASP Cheat Sheet Series was created to provide a concise collection of high value information on specific application security topics.

  • Bug-Bounty-Wordlists - A repository that includes all the important wordlists used while bug hunting.

  • seclists - SecLists is the security tester's companion. It's a collection of multiple types of lists used during security assessments, collected in one place. List types include usernames, passwords, URLs, sensitive data patterns, fuzzing payloads, web shells, and many more.

  • Payload Box - Attack payloads only 📦

  • awesome-wordlists - A curated list wordlists for bruteforcing and fuzzing

  • Fuzzing-wordlist - fuzzing-wordlists

  • Web-Attack-Cheat-Sheet - Web Attack Cheat Sheet

  • payloadsallthethings - A list of useful payloads and bypass for Web Application Security and Pentest/CT

  • pentestmonkey - Taking the monkey work out of pentesting

  • STOK suggest

Exceptional

miscellaneous

social-engineering

  • social-engineer-toolkit - The Social-Engineer Toolkit (SET) repository from TrustedSec - All new versions of SET will be deployed here.

Uncategorized

  • JSONBee - A ready to use JSONP endpoints/payloads to help bypass content security policy (CSP) of different websites.
  • CyberChef - The Cyber Swiss Army Knife - a web app for encryption, encoding, compression and data analysis
  • bountyplz - Automated security reporting from markdown templates (HackerOne and Bugcrowd are currently the platforms supported)
  • awesome-vulnerable-apps - Awesome Vulnerable Applications
  • XFFenum - X-Forwarded-For [403 forbidden] enumeration

scripts

API_key

  • keyhacks - Keyhacks is a repository which shows quick ways in which API keys leaked by a bug bounty program can be checked to see if they're valid.
  • gmapsapiscanner - Used for determining whether a leaked/found Google Maps API Key is vulnerable to unauthorized access by other applications or not.

Code_review

  • phpvuln - 🕸️ Audit tool to find common vulnerabilities in PHP source code

log-file-analyze

programs

  • disclose -Open-source vulnerability disclosure and bug bounty program database.
  • bug bounty dork - List of Google Dorks for sites that have responsible disclosure program / bug bounty program
  • crunchbase - Discover innovative companies and the people behind them
  • bounty-targets-data - This repo contains hourly-updated data dumps of bug bounty platform scopes (like Hackerone/Bugcrowd/Intigriti/etc) that are eligible for reports
  • Vdps_are_love - This repo is made for those hunters who love to hunt on VDP programs. List of Vdp programs which are not affiliated with known bug bounty platforms such as HackerOne or Bugcrowd.
  • chaos - We actively collect and maintain internet-wide assets' data, this project is meant to enhance research and analyse changes around DNS for better insights.
  • bug-bounty-list - The most comprehensive, up to date crowdsourced list of bug bounty and security vulnerability disclosure programs from across the web curated by the hacker community.

burp-suite-extesion

    Collaborator Everywhere
    XSS Validator
    Wsdler
    .NET Beautifier
    Bypass WAF
    J2EEScan
    Param Miner
    Wayback Machine
    JS Link Finder
    Upload Scanner
    Nucleus Burp Extension
    Software Vulnerability Scanner
    Active Scan++

Burp suite pro

  • Burp-Suite - || Activate Burp Suite Pro with Loader and Key-Generator ||

Scope

^.+\.company\.com$     ^443$   ^/.*

DOS

Websocket

  • STEWS - A Security Tool for Enumerating WebSockets

Smart-Contract

  • mythril - Security analysis tool for EVM bytecode. Supports smart contracts built for Ethereum, Hedera, Quorum, Vechain, Roostock, Tron and other EVM-compatible blockchains.

(back to top)