/OSINT_Encyclopedia

Your go-to resource for all things OSINT

MIT LicenseMIT

OSINT Encyclopedia

Credit: Cham423

This checklist is designed to increase the success of your open-source intelligence (OSINT) operations by collecting a comprehensive list of information about your target. Understanding the fundamentals of OSINT is a prerequisite to using this checklist, as detailed technical operations will not be captured here. This list will be a working document that is driven by the community and maintained by Optiv.

OSINT Checklist for ALL Engagements

  • Social Media
    • Corporate/Busniess Controlled Content
      • LinkedIn
      • Facebook
      • Instagram
    • Employee Controlled Content
      • Instagram facility analysis
      • Instagram hashtag review
  • Office 365
    • getuserrealm.srf
  • DNS
    • dnsdumpster
    • amass
    • horizontal (other domains owned by the same entity) and vertical (subdomain) domain enumeration
      • viewdns
      • whoisxmlapi domain research suite
      • riskiq
  • Host Enumeration
    • WHOIS
    • shodan
    • censys
    • spyse
  • Domain flyovers
    • aquatone
  • Document Metadata Analysis
    • pull large sites from google/aquatone report
    • pymeta
    • pull down manually

Meta Sites

The following links are additional lists and frameworks that can assist while performing OSINT.

Mail Blacklist Check

The following services allow you to check whether a domain or IP address is present on several blacklists. Additionally, this can help troubleshoot email delivery issues while performing phishing campaigns.

WHOIS

https://whois.arin.net/ui/advanced.jsp

  • Primary source
  • Manual web browsing

https://viewdns.info/

  • Multiple tools

https://domainbigdata.com/

  • Allows host correlation based on site registrant
  • Third-party

https://whoisology.com/#advanced

  • Reverse WHOIS search based on multiple parameters
  • Third-party

https://whoisfreaks.com/pricing/whois-database.html

https://www.whoisxmlapi.com/

  • Largest dataset available (800M+ domains)
  • $24,000 per year for full access to current and historical WHOIS data (for commercial license)
  • Has an API with many functions that is more affordable than the commercial license
  • Free license allows for 500 queries per month

Domains

https://domains-monitor.com/

  • Allows downloading a raw list of all registered domains in all zones
  • Updates quarterly with updated/deleted domains
  • Provides list of registration emails
  • $90 per year for access

https://networksdb.io/

https://www.expireddomains.net/

  • Monitors and lists domains that are expiring
  • Includes alexa rank and archive.org details for domains, allowing users to select valuable domains
  • Free to signup

DNS

https://dnsdumpster.com/

https://www.robtex.com/

Website Lookup

  • https://website.informer.com/
    • Gives generalized information about a website and a screenshot of the homepage. daily visitors, hosting info, alexa ranking
    • Paywall: no
    • Bot Detection: unknown
  • https://archive.ph/
    • Allows snapshotting of a webpage by providing a URL. also allows retrieving screenshots and text data from previously archived sites
    • Similar to wayback machine
    • Paywall: no
    • Bot Detection: unknown
  • https://www.page2images.com/URL-Live-Website-Screenshot-Generator
    • Generates screenshots of urls, 15 seconds or more per url
    • No cost solution
    • Bot detection: unknown

Phishing Site Lookup

  • https://www.phishtank.com/

    • Crowdsourced link submission and verification allows the community to determine phish validity
    • Limited reliability and visibility into anything more than the URL of a potential phishing site
    • Indicates whether site is online or offline
    • No cost solutions
    • API: yes, email verification required. commercial use allowed, has per hour request limit
    • Bot detection: hCaptcha (website)

  • https://openphish.com/

    • Raw feed of phishing urls
      • Free version updates every 12 hours, in text file format
      • Paid version updates more quickly and allows multiple formats (CSV or JSON)
    • Has IP address listing of recent phishing site
    • Provides global statistics of phishing attacks
      • What brands are being spoofed
      • What ASNs are most commonly hosting phishing attacks

    Twitter

    https://tinfoleak.com/

    • Shows devices, locations, etc. for a given Twitter handle
    • Requires email registration
    • Slow and requires capcha submitted for each request
    • No bulk capabilities

    Phone Number Validation

    https://phonevalidator.com/phone-validator-api.aspx

    • Shows phone number type (CELL PHONE, LANDLINE, VOIP, TOLL-FREE or UNKNOWN)
    • 0.004 per number pricing ($4 per 1000 phone numbers)
    • Useful for smishing to confirm that you can text a phone number

    Corporate Databases

    https://opencorporates.com/

    • Registration/incorporation articles for corporate entities
    • Shows registered trademarks, logos, and historical data
    • Shows branch locations
    • Can search by officer (person) as well to expand based on company involvement

    Github

    https://github.com/BishopFox/GitGot

    • Searches github for potentially sensitive info
    • Semi-interactive, prompts user to manually review then enumerates based on feedback
    • Python, last commit Sep 2020

    Mobile Emulators

    https://www.genymotion.com/

    • SaaS based mobile emulator
    • Pay as you go
    • Focused around app testing

Paywalled

Search engines:

  • Yandex - Russian google
  • Baidu - Chinese google
  • Goo - Japanese google
  • 2lingual.com - Can query search engines in two languages at a time, results are displayed side-by-side