/open-source-research-notebooks

Jupyter notebooks helping open source researchers, journalists, and fact-checkers use command line tools and code projects for digital investigations.

Primary LanguageJupyter NotebookMIT LicenseMIT

Open Source Research Notebooks

Bellingcat logo: Discover BellingcatDiscord logo: Join our communityHeart icon: Support our work

A collection of Jupyter Notebooks that can be to learn or run open source investigations. We cover command line tools, scripts, and methods. We aim to help open source researchers, journalists, and fact-checkers use command line tools and code projects for digital investigations.

📰 Read our guide to code notebooks here.

📹 Watch our tutorial on Youtube:

YouTube

Is this for me?

If you've used Jupyter notebooks before you should be good to navigate this right away.

If Jupyter Notebooks/Google Colaboratory don't ring any bells, you should know they are one of the easiest ways to interact with code and the command line, in essence they look like an interactive website where you execute one cell at a time to run different pieces of code, you can also edit the code and adapt it to your needs.

There's plenty of good tutorials about Jupyter Notebooks and the environments you can run them in (like Google Colab or Binder.org), we do advise you to spend 5min doing that. Additionally, we created a simple notebook that you can view (static), run on Google Colab, run on Binder, or run on Kaggle that shows the core skills needed to understand what this is all about!

Why Notebooks?

Jupyter Notebooks

  • make it easier to install and run software (less "this does not work on my machine" errors)
  • make it safer to run unknown code, when you run it on a Notebook service like Google Colaboratory, Binder, or Kaggle Notebooks
  • make it simpler to document specific code uses: they combine styled documentation with code
  • are quick to make and easy to use after you understand how to click through the code cells and edit any custom input
  • can run bash commands (command line) as well as code (Python and more)

Available Notebooks

The top part of the table consists of tools and methods developed at Bellingcat; the bottom part of community developed tools and methods.

Notebook Level Description Notebook links Tags
Telegram Phone Number Checker easy Check if phone numbers have associated Telegram accounts Colab Binder Kaggle Notebook Jupyter Notebook scraping, social-media, telegram
Wayback Google Analytics easy Uncover historical analytics ids via the Wayback Machine Colab Binder Kaggle Notebook Jupyter Notebook wayback-machine, google-analytics
Geoclustering medium Find clusters of incidents from a CSV of incidents Colab Binder Kaggle Notebook Jupyter Notebook ai, clustering, gis
vk-url-scraper medium Scrape data and download media from VKontakte URLs Colab Binder Kaggle Notebook Jupyter Notebook scraping, social-media
Search Grid Generator medium Generate grid in KML file for systematic area searches. Colab Binder Kaggle Notebook Jupyter Notebook geolocation, maps
- - - - -
OpenAI Whisper easy AI audio/video transcription and translation Colab Binder Kaggle Notebook Jupyter Notebook ai, speech-recognition
Holehe easy Find accounts associated with an email Colab Binder Kaggle Notebook Jupyter Notebook digital-footprint-tracing
Maigret easy Find accounts associated with a username Colab Binder Kaggle Notebook Jupyter Notebook digital-footprint-tracing
Wayback Machine Downloader easy Download an entire website from the Wayback Machine. Colab Binder Kaggle Notebook Jupyter Notebook wayback-machine
YT-DLP easy Download and manipulate videos from hundreds of online platforms. Includes advanced commands with Python. Colab Binder Kaggle Notebook Jupyter Notebook archiving, media-downloads
ExifTool medium Extract and analyse file metadata tags Colab Binder Kaggle Notebook Jupyter Notebook digital-forensics, metadata
Deepface medium AI face comparison and analysis Colab Binder Kaggle Notebook Jupyter Notebook ai, image-analysis
  • easy Entry level notebooks, you need only know how to run cells and edit specific values
  • medium Notebooks that require higher technical know-how, or previous setups
  • advanced Notebooks that require technical understanding or adapting code