/my-links

A collection of OSS tools and handy links

  • 98% OSS
  • 1% free-as-in-beer closed source software
  • 1% in-browser tools
  • Please don't submit to Reddit, HN, or post this on Twitter. Share, but share with close friends!
  • Josh on LinkedIn
  • Josh's Technical Blog

Table of Contents

Tools by Category

  • X Asset Management
  • X AWS Tools
  • X Backups
  • X Chef
  • X Cloud File Sync and Sharing
  • X Collaboration Tools
  • X Configuration Management
  • X Containers
  • X Data Visualization and Dashboards
  • X Distributed Systems Tools
  • X Editors
  • X Git Tools
  • X Graphics Stuff
  • X High Availability Clustering Tools
  • X HPC Tools
  • X Live CD Tools
  • X Logging
  • X Metrics
  • X Monitoring and Alerting
  • X Networking Tools
  • X Network Performance Analysis Tools
  • X Orchestration
  • X Package Patch and Repository Tools
  • X Performance Analysis Tools
  • X Provisioning Tools
  • X Python Tools and Resources
  • X Python Programming Tutorials
  • X RDBMS and SQL Tools
  • X RDBMS Performance Analysis Tools
  • X Regular Expressions
  • X Ruby Tools and Resources
  • X Ruby Programming Tutorials
  • X Secret Management
  • X Security Tools
  • X Shell Scripting and Tools
  • X Software Development Tools
  • X SSH Tools
  • X SSL Tools
  • X Storage Tools
  • X Storage Performance Analysis Tools
  • X Terminal Tools and SSH Clients
  • X Tracing and Debugging
  • X Two Factor Authentication
  • X Virtualization
  • X VMware Tools
  • X VPN Tools
  • X Web and HTTP Tools
  • X Web and HTTP Performance Analysis Tools
  • X Misc Tools and Utilities

IT News

A mix of industry and technical, traditional and next generation, dev and ops

Linux News and Articles

Linux-centric news, articles, and discussion

Linux Message Boards and Forums

Resources for troubleshooting and technical questions

Asset Management

Asset management, inventories, asset discovery, CMDB, and IPAM

  • Clusto - Inventory tracking and "cluster management" system -- Python
  • Collins - Infrastructure source of truth, created by Tumblr -- Scala
  • iTop - IT Service Management (ITSM), asset tracking, and ITIL -- PHP
  • Fusion Inventory - Multi-lingual, can be paired with GLPI for a killer solution -- perl
  • Genesis - Hardware discovery, by Tumblr, can report to Collins -- Ruby
  • GestioIP - IP address management (IPAM), web based, supports discovery -- perl
  • GLPI - Also provides license management, software auditing, and ticketing -- PHP
  • IP Plan - IPAM -- PHP
  • ITDB (IT ITems DataBase) - Includes purchase order management, floor plans, and ISO20000-like features -- PHP
  • Netdisco - Web-based network management and discovery tool, written in perl, uses SNMP -- perl
  • Network UPS Tools (NUT) - Manage power devices from over 100 manufacturers using a single web interface -- C Windows
  • NIPAP - Next-generation IPAM -- Python
  • OCS-NG (OCS Inventory NG - Automated inventory, deployment system, can sync with GLPI -- perl
  • openDCIM - Data center infrastructure management -- PHP
  • Open-AudIT - Track and report assets and configurations, supports Windows too -- PHP
  • phpIPAM - IPAM -- PHP
  • RackTables - Data center asset management -- PHP
  • racktables-contribs - RackTables user-contributed plugins -- PHP
  • Ralph - DCIM and CMDB, supports auto-discovery -- Python pip
  • Snipe-IT - Uses Bootstrap, web based, supports mobile -- PHP

AWS Tools

Tools for interacting with the Amazon Web Services ecosystem

  • Aminator - Create application-specific custom AMIs, by Netflix -- Python
  • Awesome AWS - Curated list of AWS resources -- collection
  • aws-shell (formerly Supercharged AWS CLI (SAWS)) - The best CLI for interacting with AWS -- Python pip
  • AWS SAM Local - CLI tool for local development and testing of Lambda applications -- Golang
  • Chrome AWS SAML Token Expiry Reminder (CASTER) - Automatically re-logs into AWS via ADFS before credentials expire -- Chrome
  • CloudBerry Explorer - Windows client for accessing AWS S3 buckets -- closed source Windows
  • CloudCraft - Create professional AWS architecture diagrams -- in-browser
  • credstash - Secret management using AWS KMS -- Python pip
  • Developer Tools - 3rd party tools for AWS - not screened or supported by Amazon -- various lang
  • eb_deploy - Elastic Beanstalk blue-green deployment automation -- Ruby gem
  • ec2instances.info - Open source and up to date instance price comparison tool -- Python
  • haproxy-autoscale - Wrapper for haproxy that handles auto-scaling EC2 instances -- Python
  • Ice - AWS usage tool, by Netflix -- Grails
  • ION Roller - Immutable deployment framework for web services -- Scala
  • jungle - awscli wrapper to make commands simpler and more UNIX-like -- Python pip
  • kappa - CLI tool that makes it easier to deploy, update, and test Lambda functions -- Python
  • localstack - A fully functional local AWS cloud stack for offline dev and test -- Python
  • s3cmd - Backup to AWS via the command line -- Python
  • Serverless (formerly JAWS) - Javascript + AWS stack, the entire backend is Lambda functions, server-free -- JavaScript
  • Sneaker - Store secrets on S3 using Amazon KMS -- Golang
  • ssh2ec2 - SSH into EC2 instances by tag name and/or other metadata filters -- Python pip
  • StarCluster - Toolkit for using AWS for high performance computing (HPC), by MIT -- Python
  • Stout - Easy way to reliably upload a static website to S3, capable of configuring CloudFront and Route 53 -- Golang
  • Tools for Amazon Web Services - Amazon's portal for their official tools -- various lang
  • troposphere - Python library to create CloudFormation descriptions -- Python pip

Backups

Traditional backup software - see also Cloud File Sync and Sharing

  • Amanda - The classic -- C perl
  • Attic - Written in Python, supports dedupe -- Python
  • Backupninja - Centralize way to configure and schedule many different backup utilities -- shell
  • BURP (BackUp and Restore Program) - Reduces network traffic and the amount of space required by using librsync -- C
  • BackupPC - Dedupe and a web GUI for restores -- perl
  • Backup - Gem for backup and restore, supports tons of platforms and notifiers (RDBMS, S3, Dropbox, rsync, Hipchat, Zabbix) -- Ruby gem
  • Bacula - Open source backup tool, lots of downloads so I guess it's good I don't know -- C
  • Back in Time - Similar to TimeMachine, simple GUI backup program -- Python
  • Bareos (Backup Archiving REcovery Open Sourced) - Fork of Bacula with additional features -- C
  • borgmatic - A simple wrapper script for the Borg backup software that creates and prunes backups-- Python
  • bup - Uses the git packfile format, supports global dedupe, can use "par2" redundancy -- Python
  • DAR (Disk ARchive) - Better than tar, focuses on disks instead of tapes -- C++
  • ddrecover - This should be the first data recovery tool you use -- C
  • Deja Dup - GUI for duplicity -- C
  • Duplicati - Supports encryption and dedupe -- Mono
  • Duplicity - Encrypted bandwidth-efficient backup using the rsync algorithm -- Python
  • Elkarbackup - Comes as a ready to use VM, supports Linux and Windows -- PHP
  • Fpart - Packs file systems into "partitions" so you can do multi-threaded or multi-node rsyncs -- C
  • FSArchiver - Save the contents of a file system to a compressed archive, if one of the checksums doesn't match the file is lost, not the whole backup -- C
  • Grsync - GUI for rsync -- Windows OS X C
  • Mondo Rescue - Disaster recovery, supports tapes, disks, network and CD/DVD as backup media, multiple filesystems, LVM, software and hardware RAID -- C
  • Obnam - Snapshots, dedupe, encryption -- C
  • rdiff-backup - Combines the best features of a mirror and an incremental backup in a bandwidth efficient manner -- Windows OS X Python
  • Redo Backup and Recovery - Simple bare metal backup and restore, live CD -- Windows
  • Relax and Recover (REAR) - BMR, simple, integrates with commercial backup solutions -- shell
  • rsnapshot - Uses rsync and hard links, can keep multiple full backups available while using very little disk space -- OS X perl shell
  • SafeKeep - Superb project, uses LVM snapshots -- Python
  • SnapRAID - Backup program that also stores RAID parity information -- C
  • Unison - Multi-OS file sync tool, syncs from both sides, no master -- OS X
  • UrBackup - Supports Windows, has a web interface -- Windows C++
  • ZBackup - Globally deduplicating backup tool -- C
  • zpaq - Free incremental and journaling command-line archiver -- Windows OS X C++

Chef

Chef gets it's own category because they're MY links

  • Bento - Chef's collection of baseboxes configured to work with Packer and Vagrant, great for testing -- Ruby
  • Berkshelf - Like gem bundler for Chef cookbooks, manages dependencies -- Ruby gem
  • Chake - Tool that helps you manage multiple hosts with chef-solo -- Ruby gem
  • chef-guard - feature rich Chef add-on that protects your Chef server from untested and potentially dangerous cookbooks -- Ruby
  • chef-relevant-tests - Runs Test Kitchen tests only on the parts of your code that changed (rather than the full suite) -- Ruby gem
  • chef-vault - Allow specified node(s) to dencrypt the contents of encrypted data bags -- Ruby gem
  • chef-whitelist - Library that enables easy rollout of changes to one or more hosts, by Etsy -- Ruby
  • ChefDK (Chef Development Kit) - Includes Berkshelf, Test Kitchen, ChefSpec, Foodcritic, Knife, Chef Zero, Ohai, and more -- Ruby
  • ChefSpec Examples - A billion code snippets and tests for ChefSpec -- Ruby
  • Chef Partner GitHub Repos - Official repo for tools, provisioners, and knife plugins from partners (VMware, GCE, Azure, AWS, etc) -- collections
  • Chef Sugar - Syntactic sugar for Chef by Seth V -- Ruby gem
  • Chef Supermarket (community) - Official community site for cookbooks -- various lang
  • Chef Supermarket (self-hosted) - Cookbook repository for Chef, can be hosted internally -- Ruby
  • Citadel - Cookbook to store secrets in AWS S3 buckets using EC2 and IAM -- Ruby
  • Cookbook Design Patterns - Likely the sanest cookbook architecture (Environment, Wrapper, Base, etc patterns) by JW -- article
  • Foodcritic - Chef linter and static code analysis -- Ruby gem
  • grocery-delivery - Utility for managing cookbook uploads to distributed Chef backends, by Facebook -- Ruby
  • Handlers - The official list of community written handlers (Airbrake, IRC, Campfire, etc) -- Ruby
  • Itamae - CM tool inspired by Chef, but simpler and lightweight, formerly LightChef -- Ruby gem
  • Inspec - Drop-in replacement for ServerSpec, can be used for audit purposes too -- Ruby gem
  • knife-backup - Knife plugin to help backup and restore a Chef server -- Ruby gem
  • knife-push - Official push jobs - action or a command to be executed against a subset of nodes -- Ruby
  • Knife Plugins - The official list of community written plugins for Knife -- collection
  • motherbrain - Use knife to spin up one node? Nah, use this to create a whole environment, by Riot Games -- Ruby gem
  • omnibus - Create full-stack installers for your project -- Ruby gem
  • Policy File - Policyfile is a single document to specify the cookbooks and recipes that should be applied to node(s) -- official experimental
  • Ridley - Chef API client -- Ruby gem
  • ServerSpec - RSpec tests for your servers configured by Puppet, Chef or anything else -- Ruby
  • spiceweasel - Quickly automates the creation of whole Chef environments (batch loading) -- Ruby
  • Test Kitchen - Automated testing for configuration management tools (mostly Chef) -- Ruby gem

Cloud File Sync and Sharing

Sync files to and share from public or private clouds - see also Backups

  • boar - Stores snapshots of directory trees in a local or remote repository, for BLOBs (pictures, videos , etc) -- Python C
  • Drive - Push or pull files to Google Drive via the command line -- Golang
  • git-annex Assistant - Sync folder(s) to any device (NAS, mobile, thumb, cloud, etc) via git -- C OS X mobile
  • lsyncd - Watches a local directory trees event monitor interface (inotify or fsevents) and kicks off an rsync when things change -- Lua
  • Mackup - Sync your Mac application settings to various cloud services or git -- Python OS X
  • Nextcloud - Fork of and replacement for OwnCloud, a self-hosted Dropbox -- various lang Windows OS X
  • Pydio - Formerly AjaXplorer, AGPL license -- PHP
  • Skicka - Command line tool for working with Google Drive, by Google -- Golang
  • Seafile - Also offers a paid professional edition with more features, supports most platforms -- C
  • SparkleShare - Uses git under the hood, neat -- Windows OS X
  • Syncany - Supports tons of different protocols for the transfer (SCP, FTP, Samba, S3, etc) -- Gradle
  • Syncthing - Uses an ID rather than an IP address, share your ID with friends and go -- Golang
  • Tahoe-LAFS - Free and open decentralized cloud storage system -- Python

Collaboration Tools

ChatOps, code review, groupware, webmail, code sharing, and more

  • Barkeep - Code review tool, kinda sorta like GitHub -- Ruby
  • Citadel - Messaging, collaboration tools, and groupware - an all-in-one package -- C
  • Etherpad - Enter, save, and share text/code in a web browser -- JavaScript
  • FreeIPA - Identity, policy, and audit suite, think Active Directory for Linux (LDAP, CA, x509, DNS, Kerberos) -- various lang
  • gcalcli - CLI for Google Calendar -- Python pip
  • gmvault - Export/backup and restore your Gmail account -- Python
  • Haste / hastebin - Open source pastebin alternative for sharing code, can be installed locally / on-site -- JavaScript
  • Hubot - Chat bot that can do deploys, look up images, integrate with Google Maps, and tons of other stuff -- CoffeeScript
  • Kolab - Unified communication and collaboration system -- PHP
  • LDAP Account Manager (LAM) - Full featured LDAP management GUI, can manage almost anything -- PHP
  • Let's Chat - Self-hosted open source HipChat clone -- JavaScript
  • Mattermost - OSS Slack alternative -- Golang JavaScript
  • Mailtrain - Self hosted news letter e-mail app, similar to Mailchimp -- JavaScript
  • MultiRBL - SMTP blackhole lookup tools -- in-browser
  • MX Toolbox - SMTP blacklist lookup, header analysis, and more -- in-browser
  • OpenProject - Web-based project management system built on Ruby on Rails -- Ruby
  • osTicket - Routes inquiries created via email, web-forms and phone calls into a web-based customer support platform -- PHP
  • OTRS - Open source help desk software -- perl
  • OwnTracks - Self-hosted location tracking you can share - use for diaries, work orders, etc -- various lang mobile
  • Pandoc - Convert files from one markup format to another, supports a ton of formats -- Haskell
  • Phabricator - Suite of web-based software dev collaboration tools, and all-in-one project management tool -- PHP
  • PrivateBin - Pastebin where the server has zero knowledge of pasted data, data is encrypted/decrypted in the browser using 256 bits AES -- various lang
  • QueryClips - Pastebin for Postgres or my mySQL SQL query sharing -- in-browser
  • RainLoop - Simple, modern & fast web-based email client -- PHP
  • Redmine - Project management webapp -- Ruby
  • Request Tracker - Bug tracking, help desk ticketing, customer service, workflow processes, change management and more -- perl
  • Review Board - Code review tool for multiple SCM systems -- Python
  • Rocket.Chat - OSS Slack clone built with Meteor.js -- JavaScript
  • Roundcube - Browser-based multilingual IMAP client -- PHP
  • Say What - Using speech-to-text to fully check out during conference calls -- Python
  • SOGo - Groupware that integrates with Microsoft, Android, and Apple products -- Objective-C OS X
  • Sovereign - Set of Ansible playbooks to deploy a suite of self-hosted apps (mail, colab, calendar, file sync, and more) -- Python
  • Staytus - Complete solution for publishing the latest info about issues with your web applications, networks or services -- Ruby
  • Sticky Notes - On-site / local pastebin -- PHP
  • Synergy - Share a single keyboard and mouse with multiple physical computers, only the old version is free now -- closed source Windows OS X
  • Taiga - Project management web application with agile/ scrum in mind -- Python CoffeeScript
  • TermRecord - Record and playback terminal sessions, outputs self-contained HTML -- Python
  • WeKan - OSS Trello-like kanban board -- JavaScript
  • Your Own URL Shortener (YOURLS) - Lets you run your own URL shortener a'la TinyURL or bit.ly -- PHP
  • Zulip - Group chat, by Dropbox -- various lang mobile Windows OS X

Configuration Management

OS-level configuration management, Chef has it's own category because

  • Ansible - CM and orchestration, also can do provisioning -- Python
  • Ansible Galaxy - Community site for finding, reusing, and sharing Ansible content -- various lang
  • Automatic Server Hardening - Linux hardening cookbooks/manifests/playbooks for Puppet, Chef, and Ansible -- various lang
  • Blueprint - Reverse engineer a server configuration -- Python pip
  • BOSH - Cloud orchestration, supports Docker containers too -- Ruby
  • Boxen - Mac / OS X configuration management -- Ruby gem OS X
  • etckeeper - Stores /etc in version control, super easy config management -- various lang
  • hiera-eyaml - Backend for Hiera that provides per-value encryption of sensitive data within Puppet yaml files -- Ruby
  • Juju - Ubuntu focused but works on RHEL / CentOS now -- Golang
  • kitchen-puppet - Test Kitchen provisioner for Puppet -- Ruby
  • librarian-puppet - Bundler for Puppet, manages dependencies -- Ruby gem
  • Oxidized - RANCID replacement, supports many platforms (Cisco, Brocade, Juniper, Citrix, etc) -- Ruby gem
  • puppet-lint - Check that your Puppet manifest conform to the style guide -- Ruby gem
  • Puppet Dashboard - Web based dashboard for Puppet -- JavaScript
  • Puppet Forge - The official community repository of Puppet modules -- collection
  • puppet-module-skeleton - Skeleton modules for Puppet -- Ruby
  • r10k - General purpose toolset for deploying Puppet environments and modules -- Ruby gem
  • RANCID - (Really Awesome New Cisco confIg Differ) - Pulls and saves network device configs and saves them into a CVS, now supports git -- C
  • Rudder - Automation and compliance, uses CFengine -- Scala
  • Salt / Saltstack - Orchestration, server provisioning, and configuration management -- Python

Containers

Linux containers, container orchestration, provisioning, networking, PaaS, Kubernetes, and related tools - see also (#distributed-systems-tools)Distributed Systems Tools

  • appscale - Open source implementation of Google App Engine -- Python
  • Awesome Docker - Massive Docker collection -- collection
  • cadvisor - Analyzes resource usage and performance characteristics of running containers -- Golang
  • Clear Linux - New name for Clear Containers, attempts combine the security advantages of VMs with the deployment advantages of containers -- various lang
  • Cloud Foundry - Open source PaaS by VMware and Pivotal -- Golang Ruby
  • crane - Docker orchestration, similar to Docker Compose -- Golang
  • ctop - ncurses top-like UI for containers -- Golang
  • docker-gc - Docker garbage collection of containers and images -- shell
  • DockerSlim - Uses static and dynamic analysis to create skinny image variants of your fat images -- Golang
  • Docker Compose - Define and run multi-container apps with Docker, previously known as fig, official -- Python
  • Docker Distribution - AKA Docker Registry 2.0 - pack, ship, store, and deliver containers -- Golang
  • Docker Hub - Official Docker images for many projects -- various lang
  • Docker Swarm - Docker-native clustering system -- Golang
  • Docker Toolbox - Docker Client, Machine, Compose, Kitematic, VirtualBox, and the boot2docker VM in a single package, official -- various lang
  • Dokku - Docker powered mini-Heroku (PaaS) in around 100 lines of bash -- shell
  • draft - Streamlined Kubernetes development with sandbox testing & deployment in seconds -- Golang
  • drone - CI platform built on Docker -- Golang
  • dumb-init - Minimal init system for containers, by Yelp -- C
  • Fabric8 - Open source microservices platform based on Docker, Kubernetes and Jenkins -- Java
  • Flannel - Network fabric for containers via etcd, designed for Kubernetes -- Golang
  • Flocker - Docker data management for stateful services such as DBs -- Python
  • Flynn - PaaS built on top of Docker, can run stateful services -- Golang
  • Heapster - Container cluster monitoring and performance analysis for Kubernetes and CoreOS -- Golang
  • Heptio Ark - Manage disaster recovery for your Kubernetes persistent volumes and cluster resources -- Golang
  • Heptio Sonobuoy - Kubernetes end to end conformance testing and debugging tool -- Golang
  • Kismatic - Set of production-ready defaults and best practice tools for creating enterprise-tuned Kubernetes clusters. -- Golang
  • Kitematic - GUI Docker management on Mac & Windows (laptops), official -- JavaScript Windows OS X
  • kops - CLI for managing, upgrading, maintaining, creating, etc Kubernetes clusters -- Golang
  • kubeadm-dind-cluster - kubeadm and DIND (Docker in Docker) create a multi-node Kubernetes cluster on your laptop for dev -- Shell
  • kube-applier - service that enables continuous deployment of Kubernetes objects by applying declarative configuration files from a Git repository to a Kubernetes cluster -- Golang
  • kubectx - Easily switch between kubectl contexts and namespaces -- Ruby
  • kubed - Perform periodic cluster snapshots, provide temp storage for deleted objects, automatic event forwarding, deliver notifications via various channels for Kubernetes -- Golang
  • kubernetes - Open source orchestration system for Docker containers, by Google -- Golang
  • kubicorn - Multi-provider deployment abstraction layer / library for Kubernetes -- Golang
  • Logspout - Log router for Docker containers -- Golang
  • minikube - Kubernetes environments on your laptop -- Golang
  • OpenShift Origin - Docker + Kubernetes, the upstream for Red Hat's OpenShift 3.x -- Golang
  • Panamax - Containerized app creator with an open-source app marketplace hosted in GitHub -- various lang
  • pipework - SDN for Linux Containers -- various lang
  • Portainer - Web interface for Docker aka us-for-docker -- JavaScript
  • Project Atomic - Super stripped down Linux OS designed to only run containers and nothing more, aka RHEL Atomic -- various lang
  • Rancher - Provides a complete platform for operating Docker in production -- various lang
  • Registrator - Service registry bridge for Docker, supports Consul, etcd -- Golang
  • rkt (Rocket) - New container system by the CoreOS guys -- Golang
  • Rocker - Adds new instructions to the Dockerfile instruction set for smaller images and faster builds -- Golang
  • Shipyard - Composable Docker management toolkit -- Golang
  • Swarm Visualizer - A visualizer for Docker Swarm using the Docker Remote API, Node.JS, and D3 -- JavaScript
  • Tsuru - Docker-based PaaS -- Golang
  • Watchtower - Monitors your running Docker containers and restart them when a new image is available -- Golang
  • Weave - Virtual network that connects Docker containers deployed across multiple hosts -- Golang

Dashboards and Data Visualization

Dashboards for monitoring, alerting, metrics and data visualization tools

  • Bigdesk - Live charts and statistics for Elasticsearch cluster -- JavaScript
  • Cachet - Create beautiful, responsive status pages -- PHP
  • Cacti - Web-based network monitoring and graphing tool designed as a front-end to RRDtool -- PHP
  • CactiEZ - Super simple, added features, works right out of the box -- PHP
  • collectd3 - Beautiful D3 visualizations of system performance statistics -- JavaScript
  • Dashing - The exceptionally handsome dashboard framework, no longer maintained -- Ruby gem
  • Dashing Widgets - Massive collection of widgets for Dashing -- collection
  • Facette - Time series data visualization and graphing software -- Golang
  • Flame Graphs - Stack trace visualizer by Brendan Gregg -- perl
  • Gource - Software version control visualization tool -- C++
  • Graphene - Graphite dashboard in D3 and Backbone -- JavaScript
  • Graphiti - Graphite dashboard using Ruby and Redis -- Ruby
  • Hygieia - Visualize near real-time status of the entire delivery pipeline, by Capital One -- Java
  • Hystrix Dashboard - Dashboard for Netflix's Hystrix tool -- Java
  • Kibana - Visualization component of the ELK stack -- JavaScript
  • ksar - Creates pretty graphs from sar output -- Java
  • logstalgia - Web site access log visualization tool, aka Apache Pong -- C++
  • gdash - Simple Graphite dashboard using Bootstrap -- Ruby
  • Grafana - Modern dashboard for Graphite -- JavaScript
  • Mozaik - Create beautiful dashboards using Node/React/D3 -- JavaScript
  • MRTG (Multi Router Traffic Grapher) - Still being updated -- perl
  • Nagdash - Dashboard / NOC screen for Nagios -- PHP
  • NagVis - Visualization suite for Nagios -- PHP
  • Nagiosgraph - Another visualization tool for Nagios data -- perl
  • Network Weathermap - Network visualization tool, create a "weather map" just like big ISPs use! -- PHP
  • pdash - web dashboard for linux using data mainly served by psutil -- Python pip
  • PNP4Nagios - Analyzes performance data provided by plugins and stores them automatically into RRD-databases -- PHP
  • redash - Web application that allows to easily query an existing database, share the dataset and visualize it in different ways -- various lang
  • ServerStatus - Display and monitor your servers statistics in a beautiful way -- C C++ JavaScript
  • Seyren - Alerting dashboard for Graphite -- Java
  • Smashing - Successor to Dashing -- Ruby
  • Stashboard - Status dashboard for APIs and software services -- Python
  • Tasseo - Dashboards using D3, HTML5, Bootstrap, and others -- JavaScript CoffeeScript
  • Tessera - Graphite dashboard in Python -- Python pip
  • Thruk - Web interface for Nagios, Icinga, Shinken and Naemon, can create SLA reports, has a mobile client -- JavaScript
  • Uchiwa - Dashboard for Sensu -- Golang
  • vnstati - Creates PNG images using vnStat data -- built-in

Distributed Systems Tools

DCOSes, microservices, service discovery, schedulers, and related tools for dynamic, warehouse-scale computing - see also HPC Tools and (#containers)Containers

  • Aurora - Mesos scheduler for long running services and cron jobs, similar to Marathon -- Java
  • Bamboo - HAProxy auto configuration and auto service discovery for Mesos and Marathon -- Golang
  • Celery - Async task/job queue based on distributed message passing -- Python
  • Chaperone - End-to-end Kafka auditing (data loss, latency, message duplication, etc), by Uber -- Java
  • Chronos - Distributed, fault tolerant cron scheduler that runs on top of Apache Mesos -- various lang
  • Cruise Control - Fully automate the dynamic workload rebalance and self-healing of a cluster, by Linkedin -- Java
  • crypt - Store and retrieve encrypted configs from etcd or Consul -- Golang
  • consul - Service discovery and configuration via DNS or HTTP, great for auto-scaling -- Golang
  • consul Tools - Official, includes consul-template and others -- various lang
  • Curator - ZooKeeper client wrapper and rich ZooKeeper framework, by Netflix -- Java
  • DC/OS - Pre-configured data center OS distribution that includes Mesos, Marathon, AdminRouter, and more -- Python
  • Envoy - Sidecar container for distributed applications or microservices -- C++
  • etcd - Distributed, consistent key-value store for shared configuration and service discovery -- Golang
  • eureka - AWS Service registry for resilient mid-tier load balancing and failover, by Netflix -- Java
  • fabio - Zero-conf load balancing HTTP(S) router for deploying microservices managed by Consul, by eBay -- Golang
  • Flink - Next-generation true stream processing platform for real-time analytics -- Java
  • HAproxy - Open source software load balancer -- C
  • haproxyctl - Wrapper to talk to the HAProxy socket, as well as regular init (start stop restart) shit -- Ruby gem
  • Hystrix - Library to stop cascading failures -- Java
  • Ignite - General-purpose in-memory platform for in-memory computing use cases -- Java
  • Istio - Envoy + auth, policy enforcement, telemetry, traffic flow management etc control plane that runs on top of Mesos and Kubernetes -- Various Lang
  • jespen - Call Me Maybe: Breaking distributed systems -- Clojure
  • Kafka - Stream processing platform (logs, IoT metrics, anything) -- Java
  • Kong - Microservice / API management layer -- Lua
  • linkerd - RPC proxy and request routing for microservices, includes service discovery, instrumentation, and load balancing -- Scala
  • LogDevice - A distributed data store for logs, by Facebook -- C++
  • Mantl - Complete microservices infrastructure built using OSS tools by Cisco -- various lang yowza
  • Marathon - Mesos scheduler for "always on" jobs (eg: web services) -- Scala
  • MaxScale - General purpose DB query proxy, router, and load balancer, by MariaDB -- C
  • mcrouter - memcached protocol router for scaling memcached, by Facebook -- C++
  • Mesos - Cluster manager and scheduler, similar to Google's Borg and Omega projects -- C++
  • mesos-consul - Automatically registers/deregisters services run as Mesos tasks -- Golang
  • Mesosphere - Apache Mesos schedulers for almost anything, commercial support is available -- C++
  • miniMesos - Spin up Mesos clusters on your OS X laptop, can be used for TDD too -- Java
  • Multibinder - Simple Ruby server that makes true zero downtime reloads simple, by Github -- Ruby
  • Myriad - Hadoop YARN framework/scheduler for Mesos -- Java
  • nsq - Realtime distributed messaging platform / message queue -- Golang
  • OpenTracing - Vendor-neutral distributed tracing -- Various Lang
  • Packetbeat - Distributed packet monitoring system that can be used for application performance management -- Golang
  • Pinpoint - Application Performance Monitoring (APM) for distributed systems, based on Dapper -- Java
  • Pulsar - Distributed pub-sub messaging platform, scales to millions of clients and queues -- Java
  • redis-cell - Redis module that provides rate limiting in Redis as a single command using GCRA -- C
  • Redisson - Distributed and scalable Java data structures on top of Redis -- Java
  • Riemann - Aggregates events from your servers and applications with a powerful stream processing language, for distributed systems, similar to Borgmon -- Clojure
  • Ringpop - Fault tolerant application-level sharding for Golang or Javascript -- JavaScript Golang
  • Seesaw - Load balancer based on Linux Virtual Server (LVS), by Google -- Golang
  • Serf - Decentralized solution for service discovery and orchestration -- Golang
  • SkyDNS - Service discovery via DNS for etcd -- Golang
  • SmartStack - Automated service discovery and registration framework, by Airbnb -- Ruby
  • Spark - Near real-time analytics processing platform, succeeded by Flink (real streaming vs Spark's microbatches) -- Various Lang
  • Spring Cloud Config - Allows Java Spring to read config info from service discovery or similar source -- Java
  • Synapse - Service discovery framework by Airbnb -- Ruby
  • uReplicator - Improved Kafka MirrorMaker by Uber -- Java
  • vulcand - Programmatically extendable proxy for microservices and API management -- Golang
  • Webjob - Downloads a program or script from a remote WebJob server, executes it, and sends back output to a server -- C
  • xCAT (Extreme Cloud Administration Toolkit) - Complete management solution for distributed systems (HPC, clouds, web farms, etc), supports almost every platform, by IBM -- various lang
  • Zipkin - Distributed tracing systems for SaaS and webapps, by Twitter, based on Google's Dapper -- Scala
  • Zookeeper - Distributed configuration service, synchronization service, and naming registry -- Java

Editors

IDEs, text & source editors, vim plugins, and similar tools

  • activate-power-mode - Activate POWER MODE and write code in style, an Atom plugin -- CoffeeScript
  • Atom - Superb text editor, created by GitHub -- CoffeeScript
  • Atom Vim Mode - vi/vim style controls for Atom -- CoffeeScript
  • Atom Linters - A collection of lint tools for the Atom editor -- various lang
  • Brackets - Modern editor that understands and focuses on web design, by Adobe -- JavaScript
  • LargeFile - vim plugin that automatically disables certain things so you can edit large (multi-gig) files faster -- vim
  • Light Table - Next-generation editor that gives you instant feedback -- Clojure
  • Lime - Open source API-compatible alternative to Sublime Text -- various lang
  • livedown - Live, in-browser markdown previews for vim and Sublime -- various lang
  • MacDown - Markdown editor and live preview for Mac -- Objective-C OS X
  • MacVim - Has far more features than the vim that's included with the OS -- C
  • Nuclide - Collection of packages for Atom to provide IDE-like functionality for a variety of programming languages and technologies, by Facebook -- JavaScript
  • Notepad++ - Killer GPL'ed text editor for Windows -- C++
  • OpenVim - In-browser vim tutorial -- JavaScript in-browser
  • percol - Interactive grep (search) tool -- Python
  • Powerline - Provides various statues on the bottom of your session, flexible -- Python
  • Textmate - GUI text editor for OS X -- C++
  • UltiSnips - The ultimate snippet solution for vim -- Python
  • vim-json - A better JSON plugin for vim -- vim
  • vim-snippets - snipMate & UltiSnip snippets -- vim
  • Vimium - Chrome extension that provides vi/vim style shortcuts for navigation and control -- CoffeeScript Chrome
  • Vim Adventures - A game to learn vim using your browser -- in-browser
  • Vundle - Popular plug-in manager for vim -- vim
  • wasavi - vim/vi controls in browser text areas -- JavaScript Firefox Chrome
  • YouCompleteMe - Fuzzy-search code completion engine for vim -- Python

Git Tools

Tools for interacting with git and GitHub

  • BFG Repo-Cleaner - Scrub large blobs and sensitive data from git history -- Scala
  • Completion - Shell tab completion for git branch names -- shell
  • Gerrit - Web based code review and repo management for Git -- Java
  • git-fat - Like git-media but without the Ruby dependencies -- Python
  • GitHub Pull Request Builder Plugin (gprbp) - Jenkins plugin that allows certain comment strings to kick off builds or take other actions -- Java
  • GitLab - Kinda like an open source GitHub, has both a community and paid version -- Ruby
  • GitLab CI - CI that integrates with GitLab -- Ruby
  • GitRob - Scan repos for sensitive information -- Ruby
  • gitsome - Supercharged CLI with GitHub integration -- Python
  • GitUp - Maybe the best git GUI -- Objective-C
  • gitwatch - Automatically commit changes when specified files or directories are modified -- shell
  • Git Interfaces, Frontends, and Tools - Massive list on the official kernel.org wiki -- various lang
  • Gogs - Painless, self-hosted Git service written in -- Golang
  • hub - Official CLI for GitHub -- Golang
  • libgit2 - GitHub's official library for interfacing bindings for Ruby, Python, Go, and others -- various lang
  • myrepos - Flexible tool for managing many repos -- perl
  • Signing - Sign commits and/or tags using GPG keys to verify the identity of the commiter -- built-in

Graphics Stuff

Not everything happens in the browser or on the command line

  • Avidemux - Simple, all-in-one, GUI video editor and converter -- C++ Windows OS X
  • Blender - 3D graphics software for animated films, visual effects, art, 3D printed models, and more -- C Python Windows OS X
  • blockdiag - Generate simple block/sequence/activity/network diagrams from text files -- Python pip
  • CloudCraft - Create professional AWS architecture diagrams -- in-browser
  • Darktable - Photography workflow application and RAW developer, Adobe Lightroom replacement -- C OS X
  • draw.io - Free online flow chart maker / Visio alternative, can be self-hosted -- in-browser
  • FreeMind - OSS mind mapping software, great for brainstorming -- Java Windows OS X
  • GlyphSearch - Search for icons from Font Awesome, Glyphicons, IcoMoon, Ionicons, and Octicons -- collection
  • GIMP (GNU Image Manipulation Program) - Open source Adobe Photoshop replacement -- C
  • Graphviz - Graph visualization and flow chart software -- wacky license
  • Greenshot - The best Windows screenshot tool -- C# Windows
  • Inkscape - Open source vector image editor, Adobe Illustrator replacement -- C C++ Windows Mac
  • LICEcap - Capture an area of your desktop and save it to a GIF -- C Windows OS X
  • mac2imgur - Auto-upload screenshots to Imgur -- Swift OS X
  • maim - The most powerful and flexible Linux desktop screenshot tool -- various lang
  • Media Player Classic - Home Cinema (MPC-HC) - Open source media player for Windows -- C++ Windows
  • OpenShot - Video editing software, 2.0 will support other additional platforms -- Python Windows OS X
  • Origami - interactive UI design prototyping without programming, by Facebook -- various lang
  • Pencil - Open source GUI prototyping and mockup tool, supports all platforms -- Windows OS X
  • Scribus - Open source desktop publishing (layout, typesetting, etc), Adobe InDesign alternative -- C++ Windows OS X
  • ShareX - One of the best screenshot/screencast capture and sharing tools for -- Windows OS X
  • Skitch - Feature-rich screenshot editing, sharing, and annotation tool Mac/OS X -- closed source
  • VisioCafe - The largest collection of free Visio stencils -- collection
  • VLC Media Player - Media (music, video, etc) player and streaming server -- C Windows OS X
  • yEd Graph Editor - Flowcharts and UML diagrams -- closed source

High Availability Clustering Tools

HA clustering tools including storage replication, failover, VIPs, and more

  • AN! Cluster Tutorial 2 - Building clusters with OSS software, very detailed -- article
  • Cluster Manager (cman) - Clustering framework that's built into RHEL/CentOS/Fedora -- built-in
  • Corosync - HA framework and cluster engine -- C
  • csync2 - General purpose cluster file sync tool -- C
  • DRBD (Distributed Replicated Block Device) - Mirror block devices to a remote system aka replication -- C
  • Ganeti - Wrappers around existing tools to make it easy to create HA clusters, by Google -- Python
  • HAproxy - Open source software load balancer -- C
  • haproxyctl - Wrapper to talk to the HAProxy socket, as well as regular init (start stop restart) shit -- Ruby
  • keepalived - Load balancing and high availability -- C
  • huptime - Zero downtime restarts of unmodified programs -- C
  • Linux-HA - Building blocks for high availability systems -- wiki-and-collection
  • LVS (Linux Virtual Server) - Linux-based load balancer, also includes the IPVS kernel module -- C
  • Multibinder - Simple Ruby server that makes true zero downtime reloads simple, by Github -- Ruby
  • Pacemaker - HA resource manager -- C
  • rcron - cron redundancy and failover, ensures a job will only run on the "active" machine -- lost-to-the-internet C
  • rmanager - Resource group manager daemon for cluster services -- built-in
  • Seesaw - Load balancer based on Linux Virtual Server (LVS), by Google -- Golang
  • UCARP - VIP management using the CARP protocol -- C

HPC Tools

High performance computing for simulations, supercomputing, shared memory systems, and grid computing - see also Distributed Systems Tools

Live CD Tools

Security, recovery, bootable USB/CD/DVD creation tools, diagnostics, and more

  • BlackArch Linux - Penetration testing Linux distro, based off of Arch Linux -- various lang
  • Buscador - Live CD Linux distro preconfigured for online investigators -- various lang
  • CAINE (Computer Aided INvestigative Environment) - Computer forensics on a live CD -- various lang
  • DBAN (Darik's Boot and Nuke) - Spinning disk wiper -- various lang
  • Easy2Boot - Create multiple bootable Linux ISOs on the same USB drive -- Windows
  • GParted Live - Small, bootable ISO that contains gparted - great for resizing a non-LVM root file system -- C
  • Hiren's BootCD - Everything. Seriously. -- closed source Windows
  • Kali Linux - Penetration testing Linux distro -- various lang
  • Network Security Toolkit (NST) - Live CD that includes most tools in insecure.org's top 125 tools list -- various lang
  • Rufus - Create bootable USB flash drives -- Windows
  • Stresslinux - Hardware burn-in and stress testing -- various lang
  • System Rescue CD - System recovery CD that focuses on Linux system recovery -- various lang
  • Ultimate Boot CD (UBCD) - Tons of x86 diagnostic and stress test tools on a single CD -- closed source Windows
  • UNetbootin - Create custom, bootable USB Linux CDs -- Windows OS X
  • YUMI - Multiboot USB creator, Linux and -- Windows

Logging

Log management, analysis, analytics, and collection from any source

  • Adiscon LogAnalyzer - Slick web interface for syslog messages -- PHP
  • Apache Log Generator - Drag and drop to create Apache logging syntax -- in-browser
  • Countly - Mobile and web analytics and marketing platform -- JavaScript
  • ElasticDump - Import / export tools for Elasticsearch -- JavaScript
  • ElasticHQ - Does not yet support ES 5.x -- JavaScript
  • Elasticsearch - Component of the ELK stack, used for super fast searches -- Java
  • Elasticsearch Exporter - Script to import/export data from ElasticSearch to various other storage systems -- JavaScript
  • Eliot - Outputs causal chains of actions happening within and across process boundaries aka a logical trace of the system’s operation -- Python pip
  • Errbit - Self-hosted error catcher, Airbrake API compliant -- Ruby
  • Filebeat - By Elastic, the next generation Logstash Forwarder -- Golang
  • Fluentd - Unified logging layer -- Ruby gem
  • Flume - Distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data -- Java
  • Graylog2 - Log capture and analysis -- various lang
  • IRCcat - Send log messages and alerts to an IRC channel, can also use IRC to send commands -- Java
  • lnav - Advanced log file viewer for the small-scale, terminal/ncurses based -- C++
  • Log Courier - Enhanced fork of Logstash Forwarder -- Ruby
  • Logplex - Distributed syslog log router, uses Redis, by Heroku -- Erlang
  • logstash - Collect, parse, and store logs. A component of the popular ELK stack -- Ruby
  • logstash-forwarder - Formerly Lumberjack, forwards logs to centralized server(s), very lightweight -- Golang
  • Logster - Utility for reading log files and generating metrics to configurable outputs by Etsy -- Python
  • Logwatch - Monitor logs and send an e-mail when event(s) occur -- perl
  • multilog - Reads a sequence of lines from stdin and appends selected lines to any number of logs -- DJB
  • multitail - Monitor multiple log files in a single terminal window -- C
  • netconsole - Kernel module that sends kernel log messages (dmesg, etc) to a remote system without using syslog -- C
  • NXLOG - Universal log collector and forwarder, supports many formats/platforms/sources including Windows -- C Windows
  • Open Web Analytics (OWA) - Track and analyze how people use your websites and applications -- PHP
  • pgBadger - PostgreSQL log analyzer -- perl
  • Piwik - Web analytics platform with a killer UI -- PHP
  • Sentry - Application exception logging -- Python
  • Snoopy Logger - Logs commands that are executed and saves the information to syslog -- C
  • Snowplow - Web, mobile and event analytics -- Scala
  • swatch - Simple log watcher -- built-in
  • Webalizer - Web and FTP log analyzer -- C

Metrics

Collection, analysis, and storage of metrics, telemetry, and instrumentation data from almost any source - see also Dashboards and Data Visualization and Monitoring

  • collectd - Collects system performance statistics -- C
  • collectd Related Sites - Great tools that integrate with collectd -- collection
  • collectl - sar on steroids -- C
  • Diamond - Python daemon that collects system metrics and publishes them to Graphite (or similar), has an API -- Python
  • dim_STAT - Collects almost everything and stores it in a MySQL database, produces reports too -- C
  • FastForward - Flexible system event and metric forwarding agent by Spotify -- Ruby gem
  • Graphite - Store numeric time-series data and render graphs of the data on demand -- Python
  • Graphite Tools - Tools that work with Graphite -- collection
  • Heka / hekad - Stream processing, can gather logs or performance metrics, by Mozilla, based on Borgmon -- Golang
  • Heroic - TSDB by Spotify -- Java
  • InfluxDB - Distributed time series database with no external dependencies -- Golang
  • KairosDB - Time series DB written on top of Cassandra -- Java
  • Node-Bell - Real-time anomalies detection system for periodic time series -- JavaScript
  • OpenTSDB - Store and serve massive amounts of time series data without losing granularity -- Java
  • Prometheus - Metrics collection and storage, can trigger alerts when thresholds are breached, based on Borgmon -- Golang
  • RRDtool - High performance data logging and graphing system for time series data -- C
  • Scales - Metrics collection, written in Python -- Python pip
  • Skyline - Real-time anomaly detection system to enable passive monitoring of metrics without the need to configure thresholds -- Python
  • SNMP MIB Archive - Massive archive of SMMP MIBs, please fork and contribute -- collection
  • sysinfo_influxdb - Collect system metrics and store them in an InfluxDB server -- Golang

Monitoring and Alerting

Monitor stuff, send alerts, wake you up - see also Metrics

  • Adagios - Web based Nagios configuration interface -- HTML
  • Alerta - Distributed and de-coupled, requires MongoDB -- Python
  • bluepill - Simple process monitoring, written in Ruby -- Ruby
  • Bosun - Monitoring and alerting system written by Stack Exchange, based on Borgmon -- Golang
  • Cabot - Self-hosted, easily-deployable monitoring and alerts service - like a lightweight PagerDuty -- Python
  • check_mk - Extensions / plugins for Nagios -- C
  • check_mk Multisite - GUI for check_mk -- various lang
  • Daemon Tools - Service monitoring and management tools -- DJB
  • FastForward (ffwd) - Flexible system event and metric forwarding agent by Spotify -- Ruby gem
  • Flapjack - Alert routing and rollup so you don't get tons of alerts for large failures -- Ruby gem
  • Fully Automated Nagios - Easy to install, pre-packaged Nagios that includes the most-used tools and plugins -- various lang
  • Ganglia - Monitoring software that's focused on HPC / distributed clusters -- various lang
  • Icinga - Nagios fork, updated frequently -- various lang
  • Icinga2 - Complete re-write of Icinga -- various lang
  • jmxtrans - Connector between speaking to a JVM via JMX on one end and whatever logging / monitoring / graphing package -- Java
  • Kale - Monitoring stack by Etsy -- article
  • LibreNMS - GPL fork of Observium -- various lang
  • Monit - Includes tools to automatically take action when certain conditions are met (eg: restart a process when it dies) -- C
  • Monitorix - So lightweight that it can be used in mobile devices, aka Mikaku -- perl
  • Naemon - Modular Nagios fork -- various lang
  • Nagios - One of the most widely used OSS monitoring programs -- various lang
  • nagios-herald - Gives context to Nagios alerts, by Etsy -- Ruby
  • Nagios Exchange - Centralized repository of Nagios plugins, addons, extensions, etc -- collection
  • OpenNMS - Network monitoring, also supports configuration/asset management -- various lang
  • Open Monitoring Distribution (OMD) - Nagios bundled with the most popular plugins and tools -- various lang
  • pmacct - IP and network traffic accounting / monitoring -- C
  • PHP Server Monitor - Simple monitoring package that can use built-in public SMS gateways for notifications -- PHP
  • Pynag - Interface with Nagios via Python -- Python pip
  • Sensu - Open source monitoring framework, cloud-focused -- Ruby
  • Sensu Plugins - Official community site for Sensu plugins -- various lang
  • Shinken - Nagios-compatible monitoring, supports high availability -- Python
  • StatsD - Monitoring in node.js by Etsy -- JavaScript
  • Tattle - Self service alerting and dashboard frontend for Graphite and Ganglia -- PHP
  • Uptime - Remote monitoring application using Node.js, MongoDB, and Twitter Bootstrap -- JavaScript
  • Xymon - Fork of Big Brother -- C
  • Zabbix - Stores monitoring data in a DB, has agents for almost every OS, can be a virtual appliance -- various lang
  • Zenoss Core - Supports Nagios plug-in format -- Python

Networking Tools

Hodge-podge of network tools

  • 2ping - Simple bi-directional ping utility, helps determine where packet loss occurs -- Python
  • Angry IP Scanner - GUI network scanner, supports plugins, -- Java Windows OS X
  • aria2 - CLI for downloading HTTP/HTTPS, FTP, SFTP, BitTorrent and Metalink -- C Windows OS X
  • arp-scan - Create and send ARP requests -- C
  • Awesome PCAP - Huge list of tools that work with PCAP captures -- collection
  • bbcp - Copies files using multiple TCP streams to greatly increase throughput -- C
  • bbFTP - Uses it's own transfer protocol, optimized for large file transfers -- C
  • BIRD Internet Routing Daemon (BIRD) - (Almost) fully functional IP routing daemon for Linux, supports tons of standard routing protocols -- C
  • BPF Tools - BSD Packet Filter (BPF) and pcap tools, by CloudFlare -- Python
  • Calico - L3 fabric that runs a vRouter on each node, supports containers -- Python
  • CERT NetSA Security Suite - Network flow analysis tools -- various lang
  • Cyberduck - GUI FTP, SFTP, WebDAV, S3 & OpenStack Swift browser for Mac and Windows -- itsaduck
  • Data Plane Development Kit (DPDK) - Data plane libraries and framework for fast packet processing -- C
  • dnscrypt-wrapper - Add dnscrypt support to any resolver, server-side dnscrypt proxy -- C
  • dsniff - Great for level 2 analysis or service spoofing -- C
  • ExaBGP - The BGP swiss army knife of networking -- Python pip
  • fail2ban - Watches log files to ban IPs based on rules (too many failed logins, exploit attempts, brute force attacks, etc) -- Python
  • Fast Data Transfer (FDT) - For writing at disk speed over WANs -- Java
  • FBOSS (FaceBook Open Switching System) - FB's software stack for managing and controlling whitebox switches -- various lang
  • FRRouting - Replacement for / fork of Quagga with more features -- C
  • ftptop - Monitor FTP connections in real time -- built-in
  • kcptun - Secure tunnel based on KCP -- Golang
  • ipcalc - CLI tool to calculate subnets, netmasks, IP ranges, broadcast addresses, and more -- built-in
  • iptstate - A top-like display of IP Tables state table entries -- built-in
  • geoiplookup - Uses the GeoIP DB and library to determine which physical country an IP or host originates in, includes PAM library -- built-in
  • GridFTP - Supports parallel streams, optimized for WANs, part of the Globus Toolkit -- C
  • hping3 - Create custom TCP/IP packets, very flexible -- built-in
  • lftp - Supports many protocols (FTPS, HTTPS, SFTP), scheduling, bandwidth throttling, scripting, and more - feature-rich -- C C++
  • lldpd - Daemon that can talk LLDP aka the open version of Cisco Discovery Protocol (CDP) -- C
  • moloch - Large scale IPv4 full PCAP capturing, indexing and database system -- JavaScript
  • mrsync (multicast remote sync) - Transfers from a master to many remote machines using Unix multicast sockets -- C
  • mTCP - High-performance user-level TCP stack for multicore systems -- C
  • Mulitpath TCP - Allows TCP to use multiple paths to maximize resource usage and increase redundancy -- neat
  • My Looking Glass (myLG) - All-in-one CLI network diagnostic tool -- Golang
  • NAPALM (Network Automation and Programmability Abstraction Layer with Multivendor support) -- Network automation and programmability abstraction layer, for both setting config and config validation -- Python pip
  • ncat - Improved netcat, written by the Nmap team -- C
  • netsniff-ng - High performance, zero-copy networking sniffer -- C
  • NfSen - GUI for and visualization nfdump -- perl
  • ngrep - Network grep -- C built-in
  • nmap - Classic port scanner -- various lang
  • nping - Create custom network packets -- C
  • OpenBGPD - Free implementation of BGP v4 -- C
  • OpenBMP - BGP Monitoring Protocol collector with real-time monitoring, looking glass, analytics, etc -- Various Lang
  • Open vSwitch (OVS) - Production quality software switch -- C
  • Outages Dashboard - Outages mailing list's dashboard, tons of resources for internet outages -- collection
  • PFQ - Framework that allows efficient packets capture/transmission, in-kernel functional processing, and packets steering across sockets/end-points -- C
  • PowerAdmin - Simple Web UI for PowerDNS -- PHP
  • Quagga - Routing software suite, supports most routing protocols -- C
  • scapy - Interactive packet manipulation for Python -- Python pip
  • socat - netcat on steroids, supports serial devices -- C
  • spiped - Create encrypted pipes between socket addresses using PSKs, similar to ssh -L -- C
  • sslh - Protocol multiplexer, let multiple daemons listen on a single port -- C
  • SubBrute - Subdomain enumeration tool for penetration testers -- Python
  • tcpflow - TCP demultiplexer, each flow is stored in it's own file -- C++
  • tcpreplay - Capture, edit, and replay network traffic -- C
  • tracepath - Traceroute that doesn't require root -- C
  • Trigger - Network automation toolkit written in Python, created by AOL -- Python
  • UDR - UDT wrapper for rsync that improves throughput of large data transfers over long distances -- C++
  • WireShark - The classic network analyzer -- C
  • WireShark Tools - Superb list of network tools from the WireShark wiki -- collection
  • Zenmap - Official GUI for Nmap -- C Windows OS X
  • ZeroTier - Cloud / provider-agnostic private network backplanes, network virtualization -- C++ all platforms

Network Performance Analysis Tools

Load generation, benchmarking, profiling, and latency simulation tools for TCP/IP networks - see also Web and HTTP Performance Analysis Tools

  • ARGUS (Audit Record Generation and Utilization System) - Generates network activity reports -- C
  • Augmented Traffic Control (ATC) - A tool to simulate network conditions, by Facebook -- Python
  • bmon - Console based network monitor -- C
  • clumsy - Simulate poor network conditions on Windows -- C Windows
  • Comcast - Simulate crappy network connections -- Golang OS X
  • dummynet - Simulates/enforces queue and bandwidth limitations, delays, packet losses, and multipath effects -- C Windows
  • ESnet Fasterdata Knowledge Base - Provides proven, operationally sound methods for troubleshooting and solving performance issues -- collection hpcwisdom
  • EtherApe - Graphical network monitor, pretty output -- various lang
  • Flent - Python wrapper to run mutliple netperf/iperf3/ping in parallel, formerly netperf-wrapper -- Python
  • Flowgrind - Distributed TCP traffic generator -- C
  • iftop - top for network interfaces -- C
  • intrace - Enumerate IP hops using existing TCP connections -- C
  • iperf3 - Supports tuning of various parameters related to timing, protocols, and buffers -- C
  • iptraf-ng - Updated fork of iptraf -- C
  • ITU-T Y.1564 - Ethernet network SLA validation tool -- various lang
  • jnettop - Traffic visualizer -- C
  • mtr (my traceroute) - Combines ping and traceroute into a single program -- C
  • Muxy - Muck with your system and application context layers 4-7 -- Golang OS X
  • namebench - Hunts down the fastest DNS servers for your computer to use -- Python Windows OS X
  • nbwmon - Ncurses bandwidth monitor -- C
  • netatop - Kernel module for atop to watch and report on network packets -- C
  • netem - Network emulator for testing variable delay, loss, duplication and re-ordering -- C
  • NetHogs - Displays per-process bandwidth usage -- C C++
  • Network Link Conditioner - Simulate bandwidth, latency, and packet loss, by Apple -- closed source OS X
  • nfdump - Captures network flows including sFlow, NetFlow, NetFlow v9, ipfix, etc -- perl
  • nicstat - vmstat for network interfaces -- C
  • nload - Console application that monitors network traffic and bandwidth usage in real time, neat ASCII graphs -- C++
  • ntopng / ntop-ng - New version of the popular ntop tool -- C
  • netperf - The web page looks ancient but the software's still being updated! -- C
  • Paris Traceroute - Shows proper network topology when load balancers and load-balanced routers are used -- C
  • Saboteur - Network fault injection tool -- Python
  • Sloppy - Proxy server that slows down connections for testing -- Java Windows OS X
  • SmokePing - Network latency visualizer, written by the MRTG and RRDtool guy -- perl
  • ss - Socket statistics, a modern netstat -- built-in
  • Stanford Linear Accelerator Center - Network Monitoring Tools - MASSIVE list of network monitoring tools -- collection
  • tc - Built-in Linux kernel traffic control -- built-in
  • tcpdive - TCP performance analysis tool, implemented as SystemTap scripts -- C
  • tcpdstat - Get protocol statistics from tcpdump / PCAP files -- C
  • tcptrack - Console based connection tracker -- built-in
  • TCPWatch - TCP monitoring and logging tool -- Python pip
  • Tracebox - Detect middleboxes (NAT, transparent proxies, etc) in a network path -- C++
  • trickle - Userspace bandwidth shaper -- built-in
  • vnStat - Console based traffic monitor, supports statistic collecting -- C
  • WANem - WAN emulator for testing response over low bandwidth and/or high latency connections, also comes as virtual appliance -- various lang
  • Yconalyzer - Monitor and analyze TCP connections -- C++

Orchestration

OS and VM level orchestration as well as parallel SSH tools

  • Ansible - CM and orchestration, also can do provisioning -- Python
  • Archipel - Use XMPP push notifications to manage, supervise, and orchestrate VMs -- Objective-J
  • Batou - Define and perform automated service deployments -- Python pip
  • Capistrano - Use Ruby to run scripts/commands and push software via SSH, uses a Rake DSL -- Ruby gem
  • ClusterSHISH - Cluster SSH for Windows, works with PuTTY and OpenSSH for Windows -- closed source Windows
  • ClusterSSH - Make a change on many servers at the same time -- perl
  • csshX - Cluster SSH for OS X -- C
  • Fabric - Uses paramiko to implement a higher-level API for performing commands over SSH, particularly for deployment sysadmin tasks -- Python pip
  • KeyBox - Web-based SSH console that executes commands on multiple shells simultaneously and supports terminal sharing -- Java
  • KeyBox-OpenShift - KeyBox for OpenShift gears Java
  • ManageIQ - Upstream OSS project for Red Hat's CloudForms -- Ruby
  • Mass Parallel SSH (mpssh) - Simple parallel SSH -- C
  • mcollective (Marionette Collective) - Orchestration, can be used independently of Puppet -- Ruby gem
  • mussh - MUltihost SSH -- C built-in
  • OpenLMI (Open Linux Management Infrastructure) - Manage, monitor, and configure servers via API calls (instead of SSH), included in RHEL 7 -- various lang
  • orgalorg - Next generation parallel SSH tool because most other major ones are no longer maintined -- Golang
  • parallel - Execute jobs in parallel using one or more computers -- built-in
  • parallel-ssh (pssh) - Parallel version of OpenSSH tools - comes with prsync, pscp, pnuke, and pslurp too -- Python built-in
  • Parallel Distributed Shell (pdsh) - Kick off many SSH sessions in parallel -- C
  • paramiko - SSH2 protocol library for Python, provides both client and server -- Python pip
  • PyDSH - Python Distributed Shell, parallel SSH -- Python
  • RadSSH - Paramiko-based parallel SSH -- Python pip
  • Rex - Framework for server orchestration and to simplify system administration -- perl
  • Rubber - Capistrano/rails plugin that makes it easy to deploy/manage/scale to various service providers -- Ruby
  • Rundeck - Job scheduler and runbook automation, enable self-service access to existing scripts and tools -- Groovy
  • Salt / Saltstack - Orchestration, server provisioning, and configuration management -- Python
  • Spacewalk - Remote commands/orchestration, patch management, and more - the upstream for Red Hat Satellite -- various lang
  • Teleport - Front-end for teams, includes session capture and replay, auditing, 2FA, session sharing, and more -- Golang
  • tmux-cssh - Cluster SSH via tmux -- shell

Package Patch and Repository Tools

Repos, RPM/APT packages, packaging guidelines, patching, and patch management

  • apt-dater - Simple ncurses frontend for package management via SSH, also supports yum -- C
  • aptly - Swiss army knife for Debian repository management, has the ability to take snapshots for easy rollback -- C
  • AutoPkg - Packaging and distribution for OS X, great for managing many laptops -- Python OS X
  • CentOS Errata for Spacewalk (CEFS) - Import errata information from CentOS-announce into Spacewalk -- useit
  • CentOS Repositories - Large list of both official and unofficial CentOS software repositories -- collection
  • CentOS Software Collections (SCL) - Use multiple versions of software on a system without disturbing the system default version -- C
  • Copr - Automatic build system providing a package repository as its output, by Fedora -- C
  • cowbuilder - Package builder that uses copy-on-write (COW) to speed up the build process -- C
  • DNF - Package management system that will replace Yum in upcoming Fedora/RHEL/CentOS releases -- article
  • ELRepo - Repo that focuses on hardware related packages, supports RHEL and CentOS -- repo
  • Extra Packages for Enterprise Linux (EPEL) - Supports CentOS, RHEL, Scientific Linux, and Oracle Linux -- repo
  • Extra Packaging Guidelines and Policies for EPEL - Packaging guidelines, great even if not creating EPEL stuff -- readit
  • Fedora Packaging Guidelines - Excellent information that can be applied to CentOS & RHEL -- readit
  • fpm (Effing Package Management) - Build packages for multiple platforms (deb, rpm, etc) with great ease and sanity -- Ruby gem
  • fpm-cookery - Tool for building software packages with FPM -- Ruby gem
  • Homebrew - Tons of packages for Mac -- Ruby OS X
  • Koji - Software that builds packages for Fedora, can be used for other stuff too, uses mock -- C
  • Mock - Build packages in a simple chroot so you don't blow up your system -- C
  • mrepo - RPM repository management tool supporting ftp/http/sftp/rsync/rhn/you, formerly Yam -- Python
  • Munki - Managed software installation for OS X, great for managing laptops -- Python OS X
  • OStree - Tool for managing bootable, immutable, versioned filesystem trees (not really a package manager but...) -- thefuture C
  • Package Repository Manager (PRM) - OS-independent package repo creation and management tool -- Ruby gem
  • pkgr - Made deb or RPM packages out of any Ruby, NodeJS, or Go app -- Ruby
  • Pulp - Next generation repository management, a component of Red Hat Satellite 6 -- Python
  • Red Hat Software Collections (SCL) - Use multiple versions of software on a system without disturbing the system default version -- repo
  • reposync - Synchronize yum repositories to a local directory -- built-in
  • rpm-ostree - Take RPM packages and commit them to an OSTree repository -- C thefuture
  • RPM Fusion - Provides software that the Fedora Project or Red Hat doesn't ship -- repo
  • schroot - Allow non-root users to create chroot environments, great for package testing -- built-in
  • Spacewalk - Patch management, remote commands, and more - the upstream for Red Hat Satellite -- various lang
  • Tito - Tool for managing RPM based projects using git for their source code repository -- Python
  • yum-presto - yum plugin that provides support for downloading package deltas -- article
  • yum-security - Plugin to only install security updates -- built-in

Performance Analysis Tools

Non-specific and all-in-one performance monitoring tools - web, network, storage, and RDBMS have their own categories

  • atop - Supports both real-time and historical performance monitoring -- C
  • bwm-ng - Small and simple console-based live network and disk I/O bandwidth monitor -- C
  • Conky - Lightweight system monitor for X windows -- C++
  • dstat - Versatile replacement for vmstat, iostat, netstat and ifstat -- Python
  • Glances - Real-time performance monitoring, written in curses and Python -- Python
  • hazelnut - Python lib to parse /proc/meminfo -- Python pip
  • htop - top replacement, has a few additional features -- C
  • Linux Performance Observability Tools - Awesome graphic that shows you which tool to use, by Brendan Gregg -- the man
  • mem_logger.sh - Monitor a processes' memory usage over time -- shell
  • Munin - Historical performance monitoring to help determine when you server became "slow" -- perl
  • NetData - Real time performance visualization and dashboards -- C
  • nmon - Provides both real-time and historical performance metrics -- C
  • NumaTOP - top for NUMA systems, shows hotspots, call chains, etc -- C
  • OProfile - System-wide statistical profiling tool -- C
  • perf - User-land performance analysis tool, a sampling profiler -- C
  • perf-tools - Uses perf and ftrace, includes iosnoop -- Brendan Gregg
  • Performance Co-Pilot (PCP) - System performance and analysis framework, included in RHEL 7 -- C
  • Phoronix Test Suite - Benchmarking and profiling suite, very feature-rich and versatile -- PHP
  • pidstat - vmstat type output for CPU, disk I/O, page faults, and more on a per-process basis -- built-in
  • pmap - Shell scripts for tracking memory usage using "pmap" -- shell
  • PowerTOP - Real-time power consumption on a per-process & per-thread basis, by Intel -- C++
  • ps_mem - Accurately reports core memory usage for a process -- Python
  • ptop - top/ntop-like task monitor written in Python -- Python pip
  • recap - Collects info from various standard utilities (free, sar, vmstat, etc) at specified intervals, by Rackspace -- various lang
  • saidar - ncurses based program for viewing system statistics -- built-in
  • slabtop - Tutorial on how to use slabtop, useful when you have no idea on how your RAM is being used -- article
  • smem - Reports memory usage based on proportional set size (PSS) instead of the usual resident set size (RSS) -- C
  • sysdig - Linux system exploration and troubleshooting tool with first class support for containers -- C++
  • SystemTap (stap) - Linux kernel tracing and performance analysis tool -- C C++
  • USE Method - Brendan Gregg's rosetta stone for performance analysis and tuning -- article

Provisioning Tools

OS provisioning, image creation, installation, bootstrapping, and lifecycle management

  • Box-Cutter - Hashicorp's community repo for Packer & Vagrant templates -- Ruby
  • Chef Baseboxes - Baseboxes provided by Chef, for Packer + Vagrant -- various lang
  • cloud-init - Configures settings the first time a system spins up (SSH keys, hostname, variables, etc), note NoCloud -- C
  • Cobbler - Provisioning system used in Red Hat's Satellite -- Python
  • Crowbar - Multi-purpose provisioning tool, created by Dell -- Ruby
  • edeploy - Tool to provision and update systems using trees of files instead of packages or VM images -- Python
  • Foreman - Provisioning and life cycle management -- Ruby Windows OS X
  • iPXE - GPL'ed version of PXE, official replacement for gPXE -- C
  • Katello - Candlepin + Pulp + Foreman, the core of Red Hat Satellite 6 -- various lang
  • Kickstart - The classic Red Hat tool -- various lang
  • Knife - See the Chef section for knife provisioning plugins -- Ruby gem
  • Metal as a Service (MaaS) - Ubuntu provisioning system -- various lang
  • netboot.xyz - Simple iPXE menu and installer -- shell
  • Overcast - VM orchestration and provisioning for DigitalOcean, Linode and Amazon, or locally using Vagrant + Virtualbox -- various lang
  • Packer - Automates VM creation for multiple platforms (VMware, AWS, etc) -- Golang
  • Pallet - Aims to solve the problem of providing a consistently configured running image across a range of clouds -- Clojure
  • Razor - Puppet provisioning software that handles bare metal hardware and virtual server provisioning -- Ruby
  • Terraform - Infrastructure provisioning using existing tools, supports many providers (AWS, Azure, DO, OpenStack, etc) -- Golang Hashicorp
  • Vagrant - Quickly spin up environments for local testing and development -- Ruby
  • Vagrant Plugins - A list of Vagrant plugins on the official Hashicorp wiki -- collection
  • Vagrant Manager - GUI to manage Vagrant boxes -- Windows OS X
  • vagrant-cachier - Cache BLOB downloads to reduce network usage -- Ruby
  • vagrant-host-shell - Simple plugin to run commands on the VM when it boots (think cloud-init) -- Ruby
  • vagrant-hostsupdater - Plugin to add your own /etc/hosts to the VM -- Ruby
  • vagrant-vbguest - Auto-install the latest VirtualBox tools at boot time (if necessary) -- Ruby
  • veewee - Easily (and repeatedly) build custom Vagrant base boxes, KVMs, and VM images -- Ruby
  • veewee-to-packer - translate veewee templates to Packer templates -- Ruby gem
  • Salt / Saltstack - Orchestration, server provisioning, and configuration management -- Python

Python Tools and Resources

Python stuff of note

  • 2.7 and 3.x Differences - The differences between 2.7 and 3.x with examples, via HN -- article
  • Awesome Python - Very large list of Python resources -- collection
  • bpython - Killer interface for the Python interpreter -- Python
  • Common Python Mistakes - Link and discussion on HN -- article
  • Easy-Python - Collection of Python resources that "you didn't know you would need" -- collection
  • Faker - Generate fake data easily -- Python pip
  • Google Python Style Guide - Google's Python style guide -- article
  • IPython - Interactive Python shell and the kernel for Jupyter -- Python
  • Jinja2 - Templating language for Python -- Python pip
  • Jupyter - The language-agnostic parts of IPython -- Python
  • Micro Python - Python for microcontrollers, embedded, and small memory systems -- Python
  • PyPI (Python Package Index) - Software repo for Python packages, like Ruby gems or a RPM repo -- collection
  • Python Reference - A collection of useful scripts, tutorials, and other Python-related things -- collection
  • PyPy - Python alternative with advance features (JIT compiles, sandboxing, etc) -- Python
  • python-prompt-toolkit - Library for building interactive command lines -- Python pip
  • Requests - The best HTTP library -- Python pip
  • sh - Library that allows you to call any program (shell command) as if it were a function -- Python pip
  • virtualenv - Tool to create isolated Python environments -- Python pip
  • virtualenvwrapper - Extensions to virtualenv -- Python pip
  • xlwings - Replace your Excel VBA code with Python -- Python pip

Python Programming Tutorials

Tutorials, exercises, and challenges for learning Python. I have no idea how good/bad these are

RDBMS and SQL Tools

Tools for interacting with and related to the major DBs and SQL

  • Adminer - GUI for database management in a single PHP, formerally phpMyAdmin -- PHP
  • DB Browser for SQLite - GUI database browser for SQLite instances -- C++
  • Flyway - Version control for DB schemas, supports most DBs -- Java
  • HeidiSQL - GUI SQL DB browser and editor -- Windows OS X
  • Large Hardon Migrator (LHM) - Live, online MySQL migrations -- Ruby
  • Liquibase - Tracking, managing, and applying database schema changes, SVN for DBs -- Java
  • MaxScale - General purpose DB query proxy, router, and load balancer by MariaDB -- C
  • mycli - CLI for MySQL and derivates with auto-completion and syntax highlighting -- Python
  • mydumper (MySQL Data Dumper) - Much better than mysqldump, works in parallel -- perl
  • MySQL sys schema - Collection of views, functions and procedures to help MySQL administrators get insight into MySQL usage -- SQL
  • MySQL Utilities - Official collection of MySQL tools -- various lang Python
  • MySQL Workbench - The official MySQL GUI for admins, devs, DBAs, and architects -- various lang
  • Oracle TPT Scripts - Tab completion for Oracle -- SQL
  • orchestrator - MySQL replication topology management and visualization tool, GUI -- Golang
  • Peloton - Self tuning, self optimizing DB -- C++
  • Percona Toolkit for MySQL - Percona's special toolkit -- various lang
  • pgcli - Postgres CLI with autocomplete and syntax highlighting -- Python
  • pgloader - Fast data loader for PostgreSQL -- Lisp
  • pgweb - Web-based PostgreSQL DB browser -- Golang
  • pg_repack - Remove bloat from tables and indexes without using an exclusive lock -- C
  • Phinx - Database migrations in SQL or PHP -- PHP
  • Postgres-XL - Scale-out version of PostgresSQL -- C
  • Postgres.app - All-in-one version of Postgres for local testing on a laptop -- C
  • PostgREST - Create a REST API for any Postgres DB -- Haskell
  • PostgreSQL GUI Tools - A huge list on the official wiki -- collection
  • Presto - Distributed SQL query engine for big data, by Facebook -- Java
  • rdbms-subsetter - Generate a random sample of rows from a relational database that preserves referential integrity, great for using production data for testing/dev -- Python
  • rlwrap - Add history and tab completion to SQLplus -- C
  • SchemaSpy - Analyze the metadata of a schema and visualize it in HTML -- Java
  • Sequel Pro - MySQL management GUI for Mac -- OS X
  • SQLite - Self-contained, serverless, zero-configuration, transactional SQL database engine, great for testing -- C
  • sqlmap - Detect and exploit SQL injection flaws, pen testing tool -- Python
  • SQL Fiddle - Write and test SQL -- in-browser
  • wal-e - Simple continuous archiving for PostgreSQL -- Python
  • WebScaleSQL - Huge scale MySQL with contributions from Facebook, Google, LinkedIn, and Twitter -- C C++
  • WWW SQL Designer - Designing RDBMS schemas features saving, exporting to XML, and SQL script creation, free SaaS version here -- JavaScript

RDBMS Performance Analysis Tools

Load generation, benchmarking, telemetry, and profiling for various RDBMS platforms

  • Anemometer - MySQL slow query log monitor and analyzer, by Box -- PHP JavaScript
  • db_STRESS - Database benchmarking and stress-testing kit, supports most DBs -- C
  • gh-ost - Online schema migrations for MySQL, by GitHub -- Golang
  • HammerDB - Load testing and benchmark tool, supports most DBs -- Tcl
  • innotop - "top" for MySQL -- NOTE: beware of exposing your password -- perl
  • MySQLtuner-perl - Analyzes a MySQL installation and provides suggestions to increase performance -- perl
  • MySQL Performance Analyzer - Open sourced by Yahoo -- Java
  • mytop - Another "top" for MySQL -- perl
  • Open PostgreSQL Monitoring (OPM) - Includes a web console, nice GUI -- JavaScript
  • Percona Monitoring Plugins - Some of the best mySQL plugins -- various lang
  • Percona Toolkit for MySQL - Percona's internal tools -- various lang
  • pg_activity - htop for PostgreSQL -- Python
  • pg_view - PostgreSQL monitoring, supports ncurses, console, and JSON output -- Python pip
  • pgbench-tools - PostgreSQL benchmarking suite -- various lang
  • PGObserver - Killer PostgreSQL monitor, includes dashboard -- Python
  • shift - Schema migrations for MySQL, by Square -- Ruby
  • Silly Little Oracle Benchmark 2 (SLOB2) - Stresses and benchmarks Oracle DBs, works at the RDBMS layer -- various lang
  • shawn-s - Superb Reddit post on performance evaluation and tuning -- article
  • Swingbench - Oracle load generator -- closed source
  • SysBench - Evaluates OS parameters that are important for DBs, does not require a DB to be installed -- C
  • Tsung - Distributed stress tester, also supports stress testing DBs -- Erlang
  • Use the Index, Luke - SQL indexing and tuning for developers -- article

Regular Expressions

regex checkers, creators, evaluators, debuggers, and tutorials

  • Debuggex - Online regex debugger -- in-browser
  • Learn regex the Hard Way - The hard way -- tutorial
  • perlretut - perl-focused but very useful for all regex -- article
  • Refiddle - Online, supports JavaScript, Ruby, and .NET only -- in-browser
  • regex101 - Online regex tester and debugger, supports multiple languages -- in-browser
  • RegexOne - Learn regular expressions with simple, interactive examples -- tutorial
  • regexper - Regex visualizer using railroad diagrams, great for debugging -- in-browser
  • RegExr - Another online regex tool that includes cheat sheets, examples, and community-contributed expressions -- collection
  • RegularExpressions.info - THE BEST regex site -- collection
  • Regular Expressions - A Gentle User Guide and Tutorial - The easy way -- tutorial
  • RexV.2 - Online Regex evaluator, supports multiple languages -- in-browser
  • Rubular - Online Ruby-only regex evaluator -- in-browser
  • txt2re - Regex generator, recommended -- in-browser
  • txt2regex - Converts human sentences to regex, written in bash -- shell

Ruby Tools and Resources

Ruby toolbox full of junk

  • Awesome Ruby - A collection of awesome Ruby libraries, tools, frameworks and software -- collection
  • Bundler - Manage Gem dependencies using simple config files -- Ruby gem
  • did_you_mean - Suggestions for typos in Ruby code -- Ruby
  • fog - Library for communicating with clouds -- Ruby gem
  • Omniref - A better search site for Ruby documentation -- collection
  • Pry - IRB alternative, very feature-rich, great for debugging -- Ruby gem
  • Rake - Make / Makefile-like build utility -- Ruby gem
  • rbenv - Alternative to RVM -- shell
  • replicate - Dump and load relational objects between Ruby environments (eg: prod and dev) -- Ruby gem
  • RSpec - Behavior-driven development (BDD) testing framework -- Ruby
  • RuboCop - Static code analyzer that enforces guidelines -- Ruby gem
  • Ruby Gems - The Ruby package manager, gems are essentially libraries -- Ruby gems
  • Ruby Toolbox - Trillions of Ruby projects, categorized -- collection
  • RVM (Ruby Version Manager) - Install, manage, and work with multiple Ruby environments -- Ruby

Ruby Programming Tutorials

Tutorials, exercises, and challenges for learning Ruby. I have no idea how good/bad these are

Secret Management

Password, private key, and API key storage and management tools

  • blackbox - Safely store secrets in Git, by Stack Exchange -- shell
  • Diceware - Neat IRL passphrase generator -- doitirl
  • GPG (GNU Privacy Guard) - The original! Store em in text files and encrypt -- C pro
  • GRC Password Generator - Generates 63 bit passwords -- in-browser
  • Keybase - Upload your public key and find other people's public key via their social media user name(s) -- in-browser
  • KeeFox - Auto-fill website logins using KeePass as a source, like LastPass -- C# Firefox
  • KeePass - Lightweight, easy to use, GUI password manager, runs on most platforms -- C# Windows
  • Keepass2Android - Compatible with KeePass 2.x -- Android
  • KeePassDroid - Open source version of KeePass 1.x for Android -- Android
  • KeePassXC - Supports YubiKey and TOTP, forked becasue KeePassX is no longer being updated -- C++
  • KeeWeb - Unofficial web and desktop front-ends to KeePass -- JavaScript
  • Keywhiz - A system for distributing and managing secrets (API keys, certificates, etc), by Square -- Java
  • kpcli - Command line interface / CLI for KeePass -- perl
  • lastpass-cli - CLI for LastPass -- C
  • msktutil - Keytab client for a Microsoft Active Directory environment -- built-in
  • One-Time Secret - Create links that self-destruct after a single viewing, great for sharing passwords -- Ruby
  • pass (passwordstore) - Uses GPG at it's core, supports tracking password changes in git -- shell
  • passff - Addon for interacting with the a pass aka passwordstore repository -- Firefox
  • Password Gorilla - Written in -- Tcl/Tk
  • Password Pusher - Links to passwords expire after a certain number of views and/or time has passed, RoR app -- Ruby
  • Password Safe - Designed by Bruce Schneier himself -- bam
  • pwd.sh - Script to manage passwords in an encrypted file using GPG -- shell
  • pwgen - Password generated, included in most Linux distros -- built-in
  • RatticDB - Web-based password management, includes an API, great for teams or companies -- Python
  • scrypt - Super secure against hardware brute-force attacks, is also a part of/module for most programming languages, by tarsnap -- C
  • Secrets OPerationS (SOPS) - Secret management, by Mozilla -- Python
  • TeamPass - Collaborative password management using the LAMP stack -- PHP
  • Vault - Tool for storing secrets (API keys, passwords, certs, etc) by Hashicorp -- Golang
  • webpasswordsafe - Very powerful, designed "for the enterprise" -- Java
  • xca - OSS all in one PKI -- C++
  • zxcvbn - Password strength estimator, written by Dropbox, operates in a browser window -- CoffeeScript in-browser

Security Tools

Misc security tools

  • AIDE (Advanced Intrusion Detection Environment) - File integrity checker, alternative to Tripwire -- various lang
  • Armitage - GUI for Metasploit -- Java
  • Awesome Honeypots - Curated collection of honepots and honeypot resources -- collection
  • bro - Framework for network analysis and security monitoring -- C++
  • chkrootkit - Rootkit checker, best used from a live CD -- C
  • CIS Linux Benchmarks - Linux OS hardening guides, superb! -- collection
  • ConfigServer Security and Firewall (CSF) - Stateful Packet Inspection (SPI) firewall, Login/Intrusion Detection and Security application for Linux servers -- various lang
  • Cryptkeeper - System tray applet that manages EncFS encrypted folders -- C
  • dcfldd - Enhanced dd with security and forensics features -- C
  • Dshell - Network forensic analysis framework, written by the US Army -- Python
  • EncFS - Encrypted file system in user space via FUSE -- C++
  • Forensics Wiki - Digital forensics wiki, tons of tools and information -- wiki
  • FTimes - System baselining and evidence collection tool -- C
  • fwknop (FireWall KNock OPerator) - Single Packet Authorization (SPA) aka next-generation port knocking -- various lang
  • hashcat - World's fastest CPU password cracker / password recovery -- C
  • Information Security Cheat Sheets and Checklists - An assortment of IS checklists and cheat sheets -- collection
  • Jailkit - chroot toolkit -- C
  • Lynis - Auditing and hardening tool, supports most Unix-like operating systems -- shell
  • Metasploit - Classic exploit framework -- various lang
  • nmap Scripting Engine (NSE) - Tons of scripts for nmap -- various lang
  • oclHashcat - World's fastest CPU + GPU password cracker / recovery software -- C
  • OpenSCAP - NIST Certified SCAP 1.2 toolkit -- C
  • OpenVAS - Vulnerability scanner, forked from the now closed-source Nessus scanner -- C
  • Open Web Application Security Project (OWASP) - Focus on web application security -- organization
  • Oracle Database Attacking Tool (ODAT) - Open source penetration testing tool that tests the security of Oracle databases remotely -- Python
  • OS X Auditor - Forensics tool for Mac -- OS X JavaScript
  • OSSEC - Host based intrusion detection system (HIDS), supports most Unix-like OSes -- C Windows OS X
  • pfsense - The best firewall software, supports appliances and live CDs -- C
  • Prey - Open source anti-theft software for almost all platforms -- shell Java mobile
  • Red October - Go server for two-man rule style file encryption and decryption -- way cool
  • Rootkit Hunter - Compares hashes of important files with known good hashes that are stored in online databases -- perl shell
  • Samhain - HIDS, file integrity checker, rootkit detection, log file monitoring, and more -- C
  • ScoutBot - Automated network scanner for Raspberry Pi -- various lang
  • Security Onion - Linux distro for IDS, NSM, and log management -- various lang
  • Security Technical Implementation Guide (STIG) - Guides to securing almost every application, by the US military -- collection
  • Snort - The classic network intrusion prevention system (NIPS) -- C
  • Spiderfoot - Automate the process of gathering intelligence about a given target -- Python
  • sudosh2 - Records all keystrokes and output and can play back the session as just like a VCR -- C
  • tink - Smiple, small, secure crypto library by Google -- C++
  • Tripwire - File integrity checker and monitor, replacement for the now closed-source Tripwire -- C
  • URLquery - Free service to scan a URL for web-based malware -- in-browser
  • usbkill - Kill switch that takes action when a USB device is connected -- Python
  • VirusTotal - Allows you to upload a file and have it scanned by tons of virus scanners -- in-browser
  • ZMap - Scanner designed for large address spaces -- in-browser

Shell Scripting and Tools

Shell scripting tutorials and neat shell tools

  • Advanced Bash-Scripting Guide - By the Linux Documentation Project (LDP) -- tutorial
  • Awesome dotfiles - All kinds of . files -- collection
  • autojump - Small database of directories that you visited in the past, used to quickly navigate complex directory structures -- Python
  • Awesome Shell - Massive collection of shell tools -- collection
  • Babun - Pre-configured Cygwin with many more features and a better design -- various lang
  • BashGuide - Targeted at beginners -- wiki
  • bashrc PS1 Generator - Drag and drop to create a custom PS1 prompt -- in-browser
  • Bash Hackers Wiki - Human-readable bash documentation so you don't have to dig through the man page -- wiki
  • Bash Pitfalls - Common errors that bash programmers make -- wiki
  • bro pages - Simplifed man pages and examples -- collection
  • cheat - create and view interactive cheatsheets on the command-line -- Python
  • comm - Display lines that two files have in common (eg: the opposite of diff) -- built-in
  • CommandlineFu - Killer code snippets -- collection
  • CRUSH (Custom Reporting Utilities for SHell) - Killer toolset for working on delimited data, by Google -- C
  • Cygwin - GNU shell and tools for Windows -- C Windows
  • dotfiles by Paul Miller - Beautiful and flexible Mac terminal configuration files and utilities -- shell OS X hawt
  • dotfiles - Unofficial guide to dotfiles on GitHub -- collection
  • Environment Modules - Dynamic modification of your shell environment using modules -- Tcl Windows
  • Explain Shell - Enter a command-line to see the help text that matches each argument -- in-browser
  • fu - CLI for CommandlineFu.com, allows for easy searches -- Python
  • fzf - Command line fuzzy finder, supports tmux/bash/zsh -- Golang
  • Grymoire awk Tutorial - Easy to understand awk tutorial -- tutorial
  • gt5 - A "du browser" in curses, handy for finding large files -- shell
  • icdiff - diff tool that highlights the differences -- Python OS X
  • maybe - Allows a dry run of almost any Linux binary -- Python pip
  • moreutils - "Collection of the unix tools that nobody thought to write long ago when unix was young" -- neat
  • notify - Send a SMS to your phone from the CLI -- JavaScript Shell
  • pigz - Parallel gzip for multi-processor/core systems -- C
  • pv - Shows the progress of data as it flows through a pipe -- built-in
  • ShellCheck - Checks shell scripts for common mistakes, essentially a linter -- in-browser
  • sysadmin-util - A collection of small but useful utilities -- perl
  • tldr - Simplified and community-driven man pages, cuts out a lot of cruft -- collection
  • The Art of Command Line - Master the command line, in one page -- collection
  • Unix Toolbox - A collection of Unix/Linux/BSD commands and tasks for advanced users -- collection

Software Development Tools

Build systems, stubs/mocks, CI/CD, cheat sheets, and other

  • ack2 - grep-like tool designed to search source code -- perl
  • afl-fuzz (American Fuzzy Lop) - One of the best fuzzers -- C
  • ag (aka The Silver Searcher) - Source code searching tool, a better grep -- C
  • Bazel - Google's build system -- Java
  • Bento - Links to tutorials based on subject (programming, sysadmin, protocols, standards, etc) -- collection
  • Buildbot - CI framework -- Python
  • CRIU (Checkpoint-Restore in Userspace) - Freeze a process, save it to disk, then resume it later -- insane
  • cwrap - Wrappers for creating test scenarios and faking behavior/ stubbing, mostly network focused, by the Samba guys -- C
  • deployinator - Etsy's deployment framework -- Ruby gem
  • Design Patterns - Wonderful collection of design patterns with examples -- collection
  • DevDocs - Documentation browser for almost every API -- collection
  • dev-setup - Automated setup scripts for tools like Sublime Text, AWS, Spark, Android dev, and more -- collection
  • dropwizard - Simple library for building production-ready RESTful web services -- various lang
  • Fossil - Simple all-in-one SCM -- various lang
  • fswatch - Cross-platform for watching files and taking action when they change -- C++
  • Go (GoCD) - Go continuous delivery platform by ThoughtWorks -- Java
  • Google's Technical Development Guide - Hands on, free, self-paced courses to help you develop your programming skills -- collection
  • Guard - Flexible framework to take action on file system change event -- Ruby gem
  • Gulp - Built system / toolkit that helps you automate time-consuming tasks in your development workflow -- JavaScript
  • Jenkins - The most popular CI orchestration tool, supports a billion plugins -- various lang
  • jenkins-job-dsl (Jenkins Job DSL Plugin) - Groovy-based DSL for writing Jenkins jobs -- Groovy
  • Jenkins job-config-history Plugin - Tracks changes to system and job configurations -- Java
  • JSON Server - Full fake REST API for quickly prototyping and mocking in 30 seconds -- JavaScript
  • LearnXinYMinutes - Super fast tutorials for almost every programming language -- collection
  • Mastif Visualizer - Visualizer for the Valgrind's Mastiff utility -- C++
  • Microsoft Cloud Design Patterns - Ignore the cloud and Azure stuff and it's a great reference for general software dev -- collection
  • MockServer - Web server to remotely or locally mock HTTP/HTTPS and similar -- Java
  • mountebank - Stub downstream resources for testing, supports HTTP HTTPS SMTP TCP -- JavaScript Windows OS X
  • Ninja - Small build system with a focus on speed -- Python
  • OverAPI - Large collection of cheat sheets for almost anything -- collection
  • PatchELF - Simple utility for modifying existing ELF executables and libraries -- C
  • PRoot - chroot, mount --bind, and binfmt_misc without privilege/setup -- C
  • Proxygen - Modern C++ HTTP library, by Facebook -- C++
  • REST-assured - Java DSL for testing of REST services -- Java
  • RocksDB - Library that provides an embeddable, persistent key-value store for fast storage - by Facebook -- C++
  • rpm-maven-plugin - Package artifacts as a RPM -- Java
  • Software Engineering Blogs - A curated list of software engineering blogs -- collection
  • SonarQube - Platform and dashboard for managing code quality -- Ruby Java
  • Sonatype Nexus - Software / binary artifact storage -- Java
  • Spinnaker - Cloud-focused CI/CD system for immutable infrastructure, by Netflix -- Python
  • SymbolHound - Search engine that doesn't ignore special characters, great for programming questions -- collection
  • ThinBackup - Jenkins plugin that backups configurations (not workspaces or archives) -- Java
  • Valgrind - Tool suite that includes cache profilers, heap profiles, thread race condition checkers, and more - a CPU-level emulator -- C
  • watchman - Watch files and take action when they change (eg: kick off the CI system), by Facebook -- C
  • WireMock - Flexible stubbing and mocking services -- Java

SSH Tools

Misc SSH stuff - see also Terminal Tools and SSH Clients for clients and Orchestration for parallel SSH tools

  • Advanced SSH Config - Enhances ssh_config file capabilities -- Python pip
  • autossh - Automatically restart SSH sessions that stop passing traffic -- C
  • Ballast (Balancing Load Across Systems) - Balance user load across SSH servers based on configured policies -- C
  • Corkscrew - Tunnel SSH through HTTP proxies -- C
  • Keychain - Manage SSH and GPG keys, acts as a frontend to ssh-agent, only enter passphrase once per reboot -- shell
  • Match - Creates a conditional block, great for controlling actions on a per-user and/or per-host basis in sshd_config -- built-in
  • Mosh (Mobile Shell) - Remote shell that supports roaming (client IP address changes) and intermittent connectivity, by MIT -- C++
  • rssh - Restricted shell for SSH, only allows scp, sftp, rdist, rsync, and cvs -- C
  • scponly - Shell that only allows incoming SCP, supports chroot -- C
  • ssh-chat Instead of a shell you get a chat prompt -- Golang
  • ssh-ldap-helper - Store public keys in LDAP -- built-in
  • SSHFS - Mount remote file systems using a SSH tunnel -- built-in
  • sshmuxd - SSH jumphost style proxy -- Golang
  • sshrc - Sources the ~/.sshrc on your local computer after logging in remotely -- shell
  • SSH Guard - Think fail2ban for SSH -- C
  • SSH Power Tool (sshpt) - Execute commands and upload files to many servers simultaneously via SSH without using pre-shared keys -- Python pip
  • storm - CLI and GUI tool to manage your SSH connections (add, delete, list, search) -- OS X

SSL Tools

SSL, TLS, CAs, and similar tools for interacting with HTTPS and SSL certificates - see also Web and HTTP Tools

  • BadSSL.com - Test various clients (browsers, etc) against bad SSL configs -- in-browser
  • BoringSSL - Google's fork of OpenSSL, does not guarantee API and ABI compatibility -- C
  • Calomel SSL Validation - Score and analyze the strength and settings of a SSL connection -- Firefox
  • cipherscan - Find out which SSL ciphersuites are supported by a target -- Python
  • Dogtag Certificate System - PKI component of FreeIPA, by Fedora -- C
  • duraconf - A collection of hardened configuration files for SSL/TLS services by Jacob Appelbaum -- collection
  • etcd-ca - Simple CA by the CoreOS guys -- Golang
  • HSTS (HTTP Strict Transport Secuirity) - Forces browsers to interact with a site by only using HTTPS -- article
  • Lemur - Broker between CAs and environments providing a central portal for developers to issue TLS certificates with 'sane' defaults -- Python
  • Let's Encrypt - Free SSL certs from a real CA -- in-browser
  • LibreSSL - OpenBSD's fork of OpenSSL, Linux compatible -- C
  • Mozilla Server Side TLS - Mozilla's extensive server side TLS configuration guide -- wiki
  • Mozilla SSL Configuration Generator - Generate SSL configs for Apache, Nginx, ELB, HAproxy and more -- in-browser
  • nogotofail - Spot and fix weak TLS/SSL connections and sensitive cleartext traffic, by Google -- Python
  • Qualys SSL Server Test - Evaluates and provides recommendations for the SSL settings of any web site -- in-browser
  • s2n - Amazon's implementation of the TLS/SSL protocols (simple, small, fast, secure) -- C
  • Server Name Indication (SNI) - Think vhosts for SSL -- article
  • sslconfig - CloudFlare's Internet facing SSL cipher configuration -- collection
  • SSLsplit - Transparent and scalable SSL/TLS interception -- C
  • sslyze - Fast and full-featured SSL scanner, written in Python -- Python OS X
  • SSL Checker - Type in a domain name and it will dump info about its cert -- in-browser
  • SSL Decoder - Decodes a SSL connection and displays certificate information, CLI -- PHP
  • SSL Decoder - Same as above but web based -- in-browser
  • stunnel - Create TLS tunnels for existing services (eg: telnet, nc, etc) -- C

Storage Tools

The storage junk drawer

  • ATA Secure Erase - Send a signal to an ATA drive to perform a hardware-based erase, the ONLY way to wipe a SSD properly -- article
  • BeeGFS - Parallel cluster file system, worth examining -- C++
  • Clonezilla - Disk image/cloning tool, supports most file system types -- perl shell
  • CockroachDB - Open source version of Google's Spanner storage system -- thefuture Golang
  • CrystalDiskInfo - S.M.A.R.T. GUI tool for Windows -- closed source Windows
  • EnhanceIO - SSD caching, in the mainline Linux kernel, a fork of Facebook's Flashcache project -- C
  • Fog - Cloning tool -- C++
  • FreeNAS - BSD-based NAS, supports ZFS -- C
  • fs-cache - Modern NFS client-side caching -- built-in
  • Linux-IO Target (LIO) - Create and share iSCSI, FC, FCoE, and other storage targets on Linux VMs -- built-in
  • Linux Journal - Linux Swap Space - Superb article about Linux swap, includes some tuning parameters -- article
  • lsblk - List block devices -- built-in
  • lvcache - Simple wrapper to help you create and manage LVM caches -- Python
  • ncdu - ncurses version of "du" -- built-in
  • OpenFiler - Linux-based NAS, supports most protocols and storage types -- C
  • Parted Magic - Resize, grow, shrink, clone, recovery, wiping, benchmarking, and more. Supports Linux and Windows file systems -- Windows
  • scrub - Supports many disk-wiping standards including military / government grade wipes -- built-in
  • snapper - CLI tool to manage Btrfs snapshots, snapshot timelines, and more -- various lang
  • System Storage Manager (SSM) - Generic CLI for managing all types of storage (DM, LVM, multipath), added in RHEL 7 -- built-in
  • Tomb - zsh wrapper script for cryptsetup + gpg + LUKS volumes -- shell
  • VeraCrypt - OSS successor to and fork of TrueCrypt, supports Linux, Windows, and Mac -- C C++

Storage Performance Analysis Tools

  • blktrace - mount -t debugfs debugfs /sys/kernel/debug && blktrace -d /dev/md2 -o - | blkparse -i - -- article
  • Bonnie++ - The classic, still updated -- built-in
  • Connectathon Test Suite - NFS stress testing and benchmarking tools -- various lang
  • DPACK (Dell Performance Analysis Collection Kit) - Load generation, reporting, profiling -- closed source
  • findswappers - perl script that identifies swapped processes -- perl
  • fio - Supports 19 different I/O engines (sync, mmap, libaio, posixaio, etc), very powerful -- C
  • Fnotifystat - Dumps the file system activity in a given period of time -- C
  • IObench - Generic I/O stress tool that's a part of dim_STAT -- C
  • iometer - Multi-platform, by Intel -- C++
  • ioping - Monitor I/O latency in real time -- C
  • iorate - Originally written by EMC, now open source -- C
  • iotop - top for I/O requests, displays information on a per-process basis -- Python
  • IOzone - Supports NFS, still being updated! -- C
  • Threaded I/O Tester (tiobench) - Threaded I/O tester -- C

Terminal Tools and SSH Clients

Fancy SSH clients, terminal sharing, and similar tools

  • Bitvise SSH Client - Feature-rich SSH & SFTP client for Windows, free for individual use -- closed source Windows
  • Byobu - An enhanced version of the "screen" utility -- shell OS X
  • ConnectBot - Open source SSH client for Android -- Java Android
  • Guacamole - Clientless (in-browser) remote desktop gateway, supports VNC and RDP -- various lang
  • iTerm2 - Killer terminal replacement for Mac -- OS X -- Objective-C
  • KiTTY - PuTTY fork with additional features -- Windows
  • MobaXterm - Tabbed SSH, VNC, and RDP client for Windows, free for personal use -- closed source Windows
  • mRemoteNG - Open source, tabbed, multi-protocol, remote connections manager -- Windows
  • MTPuTTY (Multi-Tabbed PuTTY) - Multiple PuTTY sessions in a single window -- Windows
  • NoVNC - Client-less VNC in a web browser, uses HTML5 and WebSockets -- in-browser
  • PuTTY - The classic SSH client -- Windows
  • PuTTYtray - PuTTY in the systray + additional features -- C
  • reptyr - Reparent a running program to a new terminal -- C
  • SuperPutty - Allows the PuTTY SSH client to be opened in tabs -- Windows
  • tmate - Instant terminal sharing, a tmux fork -- OS X
  • tmux - Alternative to GNU screen, also used for terminal sharing -- C
  • tmux-cssh - Cluster SSH for tmux -- shell
  • tmux Resurrect - Persists tmux environment across system restarts -- shell
  • WinSCP - The well-known SCP client -- C Windows
  • x2go - One of the best remote-desktop-over-SSH clients -- Windows OS X
  • Xshell - SSH client for Windows, free for personal use -- closed source Windows

Tracing and Debugging

OS and process-level tracing and debugging tools - see also Performance Analysis Tools for tools suited to performance profiling

  • bcc - Next generation, Linux 4.x kernel tracing tool suite, uses eBPF (Extended Berkeley Packet Filters) -- C
  • [CRIU](Checkpoint/Restore in Userspace)](http://criu.org/Main_Page) - Freeze a process, save it to disk, then resume it later -- insane
  • kdump - Linux kernel dump facility (where to save it, what to save, etc) -- C
  • ktap - Another lightweight kernel tracing tool for Linux -- C
  • Linux Trace Toolkit - Next Generation (LLTng) - Linux kernel tracer and profiler, lower overhead than System Tap -- C
  • ltrace - Library call tracer -- built-in
  • ltrace Tutorial - ltrace tutorial by Red Hat -- article
  • lttng-analyses - Great front-end to LLTng -- various lang
  • OpenSnoop - Continually monitor for file opens -- shell
  • pstore - Save kernel crash info in a platform-specific persistent memory so data is not lost (eg: disks failed so write dump to NVRAM instead) -- article
  • Record and Replay (rr) - Record the failure once, then debug the recording deterministically, by Mozilla -- C
  • strace - System call tracer for user space processes -- built-in
  • SystemTap (stap) - Linux kernel tracing and performance analysis tool -- C C++

Two Factor Authentication

Universal 2nd Factor (U2F) is the successor to two-factor authentication / TFA / 2FA and multi-factor authentication / MFA

  • Two Factor Auth (2FA) - Web sites that do and don't support 2FA, organized by category -- checkit collection
  • Authy SSH - Easy 2FA for SSH servers -- shell
  • CentOS 7 2FA - VPN with CentOS 7 + FreeRADIUS + SSSD + Google Authenticator -- tutorial
  • FreeOTP - Open source fork of Google Authenticator, by Red Hat -- C mobile
  • Google Authenticator - Official project, supports Android, iOS, and even has a PAM module -- C
  • Nitrokey - Open source thumb drive for authentication -- various lang
  • pam-u2f - PAM module for auth via U2F compatible devices, by YubiKey -- C
  • yubico-pam - PAM module for use with YubiKey devices -- C

Virtualization

OS-level virtualization and emulation

  • DOSbox - Open source DOS emulator, great for running old games or utilities -- C C++
  • KVM (Kernel Virtual Machine) - The one, the only -- C
  • kvm-tools - CLI tools for managing qemu-kvm domains -- C
  • KVM Management Tools - Great list of KVM management tools on the KVM wiki, updated frequently -- collection
  • libvirt - Open source API, daemon and management tool, used with many virtualization solutions -- C
  • QEMU (Quick EMUlator) - Also supports hardware emulation (SPARC, RISC, etc) -- C
  • Oracle VM VirtualBox - Easy, simple virtualization -- C C++ Windows OS X
  • oVirt - Virtualization management platform, the upstream for Red Hat Enterprise Virtualization (RHEV) -- Java
  • phpVirtualBox - Web front-end for VirtualBox -- PHP
  • virt-manager - KVM / Xen / LXC GUI -- Python
  • WINE - Compatibility layer for running Windows apps on POSIX-compliant OSes -- C OS X
  • Xen - It's a hypervisor! -- C
  • XenServer - Successor to Xen Cloud Platform (XCP) -- C
  • xhyve - Lightweight OS X virtualization based on bhyve -- C OS X

VMware Tools

Mostly unofficial VMware tools and some PowerCLI scripts

  • Compliance Checker for vSphere - Provides detailed compliance checks against vSphere hardening guidelines, official tool -- closed source
  • Flings - Tons of a little apps written by VMware engineers, worth exploring -- closed source? various lang
  • Onyx - Do something in vSphere and it will create a PowerCLI script to do that thing that you just did -- closed source PowerShell Windows
  • open-vm-tools - OSS alternative to VMware Guest Tools -- C
  • RVtools - Killer GUI for viewing information about your VMware environment -- closed source .NET Windows
  • RVtools Export - Wrapper script to save daily snapshots of your environment using RVtools -- PowerShell
  • Sexigraph - Graphite-based visualization appliance for Sexilog -- various lang
  • Sexilog - ELK stack virtual appliance designed for vSphere / VMware ESXi logs -- various lang
  • vCheck - Get an overview of a new environment or check the health of an existing one -- PowerShell
  • vDisk Informer - Check vDisks alignment and see if they have wasted space -- closed source Windows
  • vGhetto Script Repository - Various scripts from virtuallyGhetto -- various lang
  • VMTurbo Virtual Health Monitor - Monitoring and reporting tool -- closed source
  • VMware Community PowerPack - A variety of scripts from Virtu-al.net and now other blogs such as ict-freak.nl and ntpro.nl -- PowerShell
  • VMware on Github - VMware's Github repos -- various lang
  • VMware Sample Exchange - Code samples & scripts by both VMware and the community -- various lang
  • vSphere Health Check Report - Reports a massive amount of information, by VMware -- closed source perl
  • vSphere Mobile Watchlist - Monitor VMs on your phone -- closed source mobile

VPN Tools

Mostly server-side tools

  • IKEv2-setup - Simple script that sets up a IKEv2 VPN with strongSwan on Ubuntu -- shell
  • n2n - L2 over L3 VPN that uses a peer-to-peer architecture -- C
  • OpenVPN - The one and only -- C
  • Pritunl - Distributed enterprise VPN server built using the OpenVPN protocol, supports Google sign-in -- Python
  • SoftEther - Perhaps the best VPN software out there -- C
  • sshuttle - Transparent proxy server that works as a poor man's VPN, forwards over SSH, supports DNS tunneling -- Python OS X
  • strongSwan - IPsec-based -- C
  • tinc - Simple, multi-platform VPN -- C
  • WireGuard - In-kernel VPN server that's simple, performant, and provides next generation cryptography -- C

Web and HTTP Tools

RPC is dead, long live RPC - see also SSL Tools

  • API Blueprint - Supports bindings/plugins for many APIs -- various lang
  • Awesome JSON - It's the new XML -- collection
  • Awesome Nginx - A collection of resources covering Nginx, Nginx + Lua, OpenResty and Tengine -- collection
  • curlish - Improved curl, includes cookie management and OAuth 2.0 support -- Python pip
  • Fiddler - HTTP/HTTPS debugging proxy for any browser, system or platform -- closed source
  • Geocities-izer - Make any HTML web page look like a Geocities page -- in-browser sweeeeeeet
  • gor - Capture and replay HTTP traffic for testing -- Golang
  • gRPC - High performance RPC via HTTP/2, by Google -- C
  • h2c - HTTP/2 CLI client, curl/wget type behavior and other advanced features -- Golang
  • h2i - Go's interactive HTTP/2 console debugger, send raw frames, etc -- Golang
  • Hamms - Simulate/create connection failures, malformed response data, slow servers, fat headers, and more! -- Python
  • Hipache - Distributed HTTP and websocket proxy, supports load balancing -- JavaScript
  • htaccess Snippits - Huge collection of common and useful .htaccess snippets, please contribute -- collection
  • http-traceroute - Shows the entire route including cookies, redirects, and response codes -- Ruby
  • httpdiff - Perform the same reuqest against two HTTP servers and diff the results -- Golang
  • httpie - curl replacement with many new features -- Python
  • htty - A console application for interacting with web servers -- Ruby gem
  • Huginn - Build agents that perform automated tasks/workflows for you online or locally, like IFTTT -- Ruby
  • JSON Server - Full fake REST API for quickly prototyping and mocking in 30 seconds -- JavaScript
  • jq - Command line JSON processor and manipulator -- C
  • Lighthouse - Chrome extension to audit a web page for performance, accessibility, progressive web apps, and more, by Google -- in-browser
  • localtunnel - Share a HTTP/web service on your local development machine without messing with DNS and firewall settings -- JavaScript Windows OS X
  • mitmproxy - Intercept, modify, replay and save HTTP/S traffic - even edit flows on the fly -- Python pip
  • mountebank - Stub downstream resources for testing, supports HTTP HTTPS SMTP TCP -- JavaScript Windows OS X
  • Newman - CLI companion for Postman -- JavaScript
  • ng-admin - Add an AngularJS admin GUI to any RESTful API -- JavaScript
  • nghttp - CLI HTTP/2 client, similar to curl/wget and more -- C
  • ngrok - Reverse proxy that creates a secure tunnel from a public endpoint to a locally running web service and captures & analyzes all traffic over the tunnel for later inspection and replay -- rocks Golang
  • OpenResty - Turn nginx into a non-blocking API server -- C
  • OWASP Zed Attack Proxy (ZAP) - Pen testing too that focues on web applications -- Java
  • PageKite - Makes HTTP servers or SSH publicly available on any server -- Python
  • POSTman - Create and share API and HTTP requests, great for testing and sharing -- Chrome
  • Pushpin - Proxy server that adds WebSockets to existing request-response APIs -- C++
  • RapidJSON - Super fast JSON parser and generator -- C++ Windows
  • Redirect Detective - See the complete path a redirected URL goes through -- in-browser
  • Repose - REST proxy, solutions to API tasks such as auth, rate limiting, API validation, HTTP logging, and much more -- Java
  • Resty - CLI REST client you can use in shell/bash/zsh pipes -- shell
  • SecurityHeaders.io - Scan your web site's HTTP headers -- in-browser
  • Swagger - A JSON format/standard that describes a RESTful API -- various lang
  • tengine - A distribution of Nginx with some advanced features, by Alibaba -- C
  • tortilla - Easily wrap web APIs -- Python pip
  • Tyk - API gateway -- Golang
  • urlscan.io - Displays tons of stats and info about any given URL -- in-browser
  • Varnish - Caching HTTP accelerator -- C
  • vcr - Record and play back HTTP sessions -- Ruby gem
  • w3af - Web application attack and audit framework, OSS vun scanner -- Python
  • webhook - Super simple webhook server -- Golang

Web and HTTP Performance Analysis Tools

Load generation, benchmarking, and profiling for SaaS, webapps, and HTTP(S)

  • Awesome Web Performance Optimization - Collection of web performance optimization (WPO) tools, articles, and more -- collection
  • apachetop - top for Apache -- perl
  • Betwixt - Web debugging proxy with a Chrome DevTools look -- JavaScript
  • boom - HTTP load generator written in -- Golang
  • django-debug-toolbar - Panels that display profiling information about the current request/response -- Python pip
  • django-speedbar - Shows a breakdown of page loading time -- Python pip
  • Chrome DevTools - Many built-in tools for performance analysis -- mostly OSS
  • Gatling - HTTP, JMS, and WebSocket load generator -- Scala
  • GoAccess - Real-time web log analyzer and interactive viewer that runs in a terminal and/or dashboard -- Windows OS X
  • Firefox Developer Tools - A full list of built-in Firefox developer tools including performance tools -- various lang
  • flask-debugtoolbar - The Django Debug Toolbar ported to Flask -- Python
  • FunkLoad - Functional and load web tester -- Python pip
  • h2load - HTTP/2 and SPDY load generation tool, part of the nghttp2 suite -- C
  • HATop - ncurses top for HAProxy -- Python
  • httperf - HTTP load generator, originally created, by HP -- C
  • httping - Simple program that "pings" a URL and shows response time -- C
  • JChav - Visualize JMeter output -- Java
  • JMeter - Designed to load test functional behavior and measure performance, written in Java -- Java
  • Locust - Load generation tool written in Python that allows you to define user behavior -- Python
  • ngxtop - Real time top for nginx -- Python
  • PageSpeed Insights - Analyzes the content of a web page, then generates suggestions to make that page faster, by Google -- in-browser
  • PageSpeed Module - Open-source server modules that optimize your site automatically (nginx and Apache), by Google -- various lang
  • peep - Heap inspector for live memcached instances -- Ruby
  • phantomas - JS web performance metrics collector and monitoring tool -- JavaScript
  • redis-faina - Query analyzer that parses Redis' MONITOR command for counter/timing stats about query patterns, by Facebook -- Python
  • Siege - CLI based HTTP load generation tool, supports basic authentication, cookies, and HTTPS -- C
  • Speed Tracer - Helps you identify and fix performance problems in your web applications, by Google -- Chrome
  • Tsung - Distributed stress tester, also supports stress testing DBs -- Erlang
  • vegeta - HTTP load testing tool, written in Go -- Golang
  • Web Page Test - Free website speed test from multiple locations around the globe using real browsers (IE and Chrome) and at real consumer connection speeds -- in-browser
  • Web Performance Today - About an article a week, good stuff -- article
  • wrk - Multi-threaded CLI-based HTTP load generation tool -- C
  • wrk2 - Fork of wrk that fixes the "coordinated omissions problem" -- C
  • YSlow - Analyzes web pages and why they're slow based on Yahoo!'s rules for high performance web sites, customizable -- JavaScript
  • wtop - Another top for Apache logs -- Python pip
  • Varnish Dashboard - Realtime dashboard for Varnish cache servers -- JavaScript

Misc Tools and Utilities

Everything else

  • Awesome - Top level list of awesome lists -- collection
  • AutoHotKey - Automate input (desktop and web forms, data entry, keybinds, etc) -- C++ Windows
  • Awesome AutoHotKey - A collection of AutoHotKey scripts and libraries -- various lang
  • conserver - Virtual console server with many features (multi-user, console log history, etc) -- C
  • f.lux - Changes your screen from blue light to yellow light when the sun sets to tell your brain it's night time -- closed source OS X Windows
  • Google Advanced Operators for Web Search - Search modifiers for more accurate results, by Google -- collection
  • googler - Google search, news, and site search from the terminal, slick -- Python
  • inxi - "a full featured system information script" (hardware info, etc) -- shell
  • Mjolnir - Automation for OS X, think AutoHotKey for Mac -- C OS X
  • Open19 - Open data center hardware for standard size racks -- neat
  • OpenBMC - Framework to build a complete Linux image for a Board Management Controller (BMC) -- C
  • Quick Look Plugins - Collection of plugins for Quick Look in the Mac Finder -- various lang OS X
  • runit - UNIX init scheme with service supervision, great for containers, sysvinit replacement -- C
  • runwhen - Utilities for running commands at particular times, cron on steroids -- C
  • Site Reliability Engineering (SRE) by Google - Full, free, online version of the O'Reilly book -- the best
  • supercron - Cron on steroids -- Python pip
  • supervisord - Process control, kinda like an init replacement but not really -- Python
  • Unix Rosetta Stone - The one and only resource for a mixed Linux/Unix environment -- translator