/xg2xg

by ex-googlers, for ex-googlers - a lookup table of similar tech & services

A handy lookup table of similar technology and services to help ex-googlers survive the real world :) pull-requests very welcomed. Please do not list any confidential projects!

For a working example of (some) of these technologies integrated together, see: https://github.com/google/startup-os

See also: System Design Primer, The Hadoop Ecosystem Table, Map AWS services to GCP products

Technology

Core Technology

Google Internal Google External Open Source / Real-World
MapReduce Apache Hadoop, Spark
Protocol Buffer Protobuf Cap'n Proto, Thrift, Avro, Amazon Ion
Stubby gRPC Thrift, Bolt
Chubby Apache Zookeeper, etcd, HashiCorp Consul
Goops / PubSub Apache Kafka, Apache Pulsar, Facebook LogDevice

Infrastructure

Google Internal Google External Open Source / Real-World
Borg Kubernetes, Apache Mesos, HashiCorp Nomad
GSLB, GFE, Maglev ELB, HAProxy, Istio, F5, envoy
data center hardware open compute
Jupiter, Starblaze
B4, Stargate, TE
USPS, Andromeda

Storage

Google Internal Google External Open Source / Real-World
GFS/Colossus HDFS, Ceph, GlusterFS
BigTable Cloud BigTable PrestoDB, Cassandra, HBase, Accumulo, DynamoDB
Spanner Cloud Spanner Vitess, CockroachDB, TiDB
ColumnIO / Capacitor Apache Parquet
sstable levelDB RocksDB
zippy Snappy lz4

Services

Google Internal Google External Open Source SaaS
Dremel BigQuery Apache Drill, Presto, Spark(sort-of), AWS Athena, Redshift Spectrum
Dremel UI Redash, Metabase, Apache Superset
Search (Mustang, Alexandria) Elasticsearch, Solr, Lucene algolia
pubsub pubsub NATS.io, RabbitMQ, PubNub AWS SQS/SNS, AWS AppSync
Flume (Java) Apache Beam Apache Crunch
MillWheel Cloud Dataflow Apache Flink

DevOps

Google Internal Google External Real-World
Assimilator Dominator
Blaze Bazel Buck, Pants, please.build, Blade, recc
Oncall PagerDuty, OpsGenie, VictorOps
varz/borgmon/monarch Datadog, Prometheus, librato, newrelic, skylight, scout, Scotty/tricorder, also this and this
Exception/Error Tracking (??) Sentry.io, Raygun.io, Rollbar, Honeybadger, Airbrake, OverOps
styleguides google styleguides PEP-8, HoundCI auto-style-reviewer
Blaze/Forge/TAP/BuildCop Cloud Build buildkite, CircleCI, travis, jenkins, gitlabCI
Sandman(test env)/Guitar
Sisyphus / Rapid Spinnaker, lambdaCD, screwdriver.cd, CodeShip, shipit-engine, GoCD, AWS CodeDeploy, Capistrano, Fabric, ConcourseCI
borg / borgcfg / gcl AWS Cloudformation, Puppet, Chef, Salt, Ansible, Terraform, Jsonnet, kubecfg
logging StackDriver logstash, fluentd, papertrail, cernan
CodeSearch Zoekt Sourcegraph, OpenGrok
Critique, Gerrit, Mondrian etc. Gerrit Reviewable , Phabricator
cider Eclipse Che, Cloud9, gitpod.io, Coder, Code-Server (VSCode in a Tab)
buganizer JIRA, bugzilla, github issues
ToTT Google Test Blog
Copybara / MOE Copybara, MOE
workflow/dependency management luigi, airflow, digdag, packyderm, dask
ErrorProne ErrorProne SpotBugs, FindBugs

Security

Google Internal Google External Open Source
prodaccess/LOAS Keymaster
prod secrets/identity management chamber, knox, SPIFFE

IT / Operations

Google Internal Real-World
software engineering at google
valentine 1Password, Lastpass, pass
OWNERS files in repo github CODEOWNERS
snippets Khan/snippets
memegen memegen
SnipIt recordit, CloudApp, dropbox screenshots
edge, people ops training LifeLabs
googlegeist Culture Amp, humu, tinypulse, peakon
HRIS/ERP Namely, BambooHR, Workday
stuff (SaaS IT management) intello, zylo
stuff (Device Management) Fleetsmith, jamf, rippling IT
device security monitoring Red Canary
go/ links golinks, go, Goat
sharing 1-time secrets onetimesecret
google3 philosophy innersource, monorepo

also check out xoogler.co, which organizes events, slack channels etc

disclaimer: I'm not affiliated with any of the technologies mentioned above.

disclaimer: I left Google a number of years ago so some of the naming might be dated