/xg2xg

by ex-googlers, for ex-googlers - a lookup table of similar tech & services

A handy lookup table of similar technology and services to help ex-googlers survive the real world :) pull-requests are very welcomed. Please do not list any confidential projects!

For a working example of (some) of these technologies integrated together, see: https://github.com/google/startup-os

See also: System Design Primer, The Hadoop Ecosystem Table, Map AWS services to GCP products, Database of Databases, Google Open Source Glossary

Technology

Core Technology

Google Internal Google External Open Source / Real-World
MapReduce Apache Hadoop, Spark
Flume DataFlow Apache Beam
Protocol Buffer Protobuf, FlatBuffers Cap'n Proto, Thrift, Avro, Amazon Ion, CBOR, kryo
Stubby gRPC Thrift, Bolt
Chubby Apache Zookeeper, etcd, HashiCorp Consul
Goops / PubSub Cloud Pub/Sub Apache Kafka, Apache Pulsar, Facebook LogDevice
//base abseil

Infrastructure

Google Internal Google External Open Source / Real-World
Borg Kubernetes Apache Mesos, Apache Aurora, HashiCorp Nomad
GSLB Cloud Load Balancing - Internal AWS ELB, Istio, envoy, linkerd
OnePlatform API Gateway Swagger
GFE, Maglev, uberproxy Cloud Load Balancing - HTTPS / External AWS ALB, HAProxy, nginx, F5
uberproxy (sso proxy) Identity-Aware Proxy buzzfeed-sso, Pomerium
Zanzibar Zanzibar Research Paper SpiceDB/authzed, Ory Keto, topaz
data center hardware open compute
Jupiter, Starblaze
B4, Stargate, TE
USPS, Andromeda
ESDN Faucet
broccoli man

Storage

Google Internal Google External Open Source / Real-World
GFS/Colossus HDFS, Ceph, GlusterFS, MooseFS
BigTable Cloud BigTable PrestoDB, Cassandra, HBase, Accumulo, DynamoDB, ScyllaDB
Spanner Cloud Spanner Vitess, CockroachDB, TiDB
ColumnIO / Capacitor Apache Parquet, ORC
sstable levelDB RocksDB, pebble
zippy Snappy lz4
RecordIO Riegeli, TFRecords, & in OR-Tools, szl stuffed-record-stream

Services

Google Internal Google External Open Source SaaS
Dremel BigQuery Apache Drill, Presto, Spark(sort-of), AWS Athena, Redshift Spectrum, Snowflake
Dremel UI Redash, Metabase, Apache Superset
Search (Mustang, Alexandria) Elasticsearch, OpenSearch, Solr, Lucene algolia
pubsub pubsub NATS.io, RabbitMQ, PubNub AWS SQS/SNS, AWS AppSync
MillWheel Cloud Dataflow Apache Flink, beam
Colab Colaboratory Jupyter Observable
PLX Google Data Studio Mode
Monarch paper
Napa paper

DevOps

Google Internal Google External Real-World
Assimilator Dominator
Blaze Bazel Buck, Pants, please.build, Blade, recc, BuildBuddy, flare.build
Oncall PagerDuty, OpsGenie, VictorOps
varz/borgmon/monarch Datadog, Prometheus, M3, librato, newrelic, skylight, scout, Scotty/tricorder, netdata, bosun, also this and this
Viceroy Grafana
Exception/Error Tracking (thirdeye) Sentry.io, Raygun.io, Rollbar, Honeybadger, Airbrake, OverOps, ELK stack
styleguides google styleguides PEP-8, HoundCI auto-style-reviewer
Sponge EngFlow
Blaze/Forge/TAP/BuildCop Cloud Build buildkite, CircleCI, travis, jenkins, gitlabCI, github, Drone
Forge/ObjFS EngFlow, flare.build
Sandman(test env)/Guitar
Sisyphus / Rapid Spinnaker, lambdaCD, screwdriver.cd, CodeShip, shipit-engine, GoCD, AWS CodeDeploy, Capistrano, Fabric, ConcourseCI, samson
MPM Docker
borg / borgcfg / gcl Jsonnet, Cue AWS Cloudformation, Puppet, Chef, Salt, Ansible, Terraform, kubecfg, pulumi, Nix
logging, analog StackDriver logstash, fluentd, PaperTrail, cernan, loki
CodeSearch, Grimoire Zoekt kythe Sourcegraph, OpenGrok, livegrep
Critique, Gerrit, Mondrian etc. Gerrit Reviewable , Phabricator
cider Eclipse Che, Cloud9, gitpod.io, Coder, Code-Server (VSCode in a Tab)
buganizer Google Issue Tracker JIRA, bugzilla, github issues, Linear
Bugjuggler SnoozeThis
ToTT Google Test Blog Increment
Copybara / MOE Copybara, MOE
workflow/dependency management Luigi, Airflow, digdag, Pachyderm, Dask
ErrorProne ErrorProne SpotBugs, FindBugs
Dapper stackdriver trace zipkin, opentracing, jaeger, LightStep, Honeycomb
C++ Tips of the Week Abseil C++ Tips of the Week
DiRT ChaosMonkey, aws fis
Rosie microplane, silver-platter
API Improvements Proposals AIP
g4 {fix, submit} Trunk.io
probers cloudprober

Security

Google Internal Google External Open Source
prodaccess/LOAS Keymaster
prod secrets/identity management chamber, knox, SPIFFE

IT / Operations / Misc

Google Internal Real-World
software engineering at google 1, 2 Software Engineering at Google: Lessons Learned from Programming Over Time
SRE @ google
valentine Vault, 1Password, Lastpass, pass
OWNERS files in repo github CODEOWNERS
snippets Khan/snippets
SnipIt recordit, CloudApp, dropbox screenshots, Snippyly
stuff (SaaS IT management) productiv, intello, zylo
stuff (Device Management) Fleetsmith, jamf
device security monitoring Red Canary
beyondcorp beyondcorp
go/ links golinks, go, Goat, trotto, go-shorten
google3 philosophy innersource, monorepo
doing code review code review
safely sharing 1-time secrets sendsecure.ly, croc, onetimesecret, privatebin
messaging mattermost, Slack, gchat
tech talks TechTalks @ Google
g3doc writethedocs, docs-as-code
GUTS spoke, freshservice
MOMA glean

PeopleOps / Culture

Google Internal Real-World
OKR CultureAmp OKR, Lattice Goals, Ally, workboard
HRIS/ERP Namely, BambooHR, Workday, Paylocity
peer bonus bonus.ly
kudos heytaco, slack ++ bot
perks fond
talks Talks @ Google
dory slido
edge, people ops training LifeLabs
googlegeist Culture Amp, humu, tinypulse, peakon
Meng Search Inside Yourself
Lazlo Work Rules
Claire Stapleton Tech Support - existential advice for the modern tech worker
books about google How Google Works, In The Plex

also check out xoogler.co, which organizes events, slack channels etc

disclaimer: I'm not affiliated with any of the technologies/products mentioned above.

disclaimer: I left Google a number of years ago so some of the naming might be dated.