/xg2xg

by ex-googlers, for ex-googlers - a lookup table of similar tech & services

A handy lookup table of similar technology and services to help ex-googlers survive the real world :) pull-requests very welcomed. Please do not list any confidential projects!

For a working example of (some) of these technologies integrated together, see: https://github.com/google/startup-os

See also: System Design Primer, The Hadoop Ecosystem Table, Map AWS services to GCP products

Technology

Core Technology

Google Internal Google External Open Source / Real-World
MapReduce Apache Hadoop, Spark
Protocol Buffer Protobuf Cap'n Proto, Thrift, Avro, Amazon Ion, CBOR
Stubby gRPC Thrift, Bolt
Chubby Apache Zookeeper, etcd, HashiCorp Consul
Goops / PubSub Apache Kafka, Apache Pulsar, Facebook LogDevice

Infrastructure

Google Internal Google External Open Source / Real-World
Borg Kubernetes Apache Mesos, Apache Aurora, HashiCorp Nomad
GSLB, GFE, Maglev, uberproxy ELB, HAProxy, Istio, F5, envoy
uberproxy (sso proxy) buzzfeed-sso
data center hardware open compute
Jupiter, Starblaze
B4, Stargate, TE
USPS, Andromeda

Storage

Google Internal Google External Open Source / Real-World
GFS/Colossus HDFS, Ceph, GlusterFS, MooseFS
BigTable Cloud BigTable PrestoDB, Cassandra, HBase, Accumulo, DynamoDB, ScyllaDB
Spanner Cloud Spanner Vitess, CockroachDB, TiDB
ColumnIO / Capacitor Apache Parquet
sstable levelDB RocksDB
zippy Snappy lz4

Services

Google Internal Google External Open Source SaaS
Dremel BigQuery Apache Drill, Presto, Spark(sort-of), AWS Athena, Redshift Spectrum, Snowflake
Dremel UI Redash, Metabase, Apache Superset
Search (Mustang, Alexandria) Elasticsearch, Solr, Lucene algolia
pubsub pubsub NATS.io, RabbitMQ, PubNub AWS SQS/SNS, AWS AppSync
Flume (Java) Apache Beam Apache Crunch
MillWheel Cloud Dataflow Apache Flink
Colab Colaboratory Jupyter
PLX Google Data Studio Mode

DevOps

Google Internal Google External Real-World
Assimilator Dominator
Blaze Bazel Buck, Pants, please.build, Blade, recc
Oncall PagerDuty, OpsGenie, VictorOps
varz/borgmon/monarch Datadog, Prometheus, M3, librato, newrelic, skylight, scout, Scotty/tricorder, netdata, also this and this
Exception/Error Tracking (??) Sentry.io, Raygun.io, Rollbar, Honeybadger, Airbrake, OverOps
styleguides google styleguides PEP-8, HoundCI auto-style-reviewer
Blaze/Forge/TAP/BuildCop Cloud Build buildkite, CircleCI, travis, jenkins, gitlabCI, github, Drone
Sandman(test env)/Guitar
Sisyphus / Rapid Spinnaker, lambdaCD, screwdriver.cd, CodeShip, shipit-engine, GoCD, AWS CodeDeploy, Capistrano, Fabric, ConcourseCI, samson
borg / borgcfg / gcl AWS Cloudformation, Puppet, Chef, Salt, Ansible, Terraform, Jsonnet, kubecfg, pulumi
logging, analog StackDriver logstash, fluentd, papertrail, cernan
CodeSearch Zoekt Sourcegraph, OpenGrok
Critique, Gerrit, Mondrian etc. Gerrit Reviewable , Phabricator
cider Eclipse Che, Cloud9, gitpod.io, Coder, Code-Server (VSCode in a Tab)
buganizer JIRA, bugzilla, github issues
ToTT Google Test Blog Increment
Copybara / MOE Copybara, MOE
workflow/dependency management luigi, airflow, digdag, packyderm, dask
ErrorProne ErrorProne SpotBugs, FindBugs
dapper (distributed trace) stackdriver trace zipkin, opentracing, jaeger

Security

Google Internal Google External Open Source
prodaccess/LOAS Keymaster
prod secrets/identity management chamber, knox, SPIFFE

IT / Operations

Google Internal Real-World
software engineering at google
valentine 1Password, Lastpass, pass
OWNERS files in repo github CODEOWNERS
snippets Khan/snippets
memegen memegen
SnipIt recordit, CloudApp, dropbox screenshots
edge, people ops training LifeLabs
googlegeist Culture Amp, humu, tinypulse, peakon
HRIS/ERP Namely, BambooHR, Workday
stuff (SaaS IT management) productiv, intello, zylo
stuff (Device Management) Fleetsmith, jamf, rippling IT
device security monitoring Red Canary
beyondcorp beyondcorp
go/ links golinks, go, Goat, trotto
google3 philosophy innersource, monorepo
doing code review code review
safely sharing 1-time secrets croc, onetimesecret, privatebin
messaging mattermost, slack, gchat
peer bonus bonus.ly, zestful
kudos heytaco, slack ++ bot

also check out xoogler.co, which organizes events, slack channels etc

disclaimer: I'm not affiliated with any of the technologies mentioned above.

disclaimer: I left Google a number of years ago so some of the naming might be dated