A handy lookup table of similar technology and services to help ex-googlers survive the real world :)
pull-requests very welcomed. Please do not list any confidential projects!
For a working example of (some) of these technologies integrated together, see:
https://github.com/google/startup-os
See also: System Design Primer, The Hadoop Ecosystem Table, Map AWS services to GCP products, Database of Databases, Google Open Source Glossary
| Google Internal |
Google External |
Open Source / Real-World |
| MapReduce |
|
Apache Hadoop, Spark |
| Flume |
DataFlow |
Apache Beam |
| Protocol Buffer |
Protobuf |
Cap'n Proto, Thrift, Avro, Amazon Ion, CBOR, kryo |
| Stubby |
gRPC |
Thrift, Bolt |
| Chubby |
|
Apache Zookeeper, etcd, HashiCorp Consul |
| Goops / PubSub |
|
Apache Kafka, Apache Pulsar, Facebook LogDevice |
//base |
|
abseil |
| Google Internal |
Google External |
Open Source / Real-World |
| Borg |
Kubernetes |
Apache Mesos, Apache Aurora, HashiCorp Nomad |
| GSLB |
Cloud Load Balancing - Internal |
AWS ELB, Istio, envoy, linkerd |
| OnePlatform |
API Gateway |
Swagger |
| GFE, Maglev, uberproxy |
Cloud Load Balancing - HTTPS / External |
AWS ALB, HAProxy, nginx, F5 |
| uberproxy (sso proxy) |
Identity-Aware Proxy |
buzzfeed-sso, Pomerium |
| Zanzibar |
Zanzibar Research Paper |
Ory Keto |
| data center hardware |
open compute |
|
| Jupiter, Starblaze |
|
|
| B4, Stargate, TE |
|
|
| USPS, Andromeda |
|
|
| ESDN |
|
Faucet |
| broccoli man |
|
|
| Google Internal |
Google External |
Open Source / Real-World |
| GFS/Colossus |
|
HDFS, Ceph, GlusterFS, MooseFS |
| BigTable |
Cloud BigTable |
PrestoDB, Cassandra, HBase, Accumulo, DynamoDB, ScyllaDB |
| Spanner |
Cloud Spanner |
Vitess, CockroachDB, TiDB |
| ColumnIO / Capacitor |
|
Apache Parquet, ORC |
| sstable |
levelDB |
RocksDB, pebble |
| zippy |
Snappy |
lz4 |
| RecordIO |
Riegeli, TFRecords, & in OR-Tools, szl |
stuffed-record-stream |
| Google Internal |
Google External |
Open Source |
SaaS |
| Dremel |
BigQuery |
Apache Drill, Presto, Spark(sort-of), |
AWS Athena, Redshift Spectrum, Snowflake |
| Dremel UI |
|
Redash, Metabase, Apache Superset |
|
| Search (Mustang, Alexandria) |
|
Elasticsearch, OpenSearch, Solr, Lucene |
algolia |
| pubsub |
pubsub |
NATS.io, RabbitMQ, PubNub |
AWS SQS/SNS, AWS AppSync |
| MillWheel |
Cloud Dataflow |
Apache Flink, beam |
|
| Colab |
Colaboratory |
Jupyter |
Observable |
| PLX |
Google Data Studio |
|
Mode |
| Monarch |
paper |
|
|
| Napa |
paper |
|
|
| Google Internal |
Google External |
Real-World |
| Assimilator |
|
Dominator |
| Blaze |
Bazel |
Buck, Pants, please.build, Blade, recc, BuildBuddy, flare.build |
| Oncall |
|
PagerDuty, OpsGenie, VictorOps |
| varz/borgmon/monarch |
|
Datadog, Prometheus, M3, librato, newrelic, skylight, scout, Scotty/tricorder, netdata, bosun, also this and this |
| Viceroy |
|
Grafana |
| Exception/Error Tracking (thirdeye) |
|
Sentry.io, Raygun.io, Rollbar, Honeybadger, Airbrake, OverOps, ELK stack |
| styleguides |
google styleguides |
PEP-8, HoundCI auto-style-reviewer |
| Sponge |
|
EngFlow |
| Blaze/Forge/TAP/BuildCop |
Cloud Build |
buildkite, CircleCI, travis, jenkins, gitlabCI, github, Drone |
| Forge/ObjFS |
|
EngFlow, flare.build |
| Sandman(test env)/Guitar |
|
|
| Sisyphus / Rapid |
|
Spinnaker, lambdaCD, screwdriver.cd, CodeShip, shipit-engine, GoCD, AWS CodeDeploy, Capistrano, Fabric, ConcourseCI, samson |
| MPM |
|
Docker |
| borg / borgcfg / gcl |
Jsonnet, Cue |
AWS Cloudformation, Puppet, Chef, Salt, Ansible, Terraform, kubecfg, pulumi |
| logging, analog |
StackDriver |
logstash, fluentd, PaperTrail, cernan |
| CodeSearch, Grimoire |
Zoekt kythe |
Sourcegraph, OpenGrok, livegrep |
| Critique, Gerrit, Mondrian etc. |
Gerrit |
Reviewable , Phabricator |
| cider |
|
Eclipse Che, Cloud9, gitpod.io, Coder, Code-Server (VSCode in a Tab) |
| buganizer |
Google Issue Tracker |
JIRA, bugzilla, github issues, Linear |
| Bugjuggler |
|
SnoozeThis |
| ToTT |
Google Test Blog |
Increment |
| Copybara / MOE |
Copybara, MOE |
|
| workflow/dependency management |
|
Luigi, Airflow, digdag, Pachyderm, Dask |
| ErrorProne |
ErrorProne |
SpotBugs, FindBugs |
| Dapper |
stackdriver trace |
zipkin, opentracing, jaeger, LightStep, Honeycomb |
| C++ Tips of the Week |
Abseil C++ Tips of the Week |
|
| DiRT |
|
ChaosMonkey, aws fis |
| Rosie |
|
microplane, silver-platter |
| API Improvements Proposals |
AIP |
|
| Google Internal |
Real-World |
| software engineering at google 1, 2 |
Software Engineering at Google: Lessons Learned from Programming Over Time |
| SRE @ google |
|
| valentine |
Vault, 1Password, Lastpass, pass |
| OWNERS files in repo |
github CODEOWNERS |
| snippets |
Khan/snippets |
| SnipIt |
recordit, CloudApp, dropbox screenshots, Snippyly |
| stuff (SaaS IT management) |
productiv, intello, zylo |
| stuff (Device Management) |
Fleetsmith, jamf |
| device security monitoring |
Red Canary |
| beyondcorp |
beyondcorp |
| go/ links |
golinks, go, Goat, trotto, go-shorten |
| google3 philosophy |
innersource, monorepo |
| doing code review |
code review |
| safely sharing 1-time secrets |
croc, onetimesecret, privatebin |
| messaging |
mattermost, Slack, gchat |
| tech talks |
TechTalks @ Google |
| g3doc |
writethedocs, docs-as-code |
| GUTS |
spoke |
| Google Internal |
Real-World |
| OKR |
CultureAmp OKR, Lattice Goals, Ally, workboard |
| HRIS/ERP |
Namely, BambooHR, Workday, Paylocity |
| peer bonus |
bonus.ly |
| kudos |
heytaco, slack ++ bot |
| perks |
fond |
| talks |
Talks @ Google |
| dory |
slido |
| edge, people ops training |
LifeLabs |
| googlegeist |
Culture Amp, humu, tinypulse, peakon |
| Meng |
Search Inside Yourself |
| Lazlo |
Work Rules |
| Claire Stapleton |
Tech Support - existential advice for the modern tech worker |
| books about google |
How Google Works, In The Plex |
also check out xoogler.co, which organizes events, slack channels etc
disclaimer: I'm not affiliated with any of the technologies mentioned above.
disclaimer: I left Google a number of years ago so some of the naming might be dated