A handy lookup table of similar technology and services to help ex-googlers survive the real world :)
pull-requests very welcomed. Please do not list any confidential projects!
For a working example of (some) of these technologies integrated together, see:
https://github.com/google/startup-os
See also: System Design Primer, The Hadoop Ecosystem Table, Map AWS services to GCP products
| Google Internal |
Google External |
Open Source / Real-World |
| MapReduce |
|
Apache Hadoop, Spark |
| Protocol Buffer |
Protobuf |
Cap'n Proto, Thrift, Avro, Amazon Ion, CBOR |
| Stubby |
gRPC |
Thrift, Bolt |
| Chubby |
|
Apache Zookeeper, etcd, HashiCorp Consul |
| Goops / PubSub |
|
Apache Kafka, Apache Pulsar, Facebook LogDevice |
| Google Internal |
Google External |
Open Source / Real-World |
| Borg |
Kubernetes |
Apache Mesos, Apache Aurora, HashiCorp Nomad |
| GSLB, GFE, Maglev, uberproxy |
|
ELB, HAProxy, Istio, F5, envoy |
| uberproxy (sso proxy) |
buzzfeed-sso |
|
| data center hardware |
open compute |
|
| Jupiter, Starblaze |
|
|
| B4, Stargate, TE |
|
|
| USPS, Andromeda |
|
|
| Google Internal |
Google External |
Open Source / Real-World |
| GFS/Colossus |
|
HDFS, Ceph, GlusterFS, MooseFS |
| BigTable |
Cloud BigTable |
PrestoDB, Cassandra, HBase, Accumulo, DynamoDB, ScyllaDB |
| Spanner |
Cloud Spanner |
Vitess, CockroachDB, TiDB |
| ColumnIO / Capacitor |
|
Apache Parquet |
| sstable |
levelDB |
RocksDB |
| zippy |
Snappy |
lz4 |
| Google Internal |
Google External |
Open Source |
SaaS |
| Dremel |
BigQuery |
Apache Drill, Presto, Spark(sort-of), |
AWS Athena, Redshift Spectrum, Snowflake |
| Dremel UI |
|
Redash, Metabase, Apache Superset |
|
| Search (Mustang, Alexandria) |
|
Elasticsearch, Solr, Lucene |
algolia |
| pubsub |
pubsub |
NATS.io, RabbitMQ, PubNub |
AWS SQS/SNS, AWS AppSync |
| Flume (Java) |
Apache Beam |
Apache Crunch |
|
| MillWheel |
Cloud Dataflow |
Apache Flink |
|
| Colab |
Colaboratory |
Jupyter |
|
| PLX |
Google Data Studio |
|
Mode |
| Google Internal |
Google External |
Real-World |
| Assimilator |
|
Dominator |
| Blaze |
Bazel |
Buck, Pants, please.build, Blade, recc |
| Oncall |
|
PagerDuty, OpsGenie, VictorOps |
| varz/borgmon/monarch |
|
Datadog, Prometheus, M3, librato, newrelic, skylight, scout, Scotty/tricorder, netdata, also this and this |
| Exception/Error Tracking (??) |
|
Sentry.io, Raygun.io, Rollbar, Honeybadger, Airbrake, OverOps |
| styleguides |
google styleguides |
PEP-8, HoundCI auto-style-reviewer |
| Blaze/Forge/TAP/BuildCop |
Cloud Build |
buildkite, CircleCI, travis, jenkins, gitlabCI, github, Drone |
| Sandman(test env)/Guitar |
|
|
| Sisyphus / Rapid |
|
Spinnaker, lambdaCD, screwdriver.cd, CodeShip, shipit-engine, GoCD, AWS CodeDeploy, Capistrano, Fabric, ConcourseCI, samson |
| borg / borgcfg / gcl |
|
AWS Cloudformation, Puppet, Chef, Salt, Ansible, Terraform, Jsonnet, kubecfg, pulumi |
| logging, analog |
StackDriver |
logstash, fluentd, papertrail, cernan |
| CodeSearch |
Zoekt |
Sourcegraph, OpenGrok |
| Critique, Gerrit, Mondrian etc. |
Gerrit |
Reviewable , Phabricator |
| cider |
|
Eclipse Che, Cloud9, gitpod.io, Coder, Code-Server (VSCode in a Tab) |
| buganizer |
|
JIRA, bugzilla, github issues |
| ToTT |
Google Test Blog |
Increment |
| Copybara / MOE |
Copybara, MOE |
|
| workflow/dependency management |
|
luigi, airflow, digdag, packyderm, dask |
| ErrorProne |
ErrorProne |
SpotBugs, FindBugs |
| dapper (distributed trace) |
stackdriver trace |
zipkin, opentracing, jaeger |
| Google Internal |
Real-World |
| software engineering at google |
|
| valentine |
1Password, Lastpass, pass |
| OWNERS files in repo |
github CODEOWNERS |
| snippets |
Khan/snippets |
| memegen |
memegen |
| SnipIt |
recordit, CloudApp, dropbox screenshots |
| edge, people ops training |
LifeLabs |
| googlegeist |
Culture Amp, humu, tinypulse, peakon |
| HRIS/ERP |
Namely, BambooHR, Workday |
| stuff (SaaS IT management) |
productiv, intello, zylo |
| stuff (Device Management) |
Fleetsmith, jamf, rippling IT |
| device security monitoring |
Red Canary |
| beyondcorp |
beyondcorp |
| go/ links |
golinks, go, Goat, trotto |
| google3 philosophy |
innersource, monorepo |
| doing code review |
code review |
| safely sharing 1-time secrets |
croc, onetimesecret, privatebin |
| messaging |
mattermost, slack, gchat |
| peer bonus |
bonus.ly, zestful |
| kudos |
heytaco, slack ++ bot |
also check out xoogler.co, which organizes events, slack channels etc
disclaimer: I'm not affiliated with any of the technologies mentioned above.
disclaimer: I left Google a number of years ago so some of the naming might be dated