/go-dumpotron

Take pprof dumps when a go process eats too much ram or cpu

Primary LanguageGoMIT LicenseMIT

go-dumpotron

Take pprof dumps when a go process eats too much ram or cpu.

This has been running on prometheus.locotorp.info since Jan 2020

Requirements

  • graphviz
  • go 1.13
  • ipfs-cluster-ctl

Running

The following ENVs need to be set:

  • PPROF_AUTH_PASS
  • IPFS_CLUSTER_AUTH
  • GITHUB_TOKEN

Build

go build

Docker:

docker build -t go-dumpotron .

Run

Get credentials from the 1Password note named go-dumpotron credentials in the Infra team vault

As daemon, HTTP server accepting Alertmanager webhook calls

# Prepend `LOG_LEVEL=debug` for debugging logs
source .env && ./go-dumpotron -daemon

Docker:

docker run --rm --net=host --name=go-dumpotron --env-file=.dockerenv ipfsshipyardbot/go-dumpotron

One-time, generate pprof archive locally for specific instance

# Source envs from file
source .env && ./go-dumpotron gateway-bank1-ewr1.dwebops.net
# or by passing in a basic auth passwd for the endpoint
PPROF_AUTH_PASS=THE_ADMIN_HTTPASSWD ./go-dumpotron gateway-bank1-ewr1.dwebops.net

Docker:

mkdir /tmp/pprofs
docker run -it --env-file=.dockerenv -w /tmp/pprofs -v $(pwd):/tmp/pprofs ipfsshipyardbot/go-dumpotron gateway-bank1-ewr1.dwebops.net

Webhook Test

curl localhost:9096 -d @fixtures/sample.json