Memsniff

Memsniff displays the busiest keys being retrieved from your memcache server. It has been inspired by the earlier mctop and memkeys tools.

Like its predecessors, memsniff listens to network traffic and identifies responses to get requests, and is usually run directly on a memcache server host.

You can see our announcement here.

Installation

Before building you'll need to have the libpcap library and headers installed.

On Redhat-based distributions:

# yum install libpcap-devel

Or on Debian-based distributions:

# apt-get update && apt-get install libpcap-dev

Memsniff uses the standard golang toolchain, which makes installation simple. Once you have the toolchain installed and $GOPATH pointed to your working directory:

$ go get github.com/box/memsniff
$ go build github.com/box/memsniff

You will find a compiled binary at $GOPATH/bin/memsniff, ready to be transferred to your Memcache hosts or packaged in your distribution's preferred format.

Usage

On most operating systems memsniff requires superuser privileges to capture network traffic from an interface, which you specify with the -i option.

# memsniff -i eth0

See -h for more command-line options. Once running a few more keys are active:

p - Pause the updating of the display. Press p again to resume.
q - Exit memsniff.

Roadmap

Support binary memcached protocol
Support additional operations beyond GET
Support alternate sorting methods
Create a stable report format for output to disk
Automatic logging to disk when specified conditions are met (e.g. aggregate or single key traffic exceeds a threshold)
Break out traffic by client IP
Supply build support for common package formats (.deb, .rpm, …)

Developing memsniff

Want to contribute? First have a look at CONTRIBUTING.md.

Running the tests

memsniff uses the standard Go testing framework:

$ go test ./...
?   	github.com/box/memsniff	[no test files]
...
ok  	github.com/box/memsniff/vendor/github.com/spf13/pflag	0.067s

Data pipeline

Raw packets are captured on the main thread from libpcap using GoPacket.
Batches of raw packets are sent to the decode pool, where workers parse the memcached protocol looking for responses to get requests. The key and size of the value returned are extracted into a response summary.
Batches of response summaries are sent to the analysis pool where the stream is hash partitioned by cache key and sent to workers. Each worker maintains a hotlist of the busiest keys in its hash partition.
In response to periodic requests from the UI, the analysis pool merges reports from all its workers into a single sorted hotlist, which is displayed to the user.

Support

Need to contact us directly? Email oss@box.com and be sure to include the name of this project in the subject.

Copyright and License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

alum/memsniff