Question: how many read/writes can 1 cassabon server handle per second?
Closed this issue · 3 comments
Hello Jeff,
Saw your presentation on cassabon on youtube. Cassabon looks very promising but it needs a lot of work to be done. Community support is needed for this great prohect.
I am currently using Opentsdb at my company for storing tsd. Opentsdb is scalable as it based upon hbase/hdfs stack. But it lacks the ease of Graphite. Cassabon seems to be a scalable replacement of Graphite. But I bave a couple of questions about Cassabon:
- Number of read/writes per second per cassabon server (may be you can briefly explain ur use case af change.org)
- What is the retention policy for metrics in cassabon? Are you guys storing metrics for months?
Sure.
Right now, we're handing roughly 150k stats per second across 6 c3.xlarge AWS instances running Cassabon. That's probably overkill. since no instance goes over about 60% CPU usage. The retention time on most of our stats is:
6 second resolution for 6 hours
1 minute resolution for 7 days
1 hour resolution for 30 days
6 hour resolution for 1 year.
That's backed by a 10 i2.2xlarge Cassandra cluster with a replication factor of 2 using a date-tiered compaction strategy. Keeping stats for a shorter amount of time would reduce the number of Cassandra nodes needed.
A reasonable estimate for what one Cassabon instance could handle is 50,000 stats per second on a c3.xlarge.
I'll actually be spending this week putting out comprehensive documentation on the project, including a reference architecture for a metrics stack that utilizes Cassabon -- it resembles the one in my presentation, except replace Cyanite with Cassabon. Once the documentation is done, I plan on getting it out to the community since I absolutely agree that an open source project needs good community support to be successful.
@jeffpierce Thanks a lot for sharing this useful information. And I hope cassabon gets a good community support. I will be playing around with it for a couple of weeks, and if it fits our use case here then I'll start contributing too. The main piece I will be interested in would be replacing Graphite-API completely. I know that needs a whole lot of work but it can be achieved. 👍
@utkarshcmu Sounds good. Feel free to ask any more questions that come up as you're playing with it.