Pika
中文
IntroductionPika is a persistent huge storage service , compatible with the vast majority of redis interfaces (details), including string, hash, list, zset, set and management interfaces. With the huge amount of data stored, redis may suffer for a capacity bottleneck, and pika was born for solving it. Except huge storage capacity, pika also support master-slave mode by slaveof command, including full and partial synchronization. You can also use pika together with twemproxy or codis(pika has supported data migration in codis,thanks left2right and fancy-rabbit) for distributed Redis solution
UserList
Feature
- huge storage capacity
- compatible with redis interface, you can migrate to pika easily
- support master-slave mode (slaveof)
- various management interfaces
For developer
Releases
The User can download the binary release from releases or compile the source release.
Dependencies
- snappy - a library for fast data compression
- glog - google log library
Upgrade your gcc to version at least 4.8 to get C++11 support.
Supported platforms
-
linux - Centos 5&6
-
linux - Ubuntu
If it comes to some missing libs, install them according to the prompts and retry it.
Compile
Upgrade your gcc to version at least 4.8 to get C++11 support.
Get the source code
git clone https://github.com/Qihoo360/pika.git
Then compile pika, all submodules will be updated automatically.
make
Usage
./output/bin/pika -c ./conf/pika.conf
deep011)
Performance (provided bytest environment
CPU module:Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz
CPU threads:56
MEMORY:256G
DISK:3T flash
NETWORK:10GBase-T/Full * 2
OS:centos 6.6
benchmark tools
Test 1
Purpose
With different number of pika worker threads, we test pika's max QPS.
Condition
pika db_size : 800G
value : 128bytes
Threads are not bound to CPU cores.
Result
Description : Horizontal axis is the number of worker threads; Vertical axis is the QPS. The value size is 128bytes. set3/get7 means 30% set and 70% get.
Conclusion
The best pika worker threads number is 20-24.
Test 2
Purpose
With the optimal worker threads number, we test pika's round-trip time.
Condition
pika db_size:800G
value:128bytes
Result
====== GET ======
10000000 requests completed in 23.10 seconds
200 parallel clients
3 bytes payload
keep alive: 1
99.89% <= 1 milliseconds
100.00% <= 2 milliseconds
100.00% <= 3 milliseconds
100.00% <= 5 milliseconds
100.00% <= 6 milliseconds
100.00% <= 7 milliseconds
100.00% <= 7 milliseconds
432862.97 requests per second
====== SET ======
10000000 requests completed in 36.15 seconds
200 parallel clients
3 bytes payload
keep alive: 1
91.97% <= 1 milliseconds
99.98% <= 2 milliseconds
99.98% <= 3 milliseconds
99.98% <= 4 milliseconds
99.98% <= 5 milliseconds
99.98% <= 6 milliseconds
99.98% <= 7 milliseconds
99.98% <= 9 milliseconds
99.98% <= 10 milliseconds
99.98% <= 11 milliseconds
99.98% <= 12 milliseconds
99.98% <= 13 milliseconds
99.98% <= 16 milliseconds
99.98% <= 18 milliseconds
99.99% <= 19 milliseconds
99.99% <= 23 milliseconds
99.99% <= 24 milliseconds
99.99% <= 25 milliseconds
99.99% <= 27 milliseconds
99.99% <= 28 milliseconds
99.99% <= 34 milliseconds
99.99% <= 37 milliseconds
99.99% <= 39 milliseconds
99.99% <= 40 milliseconds
99.99% <= 46 milliseconds
99.99% <= 48 milliseconds
99.99% <= 49 milliseconds
99.99% <= 50 milliseconds
99.99% <= 51 milliseconds
99.99% <= 52 milliseconds
99.99% <= 61 milliseconds
99.99% <= 63 milliseconds
99.99% <= 72 milliseconds
99.99% <= 73 milliseconds
99.99% <= 74 milliseconds
99.99% <= 76 milliseconds
99.99% <= 83 milliseconds
99.99% <= 84 milliseconds
99.99% <= 88 milliseconds
99.99% <= 89 milliseconds
99.99% <= 133 milliseconds
99.99% <= 134 milliseconds
99.99% <= 146 milliseconds
99.99% <= 147 milliseconds
100.00% <= 203 milliseconds
100.00% <= 204 milliseconds
100.00% <= 208 milliseconds
100.00% <= 217 milliseconds
100.00% <= 218 milliseconds
100.00% <= 219 milliseconds
100.00% <= 220 milliseconds
100.00% <= 229 milliseconds
100.00% <= 229 milliseconds
276617.50 requests per second
Conclusion
Both the 99.9% get/set RTT are below 2ms.
Test 3
Purpose
With the optimal worker threads number, we test the max qps of different commands.
Condition
pika worker threads:20
key count:10000
field count:100(except list)
value:128bytes
commands execute times:10000000(except lrange)
Result
PING_INLINE: 548606.50 requests per second
PING_BULK: 544573.31 requests per second
SET: 231830.31 requests per second
GET: 512163.91 requests per second
INCR: 230861.56 requests per second
MSET (10 keys): 94991.12 requests per second
LPUSH: 196093.81 requests per second
RPUSH: 195186.69 requests per second
LPOP: 131156.14 requests per second
RPOP: 152292.77 requests per second
LPUSH (needed to benchmark LRANGE): 196734.20 requests per second
LRANGE_10 (first 10 elements): 334448.16 requests per second
LRANGE_100 (first 100 elements): 50705.12 requests per second
LRANGE_300 (first 300 elements): 16745.16 requests per second
LRANGE_450 (first 450 elements): 6787.94 requests per second
LRANGE_600 (first 600 elements): 3170.38 requests per second
SADD: 160885.52 requests per second
SPOP: 128920.80 requests per second
HSET: 180209.41 requests per second
HINCRBY: 153364.81 requests per second
HINCRBYFLOAT: 141095.47 requests per second
HGET: 506791.00 requests per second
HMSET (10 fields): 27777.31 requests per second
HMGET (10 fields): 38998.52 requests per second
HGETALL: 109059.58 requests per second
ZADD: 120583.62 requests per second
ZREM: 161689.33 requests per second
PFADD: 6153.47 requests per second
PFCOUNT: 28312.57 requests per second
PFADD (needed to benchmark PFMERGE): 6166.37 requests per second
PFMERGE: 6007.09 requests per second
Test 4
Purpose
Compare the pika max qps with the redis.
Condition
pika worker threads:20
key count:10000
field count:100(except list)
value:128bytes
commands execute times:10000000(except lrange)
Redis version:3.2.0
Result
Documents
Contact Us
Mail: g-infra-bada@360.cn
QQ group: 294254078
For more information about Pika, Atlas and some other technology please pay attention to our Hulk platform official account