/parseAOF

A simple and fast tool to parse the AOF file of redis / 一个简单快速的解析Redis AOF文件的工具

Primary LanguageGoMIT LicenseMIT

img

A simple and fast tool to parse the AOF file of redis

Document :中文 / English

Content

1、Introduction

A simple and fast tool to parse the AOF file of redis

(1) Features

  • Code is clean, simple and easy to customize
  • Speed up parsing through multiple goroutines
  • A list of commands will be generated after parsing for log querying

(2) Architecture

2、Install

(1) Linux/Mac

git clone https://github.com/WGrape/parseAOF
cd parseAOF
go mod download

(2) Windows

Windows is temporarily not supported

3、Usage

Run the start.sh script with the path of the aof file

bash ./start.sh /path/appendonly.aof

(1) The input file

Here's an example input file ./data/appendonly.aof for you to test

Before running, pass the path of the aof file to the start.sh script, the content is as follows

*2
$6
SELECT
$1
0
... ...

(2) The output file

Here's an example output file ./data/aof.merged for you to test

After the parsing is complete, the file aof.merged will be generated in the directory of data, the content is as follows

--------------------parseAOF | version=0.5.0--------------------
SELECT 0 
set key1 1 
set key2 2 
set key3 3 
sadd key4 1 2 3 4 
lpush key5 1 2 3 4 5 
zadd key6 1 2 3 4 5 6 

(3) Example

4、Configuration

Config fileconfig/config.yml

Key Value Default Detail
debug false / true false debug mode
maxRoutines int 1024 allow max number of goroutines

5、Performance

  • The average speed to parse is 50000 lines/s
  • The maximum size of the aof is 1GB

(1) Testing

Id Lines Size Cost CPU
1 1,2301,117 39MB 3m50s <=65%
2 3,435,263 13MB 1m12s <=65%
3 1,043,700 6.6MB 38s <=65%