alexdicianu/redis_toolkit

Speed up report generation by rewriting the reporting scripts in Go

Closed this issue · 0 comments

More as a long term goal, but right now there's a complexity level of O(n) = n * log(n) for building the prefix tree. Often there are lots of keys in Redis which tends to make the tree generation quite a lengthy process for over 100K keys.

Doing a quick test python vs. go for a complexity level of O(n) = n * n, it's pretty clear who the winner is.

# test.go
package main
import "fmt"
func main() {
    sum := 0
    for i := 0; i < 100000; i++ {
        for j := 0; j < 100000; j++ {
            sum += j
        }
    }
    fmt.Println("Sum is:", sum)
}

# test.py
sum = 0
for i in range(100000):
    for j in range(100000):
        sum += j
print "Sum is:", sum

==============================================

# Go
$ time ./test
Sum is: 499995000000000

real	0m3.535s
user	0m3.501s
sys	0m0.014s

# Python
$ time python test.py
Sum is: 499995000000000

real	7m56.064s
user	7m30.009s
sys	0m3.786s