v2: Constant CPU usage

[axllent]

Hi @ostafen. As you would probably have seen (in Mailpit), a user reported a constant CPU usage, which I believe I have traced back to CloverDB v2 (both 1 & 2 alpha).

A simple test:

package main

import (
	"time"
	"github.com/ostafen/clover/v2"
)

func main() {
	db, _ := clover.Open("", clover.InMemoryMode(true))
	defer db.Close()
	time.Sleep(600 * time.Second)
}

Run the program and check CPU usage with top/htop - the running binary will be using a constant 2-3% CPU. This does not happen with CloverDB v1.

Any ideas?

[ostafen]

Hi, @axllent. I don't see any clear reason for this to happen inside clover's code, since Open() simply creates a badger db instance and starts a background goroutine for clean-up (which runs every 5 minutes).

func (s *storageImpl) startGC() {
	s.chWg.Add(1)

	go func() {
		defer s.chWg.Done()

		ticker := time.NewTicker(s.conf.GCReclaimInterval)
		defer ticker.Stop()

		for {
			select {
			case <-s.chQuit:
				return

			case <-ticker.C:
				err := s.db.RunValueLogGC(s.conf.GCDiscardRatio)
				if err != nil {
					log.Printf("RunValueLogGC(): %s\n", err.Error())
				}
			}
		}
	}()
}

func (s *storageImpl) Open(path string, c *Config) error {
	if c.InMemory {
		path = ""
	}

	db, err := badger.Open(badger.DefaultOptions(path).WithLoggingLevel(badger.ERROR).WithInMemory(c.InMemory))

	s.db = db
	s.conf = c

	s.startGC()

	return err
}

[axllent]

But you can confirm the constant CPU usage on your end though, even using that simple test? Maybe it's the badger version then? 2-3% CPU isn't the end of the world, however it should be 0 for most of the time (when obviously not reading/writing/purging).

[ostafen]

Yes, I confirm that. However, the badger version is the same as v1.2.0

[axllent]

@ostafen I can confirm that I get the same constant 2-3% CPU usage when running badger directly (default options + in memory). I had done a complete comparison, only to realize later that v1 of CloverDB doesn't use badger for the in-memory database at all whereas v2 does. I guess that explains why v1 (in memory) doesn't have that issue then :)

I'm suspecting there isn't much, if anything, you can do here as it appears to be directly related to badger?

[ostafen]

Unless this is due to something that can be configured when opening the badger database, there's nothing I can do