jlewi/foyle

logs Analyze fails DB is temporarily unavailable - can't run logs analyze or learn while serving is running

Opened this issue · 3 comments

jlewi commented

Here are the logs

2024-05-28T15:30:03.253-0700INFOcmd/version.go:41binary version{"version": "dev", "commit": "none", "date": "unknown", "builtBy": "unknown"}
2024-05-28T15:30:03.254-0700INFOcmd/logs.go:55Processing logs{"logs": "/Users/jlewi/.foyle/logs/raw"}
2024-05-28T15:30:03.254-0700INFOanalyze/analyzer.go:55Analyzing logs{"logsDir": "/Users/jlewi/.foyle/logs/raw", "tracesDBDir": "/Users/jlewi/.foyle/logs/traces", "blocksDBDir": "/Users/jlewi/.foyle/logs/blocks"}
2024-05-28T15:30:03.254-0700INFOanalyze/analyzer.go:57Opening traces database{"database": "/Users/jlewi/.foyle/logs/traces"}
2024/05/28 15:30:03 [JOB 1] WAL file /Users/jlewi/.foyle/logs/traces/000011.log with log number 000011 stopped reading at offset: 0; replayed 0 keys in 0 batches
2024-05-28T15:30:03.373-0700INFOanalyze/analyzer.go:65Opening blocks database{"database": "/Users/jlewi/.foyle/logs/blocks"}
2024-05-28T15:30:03.393-0700INFOapplication/app.go:389Shutting down the application
Error processing logs;
 resource temporarily unavailable
could not open blocks database /Users/jlewi/.foyle/logs/blocks
github.com/jlewi/foyle/app/pkg/analyze.(*Analyzer).Analyze
/Users/jlewi/git_foyle/app/pkg/analyze/analyzer.go:68
github.com/jlewi/foyle/app/cmd.NewLogsProcessCmd.func1.1
/Users/jlewi/git_foyle/app/cmd/logs.go:57
github.com/jlewi/foyle/app/cmd.NewLogsProcessCmd.func1
/Users/jlewi/git_foyle/app/cmd/logs.go:62
github.com/spf13/cobra.(*Command).execute
/Users/jlewi/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:987
github.com/spf13/cobra.(*Command).ExecuteC
/Users/jlewi/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1115
github.com/spf13/cobra.(*Command).Execute
/Users/jlewi/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1039
main.main
/Users/jlewi/git_foyle/app/main.go:13
runtime.main
/opt/homebrew/Cellar/go/1.22.1/libexec/src/runtime/proc.go:271
runtime.goexit
/opt/homebrew/Cellar/go/1.22.1/libexec/src/runtime/asm_arm64.s:1222

I wonder if this is because I'm running the Foyle server and the Foyle server is opening the PebbleDB

foyle/app/pkg/analyze/crud.go

Line 25 in 225d30f

blocksDB, err := pebble.Open(cfg.GetBlocksDBDir(), &pebble.Options{})

So we are trying to have multiple processes open the DB which doesn't work.

jlewi commented

Confirmed as soon as I stopped the server it worked.

jlewi commented

I then tried to start the server and I got this error

Error running request;
 resource temporarily unavailable
could not open blocks database /Users/jlewi/.foyle/logs/blocks
github.com/jlewi/foyle/app/pkg/analyze.NewCrudHandler
	/Users/jlewi/git_foyle/app/pkg/analyze/crud.go:27
github.com/jlewi/foyle/app/pkg/server.NewServer
	/Users/jlewi/git_foyle/app/pkg/server/server.go:91
github.com/jlewi/foyle/app/pkg/application.(*App).SetupServer
	/Users/jlewi/git_foyle/app/pkg/application/app.go:297
github.com/jlewi/foyle/app/cmd.NewRootCmd.NewServeCmd.func1.1
	/Users/jlewi/git_foyle/app/cmd/serve.go:29
github.com/jlewi/foyle/app/cmd.NewRootCmd.NewServeCmd.func1
	/Users/jlewi/git_foyle/app/cmd/serve.go:38
github.com/spf13/cobra.(*Command).execute
	/Users/jlewi/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:987
github.com/spf13/cobra.(*Command).ExecuteC
	/Users/jlewi/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1115
github.com/spf13/cobra.(*Command).Execute
	/Users/jlewi/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1039
main.main
	/Users/jlewi/git_foyle/app/main.go:13
runtime.main
	/opt/homebrew/Cellar/go/1.22.1/libexec/src/runtime/proc.go:271
runtime.goexit
	/opt/homebrew/Cellar/go/1.22.1/libexec/src/runtime/asm_arm64.s:1222
jlewi commented

I had to remove the lock file

rm  ~/.foyle/logs/blocks/LOCK

Why didn't that get cleaned up?

I had Foyle running in a debug session that probably had an open connection to the database.