openflagr/flagr

Issue with SQLite :memory: database losing tables

snemarch opened this issue · 1 comments

snemarch commented

It seems Flagr 1.1.12 has some issues when configured with SQLite in-memory database. Refreshing the details for a flag a couple of times very quickly leads to errors like "an unknown error occurred while looking up flag 1: no such table: distributions" in the web UI.

Steps to Reproduce (for bugs)

  1. Spin up a Docker container with Flagr configured with a in-memory database: docker run -e FLAGR_DB_DRIVER=sqlite3 -e 'FLAGR_DB_DBCONNECTIONSTR=:memory:' -it -p 18000:18000 checkr/flagr.
  2. Navigate to http://localhost:18000/#/ in a browser, "Create Simple Boolean Flag" (a description of foo_bar_baz should do the trick).
  3. Click the new flag (navigate to e.g. http://localhost:18000/#/flags/1).
  4. Refresh the browser a couple of times.

Sometimes the error happens as soon as you navigate to the flag details, sometimes it survives a couple of refreshes.

Your Environment

  • Version used (flagr version): v1.1.12
  • Host OS: macOS 10.15.7
  • Docker Desktop 2.5.0.1 (Docker Engine 19.03.13)

Looking at the output from Flagr, there doesn't seem to be any mentions of errors, closing the database connection or anything, before the tables disappear:

INFO[0077] started handling request method=GET remote="172.17.0.1:37880" request=/
INFO[0077] completed handling request measure#flagr.latency=273200 method=GET remote="172.17.0.1:37880" request=/ status=304 text_status="Not Modified" took="273.2µs"
INFO[0077] started handling request method=GET remote="172.17.0.1:37880" request=/api/v1/flags/1
INFO[0077] started handling request method=GET remote="172.17.0.1:37876" request=/api/v1/flags/entity_types
INFO[0077] started handling request method=GET remote="172.17.0.1:37872" request=/api/v1/tags
INFO[0077] completed handling request measure#flagr.latency=1238200 method=GET remote="172.17.0.1:37876" request=/api/v1/flags/entity_types status=500 text_status="Internal Server Error" took=1.2382ms
INFO[0077] error/go/src/github.com/checkr/flagr/pkg/handler/crud.go:192no such table: flag_entity_types
INFO[0077] completed handling request measure#flagr.latency=1118800 method=GET remote="172.17.0.1:37872" request=/api/v1/tags status=500 text_status="Internal Server Error" took=1.1188ms
INFO[0077] error/go/src/github.com/checkr/flagr/pkg/handler/crud.go:354no such table: tags
INFO[0077] error/go/src/github.com/checkr/flagr/pkg/handler/crud.go:146no such table: constraints
INFO[0077] error/go/src/github.com/checkr/flagr/pkg/handler/crud.go:146no such table: constraints
INFO[0077] error/go/src/github.com/checkr/flagr/pkg/handler/crud.go:146no such table: constraints
INFO[0077] error/go/src/github.com/checkr/flagr/pkg/handler/crud.go:146no such table: constraints
INFO[0077] completed handling request measure#flagr.latency=4679900 method=GET remote="172.17.0.1:37880" request=/api/v1/flags/1 status=500 text_status="Internal Server Error" took=4.6799ms
INFO[0077] started handling request method=GET remote="172.17.0.1:37880" request=/favicon.png
INFO[0077] completed handling request measure#flagr.latency=353900 method=GET remote="172.17.0.1:37880" request=/favicon.png status=200 text_status=OK took="353.9µs"
ERRO[0078] reload evaluation cache error err="no such table: flags"
INFO[0078] error/go/src/github.com/checkr/flagr/pkg/handler/eval_cache_fetcher.go:144no such table: flags
INFO[0081] error/go/src/github.com/checkr/flagr/pkg/handler/eval_cache_fetcher.go:144no such table: segments
INFO[0081] error/go/src/github.com/checkr/flagr/pkg/handler/eval_cache_fetcher.go:144no such table: segments
INFO[0081] error/go/src/github.com/checkr/flagr/pkg/handler/eval_cache_fetcher.go:144no such table: segments
ERRO[0081] reload evaluation cache error err="no such table: segments"
ERRO[0084] reload evaluation cache error err="no such table: flags"
INFO[0084] error/go/src/github.com/checkr/flagr/pkg/handler/eval_cache_fetcher.go:144no such table: flags
INFO[0087] error/go/src/github.com/checkr/flagr/pkg/handler/eval_cache_fetcher.go:144no such table: segments
INFO[0087] error/go/src/github.com/checkr/flagr/pkg/handler/eval_cache_fetcher.go:144no such table: segments

zhouzhuojie commented

Hi, I think this is related to how sqlite3 handles db connections and processes that connect to the in-memory db. For example, https://www.sqlite.org/inmemorydb.html#temp_db

If two or more distinct but shareable in-memory databases are needed in a single process, then the mode=memory query parameter can be used with a URI filename to create a named in-memory database:
rc = sqlite3_open("file:memdb1?mode=memory&cache=shared", &db);

So I tried the following and it works.

FLAGR_DB_DBCONNECTIONSTR=file:memdb1?mode=memory&cache=shared

In the unit tests of flagr, we only use :memory:, it has only one process and one connection established, that's probably why it passes the unit tests.