truera/trulens

[BUG] Dashboard ignores `database_url` when opening Apps or Evaluations pages without opening Leaderboards first

theoratkin opened this issue ยท 4 comments

Bug Description
I spent many hours figuring out why the Streamlit dashboard seemingly randomly ignores the database_url and falls back to using default.sqlite. Turns out, if you happen to have Apps or Evaluations page opened at the moment you execute run_dashboard, then the Tru singleton is initialized with the default constructor, ignoring your database_url.

To Reproduce

  1. Run the dashboard with your own database URL:
DB_URL = 'postgresql+psycopg://test:12345678@localhost/test'
tru = Tru(database_url=DB_URL)
tru.run_dashboard()
  1. Go either to http://localhost:8501/Apps or http://localhost:8501/Evaluations.
  2. The page will be empty (even if you already have something in the database) and you will see a default.sqlite file created in the project directory.

Expected behavior
database_url is respected regardless of which dashboard page you open first.

Relevant Logs/Tracebacks
In the Leaderboard.py the database_url is initialized by argument parser:

if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("--database-url", default=None)
try:
args = parser.parse_args()
except SystemExit as e:
# This exception will be raised if --help or invalid command line arguments
# are used. Currently, streamlit prevents the program from exiting normally,
# so we have to do a hard exit.
sys.exit(e.code)
database_url = args.database_url
main()

However, there is no such logic in Apps and Evaluation pages, so the default singleton constructor is used.

tru = Tru()
lms = tru.db

This relies on having the Leaderboard page opened before others.

Environment:

  • OS: Linux
  • Python Version: 3.11.8
  • TruLens version: 0.26.0

It's great that you've identified this issue and provided a detailed description of the bug, along with relevant logs and tracebacks. Your understanding of the problem and its potential solution is valuable. If you're interested, we encourage you to create a pull request with your proposed fix to help improve the TruLens dashboard for everyone. Thank you for your contribution!

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a ๐Ÿ‘ if this is helpful and ๐Ÿ‘Ž if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

@piotrm0 can you take this one?

Hi @theoratkin , I believe I fixed this as part of another database-related PR (#971). This should be in the next release but you can test with the PR version for now. Let me know if the issue persists.

Sorry for late reply, didn't have the chance to test it until now. Yes, it is indeed fixed now, thanks! Closing the issue.