Run 'tru.run_dashboard(port=8502)' Leaderboard shows error.
rajpr5656 opened this issue · 10 comments
Bug Description
Run 'tru.run_dashboard(port=8502)' Leaderboard shows error.
To Reproduce
Not Sure on how to reproduce the steps as I am running standard Trulens Eval. I have enclosed screen shots as attachment and also all python packages. I am using python 3.12.2 version. Enclosed all Python packages below that are there. I have created a separate
TruLens Leaderboard Issue.docx
python environment and using there. However, I open Jupyter Lab in by generic folder.
Package Version
aiohttp 3.9.3
aiosignal 1.3.1
alembic 1.13.1
altair 4.2.2
annotated-types 0.6.0
anyio 4.3.0
asgiref 3.8.1
asttokens 2.4.1
attrs 23.2.0
backoff 2.2.1
bcrypt 4.1.2
beautifulsoup4 4.12.3
blinker 1.7.0
bs4 0.0.2
build 1.2.1
cachetools 5.3.3
cargo 0.3
certifi 2024.2.2
charset-normalizer 3.3.2
chroma-hnswlib 0.7.3
chromadb 0.4.24
click 8.1.7
colorama 0.4.6
coloredlogs 15.0.1
comm 0.2.2
contourpy 1.2.0
cycler 0.12.1
dataclasses-json 0.6.4
debugpy 1.8.1
decorator 5.1.1
Deprecated 1.2.14
dill 0.3.8
dirtyjson 1.0.8
distro 1.9.0
entrypoints 0.4
executing 2.0.1
Faker 24.4.0
fastapi 0.110.0
favicon 0.7.0
filelock 3.13.3
flatbuffers 24.3.25
fonttools 4.50.0
frozendict 2.4.0
frozenlist 1.4.1
fsspec 2024.3.1
gitdb 4.0.11
GitPython 3.1.42
google-auth 2.29.0
googleapis-common-protos 1.63.0
greenlet 3.0.3
grpcio 1.62.1
h11 0.14.0
htbuilder 0.6.2
httpcore 1.0.5
httptools 0.6.1
httpx 0.27.0
huggingface-hub 0.22.1
humanfriendly 10.0
humanize 4.9.0
idna 3.6
importlib-metadata 6.11.0
importlib_resources 6.4.0
InstructorEmbedding 1.0.1
ipykernel 6.29.4
ipython 8.22.2
ipywidgets 8.1.2
jedi 0.19.1
Jinja2 3.1.3
joblib 1.3.2
jsonpatch 1.33
jsonpointer 2.4
jsonschema 4.21.1
jsonschema-specifications 2023.12.1
jupyter_client 8.6.1
jupyter_core 5.7.2
jupyterlab_widgets 3.0.10
kiwisolver 1.4.5
kubernetes 29.0.0
langchain 0.1.13
langchain-community 0.0.29
langchain-core 0.1.35
langchain-text-splitters 0.0.1
langsmith 0.1.36
llama-index 0.10.25
llama-index-agent-openai 0.2.1
llama-index-cli 0.1.11
llama-index-core 0.10.25.post1
llama-index-embeddings-huggingface 0.2.0
llama-index-embeddings-instructor 0.1.3
llama-index-embeddings-openai 0.1.7
llama-index-indices-managed-llama-cloud 0.1.5
llama-index-legacy 0.9.48
llama-index-llms-openai 0.1.13
llama-index-multi-modal-llms-openai 0.1.4
llama-index-program-openai 0.1.5
llama-index-question-gen-openai 0.1.3
llama-index-readers-file 0.1.12
llama-index-readers-llama-parse 0.1.4
llama-parse 0.4.0
llamaindex-py-client 0.1.15
lxml 5.1.0
Mako 1.3.2
Markdown 3.6
markdown-it-py 3.0.0
markdownlit 0.0.7
MarkupSafe 2.1.5
marshmallow 3.21.1
matplotlib 3.8.3
matplotlib-inline 0.1.6
mdurl 0.1.2
merkle-json 1.0.0
millify 0.1.1
minijinja 1.0.16
mmh3 4.1.0
monotonic 1.6
more-itertools 10.2.0
mpmath 1.3.0
multidict 6.0.5
munch 4.0.0
mypy-extensions 1.0.0
nest-asyncio 1.6.0
networkx 3.2.1
nltk 3.8.1
numpy 1.26.4
oauthlib 3.2.2
onnxruntime 1.17.1
openai 1.14.3
opentelemetry-api 1.23.0
opentelemetry-exporter-otlp-proto-common 1.23.0
opentelemetry-exporter-otlp-proto-grpc 1.23.0
opentelemetry-instrumentation 0.44b0
opentelemetry-instrumentation-asgi 0.44b0
opentelemetry-instrumentation-fastapi 0.44b0
opentelemetry-proto 1.23.0
opentelemetry-sdk 1.23.0
opentelemetry-semantic-conventions 0.44b0
opentelemetry-util-http 0.44b0
orjson 3.10.0
overrides 7.7.0
packaging 23.2
pandas 2.2.1
parso 0.8.3
pillow 10.2.0
pip 24.0
platformdirs 4.2.0
posthog 3.5.0
prometheus_client 0.20.0
prompt-toolkit 3.0.43
protobuf 4.25.3
psutil 5.9.8
pulsar-client 3.4.0
pure-eval 0.2.2
pyarrow 15.0.2
pyasn1 0.6.0
pyasn1_modules 0.4.0
pydantic 2.6.4
pydantic_core 2.16.3
pydeck 0.8.1b0
Pygments 2.17.2
pymdown-extensions 10.7.1
PyMuPDF 1.24.0
PyMuPDFb 1.24.0
pyparsing 3.1.2
pypdf 4.1.0
PyPika 0.48.9
pyproject_hooks 1.0.0
pyreadline3 3.4.1
python-dateutil 2.9.0.post0
python-decouple 3.8
python-dotenv 1.0.1
pytz 2024.1
pywin32 306
PyYAML 6.0.1
pyzmq 25.1.2
referencing 0.34.0
regex 2023.12.25
requests 2.31.0
requests-oauthlib 2.0.0
rich 13.7.1
rpds-py 0.18.0
rsa 4.9
safetensors 0.4.2
scikit-learn 1.4.1.post1
scipy 1.12.0
sentence-transformers 2.6.1
setuptools 69.2.0
six 1.16.0
smmap 5.0.1
sniffio 1.3.1
soupsieve 2.5
SQLAlchemy 2.0.29
st-annotated-text 4.0.1
stack-data 0.6.3
starlette 0.36.3
streamlit 1.32.2
streamlit-aggrid 1.0.0
streamlit-camera-input-live 0.2.0
streamlit-card 1.0.0
streamlit-embedcode 0.1.2
streamlit-extras 0.4.0
streamlit-faker 0.0.3
streamlit-image-coordinates 0.1.6
streamlit-keyup 0.2.3
streamlit-toggle-switch 1.0.2
streamlit-vertical-slider 2.5.5
striprtf 0.0.26
sympy 1.12
tenacity 8.2.3
threadpoolctl 3.4.0
tiktoken 0.6.0
tokenizers 0.15.2
toml 0.10.2
toolz 0.12.1
torch 2.2.2
tornado 6.4
tqdm 4.66.2
traitlets 5.14.2
transformers 4.39.1
trulens 0.13.4
trulens-eval 0.27.0
typer 0.11.0
typing_extensions 4.10.0
typing-inspect 0.9.0
tzdata 2024.1
urllib3 2.2.1
utils 1.0.2
uvicorn 0.29.0
watchdog 4.0.0
watchfiles 0.21.0
wcwidth 0.2.13
websocket-client 1.7.0
websockets 12.0
wheel 0.43.0
widgetsnbextension 4.0.10
wrapt 1.16.0
yarl 1.9.4
zipp 3.18.1
Expected behavior
Expected to open Leaderboard and show results.
Relevant Logs/Tracebacks
Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks. If the issue is related to the TruLens dashboard, please also include a screenshot.
Environment:
- OS:Windows 11
- Python Version: Python 3.12.2
- TruLens version: 0.13.4
- Versions of other relevant installed libraries : I have enclosed all packages and its version.
Additional context
If you need logs let me know where to find.
Hey @rajpr5656! Good to see you tackling new challenges with TruLens. Let's dive into this dashboard dilemma together! 🚀
I understand that you're having trouble with the TruLens dashboard's Leaderboard not displaying as expected. Let's try to troubleshoot this issue step by step.
Firstly, it's important to ensure that all the packages are compatible with each other and with Python 3.12.2. Since Python 3.12 introduced several changes, some packages might not yet be fully compatible. You've mentioned aiohttp 3.9.3 and fastapi 0.110.0, which should be checked for compatibility with Python 3.12.2.
Secondly, although you're using TruLens version 0.13.4, it's a good practice to check if there's a newer version available that might have fixed the issue you're encountering. You can update TruLens to the latest version using the command pip install trulens --upgrade.
Next, ensure all dependencies are correctly installed and up to date. You can do this by running pip install -r requirements.txt if you have a requirements file or manually updating packages mentioned in your setup like aiohttp and fastapi using pip install aiohttp --upgrade and pip install fastapi --upgrade.
Also, ensure that the port 8502 is not already in use on your system. You can check this by using the netstat -aon | findstr 8502 command in the command prompt. If the port is in use, either close the application using it or choose a different port for the TruLens dashboard.
Sometimes, running the command prompt or PowerShell as an administrator can resolve issues related to permissions, especially on Windows 11. So, try running your application as an administrator.
Ensure that your firewall or antivirus is not blocking the application or the specific port you're trying to use. This can often cause unexpected issues.
When running tru.run_dashboard(port=8502), carefully check the console output for any errors or warnings that might give you more insight into what's going wrong.
If after following these steps the issue persists, it would be helpful to have more detailed information about the error messages you're receiving and the steps you've already taken to try to resolve the issue. This will help in diagnosing the problem more accurately.
For more information, you can refer to the TruLens codebase and documentation:
I hope this helps in resolving the issue. If you have any more questions or need further clarification, feel free to ask.
Sources
About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.
Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
If you want to continue the conversation, start your reply with @dosu-bot.
Thanks for your reply. Could you please let me know what version of Python should I reinstall to make tru.run_dashboard() to work. I will install recommended version of python. Also, pls recommend associated packages and its version to be installed. I will recreate all.
@rajpr5656 can you share the stack trace?
Could you pls provide steps? I am sorry for asking basic question. If you are asking for sequence of operations of LLM then please find below:
Followed below URL https://github.com/kevintsai/Building-and-Evaluating-Advanced-RAG-Applications/blob/main/L1-Advanced_RAG_Pipeline.ipynb
and only performed the following operations of
- 'Basic RAG pipeline'
- 'Evaluation setup using TruLens'
- Before running 'from utils import get_prebuilt_trulens_recorder
tru_recorder = get_prebuilt_trulens_recorder(query_engine,
app_id="Direct Query Engine")' I copied your util.py code that has feedback codes.
- All ran good and I was able see results by running 'records.head()'.
Issue is with 'tru.run_dashboard()'
Also, I degraded my python to 3.11.8
Here are my latest python packages:
Package Version
aiohttp 3.9.3
aiosignal 1.3.1
aiostream 0.5.2
alembic 1.13.1
altair 4.2.2
annotated-types 0.6.0
anyio 4.3.0
asttokens 2.4.1
attrs 23.2.0
backoff 2.2.1
beautifulsoup4 4.12.3
bleach 6.1.0
blinker 1.7.0
cachetools 5.3.3
certifi 2024.2.2
charset-normalizer 3.3.2
click 8.1.7
cohere 5.1.7
colorama 0.4.6
comm 0.2.2
contourpy 1.2.0
cycler 0.12.1
dataclasses-json 0.6.4
datasets 2.18.0
debugpy 1.8.1
decorator 5.1.1
Deprecated 1.2.14
dill 0.3.8
dirtyjson 1.0.8
distro 1.9.0
dnspython 2.6.1
entrypoints 0.4
executing 2.0.1
Faker 24.4.0
fastavro 1.9.4
favicon 0.7.0
filelock 3.13.3
fonttools 4.50.0
frozendict 2.4.0
frozenlist 1.4.1
fsspec 2024.2.0
gitdb 4.0.11
GitPython 3.1.42
greenlet 3.0.3
h11 0.14.0
htbuilder 0.6.2
html2text 2024.2.26
httpcore 1.0.5
httpx 0.27.0
huggingface-hub 0.22.2
humanize 4.9.0
idna 3.6
importlib_metadata 7.1.0
ipykernel 6.29.4
ipython 8.22.2
ipywidgets 8.1.2
jedi 0.19.1
Jinja2 3.1.3
joblib 1.3.2
jsonpatch 1.33
jsonpointer 2.4
jsonschema 4.21.1
jsonschema-specifications 2023.12.1
jupyter_client 8.6.1
jupyter_core 5.7.2
jupyterlab_widgets 3.0.10
kaggle 1.6.8
kiwisolver 1.4.5
langchain 0.1.13
langchain-community 0.0.29
langchain-core 0.1.36
langchain-text-splitters 0.0.1
langsmith 0.1.38
llama-index 0.10.25
llama-index-agent-openai 0.2.1
llama-index-cli 0.1.11
llama-index-core 0.10.25.post3
llama-index-embeddings-huggingface 0.2.0
llama-index-embeddings-openai 0.1.7
llama-index-indices-managed-llama-cloud 0.1.5
llama-index-legacy 0.9.48
llama-index-llms-openai 0.1.14
llama-index-multi-modal-llms-openai 0.1.4
llama-index-program-openai 0.1.5
llama-index-question-gen-openai 0.1.3
llama-index-readers-file 0.1.13
llama-index-readers-llama-parse 0.1.4
llama-parse 0.4.0
llamaindex-py-client 0.1.15
loguru 0.7.2
lxml 5.1.1
Mako 1.3.2
Markdown 3.6
markdown-it-py 3.0.0
markdownlit 0.0.7
MarkupSafe 2.1.5
marshmallow 3.21.1
matplotlib 3.8.3
matplotlib-inline 0.1.6
mdurl 0.1.2
merkle-json 1.0.0
millify 0.1.1
minijinja 1.0.16
more-itertools 10.2.0
mpmath 1.3.0
multidict 6.0.5
multiprocess 0.70.16
munch 4.0.0
mypy-extensions 1.0.0
nest-asyncio 1.6.0
networkx 3.2.1
nltk 3.8.1
numpy 1.26.4
openai 1.14.3
orjson 3.10.0
packaging 23.2
pandas 2.2.1
parso 0.8.3
pillow 10.2.0
pinecone-client 3.2.2
pip 24.0
platformdirs 4.2.0
prometheus_client 0.20.0
prompt-toolkit 3.0.43
protobuf 4.25.3
psutil 5.9.8
pure-eval 0.2.2
pyarrow 15.0.2
pyarrow-hotfix 0.6
pydantic 2.6.4
pydantic_core 2.16.3
pydeck 0.8.0
Pygments 2.17.2
pymdown-extensions 10.7.1
PyMuPDF 1.24.0
PyMuPDFb 1.24.0
pyparsing 3.1.2
pypdf 4.1.0
python-dateutil 2.9.0.post0
python-decouple 3.8
python-dotenv 1.0.1
python-slugify 8.0.4
pytz 2024.1
pywin32 306
PyYAML 6.0.1
pyzmq 25.1.2
referencing 0.34.0
regex 2023.12.25
requests 2.31.0
rich 13.7.1
rpds-py 0.18.0
safetensors 0.4.2
scikit-learn 1.4.1.post1
scipy 1.12.0
sentence-transformers 2.6.1
sentencepiece 0.2.0
setuptools 65.5.0
six 1.16.0
slack-bolt 1.18.1
slack_sdk 3.27.1
smmap 5.0.1
sniffio 1.3.1
soupsieve 2.5
SQLAlchemy 2.0.29
st-annotated-text 4.0.1
stack-data 0.6.3
streamlit 1.32.2
streamlit-aggrid 1.0.0
streamlit-camera-input-live 0.2.0
streamlit-card 1.0.0
streamlit-embedcode 0.1.2
streamlit-extras 0.4.0
streamlit-faker 0.0.3
streamlit-image-coordinates 0.1.6
streamlit-javascript 0.1.5
streamlit-keyup 0.2.3
streamlit-toggle-switch 1.0.2
streamlit-vertical-slider 2.5.5
striprtf 0.0.26
sympy 1.12
tenacity 8.2.3
text-unidecode 1.3
threadpoolctl 3.4.0
tiktoken 0.6.0
tokenizers 0.15.2
toml 0.10.2
toolz 0.12.1
torch 2.2.2
torchvision 0.17.2
tornado 6.4
tqdm 4.66.2
traitlets 5.14.2
transformers 4.39.2
trulens-eval 0.27.0
types-requests 2.31.0.20240311
typing_extensions 4.10.0
typing-inspect 0.9.0
tzdata 2024.1
tzlocal 5.2
urllib3 2.2.1
utils 1.0.2
validators 0.24.0
watchdog 4.0.0
wcwidth 0.2.13
webencodings 0.5.1
widgetsnbextension 4.0.10
win32-setctime 1.1.0
wrapt 1.16.0
xxhash 3.4.1
yarl 1.9.4
zipp 3.18.1
I'm asking to copy the error you are seeing the leaderboard. Thanks
Hi Sorry for the delay. Please find below error message. I have also enclosed doc with screenshots.
error screenshot.docx
STDERR
Exception in thread ScriptRunner.scriptThread:
Traceback (most recent call last):
File "C:\Users\rajpr\AppData\Local\Programs\Python\Python311\Lib\threading.py", line 1045, in _bootstrap_inner
self.run()
File "C:\Users\rajpr\AppData\Local\Programs\Python\Python311\Lib\threading.py", line 982, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\rajpr\AppData\Local\Programs\Python\Python311\Lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 267, in _run_script_thread
ctx = ScriptRunContext(
^^^^^^^^^^^^^^^^^
TypeError: ScriptRunContext.init() missing 1 required positional argument: 'main_script_path'
Hi Team, Do you have any updates please
@rajpr5656 I'm unable to reproduce your error, however can you try:
tru.run_dashboard(force=True, port=8502)
@rajpr5656 please give this a try, thanks!