s3cmd integration
Closed this issue · 10 comments
Hello, I work in an airgapped environment where we use our own S3 provider (Ceph), And we use s3cmd to connect to it, I was unable to use s3cmd with the extension, Can you add to the docs how to use the extension with s3cmd?
In fact, I haven't tried it either. So I'll try it out these days
I reviewed again.
boto3 is a python package provided by aws, tensorboard integrates its capabilities by default, so you may try to use boto3 to see if you can access it.
Here I gave a little introduction to how to use it, but I think you may need to configure more things here.
Visits to similar environments are usually related to specific scenarios. It may be difficult for me to fully reproduce your usage scenarios, so I cannot guarantee that my next update(time not guaranteed) will solve your problems, so you are welcome to make more attempts and exchanges.
So if I were to use boto3 and configure my connection with it, I'd be able to use the TensorBoard extension directly accessing the bucket as a logdir?
I've also tried to use your documentation of how to use with awscli, and to make my specific endpoint work I used https://pypi.org/project/awscli-plugin-endpoint/
That also hasn't worked
If you use tensorboard directly (just use tensorboard in terminal), can it work? (actually this plugin just integrates tensorboard, without invasively modifying tensorboard)
It seemed to work if I launched a tensorboard using a Termnial block or with magics, but since I launched my notebook using the jupyter labhub command(Connecting to a remove hub and the remote jupyterhub redirects to my notebook using a proxy) and I didn't configure the hub to allow communication on the tensorboard's port I couldn't verify properly.
However I can't be sure, when I use the tensorboard-pro extension, It launches the tensorboard, however I don't see the logs from my bucket, and I couldn't find any error logs in the pod's logs
You can set your tensorboard's logger to debug mode, and then print out more logs, in fact, set it like this:
_logger = logging.getLogger("tensorboard")
_logger.setLevel(logging.DEBUG)
It's just that this ability currently needs to change the python code. You can directly go to like lib/python3.8/site-packages/tensorboard/util/tb_logging.py to set it and change the code (for testing only)
Later, I will consider making this logging capability a configuration item
I published v0.3.2
You can use jupyter-lab --debug to show debug log for tensorboard
There is no updated reply, and the issue is temporarily closed
If there are any questions later, we can reopen or create a new issue.