Integrate new `lakefs` SDK wrapper into the project
Closed this issue · 3 comments
The new lakefs SDK wrapper proposal is available in beta: https://pypi.org/project/lakefs/
Should therefore be installable via pip install --pre lakefs.
Even though it is not completely ready yet, we can see if there is functionality that can already supersede some of our APIs (I'm thinking the client helpers especially), and save our learnings in this thread to ease migration to the lakefs package later.
One particularly interesting thing will be how to handle client autodiscovery: The wrapper APIs support omission of the client everywhere, which prompts an autodiscovery mechanism similarly to what we do now, but as a stateful session opened via init.
Will we use the "one file system, one session" approach or should we opt to reuse a session?
Update: the first stable releases, v0.1.0 and v0.1.1, have just landed on PyPI: https://pypi.org/project/lakefs/#history
Closed by #231.