databrickslabs/databricks-sync

If the folder is empty for dbfs files it fails to skip the object

stikkireddy opened this issue · 0 comments

error example:
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 568, in run_until_complete
return future.result()
File "/Users/itaiweiss/PycharmProjects/demoMLFlow/venv/lib/python3.7/site-packages/databricks_sync/sdk/pipeline.py", line 87, in trigger
async for item in self.generate():
File "/Users/itaiweiss/PycharmProjects/demoMLFlow/venv/lib/python3.7/site-packages/databricks_sync/sdk/pipeline.py", line 91, in generate
async for item in self._generate():
File "/Users/itaiweiss/PycharmProjects/demoMLFlow/venv/lib/python3.7/site-packages/databricks_sync/sdk/generators/dbfs.py", line 129, in _generate
for_each_var_id_name_pairs=dbfs_files_id_name_pairs)
File "/Users/itaiweiss/PycharmProjects/demoMLFlow/venv/lib/python3.7/site-packages/databricks_sync/sdk/generators/dbfs.py", line 112, in __make_dbfs_file_data
dbfs_data.upsert_local_variable(self.DBFS_FOREACH_VAR, dbfs_file_data)
AttributeError: 'NoneType' object has no attribute 'upsert_local_variable'