MartinSahlen/cloud-functions-python

Add support for bundling data files

Closed this issue · 4 comments

Add support for bundling data files

@venkatesh-1729 , Just to understand what you mean by this. You are thinking that you want to add a file named for instance config.yaml or config.json that the function should read and parse? That is probably not supported at the moment (haven't tried though). Could you add some use cases just to make sure this will be implemented and supported the way you need it?

@MartinSahlen in general we package data files using MANIFEST.in or package_data in setuptools. I am guessing this can be achieved by using --add-data option of pyinstaller here.

A quick way to do this would be mounting project root directory in docker and copy data files from either Manifest.in or package_data option and add them via --add-data in pyinstaller. thoughts ??

I think we should leave anything pertaining MANIFEST.in, and setuptools/pip from this, we don't want the users of this library to have to deal with internals of pip or understand how pip works beyond running pip install 😄 . Rather, (as you seem suggest in the last paragraph), the function builder should just scoop up any files or folders in the path and include them in the package. So if you have a file named file.yaml in the repo root, the function builder should include it so you can just use https://docs.python.org/2/library/functions.html#open as you normally would. I will look into this, it is pretty easy to do via pyinstaller I think.

@venkatesh-1729 I am pretty swamped currently, so a PR would be greatly appreciated if you need this urgently. Otherwise a workaround would be to just embed the files you need as multiline strings in your python code. I will take a look at this next week if time allows!