Upload a file to gdrive from a cloud function (gcp)

[romain-boy]

Introduction

Good morning every one, I know Iterative for two years now, and I like very much your softwares.
Then I have discovered PyDrive2 yesterday, as it is the best library to help me doing an upload to a shared gdrive folder, from
a remote host, in a cloud function. Let me show you that.

Diagram of the program

Here is the principle of my Python program :

Sum up

As you told on your documentation, for a non-human authentication, I’ve written a settings.yaml, and downloaded a service account
key, attached to my project.

But before tell you more, may I say that, the upload works well with a human authentication. My file is correctly set into the folder with its content.

In fact, my problem is that I have an error of FILE NOT FOUND, when I launch my program with GoogleAuth().ServiceAuth()

Code

def login_with_service_account():
    """
    Google Drive service with a service account.
    note: for the service account to work, you need to share the folder or
    files with the service account email.

    :return: google auth
    """
    # Define the settings dict to use a service account
    # We also can use all options available for the settings dict like
    # oauth_scope,save_credentials,etc.
    settings = {
                "client_config_backend": "service",
                "service_config": {
                    "client_json_file_path": "service-secrets.json"
                }
            }
    # Create instance of GoogleAuth
    gauth = GoogleAuth(settings=settings)
    # Authenticate
    gauth.ServiceAuth()
    return gauth

def upload_to_gdrive(filename="download.png"):
    global GDRIVE_FOLDER_ID
    mimetypes = get_mimetypes()
    fileformat = filename.split(".")[-1]

    

    # Create GoogleDrive instance with authenticated GoogleAuth instance.
    drive = GoogleDrive(login_with_service_account())
    metadata = {
        'parents': [
            {"id": GDRIVE_FOLDER_ID}
        ],
        'title': filename,
        'mimeType': mimetypes[fileformat]
    }
    # Create file
    file = drive.CreateFile(metadata)
    file.SetContentFile(filename)
    file.Upload()


def dofunction():
    upload_to_gdrive()
    return "Return function"


if __name__ == "__main__":
    dofunction()

Error

It sent (I've changed sensible information in the text).

C:\Users\name.firstname\Documents\Venvs\GCP_venv\Scripts\python.exe "C:\Users\name.firstname\Documents\Perf\Program\main.py" 
Traceback (most recent call last):
  File "C:\Users\name.firstname\Documents\Venvs\GCP_venv\lib\site-packages\pydrive2\files.py", line 678, in _FilesInsert
    self.auth.service.files()
  File "C:\Users\name.firstname\Documents\Venvs\GCP_venv\lib\site-packages\googleapiclient\_helpers.py", line 130, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "C:\Users\name.firstname\Documents\Venvs\GCP_venv\lib\site-packages\googleapiclient\http.py", line 902, in execute
    _, body = self.next_chunk(http=http, num_retries=num_retries)
  File "C:\Users\name.firstname\Documents\Venvs\GCP_venv\lib\site-packages\googleapiclient\_helpers.py", line 130, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "C:\Users\name.firstname\Documents\Venvs\GCP_venv\lib\site-packages\googleapiclient\http.py", line 1022, in next_chunk
    raise ResumableUploadError(resp, content)
googleapiclient.errors.ResumableUploadError: <HttpError 404 when requesting None returned "File not found: sfqjqDJFfdshsfMKJSDfmJDSFsdsmfjmSLF". Details: "[{'domain': 'global', 'reason': 'notFound', 'message': 'File not found: sfqjqDJFfdshsfMKJSDfmJDSFsdsmfjmSLF', 'locationType': 'other', 'location': 'file'}]">

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\name.firstname\Documents\Perf\Program\main.py", line 122, in <module>
    dofunction()
  File "C:\Users\name.firstname\Documents\Perf\Program\main.py", line 117, in dofunction
    upload_to_gdrive()
  File "C:\Users\name.firstname\Documents\Perf\Program\main.py", line 113, in upload_to_gdrive
    file.Upload()
  File "C:\Users\name.firstname\Documents\Venvs\GCP_venv\lib\site-packages\pydrive2\files.py", line 512, in Upload
    self._FilesInsert(param=param)
  File "C:\Users\name.firstname\Documents\Venvs\GCP_venv\lib\site-packages\pydrive2\auth.py", line 84, in _decorated
    return decoratee(self, *args, **kwargs)
  File "C:\Users\name.firstname\Documents\Venvs\GCP_venv\lib\site-packages\pydrive2\files.py", line 683, in _FilesInsert
    raise ApiRequestError(error)
pydrive2.files.ApiRequestError: <HttpError 404 when requesting None returned "File not found: sfqjqDJFfdshsfMKJSDfmJDSFsdsmfjmSLF". Details: "[{'domain': 'global', 'reason': 'notFound', 'message': 'File not found: sfqjqDJFfdshsfMKJSDfmJDSFsdsmfjmSLF', 'locationType': 'other', 'location': 'file'}]">

Process finished with exit code 1

Question

I really don't understand, why it is complaining about the File not found, whereas it is a folder, except if it could be a problem with the service account.

Do you have any kind of idea to help me resolve this problem, or is it something not implemented yet ?

Thanks in advance.

[romain-boy]

The problem has been solved, by sharing the gdrive folder, with the email address of the service account.

Thanks for your program PyDrive2 !