doc upload fails with error 400

[kbseah]

Hello,

I'm using geonodectl v0.1 installed via pip from Github as described in the readme of this repo.

When attempting to upload a document to a Geonode server, I get the following error message:

ERROR:root:{'success': False, 'errors': ['Dieses Feld ist erforderlich.'], 'code': 'invalid'}
400 Client Error: Bad Request for url: https://tisdar.dev.thuenen.de/api/v2/documents

The command was geonodectl doc upload -f archive.tar. Attempts with other file types (PNG) failed too.

However, via the web interface of the Geonode instance I could upload the same archive.tar document

I was able to run subcommands doc list, users list, ds list, ds upload successfully. With ds upload I uploaded a test KML file without errors, using the same credentials.

Could you please help? It's not clear to me which metadata field is missing and how this can be supplied as an argument to geonodectl.

[gannebamm]

Hi Brandon,
Could you please test these examples: https://docs.geonode.org/en/master/devel/api/usage/index.html#resource-upload

Just to be sure it is not something on our dev instance.

[kbseah]

Thanks for looking into this. I adapted example 2: changed filename and path to an actual local file. Also "basic auth" is not explained in the geonode documentation (i did a search for the string "basic auth"). I guessed that it is the same value required as an environment variable by geonodectl (see https://github.com/GeoNodeUserGroup-DE/geonodectl/blob/main/geonodectl#L87)

Here's what i typed in python:

import requests
url = "https://tisdar.dev.thuenen.de/api/v2/"
payload = { 'title' : 'An example image' }
files = [ ('doc_file', ('Capture.PNG', open('./Capture.PNG', 'rb'), 'image/png')) ]
headers = { 'Authorization' : 'Basic ...' } # replace "..." with the output of $(echo -n user:password | base64) with my credentials
response = requests.request ("POST", url, headers=headers, data=payload, files=files)

Here is the result returned:

>>> response
<Response [405]>
>>> response.json()
{'success': False, 'errors': ['Methode "POST" nicht erlaubt.'], 'code': 'method_not_allowed'}

I then tried to post a link (example 2b) by modifying payload

payload = {'title' : 'example image', 'doc_url' : 'https://example.com/example.jpg' }
response = requests.request('POST', url, headers=headers, data=payload)

Response was the same:

>>> response
<Response [405]>
>>> response.json()
{'success': False, 'errors': ['Methode "POST" nicht erlaubt.'], 'code': 'method_not_allowed'}

software versions: requests v2.31.0, python 3.10.6

[gannebamm]

I was able to reproduce this on https://stable.demo.geonode.org

The issue is listed above this comment

[kbseah]

thanks, Florian!

[gannebamm]

@kbseah @ahmdthr please see Giovannis comment in upstream. Seems like we are using the wrong endpoint

[kbseah]

@gannebamm ah okay, it turns out there are two different issues.

The error with the wrong endpoint (response 405) in GeoNode/geonode#11703 relates to the commands i typed at the python REPL. That is now resolved with the response above.

However when uploading with the command geonodectl documents upload, the error code is 400. geonodectl should already point to the correct endpoint, see https://github.com/GeoNodeUserGroup-DE/geonodectl/blame/e9d9506cd1b8061ced173fa058d5a2f065d61999/geonoderest/documents.py#L103

My guess is that the title metadata is missing when uploading with geonodectl. I was able to get the same error code 400 at the python REPL by omitting the data=payload parameter from the requests.request command in the code snippet.

In the README of the geonodectl repo, there is an example command geonodectl ds upload -f ~/data/geolocation.shp -t example-shape where the parameter -t appears to be the title metadata. However this option -t is not recognized in the current version.

So my question is: how to specify the metadata required when uploading documents with geonodectl?

[mwallschlaeger]

yes I removed the the title argument in one of the last commits as it was not working when uploading a dataset. I think i additionally removed it for documents as well this needs to be fixed.

@kbseah at the current state of this project its not possible to directly upload a document or a dataset and edit the metadata at the same time. Currently you need to patch the object afterwards

[kbseah]

Thanks for the clarification, @mwallschlaeger . The issue is that the upload of a document fails if the title is not provided.

[ahmdthr]

Will it be desirable to let the upload happen when the title is not provided and set the title to the name of the file?

[gannebamm]

Will it be desirable to let the upload happen when the title is not provided and set the title to the name of the file?

I think this is a good default and also the way the UI handles the upload. @mwallschlaeger your thoughts?

[mwallschlaeger]

yes i would really appreciate this change. maybe make the title parameter still optional, but if set patch the uploaded dataset. I forsee the problem that uploads which take longer require geonodectl to wait until the upload is complete to change the title.

[mwallschlaeger]

just for documentation: #19

[mwallschlaeger]

I think this is fixed by now