princenyeche/jiraone

Issue using download_attachments

JoeGT opened this issue · 5 comments

JoeGT commented

I have been using your fantastic JIRAONE library for a migration project for some of our JIRA projects to a new instance.
I am having issues trying to get the code working correctly for downloading attachments locally. I am sure it is just that I don’t understand your documentation properly:

https://jiraone.readthedocs.io/en/latest/apis.html#project
staticdownload_attachments

My script looks like this :

from jiraone import LOGIN, PROJECT
from threading import Thread
LOGIN.api = False
user = "xxx"
password = "xxxx
link = xxxxx
LOGIN(user=user, password=password, url=link)
#file = "conf.json"
#config = json.load(open(file))
#LOGIN(**config)
project = 'DPLSCE'
print ("Downloading attachments for Project - "+ project + "")
if name == 'main':
the output of the file would be absolute to the directory where this python file is being executed from
jql = f"project = {project}"
the below method, helps you download a report of a list of files per issue on a project or on projects
Thread(target=PROJECT.get_attachments_on_projects(query=jql)).start()
afterwards, you can use the below method to move attachments across instances without downloading it
if you're using your own file structure say a csv file, you need to identify the index of the attachment
for this, 3 keyword args are used key=0, attach=1, and file=2 -> all requires an integer value.
PROJECT.move_attachments_across_instances(attach_file="new.csv", key=0, attach=1, file=2)
To download an attachment locally use
PROJECT.download_attachments(download_path="C:\Temp\Download", attach=1, file=2)

But this throws the error:

Traceback (most recent call last):
File "C:\Temp\Python\jira-attachment-download.py", line 34, in
PROJECT.download_attachments(download_path="C:\Temp\Download", attach=1, file=2)
File "C:\Program Files\Python39\lib\site-packages\jiraone\reporting.py", line 753, in download_attachments
read = file_reader(folder=file_folder, file_name=file_name, **kwargs)
File "C:\Program Files\Python39\lib\site-packages\jiraone\reporting.py", line 3267, in file_reader
file = path_builder(path=folder, file_name=file_name)
File "C:\Program Files\Python39\lib\site-packages\jiraone\reporting.py", line 3150, in path_builder
base_dir = os.path.join(WORK_PATH, path)
File "C:\Program Files\Python39\lib\ntpath.py", line 117, in join
genericpath._check_arg_types('join', path, *paths)
File "C:\Program Files\Python39\lib\genericpath.py", line 152, in _check_arg_types
raise TypeError(f'{funcname}() argument must be str, bytes, or '
TypeError: join() argument must be str, bytes, or os.PathLike object, not 'NoneType'

I am unable to work out what I need to adjust to make this run successfully and would love you help to understand what I need to do. The PROJECT.get_attachments_on_projects runs successfully.

Hey @JoeGT

The download_attachments needs to read the file from the previous method but you didn't indicate where it should read it from hence the error of NoneType as the default from the folder and file is set to none. Since you used the default values from PROJECT.get_attachments_on_projects, you should add the below arguments to the download_attachments method so it can work for you.

PROJECT.download_attachments(download_path="C:\Temp\Download", attach=1, file=2,
file_folder="Attachment", file_name="attachment_file.csv")

Let me know how it goes.

JoeGT commented

Thank you @princenyeche. That got me headed in the right direction however there are still some issues and questions.

If I rely on the output from

PROJECT.get_attachments_on_projects

Then I get this error when running

PROJECT.download_attachments(download_path="C:\Temp\Download", key=1, attach=8, file=6, file_folder="Attachment", file_name="attachment_file.csv")

File "C:\Program Files\Python39\lib\site-packages\requests\models.py", line 439, in prepare_url
    raise MissingSchema(
requests.exceptions.MissingSchema: Invalid URL 'Attachment url': No scheme supplied. Perhaps you meant https://Attachment url?

It looks like it is reading the header row of the CSV that PROJECT.get_attachments_on_projects generated and trying to interpret the index 8 for attachments on that column value "Attachment url" - which is obviously not an actual valid URL.

If I remove the header row and then re-run, the file output is generated successfully.

Is there any way of getting the output to maintain the folder structure of the JIRA servers file system and to output using the fil ID rather than converting to the real file name ?

It is great to have the files able to be pulled but for the destination system, the folder structure needs to be duplicated to allow for the importer to find and reference all of the attached files using their ID (the real file name is just stored in the DB). For instance as an example, I have a project with the key "AIR" that has this folder structure on the file system:

Folder PATH listing for volume OSDisk
Volume serial number is 100C-1728

AIR
+---10000
| +---AIR-100
| | ---thumbs
| | _thumb_84697.png
| | _thumb_84743.png
| | _thumb_84781.png
| |
| +---AIR-1002
| | 195089
| |
| +---AIR-1003
| | | 174593
| | |
| | ---thumbs
| | _thumb_174593.png
| |
| +---AIR-1005
| | | 192310
| | | 192353
| | | 193092
| | |
| | ---thumbs
| | _thumb_192353.png
| |
| +---AIR-1006
| | | 174781
| | | 174782
| | |
| | ---thumbs
| | _thumb_174781.png
| | _thumb_174782.png

Thank you

Joe

Hey @JoeGT,

Yes, that's exactly what it does. You can also try this next time for the headers to prevent that error.

PROJECT.download_attachments(download_path="C:\Temp\Download", attach=1, file=2,
file_folder="Attachment", file_name="attachment_file.csv", skip=True)

The skip argument equal to true should skip the first row which contains the headers. Regarding your other question, the download doesn't store it based on the id but that shouldn't be difficult to implement. It also doesn't store it based on project keys and issue keys in the folders. That would be a nice feature to have there on that method, don't think it should take much time to implement, so I'll work on it for the next release. Add more context to your ask, then I'll see how I can add that to the next release

huyz commented

I have the exact same ask, and I'm trying to make changes to the code for that

v0.8.4 resolves this issue.