nicolevanderhoeven/deroamify

os.replace() not allowed while file still open in loop (Windows)

Closed this issue · 2 comments

It took a bit, but I was able to find the solution with some help. From what I can see, this issue is specific to windows. See more here:
https://stackoverflow.com/questions/75140357/need-a-workaround-for-known-issue-with-os-replace-function-on-windows?noredirect=1#comment132599049_75140357

I also found that when adding the reference back to the notes, having the folder 'assets/' appended to the entry could present problems down the line. If the files are ever moved, they will de-link from their references, but if they are only referenced by the file name, Obsidian can find the file as long as it is located within the vault and vault subfolders. With a few added notes, here was the code I successfully ran in the end. (I chose to call my folder "Attachments".)

import re
import glob
import os
import requests
import calendar
import time

vaultDir = '/Users/kacar/Firebase Temp/Firebase-Test2'

firebaseShort = 'none'
fullRead = 'none'
fileFullPath = ''
fullTempFilePath = ''
i = 0
ext = ''

# Walk through all files in all directories within the specified vault directory
for subdir, dirs, files in os.walk(vaultDir):
    for file in files:
        # Open file in directory
        fileFullPath = os.path.join(subdir,file)
        fhand = open(fileFullPath, errors='ignore')
        for line in fhand:
            # Download the Firebase file and save it in the assets folder
            if 'firebasestorage' in line:
                try:
                    # If it's a PDF, it will be in the format {{pdf: link}}
                    if '{{pdf:' in line:
                        link = re.search(r'https://firebasestorage(.*)\?alt(.*)\}', line)
                    else:
                        link = re.search(r'https://firebasestorage(.*)\?alt(.*)\)', line)
                    firebaseShort = 'https://firebasestorage' + link.group(1) # https://firebasestorage.googleapis.com/v0/b/firescript-577a2.appspot.com/o/imgs%2Fapp%2FDownloadMyBrain%2FLy4Wel-rjk.png
                    firebaseUrl = link.group(0)[:-1] # https://firebasestorage.googleapis.com/v0/b/firescript-577a2.appspot.com/o/imgs%2Fapp%2FDownloadMyBrain%2FLy4Wel-rjk.png?alt=media&token=0fbafc8f-0a47-4720-9e68-88f70803ced6
                    # Download the file locally
                    r = requests.get(firebaseUrl)
                    timestamp = calendar.timegm(time.gmtime())
                    # Get file extension of file. Ex: .png; .jpeg
                    reg = re.search(r'(.*)\.(.+)', firebaseShort[-5:]) # a.png / .jpeg
                    ext = '.' + reg.group(2) # .jpeg
                    # Create assets folder if it doesn't exist
                    if not os.path.exists(vaultDir + '/Attachments'):
                        os.makedirs(vaultDir + '/Attachments')
                    # Create new local file out of downloaded firebase file
                    #old -newFilePath = 'assets/' + str(timestamp) + '_' + str(i) + ext
                    newFilePath = str(timestamp) + '_' + str(i) + ext
                    ## FOR FINAL RUN remove 'assets/' from newFilePath                 
                    # print(firebaseUrl + '>>>' + newFilePath)
                    with open(vaultDir + '/Attachments/' + newFilePath,'wb') as output_file:
                        output_file.write(r.content)
                except AttributeError: # This is to prevent the AttributeError exception when no matches are returned
                    continue
                # Save Markdown file with new local file link as a temp file
                # If there is already a temp version of a file, open that.
                fullTempFilePath = vaultDir + '/temp_' + file
                if os.path.exists(fullTempFilePath):
                    fullRead = open(fullTempFilePath, errors='ignore')
                else:
                    fullRead = open(fileFullPath, errors='ignore')
                data = fullRead.read()
                data = data.replace(firebaseUrl,newFilePath)
                fullRead.close()
                with open(fullTempFilePath, 'wt') as temp_file:
                    temp_file.write(data)
                    i = i + 1
        # Close file
        fhand.close()

        #Had to move this out of the main loop to get it to work on Windows os.replace() was throwing an error
        if os.path.exists(fullTempFilePath):
                #This renames the temp file name with the original file, deleting the original file, but produces the following error in Windows: "PermissionError: [WinError 5] Access is denied"
            path = os.replace(fullTempFilePath,fileFullPath)

This issue is resolved.