GitHubRGI/geopackage-python

Race condition found

davidintelinair opened this issue · 0 comments

using linux 64

There is a race condition in gdal2tiles_parallel at line ~1866

# Create directories for the tile
if not path.exists(path.dirname(tilefilename)):
    makedirs(path.dirname(tilefilename))

During the generation of the overview tiles, two of the sub processes could run into a well known race condition and cause a FileExistsError exception. There are two possible solutions... Since our private fork has some other changes and we are python 3 only, we fixed this with
makedirs(path.dirname(tilefilename), exist_ok=True )
It is debatable that you could also use something like

if not path.exists(path.dirname(tilefilename)):
    try:
        makedirs(path.dirname(tilefilename))
    except OSError as exception:
        if exception.errno != errno.EEXIST or not os.path.isdir(path):
            raise

There is a long discussion here about the pros and cons of various solutions. Since I am a neophyte python programmer, I defer to those more knowledgeable:

NOTE This will also allow for a single execution of the main path by removing the second call to Main()

if __name__ == '__main__':
    main(None)
    main(None)

becomes

if __name__ == '__main__':
    main(None)