encountering various errors with other cities
Closed this issue · 8 comments
Nice script - thought I'd test it with some other cities, but ran into 3 errors you might want to be aware of :)
Nortorf
>>> python arnis.py --city "Nortorf" --state "Schleswig Holstein" --country "Deutschland" ...
Fetching data...
Chosen server: https://overpass.kumi.systems/api/interpreter
Parsing data...
Processing data...
Traceback (most recent call last):
File "C:\path\arnis.py", line 592, in <module>
imgarray = processData(rawdata)
File "C:\path\arnis.py", line 350, in processData
if (imgLanduse[i[1]][i[0]] == 0): imgLanduse[i[1]][i[0]] = landuseType
IndexError: index 8846204 is out of bounds for axis 0 with size 4818
Gnutz
python arnis.py --city "Gnutz" --state "Schleswig Holstein" --country "Deutschland" ...
Fetching data...
Chosen server: https://maps.mail.ru/osm/tools/overpass/api/interpreter
Parsing data...
Processing data...
Traceback (most recent call last):
File "C:\path\arnis.py", line 592, in <module>
imgarray = processData(rawdata)
File "C:\path\arnis.py", line 504, in processData
if (str(img[y][x][0])[:1] != 5 and str(img[y][x][0])[:1] != 6 and str(img[y][x][0])[:1] != 7): img[i[1]][i[0]] = int("2" + str((wallHeight + 1)))
UnboundLocalError: local variable 'y' referenced before assignment
Kiel tries to request 590 GiB of memory - didn't happen with Hamburg, maybe it didn't get that far though (ran into the same IndexError as Nortorf).
python arnis.py --city "Kiel" --state "Schleswig Holstein" --country "Deutschland" ...
Fetching data...
Chosen server: https://overpass-api.de/api/interpreter
Parsing data...
Traceback (most recent call last):
File "C:\path\arnis.py", line 592, in <module>
imgarray = processData(rawdata)
File "C:\path\arnis.py", line 199, in processData
img = np.zeros((round((greatestElementY - lowestElementY) / resDownScaler) + 5, round((greatestElementX - lowestElementX) / resDownScaler) + 5, 1), np.uint8)
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 590. GiB for an array with shape (1113821, 568481, 1) and data type uint8
Hi there, thanks for opening an issue! Just fixed the Gnutz and Kiel issue.
The Nortorf crash is most likely caused by two cities with the same name in the same state. I already had that problem during development and it took me quite a while to figure out haha! That's why I included the state tag, which had the purpose of minimizing the chance to get two different cities with the same name and a large distance between them returned. I'm not 100% sure tho if this really causes the crash in this case, but I think so. For now I can implement another check which should avoid that exact error to occur again. Will do that later!
$ python3 arnis.py --city Arnis --state "Schleswig Holstein" --country "Deutschland" --path ~/.minecraft/saves/asdf
Fetching data...
Chosen server: https://maps.mail.ru/osm/tools/overpass/api/interpreter
Parsing data...
Traceback (most recent call last):
File "/home/sparr/src/arnis/arnis.py", line 571, in <module>
imgarray = processData(rawdata)
File "/home/sparr/src/arnis/arnis.py", line 178, in processData
img = np.zeros((round((greatestElementY - lowestElementY) / resDownScaler) + 5, round((greatestElementX - lowestElementX) / resDownScaler) + 5, 1), np.uint8)
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 4.94 TiB for an array with shape (993759, 5466422, 1) and data type uint8
I get the memory allocation problem even with the documented and namesake city. Changing resDownScaler = 100
to resDownScaler = 1000
eliminates the problem by presumably reducing the memory allocation attempt by a factor of 10x10=100 to a barely manageable 49.4GiB, although I wasn't willing to wait for it to run to completion. Changing it to 3000 for another 9x reduction to fit in my available RAM made it run quickly, although I can't confirm it had any effect on my game world.
That's definitely an unexpected behavior. I just tried several ways to reproduce this issue on multiple machines but none of them run into the same problem.
Can you run the script again with the --debug parameter and send me the arnis-debug-raw_data.json file over e.g. pastebin?
After update to current main I cannot reproduce that error.
I will leave my found issue here as well:
This error occurs for Augsburg, Bayern, Deutschland
Traceback (most recent call last):
File "C:\Users\andil\Desktop\arnis\arnis.py", line 10, in <module>
run()
File "C:\Users\andil\Desktop\arnis\src\main.py", line 119, in run
imgarray = processData(rawdata, args)
File "C:\Users\andil\Desktop\arnis\src\processData.py", line 162, in processData
and int(element["tags"]["building:levels"]) <= 8
ValueError: invalid literal for int() with base 10: '1.5'
Just released a hotfix for this problem. Thanks for the report!
Wien, Österreich
Traceback (most recent call last):
File "/home/.../Desktop/arnis/arnis.py", line 10, in
run()
File "/home/.../Desktop/arnis/src/main.py", line 123, in run
imgarray = processData(rawdata, args)
File "/home/.../Desktop/arnis/src/processData.py", line 199, in processData
and int(float(element["tags"]["building:levels"])) <= 8
ValueError: could not convert string to float: 'EG'
@savvasenok Pushed a hotfix, should be working now. I hope that Wien doesn't make a big problem with its size though. Thanks for the report!