Scraping roms assets/metadata broken
Closed this issue · 3 comments
Hybrid512 commented
Hi,
I'm trying the make AEL work on Kodi Matrix with your latest Python3 branch but I'm still facing some issues.
I just added a new standalone launcher (for Dolphin) as I used to do with Kodi Leia/AEL, creating the launcher works as expected but scanning new roms fails when using TheGamesDB for both assets and metadata.
I enabled debugging and I get this in the logs :
2021-05-29 21:50:54.178 T:6023 INFO <general>: AEL INFO : ============================== Processing ROMs ===============================
2021-05-29 21:50:54.178 T:6023 INFO <general>: AEL DEBUG: ------------------------------ Processing cached file -------------------
2021-05-29 21:50:54.178 T:6023 INFO <general>: AEL DEBUG: ROM.getPath() "/data/emulation/ROMS/gc/1080: Avalanche.iso"
2021-05-29 21:50:54.178 T:6023 INFO <general>: AEL DEBUG: ROM.getOriginalPath() "/data/emulation/ROMS/gc/1080: Avalanche.iso"
2021-05-29 21:50:54.179 T:6023 INFO <general>: AEL DEBUG: Expected 'iso' extension detected
2021-05-29 21:50:54.180 T:6023 INFO <general>: AEL DEBUG: ROM does not belong to a multidisc set.
2021-05-29 21:50:54.180 T:6023 INFO <general>: AEL DEBUG: File not in launcher ROM list. Processing...
2021-05-29 21:50:54.180 T:6023 INFO <general>: AEL DEBUG: FilterROM::ROM_is_filtered() Testing "1080: Avalanche"
2021-05-29 21:50:54.180 T:6023 INFO <general>: AEL DEBUG: ScrapeStrategy.scanner_process_ROM_begin() Determining metadata and asset actions...
2021-05-29 21:50:54.180 T:6023 INFO <general>: AEL DEBUG: NFO file NOT found "/data/emulation/ROMS/gc/1080: Avalanche.nfo"
2021-05-29 21:50:54.180 T:6023 INFO <general>: AEL DEBUG: Metadata policy: Read NFO file OFF | Scraper ON
2021-05-29 21:50:54.180 T:6023 INFO <general>: AEL DEBUG: Metadata policy: Using metadata scraper TheGamesDB
2021-05-29 21:50:54.180 T:6023 INFO <general>: AEL DEBUG: assets_search_local_cached_assets() Searching for ROM local assets...
2021-05-29 21:50:54.181 T:6023 INFO <general>: AEL DEBUG: Missing Fanart
2021-05-29 21:50:54.181 T:6023 INFO <general>: AEL DEBUG: Missing Banner
2021-05-29 21:50:54.181 T:6023 INFO <general>: AEL DEBUG: Missing Clearlogo
2021-05-29 21:50:54.181 T:6023 INFO <general>: AEL DEBUG: Missing Title
2021-05-29 21:50:54.181 T:6023 INFO <general>: AEL DEBUG: Missing Snap
2021-05-29 21:50:54.181 T:6023 INFO <general>: AEL DEBUG: Missing Boxfront
2021-05-29 21:50:54.181 T:6023 INFO <general>: AEL DEBUG: Missing Boxback
2021-05-29 21:50:54.181 T:6023 INFO <general>: AEL DEBUG: Missing 3D Box
2021-05-29 21:50:54.181 T:6023 INFO <general>: AEL DEBUG: Missing Cartridge
2021-05-29 21:50:54.181 T:6023 INFO <general>: AEL DEBUG: Missing Flyer
2021-05-29 21:50:54.181 T:6023 INFO <general>: AEL DEBUG: Missing Map
2021-05-29 21:50:54.181 T:6023 INFO <general>: AEL DEBUG: Missing Manual
2021-05-29 21:50:54.181 T:6023 INFO <general>: AEL DEBUG: Missing Trailer
2021-05-29 21:50:54.181 T:6023 INFO <general>: AEL DEBUG: Asset policy: Local images OFF | Scraper ON
2021-05-29 21:50:54.181 T:6023 INFO <general>: AEL DEBUG: Scraping Fanart with TheGamesDB.
2021-05-29 21:50:54.181 T:6023 INFO <general>: AEL DEBUG: Scraping Banner with TheGamesDB.
2021-05-29 21:50:54.181 T:6023 INFO <general>: AEL DEBUG: Scraping Clearlogo with TheGamesDB.
2021-05-29 21:50:54.181 T:6023 INFO <general>: AEL DEBUG: Scraping Title with TheGamesDB.
2021-05-29 21:50:54.181 T:6023 INFO <general>: AEL DEBUG: Scraping Snap with TheGamesDB.
2021-05-29 21:50:54.181 T:6023 INFO <general>: AEL DEBUG: Scraping Boxfront with TheGamesDB.
2021-05-29 21:50:54.181 T:6023 INFO <general>: AEL DEBUG: Scraping Boxback with TheGamesDB.
2021-05-29 21:50:54.181 T:6023 INFO <general>: AEL DEBUG: Scraper TheGamesDB does not support 3D Box. Local asset not found.
2021-05-29 21:50:54.182 T:6023 INFO <general>: AEL DEBUG: Scraper TheGamesDB does not support Cartridge. Local asset not found.
2021-05-29 21:50:54.182 T:6023 INFO <general>: AEL DEBUG: Scraper TheGamesDB does not support Flyer. Local asset not found.
2021-05-29 21:50:54.182 T:6023 INFO <general>: AEL DEBUG: Scraper TheGamesDB does not support Map. Local asset not found.
2021-05-29 21:50:54.182 T:6023 INFO <general>: AEL DEBUG: Scraper TheGamesDB does not support Manual. Local asset not found.
2021-05-29 21:50:54.182 T:6023 INFO <general>: AEL DEBUG: Scraper TheGamesDB does not support Trailer. Local asset not found.
2021-05-29 21:50:54.182 T:6023 INFO <general>: AEL DEBUG: Getting metadata candidate game
2021-05-29 21:50:54.182 T:6023 INFO <general>: AEL DEBUG: Searching games with scaper TheGamesDB
2021-05-29 21:50:54.182 T:6023 INFO <general>: AEL DEBUG: Scraper._load_disk_cache() Loading cache "candidates"
2021-05-29 21:50:54.182 T:6023 INFO <general>: AEL DEBUG: Cache file not found. Resetting cache.
2021-05-29 21:50:54.182 T:6023 INFO <general>: AEL DEBUG: ROM "/data/emulation/ROMS/gc/1080: Avalanche.iso" NOT in candidates cache.
2021-05-29 21:50:54.182 T:6023 INFO <general>: AEL DEBUG: use_from_cache "False"
2021-05-29 21:50:54.182 T:6023 INFO <general>: AEL DEBUG: Scraper.clear_cache() Clearing caches "1080: Avalanche.iso" "Nintendo GameCube"
2021-05-29 21:50:54.182 T:6023 INFO <general>: AEL DEBUG: Scraper._load_disk_cache() Loading cache "metadata"
2021-05-29 21:50:54.182 T:6023 INFO <general>: AEL DEBUG: Cache file not found. Resetting cache.
2021-05-29 21:50:54.182 T:6023 INFO <general>: AEL DEBUG: Scraper._load_disk_cache() Loading cache "assets"
2021-05-29 21:50:54.182 T:6023 INFO <general>: AEL DEBUG: Cache file not found. Resetting cache.
2021-05-29 21:50:54.182 T:6023 INFO <general>: AEL DEBUG: Scraper._load_disk_cache() Loading cache "internal"
2021-05-29 21:50:54.182 T:6023 INFO <general>: AEL DEBUG: Cache file not found. Resetting cache.
2021-05-29 21:50:54.183 T:6023 INFO <general>: AEL DEBUG: TheGamesDB.get_candidates() search_term "1080 Avalanche"
2021-05-29 21:50:54.183 T:6023 INFO <general>: AEL DEBUG: TheGamesDB.get_candidates() rombase_noext "1080: Avalanche"
2021-05-29 21:50:54.183 T:6023 INFO <general>: AEL DEBUG: TheGamesDB.get_candidates() AEL platform "Nintendo GameCube"
2021-05-29 21:50:54.183 T:6023 INFO <general>: AEL DEBUG: TheGamesDB.get_candidates() TheGamesDB platform "2"
2021-05-29 21:50:54.184 T:6023 INFO <general>: AEL DEBUG: net_get_URL() GET URL "https://api.thegamesdb.net/v1/Games/ByGameName?apikey=***&name=1080+Avalanche&filter[platform]=2"
2021-05-29 21:50:54.433 T:6023 INFO <general>: AEL DEBUG: net_get_URL() Read 475 bytes
2021-05-29 21:50:54.433 T:6023 INFO <general>: AEL DEBUG: net_get_URL() HTTP status code 200
2021-05-29 21:50:54.434 T:6023 INFO <general>: AEL DEBUG: net_get_URL() encoding application/json
2021-05-29 21:50:54.434 T:6023 INFO <general>: AEL DEBUG: TheGamesDB._check_overloading() remaining_monthly_allowance = 2994
2021-05-29 21:50:54.434 T:6023 ERROR <general>: AEL ERROR: SingleInstance::__exit__() Unhandled excepcion in protected code
2021-05-29 21:50:54.434 T:6023 INFO <general>: AEL DEBUG: SingleInstance::__exit__() Releasing lock
2021-05-29 21:50:54.445 T:6023 ERROR <general>: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
- NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
Error Type: <class 'TypeError'>
Error Contents: '>' not supported between instances of 'str' and 'int'
Traceback (most recent call last):
File "/home/kodi/.kodi/addons/plugin.program.advanced.emulator.launcher/addon.py", line 37, in <module>
main.Main().run_plugin(sys.argv)
File "/home/kodi/.kodi/addons/plugin.program.advanced.emulator.launcher/resources/main.py", line 411, in run_plugin
self.run_protected(command, args)
File "/home/kodi/.kodi/addons/plugin.program.advanced.emulator.launcher/resources/main.py", line 505, in run_protected
self._roms_import_roms(args['launID'][0])
File "/home/kodi/.kodi/addons/plugin.program.advanced.emulator.launcher/resources/main.py", line 9551, in _roms_import_roms
scraper_strategy.scanner_process_ROM_begin(romdata, ROM, ROM_checksums)
File "/home/kodi/.kodi/addons/plugin.program.advanced.emulator.launcher/resources/scrap.py", line 599, in scanner_process_ROM_begin
self._scanner_get_candidate(romdata, ROM, ROM_checksums,
File "/home/kodi/.kodi/addons/plugin.program.advanced.emulator.launcher/resources/scrap.py", line 731, in _scanner_get_candidate
candidates = scraper_obj.get_candidates(
File "/home/kodi/.kodi/addons/plugin.program.advanced.emulator.launcher/resources/scrap.py", line 2248, in get_candidates
candidate_list = self._search_candidates(search_term, platform, scraper_platform, st_dic)
File "/home/kodi/.kodi/addons/plugin.program.advanced.emulator.launcher/resources/scrap.py", line 2393, in _search_candidates
candidate_list = self._retrieve_games_from_url(url, search_term,
File "/home/kodi/.kodi/addons/plugin.program.advanced.emulator.launcher/resources/scrap.py", line 2430, in _retrieve_games_from_url
if scraper_platform > 0 and platform == scraper_platform: candidate['order'] += 1
TypeError: '>' not supported between instances of 'str' and 'int'
-->End of Python script error report<--
2021-05-29 21:50:54.522 T:6023 INFO <general>: Python interpreter stopped
Hope this helps.
Best regards, AEL is a great stuff !
Wintermute0110 commented
I think the bug is fixes. Please have a try and if OK close the issue.
WarsTime commented
I had the same bug previously but I confirm, the new py3 version fixes it.
Hybrid512 commented
Confirmed ! Thanks !