[Request] FTP Support?
Closed this issue · 15 comments
Me and a couple of friends have a FTP server with a few ROMs, however AEL is not able to launch the ROMs from the server. It seems to only work via SMB.
Would you be able to implement support for FTP?
It depends on how Python file functions can handle them. In the case of SMB files Python file functions handle them transparently.
Also, keep in mind that SMB support in AEL is only for artwork/assets! Launcher ROMs must be locally mounted and the reason is that most emulators/external applications won't support the SMB/FTP protocol.
We've managed to run the ROMs remotely via SMB, but since SMB is inherently an unsecure protocol, we'd prefer to use FTP. However, when we try to launch the ROMs via FTP, we get the following error message: "ROM not found".
Can you tell me more about your configuration? What OS you are using, what emulator, etc.
The ROMs are stored on a ftp server with Debian 8.0.7 running vsftpd.
Clients are a mix of Linux, Windows 10 and Android, all running retroArch 1.4.0, Dolphin 5.0 and KODI 17 rc4.
We use KODI to connect to the FTP server to retrieve data.
We tried this very solution with the Rom Collection Browser add-on,
and it was able to mount and run the games via our FTP.
However we would much prefer to use AEL for this as it's more robust.
I see. The paths of your ROMs in the RCB database are like ftp://server.name/romdir/romname.zip? Also, are you using the Remote Copy patch by sklick?
Correct (we tested this with .sfc and .iso files).
No, we're not using the Remote Copy patch.
We mounted the source in Kodi File Manager. :)
Then I don't understand... if you are calling to Retroarch externally then the ROM file must locally exist: either you have the file on a local filesystem or a remote filesystem is mounted. If you are using Kodi Retroplayer I believe that the Retroplayer cores (or at least some of them) support Kodi's filesystem, so ROMs can be in any remote filesystem Kodi can access.
Can you confirm that you are using Kodi's Retroplayer? If so, in latest AEL from github you can launch ROMs using Kodi Retroplayer. Please give a try.
Alternatively, in Linux you can mount your server locally using FUSE SSHFS, so your ROMs will be on the remote server but you can access them from a locally mounted directory. This will be OK in Linux but won't work in Windows.
EDIT: I just reread that you said you were using external Retroarch and Dolphin... I don't understand how the emulators work with RCB and not making a local copy of the ROM/ISO...
Hi Wintermute.
Let me try to explain.
No we are not using retroplayer. This is a plain kodi installation with the AEL addon. We use RetroArch and Dolphin emulators, which are installed locally.
As for the ROMs, they are stored on a FTP server. We mount the FTP server in kodi.
The source in kodi will look somthing like this:
ftps://username:password@ftpserver.com:990/Path/roms
This will be our remote filesystem mounted in kodi. The only difference is that kodi will read from a FTPS source rather than via SMB.
LAN or WAN, dosen't matter, kodi is able to read from the source.
AEL is able to see the folders and the roms. But when we try to launch a game it throws a error saying "ROM not found"
We know this method should work because we have tested it using a kodi addon called Rom Collection Browser.
However as we mentioned before, Rom Collection Browser is outdated and we would much prefer to use AEL.
Hi @overdox,
No we are not using retroplayer. This is a plain kodi installation with the AEL addon. We use RetroArch and Dolphin emulators, which are installed locally. As for the ROMs, they are stored on a FTP server. We mount the FTP server in kodi.
OK.
The source in kodi will look somthing like this:
ftps://username:password@ftpserver.com:990/Path/roms
This will be our remote filesystem mounted in kodi. The only difference is that kodi will read from a FTPS source rather than via SMB. LAN or WAN, dosen't matter, kodi is able to read from the source.
Yes, Kodi is able to see the file in remote SMB or FTP because it has libraries to access those remotes. However, Kodi does not mount the remote filesystem so, in theory, other applications cannot access files in remotes "mounted" by Kodi. For example, suppose you want to launch Retroarch. The launching command AEL issues is going to be something like this
/home/kodi/bin/retroarch -L /home/kodi/bin/libretro/picodrive_libretro.so ftp://servername/ROMs-genesis/Sonic.smd
So, because Retroarch cannot handle the ftp protocol I believe the ROM must not launch...
We know this method should work because we have tested it using a kodi addon called Rom Collection Browser. However as we mentioned before, Rom Collection Browser is outdated and we would much prefer to use AEL.
Thank you guys for using AEL. One of the motivations to develop AEL was to overcome some of the limitations of RCB, especially for big ROM collections which AEL can handle without trouble.
AEL is able to see the folders and the roms. But when we try to launch a game it throws a error saying "ROM not found".
That check is to prevent the user to launch non-existent ROMs. For example, when you audit a launcher with a No-Intro DAT file the missing ROMs you don't have but are in the DAT file will be added to the launcher as "fake" ROMs.
OK, I have coded a quick patch for you to do your tests. In last Github commit I have disabled aborting launching if ROMs does not exist. This should enable to launch ROMs with filenames like ftp://servername/romfile. Please install this very last version from Github and report if it works or not. If it works I will investigate this matter further. Also, I did some other changes in AEL (I added new metadata fields for ROMs in 0.9.6) so there could be some bugs here and there.
Hi, thanks for giving it a try.
Unfortunately it looks like it diden't do much.
Here is the log from kodi:
(In this case I used the retroarch_debug.exe to see if I could find anything in retroarch's log. It still gives same error with the regular launcher)
17:29:36.790 T:1424 ERROR: AEL INFO : _command_run_rom() Launching ROM in Launcher...
17:29:36.790 T:1424 ERROR: AEL INFO : _command_run_rom() Using Launcher standard arguments
17:29:36.790 T:1424 ERROR: AEL INFO : _command_run_rom() categoryID c3527aae4ebe086f43500b466a37b91f
17:29:36.790 T:1424 ERROR: AEL INFO : _command_run_rom() launcherID caee0d992d1ff6214db83dadc7b5d4f0
17:29:36.790 T:1424 ERROR: AEL INFO : _command_run_rom() romID 95ad73eb35c2377b676440afc0a67718
17:29:36.790 T:1424 ERROR: AEL INFO : _command_run_rom() romfile "ftps://username:password@ftp.com:990/Data/ROMs/Nintendo SNES/Chrono Trigger/Chrono Trigger.sfc"
17:29:36.790 T:1424 ERROR: AEL INFO : _command_run_rom() rompath "ftps://username:password@ftp.com.tv:990/Data/ROMs/Nintendo SNES/Chrono Trigger"
17:29:36.790 T:1424 ERROR: AEL INFO : _command_run_rom() rombase "Chrono Trigger.sfc"
17:29:36.790 T:1424 ERROR: AEL INFO : _command_run_rom() rombasenoext "Chrono Trigger"
17:29:36.790 T:1424 ERROR: AEL INFO : _command_run_rom() romtitle "Chrono Trigger"
17:29:36.790 T:1424 ERROR: AEL INFO : _command_run_rom() application "C:\emulator\retroarch_debug.exe"
17:29:36.790 T:1424 ERROR: AEL INFO : _command_run_rom() apppath "C:\emulator"
17:29:36.790 T:1424 ERROR: AEL INFO : _command_run_rom() romext "sfc"
17:29:36.790 T:1424 ERROR: AEL ERROR: ROM not found "ftps://username:password@ftp.com:990/Data/ROMs/Nintendo SNES/Chrono Trigger/Chrono Trigger.sfc"
17:29:36.791 T:1424 ERROR: AEL INFO : _command_run_rom() raw arguments "-L "C:/emulator/cores/snes9x_libretro.dll" -f "%rom%""
17:29:36.791 T:1424 ERROR: AEL INFO : _command_run_rom() final arguments "-L "C:/emulator/cores/snes9x_libretro.dll" -f "ftps://username:password@ftp.com:990/Data/ROMs/Nintendo SNES/Chrono Trigger/Chrono Trigger.sfc""
17:29:36.799 T:1424 ERROR: AEL INFO : fs_write_Favourites_JSON() File special://profile/addon_data\plugin.program.advanced.emulator.launcher\most_played.json
17:29:36.807 T:1424 ERROR: AEL INFO : _command_run_rom() Launcher is not Kodi Retroplayer.
It still throws the "ROM not found" in the log.
And I coulden't see any error in retroarch.
The thing is, we did use RetroArch as a emunlator for RCB when we got it to launc the ROMs over FTP.
So I don't think the issue is with RetroArch. But I could always take it up with them and ask.
I disabled aborting the launching if the ROM was not found, but it's still printed in the log if the ROM is not found.
Please do a quick test. Open a terminal in Linux or Windows and try to launch Retroarch from there:
C:\emulator\retroarch_debug.exe -L "C:/emulator/cores/snes9x_libretro.dll" -f "ftps://username:password@ftp.com:990/Data/ROMs/Nintendo SNES/Chrono Trigger/Chrono Trigger.sfc"
If I'm correct it shouldn't work... I believe that Retroarch (and Dolphin) don't support ROMs in FTP remotes. Now the question is why it works in RCB. Maybe the remote copy patch was integrated into RCB (the patch is old so maybe Malte merged into official RCB). Then, if RCB copies ROMs from the FTP to a local temporary location... do you observe a delay at launching, like a couple of seconds? Better, try Dolphin with an ISO image on the FTP. Does it launch instantly or you see a delay? (A big ISO image must take some time to transfer). If you can provide a debug log of RCB that also will be helpful to solve the conundrum.
It seems you might be right.
results from running:
C:\emulator\retroarch_debug.exe -L "C:/emulator/cores/snes9x_libretro.dll" -f "ftps://username:password@ftp.com/Data/ROMs/game.zip" >> log.txt 2>&1
Failed to open ftps://username:password@ftp.com/Data/ROMs/game.zip: Invalid argument
RetroArch [INFO] :: Loading content file: ftps://username:password@ftp.com/Data/ROMs/game.zip.
Failed to open ftps://username:password@ftp.com/Data/ROMs/game.zip: Invalid argument
RetroArch [ERROR] :: Could not read content file "ftps://username:password@ftp.com/Data/ROMs/game.zip"
I will take the issue up with the retroarch team and see if they have a suggestion or possible solution.
I guess it is possible that RCB did something to copy the files locally. We only did testing with fairly small roms and we couldent see any noticeable delay.
Anyways, thanks for the help. My apologies for wasting your time with this :)
OK. Now things start to make sense. A few comments...
a) I don't think Retroplayer coders will add support for FTP remotes. If they do, then some other people will say "Why you don't add protocol XXX?". I think coders don't want Retroplayer to be bloated with libraries to access remotes.
b) Copy a ROM from a remote, like RCB does, could be practical for console systems. However, for MAME and for ISO images is not. In the case of MAME, many ROMs have dependencies like BIOS and devices, so RCB must copy the dependencies as well. Otherwise, the game won't work. For ISO images the reason is obvious: if you are connected with a wired ethernet it will take a few seconds but if you are on wireless you will have to wait minutes to download the ISO image, every time you want to play!
c) You can try Kodi Ledia which includes Retroplayer. I don't know if it's very stable at the moment or not. Retroplayer cores support virtual filesystem, which means you ROMs can be located in any filesystem Kodi understands, like SMB, FTP, SSHFS, etc.
d) In LINUX ONLY: you can mount the remote using SSH. Google for FUSE SSHFS. You can even use autofs to do the mount on demand, just when's needed.
e) In LINUX and Windows: AEL is able to pass the launching application, the arguments and the ROMs to the launching script. Instead of calling the emulator, you create a wrapper script that uses wget (for example) to download the ROM from the FTP to a local temp file and then launch the emulator.
An update of the situation for you guys.
Kodi Leia will have Retroplayer integrated and will support Kodi virtual filesystem (VFS). In other words, you will be able to have your ROMs and assets in any remote filesystem Kodi supports (SMB, FTP, SSHFS, HTTP, NFS, etc.). AEL will of course properly support Kodi Leia in the future but I guess for some months (until a Leia release candidate is ready) I won't start working on it. From now on until Leia RC is released I will focus on Krypton only.
Well this is good news for us, we will just have to put our project on ice for now. Thanks again for the help.