Ymodem-G issues
Closed this issue · 21 comments
I still have issues with ymodem-g. I tried uploading files with native MysticBBS Ymodem, the result I get is:
"Expected (N)ACK got 24"
When using SEXYZ Ymodem-G the transfer stalls at 100%.
Could you try to upload a file by one of these protocols on bbs.theforze.eu with ftelnet? Or use the http url, bbs.theforze.eu. Thank you in advance! And keep up the great work.
Just to be sure I just tried https://embed-v2.ftelnet.ca/ also. Same result on my board.
Can you try uploading a text file to see if that completes? And if it does, try removing any 0xFF bytes from the zip file to see if that completes?
It's been awhile since I worked on the ymodem code, but I do remember the IAC expansion causing issues way back when, so I'm wondering if that's what's to blame here.
I tried uploading an UTF-8, CP437 and IMB855 .txt file with the same result.
When I try a very small .txt file with the following content
APDZF installateur code
I received this error:
When the upload fails the whole session becomes very sluggish. Like there is some major activity on the background.
I just tried to download a file with both internal Mystic Ymodem-g and SEXYZ Ymodem-G the result for both is bad blocks:
The good news is, I'm able to reproduce the problem here. The bad news is, I'm not sure what's going wrong. My totally wild guess is that the sending side is overwhelming the receiving side by writing all the data as fast as possible, but I'm not really a networking guy so I'm not sure whether that's a plausible explanation.
Either way, disabling "G" mode so it waits for an ACK after sending each block seems to work, so that may be an option. The only drawback is that it'll be slower, but I guess a slower working implementation is better than a faster broken one!
Hopefully I'll get an update out later this weekend...need to clean up the code a bit, and also switch the receiving code to ACK as well.
I've done some additional testing, and with Synchronet and its bundled websocketservice.js proxy, things are working great (after some tweaks to the proxy).
Using fTelnetProxy things are still broken though, so I'll need to do some troubleshooting with that tomorrow.
I also cleaned up the "plain YMODEM" code and pushed it in a branch here if you want to test it out: https://github.com/rickparrish/fTelnet/tree/ymodem
Hopefully once fTelnetProxy is fixed the non-G code won't be needed though.
@opicron Just updated my public proxies with a new fTelnetProxy build, which should fix the YMODEM-G issues (uploading text and binary files both work on Synchronet+sexyz for me now).
I confirm that Ymodem-G upload works on my board through ftelnet with proxy! Great, thank you very much.
Am I correct to read between the lines that download is not working yet? For me that fails still.
I just registered and was able to see what you mean. I captured the header packet so will take a look to see why fTelnet isn't liking it.
I also tried with SEXYZ, but it aborted before any data transfer took place. Does that one work for you?
Looks like Mystic is not escaping the telnet IAC character. The opening header sequence should be:
0x01 (indicates 128 byte block), 0x00 (indicate block number is zero), 0xFF (inverse block number, aka 255 - block number)
Since the inverse block number for block 0 is 255, and 255 is the telnet IAC character, it needs to be doubled-up. Here's the difference between bbs.theforze.eu and bbs.ftelnet.ca:
Downloading file from bbs.theforze.eu
IN: 0x01 0x00 0xFF LORD48.ZIP...truncated rest of header
Downloading file from bbs.ftelnet.ca
IN: 0x01 0x00 0xFF 0xFF LORD48.ZIP...truncated rest of header
I cant seem to make SEXYZ download work. It exits immediately when trying to use STDIO. Will have to find a more recent compiled version, as this one is v2.0. I think v3.x is out but I cannot compile myself.
If the implementation of Ymodem-G in Mystic is incorrect there is nothing we can do. Ill post a message to g00r00 about the Ymodem-G in Mystic. Hopefully this will be fixed ;).
For now thank you very much for your time and effort!
I'm curious, if you use SyncTerm to download using YModem-G, does that work?
It seems if I tag multiple files in Mystic I can use SEXYZ. I just tried and it gives the following error:
Will try SyncTerm now.
Yeah that makes sense SEXYZ would fail too. If I remember right when it failed there was a message about the socket handle not being passed, so it was using STDIO mode, so it would be up to Mystic to escape the IACs, which we know it's not.
Is it possible to have SEXYZ use a socket handle under linux, instead of STDIO mode? I would bet that would do the trick.
If I know how to do that I would. I tried offering the 2222 port which is the internal docker port. But that crashes SEXYZ. Im not familiar how to do it otherwise. I tried the -telnet paramer but it has no effect.
Edit: I found that adding %0 or %H adds the communication handle to the protocol. However under linux it is empty, wheras under DOS it would link to the comport.
EXTERNAL DOORS MYSTIC
%H = The current socket handle (added 1.12 A34)
%0 = The current communication handle (socket handle)
Syncterm error:
Lets leave it at that then. The download doesnt need to be fixed as the issue is not with fTelnet.
Thanks again for your support!
Yeah it's failing for the exact same reason, just with a different output (fTelnet outputs the raw inverse block number, SyncTerm outputs its ones-complement. And the ones-complement of 0xb0 is 79, so they're both complaining about the same thing)
ASCII character 79 is an O, so I think the file you're trying to download has an O as the second character?
Anyway, it's safe to say it's not something on the fTelnet/SyncTerm side, but it's not clear if it's something that Mystic needs to fix, or something else. If Mystic is also running in STDIO mode, then it wouldn't be expected to double-up the IAC characters. Instead, whatever is answering the incoming telnet connection and spawning Mystic in STDIO mode is what would need to do it.
Just for completeness: I just tried to download a ZIP file where the second character is a K. With syncterm it mentions 0xb1 error. I do not remember which was the other file with 0xb0.
With 0xb1 I would have expected the second letter to be an N. For it to be a K it should be 0xb4. That's assuming I'm "reverse engineering" the error correctly, which is entirely possible I'm not!
For anybody reading into this issue: adding -telnet to SEXYZ command argument doubles the Telnet IAC character when it appears in the transmit datastream.
I thought I tried that before, but apparently I am a idiot. Went ahead and requested this functionality from Digital Man who kindly pointed out this is implemented.
Thanks to @rickparrish upload is working, and download has been working all along as far as I could tell.
Thanks for following up, glad it's working now!