`basictool` output doesn't match BASIC IV

[tom-seddon]

An initial REM with two consective 0a bytes, that basictool turns into a single 0a. Judging by the output of *SPOOL on my real Master, BASIC calls OSWRCH for both 0a bytes, so I assume they should both be emitted.

Zip attached. There are four files in the zip:

• .BBCBasicToText.txt - output from BBCBasicToText.py, which I'm trying to make match output from the real BASIC ROM
• .basictool.txt - output from basictool --input-tokenised --ascii
• .original.dat - the original file
• .stripped.dat - just the BASIC part of the original file, excluding anything past TOP (in this case there's an additional copyright string after the BASIC)

53ae23f87c28171e1b39a9f02107d32810ca09bfb32653f0576c30744bbb0268.zip

[ZornsLemma]

Thanks Tom! I think this is an easy fix - the state machine used to decide what output to allow through from the emulated machine to the host was stripping blank lines, so I've tweaked it to not do that when it's doing a LIST internally. I had no thought of such "crazy" things as embedded LF codes inside the BASIC when I originally wrote basictool, but it's good to handle them correctly all the same.

I've pushed the fix onto the https://github.com/ZornsLemma/basictool/tree/issue-4 branch, please give that a try and let me know how you get on. With this I seem to get byte-for-byte identical output to your .BBCBasicToText.txt file.

[ZornsLemma]

There is maybe a grey area here - for ASCII output, the file is fopen()-ed in mode "w", not "wb". On our Unix-like systems there won't be any translation of 0xa codes going on, but someone trying to do what you're trying to do here on Windows would probably run into problems with the C library's translation. I don't know if this is a problem or not, and I'm not entirely sure what to do about it, except offering a --binary-output option to force "wb" mode even for ASCII.

(It's even more fiddly if we're outputting to stdout. AFAICS there's no portable way to reopen stdout as binary; https://stackoverflow.com/questions/16888339/what-is-the-simplest-way-to-write-to-stdout-in-binary-mode suggests wrapping a Windows-specific call in a #ifdef, which I guess isn't the end of the world even though I was trying to keep the code as platform-independent as possible.)

[tom-seddon]

Thanks - issue-4 seems to have fixed this.

The binary issue is indeed a potential problem, though I'm testing on macOS so I can manage. I wouldn't worry about the output-to-stdout case - if it doesn't work, it doesn't work.

[ZornsLemma]

I've added an --output-binary option (on the issue-11 branch, because that was easiest...), although it's only had the most cursory testing since I'm on Linux. I've followed your advice to not worry about stdout for now, although there is (and was) a TODO in the code so maybe I'll come back to this aspect later.

[tom-seddon]

I should be able to run my tests on Windows at some point in the next week, and I'll report back.

[ZornsLemma]

Thanks Tom, it would be good to get confirmation this works.

[tom-seddon]

I think #13 closed this off.