Remote CLI combined with Local CLI causes IO Issues

[e-fin]

Long story short, I have a terminal using this library and one of the options connects to a remote CLI listening on some port using this library. I can successfully connect, but once I close the connection to the remote CLI and return to the original prompt it seems to only read half of the keys im pressing. Ive made a small POC instead of sending over all my applications:

Client:
client.txt

Server:
server.txt

to re-create the error compile both binaries and do the following:

root@localhost:/readlineTest/client$ ./server &
root@localhost:/readlineTest/client$ ./client
»
»
»
» help
commands:
...
» remote
readline-remote: help
receive:help
readline-remote: test
receive:test
readline-remote: exit
»

after you type "exit" in the "readline-remote" context and it returns to the original prompt, it will not read every key you type but the prompt still works if you can struggle to get all the characters typed in.

Ive tried many things like properly closing the remote prompt, killing and re-starting the original readline prompt but no luck, the IO always seems screwed up after doing this. Ive tried messing around with goroutines but no luck with that either

Edit: sorry I tried to use code blocks for the actual code but githubs code blocks dont like it when you put code in them apparently....