Prompt wraps incorrectly in Bash 5.1

[ben-foxmoore]

Highly related to the issue being discussed towards the end of #547: The prompt wraps incorrectly when running in a clean Bash environment.

It's difficult to describe this issue with much more detail - I've been using Liquidprompt for a number of months without this issue, but something appears to have changed recently which has raised the problem. I'm running Bash on an Arch Linux-based WSL2 install and have tested with both the latest released version of Liquidprompt (2.1.2) as well as the latest on master. In both cases, as soon as I activate liquidprompt, the wrapping problem appears.

Shell: bash 5.1.16(1)-release
Operating system: Linux Nomos 5.15.90.1-microsoft-standard-WSL2, x86_64, Arch Linux
Liquid Prompt version: 9d02aeb
Using Microsoft/Windows Terminal

Steps to Reproduce

$ env -i $(which bash) --norc --noprofile
bash-5.1$ export TERM=xterm-256color
bash-5.1$ export LANG=en_US.UTF-8
bash-5.1$ source /usr/bin/liquidprompt
⏚ [ben:~/home/ben] $ echo $COLUMNS
149
⏚ [ben:~/home/ben] $ # type more characters than remaining columns

Screenshot of this occuring:

Expected Behavior

Text should wrap to the next line rather than overwriting the start of the prompt on the current line. This eventually happens once the entire first line has been (over)written for a second time:

[Rycieos]

Thanks for the report. I'm completely willing to help track this issue down with you (and fix it if possible), but it will probably take a while; so be ready for a long haul.

Good job on isolation so far, that eliminates a lot of the possible issues.

The first step is to track down a minimum reproduction PS1. Here is the process to follow:

1. Print out your current PS1: printf '%q\n' "$PS1"
2. Turn off Liquid Prompt: prompt_OFF
3. Set PS1 to a substring of the original PS1, something like PS1=$'\\[\E[1m\E[33m\\]1&\\[\E(B\E[m\\] [user'
4. Test if the issue still exists.
5. Go to step 3 until you have a very short string that still has the issue.

Likely, there is one of two things causing this:

1. A Unicode character, like the ⏚ symbol at the start of your prompt.
2. A terminal formatting sequence (which Bash is supposed to ignore using the formatting escapes) is not being ignored correctly. Make sure in your above testing that you keep the \\[ and \\] sequences surrounding the formatting sequences to avoid invalidating the test.

And of course anything else you can narrow down, or prove it's not really a variable will help.

[ben-foxmoore]

@Rycieos thanks for the support - I'm happy to try and work through this together with you as well 😄

As you suggested, it seems to be related to the Unicode character (I think I actually suspected that when testing the other day, but forgot to mention it in the issue).

The PS1 I had in the screenshots above is

$'\\[\E[32m\\]\342\217\232\\[\E(B\E[m\\] [ben\\[\E(B\E[m\\]\\[\E[32m\\]:\\[\E(B\E[m\\]\\[\E(B\E[m\\]~\\[\E(B\E[m\E[90m\\]/\\[\E(B\E[m\\]home\\[\E(B\E[m\E[90m\\]/\\[\E(B\E[m\E[1m\\]ben\\[\E(B\E[m\\]] \\[\E[35m\\]130\\[\E(B\E[m\\] \\[\E[1m\\]$\\[\E(B\E[m\\] '

Removing the ⏚ symbol from the definition immediately resolves the issue

[ben:~/home/ben] 130 $ PS1=$'[ben\\[\E(B\E[m\\]\\[\E[32m\\]:\\[\E(B\E[m\\]\\[\E(B\E[m\\]~\\[\E(B\E[m\E[90m\\]/\\[\E(B\E[m\\]home\\[\E(B\E[m\E[90m\\]/\\[\E(B\E[m\E[1m\\]ben\\[\E(B\E[m\\]] \\[\E[35m\\]130\\[\E(B\E[m\\] \\[\E[1m\\]$\\[\E(B\E[m\\] '
[ben:~/home/ben] 130 $ aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaa

and correspondingly setting the PS1 just to that symbol still has the same problem

 $ PS1=$'\\[\E[32m\\]\342\217\232\\[\E(B\E[m\\] '
⏚
⏚
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

(The last line had the ⏚ prompt but was overwritten)

[Rycieos]

Note your final test has both colors/formatting ($'\\[\E[32m\\]\\[\E(B\E[m\\] ') and a Unicode non-standard width character (⏚). A better test would be to try them by themselves.

Try all of these combinations:

PS1=$'\342\217\232 '
PS1=$'\\[\E[32m\\]$ '
PS1=$'\\[\E[32m\\]$\\[\E(B\E[m\\] '
PS1=$'\E[32m$\E(B\E[m '

Note: the last one should break.

If you can test with other Bash versions and other terminal emulators that would help as well.

[ben-foxmoore]

Oh, good point. The second one from that list wraps/works properly, but the rest all exhibit the same problem.

[Rycieos]

Well that's not expected. Let's stick with the first one for now. Can you test with any other environment? Different Bash version, terminal emulator, OS, anything?

[ben-foxmoore]

I've just tested with Mobaxterm rather than Windows Terminal - although Mobaxterm isn't rendering the ⏚ properly, the behaviour is still wrong. (For completeness, I tested all 4 prompts, and again only the second one wraps properly.)

I'm not sure what other options I have on Windows for alternative emulators..

I'll try with a different Bash version, but it might take some time to take an older PKGBUILD from Arch's repos and get it built alongside the current version.

[Rycieos]

Another terminal option is Mintty (bundled with Cygwin, GitBash, and others). It's my emulator of choice on Windows.

I agree; building a specific version of Bash is a giant pain. I would never attempt to do such a thing on Windows. I'll try building that specific Bash version on Linux (Rocky) and test on Mintty over SSH.

[Rycieos]

I tested with Bash 5.1.16(1)-release (over SSH) and 5.2.15(3)-release locally, on both Mintty and Windows Terminal, and I can not reproduce this. I'm not exactly sure what could be causing this now.

Maybe the WSL layer is causing some weirdness here? If you could eliminate that layer somehow, maybe with Cygwin or Git Bash and running Bash directly from Windows Terminal, maybe that would prove something. Testing with a different version of Bash might also provide some insight.

[ben-foxmoore]

I'm no longer seeing this behaviour so something has changed over the last few months. I believe I'm still using the same version of liquidprompt so the bug and related fixes were likely elsewhere all along. Now on these versions:

Shell: bash 5.2.26(1)-release
Operating system: Linux Nomos 5.15.146.1-microsoft-standard-WSL2, x86_64, Arch Linux
Liquid Prompt version: 9d02aeb
Using Microsoft/Windows Terminal

[Rycieos]

Well, that means it's time to search the patch notes!

I found these line items that may be related:
https://github.com/bminor/bash/blob/f3b6bd19457e260b65d11f2712ec3da56cef463f/CHANGES#L132-L133

Fixed a bug that caused rl_eof_found to be set prematurely while reading a multi-character key sequence in callback mode.

https://github.com/bminor/bash/blob/f3b6bd19457e260b65d11f2712ec3da56cef463f/CHANGES#L198-L199

Fixed an issue with multi-line prompt strings that have one or more invisible characters at the end of a physical line.

https://github.com/bminor/bash/blob/f3b6bd19457e260b65d11f2712ec3da56cef463f/CHANGES#L270-L271

Fixed an off-by-one error that caused a read past the end of a buffer when reading a multibyte character from the output of a command substitution.

https://github.com/bminor/bash/blob/f3b6bd19457e260b65d11f2712ec3da56cef463f/CHANGES#L395-L397

Fixed a display problem that caused the prompt to be wrapped incorrectly if the screen changed dimensions during a call to readline() and the prompt became longer than the screen width.

https://github.com/bminor/bash/blob/f3b6bd19457e260b65d11f2712ec3da56cef463f/CHANGES#L405-L407

Fixed a problem with line wrapping prompts when a group of invisible characters runs to the right edge of the screen and the prompt extends longer then the screen width.

None of these seem very likely to be the cause, but maybe some combination of them caused it. I also checked all 26 patches for 5.2, and none of them seem to be related. I'm going to assume that one of these Bash changes between 5.1 and 5.2 fixed this. Thanks for checking back in!