No Visual highlight with new themes on tty (t_Co=8)

Question

No Visual highlight with new themes on tty (t_Co=8)

sandwm opened this issue 3 years ago · 31 comments

When using vim on a Linux tty and pressing V (or its variants) I noticed there is no visual indication of which parts of the text are selected. My colorscheme is elflord, but other themes are affected too (see first comment below). The issue started about a week ago, when Arch Linux shipped the new vim version (8.2.4981) which replaced the legacy themes with the new ones.

Here is the output of source /usr/share/vim/vim82/syntax/hitest.vim with legacy elflord.vim:

And here it is with the current elflord.vim:

You see that Visual as well as some other highlighting groups which previously had some highlighting applied are now missing it.

For better reproducibility and to make sure it's not related to my personal setup, I reproduced the issue with a fresh Arch Linux virtual machine: Get the iso image from https://archlinux.org/download/, start the VM from the iso and run:

pacman -Sy vim vim-runtime  # update vim packages
vim /etc/vimrc # add 'colorscheme elflord'
mkdir -p ~/.vim/colors/
cd ~/.vim/colors/
curl -o elflord.vim https://raw.githubusercontent.com/vim/colorschemes/master/legacy_colors/elflord.vim
# first image above produced by these two commands
vim -c 'source /usr/share/vim/vim82/syntax/hitest.vim'
vim -u NONE -c 'colorscheme elflord' -c 'source /usr/share/vim/vim82/syntax/hitest.vim'
rm elflord.vim
# second image above produced by these two commands
vim -c 'source /usr/share/vim/vim82/syntax/hitest.vim'
vim -u NONE -c 'colorscheme elflord' -c 'source /usr/share/vim/vim82/syntax/hitest.vim'

There was no relevant difference in the output between including -u NONE -c 'colorscheme elflord' or just using the /etc/vimrc (apart from having a minimal status line or not).

The first command (pacman) updates the vim packages so they contain the new themes. This won't be necessary once the 2022.06.01 image is released, which will already contain a newer vim version. If you want to see the very minimal /etc/vimrc and archlinux.vim, which is sourced by the former, you can find them here: https://github.com/archlinux/svntogit-packages/tree/packages/vim/trunk

Of course apart from the pacman command all other steps should also work on any other distro and a recent vim.

Answer 1 · 2022-05-29T16:54:51.000Z

Additional info:

:set t_Co? gives t_Co=8 on the tty
/usr/share/vim/vim82/colors/elflord.vim is identical to https://github.com/vim/colorschemes/blob/master/colors/elflord.vim
The issue affects almost all themes shipped by vim 8.2.4981, except default.
- These have no highlighting for Visual, as well as several other groups:
  - blue
  - delek
  - desert
  - elflord
  - evening
  - industry
  - morning
  - murphy
  - pablo
  - peachpuff
  - shine
  - slate
  - torte
  - zellner
- These indeed have highlighting for Visual, but not for several other groups (not sure if that's a bug, too):
  - darkblue
  - koehler
  - ron

Note that I didn't do a full comparison for the non-elflord themes of legacy vs. new version, but just ran this in /usr/share/vim/vim82/colors/:

for file in *.vim; do vim -u NONE -c "colorscheme ${file%.vim}" -c "source /usr/share/vim/vim82/syntax/hitest.vim"; done

Answer 2 · 2022-05-29T17:06:55.000Z

Thanks for the report, @sandwm

We aimed for GUI > 256 > 16 > 0 and for some reason missed 8.

I believe, this has to be fixed

Answer 3 · 2022-05-29T18:32:34.000Z

Can you set it to use 16 colors? What do you mean with "Linux tty", what is 'term' set to?

Answer 4 · 2022-05-29T21:54:46.000Z

Can you set it to use 16 colors?

If memory serves, doing set t_Co=16 in Vim does the job.

What do you mean with "Linux tty", what is 'term' set to?

Another name for what OP is talking about is "virtual console" or "virtual terminal" or "VT". You can get there from your desktop environment with Ctrl+Alt+F7, Ctrl+Alt+F8, etc. $TERM is usually linux, there.

Answer 5 · 2022-05-29T22:25:22.000Z

Yes, like @romainl said, I meant a virtual console like tty1, tty2, ... accessible via Ctrl+Alt+F1, Ctrl+Alt+F2, ...
$TERM is linux.
:set t_Co=16 works.

Answer 6 · 2022-05-30T11:25:58.000Z

Well, you can set it to 16 colors, but it looks like it really only supports 8.
With the default colors Visual uses background 8, which is just black and not grey.
With 8 colors it uses reverse, which does work. Thus 16 colors is wrong.

Looks like the color schemes should support 8 colors, there is no workaround.

Answer 7 · 2022-05-30T13:03:36.000Z

In the virtual console, we can…

use colors 0-7 for fg and bg (red, darkred, lightred are all synonyms),
use colors 0-7 for fg with bold to obtain brighter colors,
get white text with ctermfg=white cterm=bold.

and we can't…

use colors 8-15 directly for anything,
have a white bg.

which means that…

we have 16 colors at our disposal for fg
white is only available for text,
there is only one shade of grey for bg and it is quite light

Anything else I missed?

Answer 8 · 2022-05-30T13:15:53.000Z

have a white bg.

What would we do with delek, peachpuff, morning, shine, zellner?

Answer 9 · 2022-05-30T14:10:32.000Z

What would we do with delek, peachpuff, morning, shine, zellner?

Either accept to have it light grey instead of white or (╯°□°)╯︵ ┻━┻.

Answer 10 · 2022-05-30T14:22:30.000Z

Either accept to have it light grey

Ah, light grey is available, this might work

Answer 11 · 2022-06-04T18:18:41.000Z

To track status:

pablo
industry
elflord
blue
darkblue
delek
desert
evening
morning
murphy
peachpuff
shine
slate
torte
zellner
koehler
ron

Answer 12 · 2022-06-08T12:18:17.000Z

@romainl pls check light bg colorschemes

PS, I have to do other things so the last one is on you :)

Answer 13 · 2022-06-08T12:39:44.000Z

@habamax thank you for all that great work.

Answer 14 · 2022-06-08T14:26:46.000Z

Thanks to both of you! That looks much better in several places. :)

Unfortunately there's also a regression: SpellBad, SpellCap, SpellRare, SpellLocal lost all highlighting, compared to what's in vim 8.2.5046. (Actually, for SpellLocal that's not really a regression, since previously it had fg color = bg color.) I don't really mind, since I don't do spell checking in a tty, but wanted to note it nonetheless. Checked with elflord and pablo.

Answer 15 · 2022-06-08T14:32:11.000Z

@sandwm it actually shouldn't. Any exact colorscheme or all of them?

Answer 16 · 2022-06-08T14:34:45.000Z

This is shine:

desert:

evening

etc

Answer 17 · 2022-06-08T14:36:05.000Z

Ah, elflord and pablo:

elflord:

pablo:

Answer 18 · 2022-06-08T14:38:30.000Z

Aha, if it is in tty (or whatever it is called in Ctrl+Alt+F3) it is indeed have no highlighting.

Not sure how to tackle it...

Answer 19 · 2022-06-08T14:40:48.000Z

I can't make screenshots there :)

But having those 3 lines I have nothing only for "Normal" (first line), everything else is highlighted.

Answer 20 · 2022-06-08T14:47:45.000Z

underline makes it all the same color and for elflord it is the same as normal text.

We could remove it for 8c completely... @romainl what do you think?

Answer 21 · 2022-06-08T14:54:35.000Z

Yes, my comment was about vim in a tty with t_Co=8. I made screenshots from a virtual machine (vim with source /usr/..../syntax/hitest.vim).

Before:

After:

I think underline isn't supported there anyhow, so that could be removed. I'm gonna check.

Answer 22 · 2022-06-08T14:55:16.000Z

@sandwm could you pls check if

:hi SpellRare cterm=NONE
:hi SpellLocal cterm=NONE
:hi SpellBad cterm=NONE
:hi SpellCap cterm=NONE

Make it properly highlighted?

Answer 23 · 2022-06-08T14:56:41.000Z

I think underline isn't supported there anyhow, so that could be removed. I'm gonna check.

We can't probably detect if it is tty or xterm with 8c support, so if we are to remove it -> it would be for all terminals.

Answer 24 · 2022-06-08T14:57:26.000Z

And we can make it reverse though...

Answer 25 · 2022-06-08T15:02:46.000Z

@sandwm could you pls check if
* `:hi SpellRare cterm=NONE`

* `:hi SpellLocal cterm=NONE`

* `:hi SpellBad cterm=NONE`

* `:hi SpellCap cterm=NONE`
Make it properly highlighted?

Yes, that works.

We can't probably detect if it is tty or xterm with 8c support, so if we are to remove it -> it would be for all terminals.

Shouldn't xterm have more colors? For me :set t_Co? in an actual xterm gives 256 (and underlines work).

Answer 26 · 2022-06-08T15:14:45.000Z

I think underline isn't supported there anyhow, so that could be removed. I'm gonna check.

Yes, no underline in tty (on a color display – so any display relevant here).

cterm=reverse works, although I think personally I would prefer just changing the color. That's more readable and a word which is supposedly spelled wrong (even if it's just a term the dictionary doesn't know) doesn't need the highlighting of e.g. an ErrorMsg.

Answer 27 · 2022-06-08T15:50:29.000Z

We can't probably detect if it is tty or xterm with 8c support, so if we are to remove it -> it would be for all terminals.

Shouldn't xterm have more colors? For me :set t_Co? in an actual xterm gives 256 (and underlines work).

Ah, looks like some older xterms indeed only supported 8 colors: https://stackoverflow.com/questions/10003136/what-is-the-difference-between-xterm-color-xterm-256color/10039347#10039347
I don't know whether they supported underline.

Side note: Even with export TERM=xterm-color (for 8 color xterm) which also leads to tput colors reporting 8, in vim I get t_Co=256 (which is indeed what this xterm supports).

Answer 28 · 2022-06-08T16:35:08.000Z

I will have a look into this once everything is merged in

Answer 29 · 2022-06-11T08:40:12.000Z

I couldn't find the way in templates how to clear the default reverse, e.g. if I remove underline then spell highllghts are reverse by default (at 8c).

So I made them reverse explicitly.

Answer 30 · 2022-06-11T11:40:13.000Z

All of the colorschemes have 8c support now.

@romainl please check the latest darkblue and it might be a good idea to create PR against main vim repo to include them before vim9 is out

Answer 31 · 2022-06-12T20:13:15.000Z

vim/vim#10563