How to mitigate highlighting issues when switching colorschemes?
lifepillar opened this issue · 32 comments
As you know, filetype-specific syntax items and colorscheme's switching don't go very well together—the emblematic discussion of the topic likely being altercation/solarized#102.
In a nutshell, the issue can be reproduced as follows (using Vim 8.1.1300):
- Create a minimal colorscheme
~/.vim/colors/test.vim:
set background=dark
hi clear
if exists('syntax_on')
syntax reset
endif
let g:colors_name = 'test'
hi! link vimCommand ErrorMsg
hi vimOption cterm=reverse gui=reverse-
Edit your
vimrc; -
:colorscheme test -
:colorscheme default
You will see that vimCommand is still linked to ErrorMsg and vimOption is cleared.
I have thought hard about possible workarounds and my conclusion is that this issue has no solution—except avoiding using syntax items in colorschemes—unless some new mechanism (a new command?) is introduced in Vim to deal specifically with it. Am I right?
Note that avoiding syntax items in colorschemes is somewhat limiting, as one can realize by looking at two of the most popular colorschemes (Solarized and Gruvbox): people constantly ask to tweak the highlighting for specific filetypes.
I've traced all code paths and there is no way to clear/restore a highlight group so that "hi def" will work as expected- switching colorschemes will always leave problems behind.
This function call would work, but there's no vim script which will use this combination of parameters.
do_highlight(name, TRUE, TRUE); // force init a group
There is no way to restore the highlights to "factory" settings (i.e., after starting vim and enabling syntax).
IMHO, the problem would be easily solved, if:
hi default link {group}could reset the highlight group that is defined in the syntax file, when the group has been cleared. Instead this command fails if the group exists and has been cleared, while this command
hi! default link {group}restores the highlight, but does it even if it's not currently cleared (it totally ignores the default part). It's the default option that is not working as it should (IMHO).
It's not as simple as that, I believe. For that to work, syntax reset (or something else) should be extended to clear all syntax groups Vim currently knows about. That might be easy to implement, I guess. That would also clear highlight groups defined in a user's vimrc: this would break backward compatibility, but Vim might start recommending putting one's customizations in after/syntax/<filetype>.vim rather than in vimrc (in :help hi-default).
But then, Vim should also reload syntax/<filetype>.vim for each filetype with a buffer currently opened before loading the new colorscheme (but after syntax reset), and after/syntax/<filetype>.vim after the colorscheme has been loaded. Failing to do so would result in syntax highlighting be off for most of your code. Or should the user be left responsible for reloading a filetype plugin after changing the colorscheme?
Vim should also reload syntax/.vim for each filetype with a buffer currently opened
vim actually does reload the syntax files for each buffer when the commands :syntax on or :syntax enable are used. See
Line 40 in 110289e
The help says "If you want Vim to overrule your settings with the defaults, use: :syntax on" although this doesn't seem to be completely true.
For what I could understand, the sequence is:
- a colorscheme is loaded (with all groups it defines)
- syntax is loaded, when filetype is applied.
hi def linkfills the gaps, if the colorscheme didn't define some groups. - you load a new colorscheme. All syntax is cleared and reset.
hi def linkthis time won't do anything at all: groups that have been cleared, won't be restored to their default. - the new colorscheme applies its own groups: if some groups had previously been defined, they've been cleared, and if the new colorscheme does not redefine them, they stay cleared. And this is the problem.
If this is right, imho having hi def link (called from the syntax file) to be able to restore a group that has been cleared, would solve the issue. The new colorscheme would then apply its own groups, but default groups would still be working, even if the new colorscheme does not define them.
What would be missing, would be the group that are defined by the old scheme, and that are not defined in the syntax file (hi def link) or in the new colorscheme. These groups would be cleared and stay cleared, but I think it would be a good thing, why should they carry over to schemes that don't define them, if they don't belong to the syntax file either? They belong to that colorscheme only and it's right if they are removed as well.
vim actually does reload the syntax files for each buffer when the commands :syntax on or :syntax enable are used.
But a colorscheme only executes syntax reset. Am I missing something? Or are you suggesting that a colorscheme use syntax on instead?
@mg979 When you load a new colorscheme, groups with settings are cleared, but groups that link to another group are not touched (see vimCommand in my first post). How do you deal with those?
But a colorscheme only executes syntax reset
Yes you are right. My point was, a user who really wants a colorscheme to look exactly as intended might think to do this when changing schemes:
let g:colors_name = "new_scheme"
syntax on
This almost works. But because of the "hi def" issue it doesn't always.
I must be wrong then. I tried to use this command in a syntax file:
command! -nargs=1 HiDefault call s:HiDefault(<q-args>)
function! s:HiDefault(groups)
try
let newHi = split(a:groups)[0]
if match(execute('hi ' . newHi), 'xxx cleared') >= 0
exe 'hi! default link' a:groups
echom newHi 'had been cleared'
endif
catch
exe 'hi default link' a:groups
endtry
endfunctionand replaced all hi def link commands with this one, but I can't see any change. If I run filetype detect after colorscheme change, broken highlight gets fixed though.
Edit: the command is able to restore broken highlight on filetype detect at least, while it doesn't work without it. So I still think that hi def link is misbehaving when a syntax has been cleared by a colorscheme.
How about a solution similar to b:undo_ftplugin and b:undo_indent?
let g:undo_colorscheme = 'highlight clear'
\ .. ' highlight! link <one of our linked group> NONE'
\ .. ' highlight! link <another one of our linked group> NONE'
\ .. ' highlight! link <and another one of our linked group> NONE'
Like those features, g:undo_colorscheme could be executed to undo everything done in the current colorscheme and, essentially, go back to a pristine default before sourcing the new colorscheme.
I like @romainl's idea.
I've tried to automate this in the past. But I found it distasteful and stopped using it shortly thereafter, so I'm not sure how reliable it is. Autocommands are tricky things.
The idea: record existing links before changing them. For filetype-specific syntax items, record the existing link, unless there is none yet. In that case defer creating the new link until after the syntax file has been sourced. When switching to another colorscheme, restore links using the gathered information.
I only do this below when linking, but I guess you could also make s:fthi() for setting colours directly too.
" colors/linktest.vim
hi clear
let s:name = expand('<sfile>:t:r')
let colors_name = s:name
fun! s:hi(grp, fg, bg, attr)
exe 'hi! ' . a:grp . ' ctermfg='.a:fg . ' ctermbg='.a:bg . ' cterm='.a:attr
endfun
com! -nargs=+ Hi call s:hi(<f-args>)
let s:links = {}
fun! s:link(to, ...)
for o in a:000
sil! let old = matchstr(execute('hi ' . o), 'links to \zs.*')
let s:links[o] = old == '' ? 'NONE' : old
exe 'hi! link ' . o . ' ' . a:to
endfor
endfun
com! -nargs=+ Link call s:link(<f-args>)
" only checks first item for a link
fun! s:ftlink(ft, to, ...)
sil! let old = matchstr(execute('hi ' . a:000[0]), 'links to \zs.*')
if old == ''
exe 'au colorscheme_' . s:name . ' filetype ' . a:ft .
\ ' call s:link("' . a:to . '", "' .
\ join(a:000, '", "') . '")'
else
call call('s:link', [a:to]+a:000)
endif
endfun
com! -nargs=+ Ftlink call s:ftlink(<f-args>)
fun! s:cleanup()
for [link, old] in items(s:links)
exe 'hi link ' . link . ' ' . old
endfor
endfun
exe 'augroup colorscheme_' . s:name
au!
au colorschemepre * call s:cleanup()
exe 'au colorschemepre * au! colorscheme_' . s:name
augroup END
Hi normal 7 0 NONE
Hi emphasis bg fg NONE
Hi surprise 15 5 bold
Link emphasis directory whatever
Ftlink help emphasis helphypertextjump otherstuff
Ftlink vim surprise vimcomment vimstring
delcom Hi
delcom Link
delcom Ftlink
I think vimscript masochism in this case won't help, imho the solution is easy (but must be implemented in the C source): don't keep cleared groups around. If a group has been cleared, delete it, so that hi default link will be able to restore the default highlight group as defined by the syntax file.
Yeah, I know, but workarounds were being discussed. However I couldn't agree more. That's why I stopped trying to use a workaround myself -- it's not the right thing to do.
A more palatable workaround perhaps:
" colors/boring.vim
...
let s:restore = {}
...
" break a help highlighting link
hi helphypertextjump ctermfg=5 ctermbg=NONE cterm=NONE
let s:restore.helphypertextjump = 'identifier'
...
fun! s:fixlinks()
for [group, link] in items(s:restore)
exe 'hi! link ' . group . ' ' . link
endfor
endfun
augroup colorscheme_boring
au!
au colorschemepre * call s:fixlinks() | au! colorscheme_boring
augroup END
@adscriven My proposal doesn't imply using autocommands. I tried it in the past, too, and didn't like it either. My point is rather that there already is a similar mechanism in place for filetype plugins and indent scripts and implementing it in core for colorschemes shouldn't be too complicated. From there, it would be a matter of updating the doc, the default colorschemes (we are working on it), and the existing templating solutions and wait for everyone to update their colorschemes accordingly.
Fixing the problem at its root would certainly be the way to go but links can be legit (those made in syntax scripts) or viral (those made in colorschemes) and a perfect solution shouldn't nuke the legit ones while trying to eradicate the viral ones. With my proposal, colorscheme authors clean up their mess and only their mess.
Note that b:undo_ftplugin itself is implemented through an autocommand:
Lines 11 to 35 in f0afd9e
@romainl Yeah, again, I agree with having a built-in standard way. I also agree that colorschemes should fix what they break. I'm just trying to think of something that can be done now (and I think it's basically the same as your proposal, but with an ad hoc autocommand).
Also there's no avoiding reading the syntax files in question: linking to NONE isn't enough, you have to restore the default link. And there's a minor complication of somebody using different syntax file to that in $VIMRUNTIME which may have different linkage to standard. So I find something like reverting to hi def link ... definitions after a hi clear more appealing.
but links can be legit (those made in syntax scripts)
@romainl aren't those using hi default link? Because this is what is currently failing (legit highlight links not being restored due to the group having been 'cleared').
I have changed function hl_has_settings adding additional check for sg_cleared flag making it return false if highlight group was cleared (like highlight group has no settings if it is cleared):
Line 1632 in 06cf97e
/*
* Return TRUE if highlight group "idx" has any settings.
* When "check_link" is TRUE also check for an existing link.
*/
static int
hl_has_settings(int idx, int check_link)
{
return HL_TABLE()[idx].sg_cleared == 0 &&
( HL_TABLE()[idx].sg_term_attr != 0
|| HL_TABLE()[idx].sg_cterm_attr != 0
|| HL_TABLE()[idx].sg_cterm_fg != 0
|| HL_TABLE()[idx].sg_cterm_bg != 0
#ifdef FEAT_GUI
|| HL_TABLE()[idx].sg_gui_attr != 0
|| HL_TABLE()[idx].sg_gui_fg_name != NULL
|| HL_TABLE()[idx].sg_gui_bg_name != NULL
|| HL_TABLE()[idx].sg_gui_sp_name != NULL
|| HL_TABLE()[idx].sg_font_name != NULL
#endif
|| (check_link && (HL_TABLE()[idx].sg_set & SG_LINK)));
}
With that change I can "restore" the colors with:
:colo minfrom OP:colo default--> colors are broken:syntax clear:syntax on--> default syntax highlighting is restored
PS, actually it is enough to do :syn enable without :syn clear
PPS, it works with all my colorschemes that redefine filetype highlight groups, all you have to do is to run :syn on or :syn enable after colorscheme
Nice. This is the sort of thing I was thinking of: https://asciinema.org/a/k0WvrmJuSfp7yybxPegORs9nx
I added int sg_deflink to hl_group_T. Saved in do_highlight(). Restored in highlight_clear(). I've probably broken something else, but you get the idea.
Edit: linkspatch.txt
Edit: you could argue that there should be a separate :hi reset link, but on the other hand by combining the two operations colorschemes appear to work as expected for me now, without any intervention.
Basically with the change (treat cleared as a hi-group without settings) one have to use syntax on instead of syntax reset when authoring colorscheme files:
hi clear
if exists('syntax_on')
syntax on
endif
But it wouldn't work for a default colorscheme.
But it wouldn't work for a default colorscheme.
Because it isn't a proper colorscheme, which is a problem in and of itself.
Note that my attempt appears to require no changes to colorschemes.
Note that my attempt appears to require no changes to colorschemes.
I am for any solution that would fix the issue.
Heh, but mine is also a quick hack, doesn't handle forceit for example, and I can't claim to really understand the highlight code to be honest. Offered as an idea only.
Basically with the change (treat cleared as a hi-group without settings) one have to use
syntax oninstead ofsyntax resetwhen authoring colorscheme files:hi clear if exists('syntax_on') syntax on endifBut it wouldn't work for a default colorscheme.
@habamax using syntax on could make colorscheme loading much slower (as far as I can remember). Instead, with your patch, if I understood correctly, one could do:
au Colorscheme * call ResetSyntax()
fun! ResetSyntax()
for buf in range(1, bufnr('$'))
if buflisted(buf)
silent! unlet b:current_syntax
exe 'au BufEnter <buffer=' . buf . '> ++once doautocmd Syntax'
endif
endfor
endfunThis should reset the b:current_syntax variable in all buffers and let the default highlight groups be applied again the next time you enter the buffer. But maybe it's too much (it should be enough once for each filetype).
@mg979 I dont disagree :)
Although colorscheme loading is not the thing you run in a loop to notice performance issues.
True, but using syntax on would require all colorschemes to be updated, and if they did, the benefit would only be for the newest version, while older versions would have slower colorschemes. I think an autocommand of some sort would work better.
Another alternative would be to modify also syntax files, and put the hi default link commands in functions, such as SomeFileypeDefaultHighlight(), so that after a colorscheme is changed, they can be called without reloading the whole syntax file, also from different filetypes, and without unsetting buffer variables. But this also requires lots of changes.
True, but using
syntax onwould require all colorschemes to be updated, and if they did, the benefit would only be for the newest version, while older versions would have slower colorschemes. I think an autocommand of some sort would work better.
I am not against anything builtin that would solve the issue, be it autocommand or smth else. I can live with the way currently colorschemes are authored, just let me really reset to defaults and apply my colors at least in my vimrc :)
Yes, totally, your patch would basically fix the issue in one way or another.
Created a PR just to push it forward.