Async searching gives "E117: Unknown function: ferret#private#shared#finalize_search"
Closed this issue · 10 comments
Trying an async search gives the following error message:
Error detected while processing function ferret#private#async#close_cb:
line 10:
E117: Unknown function: ferret#private#shared#finalize_search
Vim becomes unresponsive for the duration of the "search", though no results show up. It looks like this might have happened in 1c210c9 as the finalize_search function only seems to be called, not defined:
% pwd
/Users/johno/src/dotfiles/vim/bundle/ferret
% rg finalize_search
autoload/ferret/private/async.vim
107: call ferret#private#shared#finalize_search(l:info.output, l:info.ack)
117: call ferret#private#shared#finalize_search(l:info.output, l:info.ack)
autoload/ferret/private/vanilla.vim
7: call ferret#private#shared#finalize_search(l:output, a:ack)
Async execution is confirmed working with other plugins (i.e. https://github.com/skywind3000/asyncrun.vim). Here's the :version output if that's helpful:
:version
VIM - Vi IMproved 8.0 (2016 Sep 12, compiled Apr 30 2017 10:36:10)
MacOS X (unix) version
Included patches: 1-589
Compiled by Homebrew
Huge version without GUI. Features included (+) or not (-):
+acl +clipboard +dialog_con +file_in_path +job -lua +mouse_sgr +path_extra +rightleft +tag_old_static +user_commands +writebackup
+arabic +cmdline_compl +diff +find_in_path +jumplist +menu -mouse_sysmouse +perl +ruby -tag_any_white +vertsplit -X11
+autocmd +cmdline_hist +digraphs +float +keymap +mksession +mouse_urxvt +persistent_undo +scrollbind -tcl +virtualedit -xfontset
-balloon_eval +cmdline_info -dnd +folding +lambda +modify_fname +mouse_xterm +postscript +signs +termguicolors +visual -xim
-browse +comments -ebcdic -footer +langmap +mouse +multi_byte +printer +smartindent +terminfo +visualextra -xpm
++builtin_terms +conceal +emacs_tags +fork() +libcall -mouseshape +multi_lang +profile +startuptime +termresponse +viminfo -xsmp
+byte_offset +cryptv +eval -gettext +linebreak +mouse_dec -mzscheme -python +statusline +textobjects +vreplace -xterm_clipboard
+channel +cscope +ex_extra -hangul_input +lispindent -mouse_gpm +netbeans_intg +python3 -sun_workshop +timers +wildignore -xterm_save
+cindent +cursorbind +extra_search +iconv +listcmds -mouse_jsbterm +num64 +quickfix +syntax +title +wildmenu
-clientserver +cursorshape +farsi +insert_expand +localmap +mouse_netterm +packages +reltime +tag_binary -toolbar +windows
system vimrc file: "$VIM/vimrc"
user vimrc file: "$HOME/.vimrc"
2nd user vimrc file: "~/.vim/vimrc"
user exrc file: "$HOME/.exrc"
defaults file: "$VIMRUNTIME/defaults.vim"
fall-back for $VIM: "/usr/local/share/vim"
Compilation: clang -c -I. -Iproto -DHAVE_CONFIG_H -DMACOS_X_UNIX -g -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1
Linking: clang -L. -L/usr/local/lib -L/usr/local/lib -o vim -lm -lncurses -liconv -framework Cocoa -fstack-protector -L/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE -lperl -L/usr/local/opt/python3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/config-3.6m-darwin -lpython3.6m -framework CoreFoundation -lruby.2.0.0 -lobjc
Thanks for the report. Not at my computer now, but this is almost certainly a case of me being stupid and forgetting to git add a new file. Will fix shortly.
Fixed in c5ba376. Thanks once again for the report @johnoshea, and sorry for the inconvenience.
Hi Greg, thanks for for sorting that.
I'm now seeing the plugin work, but not asynchronously (see https://vimeo.com/215491417). My vim has +job enabled, 'rg' is installed and setting g:FerretDispatch=0 makes no difference.
Putting echom lines at the start and end of ferret#private#async#search() has both messages show up immediately but vim still blocks for the duration of the search (i.e. the message at the end of the function shows up well before the quickfix window opens).
How can I help debug this? My knowledge of vim's channels & jobs is extremely limited but I'm happy to poke at things if you can give some pointers around where to look and what to try.
Thanks,
John
Interesting. I'm also using rg and normally don't see this. Lately though, I have seen Vim block when searching a very large directory hierarchy (like my home dir) for a common string. Is that what you're doing here?
It seems to be related to dealing with large amounts of output. I suspect a regression in Vim, but either way, would like to find a mitigation. Speaking of versions, what Vim version are you on?
Another thing I should add. I had seen lock-ups before when processing results with very long lines; this was due to this issue in Vim (see mitigations in 06eaeed and 3379d5d). I wouldn't be surprised if long lines are involved again, but I don't have any proof for that, just intuition.
I am, indeed, searching my home folder for the string 'foo' so it's not exactly a real-world need.
My current Vim version info:
VIM - Vi IMproved 8.0 (2016 Sep 12, compiled May 1 2017 14:20:35)
MacOS X (unix) version
Included patches: 1-594
Compiled by Homebrew
Huge version without GUI. Features included (+) or not (-):
+acl +clipboard +dialog_con +file_in_path +job -lua +mouse_sgr +path_extra +rightleft +tag_old_static +user_commands +writebackup
+arabic +cmdline_compl +diff +find_in_path +jumplist +menu -mouse_sysmouse +perl +ruby -tag_any_white +vertsplit -X11
+autocmd +cmdline_hist +digraphs +float +keymap +mksession +mouse_urxvt +persistent_undo +scrollbind -tcl +virtualedit -xfontset
-balloon_eval +cmdline_info -dnd +folding +lambda +modify_fname +mouse_xterm +postscript +signs +termguicolors +visual -xim
-browse +comments -ebcdic -footer +langmap +mouse +multi_byte +printer +smartindent +terminfo +visualextra -xpm
++builtin_terms +conceal +emacs_tags +fork() +libcall -mouseshape +multi_lang +profile +startuptime +termresponse +viminfo -xsmp
+byte_offset +cryptv +eval -gettext +linebreak +mouse_dec -mzscheme -python +statusline +textobjects +vreplace -xterm_clipboard
+channel +cscope +ex_extra -hangul_input +lispindent -mouse_gpm +netbeans_intg +python3 -sun_workshop +timers +wildignore -xterm_save
+cindent +cursorbind +extra_search +iconv +listcmds -mouse_jsbterm +num64 +quickfix +syntax +title +wildmenu
-clientserver +cursorshape +farsi +insert_expand +localmap +mouse_netterm +packages +reltime +tag_binary -toolbar +windows
system vimrc file: "$VIM/vimrc"
user vimrc file: "$HOME/.vimrc"
2nd user vimrc file: "~/.vim/vimrc"
user exrc file: "$HOME/.exrc"
defaults file: "$VIMRUNTIME/defaults.vim"
fall-back for $VIM: "/usr/local/share/vim"
Compilation: clang -c -I. -Iproto -DHAVE_CONFIG_H -DMACOS_X_UNIX -g -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1
Linking: clang -L. -L/usr/local/lib -L/usr/local/lib -o vim -lm -lncurses -liconv -framework Cocoa -fstack-protector -L/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE -lperl -L/usr/local/opt/python3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/config-3.6m-darwin -lpython3.6m -framework CoreFoundation -lruby.2.0.0 -lobjc
Update: I think you're spot-on with the "very long lines" theory. If I search for 'foo' in my vim folder it takes ages because it finds a ton of matches in the vim/thesaurus/mthesaur.txt I have loaded as part of a plugin. Searching the same folder for 'ferret' is almost instantaneous, whereas the previous search locks vim up for maybe 5-10s.
Should you be curious the https://github.com/skywind3000/asyncrun.vim plugin doesn't block in the same way if I: open the quickfix window first (it doesn't do it automatically) and then do :AsyncRun rg foo - the results take the same 5-10s but the interface is responsive throughout.
Anyway, I just won't do daft searches and that'll keep me more than happy - thanks again!
Anyway, I just won't do daft searches and that'll keep me more than happy - thanks again!
Yeah fair enough, although I will try to find a mitigation. I can repro this pretty easily on my machine. For example, if I search for SomeJavaClassWhichExists in a repo with 1m+ files, everything works fine. If I search for irsntirnstienrstienrst in my home dir, everything works fine. If I search for test in my home dir, rg produces 15G(!!!) of output, including some terribly long lines, and Vim reliably blocks. We clearly don't want to penalize people for making innocent searches, so will come up with a workaround one way or another.
For reference, one of the last 3 commits has eliminated all obvious delays doing a "search for 'foo' in my dotfiles directory" (which containing the thesaurus file with a ton of long-line matches and was previously giving me 5-10s delays).
Doing a "search for 'foo' in my home directory" also keeps Vim responsive throughout, even if poor old 'rg' is grinding away trying to services a frankly bonkers search request.
Thanks for a really speedy - and effective - response to a not-entirely-mainstream request.
Awesome. Even though searching an entire $HOME dir isn't exactly the intended use case, it's easy enough to accidentally do so, so I think it's pretty important to avoid locking up the editor. Glad to hear it's working better for you now.
I still have one last idea for a mitigation for the case where rg produces a lot of output (even in the absence of long lines) -- eg. search for "test" or "the" in home directory -- but will get to that later.