Improve Telescope picker

Currently, the Telescope picker is quite slow. The reason for this is probably the entry_display.create() function that is called for each line in the results window (that's the list of Author (Year) Title lines). The function is used to define which part of the string is to be highlighted with which highlight group (e.g. chars 1-5 with highlight group X, chars 6-10 with group Y, etc.). Because the length of the author and title string is different for each line, the function needs to be called for each line. Telescope's creators explicitly say in :h telescope.pickers.entry_display that this can slow things down.

An alternative is to have predefined lengths for each element in the results window (e.g. year -> 4 chars , author -> 20 chars, title -> remaining chars). It would look something like how citar does it:

Advantages:

entry_display.create() only has to be called once when the picker is opened, which should speed things up

Disadvantages:

wastes space, which can be relevant especially when the picker window is small (applies to me as I usually have two documents open side-by-side

What do you think?

I would switch to a fixed-width approach if it meant speeding up the selection process. Even if you have 30 characters for the authors, 8 for the year (including padding) and 42 for the title, that's only 80 columns and should fit even if you have a vertical split.

But I like having options, so why not offer both?

I'm gonna play around with this probably on the weekend. I'll see if it's manageable to have both versions. I'm not sure it's worth the added complexity, but let's see :)

Personally, I also really want those sexy icons that show at a glance whether the entry has attached notes/files. 😃

I just tested my theory about what makes the picker a bit slow, and I was wrong. Even when I remove all the highlighting, the picker is still on the sluggish side (like it's better, but I don't think the change is that noticeable). So for now I'm dropping implementing an alternative results style and will rather try think of other ways of making things speedier 🤔

That's interesting. I'll also have a look to see if I can find the bottleneck.

Hi again, so I wanted to get back on this issue because the picker is still quite slow considering I have only 700 entries in my database. Telescope find_files in directories with thousands of files is much faster.

I experimented a bit but have not found a way to speed it up yet. Here's what I tried:

https://github.com/jghauser/papis.nvim/blob/main/lua/telescope/_extensions/papis.lua

commenting out previewer - no improvement
commenting out sorter - no improvement
swapping the generic sorter with https://github.com/nvim-telescope/telescope-fzy-native.nvim - no improvement

Unless I miss something, this should leave the finder of the picker as bottleneck. I timed the execution of the sqlite database connection:

papis.nvim/lua/telescope/_extensions/papis.lua

Line 49 in 31e7e72

local entry_pre_calc = db["search"]:get(entry["id"])[1]

This function is executed in sub-millisecond level, so it should not be the bottleneck either.

This is kind of where my lua/telescope understanding leave me. Do you have a suggestion of what else I could try?

Great to see you back at this! :) I'll play around as well and see what else could be at fault. It might be useful to have someone chime in who has a better understanding of telescope.nvim -- maybe one of us could ask a question over there.

Out of curiosity, what is slow for you and what system are you on? For me, with 1700 entries, searches complete maybe not quite as fast as I type but definitely no more than a split second or so after i finish typing. It does feel sluggish, but not immensely so. I do have quite a fast system though (Intel i5-1240P).

It takes around 1 second from the telescope window opening to the database entries to appear. When typing a key to narrow down the selection, the characters I type do not appear instantly, the window takes again a fraction of a second to update. From the initial database to a selection, it's about 2 seconds, assuming after I finish typing. I'm also on an Intel i5.

It's not a dealbreaker but definitely much more sluggish than other telescope interfaces. We can do better :)

Sure, we could ask over there. Two alternative approaches: I can rebuild the picker from scratch and see at what part it gets slow, and we could see if using one of the fzf plugins instead of telescope makes a difference.

Hui, that is very sluggish. In my case, the picker opens almost instantly. What year is your i5 from? I'm asking to exclude the possibility that something else on your system might slow things down. But in any case, papis.nvim shouldn't require last year's CPU to work semi-fluently 😅.

I would prefer sticking to Telescope as most people already have that installed, and I think it might be hard to get the same functionality with a different system. But if Telescope can't be sped up, I'm open to having an alternative minimal-but-speedy option.

Maybe this profiler or something similar could help with identifying what's slowing things down.

EDIT: I also found these suggestions.

I finally took a serious look at this and I think I found various ways to dramatically speed up the Telescope picker. In my tests the picker is about 10x faster (on the 2nd and subsequent invocations). When the picker is first opened, it needs to calculate various things and hence the speed-up is here only about 10-15%. If you're still using papis.nvim, could you give the branch in #72 a try? I would really appreciate some feedback! 😊

Hi! Yes, I had noticed that you have been working on papis.nvim lately, that's great! I'm still using it, I'll check out the branch and will let you know my experience.

I've merged #72 as it seemed to work well and give significant improvements. Still happy about feedback when you get to check out the changes :)

It's much, much faster! Great job! :-)