Listing tickets is very slow
nikreiman opened this issue · 4 comments
I've got a project with about 30 open issues and 20 closed ones, and yet it takes over 2 seconds to run a simple ti list. I'm not sure exactly what the bottleneck is here, could it be network related?
Hi, thanks for filing this bug!
First off, I need to let you know that due to complications earlier in the project history the official TicGit-ng repository is now at http://github.com/jeffWelling/ticgit . Long story short, the original author abandoned TicGit without (at the time) providing access to the repository or gem, which meant that when other people wanted to apply fixes to ticgit they had to rename it to avoid namespace clashes. If you are using the 'ticgit' gem, your bugs should be posted here, if you are using the 'TicGit-ng' gem, your bugs should be filed at http://github.com/jeffWelling/ticgit/issues .
This particular bug is a symptom of another problem. TicGit-ng (and ticgit) currently uses the 'git' gem, which IIUC calls out to the git executable to perform operations. This is inefficient and work is actively being done to switch to the Rugged gem, which is a set of Ruby bindings for a pure C implementation of git functions written in library form known as libgit2. At present, work is needed to write a Ruby binding for the freshly implemented 'repo.fetch' functionality that was recently committed to the libgit2 project. I am making slow but steady progress on this front myself.
In the meantime, small improvements may be gained from grooming the existing code so if you are a developer and you have a moment, please do browse through the code. Otherwise, I invite you to wait a short while longer and when I expect to have this problem resolved.
It is quite an irritating problem so please bear with me, if you absolutely need a fix for this ASAP, then this;
- manually checking out the 'ticgit' branch
- deleting some of the ticket directories that you no longer need
- delete your ticgit cache (also referred to as 'index') by doing
rm -r ~/.ticgit. - checkout your original branch (any branch, just check out a branch that is not 'ticgit' before attempting a ticgit operation).
This should speed things up a little bit for you until a canonical fix can be released.
Thanks for your patience!
Jeff Welling
Hi Jeff,
Thanks for your response! I had read about the split between TicGit and TicGit-ng, bug as I had installed TicGit via apt-get, I was indeed using the old version. I have since uninstalled that and manually installed TicGit-ng via the gem tool.
I guess the libgit2 migration will fix some of the performance issues with TicGit-ng, so I'm happy to sit tight and wait for an update. The current speed of listing bugs (and other operations) is a bit annoying, but definitely not a deal-breaker for me.
While I am a developer, I don't know ruby so I can only assist in testing and providing feedback, both of which I would be happy to do.
Oh you're a developer? You'll be able to appreciate this then lol.
Ticgit had/has a bug where a series of operations (Checkout the 'ticgit' branch -> do ticgit operation such as ti list) can cause Ticgit to have an inconsistent state. This is most commonly seen when someone does a ticgit operation and they want to back out of it by checking out the branch and deleting the associated file, without deleting the appropriate cache files in ~/.ticgit/ . Now, I thought I had fixed this with a patch a year or two ago, and I had resolved that behaviour, but I created this delay.
When I first started out maintaining Ticgit as a side project, I was making mistakes all over the place and still learning how Ticgit (and git and Ruby for that matter) worked, and so I came across this 'cache corruption' bug faster than most people would in normal use. At that time, I was unfamiliar with how Ticgit worked, wasn't as skilled with Ruby as I am today, and wasn't sure how to fix the problem. Some poking and prodding eventually led me to just trying to wipe out the cache at the beginning of every Ticgit run (I know, I know, I was young and ignorant). It fixed the problem that I was seeing, but nowadays I would barely classify that fix as 'bandaid' and more 'swept under the rug'. The problem with wiping the cache out at the beginning of each operation as you probably already realized, is that it makes the cache superfluous and means you have to read everything from disk anyway.
Recently I began using Ticgit to keep notes while at work, and quickly grew the list of 'tickets' to a more than trivial amount, which is when I also ran into this problem yet again. I spent some time reviewing it with a coding buddy and we eventually tracked the speed problem down the way Ticgit has to re-read everything from disk on each operation as a result of the cache being wiped out. Some banter, several distractions, and a meal or two later we came up with the eventual solution that I intend to implement;
The cache cannot be wiped out on each run, that needs to be reverted.
The cache state needs to be maintained on each read and write through Ticgit.
A comparison between the 'ticgit' branch and the cache must be done at the beginning of each run to detect manual fixes in git, which is when it would be appropriate to blow the cache away.
The real question is: do I switch my focus from the Rugged switch to this cache corruption / speed bug, which would quickly resolve the problems that you and I are seeing. or do I stick with the switch to Rugged, which potentially will fix these problems but will also help fix several others? At this point I think I'm leaning towards sticking with the switch to Rugged, because I'm hoping that in doing so Ticgit won't need to maintain a cache anymore, but I haven't proven that it won't.
What are your thoughts as a developer?
Curious afterthought: I thought I had written scalability tests that compared the time needed to do a Ticgit operation on a repo with hundreds of tickets vs a repo that has 1 ticket. Clearly that needs reviewing as well, note to self.
I'd say to continue with the switch to Rugged. Certainly that will at least alleviate some of the performance problems, and will also make TicGit-ng much easier to maintain in the future. Plus, it's something which you seem to already have some momentum doing.
Not knowing the TicGit-ng codebase at all, I'd still guess that at least some of your improvements to the caching system would need to be rewritten after migrating to Rugged, which would be another argument for waiting on these improvements.