We still spend time to traverse "ignored" directory/file while generating site
TheKK opened this issue · 2 comments
Currently implementation is like
neuron/src/Neuron/Reactor/Build.hs
Line 235 in 49a4beb
the filtering is after the buildDirTree.
In my case this makes generating site becomes very slow, since I'm using nix direnv and neuron still tries to parse .direnv which contains all the source of nix input.
I think the real issue is from https://github.com/obsidiansystems/directory-contents, which has issue with the DirTree a type. The representation is quite inefficient since it use String/FilePath. The experiment with my quite small blog project to change String to Data.Text shows that the heap usage is reduced from 8 Mib to 5 Mib. I'll open an issue there to discuss this issue.
We use the directory-tree package only because it makes it simpler to create something like the dirtree plugin. But maybe that decision wasn't perfect.
In retrospect System.FilePattern.Directory (from filepattern) would have been simpler choice, with custom code to build the tree from a list of files.
I should keep that in mind for srid/ema#22
FTR, Emanote avoids this library and uses filepattern directly. It's also got an innotative 'union mount' system for O(1) update,
https://github.com/srid/emanote/blob/master/src/Emanote/Source/Mount.hs
Closing per https://neuron.zettel.page/next