Add CachingFileProvider
davidfowl opened this issue · 2 comments
@pakrym ran some performance tests on our static file server and we came to the conclusion that the performance can be improved with a bit of caching. Most of the time (on windows at least) seems to be spent in GetFileAttributes
About a Clean() method, I just ask because I have seen the request for IMemoryCache. But personnally I don't know scenarios.
For the synchronization, I was thinking that a physically deleted file would be 'visible' until its expiration (and same principle for update). IFileProvider.Watch() may be used too, but I fear it would have performance problems in case of large file tree.
Or do you want both options?
An internal structure FileInfoCacheEntry { IFileInfo, Last Access Date, Expiration Date... }.
Two internal dictionaries:
- SubPath -> FileInfoCacheEntry // Filled by GetFileInfo() and GetDirectoryContents() results
- SubPath -> { List<FileInfoCacheEntry>, Last Access Date, Expiration Date,... } // Used to store the GetDirectoryContents() results
Caching options (excluding the content cache for the moment):
- Expiration date (Nullable): Indicates how long a cache entry is considered as valid (
nullto useIFileProvider.Watch()???). - Max entry count (int): Necessary to be sure to not accumulate too much data. The olders entries (in sense of last access date, not file last modification) are removed if necessary in a background thread.
- Hysteresis (int): Value between 0 and 'Max Entry Count' used to calculate how many old entries must be removed when the maximum entry count has been exceeded. It allows to avoid hysteresis problems.
- ExcludeFilter (
Func<string, IFileInfo, bool>): Allows to exclude a single file from the cache. Special case: if at least one of the entries of aGetDirectoryContents()is excluded by this filter, theGetDirectoryContents()list is not cached (but the otherIFileInfoare), because we consider that the file removal event cannot be cached.