google/jimfs

Slower than regular file system?

fcurts opened this issue · 1 comments

Running my integration tests (which write and read back a couple hundred small-ish HTML files) with jimfs takes about twice as long as running them with the regular file system (MacBook Pro 2015, SSD).

Is this expected? More generally, are there any good reasons for using jimfs for tests that need a file system?

Hard to say whether this is expected or not based on the information you've given here*, but:

  • Modern file systems can be quite fast on SSDs.
  • To get the best speed with Jimfs, you'd need to ensure you aren't creating a new file system instance for each test (which requires making the file system a static field since each test is run in a separate instance of the test class), since Jimfs caches the memory blocks it stores files in. That does require ensuring that files you create are deleted after each test the same way you would for a normal file system, though.
  • Since it's Java, Jimfs won't perform at its best until the codepaths have been run enough to fully JIT them, which could certainly affect overall test time. Native code for the local file system obviously doesn't have this issue.
  • Depending on the average file size, speed might benefit from tuning the block size to something less than the default 8K, but in general I wouldn't recommend worrying about this.

Ignoring speed, some reasons to use Jimfs for tests are:

  • If you use a different file system instance for each test, you don't need to do anything to ensure files are cleaned up.
  • In some environments it can be useful just because it doesn't actually touch the local file system; obviously this isn't the case for you.
  • You want to be able to test against a Linux-like or Windows-like file system and have those tests run regardless of what environment you're actually running in.

Whether any of those are useful will depend on the individual situation.

* Well, twice as long would certainly be unexpected if you aren't creating a new file system instance for each test. I did a bunch of benchmarking back when I created this and generally found that it could perform at least as well as an SSD with JIT warmed up. Of course, SSDs have been getting faster since then.