Filebench - A Model Based File System Workload Generator ======================================================== INTRODUCTION ------------ Filebench is a file system and storage benchmark that can generate a large variety of workloads. Unlike typical benchmarks it is extremely flexible and allows to specify application's I/O behavior using its extensive Workload Model Language (WML). Users can either describe desired workloads from scratch or use (with or without modifications) workload personalities shipped with Filebench (e.g., mail-, web-, file-, and database-server workloads). Filebench is equally good for micro- and macro-benchmarking, quick to setup, and relatively easy to use. INSTALLATION ------------ Filebench compilation and installation is simple, and can be completed in two steps. However, if you have downloaded a release tarball from Github (https://github.com/filebench/filebench/releases), you can go ahead and skip to Step 2. Step 1: Generating autotool scripts Makefile.in and configure files are not included in the repo, so they will have to be generated. To do so, make sure you have libtoolize and automake tools installed and run the following commands: $ libtoolize $ aclocal $ autoheader $ automake --add-missing $ autoconf Step 2: Compilation and installation Before proceeding, make sure yacc and lex are available in your system. Then, you can proceed with configuration, compilation, and installation as usual: $ ./configure $ make $ sudo make install QUICK-START GUIDE ----------------- To quickly introduce a new user to Filebench we use two examples demonstrating two different typical Filebench workflows: 1) describe a user-defined workload in Workload Model Language (WML) and generate the workload; 2) generate one of the predefined Filebench workloads. The description in this guide assumes Filebench 1.5-alpha1 or higher. Example 1: User-defined workloads First step is to create the description of the workload (so called workload personality) in WML language. Typically, workload personalities are stored in files with '.f' extension. In this example we describe a very simple workload consisting of two processes with three threads each. Every thread continuously picks a file among many, reads it, and then closes the file. Here is the corresponding workload personality: 01 define fileset name="testF",entries=10000,filesize=16k,prealloc,path="/tmp" 02 03 define process name="readerP",instances=2 { 04 thread name="readerT",instances=3 { 05 flowop openfile name="openOP",filesetname="testF" 06 flowop readwholefile name="readOP",filesetname="testF" 07 flowop closefile name="closeOP" 08 } 09 } 10 11 run 60 Four main entities in Filebench are 'filesets', 'processes' consisting of 'threads', and 'flowops'. In the first line we define a fileset containing 10,000 files of 16KiB size each in /tmp directory. Filebench is instructed to precreate all files in the fileset before executing the actual workload. In the third and fourth lines we define two identical processes each consisting of three identical threads. Every thread in Filebench repeats flowops (operations) defined in it in a loop. Lines 05-07 describe the flowops that constitute the threads: open a file in "testF" fileset, read the file completely, and close it. Finally, in the 11th line we indicate to run the workload for 60 seconds. In more complex workloads one can define any number of filesets, multiple different processes and threads, use a variety of flowops and attributes, and more. Refer to the complete WML vocabulary for details at https://github.com/filebench/filebench/wiki/Workload-model-language Assuming that the personality is saved in 'readfiles.f' file, one can then generate corresponding workload by running 'filebench -f readfiles.f' command. Example 2: Pre-defined workloads Filebench comes with several predefined micro- and macro-workloads (e.g., webserver, fileserver, mailserver) which are also described in WML, not differently than the workload in Example 1 above. In the source code tree, workloads are located in the workloads/ directory and are typically installed in /usr/local/share/filebench/workloads/ during 'make install' (though this can differ from one installation to another). We do *not* recommend to directly use workload files from workloads/ or /usr/local/share/filebench/workloads/ directories. The main reason is that these workloads *are not properly sized* (e.g., in terms of the dataset sizes) to a particular system. For instance, the initial dataset size of the webserver workload is only slightly larger than 16MiB, which is typically not the size you want to test the system containing multiple gigabytes of RAM with. So, instead, copy webserver workload to any other directory: $ cp /usr/local/share/filebench/workloads/webserver.f mywebserver.f Then edit the copied file to increase the dataset size by setting the number of files ('entries' attribute of a fileset) to an appropriate value. Finally, run the workload: $ filebench -f mywebserver.f An extended discussion on how to scale Filebench workloads can be found at https://github.com/filebench/filebench/wiki/Scaling-Filebench-workloads SUPPORT ------- To ask questions about Filebench, report bugs, or request new features, you can use GitHub's issue tracking system. This is the central hub for both user support and bug tracking, and can be found at https://github.com/filebench/filebench/issues
ZhiqiangLiu26/filebench
File system and storage benchmark that uses a custom language to generate a large variety of workloads.
CNOASSERTION