micromatch/glob-fs

file globbing for node.js. speedy and powerful alternative to node-glob. This library is experimental and does not work on windows!

glob-fs

file globbing for node.js. speedy and powerful alternative to node-glob.

Usage

var glob = require('glob-fs')({ gitignore: true });
var files = glob.readdirSync('**/*.js');

Run actual examples:

Jump to docs sections:

• glob.readdir
• glob.readPromise
• glob.readStream
• glob.readdirSync

Table of contents

• Install
• Usage
• API
• Middleware
  • Middleware examples
  • Middleware conventions
  • Advice for middleware authors
• Globbing examples
  • async
  • promise
  • stream
  • sync
• Events
  • Event examples
• FAQ
• TODO
• Community middleware
• Related projects
• Running tests
• Contributing
• Author
• License

(Table of contents generated by verb)

Install

Install with npm

$ npm i glob-fs --save

Usage

Params

All "read" methods take a glob pattern and an options object.

• pattern {String}: Glob pattern to use for matching. (multiple pattern support is planned)
• options {Object}: Options for glob-fs or middleware.

Examples:

// sync
var files = glob.readdirSync('*.js', {});

// async
glob.readdir('*.js', function(err, files) {
  console.log(files);
});

// stream
glob.readdirStream('*.js', {})
  .on('data', function(file) {
    console.log(file);
  });

// promise
glob.readdirPromise('*.js')
  .then(function(files) {
    console.log(file);
  });

API

.readdir

Asynchronously glob files or directories that match the given pattern.

Params

• pattern {String}: Glob pattern
• options {Object}
• cb {Function}: Callback

Example

var glob = require('glob-fs')({ gitignore: true });

glob.readdir('*.js', function (err, files) {
  //=> do stuff with `files`
});

.readdirSync

Synchronously glob files or directories that match the given pattern.

Params

• pattern {String}: Glob pattern
• options {Object}
• returns {Array}: Returns an array of files.

Example

var glob = require('glob-fs')({ gitignore: true });

var files = glob.readdirSync('*.js');
//=> do stuff with `files`

.readdirStream

Stream files or directories that match the given glob pattern.

Params

• pattern {String}: Glob pattern
• options {Object}
• returns {Stream}

Example

var glob = require('glob-fs')({ gitignore: true });

glob.readdirStream('*.js')
  .on('data', function (file) {
    console.log(file.path);
  })
  .on('error', console.error)
  .on('end', function () {
    console.log('end');
  });

Glob

Optionally create an instance of Glob with the given options.

Params

• options {Object}

Example

var Glob = require('glob-fs').Glob;
var glob = new Glob();

.use

Add a middleware to be called in the order defined.

Params

• fn {Function}
• returns {Object}: Returns the Glob instance, for chaining.

Example

var gitignore = require('glob-fs-gitignore');
var dotfiles = require('glob-fs-dotfiles');
var glob = require('glob-fs')({ foo: true })
  .use(gitignore())
  .use(dotfiles());

var files = glob.readdirSync('**');

.exclude

Thin wrapper around .use() for easily excluding files or directories that match the given pattern.

Params

• pattern {String}
• options {Object}

Example

var gitignore = require('glob-fs-gitignore');
var dotfiles = require('glob-fs-dotfiles');
var glob = require('glob-fs')()
  .exclude(/\.foo$/)
  .exclude('*.bar')
  .exclude('*.baz');

var files = glob.readdirSync('**');
//=> ['index.js', 'README.md', ...]

Middleware

glob-fs uses middleware to add file matching and exclusion capabilities, or other features that may or may not eventually become core functionality.

What is a middleware?

A middleware is a function that "processes" files as they're read from the file system by glob-fs.

Additionally, middleware can:

• be chained
• include or exclude a file based on some condition, like whether or not one of its properties matches a regex or glob pattern.
• determine whether or not to continue recursing in a specific directory
• modifying an existing property to the file object
• add a new property to the file object

Middleware examples

Ignoring files

In the following example, notemp is a complete and functional middleware for excluding any filepath that has the substring temp:

var glob = require('glob-fs')();

function notemp(file) {
  if (/temp/.test(file.path)) {
    file.exclude = true;
  }
  return file;
}

glob.use(notemp)
  .readdirStream('**/*.js')
  .on('data', function(file) {
    console.log(file.relative);
  });

Matching

Pattern matching is done by default in glob-fs, but you get disable the built-in matchers or get more specific by adding a middleware that uses [micromatch][] or minimatch for matching files.

var glob = require('glob-fs')({ gitignore: true });
var mm = require('micromatch');

glob.use(function(file) {
    if (mm.isMatch(file.relative, 'vendor/**')) file.exclude = true;
    return file;
  })
  .readdirStream('**/*.js')
  .on('data', function(file) {
    console.log(file.relative);
  });

recursion

Here is how a middleware might determine whether or not to recurse based on a certain pattern:

var glob = require('glob-fs')();

// this specific check is already done by glob-fs, it's just used here as an example 
function recurse(file) {
  // `file.pattern` is an object with a `glob` (string) property
  file.recurse = file.pattern.glob.indexOf('**') !== -1;
  return file;
}

// use the middleware
glob.use(recurse)
  .readdir('**/*.js', function(err, files) {
    console.log(files);
  });

Built-in middleware

Currently glob-fs includes and runs the following middleware automatically:

• glob-fs-dotfiles: glob-fs middleware for automatically ignoring dotfiles.
• glob-fs-gitignore: glob-fs middleware for automatically ignoring files specified in .gitignore

Disabling built-ins

To disable built-in middleware and prevent them from running, pass builtins: false on the global options. This will disable all built-in middleware.

Example:

var glob = require('glob-fs')({builtins: false});

To disable a specific middleware from running, you can usually pass the name of the middleware on the options, like dotfiles: false, but it's best to check the readme of that middleware for specifics.

Middleware conventions

• Naming: any middleware published to npm should be prefixed with glob-fs-, as in: glob-fs-dotfiles.
• Keywords: please add glob-fs to the keywords array in package.json
• Options: all middleware should return a function that takes an options object, as in the Middleware Example
• Return file: all middleware should return the file object after processing.

Advice for middleware authors

• A middleware should only do one specific thing.
• Multiple middleware libs can be bundled together to create a single middleware.
• Pattern matching should be extremely specific. Don't force downstream middleware to reverse your mistakes.
• As mentioned in the middleware conventions section, always return the file object.
• A single conditional should only set file.exclude to true, or file.include to true, never both.
• It's completely okay to check this.options
• Middleware modules should be fully documented.

Globbing examples

Note that the gitignore option is already true by default, it's just shown here as a placeholder for how options may be defined.

async

var glob = require('glob-fs')({ gitignore: true });

glob.readdir('**/*.js', function(err, files) {
  console.log(files);
});

promise

var glob = require('glob-fs')({ gitignore: true });

glob.readdirPromise('**/*')
  .then(function (files) {
    console.log(files);
  });

stream

var glob = require('glob-fs')({ gitignore: true });

glob.readdirStream('**/*')
  .on('data', function (file) {
    console.log(file.path);
  })

sync

var glob = require('glob-fs')({ gitignore: true });

var files = glob.readdirSync('**/*.js');
console.log(files);

Events

(WIP)

The following events are emitted with all "read" methods:

• read: emitted immediately before an iterator calls the first middleware.
• include: emits a file object when it's matched
• exclude: emits a file object when it's ignored/excluded
• file: emits a file object when the iterator pushes it into the results array. Only applies to sync, async and promise.
• dir: emits a file object when the iterator finds a directory
• end when the iterator is finished reading
• error on errors

Event examples

async

var glob = require('..')({ gitignore: true });

glob.on('dir', function (file) {
  console.log(file);
});

glob.readdir('**/*.js', function (err, files) {
  if (err) return console.error(err);
  console.log(files.length);
});

promise

var glob = require('glob-fs')({ gitignore: true });

glob.on('include', function (file) {
  console.log('including:', file.path);
});

glob.on('exclude', function (file) {
  console.log('excluding:', file.path);
});

glob.readdirPromise('**/*');

sync

Also has an example of a custom event, emitted from a middleware:

var glob = require('glob-fs')({ gitignore: true })
  .use(function (file) {
    if (/\.js$/.test(file.path)) {
      // custom event
      this.emit('js', file);
    }
    return file;
  });

glob.on('js', function (file) {
  console.log('js file:', file.path);
});

glob.on('exclude', function (file) {
  console.log('excluded:', i.excludes++);
});

glob.on('include', function (file) {
  console.log('included:', i.includes++)
});

glob.on('end', function () {
  console.log('total files:', this.files.length);
});

glob.readdirSync('**/*.js');

stream

var glob = require('glob-fs')({ gitignore: true })

glob.readdirStream('**/*')
  .on('data', function (file) {
    console.log(file.path)
  })
  .on('error', console.error)
  .on('end', function () {
    console.log('end');
  });

FAQ

• when files are read from the file system, an object is created to keep a record of the file's path, dirname, and fs stat object and other pertinent information that makes it easier to make decisions about inclusion and exclusion later on.
• file objects are decorated with a parse method that is used to calculate the file.relative and file.absolute properties.
• the file.parse() method is called in the iterator, right after the call to fs.stats and just before the call to the middleware handler (.handle()). This ensures that all middleware have access to necessary path information.
• file.relative is the file path that's actually pushed into the files array that is ultimately returned.
• file.relative is calculated using path.relative(file.path, cwd), where cwd is passed on the options (globally, or on a middleware), and file.path is typically the absolute, actual file path to the file being globbed.

TODO

middleware

• middleware
• middleware handler
• externalize middleware to modules (started, prs welcome!)

events

• events

tests

• unit tests (need to be moved)

iterators

• sync iterator
• async iterator
• stream iterator
• promise iterator

read methods

• glob.readdir (async)
• glob.readdirSync
• glob.readdirStream
• glob.readdirPromise

patterns

• Multiple pattern support. will need to change pattern handling, middleware handling. this is POC currently
• Negation patterns (might not do this, since it can be handled in middleware)
• matching method, memoized/cached/bound to a glob pattern or patterns, so it can be reused without having to recompile the regex.

other

• clean up ./lib
• comparsion to [node-glob][]

Community middleware

(Add your project to the .verb.md template do a PR!)

• glob-fs-dotfiles: glob-fs middleware for automatically ignoring dotfiles.
• glob-fs-gitignore: glob-fs middleware for automatically ignoring files specified in .gitignore

Related projects

• braces: Fastest brace expansion for node.js, with the most complete support for the Bash 4.3 braces… more
• fill-range: Fill in a range of numbers or letters, optionally passing an increment or multiplier to… more
• is-glob: Returns true if the given string looks like a glob pattern.
• micromatch: Glob matching for javascript/node.js. A drop-in replacement and faster alternative to minimatch and multimatch. Just… more

Running tests

Install dev dependencies:

$ npm i -d && npm test

Contributing

Pull requests and stars are always welcome. For bugs and feature requests, please create an issue

Author

Jon Schlinkert

• github/jonschlinkert
• twitter/jonschlinkert

License

Copyright © 2015 Jon Schlinkert Released under the MIT license.

---

This file was generated by verb-cli on July 11, 2015.