Netlify Search Index Plugin

Generate a Search Index you can query via a static JSON blob or a Netlify Function!

You may not need this - There are other ways to add search to your site, like using Algolia or Vanilla JS with a custom search Index.

However, you may wish to have a way to generate this index based ONLY on crawling your generated static site, or you may wish to do index searches in a serverless function instead of making your user download the entire index and run clientside.

Demo

Demo site: https://netlify-plugin-search-index.netlify.app
Demo search function: https://netlify-plugin-search-index.netlify.app/.netlify/functions/search?s=web
Demo JSON blob: https://netlify-plugin-search-index.netlify.app/searchIndex.json

Usage

To install, add the plugin in your netlify.toml. No config is required but we show the default options here.

Generating both serverless function and clientside JSON

[[build]]
  functions = functions # must specify a functions folder for this to work
[[plugins]]
  package = netlify-plugin-search-index
    # all inputs is optional, we just show you the defaults below
    # [plugins.inputs]
      # ignore = ["/ignore-this-file.html"] # don't index this file
      # generatedFunctionName = search # change the name of generated folder in case of conflicts, use `null` to turn off
      # publishDirJSONFileName = searchIndex # also use null to turn off

Without config, this would generate:

a function at https://yoursite.netlify.com/.netlify/functions/search and
a clientside JSON blob at https://yoursite.netlify.com/searchIndex.json

Generating serverless function only

To use this plugin only for the generated serveless function, supply null to the publishDirJSONFileName:

[[plugins]]
  package = netlify-plugin-search-index
    [plugins.inputs]
      generatedFunctionName = mySearchFunction
      publishDirJSONFileName = null

This would generate a Netlify function at https://yoursite.netlify.com/.netlify/functions/mySearchFunction which you can query with https://yoursite.netlify.com/.netlify/functions/mySearchFunction?search=foo.

Generating clientside JSON only

To use this plugin only for the clientside JSON file, supply null to the generatedFunctionName:

[[plugins]]
  package = netlify-plugin-search-index
    [plugins.inputs] = 
      generatedFunctionName = null
      publishDirJSONFileName = mySearchIndex # you can use / to nest in a directory

This would generate a clientside JSON at https://yoursite.netlify.com/mySearchIndex.json.

Supplying null to both generatedFunctionName and publishDirJSONFileName would be meaningless (because there would be nothing to generate) and cause an error.

More options

Exclude files

Your project probably contains some content files that you don't want your users to search. Pass an array of paths (or regex) to the files you don’t want to be indexed to dismiss them:

[[plugins]]
  package = netlify-plugin-search-index
    [plugins.inputs] = 
      exclude = ['/ignore-this-file.html', /^\/devnull\/.*/]

Search params

At the moment of writing, it is not possible to pass custom options to FuseJS. If it's something that you would like to see implemented, let us know! ✌️

What It Does

After your project is built:

this plugin goes through your HTML files
extracts their metadata (title, description, keywords) with unified
converts them to searchable content, weighted by field type
stores them as a JSON blob in /searchIndex/searchIndex.json
generates a Netlify Function that fuzzy searches against a query string with fuse.js

You can use this plugin in two ways:

Client-side: You can simple require the JSON blob in your clientside JavaScript if it isn't too big:
```
// app.js
import searchIndex from './searchIndex.json'
```

Serverless-side: You can use the generated function that reads the JSON and returns fuzzy search results to be lighter on your frontend. The generated function is available at .netlify/functions/searchIndex and you can use it with a search term like .netlify/functions/searchIndex?s=foo or .netlify/functions/searchIndex?search=foo:

// app.js
document.getElementById('myForm').addEventListener('submit', async event => {
  event.preventDefault()
  const result = await fetch(`/.netlify/functions/searchIndex?search=${event.target.searchText.value}&limit=25`).then(x => x.json())
  document.getElementById('result').innerText = JSON.stringify(result, null, 2)
})

You can use an optional limit parameter to limit the length of returned results.

Under the hood, the search function uses fuse.js and in future we may expose more configurations for this. Fuse.js search is fuzzy, therefore for short strings you may get a lot of results that don't seem relevant. In future we may make Fuzzy matching opt-in, and just use String.toLowerCase and String.includes - please see more details here.

Notes for contributors

We had to use patch-package to fix this bug: paulmillr/readdirp#157 - readdirp is a dependency of ``copy-template-dir` which we use.

Hopefully it will be resolved or we can just fork copy-template-dir entirely locally.

Future plans

WE ARE SEEKING MAINTAINERS.

Micro Todo:

expose fuse.js and html parse search options for more configurability
filtering results
support non html files?

h-dong/netlify-plugin-search-index