tuler/grails-prerender

Prerender Plugin

Nowadays it's fairly common to implement websites using lots of AJAX and javascript generated contents. This improves usability but it's bad for search engines, because javascript is not processed by them.

Google came up with a proposal to enhance the SEO quality of such websites in a "blessed" way: https://developers.google.com/webmasters/ajax-crawling/docs/getting-started.

One of the steps of the process is to generate HTML snapshots of your webpages. There comes prerender.io to the rescue. This is a server written in node.js which uses phantomjs to render pages on demand.

All you need to do is proxy requests coming from crawlers to a prerender server. This grails plugin add a servlet filter which makes this proxy.

Actually this is just a thin wrapper around the filter provided by https://github.com/greengerong/prerender-java.

Using

Add a dependency to BuildConfig.groovy:

plugins {
	runtime ":prerender:1.0.7"
	...
}

The default configuration installs a servlet filter that proxies the request to http://prerender.herokuapp.com according to the rules defined by google.

Basically it checks if the User-Agent is a crawler or if there is a query parameter called "_escaped_fragment_".

Configuration

The plugin is configured through Config.groovy.

You can change the prerender service url using the following configuration:

prerender.prerenderServiceUrl = 'http://localhost:3000'

The default value is http://prerender.herokuapp.com

If you want to use the prerender service, you can configure it using the prerender.prerenderToken configuration key.

You can also change the user agents the filter applies to, separated by comma. This will be added to the default ones. Example:

prerender.crawlerUserAgents = 'Ezooms,MSNBot,Exabot'

The filter will ignore all url's that contains any of the following extensions: .js, .css, .less, .png, .jpg, .jpeg, .gif, .pdf, .doc, .txt, .zip, .mp3, .rar, .exe, .wmv, .doc, .avi, .ppt, .mpg, .mpeg, .tif, .wav, .mov, .psd, .ai, .xls, .mp4, .m4a, .swf, .dat, .dmg, .iso, .flv, .m4v, .torrent. In addition to those, you can define the following configuration, which should be a comma separated string with the extensions you want to add to the above list.

prerender.extensionsToIgnore = '.xml,.json'

Changelog

• 1.0.7: Added prerenderToken configuration.
• 1.0.6: Upgraded prerender-java dependency to 1.6.1.
• 1.0.5: Upgraded prerender-java dependency to 1.5.
• 1.0.4: Upgraded prerender-java dependency to 1.4.
• 1.0.3: Upgraded prerender-java dependency to 1.3.
• 1.0.2: Extension list to ignore.
• 1.0.1: Upgraded prerender-java dependency to 1.2. Fix for User-Agent detection code.
• 1.0.0: First working version