This code goes along with an article I published on Medium: SUPER SIMPLE Static Site Generation with Node, Jade, Gulp and JSON
###Static Site? But...why? If you are asking this question, do yourself a favor and skim through the free O’Reilly book Static Site Generators: Modern Tools for Static Website Development. The quick answer is because you can have a static website and still have all the bells and whistles. The slightly longer answer is a static website will perform well, can be hosted anywhere, is secure and can easily be version controlled. For more on these points, read the book mentioned above.
What I didn’t realize until I noticed an email from O’Reilly about the book, is that I’m already using the concepts of Static Site Generators, except without the Generator part. I recently rebuilt my employer’s website using gulp, Jade templates and JSON. It is a homepage with some case study pages. I wanted to learn Jade because I’m also dabbling in Express.js and html pages fulfilled the requirements of the project. Piping the data into Jade was something I had to work hard to understand, so hopefully this helps someone else.
###The Goal. So after reading the book, I thought it would helpful for other developers to see my approach. To-do MVC is the standard for example projects like this, but to me it was more comparable to a simple WordPress website. So what is appealing about the basic WordPress site the we often build? I thought it was the stuff you get right out of the box with the 2016 theme:
- Theming/Templates (header, footer, shared markup)
- Easy page generation
- Easy post generation
- Easy to get a post loop in a page (via the loop)
- Sorting posts by categories (vie the loop)
So the approach I took was building a new portfolio site for myself, just lacking the final content. Which would lead to the following pages:
- Home => index.html
- About Me => about.html
- Resume => resume.html
- Contact Me => contact.html
- Blog => blog/index.html which loops over all blog posts and provides a teaser
- Blog Pages => blog/[PAGE NAME].html
- Blog Categories => blog/category-[CATEGORY NAME].html
- Assets => public/ just a home for images, css, js
###The Core Tools. Ok, if you aren’t slightly familiar with these tools, check them out and double back. I’m going to jump right in.
- Node.js: node serves as the server and the engine in the build process
- Gulp.js: the task manager
- Jade Templates (gulp-jade): the templating system
- JSON: serves as our database
###The build. I have a personal preference that I don’t like gulp and node messing up my root folder, so I like to tuck the gulp build into a _src directory. So I’ll start by breaking down what is in there.
/_src
gulpfile.js (the directions for the tasks)
package.json (default npm file with node modules,etc)
markup/ (where the jade templates live)
-- /_inc/ (basic template files, or includes. a grouping of markup that is included in each page)
---- foot.jade (usually just the scripts or GA tracking)
---- footer.jade (the site footer menu and copyright, loops over post list and sections)
---- head.jade (repetitive html head content)
---- header.jade (the site header, main navigation, loops over sections)
-- _templates/ (templates for page types)
---- main.jade (core page template)
---- post.jade (post page template)
-- blog/
---- about-content.jade (a blog post)
---- about-gulp.jade (a blog post)
---- category.jade (template page for category pages)
---- index.jade (page the previews the posts, loops over posts and categories)
---- why.jade (a blog post)
-- data/
---- resume.json (sample resume data)
---- services.json (sample services data)
---- skills.json (sample skills data)
-- about.jade (content for about us page)
-- contact.jade (content for about us page)
-- index.jade (content for about us page, loops over services and skills)
-- resume.jade (content for about us page, loops over resume data)
Three collections of data are mentioned but are not JSON files.
- sections is an array called ‘websiteLinks’ that is in the gulpfile.js
- blog posts is data that is created via the gulpfile.js, stored in memory and then used by the gulpfile.js
- blog categories is data that is created via the gulpfile.js, stored in memory and then used by the gulpfile.js
###DATA! There are three ways data is used to build this project. All three methods are then passed into the Jade build using Jade locals.
- JSON files in /_src/markup/data => the idea here is that you can use Excel and make a datatable, export as CSV and convert it to JSON. I usually end up on this website to convert to JSON.
- javascript array for the websiteLinks, or sections as they are called once they enter Jade
- using node fs to loop over and read the files in the build to create a dataset for blog posts and blog categories
###MARKUP! The markup is all housed in the /_src/markup directory. This is just your standard Jade build. If you don’t understand this section, learn more about Jade and come on back. The real magic is using Jade locals to pass data in, so I’ll focus more on that. A short description of each file is above in the build.
###STYLES! I added just enough CSS to make the page have minimal layout. This article is about the markup, so I’m not going to focus on CSS.
###JAVASCRIPT! See the paragraph on styles above and replace ‘CSS’ with ‘javascript’.
###TASK MANAGEMENT! The gulpfile.js, this is where it all goes down. You can see the whole file in the Github repo. I’ll break down each section.
####The variables:
The modules includes:
var fs = require('fs'),
gulp = require('gulp'),
gutil = require('gulp-util'),
jade = require('gulp-jade'),
rename = require("gulp-rename"),
Some Regular Expression caching
titleRegExp = /var title =(.*?)\n/g,
catRegExp = /var categories =(.*?)\n/g,
descriptionRegExp = /var description =(.*?)\n/g,
publishDateRegExp = /var publish_date =(.*?)\n/g,
Initializing some variables
fileName = null,
pageTitle = null,
pageCategories = null,
Initializing the js objects for data storage
blogPostJson = new Object(),
blogCategoryJson = new Object(),
currentFile = null,
postCounter = null,
Storing some data
websiteLinks = ['HOME','ABOUT','RESUME','BLOG','CONTACT'];
The HTML task
gulp.task('html', function() {
postCounter = 0;
Use node’s fs module to read the blog directory and pass the data to the buildBlogData function
fs.readdir("./markup/blog", function(err,files){
if (err) throw err;
buildBlogData(files);
});
var buildBlogData = function(data){
Use map to loop through the data and collect the relevant information
data.map(function(url, ind, arr){
If the page is jade and is not the index or category page, then collect information from it
if(url.indexOf(".jade") > 0 && url.indexOf("index") < 0 && url.indexOf("category") < 0){
Use node’s fs.readFileSync to read the contents of each file
currentFile = fs.readFileSync("./markup/blog/" + url, 'utf8');
fileName = url; // url is from map’s anonymous function
pageTitle = currentFile.match(titleRegExp)[0].replace('var title = '','').replace(''\n','') || "NO VALUE"; // use RegExp to find the line with the title in it, and then strip out everything except the value. The next three do the same thing but for categories, description and published date.
pageCategories = currentFile.match(catRegExp)[0].replace('var categories = '','').replace(''\n','').split(",") || "NO VALUE";
pageDescription = currentFile.match(descriptionRegExp)[0].replace('var description = '','').replace(''\n','') || "NO VALUE";
pagePublishedDate = currentFile.match(publishDateRegExp)[0].replace('var publish_date = '','').replace(''\n','') || "NO VALUE";
Add the data to the blogPostJson object
blogPostJson["post" + postCounter] = {
"file":fileName,
"title":pageTitle,
"categories":pageCategories,
"description":pageDescription,
"published":pagePublishedDate
}
And use the pageCategories data to build out and object for the blogCategoryJson
pageCategories.map(function(category, ind, arr){
if(blogCategoryJson.hasOwnProperty(category)){
// do nothing
} else {
blogCategoryJson[category] = {
"files": new Array()
}
}
blogCategoryJson[category]["files"].push(fileName)
});
postCounter++;
}
});
buildHtml();
}
And back the the normal gulp-jade task where we then pass all the data collected above into the jade build.
var buildHtml = function(){
Build the pages
gulp.src('./markup/*.jade')
.pipe(jade({
pretty: false,
locals: {
'posts': blogPostJson, // built via our loop above
'sections': websiteLinks, // built in variables
'categories': blogCategoryJson, // build via our loop above
'skills': JSON.parse( fs.readFileSync('./markup/data/skills.json', { encoding: 'utf8' }) ), // an external JSON file
'services': JSON.parse( fs.readFileSync('./markup/data/services.json', { encoding: 'utf8' }) // an external JSON file),
'resume': JSON.parse( fs.readFileSync('./markup/data/resume.json', { encoding: 'utf8' }) ) // an external JSON file
}
}).on('error', gutil.log))
.pipe(gulp.dest('../'))
Build the blog index and post pages
gulp.src('./markup/blog/*.jade')
.pipe(jade({
pretty: false,
locals: {
'sections': websiteLinks,
'posts': blogPostJson,
'categories': blogCategoryJson
}
}).on('error', gutil.log))
.pipe(gulp.dest('../blog/'))
buildCategories(blogCategoryJson);
}
And build the category pages. This one loops through blogCategoryJson and builds a page for each key in the object
var buildCategories = function(data){
for(key in data){
console.log("buildCategories", key, data[key]);
gulp.src('./markup/blog/category.jade')
.pipe(jade({
pretty: false,
locals: {
'key': key,
'categories': data,
'sections': websiteLinks,
'posts': blogPostJson,
}
}).on('error', gutil.log))
rename the file based on the key in the object
.pipe(rename({
basename: key,
prefix: "category-",
}))
.pipe(gulp.dest('../blog/'))
}
}
});
So that was a lot to take in, but I hope my comments cleared up what the intention is at each pause.
So now we have all the data we need available for our Jade loops. You’ll notice that we call the ones used on each page in the template files. (_src/markup/templates/ main.jade and post.jade)
- var sections = locals['sections']
- var posts = locals['posts']
And then we call specific data on pages where it is needed, like in the markup/index.jade file. The page loops over services and skills, so I needed to include the data there
- var services = locals['services']
- var skills = locals['skills']
That’s it. Complicated and simple all in one!