Benchmark Tool

This repo is intended to measure performance of all popular Node.js web frameworks alongside with the ORM.

List of web framework included in test:

List of ORM included in test:

Bookshelf
Knex
mysql
mysql2
Sequelize
static (hardcoded using Promise.resolve)

Running

Running benchmark

npx tsc --project .
node bench.js

Running individual web framework + specific ORM:

npx tsc --project .
# node index.js [web-framework] [orm]
node index.js bare static # open http://localhost:3000

To run the ORM benchmark:

You need to have a MySQL running in your machine.
Create a database and import the CountryCodes.sql file.

mysql -u root -p -e 'CREATE DATABASE Geo' # create new database
mysql -u root -p Geo < CountryCodes.sql   # import from file

Project Structure

app
  [name].js
orm
  [name].js

The app folder contains web framework entrypoint. Meanwhile the orm folder contains ORM definition.

Writing New Web Framework to be Tested

// @ts-check
'use strict'

const c = require('../constanta')

/**
 * @param {() => Promise<import("../lib").ICountryCodePair[]>} fetchAll
 */
const createApp = fetchAll => {
  // 1. Initialize your application
  const app = new MyApp()

  // 2. Overview: fetchAll is an async function, your ROOT route of your application
  //    should be able to handle incoming GET request.
  app.get('/', (req, res) => {
    fetchAll()
      .then(data => {
        // If there is no error:
        // 2.1. The status code must be 200.
        // 2.2. The response must have `Content-Type: application/json; charset=utf-8` header.
        // 2.3. The response body must be a valid JSON, which it must be the representation of
        //      anything that fetchAll function returns.
        res.status(200).json(data)
      })
      .catch(error => {
        // If there's an error:
        // 2.4. The status code must be 500.
        // 2.5. The response must have `Content-Type: application/json; charset=utf-8` header.
        // 2.6. The response body must be a valid JSON, which it must be the representation of
        //      { message: string }
        res.status(500).json({ message: error.message )
      })
  })

  // 3. The application must listen on port 3000.
  app.listen(3000, () => {
    // 3.1. After the application ready to serve connection, send a message to parent process.
    process.send && process.send(c['SERVER_EVENT::SERVER_START'])
  })
}

Writing New ORM to be Tested

// @ts-check
'use strict'

// The ORM should use MySQL as it's datasource
const orm = new MyOrm({
  host: 'localhost',
  user: 'root',
  password: 's3cr3t',
  database: 'Geo',
  // If you use pool connection, use this configuration:
  // - minimum number of connection in your pool: 5
  // - maximum number of connection in your pool: 50
  pool: {
    min: 5,
    max: 50
  }
})


/**
 * @returns {Promise<import("../lib").ICountryCodePair[]>}
 */
const fetchAll = () => new Promise((resolve, reject) => {
  // The fetchAll should get all contents in `CountryCodes` table.
  orm.fetchAll('CountryCodes', (err, result) {
    if (err) {
      reject(err)

      return
    }

    resolve(result)
  })
})
// Lastly, export the fetchAll function.
exports.fetchAll = fetchAll

Result

Configuration:

ORM: static
AWS tier: t2.large
Node.js version: v11.13.0
OS: Ubuntu 18.04.2 LTS
Kernel: 4.15.0-1032-aws
Commit ID: 40a1bde