Learn how to build a service that manages JSON data using the REST architecture. We'll also learn how to use bearer tokens to authenticate our API, and Insomnia to test it in development.
REST is a set of recommendations for how to structure services that communicate over HTTP. It's not an official standard or specification, but it is designed to fit the HTTP protocol.
For example you access data via HTTP methods and paths. A request like GET /api/todos would return a list of all the todos. DELETE /api/todos/1 would delete the todo with ID 1. The path represents a "resource".
Resources are an important HTTP/REST concept. A URL is a "uniform resource locator"—http://example.com/api/todos tells you/the client where the /api/todos resource is located. The HTTP method is how the client tells the server what action it wishes to take on the resource (e.g. retrieving it, updating it, deleting it etc).
A resource can be a real file (e.g. todos.json) or it can be generated on the fly by a dynamic server pulling the todo data from a database. REST is designed so the client doesn't have to be aware of how the data is stored.
REST APIs usually implement four types of data access: create, read, update & delete. These can be represented by both HTTP methods and SQL commands:
| Operation | HTTP | SQL |
|---|---|---|
| Create | POST |
INSERT |
| Read | GET |
SELECT |
| Update | PUT |
UPDATE |
| Delete | DELETE |
DELETE |
This means for any given resource you're likely to have 4 routes: POST /resource, GET /resource/:id, PUT /resource/:id and DELETE /resource/:id. REST APIs often have a way to list all instances of a resource with GET /resource (without an identifier).
- Clone this repo
- Run
npm install - Run
npm run dev
The workshop/ directory contains a fake database that writes to workshop/database/db.json. It also contains a workshop/model/ directory that manages the data access. These are written for you so you can focus on the Express handlers and middleware.
You're going to build a REST API for a database of dogs. There are two "tables" in the "database": "users" and "dogs". Each dog has an "owner" property that is a user ID. Each user can own many dogs. You can see the data the database starts with in workshop/database/init.json.
Up until now our servers have mostly been responding with HTML. Since this server is designed to be a general purpose API (that could be consumed by a web app, mobile app etc) it's best if we respond with data in a useful generic format. JSON is usually the format of choice for modern APIs. Luckily Express handles JSON responses for us, so we can keep using res.send to send response bodies.
We'll start with reading data so we can get our server returning something. Create a workshop/handlers/dogs.js file and import the dogs model from ../model/dogs.
This file is where all the handlers for our dogs resources will live. Create a getAll handler function and export it. This handler should use model.getAllDogs to retrieve all the dogs in the database, then send them as the response body using res.send.
All the model functions return promises, so you'll need to use .then and .catch with them. If the handler catches an error it should call the next argument with that error to let Express handle it.
Solution
const model = require("../model/dogs");
function getAll(req, res, next) {
model
.getAllDogs()
.then((dogs) => {
res.send(dogs);
})
.catch(next);
}Create a GET /dogs route in workshop/server.js that uses the dogs.getAll handler. Visit http://localhost:3000/dogs in your browser and you should see an array containing a single dog object.
Solution
const dogs = require("./handlers/dogs");
server.get("/dogs", dogs.getAll);Now we need to do the same for the GET /dogs/:id route. Create a handler named get that uses model.getDog(id) to retrieve a single dog from the database. Add this route to your server, then visit http://localhost:3000/dogs/1586691897927 and you should see a single dog object.
Remember you can access route params (placeholder values) on the req.params object.
Solution
function get(req, res, next) {
const id = req.params.id;
model
.getDog(id)
.then((dog) => {
res.send(dog);
})
.catch(next);
}server.get("/dogs/:id", dogs.get);A web browser is not a great tool for developing JSON APIs. Chrome especially has no built-in JSON formatting, making it a bit awkward. It's also annoying to send anything but a GET request. Instead, we'll be using Insomnia to test our server. This is a nice tool for sending any type of HTTP request. You may have tried a popular app called Postman — we're using Insomnia instead as the interface is simpler and a bit more intuitive.
Create a new request (Ctrl-N) called 'Get Dogs'. Copy localhost:3000/dogs/ into the request text input, click "Send" you should see the JSON response appear below.
Now we need to add a route for adding new dogs to the database. Since this is a JSON API we need to be able to receive JSON data. For example until now we've had data submitted via HTML forms, which means the POST bodies had a content-type of x-www-form-urlencoded. Now we'll be receiving bodies formatted as application/json.
Luckily Express has built-in support for JSON bodies, just like urlencoded ones. Add the express.json middleware to your server in workshop/server.js.
Solution
server.use(express.json());Create a handler function named post that gets the submitted data from req.body, then uses model.createDog to add a dog to the database. Once the dog is created respond with a 201 status code and the new dog object. 201 means "new resource created". Make sure you respond with the object that createDog resolves with (rather than the user-submitted one) since it will contain the database-generated ID.
Add this handler to your server for the POST /dogs/ route.
We can test this endpoint using Insomnia. Create a new request and change the method to POST. Keep the URL as localhost:3000/dogs. Click the drop down labelled 'Body' and select JSON. This will let us send a JSON body with our POST request (and automatically add a header of Content-Type: application/json). Enter this data in the text area:
{
"name": "Pongo",
"breed": "Dalmation",
"owner": 1586691863221
}For now we're hard-coding the owner, as we haven't implemented authentication yet. Submit this request and you should see a 201 response with a body like this:
{
"id": 1586716616202,
"name": "Pongo",
"breed": "Dalmation",
"owner": 1586691863221
}Solution
function post(req, res, next) {
const newDog = req.body;
model
.createDog(newDog)
.then((dog) => {
res.status(201).send(dog);
})
.catch(next);
}server.post("/dogs", dogs.post);
Finally we need a route for deleting dogs from the database. Create a new route for DELETE /dogs/:id. Write a new handler called del (delete is a reserved word in JS so we can't use that). It should use the ID route param to delete the dog from the database with model.deleteDog. Once this is done respond with a 204 ("no content") status code and an empty body. There's nothing to return from a delete (other than indicating the operation was successful).
Solution
function del(req, res, next) {
const dogId = req.params.id;
model
.deleteDog(dogId)
.then(() => {
res.status(204).send();
})
.catch(next);
}Test this using Insomnia to send a DELETE request with the ID of the dog you want to delete in the URL. You should see a 204 response with no body.
Our API is currently totally unprotected. This means anyone can add a dog with any owner, and delete any dog. For most general APIs only the GET routes should be public: anyone can see a list of the dogs, but only the dog's owner should be able to change or remove them.
First we need a way for users to sign up. Our API should treat users like any other resource—ideally we should have routes for creating, reading, updating and deleting them. In the interests of time we'll just implement creation right now.
Create a new file at workshop/handlers/users.js. Inside create a function called post that gets the submitted user data from req.body and creates a new user with model.createUser. Respond with a 201 status code and the the new user object (except for the password, which should be secret!).
Note: in a real API we'd want to verify that the user's email was unique, since we'll be using this to verify users.
Add a new POST /users route to your server that uses the users.post handler.
Solution
function post(req, res, next) {
const userData = req.body;
model
.createUser(userData)
.then((user) => {
const response = {
id: user.id,
name: user.name,
email: user.email,
};
res.status(201).send(response);
})
.catch(next);
}Test this using Insomnia by sending a POST request to localhost:3000/users with a body like:
{
"email": "oli@example.com",
"password": "123",
"name": "oli"
}You should see a 201 response with the new user object as the body.
{
"id": 123456,
"email": "oli@example.com",
"name": "oli"
}We can create a user, but we have no way for subsequent requests to prove they have been made by that user. If we provide a token containing their ID they can send this on all subsequent requests to authenticate themselves. This is similar to the browser sending a cookie with every request.
We'll use JWTs as our tokens. We need a secret to sign our JWTs with so we can verify they haven't been tampered with. We should keep this secret, well, secret, otherwise anyone can sign our tokens and we can't trust any of them.
Add a .env file to the root of the project with a JWT_SECRET environment variable. This needs to be a long random string.
JWT_SECRET=mn6Ak%8fbaf$ur2u£uka*8avaNow we can use dotenv to access this on process.env.JWT_SECRET in our workshop/handlers/users file. Use the secret to sign a JWT containing the newly created user's ID. You should also set an expiry for the token so the user isn't logged in forever:
jwt.sign(userStuff, SECRET, { expiresIn: "1h" });We need to send the JWT as part of the response object, so add an access_token property to the object we're sending.
Try sending a POST /users again in Insomnia. Now you should see an extra access_token property in the response. This is a JWT containing the user's ID.
{
"id": 123456,
"email": "oli@example.com",
"name": "oli",
"access_token": "ey234adsfd.afnd..."
}Solution
require("dotenv").config();
const SECRET = process.env.JWT_SECRET;
function post(req, res, next) {
const userData = req.body;
model
.createUser(userData)
.then((user) => {
const token = jwt.sign({ user: user.id }, SECRET, { expiresIn: "1h" });
const response = {
id: user.id,
name: user.name,
email: user.email,
access_token: token,
};
res.status(201).send(response);
})
.catch(next);
}Our token currently expires after one hour. Users can't keep creating new accounts to get new tokens, so we need to add a route allowing them to log in and get a new token. Create a POST /login route on your server.
Create a handler named login that gets the submitted email and password from req.body. It should then use model.getUser(email) to find the user in the database and compare the submitted password to the stored one.
Note: our passwords are in plaintext here. This is bad so never do it in a real app. Implementing hashing here would distract from the goal of this workshop.
If the passwords do not match then create a new error with a message of "Unauthorized". Set a status property on the error object with a value of 401. Then call next with the error to pass it on to the error-handling middleware.
If the passwords match then sign a new JWT using process.env.JWT_SECRET and respond with a 200 and an object with just the access_token property. We don't need to send the whole user object since this endpoint is just for logging in and getting a token.
Solution
require("dotenv").config();
const SECRET = process.env.JWT_SECRET;
function login(req, res, next) {
const email = req.body.email;
const password = req.body.password;
model
.getUser(email)
.then((user) => {
if (password !== user.password) {
const error = new Error("Unauthorized");
error.status = 401;
next(error);
} else {
const token = jwt.sign({ user: user.id }, SECRET, { expiresIn: "1h" });
res.status(200).send({ access_token: token });
}
})
.catch(next);
}Since this API may be used by non web browser clients (like mobile apps) it can't use cookies. Instead any request for a protected resource must have an authorization header containing a "bearer token". E.g. Bearer 12345 (where 12345 is a valid token). Let's create a middleware that can grab this token, extract the user, then put it on the request object for subsequent handlers to use.
Create a new file workshop/middleware/auth.js. Write a middleware function named verifyUser. It should read the authorization header from the request object to get the bearer token. If the header isn't present create a new error with a status property of 400 ("bad request"). Call next with the error to pass it to the error-handling middleware.
To get the token out of the header you'll need to remove the "Bearer " bit (hint: string.replace).
Once you have just the token use the JWT library to verify it and get the decoded user ID. Don't forget to use dotenv to access the JWT_SECRET environment variable.
We want to grab the user whose ID matches the one in the token from the database, then attach it to the request object. That way all our other handlers will be able to access the authenticated user.
You can use model.getUserById to get the user from the database. Don't forget to call next() when you're done to pass the request on to the next handler.
If the JWT verification fails you should create an error with a status property of 401 and call next with it.
Solution
function verifyUser(req, res, next) {
const authHeader = req.headers.authorization;
if (!authHeader) {
const error = new Error("Authorization header required");
error.status = 400;
next(error);
}
const token = authHeader.replace("Bearer ", "");
try {
const data = jwt.verify(token, SECRET);
model
.getUserById(data.user)
.then((user) => {
req.user = user;
next();
})
.catch(next);
} catch (_error) {
const error = new Error("Invalid token");
error.status = 401;
next(error);
}
}Now that we have users and tokens working we can protect the routes that should be private. Add the verifyUser middleware before the POST /dogs and DELETE /dogs/:id route handlers. Now requests with no token (or an invalid token) will be rejected before reaching our handlers.
Solution
server.post("/dogs", verifyUser, dogs.post);
server.put("/dogs/:id", verifyUser, dogs.put);
server.delete("/dogs/:id", verifyUser, dogs.del);Test this in Insomnia by trying to create a dog without sending an authorization header. You should receive a 400 error.
Now add a random invalid authorization header. Insomnia has two ways to do this. You can go to the dropdown labelled 'Auth' and select 'Bearer Token', and set the Token value to 1234.
It's probably best to use the inbuilt authorisation functionality, however you can manually set the header in the 'Header' tab (with a key of "authorization" and a value of Bearer 1234). You should receive a 401 error either way.
Finally log in as a real user, then send their access_token as the authorization header. The dog should be created successfully.
However this isn't quite enough. Currently any logged in user can delete any dog, since we're just checking for a valid token. We need to amend our DELETE handler to get the authenticated user's ID from req.user, then check if that ID matches the dog's owner property. If it does not create a new error with a 401 status property and call next with it.
Solution
function put(req, res, next) {
const userId = req.user.id;
const dogId = req.params.id;
const newDog = req.body;
model
.getDog(dogId)
.then((dog) => {
if (dog.owner !== userId) {
const error = new Error("Unauthorized");
error.status = 401;
next(error);
} else {
model.updateDog(dogId, newDog).then((dog) => {
res.status(200).send(dog);
});
}
})
.catch(next);
}
// del is the sameTest this in Insomnia by creating a new user, then trying to delete another user's dog sending the new user's token in the authorization header. You should get a 401 response. Then log in as the dog's owner and try with their token. This time the dog should be deleted successfully.
The final step is to amend our POST handler. Currently it allows the dog's owner to be set to any user ID. We should instead take the user ID from req.user to ensure that a dog's owner is always the currently authenticated user.
Solution
function post(req, res, next) {
const user = req.user;
const dog = req.body;
dog.owner = user.id;
model
.createDog(dog)
.then((dog) => {
res.status(201).send(dog);
})
.catch(next);
}Test this in Insomnia by sending a POST request with a new dog object. Make sure you have a valid authorization header set. Don't send an owner property. The response should contain an owner property with the ID of the logged in user.
Since this is a general purpose API it will probably be accessed from domains other than the one it's deployed on. Make sure your server handles cross-origin requests correctly and responds with the right headers to allow anyone to talk to the API. Check out the cors middleware.
General purpose APIs should usually have version numbers. This lets you make breaking changing whilst maintaining backwards compatibility. For example if you wanted to change the name of one of the endpoints you'd have to make sure every client using your API had updated before you made the change, or you'd break all those apps.
Instead you can put a version number in the URL so clients can keep using the version they're on forever without it breaking. Add /v1/ to the front of all your routes. Now if you ever want to make a breaking change you can release a new version with /v2/ for all the URLs.