Some notes and examples of basic NodeJS (and JS in general) concepts.
- In a new directory, run
npm init
(guide you through creatingpackage.json
). - Create
src/index.js
(or whichever path appears asmain
in yourpackage.json
) - Edit
package.json
, and add thestart
script (the toscripts
objects) - Run
npm start
- Install a test runner,
npm install mocha --save-dev
- Install an assertion library,
npm install --save-dev chai
- Add
test/test.js
- Now, run the tests using
node_modules/.bin/mocha
Note: We runmocha
fromnode_modules/.bin
, so we don't depend on Mocha being globally available.
Once we change the test
script in package.json
, we can npm test
.
- Push the project into a GitHub repo
- Enable Travis-CI on that repo (in the repo's settings)
- Add
.travis.yml
and push to GitHub - Now, you can go to Travis-CI and see the build.
We can embed the badge to indicate the status of the latest build.
- Create
src/hello.js
- In
src/hello.js
, we populate themodule.exports
with values (usually functions). - In other modules (e.g.
src/index.js
) werequire(PATH_OR_NAME)
, which gives us back theexports
object. - After updating the code in
index.js
to use the function fromhello.js
, you cannpm start
. - Also, update the tests, and
npm test
.
ES6 (aka ES2015) is the newest standard of JavaScript. It's great, but there's one problem - It's not fully supported everywhere (in fact, in most runtimes it's not fully supported yet).
The solution - Transpile (i.e. translate/compile) ES6 source to ES5 (or any other JavaScript specs that are fully supported). Currently, the industry stanrad is Babel.
First, setup dependencies:
- Install dependencies
npm install --save-dev babel-core babel-cli babel-preset-es2015
- Configure Babel default by creating
.babelrc
Next, we can update our src/
and test/
code to use ES6.
Finally, make sure that npm start
or npm test
use the transpiled (not the original) source code:
- Setup the
scripts
object inpackage.json
.
To get a better feel for what programming in Node.js is like, we'll write a
small web scraping program in index.js
.
OK, we wrote a quick script, but now, let's do things right ...
Let's create some unit tests and the WebScraper
class to pass them.
Finally, we update the script to use our WebScraper
and the isomorphic-fetch
library. We now have a small utility that scrapes catalog pages from
www.ssense.com and prints JSON data to the console.
Next, we improve the API by supporting both sync and async operations -
Change scrape(html)
to scrape(html, callback)
and the tests that specify the
behaviour. (Q: Why/when is it important?)
Next, we will insert the information we scrape into a database. If you have Docker installed, you can easily start a test MySQL server:
docker run -p3306:3306 --name dev-server -e MYSQL_ROOT_PASSWORD=123456 -d mysql
-p3306:3306
is used for port forwarding, to allow us to connect to the server from the host machine atlocalhost:3306
.
Before we run our script, let's create a table for it to store the data.
First, find out the ip of your container:
# Find the id of the container
docker ps
# Get the IP
docker inspect --format '{{ .NetworkSettings.IPAddress }}' CONTAINER_ID
Then, connect to the server from a MySQL client:
docker run -it --link dev-server:mysql --rm mysql sh -c 'exec mysql -hCONTAINER_IP -P3306 -uroot -p123456'
And, setup the database and table(s):
create database test_db;
use test_db;
CREATE TABLE products (
sku VARCHAR(24) NOT NULL,
brand VARCHAR(128) NOT NULL,
name VARCHAR(128) NOT NULL,
image VARCHAR(1024) NOT NULL,
PRIMARY KEY (sku)
);
This script is very similar to our index.js, only it saves the scraped data in the database.