Intro
In the past couple weeks, I have been developing GoChat, a Facebook messenger bot that allows you to play a game of Go (Weiqi, Baduk) right in Facebook Messenger. It is essentially a Node server that accepts callbacks from Facebook whenever someone messages my Facebook page.
I have learned a ton from the project and I think what I used would be useful to y'all as well. I extrapolated a few things that I think any node projects improve any Node projects. Keep reading if you're working on a Node or Javascript project!
Source code
See source code for reference.
1. Install Babel
Everyone is using Babel and you should too. Babel is a Javascript compiler that allows you to write modern specification of Javascript (JS6 and proposed JS7) that the V8 engine or your browsers have yet to support. This page has a detailed list of nice features for ES6. There are tons of goodies like arrow functions, destructing objects and classes.
To take advantage of this right away, install the following. babel-cli is the command line interface for Babel. babel-preset-es2015 consists of all the plugins proposed for ES6.
> npm install --save-dev babel-preset-es2015
> npm install --save-dev babel-cli
Then create a .babelrc file in the root directory. This file will define the rules for the compiler.
{
"presets": ["es2015"],
}
Now, put all your source code in .src/ directory. We will setup npm scripts that compiles your code to .build/ folder.
// package.json
"scripts": {
"watch": "babel --watch=./src --out-dir=./build --source-maps inline --copy-files",
"build": "babel ./src --out-dir=./build --source-maps inline --copy-files",
...
}
When you develop, have a separate session (tmux) open that runs npm run build. The script will then listen to any changes in the src folder and auto-compiles the code. To make the process even better, add this to package.json. This will restart the node server when anything in build changes.
> npm install --save-dev nodemon
// package.json
"scripts": {
"server": "nodemon build/app.js --watch build",
...
}
Awesome! Now you can use any ES6 features in your Node app. Keep in mind that you can always include more babel plugins.
2. Flow typechecker
Like most scripting languages, Javascript is dynamically typed which means even tho you can code a bit faster but you would inevitably waste time catching stupid bugs. This is when Flow comes in. It uses type inference and type annotation to catch those stupid bugs.
> npm install --save-dev flow
Once you install flow, you need to specify which files you want flow to analyze. Add the following:
// @ flow
Then simply run the flow command to start checking your code!
You may notice that adding flow annotation is breaking babel compiling. No worries, simply install
> npm install --save-dev babel-plugin-transform-flow-strip-types
And add the babel plugin to the .babelrc file
{
"presets": ["es2015"],
"plugins": [
"transform-flow-strip-types",
],
}
Not only does Flow help catch bugs, it also makes your code much more readable! Knowing precisely what the input format and the return type is are extremely helpful to create a mental model for the program. Imagine the following function from GoChat which renders a list of stone positions on a board image.
playStones(stones, board, cb) {
lwip.open(board, function(err, boardImage) {
if(err) {
error('Cannot open board image', err);
cb(err);
} else {
_playStonesHelper(stones, boardImage, cb);
}
});
}
If I'm brand new to the code base this would be going through my head, "What are stones? Are they objects or array? What properties does it have? cb seems like callback but can I be exactly sure? Oh yeah okay it is..." As you see, the code could be interpreted in different ways and you are required to trace through other functions to have a better understanding.
Now, take a look at how this would look like using flow.
type StoneColor = 'black' | 'white';
type Stone = {
x: number,
y: number,
color: StoneColor,
};
playStones(stones: Array<Stone>, board: string, cb: Function): void {
lwip.open(board, function(err, boardImage) {
if(err) {
error('Cannot open board image', err);
cb(err);
} else {
_playStonesHelper(stones, boardImage, cb);
}
});
}
Much cleaner.
3. Linting with ESLint
Programmers are lazy and prone to make careless mistakes. And that's okay, as long as there are programs that clean up our messes. Linting tools are designed to do exactly that. JSLint and ESLint are both popular for javascript but the latter offers much better flexibility to add custom linting rules.
> npm install --save-dev eslint
Then add a .eslintrc file. Then you can define the rules you want from all the supported rules.
One tricky thing here is that since we are using Flow, our source code are not proper Javascript so ESLint will have parsing issues. Good news is that ESLint allows you to define custom parsers to transform into code that ESLint could understand.
> npm install --save-dev babel-eslint
Then in .eslintrc, define...
{
"parser": "babel-eslint",
"env": {
// in CommonJS
"node": true
},
"rules": {
...
}
}
ESlint is great for maintaining consistent coding styles. Personally, I prefer using trailing commas and the rule comma-dangle has been wonders. By running eslint src --fix with the fix param, ESLint even auto fixes places where I don't have trailing commas.
If you ever feel overwhelmed and not sure what rules to you use, you can always download the standard rule sets and call it a day.
4. invariant
Invariant is a little handy module that I used very heavily in my app.
> npm install --save invariant
Without getting into too much details of its importance in Software Engineering, using invariants helps me keep track of the correctness and assumptions of my functions. I use invariants a lot to assert the type and data correctness of function arguments and also to verify an action is proper given the current application/user state. For example, I have a series of functions that are only applicable to users currently in a game, so I would throw an invariant in these functions:
async resignGame(user: User): Promise<void> {
invariant(user.isPlaying(), 'User should be playing');
...
}
This not only catches careless errors but is a great code documentation for me and my partner. Install now and try it for yourself!
5. async and await
Javascript is asynchronous and single-threaded. This means for many operations such as talking to the DB, making a http request, the application will kickoff the function and then process thread will "unblock" itself and run on something else. That means your code will most likely have a bunch of callbacks. A hell of nested callbacks!
function doSomething(callback): void {
doSomethingFirst((result1) => {
okay(result1);
doSomethingSecond((result2) => {
more(result2);
doSomethingThird((result3) => {
callback(result3);
});
});
});
}
There are methods to make the code cleaner when you want to avoid nested callback hells - avoid nesting functions, modularize the callbacks, etc. Personally, callbacks just make make my code much more unreadable and makes it difficult to handle exceptions. I prefer writing serial code and the framework should handle the asynchronous processing.
Fortunately, you can use await and async functions proposed in ES7 features. The above code will be equivalent to
async function doSomething(): Promise<void> {
const result1 = await doSomethingFirst();
okay(result1);
const result2 = await doSomethingSecond();
more(result2);
return await doSomethingThird();
}
Isn't that much nicer? It is very clear how the logic flows and the code is much more concise.
You can use it right away using Babel compiler. Under the hood, it will convert async/await to generators.
> npm install --save-dev babel-plugin-syntax-async-functions
> npm install --save-dev babel-plugin-transform-async-to-module-method
And update your .babel
{
"presets": ["es2015"],
"plugins": [
...
"syntax-async-functions",
["transform-async-to-module-method", {
"module": "bluebird",
"method": "coroutine"
}]
],
}
Finally, since v8 doesn't provide generator functions, you need to import the Babel polyfill environment which adds a bunch of ES2015 features to the global scope.
> npm install --save-dev babel-polyfill
Then add to app.js
import 'babel-polyfill';
6. Test with Mocha
Always test your app. It's easy to forget about it and only focus on the business logic but trust me, at the end of the day, the only thing that makes me sleep at night is knowing that my code passes all the tests. When your app become more complicated, unit tests or integrations tests ensure any new changes don't result in regression or bugs.
For testing in Javascript there are plenty of options but I really enjoy using Mocha for its simplicaity.
> npm install --save-dev mocha
> npm install --save-dev chai
A tricky thing I had was being able to use async/await for my unit tests. It's actually quite simple, when starting the mocha script, you could specify Babel as the compiler. It will then compile the code on the fly and run the tests.
// package.json
"scripts": {
"test": "NODE_ENV=test mocha --compilers js:babel-register",
...
}
Let's see how to make a simple integration test that checks GET / returns 200. Note that I'm starting a new instance of server for each test case. This makes sure theres no order dependency and each test starts with a clean state.
// test/basicServerTest.js
import 'babel-polyfill';
import {expect} from 'chai';
import {mochaAsync} from './TestHelpers';
import request from "supertest-as-promised";
import makeServer from '../src/server';
describe('Basic server test', function() {
this.timeout(1000);
var server;
beforeEach(mochaAsync(async () => {
server = await makeServer(true /* silent */);
}));
afterEach(function (done) {
server.close(done);
});
it('Server healthy', mochaAsync(async () => {
await request(server).get('/')
.expect(200);
}));
});
Then run the test.
> npm run test
npm run test
> Node_tutorial@1.0.0 test /Users/spchuang/github/GoChat-tutorial/6-things-to-add-to-your-node-projects
> NODE_ENV=test mocha --compilers js:babel-register
Basic server test
✓ Server healthy (44ms)
1 passing (212ms)
That's it!
Alright, that's it!
One last thing, I generally want to run flow, lint and tests together when I develop. This is the handy script I use:
// package.json
"scripts": {
"check": "npm run lint && npm run flow && npm run test",
...
}
I hope you can try them out for your Node project because they have certain made my Node development a much smoother experience!