Hi, I'm Frederic.

Web Developer & Entrepreneur
blogging about life and programming in Node.js, Go and React.

GithubTwitterEmailKeybase

Testing web applications made fast and easy

Many other content already discusses about the advantages of testing. The advice almost always goes something like this: “Testing won’t add much to development time but will save your ass more than once on bugs and that’s before they even reach your customers. Plus, it has the nice side effect of making you write cleaner code if you write tests before code”.

I am here to talk about the two main pain-points developers have with writing tests, they are hard to write and lengthen the feedback loop (especially with large codebases).

Now those two factors are probably the biggest detractors of, first, the people looking to get into testing and TDD and, for the second, people writing test but hating it as their test suite is so slow it can take 1 hour to run.

That often leads to you pushing to the CI and hoping what you wrote didn’t break anything elsewhere, it’s an hour wait so you context switch to an other task, come back to it later, it failed, re-checkout the git branch fix the little details, push again…


Now, I am no different and here is what I propose: ~80% of tests you write are unit tests. That is 80% of tests you write and, in terms, ~80% of the running time of your test suite. Why not, forget about a real database, forget about HTTP, forget about all dependencies mock them all and only run (in Node.js’s case) pure JavaScript across just the few lines in the function you are currently testing, no other part of the codebase.

Here, let’s say you have this controller with a method fetching a list of users:

class UsersController {
  constructor(userRepository) {
    this.userRepository = userRepository;
  }

  users(req, res) {
    const limit = req.query.limit || 20;
    const order = req.query.order || 'created';

    return this.userRepository.find({limit, order})
      .then(users => {
        res.status(200);
        res.json({data: users});
      }, next)
  }
}

Now, the route most often taken to test this part of the code is to reach out for a library to do an http request and make sure you have a test database setup and also create few models so that you know what to look for in the controller’s response, all while making sure the database tables are cleared in between tests as you don’t want all those models from other tests showing up in the controllers response…

A lot to think about, a lot of setup, quite slow because of all the moving components and really, looks more like integration testing (which you should still be doing here and there) than unit testing.


How about creating a fake request object and a fake response, and just for this controller a fake userRepository that will help us verify that for a given input, the correct calls are made by the piece of code being tested.

// fake-request.js
class FakeRequest {
  constructor() {
    this.query = {};
    this.body = {};
    this.params = {};
  }
}
// fake-response.js
class FakeResponse {
  constructor() {
    this.statusCode = 200;
    this.jsonBody = null;
    this.endCalled = false;
  }

  json(value) {
    this.jsonBody = value;
  }

  status(code) {
    this.statusCode = code;
  }

  end() {
    this.endCalled = true;
  }

  // ...
}

Then with those two you can start writing a test specific to that controller’s users method:

// test/controllers/users.js
const assert = require('assert');
const FakeRequest = require('...');
const FakeResponse = require('...');
const UsersController = require('...');

describe('controllers:users', () => {
  let fakeUserRepository = {};
  let fakeRequest;
  let fakeResponse;
  let fakeNext;
  let usersContoller;

  beforeEach(() => {
    usersContoller = new UsersController(fakeUserRepository);
    fakeRequest = new FakeRequest();
    fakeResponse = new FakeResponse();

    fakeUserRepository.findOptions = null;
    fakeUserRepository.find = (options) => {
      fakeUserRepository.findOptions = options;
      return Promise.resolve([]);
    };

    fakeNext = (error) => {
      fakeNext.givenError = error;
    };
  });

  describe('users()', () => {
    it('calls userRepository defaulting to a limit of 20', () => {
      return usersContoller.users(fakeRequest, fakeResponse)
        .then(() => {
          assert(fakeUserRepository.findOptions);
          assert.equal(fakeUserRepository.findOptions.limit, 20);
        });
    });

    it('calls userRepository defaulting to ordering by creation date', () => {
      return usersContoller.users(fakeRequest, fakeResponse)
        .then(() => {
          assert(fakeUserRepository.findOptions);
          assert.equal(fakeUserRepository.findOptions.order, 'created');
        });
    });

    it('calls userRepository respecting query parameters', () => {
      fakeRequest.query.limit = 5;
      fakeRequest.query.order = 'name';

      return usersContoller.users(fakeRequest, fakeResponse)
        .then(() => {
          assert(fakeUserRepository.findOptions);
          assert.equal(fakeUserRepository.findOptions.limit, 5);
          assert.equal(fakeUserRepository.findOptions.order, 'name');
        });
    });

    it('calls next on userRepository error', () => {
      fakeUserRepository.find = () => Promise.reject('repo error');

      return usersContoller.users(fakeRequest, fakeResponse, fakeNext)
        .then(() => {
          assert.equal(fakeNext.givenError, 'repo error');
        });
    });

    it('sends the right response', () => {
      const result = [{id: 89, name: 'Jack'}, {id: 41, name: 'Dooey'}];
      fakeUserRepository.find () => Promise.resolve(result);

      return usersContoller.users(fakeRequest, fakeResponse)
        .then(() => {
          assert.equal(fakeResponse.statusCode, 200);
          assert.deepEqual(fakeResponse.jsonBody, {data: result});
        });
    });
  });
});

Sorry for the long file, but that’s all there is to it, running that test file takes no more that 2-3 milliseconds. A full test suite for a bigger project might be more like 20 seconds, a far cry from 20 minutes.

I know it isn’t perfect, there is a lot that happens in between component and you really can’t always write tests verifying exactly the right outputs but that’s why integration testing still has it’s place, to make sure everything integrates properly. Just don’t do it in place of actual unit tests and as 90% of the test suite.

As always, take this with a grain of salt, your projects are different than mines for sure, if you find that way of writing minimal unit tests promising try it out see if it sticks.

Bringing sanity to growing Node.js applications

It seems like most of the content written and blogged about Node.js, even now, 6 years in, takes a really basic approach to showing you how to build applications.

A lot of Node.js articles explain Express.js the leading web framework, the problem is, this framework is comparable to Sinatra in Ruby or Flask in Python or Silex in PHP. Good for small few pages website, basically gives you routing and an interface to HTTP but not much more.

Now Ruby, Python and others have bigger frameworks that are well suited for larger project where you benefits from more architecture, opinionated defaults and supporting modules (ORMs, utilities, rendering, mailing, background workers, assets pipelines). The story is a bit different in Node.js as it promotes small npm modules (i.e. gems, packages) that you put together by yourself, which, is a good thing, most experienced developers prefer libraries over frameworks, but, there is no literature or examples of how this can be done within the Node.js ecosystem.

So, to solve this, let’s try and define few libraries or simple files that can help us out with our growing codebase.

Goals

The goals here are to have something easier to maintain than an app.js file, a routes/, models/ and views/ folder that’s it. To achieve this we are going to go on a hunt and steal few time tested tricks from other ecosystems.

Dependency injection

Some people seem to dread this one, others love it. Having experienced it a lot in Laravel a great framework for PHP and in Java in quite a few places, dependency injection can help keep all our application parts and files decoupled. Leading to way easier unit testing and modification of dependencies.

The concept is, all of your request handlers/controllers have dependencies, your models/repositories/entities too, you could go and hard code them by requiring the right file and using it but if you let a dependency injection container do it for you, you can more easily change that required components by a different implementation of it and, when testing, you can directly pass in stubs/mocks without any trickery or magic.

So, how would we go about implementing this?

First step is to have a file that represents the global instance of the container. That is, where all instances will be stored and the tool that resolve needed dependencies when you’ll want to instantiate a controller.

It would look like this:

import Container from './lib/contrainer';
export default new Container();

Then, in your app.js you can register libraries you want to make available to the following classes you’ll register/use.

import express from 'express';
import container from './container';

let app = express();
// ... middlewares, config ...

// Manually setting intance
import EventEmitter from 'events';
container.set('events', new EventEmitter());

// Automatically resolving dependencies and setting an instance
container.load(require('./lib/config'));
container.load(require('./lib/db'));
container.load(require('./lib/auth'));
container.load(require('./repositories/user'));
container.load(require('./services/billing'));

// Using container to resolve dependencies but
// giving back the instance insted of setting it.
let requireUser = contrainer.get('auth').requireUserMiddleware;
let userController = container.create(require('./controllers/user'));
app.get('/users/:id', requireUser, userController.showUser);
app.get('/users/create', requireUser, userController.showCreateUser);
app.post('/users', requireUser, userController.createUser);

// ... error handling ...

app.listen(contrainer.get('config').get('port'));
container.get('events').emit('app:started');

(When you grow to have many more routes, extracting those to their own routes.js is a good idea)

The final piece being the DI container it-self. I tried making it as compact as possible.

import R from 'ramda';

export default class Container {
  constructor() {
    this.contents = {};
  }

  get(name) {
    if (!(name in this.contents)) {
      throw Error('Container has nothing registered for key ' + name);
    }
    return this.contents[name];
  }

  set(name, instance) {
    this.contents[name] = instance;
  }

  create(klass) {
    if (!('length' in klass.dependencies)) {
      throw new Error('Invariant: container can\'t resolve a class without dependencies');
    }

    var dependencies = R.map(function(dependencyName) {
      return this.get(dependencyName);
    }.bind(this), klass.dependencies);

    return applyToConstructor(klass, dependencies)
  }

  load(klass) {
    if (typeof klass.dependencyName !== 'string') {
      throw new Error('Invariant: container can\'t resolve a class without a name');
    }

    this.set(klass.dependencyName, this.create(klass));
  }

  unset(name) {
    delete this.contents[name]
  }

  reset() {
    this.contents = {};
  }
}

function applyToConstructor(constructor, args) {
  var newObj = Object.create(constructor.prototype);
  var constructorReturn = constructor.apply(newObj, args);

  // Some constructors return a value; let's make sure we use it!
  return constructorReturn !== undefined ? constructorReturn : newObj;
}

Repositories, Entities and Services instead of large Models

It’s been told on many blog posts and talks for a good while now that fat models are evil. The ActiveRecord pattern that’s so prevalent in Rails and many ORMs is easily replaced by separating concerns:

  • Data representation goes in models. Those are as dump as possible, optimally immutable.
  • Fetching/Saving/Database interactions are made in repositories. Those take plain models and knows how to persist them and query datastores.
  • Business logic goes into services. Services is the place where most of the complexity resides, it’s what controllers call with input, what validates business rules, what’s calling repositories and external apis.

To give concrete examples:

An entity is a simple POJO/PORO/POCO…

import R from 'ramda';

export default class InvoiceLine {
  constructor(params) {
    R.mapObjIndexed((v, k) => this[k] = v, R.merge(User.defaults, params));
  }

  taxAmount() {
    return this.price * this.taxes;
  }

  total() {
    return this.price + this.taxAmount();
  }
}
InvoiceLine.defaults = {price: 0, taxes: 0.15, created: Date.now()};

A repository will most likely take a database object in it’s constructor to be able to interact with the datastore. Repositories are singletons loaded once when stating the app using the container’s load method.

import User from '../entities/user';
const TABLE_NAME = 'users';

export default class UserRepository {
  constructor(db) {
    this.db = db;
  }

  findByEmail(email) {
    return this.db.select('id, name, email, ...')
      .from(TABLE_NAME)
      .where('email = ?', email)
      .limit(1)
      .exec();
  }
}
UserRepository.dependencyName = 'repositories:user';
UserRepository.dependencies = ['db'];

A service is the simplest of the 3 in form but the one in which most complexity will hide. It simply has instance methods and dependencies listed to it can be registered in the container for controllers to depend on.

export default class BillingService {
  constructor(userRepository, stripeService, mailer) {
    this.userRepository = userRepository;
    this.stripeService = stripeService;
    this.mailer = mailer;
  }

  createNewAccout(name, email, password, stripeToken) {
    // validate
    // create user
    // create stripe customer
    // update db user
    // send welcome email
    // ...
  }

  // ...
}
BillingService.dependencyName = 'services:billing';
BillingService.dependencies = [
  'repositories:user', 'services:stripe', 'mailer'
];

Slimmer Controllers in favor of Services

Now that we have a dedicated place to put business logic, you should aim to slim down those controllers to their essential job: mapping requests and the http protocol oddities to method calls/actions to be taken.

This simple action has the new benefit of decoupling yourself from the transport protocol enabling reuse of all that business logic by other consumers like: background workers, a websocket endpoint, a protobuff endpoint even a separate codebase if you decide to extract the core of your app into a library when you grow bigger.

Factories

As your project grows and your entities become more complex you may come to a point where you find yourself spending a lot of lines initializing entities in your services, it’s a good idea to extract those to factories. Those object will give you a clean way to encapsulate complex entity construction with many branches.

The lib folder still exists

Not everything fits into the concepts we just went over, there are few middlewares, really simple libs or wrapper and definitively have their place in your lib folder, just try to keep it lean and mean, most of your code is supposed to be elsewhere.

Conclusion

I hope this post gave you ideas on how to reduce the size and complexity of your routes files/folder. Code organization (/architecture) starts simple in a new project but needs to grow linearly as your project matures or your productivity will suffer quite a bit.

I would love to know how you deal with growing codebases too! DM on Twitter or send me an email.

Welcome to Hugo!

It’s been a little while I wanted to switch to a new bloging platform. My old solution was a tool a built myself (often unwise when a lot already exists) and was pretty basic. It got the job done, but did really have space for growth.

I am now joining the ranks of people using the static website generator called Hugo. Advantages it encompasses go from blazing fast compilation, by more than simply bloging, to simple in design but infinite in possibilities with concepts like content types (not plain blog posts) and taxonomies). Making themes for it was not an afterthought and was really well designed.

This was an occasion to try something new design wise, I wanted something more simple, a different layout, and, mainly, simple CSS and a serif font.

How hard is it?

Well, Hugo is a bit different in the way it approaches content but still easy to understand if you come from other static website generators like Jekyll or Metalsmith.

For a new site simply invoke (from the command like)

hugo new site <FOLDER>

When I wish to create a new post I type:

hugo new posts/2015-09-18-welcome-to-hugo.md

I will then base itself from the template at _archetypes/default.md and fill in the title and current date for me.

Next, firing up a development server is as simple as:

hugo server --buildDrafts -w

The --buildDrafts ensure you preview the posts that have draft = true in their metadata, allowing you to work on posts, put them on hold, and still keep publishing other posts.

When ready to publish something you simply invoke:

hugo

This will generate all the necessary html & static assets in the public directory that you can them upload or commit to Github Pages.

Hope this makes it less scary to get into! Hugo has good documentation as really is a seriously good option in the market of static site generators used for blogs.

4 reasons for slowing down on the grind and enjoying life

Every now and then you’ll have moments in your life here it feels way more difficult to to get focused and work. You try to force it but still end up procrastinating. It’s almost impossible to get in the flow, get more that an hour or two of straight work.

I think possible reasons for that are, in my case at least, a new environment to adapt to, lack of routine, a bad diet, being in between two projects, no really motivating/interesting project, you could surely uncover more…

Here’s the thing: you shouldn’t be that situation, make the best of it instead. Not fighting it will even help you get your productivity back faster in my experience.

  1. There is no time like now. You wont get to relive that week, month, year, decade so, keep that in mind and enjoy yourself. Worrying about the future or regretting the past wont lest to a great future neither a nice past to look back to. Plus, traveling and many other activities are really not experienced the same at 25 than at 45 or 65 so, don’t always push back and just do it.
  2. Spend time working on other spheres of your life. Work is not everything, there is more to it, and, saying you are working “for your family” doesn’t mean you can omit spending real time with them NOW. Those personal relationships with your loved one, kids, best friends is something that lives on the most important resource you have: time. Without it, they die, and I am telling you, no work, how ever fulling is worth loosing a dear one. And when I say “other spheres” that includes taking care of your health, making time for a good diet, having hobbies.
  3. Balance will make you perform better in everything. Countless researches shown that exercising, working normal days (not more that 10ish hours), having diversified activities, eating well, makes you time working way more productive that otherwise. I see no disadvantage here, you live a more healthy and fulfilling life while doing better work.
  4. Slowing down is normal, and you will spring back soon enough. The important thing here is not to fight those less productive moments. Sometimes it’s because something else in your life needs fixing or simply more attention. Sometimes it’s because work is not that interesting. Let it flow normally and I guarantee this wont be permanent, you’ll will spring back in hyper focused and productive mode sooner than you think.

My Housing Situation

Nowadays I always try to consider all my options, especially including the uncommon ones, even more when it comes to lifestyle choices.

At the moment I am torn apart in terms of choosing a place to live from. I have quite a few options, some common, some uncommon, some expensive, some cheap, some classy, some less desirable. I hope it gives you ideas of what is possible.

Let’s explore some options you might be faced with:

  • Rent an apartment in a Canadian city center (Montreal)
    Price: 410 — Comfort: 610
    Advantages: Developed city, your own apartment, affordable (~900-1100$CAD/m). Personally: I know the city and have friends there.
    Disadvantages: Can’t cancel 12 month bail easily, not many travel destination close without crossing the ocean.
  • Rent an apartment in an eastern European city center (Warsaw)
    Price: 610 — Comfort: 710
    Advantages: Developed city, your own apartment, can travel easily to the whole Europe, cost of life can be cheaper than in the West.Personally: I get to learn more Polish.
    Disadvantages: Getting pretty costly, again, commit for 12 months renting the place.
  • Take AirBnB’s where I want 1 month at a time
    Price: 710 (can vary) — Comfort: 4
    Advantages: You get to see many cultures, there are some nice AirBnB but I still not like owning the place, you don’t have fixed commitment, travel to new places as you wish.
    Disadvantages: No stability can be exhausting, difficult to get something cheap that also looks nice in most Western cities, not that comfortable as you constantly feel you are living in someone else’s things.
  • Stay in hotels suites /w a kitchenette on long-term stay prices
    Price: 1010 — Comfort: 8
    Advantages: You live the high life/feel jet-set. Suites are really nice, they often include cleaning, internet, access to a gym, front desk 247. No commitment. You get to see many cities. Often really well located in cities.
    Disadvantages: Extremely costly (for my means), north of 3000$/month. Tiring to always move. Feels really temporary as, well, it’s an hotel!
  • Buy a small apartment in the greater area of an Eastern European city (Warsaw)
    Price: 510 — Comfort: 910
    Advantages: Best value for the money. It’s an investment, not simply giving you money away. Good for permanence/stability. In Europe so nice for travelling to neighbouring countries. Really nice to live from as it’s yours you can renovate and furnish as you like.
    Disadvantages: Long term commitment (I guess you could sublet). A bit less fun than living in the center. You still pay for it when you make a trip elsewhere.
  • Stay at parents or friends place
    Price: 110 — Comfort: 310
    Advantages: Really cheap.
    Disadvantages: Often not close to the city center, often not even in a city you’d like to be in. At best temporary. Doesn’t feel like home. You don’t want to bother your hosts with your presence for too long.
  • Camping in state parks, sleeping in your car or tent
    Price: 310 — Comfort: 210
    Advantages: Pretty cheap. Makes you see tons of amazing landscapes. Access to outdoor activities.
    Disadvantages: Finding WIFI is your constant quest, or, stay in 4G range and pay a bunch! Living conditions/standards lower than when living in a house for sure, but I can deal with this, I think. Eating personal hygiene occupies a good part of you day. You feel isolated and lonely pretty quickly.
  • Living on trains /w a monthly pass
    Price: 310 — Comfort: 110
    Advantages: Not costly at all. Can be fun at start. You get to see many cities. You can meet many people every day. You see nice landscapes in the viewing cart, if there’s one.
    Disadvantages: You are always siting. Personal hygiene is more tricky. Your food choices are limited if you don’t get off often. You don’t really build any stable friendships. You can get bored of it quickly, I guess.
  • Living in planes and airport lounges
    Price: 110 or 1010 — Comfort: 510
    Advantages: Living the jet-set life
    Disadvantages: Far from a permanent living situation. Rough on your body. You must be a hell of a travel hacker to pull it off. Still no kitchen and stable friendships.

I am still debating and researching many of there. I might do an update post when I settle on an option.