jamesingold@home:~$

Blog 

About 

Books 

Quotes 

RSS

Setting up a Serverless Project with Webpack, Babel, and Knex

Using Webpack with the Serverless Framework is handy if you want to use the latest Javascript features along with Babel. It also helps to optimize the packaging of functions so we can make sure we’re only shipping code that is lean and mean. However, adding the delightful query builder Knex to the mix can cause some issues that I spent a good amount of time on. Hopefully this article will help anyone dealing with similar issues save some time.

In this article we’ll go through setting up a Serverless project with Webpack, Babel, and Knex along with Prettier and Eslint. We’ll focus on specific issues with Knex in this scenario and how to solve them. If you want a TLDR; here’s the final output, a Serverless starter template with Webpack, Babel, and Knex ready to go.

Project Setup

Install serverless globally

npm install serverless -g

First, we’ll set up a new Serverless project using a default aws-nodejs template:

serverless create --template aws-nodejs

This will create a bare handler.js and a serverless yaml file to get us started.

Next add a package.json file to manage our dependencies

npm init -y

Add Dev Dependencies and Webpack:

We’re going to add Babel to get access to the latest Javascript features and then we’ll add Webpack to transform our Javascript code in a way that the Serverless platforms (AWS) can handle. We’ll also add Serverless-Offline which emulates AWS and AWS Gateway, allowing us to run our functions locally.

npm install --save-dev @babel/core @babel/preset-env webpack serverless-webpack serverless-offline babel-loader dotenv

Adding Source Map Support

It’s always nice to get stack traces, let’s set up source map support.

npm install source-map-support --save npm install
babel-plugin-source-map-support --save-dev

The source-map-support module provides source map support for stack traces in node via the V8 stack trace API

Babel-plugin-source-map-support prepends this statement to each file, giving us stack traces with the source-map-support package:
import 'source-map-support/register';

Setting up Babel

Create a .babelrc file in the root of the project to handle our Babel configuration:

.babelrc

{
  "plugins": ["source-map-support"],
  "presets": [
    [
      "@babel/preset-env",
      {
        "targets": {
          "node": true
        }
      }
    ]
  ]
}

Adding Knex

Next, we’ll add Knex and MySQL as the driver of choice for this purpose:

npm install --save mysql2 knex

Setting up Knex

Create a knexfile.js in the project root:

import dotenv from "dotenv";
dotenv.config({ silent: true });

module.exports = {
  development: {
    client: "mysql2",
    connection: {
      host: process.env.DB_HOST,
      user: process.env.DB_USER,
      password: process.env.DB_PW,
      database: process.env.DB_DB
    }
    // migrations: {
    // directory: './database/migrations',
    // },
    // seeds: { directory: './database/seeds' }
  },
  staging: {
    client: "mysql2",
    connection: {
      host: process.env.DB_HOST,
      user: process.env.DB_USER,
      password: process.env.DB_PW,
      database: process.env.DB_DB
    }
  },
  production: {
    client: "mysql2",
    connection: {
      host: process.env.DB_HOST,
      user: process.env.DB_USER,
      password: process.env.DB_PW,
      database: process.env.DB_DB
    }
  }
};

Create a folder called queries in your project root, this will be where the data retrieval functions will go:

mkdir queries

Add a knex file: knex.js

const knex = require("knex");

const knexfile = require("../knexfile");

const env = process.env.NODE_ENV || "development";

const configOptions = knexfile[env];

module.exports = knex(configOptions);

Example query file - games.js:

const knex = require("./knex");

export async function getAll() {
  const res = await knex("matches").select("*");
  return res;
}

Setting up Webpack

In the root of the project create a webpack.config.js file and configure Webpack to use Babel to bundle up our Serverless functions. We’ll also exclude node development dependencies using the node externals package.

npm install webpack-node-externals --save-dev

webpack.config.js:

const slsw = require("serverless-webpack");
const nodeExternals = require("webpack-node-externals");

module.exports = {
  entry: slsw.lib.entries,
  devtool: "source-map",
  // Since 'aws-sdk' is not compatible with webpack,
  // we exclude all node dependencies
  externalsPresets: { node: true },
  externals: [nodeExternals()],
  mode: slsw.lib.webpack.isLocal ? "development" : "production",
  optimization: {
    minimize: false
  },
  performance: {
    // Turn off size warnings for entry points
    hints: false
  },
  // Run babel on all .js files - skip those in node_modules
  module: {
    rules: [
      {
        test: /\.js$/,
        loader: "babel-loader",
        include: __dirname,
        exclude: /node_modules/
      }
    ]
  },
  plugins: []
};

Setting up Serverless

Add our plugins to the serverless.yaml file:

- serverless-webpack
- serverless-offline

Add serverless-webpack configuration to serverless.yaml

custom:
  webpack:
    webpackConfig: ./webpack.config.js
    includeModules: true # enable auto-packing of external modules

We’ll add an http endpoint to the default hello handler, so that we can test our api endpoint out:

events:
  - http:
    path: hello
    method: get
    cors: true

Full Serverless.yaml

service: serverless-webpack-babel-knex-starter
frameworkVersion: "2"

provider:
name: aws
runtime: nodejs12.x
apiGateway:
  shouldStartNameWithService: true

plugins:

- serverless-webpack
- serverless-offline

functions:
  hello:
    handler: handler.hello
      events:
        - http:
          path: hello
          method: get
          cors: true

custom:
  webpack:
  webpackConfig: ./webpack.config.js
  includeModules: true # enable auto-packing of external modules

Running and Knex Issues

Let’s test it out! Add a start npm script to package.json

"start": "serverless offline start --stage dev --noAuth"

Call our API

curl --location --request GET 'http://localhost:3000/dev/hello'

Knex Runtime Issues:

  • ES Modules may not assign module.exports or exports.*, Use ESM export syntax, instead: ./knexfile.js

It doesn’t like that we’re using module.exports in our knexfile, one potential solution would be to use es6 default export syntax export default {}

This ended up causing more problems then it solved dealing with the internal knex library which doesn’t play well with ES modules.

The solution I went for is to use a Babel plugin to transform ESM to CommonJS Modules which is the standard for Node modules. Client-side JavaScript that runs in the browser use another standard, called ES Modules or ESM. In CommonJS, we export with module.exports and import with require statements. Since we’re using Babel we can use import/export and our code will be transformed into CommonJS modules.

npm install --save-dev @babel/plugin-transform-modules-commonjs

Add to our plugins section in .babelrc

{
  "plugins": ["source-map-support", "@babel/plugin-transform-modules-commonjs"],
  "presets": [
    [
      "@babel/preset-env",
      {
        "targets": {
          "node": true
        }
      }
    ]
  ]
}

Using CommonJS should be enough to get you going but you might run into the next issue:

  • Can’t resolve runtime dependencies
Module not found: Error: Can't resolve 'oracledb'
Module not found: Error: Can't resolve 'pg-native'
Module not found: Error: Can't resolve 'pg-query-stream'
Module not found: Error: Can't resolve 'sqlite3'

If you receive module not found errors for packages that you are not using, then we can fix this by ignoring those drivers/packages. There are different ways this can be approached with Webpack and with Serverless but the solution that I landed on was to use the NormalModuleReplacementPlugin which is bundled with Webpack. This plugin allows you to replace resources that match a regular expression with another resource. We’ll add the noop2 package to replace the drivers we’re not using with a “no operation module”.

npm install --save-dev noop2

const { NormalModuleReplacementPlugin } = require("webpack");

plugins: [
  // Ignore knex runtime drivers that we don't use
  new NormalModuleReplacementPlugin(
    /mssql?|oracle(db)?|sqlite3|pg-(native|query)/,
    "noop2"
  )
];

Adding Eslint and Prettier

To finish this starter template, we’ll add some niceness to the project with eslint and prettier.

npm install --save-dev @babel/eslint-parser eslint eslint-config-prettier eslint-plugin-lodash eslint-plugin-prettier prettier

prettierrc.json

{
  "trailingComma": "none",
  "tabWidth": 2,
  "semi": false,
  "singleQuote": true,
  "printWidth": 120
}

.eslintrc.js

module.exports = {
  env: {
    node: true
  },
  plugins: ["prettier"],
  parser: "@babel/eslint-parser",
  parserOptions: {
    sourceType: "module",
    ecmaFeatures: {
      classes: true,
      experimentalObjectRestSpread: true
    }
  },
  extends: [
    "eslint:recommended",
    "plugin:prettier/recommended",
    "plugin:lodash/recommended"
  ],
  rules: {
    "prettier/prettier": "error"
  }
};

Starter Project

Now we have a nice starter project to get us off the ground with Serverless, Webpack, Babel, and Knex.

To grab all this goodness or if you have improvements, check out the Github repository