Lambda API: v0.8 Released

Lambda API v0.8 is finally here, but I think it was well worth the wait! New features include allowing middleware to accept multiple handlers, new convenience methods for cache control and signing S3 URLs, and async/await support for the main function handler. And best of all, new LOGGING and SAMPLING support for you to add more observability into your APIs and web applications.

This version is a huge step forward. There are a few more features to add, but we are very close to releasing this as a stable version 1.0. Your feedback is greatly appreciated, so please submit feature requests and issues. If you want to contribute, even better! Pull requests are always welcome.

NPM: https://www.npmjs.com/package/lambda-api

GitHub: https://github.com/jeremydaly/lambda-api

Specifying multiple middleware

In addition to restricting middleware to certain paths (added in v0.7), you can now add multiple middleware using a single use() method. This is a convenient way to assign several pieces of middleware to the same path or to minimize your code.

javascript
const middleware1 = (req,res,next) => {
  // middleware code
}

const middleware2 = (req,res,next) => {
  // some other middleware code
}

// Restrict middleware1 and middleware2 to /users route
api.use('/users', middleware1, middleware2)

// Add middleware1 and middleware2 to all routes
api.use(middleware1, middleware2)

Generate signed S3 URLs

Serving up files from an S3 bucket is incredibly simple with Lambda API. However, transferring large files through your Lambda function and API Gateway adds unnecessary latency and cost. v0.8 introduces a new getLink() method that returns a signed URL to the referenced file in S3 (using the s3://{my-bucket}/{path-to-file} format). You can optionally pass in an integer as the second parameter that will changed the default expiration time of the link. The expiration time is in seconds and defaults to 900. In order to ensure proper URL signing, the getLink() call must be asynchronous, and therefore returns a promise. You must either await the result or use a .then() to retrieve the value.

There is an optional third parameter that takes an error handler callback. If the underlying getSignedUrl() call fails, the error will be returned using the standard res.error() method. You can override this by providing your own callback.

javascript
// async/await
api.get('/getLink', async (req,res) => {
  let url = await res.getLink('s3://my-bucket/my-file.pdf')
  return { link: url }
})

// promises
api.get('/getLink', (req,res) => {
  res.getLink('s3://my-bucket/my-file.pdf').then(url => {
    res.json({ link: url })
  })
})

Want even more convenience? The redirect() method now accepts S3 file references and will automatically generate a signed URL and then redirect the user's browser.

javascript
// This will redirect a signed URL using the getLink method
api.get('/redirectToS3File', (req,res) => {
  res.redirect('s3://my-bucket/someFile.pdf')
})

`async/await` support for the main handler

Lambda API added async/await support for route handlers and middleware in v0.6, but now you can use it for your main function handler as well.

javascript
// Require the framework and instantiate it
const api = require('lambda-api')()

// Define a route
api.get('/status', async (req,res) => {
  return { status: 'ok' }
})

// Declare your Lambda handler
exports.handler = async (event, context) => {
  // Run the request
  return await api.run(event, context)
}

This is a more modern approach using the Node.js v8.10 runtime. The old way still works, just pass the callback in as the third parameter to the api.run() function.

default
// Declare your Lambda handler with a callback
exports.handler = (event, context, callback) => {
  // Run the request
  api.run(event, context, callback)
}

More Cache Control

The new cache() convenience method adds a cache-control header to responses. If the first parameter is an integer, it will add a max-age to the header. The number should be in milliseconds. If the first parameter is true, it will add the cache headers with max-age set to 0 and use the current time for the expires header. If set to false, it will add a cache header with no-cache, no-store, must-revalidate as the value. You can also provide a custom string that will manually set the value of the cache-controlheader. An optional second argument takes a boolean and will set the cache-control to private. This method is chainable.

javascript
res.cache(false).send() // 'cache-control': 'no-cache, no-store, must-revalidate'
res.cache(1000).send() // 'cache-control': 'max-age=1'
res.cache(30000,true).send() // 'cache-control': 'private, max-age=30'

Also, a new modified() method adds a last-modified header to responses. A value of true will set the value to the current date and time. A JavaScript Date object can also be passed in. Note that it will be converted to UTC if not already. A string can also be passed in and will be converted to a date if JavaScript's Date() function is able to parse it. A value of false will prevent the header from being generated, but will not remove any existing last-modified headers.

These two methods allow you to tell the browser to reuse locally cached versions of your Lambda API resources. This can save a tremendous amount of executions and API calls, especially for things like serving up HTML pages or infrequently updated JSON responses from API routes.

Logging 🙌

Lambda API now includes a robust logging engine specifically designed to utilize native JSON support for CloudWatch Logs. Not only is it ridiculously fast, but it's also highly configurable. Logging is disabled by default, but can be enabled by passing { logger: true } when you create the Lambda API instance (or by passing a Logging Configuration definition object, see below).

The logger is attached to the REQUEST object and can be used anywhere the object is available (e.g. routes, middleware, and error handlers).

javascript
const api = require('lambda-api')({ logger: true })

api.get('/status', (req,res) => {
  req.log.info('Some info about this route')
  res.send({ status: 'ok' })
})

In addition to manual logging, Lambda API can generate "access" logs for your API requests. API Gateway can also provide access logs, but they are limited to contextual information about your request (see here). Lambda API allows you to capture the same data PLUS additional information directly from within your application.

Logging Configuration

Logging can be enabled by setting the logger option to true when creating the Lambda API instance. Logging can be configured by setting logger to an object that contains configuration information. The following table contains available logging configuration properties.

Property	Type	Description	Default
access	`boolean` or `string`	Enables/disables automatic access log generation for each request. See Access Logs.	`false`
customKey	`string`	Sets the JSON property name for custom data passed to logs.	`custom`
detail	`boolean`	Enables/disables adding `REQUEST` and `RESPONSE` data to all log entries.	`false`
level	`string`	Minimum logging level to send logs for. See Logging Levels.	`info`
levels	`object`	Key/value pairs of custom log levels and their priority. See Custom Logging Levels.
messageKey	`string`	Sets the JSON property name of the log "message".	`msg`
nested	`boolean`	Enables/disables nesting of JSON logs for serializer data. See Serializers.	`false`
timestamp	`boolean` or `function`	By default, timestamps will return the epoch time in milliseconds. A value of `false` disables log timestamps. A function that returns a value can be used to override the default format.	`true`
sampling	`object`	Enables log sampling for periodic request tracing. See Sampling.
serializers	`object`	Adds serializers that manipulate the log format. See Serializers.
stack	`boolean`	Enables/disables the inclusion of stack traces in caught errors.	`false`

Example:

javascript
const api = require('lambda-api')({
  logger: {
    level: 'debug',
    access: true,
    customKey: 'detail',
    messageKey: 'message',
    timestamp: () => new Date().toUTCString(), // custom timestamp
    stack: true
  }
})

Log Format

Logs are generated using Lambda API's standard JSON format. The log format can be customized using Serializers.

Standard log format (manual logging):

javascript
{
  "level": "info", // log level
  "time": 1534724904910, // request timestamp
  "id": "41b45ea3-70b5-11e6-b7bd-69b5aaebc7d9", // awsRequestId
  "route": "/user/:userId", // route accessed
  "method": "GET", // request method
  "msg": "Some info about this route", // log message
  "timer": 2, // execution time up until log was generated
  "custom": "additional data", // addditional custom log detail
  "remaining": 2000, // remaining milliseconds until function timeout
  "function": "my-function-v1", // function name
  "memory": 2048, // allocated function memory
  "sample": true // is generated during sampling request?
 }

Access Logs

Access logs generate detailed information about the API request. Access logs are disabled by default, but can be enabled by setting the access property to true in the logging configuration object. If set to false, access logs will only be generated when other log entries (info, error, etc.) are created. If set to the string 'never', access logs will never be generated.

Access logs use the same format as the standard logs above, but include additional information about the request. The access log format can be customized using Serializers (see below).

Access log format (automatic logging):

javascript
{
  ... Standard Log Data ...,
  "path": "/user/123", // path accessed
  "ip": "12.34.56.78", // client ip address
  "ua": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6)...", // User-Agent
  "version": "v1", // specified API version
  "device": "mobile", // client device (as determined by CloudFront)
  "country": "US", // client country (as determined by CloudFront)
  "qs": { // query string parameters
    "foo": "bar"
  }
}

Logging Levels

Logging "levels" allow you to add detailed logging to your functions based on severity. There are six standard log levels as specified in the table below along with their default priority.

Level	Priority
`trace`	10
`debug`	20
`info`	30
`warn`	40
`error`	50
`fatal`	60

Logs are written to CloudWatch Logs ONLY if they are the same or higher severity than specified in the level log configuration.

javascript
// Logging level set to "warn"
const api = require('lambda-api')({ logger: { level: 'warn' } })

api.get('/', (req,res) => {
  req.log.trace('trace log message') // ignored
  req.log.debug('debug log message') // ignored
  req.log.info('info log message') // ignored
  req.log.warn('warn log message') // write to CloudWatch
  req.log.error('error log message') // write to CloudWatch
  req.log.fatal('fatal log message') // write to CloudWatch
  res.send({ hello: 'world' })
})

Custom Logging Levels

Custom logging "levels" can be added by specifying an object containing "level names" as keys and their priorities as values. You can also adjust the priority of standard levels by adding it to the object.

javascript
const api = require('lambda-api')({
  logger: {
    levels: {
      'test': 5, // low priority 'test' level
      'customLevel': 35, // between info and warn
      'trace': 70 // set trace to the highest priority
    }
  }
})

In the example above, the test level would only generate logs if the priority was set to test. customLevel would generate logs if level was set to anything with the same or lower priority (e.g. info). trace now has the highest priority and would generate a log entry no matter what the level was set to.

Adding Additional Detail

Manual logging also allows you to specify additional detail with each log entry. This is especially handy if you need to include structured data as part of your log entry. Details can be added by suppling any variable type as a second parameter to the logger function.

javascript
req.log.info('This is the main log message','some other detail') // string
req.log.info('This is the main log message',{ foo: 'bar', isAuthorized: someVar }) // object
req.log.info('This is the main log message',25) // number
req.log.info('This is the main log message',['val1','val2','val3']) // array
req.log.info('This is the main log message',true) // boolean

If an object is provided, the keys will be merged into the main log entry's JSON. If any other type is provided, the value will be assigned to a key using the customKey setting as its property name. If nested is set to true, objects will be nested under the value of customKey as well.

Serializers

Serializers allow you to customize log formats as well as add additional data from your application. Serializers can be defined by adding a serializers property to the logger configuration object. A property named for an available serializer (main, req, res, context or custom) needs to return an anonymous function that takes one argument and returns an object. The returned object will be merged into the main JSON log entry. Existing properties can be removed by returning undefined as their values.

javascript
const api = require('lambda-api')({
  logger: {
    serializers: {
      req: (req) => {
        return {
          apiId: req.requestContext.apiId, // add the apiId
          stage: req.requestContext.stage, // add the stage
          qs: undefined // remove the query string
        }
      }
    }
  }
})

Serializers are passed one argument that contains their corresponding object. req and main receive the REQUEST object, res receives the RESPONSE object, context receives the context object passed into the main run() function, and custom receives custom data passed in to the logging methods. Note that only custom objects will trigger the custom serializer.

If the nested option is set to true in the logger configuration, then JSON log entries will be generated with properties for req, res, context and custom with their serialized data as nested objects.

Log Sampling 🤘🏻

Log sampling allows you to periodically generate log entries for all possible severities within a single request execution. All of the log entries will be written to CloudWatch Logs and can be used to trace an entire request. This can be used for debugging, metric samples, resource response time sampling, etc.

Sampling can be enabled by adding a sampling property to the logger configuration object. A value of true will enable the default sampling rule. The default can be changed by passing in a configuration object with the following optional properties:

Property	Type	Description	Default
target	`number`	The minimum number of samples per `period`.	1
rate	`number`	The percentage of samples to be taken during the `period`.	0.1
period	`number`	Number of seconds representing the duration of each sampling period.	60

The example below would sample at least 2 requests every 30 seconds as well as an additional 0.1 (10%) of all other requests during that period. Lambda API tracks the velocity of requests and attempts to distribute the samples as evenly as possible across the specified period.

javascript
const api = require('lambda-api')({
  logger: {
    sampling: {
      target: 2,
      rate: 0.1,
      period: 30
    }
  }
})

Additional rules can be added by specifying a rules parameter in the sampling configuration object. The rules should contain an array of "rule" objects with the following properties:

Property	Type	Description	Default	Required
route	`string`	The route (as defined in a route handler) to apply this rule to.		Yes
target	`number`	The minimum number of samples per `period`.	1	No
rate	`number`	The percentage of samples to be taken during the `period`.	0.1	No
period	`number`	Number of seconds representing the duration of each sampling period.	60	No
method	`string` or `array`	A comma separated list or `array` of HTTP methods to apply this rule to.		No

The route property is the only value required and must match a route's path definition (e.g. /user/:userId, not /user/123) to be activated. Routes can also use wildcards at the end of the route to match multiple routes (e.g. /user/* would match /user/:userId AND /user/:userId/tags). A list of methods can also be supplied that would limit the rule to just those HTTP methods. A comma separated string or an array will be properly parsed.

Sampling rules can be used to disable sampling on certain routes by setting the target and rate to 0. For example, if you had a /status route that you didn't want to be sampled, you would use the following configuration:

javascript
const api = require('lambda-api')({
  logger: {
    sampling: {
      rules: [
        { route: '/status', target: 0, rate: 0 }
      ]
    }
  }
})

You could also use sampling rules to enable sampling on certain routes:

javascript
const api = require('lambda-api')({
  logger: {
    sampling: {
      rules: [
        { route: '/user', target: 1, rate: 0.1 }, // enable for /user route
        { route: '/posts/*', target: 1, rate: 0.1 } // enable for all routes that start with /posts
      ],
      target: 0, // disable sampling default target
      rate: 0 // disable sampling default rate
    }
  }
})

If you'd like to disable sampling for GET and POST requests to user:

javascript
const api = require('lambda-api')({
  logger: {
    sampling: {
      rules: [
        // disable GET and POST on /user route
        { route: '/user', target: 0, rate: 0, method: ['GET','POST'] }
      ]
    }
  }
})

Any combination of rules can be provided to customize sampling behavior. Note that each rule tracks requests and velocity separately, which could limit the number of samples for infrequently accessed routes.

Full Release Notes: https://github.com/jeremydaly/lambda-api/releases/tag/v0.8.0

NPM: https://www.npmjs.com/package/lambda-api

GitHub: https://github.com/jeremydaly/lambda-api

Tags: #aws, #amazon-web-services, #lambda-api, #serverless, #api-gateway, #aws-lambda, #web-framework