Verifying self-signed JSON Web Tokens (JWTs) with AWS HTTP APIs
Hosting your own serverless OIDC Conformant "server" to verify self-signed JSON Web Tokens with HTTP APIs is actually quite simple. This post will show you how.
"Trust no one." Or at least that's what Fox Mulder told me back in the 90s.
With the recent GA of HTTP APIs for API Gateway, I decided to start evaluating my existing API Gateway REST APIs to see if I could migrate them over to take advantage of the decreased latency and reduced cost of the new HTTP APIs. Several of them were disqualified because they utilize service integrations (a feature that AWS is working to add), but for one of my largest applications, the lack of Custom Authorizers is what brought me to a dead end. Or so I initially thought. 😉
After a bit of research (okay, it was actually several hours because I decided to read through a bunch of specs and blog posts and then run a ton of experiments), it turns out that hosting your own OIDC Conformant "server" to verify self-signed JSON Web Tokens with HTTP APIs is actually quite simple. So as long as you can use JWT for your bearer tokens, you can utilize your existing authentication service (and probably dramatically reduce your latency and cost).
In this post, I'll show you everything you need to know to set this up yourself. We'll generate certificates, create our OIDC discovery service, set up our HTTP API authorizers, generate and sign our JWTs, and protect routes with scopes.
Why a custom authentication system?
That's a really, really good question. In a perfect world, using something like Cognito, Auth0, or Firebase to handle my app's authentication would be my first choice. These are well-established, trusted solutions that provide drop-in authentication capabilities. They also all support JWT, making them a perfect fit for HTTP APIs for API Gateway.
Unfortunately, we don't live in a perfect world. Sometimes we have legacy authentication systems that we need to use to control access, or additional regulations that need to be followed, or maybe very specific login or password reset flows that aren't handled correctly by these providers. And if you have a lot of users, these services can get very expensive, very quickly.
So whatever your reason, it is possible that an off-the-shelf solution might not be right for you. If you find yourself in this situation, then keep reading.
Why JWT?
Well, first of all, that's the only choice right now with HTTP API authorizers, so there's that. Also, HTTP API authorizers utilize OpenID Connect discovery, which makes setup super easy (more on this in a bit). But the biggest reason is because JWTs allow for stateless authorization, making them a really good fit for serverless applications. I know, I know. You've read all the articles and posts about how JWTs are a terrible idea for session tokens because you can't invalidate them without some sort of coordinated black list which defeats the purpose of being stateless in the first place. I (and many others) say, that's a bunch of hogwash.
Sure, having the ability to invalidate tokens can be super useful, but in most cases, a JWT with some reasonable expiration will be just fine. If you're familiar with OAuth2 (and if you're working with a custom authentication system, then I really hope you are), long-lived refresh tokens can be used to generate new JWTs when old ones expire. So if you wanted to really limit the usage of a JWT, set a very short expiration, and refresh them. Your refresh endpoint can be used to invalidate any compromised tokens.
The data in JWTs can be encrypted, but most of the time this seems like overkill. You obviously don't want to store unencrypted credit card info in a JWT, so a few claims (like name, email, etc.) and some scopes, are likely fine to leave unencrypted. Also, if your communicating your tokens over an SSL connection (which I hope you are), and your not saving them in a browser's local storage (which I hope you're not), then the likelihood of them getting stolen is very low.
And back to this stateless thing for a second. With API Gateway REST API Custom Authorizers, a Lambda function would need to be triggered to generate a policy document that would get cached (for a short while) and passed into your Lambda. In my case, I was just generating the same policy document over and over again, each time needing to lookup the token in DynamoDB to verify it. That's a lot of extra overhead to get the same information that can easily (and confidently) be passed and verified in a JSON Web Token. This makes endpoint authorization super fast, and saves you all those Lambda invocations and database round trips.
Okay, that's enough of a preamble. Let's get building!
Generating an RSA key pair
The first thing we need to is generate our RSA key pair so that we can sign our JWTs and so that the HTTP API authorizers can verify the signatures. We can do this by running the following commands:
defaultopenssl genrsa -out private.key 4096 openssl rsa -in private.key -pubout -out public.key
This will give you a PRIVATE key like this:
default-----BEGIN RSA PRIVATE KEY----- MIIJKQIBAAKCAgEA3E7BTT9I18Yo1UX/UJrK4DXT6EHbFHvEgPsEHywn/D1zGxFZ 3XVpoeRYgLIZLFlCjt6etnbrkdaKJbM2FhWT8XexYHAXUafYxuUOQFRCHuYCMP9y ... (truncated for security) ... YtySHxcptFht1xUW7W5KVvWNS7W+mYi2m6qbeSnLK8O5t8FYymDuFJBFkxp19EN1 SN+IQF2xb0vShXazMsMKhEYbzoGxco9Kwh3qcJPHd29Er7sa5AQAHlXJPSTh -----END RSA PRIVATE KEY-----
And a PUBLIC key like this:
default-----BEGIN PUBLIC KEY----- MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA3E7BTT9I18Yo1UX/UJrK 4DXT6EHbFHvEgPsEHywn/D1zGxFZ3XVpoeRYgLIZLFlCjt6etnbrkdaKJbM2FhWT 8XexYHAXUafYxuUOQFRCHuYCMP9yVyCcteNalStmcJ7Tm3KxTOgZh3Jslf7Myy3v zND6i71vdaoslfjwFmqsblmq+xbOL7mv1/6yrGO1PPaGxumQM+7uafTgiUSue0Hw yuJtf0rdb7dj5DUmekKVUIp0sfeTmXCcX/KLmstAyzroVveUcNEdMrKge53OfLT4 alWS8JLDUAJBCYzBXsswGOErR22GDWQxccochlqUh9Sn+kdyMVpfLYJP6aiAU0ag 86E9gvBXwKSF/qCahSs6LOcrGWiHdjlv2CdVU7W5XtXjUEiuegJkai/qGyM+ndJl A3mtYl8eMxaQhABj8xNGXgu12KUlqPhmIySQeOPzGwnVCWDpB/LVPGNM1KPX4Che r0P+6PCdIWmrFE6q6CsQHwrW/UzXmXlEm0Rzelnpoo80sB5kpVPUMT+vkYrNhwxy dobcRHKKvz5giezbLhnoH/s/Tb5jVupF6xFBIutwfOKO5z8fEXix58DxduYHdSXM vnB5iuS6QhQVNt6DZ+QIXk1bAt6B8/6/ZEeOshFsUfWi29pEhPE8fTwLCvxhXW/x 2erJTivf4xTcwBbnpdeQr+sCAwEAAQ== -----END PUBLIC KEY-----
Keep these keys handy, we're going to need them in a few minutes.
Creating your OIDC Conformant "Server"
As I said earlier, HTTP APIs for API Gateway utilize OpenID Connect discovery documents to locate your PUBLIC key. An HTTP API authorizer will use your PUBLIC key to verify the signature of incoming JSON Web Tokens, and then pass the claims to your Lambda function. This is a relatively straightforward process, and only requires two STATIC files in order to work correctly. This means our OpenID Connect "server" can be any publicly accessible endpoint, like an S3 bucket.
The first file we need is our OpenID Connect discovery document. The OpenID spec describes lots of required fields, and you're welcome to study this and follow it to the letter. However, HTTP API authorizers only verify the signature of our JWT, so all of that other stuff (at least in our case) is unnecessary. In fact, we only need two fields: issuer and jwks_uri.
The spec says we need to create a JSON document available at /.well-known/openid-configuration. Using the two fields we need for our authorizer, it should look like this:
default{ "issuer":"https://[my-domain-name]/", "jwks_uri":"https://[my-domain-name]/.well-known/jwks.json" }
The issuer must match the domain where you are hosting these files. So if you were to put them in an S3 bucket named "my-jwt-test", then the issuer should be https://my-jwt-test.s3.amazonaws.com/ or your custom domain that is pointed to it.
You'll also notice that this specifies the jwks_uri (that's our JSON Web Key Set). This could be anything, but standard convention dictates that we name it jwks.json and put it in the .well-known directory. This file will contain our PUBLIC key and Key ID that will be used by the authorizer.
Before we move on, we should quickly cover JSON Web Key Sets. Of course, there is another spec that you can read (or you can get a quick overview here), but for our purposes, we just need to know a few basics. There are two ways to add our PUBLIC key. We can either add the certificate (in its ASCII PEM format sans whitespace characters) in the X.509 Certificate Chain parameter (x5c), or provide the modulus and the exponent for the RSA public key in the n and e parameters respectively. You could also provide both, but HTTP API authorizers will use whichever one is available. I'll show you how to get these in a minute.
The only other required parameter necessary for HTTP API authorizers to work is the Key ID (kid). This parameter is used to identify which key a JWT should use to verify its signature. It is possible to specify multiple certificates, so you can utilize this by passing it in the header of your JWTs. The value of your Key ID is actually completely arbitrary, meaning it can technically be whatever you want it to be, it just needs to be unique within your key set.
The HTTP API authorizers only requires these two parameters, however, adding some additional meta data (so you remember what this key is) isn't a bad idea. I would add at least the Algorithm (alg), Key Type (kty), and Public Key Use (use) parameters as well. Okay, let's create our jwks.json file.
Adding our Public Key - Option #1:
The first public key option (x5c) is super easy. Just open your public.key file we created earlier, and copy everything in between the -----BEGIN PUBLIC KEY----- and -----END PUBLIC KEY-----. Then, replace any whitespace and newlines so we end up with one big long string. The x5c parameter is an array, so we'll add this key string as the first element (see below).
Adding our Public Key - Option #2:
The second option is a bit tricky if you're not a cryptography junkie. You need to extract the modulus and exponent from your public key and base64 encode them. You can do this in the programming language of your choice OR you can use this site to do it for you. If you really want to go down the rabbit hole on this, this post will get you started in Node.
Just show me the darn file already!
So below is what our final jwks.json file should look like (using option 1 for our key). We're just using 'my-key-id' for our Key ID, but as I said, this can be whatever you want.
default{ "keys": [ { "alg": "RS256", "kty": "RSA", "use": "sig", "kid": "my-key-id", "x5c": [ "MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA3E7BTT9I18Yo1UX/UJrK4DXT6EHbFHvEgPsEHywn/D1zGxFZ3XVpoeRYgLIZLFlCjt6etnbrkdaKJbM2FhWT8XexYHAXUafYxuUOQFRCHuYCMP9yVyCcteNalStmcJ7Tm3KxTOgZh3Jslf7Myy3vzND6i71vdaoslfjwFmqsblmq+xbOL7mv1/6yrGO1PPaGxumQM+7uafTgiUSue0HwyuJtf0rdb7dj5DUmekKVUIp0sfeTmXCcX/KLmstAyzroVveUcNEdMrKge53OfLT4alWS8JLDUAJBCYzBXsswGOErR22GDWQxccochlqUh9Sn+kdyMVpfLYJP6aiAU0ag86E9gvBXwKSF/qCahSs6LOcrGWiHdjlv2CdVU7W5XtXjUEiuegJkai/qGyM+ndJlA3mtYl8eMxaQhABj8xNGXgu12KUlqPhmIySQeOPzGwnVCWDpB/LVPGNM1KPX4Cher0P+6PCdIWmrFE6q6CsQHwrW/UzXmXlEm0Rzelnpoo80sB5kpVPUMT+vkYrNhwxydobcRHKKvz5giezbLhnoH/s/Tb5jVupF6xFBIutwfOKO5z8fEXix58DxduYHdSXMvnB5iuS6QhQVNt6DZ+QIXk1bAt6B8/6/ZEeOshFsUfWi29pEhPE8fTwLCvxhXW/x2erJTivf4xTcwBbnpdeQr+sCAwEAAQ=="] } ] }
Now we just upload these to our S3 bucket (or static hosting service of choice) and we're ready to create our authorizers. If you are hosting these on S3, just make sure you set the Content-Type headers to application/json for both files.
Setting up our HTTP API Authorizers
Once you've created your HTTP API and added a route, click on the "Attach authorizer" and fill in the info below.
Be sure to change the Issuer URL to the domain you set up above, and make sure you include the trailing slash. Then hit "Create and attach" and (assuming everything above was done correctly), it should attach the authorizer.
Signing our JSON Web Tokens
If you want to run some quick tests, you can encode your JWTs using the tools available at jwt.io. If you'd like to do this programmatically, there are plenty of great libraries available that are also listed on the jwt.io site. For our example, we'll use Node.js and the jsonwebtoken library by Auth0 (GitHub repo).
First we'll install the dependency:
defaultnpm install jsonwebtoken
Then we'll set up a quick script to generate and sign our tokens:
javascript// Require token signer and file system const jwt = require('jsonwebtoken') const fs = require('fs') // Load my private key const privateKey = fs.readFileSync('./private.key') // Create and sign the token const token = jwt.sign({ foo: 'bar', // this our payload }, privateKey, { algorithm: 'RS256', expiresIn: '2H', audience: 'my-audience', // same as in our HTTP API authorizer issuer: 'https://[my-domain-name]/', keyid: 'my-key-id' // same as in our jwks.json } ) // return the token console.log(token)
This should return a beautiful JWT like this:
defaulteyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Im15LWtleS1pZCJ9.eyJmb28iOiJiYXIiLCJpYXQiOjE1ODUyNDY4OTksImV4cCI6MTU4NTI1NDA5OSwiYXVkIjoibXktYXVkaWVuY2UiLCJpc3MiOiJodHRwczovL1tteS1kb21haW4tbmFtZV0vIn0.SVuBvH2OhBrMkZ4sukgZzQnrJqCB1P_pCY916sJQ_vhZUvepby5SU2HCVEqjjSbiUl1rXLYhi_y_dv_SmegS3PupfOpNNcYMEswj0cgaJ_VQTUcUBJPYP5m10xCuCvsthdgO6djiP4ZlkhqyncLfYo7vvYxeNRBY_TEE4n1jjtrO9veipHi1nXDqVvnmT4C11xhVbkMNORGU02wXFC-OMP_6BAOREns3hJ0FKsLrwPxeYica_KJr5P5fhIfWyHtn8YKiEiGSHlDHEmmkJ0bSnXGhdj_Cx6G_y2-LoM6xNiljjSR5hEwi4rjwWtyuepye4DpJsJLB1TzqIEdIHJ3Qwjhbk8CLZmFPtUWbr9sgrkbEh2o3tPzHc76I-0bvMhPoPwdbbedEiQUEKUGd5v4HbOo_6U57HRhX5_fmMbAk97Z9k0qD9V164FVFS7F86hRWGuPpERa9v_13hWX225IML35O62Go41dikQKrw-MDlBYR5DMchZfNo-exVzEG3lUFbTm6vj0SGltPYaTldyXgyGhAwQHDRxsJriaNjbzeKU5MXKRQrZERgBiL8fnqV6lvapY3LL2vrR18nc9-kgZfwxL4BhgP2Y5Z8FMigVoM3HiJVdt8YxSVnyWtuwNdSYMNf7vRCSWHevpIN2dDZOQynUZYzV8SrOR5HxKsooDnlmU
Now we can call our HTTP API endpoint and send this token in as the Authorization header (don't forget to include bearer in there). You should get a Status 200 OK. If not, check the www-authenticate response header for more information on why it failed.
Cool, now what?
Assuming this is working for you, now you can do all kinds of cool things. First of all, the payload of your JWT is now available within the requestContext.authorizer of your event JSON passed to your Lambda function. It'll look something like this:
javascript{ "requestContext": { "authorizer": { "jwt": { "claims": { "aud": "my-audience", "exp": "1585252914", "foo": "bar", "iat": "1585245714", "iss": "https://[my-domain-name]/" }, "scopes": null } } } }
Notice our foo parameter was passed in the claims. We can add all kinds of claims to our payload that can be used by our Lambda functions. There are some reserved words, but other than the top seven, you can likely use whatever you want.
Another really cool feature is HTTP API authorizer scopes. You can limit access to particular endpoints by adding an Authorization scope. In the console, you can go to the "Authorization" tab, select the route, and then add scopes using the provided form.
This endpoint will now require the "admin" scope for access. Adding a scope to our JWT is as simple as adding a space delimited list of scopes to our payload. For example:
default// Create and sign the token const token = jwt.sign({ foo: 'bar', scope: 'admin user.id user.email' // these are our scopes }, privateKey, { algorithm: 'RS256', expiresIn: '2H', audience: 'my-audience', // same as in our HTTP API authorizer issuer: 'https://[my-domain-name]/', keyid: 'my-key-id' // same as in our jwks.json } )
If a token has a valid scope for the endpoint, access will be authorized and the "scopes" will be available in the requestContext.authorizer.
default{ "requestContext": { "authorizer": { "jwt": { "claims": { "aud": "my-audience", "exp": "1585252914", "foo": "bar", "iat": "1585245714", "iss": "https://[my-domain-name]/" }, "scopes": [ "admin", "user.id", "user.email" ] } } } }
Should I actually do this?
And here in lies the ultimate question. I did this as an experiment to see if I could make it work. I'm not using it in production (yet), but I think you should always think long and hard about the value that building a solution like this adds for your customers. If you can use Cognito or Auth0, then I say do that. If you have a need to do this yourself (and you're willing to study up on cryptography and security), then go for it. After all, you're still using the JWT standard to verify signatures, so as long as you keep your private keys safe, the solution is solid.
I'm sure that AWS will launch additional authorization methods in the future, but for now, this will allow me to use HTTP APIs with my legacy authentication components. I can also see this being extremely useful for signing tokens for internal communications too. I'm sure there are plenty of other use cases as well.
Hopefully you find this useful. Please feel free to let me know why this is such a bad idea in the comments. 😉