Webhooks: An essential guide

Published on: Wed Aug 17 2022

Series

Goals

By the end of this guide, you will gain a better understanding of:

✅ Webhooks — what they are and when to use them

✅ The common security attack vectors

✅ The techniques used to mitigate security issues in practice

✅ HS256 — the most common authentication method for webhooks

Content

Introduction

Most modern applications we build today require integration with at least one of more 3rd party service(s).

These may include services such as:

  • Stripe (Payments)
  • SendGrid (Email)
  • Auth0 (Accounts)
  • HubSpot or Salesforce - Customer relation management (CRM)
  • So forth…
Webhook microservice diagram

Because of this, our applications is actually designed in a very modular way — or in other words, using microservices.

Now comes the question of how do we coordinate between these services. Even more importantly, how do we keep the data in sync ?

Well, there are many ways to achieve this. A very common technique, recommended by many 3rd party services, is using a webhook.

This technique allows us keep the data in both systems in alignment over time as they change by using a HTTP request.

This is what we will explore in this guide. Let’s take a look at how it works.

The Webhook

The webhook itself is nothing special, it is leveraging the HTTP protocol in order to transfer the data over the network to keep the data in alignment.

It uses what is referred to as a “push model”, where the updates from the 3rd party service is pushed to the receiving endpoint (your application).

The HTTP request consist of:

  • URL - Where to send the data

  • Body - The data sent

  • HTTP header - Details about how the data is formatted

  • Request Method - The http method, it is typically using a POST method for webhooks

To get a better idea of this technique, let’s go through an example using Stripe.

Example - Stripe webhook

1. The Registration

When we first start to integrate a webhook with Stripe, we need to register our URL.

This is the API endpoint which Stripe will use to push their updates to our application.

Registering our webhook url with Stripe

2. Pushing the update

Once we’ve registered the webhook, let’s say on our application, we can start receiving updates.

First, we would make a payment request to Stripe to initialize a payment transaction.

Then, Stripe would receive that request then process it whenever their system gets around to it.

Once the transaction has been processed, it would push the updates to our webhook URL.

This would include information such as the status of that payment transaction and other relevant details about that transaction.

Stripe pushing the update to our endpoint

3. Receiving the update

Once we’ve received the update on our end, we can do whatever we like with the latest data.

That’s really what webhooks do in a nutshell.

Our application receiving the update from Stripe

Now, in practice, there are a few things to consider such as security.

Let’s take a look at what are some of the common attacks vectors and the approaches to take in order to mitigate them.

Security

Here are a few security measures we should be mindful of when it comes to creating a webhook.

These security measures are:

  • Person-in-the-middle attacks

  • Unauthorized Access

  • Replay attacks

Person-in-the-middle attacks

To ensure to our connection is safe from person-in-the-middle attacks where malicious users can sniff the traffic and gain details about the request.

How do we overcome this ?

You should always ensure that the communication channel between your application and the 3rd party service is encrypted. This means using HTTPS (TLS/SSL).

Unauthorized Access

Using the webhook to receive updates likely means that we will need to expose a public endpoint.

So, how do we ensure that the events that we receive are only from a trusted and valid source ?

This is where authentication comes in.

There are a few methods that can be used to achieve this.

API token

Depending on the 3rd party service, some may provide an option to register a webhook with an API token or even generate one for you.

This token will be added to the Authorization header, so that you can verify the webhook requests that you receive.

This technique is not very common among the various 3rd party services that offer webhooks. Nevertheless, it is still an option some 3rd party services offers.

Digital signatures - HS256

Another technique for authentication is using HS256.

This technique is more common across the various 3rd party services, where it makes use of digital signatures for verification.

So, rather than using a token, it makes use of cryptographic signatures (or hashes, checksums, digital signatures) to ensure that request is coming from a valid source by verifying the signature from the sender.

HS256 stands for HMAC with SHA-256.

HMAC stands for hash-based message authentication code, which allows parties or systems to establish a trust relationship (If you like to learn more, check out the in-depth explaination below).

HMAC requires the following:

  • A shared secret (or key)

  • A Hashing algorithm (in our case it would be SHA-256)

Signature formula: SHA256(data, secret)

Webhooks digital signature

A good analogy of this process would be like a coat check at a venue, where the attendant would hand you a piece of paper with a number.

When you are ready to get your coat, you’d hand the attendant the number you have and you can retrieve your coat.

So, in this process, instead of getting verified through your identity, you were verified by the piece of paper with the number.

Let’s re-visit our example with Stripe again but now with signature verification.

Webhook process with signature Verification:

  1. Register with a webhook with Stripe with URL and shared secret

  2. Make a payment request to Stripe from our application

  3. Stripe would process it, then proceed to generate a signature by hashing the payload and the shared secret with SHA-256

  4. Stripe then sends the webhook request, and adds the signature to the header (ie Stripe-Signature )

  5. Our application receives the event, we re-generate the signature using the payload and the shared secret with SHA-256

  6. Now we match the generated signature against the signature received through the header of the request

  7. If it matches then we are good to go, otherwise, we should reject the request

IP restrictions

Another form of restricting access to our endpoint is adding an allow list of trusted IP. This gives us more control over which clients can make requests to the endpoints.

That said, this doesn’t replace having a method for authentication; However, it does reduce the attack surface.

Replay attacks

You can probably guess what the security concern is just from the name. Basically, it is an attack where malicious users some how gets access to a request that have been already received, and tries to replay them by making another request.

This may be due to a traffic log with details getting leaked or the request was sniffed in the network. So, it can happen.

Depending on the webhook, it can have impact on your system especially if it integrates anything that deals with payments.

So, what can we do about it ?

Timestamps

Stripe actually provide a neat way to overcome this challenge.

Within the webhook request, a timestamp is added as part of the signature, for example:

Stripe-Signature:
t=1492774577,
v1=5257a869e7ecebeda32affa62cdca3fa51cad7e77a0e56ff536d0ce8e108d8bd,
v0=6ffbb59b2300aae63f272406069a9788598b792a944a07aba816edb039989a39

by using the t value, we can know when this webhook was sent then we can set an expiry interval for our webhook requests.

That means any events received beyond the interval we set would be considered invalid and should be rejected (ie t <= t + 5 minutes).

Webhooks with expiry time interval

Keeping record

An even more robust solution is storing the webhooks that was received or a signature of it in a database or a cache (ie Redis).

That way every time you receive a webhook request, you can check the signature against the store to see if it was already received.

Conveniently, this implementation also handles duplicates if we ever receive them!

The down side of this approach is that the memory use may be large depending on the numbers of events you receive.

Conclusion

Before we conclude, let’s do a quick recap.

Recap:

  • The way we build applications today (using microservices) require some sort of way for us coordinate between the services

  • Webhook is technique that helps you coordinate between your application and a 3rd party service

  • Webhooks at its core are just http requests

  • The most common way to authenticate when using webhooks is via digital signatures using HS256

  • There are a few techniques to consider to keep your webhook endpoint secure in practice:

    • IP allow list
    • Authentication
    • Timestamp / Expiry interval
    • Encrypted connection (HTTPS)

That’s it! I hope you found this guide helpful! If you did, please share this article with a friend or co-worker 🙏!

Appendix

HS256 explained

As mentioned, HMAC stands for Hash-based authentication code, and this allows us to establish trust between parties and systems.

HMAC requires the following:

  • A shared secret (or key)

  • Hashing algorithm (in our case it would be SHA-256)

Why does this work ?

It works mainly because of SHA-256.

Hashing algorithms, like SHA-256, are one-way cryptographic hash functions that would always generate the same output given the same inputs.

So, that means whenever we receive webhooks request, we can re-generate the same signature provided that we have the shared secret (which we do).

Generating digital signature with SHA-256

What you need to know about SHA-256

SHA-256 meets all criterion of what is considered a secure hashing function.

The criterions are:

  • Preimage resistenace - The input cannot be determined from the output

  • Second preimage resistenace (Weak collision resistance) - given m1, you cannot find m2 such that the hash of m1 is equal to hash of m2, h(m1) = h(m2)

  • Collision resistance (Strong collision resistance) - It is difficult to find any m1 and m2 such that h(m1) = h(m2)

So, what are the chances someone of a collision ?

1 in 1^256 (This is a very small number, it is basically zero but not impossible!)

Note: Hashing functions are not to be confused with an encryption functions, you cannot reverse (or decrypt) a hash to determine its message.

📝 Helpful reference:


Enjoy the content ?

Then consider signing up to get notified when new content arrives!

Jerry Chang 2022. All rights reserved.