What is Cloudwatch Embedded Metrics ?

Goals

After reading the article, you should:

• ✅ Be able to differentiate between Logs & Metrics

• ✅ Know what and why of Cloudwatch Embedded metrics

• ✅ Gain a high-level understanding of what it takes to convert logs into metrics in AWS

Content

• Introduction
• What is it ?
• Why it matters
• Supported Platform
• Example - using AWS Lambda
• Client Libraries
• Conclusion

Introduction

If you use AWS then there is a good chance you are using AWS Cloudwatch logs for your logging.

Logs are very useful for specific events within your code between requests.

However, if you want to gain high level insights, you want to be looking at an aggregation of events at a point in time for a system rather than a specific event. This is where metrics comes in.

Now the question become — how do you go from the logs to the metrics ?

If you are using AWS, then there is an easy way to achieve this, and that is using AWS Cloudwatch embedded metrics.

The gist of it is that these are logs formatted in a specific format that AWS uses to convert these logs into metrics.

Let’s go through the details.

What is it ?

AWS Cloudwatch embedded metrics are just logs in a specific format.

It needs to be a valid JSON, and contain the right details so AWS can convert these logs into metrics.

Format: x-amzn-logs-format: json/emf

Here is a sample of the format:

{
  "_aws": {
    "Timestamp": 1574109732004,
    "CloudWatchMetrics": [
      {
        "Namespace": "lambda-function-metrics",
        "Dimensions": [["functionVersion"]],
        "Metrics": [
          {
            "Name": "time",
            "Unit": "Milliseconds"
          }
        ]
      }
    ]
  },
  "functionVersion": "$LATEST",
  "time": 100,
  "requestId": "989ffbf8-9ace-4817-a57c-e4dd734019ee"
}

📝 Helpful reference:

• AWS - Cloudwatch Embedded metrics specification

Why it matters

Here are a few points of why I think AWS Cloudwatch embedded metrics matters.

Metrics are actionable

You go from logs (specific events) to high level insights (metrics), you can’t improve or analyze what you can’t see.

These metrics can be used to define contracts such as SLAs, SLIs and SLOs.

AWS Cloudwatch embedded metrics make this process that much easier and simpler.

Great Developer Experience (DX)

This feature is an opt-in feature, meaning its optional.

When you want to “enable” it, you just format your log in a specific way.

No friction, no extra steps — It simply just works! In my opinion, that’s a big win.

Shifted operational responsibility

In this setup, AWS assumes almost all the responsibility of this feature and infrastructure.

This reduces the operational burdens that you or your team is responsible for.

To me, that’s another big win!

Supported Platforms

Using AWS Lambda, you don’t need any further configuration, it just works.

If you are using AWS ECS or EKS, then the cloudwatch agent is required to make it work.

• AWS Lambda - Cloudwatch embedded metrics are supported out of the box

• AWS ECS and EKS - Requires the CloudWatch agent to send Cloudwatch embedded metrics

Example - using AWS Lambda

Here is an example of how AWS Cloudwatch embedded metrics works with AWS Lambda. Again, this works out of the box.

Sample AWS Architecture illustrating how Cloudwatch embedded metrics work

The steps:

1. API gateway receives request

2. API Gateway invokes Lambda Alias

3. Lambda processes the event

4. Lambda logs to cloudwatch logs ⭐️

5. Logs are automatically converted to Cloudwatch metrics

Step 1-4 are pretty typical in an AWS Lambda setup, the big difference is that when you log in a certain format, AWS will pick that up and convert those logs into metrics.

Step 4 is where the magic happens!

You can format these logs manually or use a client library provided by AWS (recommended).

Client Libraries

To make it easier, AWS has provided client libraries to generate these formatted logs.

It provides an API to help you create formatted logs that results in desired cloudwatch metrics based on the settings you provide.

The client libraries:

• Node.js
• Python
• Java
• C#

Example using the client library with Node.js:

const { createMetricsLogger, Unit } = require("aws-embedded-metrics");

const sendMetrics = async () => {
  const metrics = createMetricsLogger();
  metrics.putDimensions({ Service: "Aggregator" });
  metrics.putMetric("ProcessingLatency", 100, Unit.Milliseconds);
  metrics.setProperty("RequestId", "422b1569-16f6-4a03-b8f0-fe3fd9b100f8");
  // ...
  await metrics.flush();
};

await sendMetrics();

Conclusion

So, to recap:

• Metrics provide high-level insights while logs provide details about specific events

• AWS Cloudwatch embedded metrics converts your AWS Cloudwatch logs into AWS Cloudwatch metrics

• When using AWS Lambda, embedded metrics are supported out of the box

• When using AWS ECS & EKS, the cloudwatch agent will need to be configured

• AWS provides client libraries to makes it easier for you manage the resulting embedded metrics

And... That’s all. I hope you found that helpful!

If you learned something new, please share this article with a friend or co-worker 🙏❤️! (Thanks!)