Skip to content
Honeybadger Docs

Ship your CloudWatch Logs to Honeybadger Insights

View Markdown

Ingesting logs from CloudWatch Logs requires setting up a Data Firehose stream with a HTTP Endpoint destination that sends events to our API. Once you create subscription filters for the desired log groups, log data from those groups will start flowing into Insights.

The easiest way to set this up is to use our CloudFormation template to create a CloudFormation stack in your AWS account. It will prompt you to enter your Honeybadger API key and the name of a log group that you want to connect to Data Firehose. You can quickly launch this template in your AWS account and then create additional log group subscriptions for other log groups you wish to monitor.

If you set up the Data Firehose stream manually, choose HTTP Endpoint as the destination and use the following URL as the HTTP Endpoint URL in the destination settings:

https://api.honeybadger.io/v1/data-firehose-events?api_key=PROJECT_API_KEY

Setting default fields with a query parameter

Section titled “Setting default fields with a query parameter”

Events from this endpoint are built from the CloudWatch Logs record, so they always have the same shape: a ts, a message, and the logGroup and logStream the record came from. If you want more than that — the environment, the region, the name of the app — you can add a defaults query parameter to the HTTP Endpoint URL containing a URL-encoded JSON object, and its fields will be merged into every event the stream delivers:

https://api.honeybadger.io/v1/data-firehose-events?api_key=PROJECT_API_KEY&defaults={"environment":"production","region":"us-east-1"}

With that URL, an event that would otherwise be stored as:

{"ts": "2023-08-31T09:19:30.000Z", "logGroup": "/aws/lambda/checkout", "logStream": "2023/08/31/[$LATEST]abc123", "message": "This is a log line"}

…is stored as:

{"ts": "2023-08-31T09:19:30.000Z", "logGroup": "/aws/lambda/checkout", "logStream": "2023/08/31/[$LATEST]abc123", "message": "This is a log line", "environment": "production", "region": "us-east-1"}

Because the parameter lives on the destination URL, each Data Firehose stream can carry its own metadata — a useful way to tag events by environment or account when you’re shipping logs from more than one place, without running a transformation Lambda to rewrite the payloads.

The defaults parameter has a few restrictions:

  • It must be a flat JSON object; values must be strings, numbers, or booleans.
  • The keys event_type and ts are reserved and will be ignored.
  • It’s limited to 16 keys and 2kB (URL-decoded).
  • Fields we build from the log record always win — a default named message or logGroup won’t overwrite the real one.

An invalid defaults parameter never causes the delivery to fail: entries that break the rules above are dropped (an unparseable or oversized parameter is ignored entirely), and the events are ingested without them. Likewise, if merging the defaults would push an event past the 100kB per-event size limit, we drop the defaults for that event rather than the event itself.