Collecting and Reporting Metrics

Honeybadger's Ruby gem (v5.11+) can be used to collect metrics and send them to Insights.

Automatic Instrumentation

You can instrument your Ruby and Rails apps by enabling the automatic instrumentation in your honeybadger.yml configuration file:

yaml

insights:
  enabled: true

Metric Collection

Enabling automatic instrumentation will collect metric data for the following libraries, where they will be displayed in the Insights section of your project:

Metric Source (`metric_source::str`)	Metric Name (`metric_name::str`)
`rails`	`duration.sql.active_record`
	`duration.process_action.action_controller`
	`db_runtime.process_action.action_controller`
	`view_runtime.process_action.action_controller`
	`duration.cache_read.active_support`
	`duration.cache_fetch_hit.active_support`
	`duration.cache_write.active_support`
	`duration.cache_exist?.active_support`
	`duration.render_partial.action_view`
	`duration.render_template.action_view`
	`duration.render_collection.action_view`
	`duration.perform.active_job`
`sidekiq`	`active_workers`
	`active_processes`
	`jobs_processed`
	`jobs_failed`
	`jobs_scheduled`
	`jobs_enqueued`
	`jobs_dead`
	`jobs_retry`
	`queue_latency`
	`queue_depth`
	`queue_busy`
	`capacity`
	`utilization`
`solid_queue`	`jobs_in_progress`
	`jobs_blocked`
	`jobs_failed`
	`jobs_scheduled`
	`jobs_processed`
	`active_workers`
	`active_dispatchers`
	`queue_depth`
`autotuner`	`diff.minor_gc_count`
	`diff.major_gc_count`
	`diff.time`
	`request_time`
	`heap_pages`
`puma`	`pool_capacity`
	`max_threads`
	`requests_count`
	`backlog`
	`running`
	`worker_index`
`net_http`	`duration.request`
`karafka`	`messages_consumed`
	`messages_consumed_bytes`
	`consume_attempts`
	`consume_errors`
	`receive_errors`
	`connection_connects`
	`connection_disconnects`
	`network_latency_avg`
	`network_latency_p95`
	`network_latency_p99`
	`consumer_lags`
	`consumer_lags_delta`
	`consumer_aggregated_lag`
	`error_occurred`
	`listener_polling_time_taken`
	`listener_polling_messages`
	`consumer_messages`
	`consumer_batches`
	`consumer_offset`
	`consumer_consumed_time_taken`
	`consumer_batch_size`
	`consumer_processing_lag`
	`consumer_consumption_lag`
	`consumer_revoked`
	`consumer_shutdown`
	`consumer_tick`
	`worker_total_threads`
	`worker_processing`
	`worker_enqueued_jobs`
	`worker_processing`

These metrics may be found using the following BadgerQL query:

  

fields @ts, @preview
| filter event_type::str == "metric.hb"
| filter metric_source::str == "sidekiq"
| filter metric_name::str == "active_workers"
| sort @ts

Disabling Insights for a Specific Plugin

When Insights is active, all plugins are enabled if the required library is present. For example, if Sidekiq is present in your app, the Sidekiq plugin will be loaded. You can disable automatic Insights instrumentation for a specific plugin by adding a configuration like this:

yaml

sidekiq:
  insights:
    enabled: false

This will only affect Insights-related data capture and not the error notification portion of the plugin.

Some plugins allow for an easy way to choose if you want to capture events, metrics, or both for a particular plugin. The following configuration options are available:

yaml

  

rails:
  insights:
    events: true
    metrics: false
karafka:
  insights:
    events: true
    metrics: false
sidekiq:
  insights:
    events: true
    metrics: false
net_http:
  insights:
    events: true
    metrics: false

Event options are all true by default. It is recommened to turn off events for plugins that may be producing more data than you actually need. Metric data collection is false by default since most metrics can be calculated from events.

Customizing Cluster Metrics Collection

Metrics collection is limited by a polling interval. This is set to 60 seconds by default. Honeybadger will execute the collect method of any enabled plugins sequentially, and then sleep for 60 seconds before calling calling the collect methods again. This can be tailored per plugin through a configuration parameter:

yaml

  

sidekiq:
  insights:
    collection_interval: 10

solid_queue:
  insights:
    collection_interval: 5

This will increase the frequency of the Sidekiq metrics collection to run every 10 seconds, and the SolidQueue metrics collection to run every 5 seconds.

Some metrics collection methods collect data based on the entire cluster of an application. In these cases, you would only need to collect data from a single instance of the Honeybadger gem. This helps save on unecessary load as well as data usage. Plugins collect data by default, but can be customized through configuration.

yaml

  

sidekiq:
  insights:
    cluster_collection: false

solid_queue:
  insights:
    cluster_collection: false

You can use this configuration paramter to control which instances you want collecting cluster based data. If you are using Sidekiq Enterprise, we automatically detect the leader instance and will enable cluster collection on that instance and disable it on others without any additional configuration.

Manually Collecting Your Own Metrics

The Honeybadger gem provides a API for defining and collecting your own metrics to feed into Insights.

Data Aggregation

When you collect metrics using the Honeybadger gem, the gem will aggregate data by default and report the results to Insights every 60 seconds. This allows you to collect data as much and as quickly as you want, while making efficient use of your daily data quota. If you want to tweak the resolution of the timing, you can configure it in the honeybadger.yml config file.

yaml

insights:
  enabled: true
  registry_flush_interval: 120

The above configuration will adjust the metric registry so that it reports every 2 minutes and help save more on data usage.

Types of Metrics

Gauge

A gauge tracks a specific value at a point in time. During aggregation, the metric will record the values: max, min, avg, and latest.

ruby

Honeybadger.gauge('data_size', ->{ file.byte_size })

Timer

Timers are similar to gauges in that they track a specific value in time. However, the time methods provides a convenient way to measure duration across your ruby operations.

ruby

Honeybadger.time('process_application', ->{ application.process })

Counter

Counters are simple numbers that you can increment or decrement by any value you wish.

ruby

Honeybadger.increment_counter('add_to_basket', { item_id: item.id })

Histogram

Histograms allows you to collate data values into predefined bins. The default bins are [0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10]. You can define your own set by passing a bins attribute to the metric. You may pass a callable lambda, which will be timed and the duration recorded. Or you may also pass a duration keyword argument if you have the value at hand.

ruby

Honeybadger.histogram('execute_request', ->{ request.execute })

# or

Honeybadger.histogram('execute_request', duration: duration)

Helper module

You can also include the helper module Honeybadger::InstrumentationHelper into any of your classes. The module provdes shortened forms to create the same metrics as metioned above, as well as other helper methods to customize your metrics.

Here is an example of how we can rewrite the example metrics above, while adding more custom attributes:

ruby

  

class MyMetrics
  include Honeybadger::InstrumentationHelper

  attr_reader :region

  def initialize(region)
    @region = region
  end

  def example
    metric_source 'custom_metrics'
    metric_attributes { region: region }

    gauge 'data_size', ->{ file.byte_size }
    time 'process_application', ->{ application.process }
    increment_counter 'add_to_basket', { item_id: item.id }
    histogram 'execute_request', ->{ request.execute }
  end
end

Aside from a less verbose API, there are two available helper methods that will aid in organizing your metrics. The metric_source method accepts the name of where your metrics are coming from. This can be the name of a library, or the class you are calling from. The metric_attributes method accepts a hash that will be passed to all metrics that follow.

Then you can find these metrics by using the following BadgerQL query:

  

fields @ts, @preview
| filter event_type::str == "metric.hb"
| filter metric_source::str == "custom_metrics"
| filter region::str == "some-region"
| sort @ts

Ignoring Metrics

You can use the before_event callback to inspect or modify metric data, as well as calling halt! to prevent the metric from being sent to Honeybadger:

ruby

  

Honeybadger.configure do |config|
  config.before_event do |event|
    if event.event_type == "metric.hb" && event[:metric_name] == "jobs_processed"
      event.halt!
    end
  end
end

before_event can be called multiple times to add multiple callbacks.

Similarly, you may also ignore metric events by configuring your honeybadger.yml config file by specifying a hash object:

yaml

  

events:
  ignore:
    - event_type: 'metric.hb'
      metric_name: 'jobs_processed'

Puma

Puma has it's own plugin system and requires a small change to your puma.rb. The Honeybadger gem comes with Puma plugin and can be enabled by adding the following to your puma.rb:

ruby

plugin :honeybadger

More Automatic Instrumentation

The Honeybadger Ruby gem provides more instrumentation than just metrics. When you enable Insights, you also enable automatic event logging. Check out Sending Events to Insights for more information.