Data Collection

Circonus brokers have the ability to both actively collect and passively receive telemetry data from systems.


Active Collection

Active collection (also known as polling) is when the broker plays the active role in collecting data from a system. This usually is as simple as the broker asking a system a question and then waiting for the answer(s).


Circonus's architecture is specifically designed to overcome scaling challenges often present in polling-based monitoring systems. Circonus can scale to hundreds of thousands of polling checks without issue.


Passive Collection

The opposite of active collection is when the broker plays a passive role in data collection. In this scenario, the broker collects telemetry data from the system being monitored as it is emitted.


Choosing Active vs. Passive

Circonus's design eliminates the scaling challenges associated with active data collection, allowing users to choose the method that offers the most value on the data in question.

The decision usually depends on data velocity and observability. If the data you are interested in is occurring quickly and persistently (a high maintained velocity), then it is often the case that you want to assess the system by observing actual work. This would mean we should use a passive monitoring setup to collect such data. However, not all systems expose this data in a way that can be observed passively, so you are limited to systems that expose data in this way (over protocols such as statsd, collectd, or HTTPtrap).


ACTIVE VS PASSIVE DATABASE EXAMPLE

For example, say you would like to monitor a MySQL database server and must decide whether to use Active or Passive Collection. Ideally you would like to know how many transactions per second are ongoing. The way tools typically do this is: (1) contact the server and ask how many total transactions have been performed, (2) wait, (3) contact the server and ask again. By subtracting the the two results and dividing the difference by the elapsed time between queries we can determine how many queries/second were executed during that time period. This is an active collection method.


Passive collection of the same database requires the observation of every query in the system and the submission of its latency. This is possible within Circonus for some databases. For example, the PostgreSQL database has a plugin that will emit telemetry data regarding every query via the statsd protocol. Circonus can be configured to receive this statsd data and provide rich and wholly-representative information about query latencies over time using histograms.


Note

Some systems will deploy an agent on the database server that actively checks the instance at regular intervals and then passively submits the data to a monitoring system. This hybrid approach (while workable) is inappropriate for Circonus. It is intended to avoid problems of scalability that are simply not present in Circonus and removes the value of real-time, on-demand data collection.