Difference between revisions of "Smoothing observations"
(Created page with "Our plugins can reduce the amount of observations produced or sent to the platform by smoothing them. For instance of sending one observation each time a scanner detects a bea...") |
|||
| Line 1: | Line 1: | ||
| − | + | Many sensors produce observations every few seconds and it is very frequent to process all of them which implies high CPU use or network traffic. Sometimes, only a few of those observations are needed to be processed or taken into account like when there is a meaningful change on the monitored thing or situation. For instance, when the battery of a telephone is monitored, it could be interesting to know when it goes low (below 15% for instance) or critically low (let us say below 5%) to charge it, or when it is 100% to know that the battery is fully charged, while other values could lead to no action at all. Filtering the sensor observations, could help to reduce both network traffic and CPU processing time. This process of observation filtering is called "Smoothing" and our bridge allows to define which criteria should be use to reduce the observation production. | |
| + | |||
| + | The smoothing can be configured for each observation producer, because for one activity it could be interesting to detect every value change (to keep track of changes, for instance) but for others only some changes could be needed to be processed (to automate responses, alerts, etc). | ||
| + | |||
| + | Also, one producer could process observations from several devices, like in the case of beacon scanners, and so each one of them should be smoothed independently one of the other. For instance, the beacon's battery load values are kept for each one of the detected beacons, so the smoothing criteria old value of the corresponding beacon and not that of a different beacon. | ||
| + | |||
| + | Several field-wise criteria are proposed. | ||
| + | == Threshold comparison. == | ||
| + | The newly observed field value is compared to a given threshold and the observation is accepted to be produced whenever the condition is satisfied. It only works for numerical values and the options are | ||
| + | * greater than (>) | ||
| + | * greater than or equal (>=) | ||
| + | * less than (<) | ||
| + | * less than or equal (<=) | ||
| + | |||
| + | == State change.== | ||
| + | The observation is accepted when the defined condition is satisfied. The options are | ||
| + | * value has changed: the new value is different from previously observed one | ||
| + | * equals: the value is equal to the given one | ||
| + | * not equals: the value is not equal to the given one | ||
| + | |||
| + | == Variation threshold. == | ||
| + | The observation is accepted if the new difference between the new and old value satisfy the condition | ||
| + | * Percentage change: the difference is larger than the given percent from the old value | ||
| + | <nowiki> |'''newValue''' - '''oldValue'''|/|'''oldValue'''| >= '''percentage'''/100</nowiki> | ||
| + | or, when '''oldValue'''<1e-7, | ||
| + | <nowiki> |'''newValue''' - '''oldValue'''|/1e-7 >= '''percentage'''/100</nowiki> | ||
| + | |||
| + | == Category changes. == | ||
| + | Given a categorization of the values, the observation is accepted when the category associated to the old value, is not the same than this of the new value. The categorizations proposed are | ||
| + | * Partitioned value ranges | ||
| + | Given a list of partition limits, the category is given by the partition interval a value goes on. For instance given the limits 5,15 and 100, the intervals would be (-inf,5), [5,15), [15,100) and [100,+inf) and, thus, the observation is accepted when the field value changed from 4 to 6 but not when it went from 10 to 13. | ||
Revision as of 11:05, 2 February 2018
Many sensors produce observations every few seconds and it is very frequent to process all of them which implies high CPU use or network traffic. Sometimes, only a few of those observations are needed to be processed or taken into account like when there is a meaningful change on the monitored thing or situation. For instance, when the battery of a telephone is monitored, it could be interesting to know when it goes low (below 15% for instance) or critically low (let us say below 5%) to charge it, or when it is 100% to know that the battery is fully charged, while other values could lead to no action at all. Filtering the sensor observations, could help to reduce both network traffic and CPU processing time. This process of observation filtering is called "Smoothing" and our bridge allows to define which criteria should be use to reduce the observation production.
The smoothing can be configured for each observation producer, because for one activity it could be interesting to detect every value change (to keep track of changes, for instance) but for others only some changes could be needed to be processed (to automate responses, alerts, etc).
Also, one producer could process observations from several devices, like in the case of beacon scanners, and so each one of them should be smoothed independently one of the other. For instance, the beacon's battery load values are kept for each one of the detected beacons, so the smoothing criteria old value of the corresponding beacon and not that of a different beacon.
Several field-wise criteria are proposed.
Threshold comparison.
The newly observed field value is compared to a given threshold and the observation is accepted to be produced whenever the condition is satisfied. It only works for numerical values and the options are
- greater than (>)
- greater than or equal (>=)
- less than (<)
- less than or equal (<=)
State change.
The observation is accepted when the defined condition is satisfied. The options are
- value has changed: the new value is different from previously observed one
- equals: the value is equal to the given one
- not equals: the value is not equal to the given one
Variation threshold.
The observation is accepted if the new difference between the new and old value satisfy the condition
- Percentage change: the difference is larger than the given percent from the old value
|'''newValue''' - '''oldValue'''|/|'''oldValue'''| >= '''percentage'''/100 or, when oldValue<1e-7, |'''newValue''' - '''oldValue'''|/1e-7 >= '''percentage'''/100
Category changes.
Given a categorization of the values, the observation is accepted when the category associated to the old value, is not the same than this of the new value. The categorizations proposed are
- Partitioned value ranges
Given a list of partition limits, the category is given by the partition interval a value goes on. For instance given the limits 5,15 and 100, the intervals would be (-inf,5), [5,15), [15,100) and [100,+inf) and, thus, the observation is accepted when the field value changed from 4 to 6 but not when it went from 10 to 13.
