Anomaly detection¶
The goal is to detect anomalies on the basis of given difference of quantity values measured in two different time points. One time point is current time and the second one is selected from data that has been recently measured.
The configuration parameters in the examples below can be changed
in the configuration file config.ini
that is described in section Configuration file.
CO2¶
Detection of anomalies is based on difference in carbon dioxide concentration values measured in two various time points.
"""Anomaly detection - CO2.
"""
from os.path import dirname, abspath, join
import sys
sys.path.append(abspath(join(dirname(__file__), '../..', '')))
from dm.ConnectionUtil import ConnectionUtil as cu
from dm.models.ModelsUtil import ModelsUtil
from dm.WundergroundCom import WundergroundCom
from dm.DateTimeUtil import DateTimeUtil
if __name__ == '__main__':
cu.setup_logging()
cls = cu.setup_clients()
actual_time = int(DateTimeUtil.local_time_str_to_utc('2019/02/20 03:00:00').timestamp())
lat = 49.1649894
lon = 16.562262499999974
w = WundergroundCom()
w.api_key = cu.wunderground_api_key()
devs = [
{
'db_column_name': 'co2_in_ppm',
'name': '',
'gateway_id': '1816820318180747',
'device_id': '0xa900811026800001',
'module_id': 2,
'server_name': 'ant-work',
}
]
notification = ModelsUtil.anomaly_diff(devs, cls, lat, lon, actual_time, w, 2, 7200, 'co2')
ModelsUtil.json_to_file(notification, 'co2_notification.doc.json', log_notification=True)
It is necessary to set clients that can communicate with a remote server using the function setup_clients
.
Current time is stored in the variable actual_time
. Latitude and longitude that enable to get weather data
according to a sensor position are stored in the variable lat
and lon
. The API key allows to get weather data
from server and it is obtained by the function wunderground_api_key
. The sensors used to gather data are stored
in the variable devs
. Anomaly detection is performed by the function anomaly_diff
and the result is stored
in the variable notification
.
An output of the detection is a notification that contains basic information described in the section Notifications and the following information:
actual_diff - current difference in carbon dioxide concentration values,
anomaly - if anomaly is detected,
min_anomaly_diff - minimal difference in carbon dioxide concentration values to detect anomaly,
min_anomaly_time - time interval between two time points,
type - what quantity is used for prediction.
The example of a notification follows.
{
"data": {
"actual_diff": 0.0,
"anomaly": false,
"min_anomaly_diff": 2,
"min_anomaly_time": 120,
"type": "anomaly_co2"
},
"device_id": "0xa900811026800001",
"event": "env-notification-pre",
"gateway_id": "1816820318180747",
"raise": false,
"raise_catch": false,
"readable": "2019-02-20 03:00:00",
"timestamp": 1550628000
}
The example of a command that can be used to detect anomalies on the basis of carbon dioxide concentration difference.
python examples2/0305_anomaly_detection/co2_run.py
Temperature¶
Detection of anomalies is based on difference in temperature values measured in two various time points.
"""Anomaly detection - temperature.
"""
from os.path import dirname, abspath, join
import sys
sys.path.append(abspath(join(dirname(__file__), '../..', '')))
from dm.ConnectionUtil import ConnectionUtil as cu
from dm.models.ModelsUtil import ModelsUtil
from dm.WundergroundCom import WundergroundCom
from dm.DateTimeUtil import DateTimeUtil
if __name__ == '__main__':
cu.setup_logging()
cls = cu.setup_clients()
actual_time = int(DateTimeUtil.local_time_str_to_utc('2019/02/20 03:00:00').timestamp())
lat = 49.1649894
lon = 16.562262499999974
w = WundergroundCom()
w.api_key = cu.wunderground_api_key()
devs = [
{
'db_column_name': 'temperature_in_celsius',
'name': 'BeeeOn sensor',
'gateway_id': '1816820318180747',
'device_id': '0xa900811026800001',
'module_id': 0,
'server_name': 'ant-work',
}
]
notification = ModelsUtil.anomaly_diff(devs, cls, lat, lon, actual_time, w, 2, 7200, 'temperature')
ModelsUtil.json_to_file(notification, 't_notification.doc.json', log_notification=True)
It is necessary to set clients that can communicate with a remote server using the function setup_clients
.
Current time is stored in the variable actual_time
. Latitude and longitude that enable to get weather data
according to a sensor position are stored in the variable lat
and lon
. The API key allows to get weather data
from server and it is obtained by the function wunderground_api_key
. The sensors used to gather data are stored
in the variable devs
. Anomaly detection is performed by the function anomaly_diff
and the result is stored
in the variable notification
.
An output of the detection is a notification that contains basic information described in the section Notifications and the following information:
actual_diff - current difference in temperature values,
anomaly - if anomaly is detected,
min_anomaly_diff - minimal difference in temperature values to detect anomaly,
min_anomaly_time - time interval between two time points,
type - what quantity is used for prediction.
The example of a notification follows.
{
"data": {
"actual_diff": 0.22,
"anomaly": false,
"min_anomaly_diff": 2,
"min_anomaly_time": 120,
"type": "anomaly_temperature"
},
"device_id": "0xa900811026800001",
"event": "env-notification-pre",
"gateway_id": "1816820318180747",
"raise": false,
"raise_catch": false,
"readable": "2019-02-20 03:00:00",
"timestamp": 1550628000
}
The example of a command that can be used to detect anomalies on the basis of temperature difference.
python examples2/0305_anomaly_detection/t_run.py
Humidity¶
Detection of anomalies is based on difference in humidity values measured in two various time points.
"""Anomaly detection - humidity.
"""
from os.path import dirname, abspath, join
import sys
sys.path.append(abspath(join(dirname(__file__), '../..', '')))
from dm.ConnectionUtil import ConnectionUtil as cu
from dm.models.ModelsUtil import ModelsUtil
from dm.WundergroundCom import WundergroundCom
from dm.DateTimeUtil import DateTimeUtil
if __name__ == '__main__':
cu.setup_logging()
cls = cu.setup_clients()
actual_time = int(DateTimeUtil.local_time_str_to_utc('2019/02/20 03:00:00').timestamp())
lat = 49.1649894
lon = 16.562262499999974
w = WundergroundCom()
w.api_key = cu.wunderground_api_key()
devs = [
{
'db_column_name': 'rh_in_percentage',
'name': 'BeeeOn sensor',
'gateway_id': '1816820318180747',
'device_id': '0xa900811026800001',
'module_id': 1,
'server_name': 'ant-work',
}
]
notification = ModelsUtil.anomaly_diff(devs, cls, lat, lon, actual_time, w, 2, 7200, 'humidity')
ModelsUtil.json_to_file(notification, 'h_notification.doc.json', log_notification=True)
It is necessary to set clients that can communicate with a remote server using the function setup_clients
.
Current time is stored in the variable actual_time
. Latitude and longitude that enable to get weather data
according to a sensor position are stored in the variable lat
and lon
. The API key allows to get weather data
from server and it is obtained by the function wunderground_api_key
. The sensors used to gather data are stored
in the variable devs
. Anomaly detection is performed by the function anomaly_diff
and the result is stored
in the variable notification
.
An output of the detection is a notification that contains basic information described in the section Notifications and the following information:
actual_diff - current difference in humidity values,
anomaly - if anomaly is detected,
min_anomaly_diff - minimal difference in humidity values to detect anomaly,
min_anomaly_time - time interval between two time points,
type - what quantity is used for prediction.
The example of a notification follows.
{
"data": {
"actual_diff": 1.5,
"anomaly": false,
"min_anomaly_diff": 2,
"min_anomaly_time": 120,
"type": "anomaly_humidity"
},
"device_id": "0xa900811026800001",
"event": "env-notification-pre",
"gateway_id": "1816820318180747",
"raise": false,
"raise_catch": false,
"readable": "2019-02-20 03:00:00",
"timestamp": 1550628000
}
The example of a command that can be used to detect anomalies on the basis of humidity difference.
python examples2/0305_anomaly_detection/h_run.py