Window opening detector

The aim is to design a system detecting window opening events on the basis of a carbon dioxide concentration or humidity change. The window opening detector according to a carbon dioxide concentration is based only on the course of indoor carbon dioxide concentration because indoor and outside relative humidity and temperature did not influence the success rate of a detector. The detector based on humidity considers also the course of temperature.

The configuration parameters in the examples below can be changed in the configuration file config.ini that is described in section Configuration file.

CO2

Today, people spend a considerable part of their life in office and living spaces. Therefore it is necessary to maintain healthy indoor environment. The concentration of carbon dioxide is an indicator of the quality of the indoor environment. Excessive concentration can cause fatigue, headache or sore throat. The long-term effect of high concentration of carbon dioxide on the human body can lead to asthma and other respiratory diseases. The concentration of carbon dioxide can be influenced by ventilation. If the ventilation is performed right, no health problems with the respiratory tract and with the concentration of people occur. Moreover, ventilation only for the necessary time contributes to saving energy.

Initialize general models from local database

A local database was created for debugging purposes. The database includes pre-processed data that enables fast computing of required attributes. The example of a model created using data stored in the database follows.

"""Initialize general models from local database.
"""
from os.path import dirname, abspath, join
import sys
sys.path.append(abspath(join(dirname(__file__), '../..', '')))

import os

from dm.models.ModelsUtil import ModelsUtil
from dm.models.open_detector.generic_training_file_from_local_db import *
from dm.ConnectionUtil import ConnectionUtil as cu


if __name__ == '__main__':
    cu.setup_logging()

    table_name = 'measured_filtered_peto'
    columns = ColumnMapper.OPEN_CO2
    no_event_shift = int(cu.open_detector('attrs.no_event.time_shift'))

    directory = cu.open_detector('generic.directory')
    if not os.path.isdir(directory):
        os.mkdir(directory)

    co2_csv = cu.open_detector('generic.co2.data_file.name') + '_from_local_db.bin'
    co2_model = cu.open_detector('generic.co2.model.name') + '_from_local_db.bin'
    data_co2 = training_set_co2(cu.package('co2.event_file.name'), no_event_shift, table_name,
                                co2_csv, columns)
    ModelsUtil.write_model(data_co2, co2_model, ModelsUtil.replace_nothing_open)

The dataset in the example is created using columns of the table measured_filtered_peto, the columns are stored in the variable OPEN_CO2. It is required to create a dataset that contains the same number of events when a window was open and when it was closed. To select the events when the window was closed the variable no_event_shift is defined. Its value can be set using the parameter attrs.no_event.time_shift. The variable directory that can be set using the parameter generic.directory denotes directory where a created model will be stored. A model is created using the function training_set_co2 that uses SVM classifier (sklearn library). The created model is written to a binary file using the write_model function.

The example of a command which can be used to create a model using data stored in the database is below.

python3 examples2/0300_open_detector/co2_init_general_models_from_local_db.py

Initialize general models from a remote server

The example of a model created using data saved on the server follows.

"""Initialize general models from a remote server.
"""
from os.path import dirname, abspath, join
import sys
sys.path.append(abspath(join(dirname(__file__), '../..', '')))

import json

from dm.models.open_detector.generic_training_file import generic_training_file
from dm.ConnectionUtil import ConnectionUtil as cu
from dm.WundergroundCom import WundergroundCom
from dm.models.open_detector.create_attrs import ColumnMapper
from dm.models.ModelsUtil import ModelsUtil


if __name__ == '__main__':
    cu.setup_logging()
    cls = cu.setup_clients()

    no_event_shift = int(cu.open_detector('attrs.no_event.time_shift'))

    with open(cu.open_detector('generic.devices.path'), 'r') as f:
        devs = json.load(f)

    w = WundergroundCom()
    w.api_key = cu.wunderground_api_key()

    data_co2 = generic_training_file(cu.package('co2.event_file.name'),
                                     no_event_shift, 'co2', ColumnMapper.OPEN_CO2, cls,
                                     devs, w)
    ModelsUtil.write_model(data_co2, cu.open_detector('generic.co2.model.name'),
                           ModelsUtil.replace_nothing_open)

At first, it is necessary to set clients that can communicate with a remote server using the function setup_clients. Then it is required to create a dataset that contains the same number of events when a window was open and when it was closed. To select the events when the window was closed the variable no_event_shift is defined. Its value can be set using the parameter attrs.no_event.time_shift. Devices that were used to gather data are stored in the variable devs. The API key allows to get weather data from server and it is obtained by the function wunderground_api_key. A model is created using the function generic_training_file that uses SVM classifier (sklearn library). The created model is written to a binary file using the write_model function.

The example of a command which can be used to create a model using data saved on the remote server is below.

python examples2/0300_open_detector/co2_init_general_models.py

Initialize adapted model

The example of a model adapted to a given room follows.

"""Initialize adapted model.
"""
from os.path import dirname, abspath, join
import sys
sys.path.append(abspath(join(dirname(__file__), '../..', '')))

import time
import requests
import json

from dm.WundergroundCom import WundergroundCom
from dm.ConnectionUtil import ConnectionUtil as cu
from dm.models.open_detector.adapted_models import prepare_adapted_data_co2


def prepare_adapted_co2(cls, actual_time):
    url_req = 'https://api.aurorahub.io/watchdogs-refresh'
    json_data = requests.get(url_req).text
    python_obj = json.loads(json_data)['co2_open_window']

    cl = cls['rehivetech']

    for item in python_obj:
        first_seen = None
        out_dev = []

        for dev in item['input']:
            res = cl.device_info(dev['gateway_id'], dev['device_id'])

            if first_seen is None:
                first_seen = res['first_seen']

            if first_seen is not None:
                first_seen = max(first_seen, res['first_seen'])

            if 'open_close' in dev['modules']:
                out_dev.append(
                    {
                        'db_column_name': 'open_close',
                        'name': 'BeeeOn sensor',
                        'gateway_id': str(dev['gateway_id']),
                        'device_id': str(dev['device_id']),
                        'module_id': dev['modules'].index('open_close'),
                        'server_name': 'rehivetech',
                    }
                )

            if 'co2' in dev['modules']:
                out_dev.append(
                    {
                        'db_column_name': 'co2_in_ppm',
                        'name': 'BeeeOn sensor',
                        'gateway_id': str(dev['gateway_id']),
                        'device_id': str(dev['device_id']),
                        'module_id': dev['modules'].index('co2'),
                        'server_name': 'rehivetech',
                    }
                )

        coord = item['location']['coord']
        prepare_adapted_data_co2(out_dev, cls, first_seen, actual_time,
                                 coord['lat'], coord['lon'], w)


if __name__ == '__main__':
    cu.setup_logging()
    cls = cu.setup_clients()

    actual_time = int(time.time())

    w = WundergroundCom()
    w.api_key = cu.wunderground_api_key()

    prepare_adapted_co2(cls, actual_time)

It is necessary to set clients that can communicate with a remote server using the function setup_clients. The current time is stored in the variable actual_time. The API key allows to get weather data from server and it is obtained by the function wunderground_api_key. The function prepare_adapted_co2 creates a model using data gathered by devices up to the current time. The devices are stored in the variable out_dev. To create the model data from the general model can be also used. This option can be enabled or disabled using the parameter adapted.combined_with_generic_data in the configuration file config.ini. If the parameter is set to Yes, data from the general model will be used, if it is set to No, data will not be used. The resulting model is stored in the directory adapted_data by default.

The example of a command which can be used to adapt a model to a certain room is below.

python examples2/0300_open_detector/co2_init_adapted_model.py

Detector

The example of the use of a model for window opening detection follows.

"""Detector.
"""
from os.path import dirname, abspath, join
import sys
sys.path.append(abspath(join(dirname(__file__), '../..', '')))

from dm.ConnectionUtil import ConnectionUtil as cu
from dm.models.ModelsUtil import ModelsUtil
from dm.WundergroundCom import WundergroundCom
from dm.DateTimeUtil import DateTimeUtil


if __name__ == '__main__':
    cu.setup_logging()
    cls = cu.setup_clients()

    actual_time = int(DateTimeUtil.local_time_str_to_utc('2019/02/20 03:00:00').timestamp())
    lat = 49.1649894
    lon = 16.562262499999974

    w = WundergroundCom()
    w.api_key = cu.wunderground_api_key()

    devs = [
        {
            'db_column_name': 'co2_in_ppm',
            'gateway_id': '1816820318180747',
            'device_id': '0xa900811026800001',
            'module_id': 2,
            'server_name': 'ant-work',
        }
    ]

    notification = ModelsUtil.estimate_open_co2(devs, cls, lat, lon, actual_time, w)
    ModelsUtil.json_to_file(notification, 'co2_notification.doc.json', log_notification=True)

It is necessary to set clients that can communicate with a remote server using the function setup_clients. Detection is performed on the basis of data measured during a given time interval. Time that defines end of a time interval is stored in the variable actual_time. Latitude and longitude that enable to get weather data according to a sensor position are stored in the variable lat and lon. The API key allows to get weather data from server and it is obtained by the function wunderground_api_key. The sensor used to gather data is stored in the variable devs. Estimation if a window was open or not based on the concentration of carbon dioxide at the given time is performed by the function estimate_open_co2 and the result is stored in the variable notification.

An output of a detection is a notification that contains basic information described in the section Notifications and information if a window was open or not which is stored in the key estimate_open.

{
    "data": {
        "estimate_open": false,
        "type": "co2_open"
    },
    "device_id": "0xa900811026800001",
    "event": "env-notification-pre",
    "gateway_id": "1816820318180747",
    "raise": true,
    "raise_catch": false,
    "readable": "2019-02-20 03:00:00",
    "timestamp": 1550628000
}

The example of a command which can be used to the use of a model for window opening detection is below.

python examples2/0300_open_detector/co2_detector.py

Temperature and humidity

People spend about 90 % of their life indoors. Therefore it is necessary to maintain a healthy indoor environment that is significantly affected by humidity. It is possible to determine the common symptoms of excessive humidity in buildings. For example, people can suffer from allergy or asthma thanks to airborne dust mites and mould spores that are spread in the excessively humid air. A big difference between indoor and outdoor temperature can cause condensation when water drops or fog can occur on the window glass. Mould spots and musty odours indicate the presence of mould and mildew that can result in health problems. A humid environment encourages mildew, mould and bacterial growth that negatively influence indoor air quality. Ventilation is able to remove pollutants and humidity forming indoors or reduce their concentrations to admissible levels for the occupant health and comfort. It should be energy efficient, preserve indoor air quality and it should not harm the occupants or the building.

Initialize general models from local database

A local database was created for debugging purposes. The database includes pre-processed data that enables fast computing of required attributes. The example of a model created using data stored in the database follows.

"""Initialize general models from local database.
"""
from os.path import dirname, abspath, join
import sys
sys.path.append(abspath(join(dirname(__file__), '../..', '')))

import os

from dm.models.ModelsUtil import ModelsUtil
from dm.models.open_detector.generic_training_file_from_local_db import *
from dm.ConnectionUtil import ConnectionUtil as cu


if __name__ == '__main__':
    cu.setup_logging()

    table_name = 'measured_klarka'
    columns = ColumnMapper.OPEN_T_H
    no_event_shift = int(cu.open_detector('attrs.no_event.time_shift'))

    directory = cu.open_detector('generic.directory')
    if not os.path.isdir(directory):
        os.mkdir(directory)

    t_h_csv = cu.open_detector('generic.t_h.data_file.name') + '_from_local_db.bin'
    t_h_model = cu.open_detector('generic.t_h.model.name') + '_from_local_db.bin'
    data_t_h = training_set_t_h(cu.package('t_h.event_file.name'), no_event_shift, table_name,
                                t_h_csv, columns)
    ModelsUtil.write_model(data_t_h, t_h_model, ModelsUtil.replace_nothing_open)

The dataset in the example is created using columns of the table measured_filtered_klarka, the columns are stored in the variable OPEN_T_H. It is required to create a dataset that contains the same number of events when a window was open and when it was closed. To select the events when the window was closed the variable no_event_shift is defined. Its value can be set using the parameter attrs.no_event.time_shift. The variable directory that can be set using the parameter generic.directory denotes directory where a created model will be stored. A model is created using the function training_set_t_h that uses SVM classifier (sklearn library). The created model is written to a binary file using the write_model function.

The example of a command which can be used to create a model using data stored in the database is below.

python3 examples2/0300_open_detector/t_h_init_general_models_from_local_db.py

Initialize general models from a remote server

The example of a model created using data saved on the server follows.

"""Initialize general models from a remote server.
"""
from os.path import dirname, abspath, join
import sys
sys.path.append(abspath(join(dirname(__file__), '../..', '')))

import json
from dm.models.open_detector.generic_training_file import generic_training_file
from dm.ConnectionUtil import ConnectionUtil as cu
from dm.WundergroundCom import WundergroundCom
from dm.models.open_detector.create_attrs import ColumnMapper
from dm.models.ModelsUtil import ModelsUtil


if __name__ == '__main__':
    cu.setup_logging()
    cls = cu.setup_clients()

    no_event_shift = int(cu.open_detector('attrs.no_event.time_shift'))

    with open(cu.open_detector('generic.devices.path'), 'r') as f:
        devs = json.load(f)

    w = WundergroundCom()
    w.api_key = cu.wunderground_api_key()

    data_t_h = generic_training_file(cu.package('t_h.event_file.name'),
                                     no_event_shift, 't_h', ColumnMapper.OPEN_T_H, cls,
                                     devs, w)
    ModelsUtil.write_model(data_t_h, cu.open_detector('generic.t_h.model.name'),
                           ModelsUtil.replace_nothing_open)

At first, it is necessary to set clients that can communicate with a remote server using the function setup_clients. Then it is required to create a dataset that contains the same number of events when a window was open and when it was closed. To select the events when the window was closed the variable no_event_shift is defined. Its value can be set using the parameter attrs.no_event.time_shift. Devices that were used to gather data are stored in the variable devs. The API key allows to get weather data from server and it is obtained by the function wunderground_api_key. A model is created using the function generic_training_file that uses SVM classifier (sklearn library). The created model is written to a binary file using the write_model function. The example of a command which can be used to create a model using data saved on the remote server is below.

python examples2/0300_open_detector/t_h_init_general_models.py

Initialize adapted model

The example of a model adapted to a given room follows.

"""Initialize adapted model.
"""
from os.path import dirname, abspath, join
import sys
sys.path.append(abspath(join(dirname(__file__), '../..', '')))

from dm.ConnectionUtil import ConnectionUtil as cu
from dm.models.open_detector.adapted_models import prepare_adapted_data_t_h
from dm.WundergroundCom import WundergroundCom
from dm.DateTimeUtil import DateTimeUtil


if __name__ == '__main__':
    cu.setup_logging()
    cls = cu.setup_clients()

    actual_time = int(DateTimeUtil.local_time_str_to_utc('2019/02/20 03:00:00').timestamp())
    lat = 49.1649894
    lon = 16.562262499999974

    w = WundergroundCom()
    w.api_key = cu.wunderground_api_key()

    devs = [
        {
            'db_column_name': 'open_close',
            'gateway_id': '1908402624139667',
            'device_id': '0x900000000197053',
            'module_id': 0,
            'server_name': 'ant-work',
        },
        {
            'db_column_name': 'temperature_in2_celsius',
            'gateway_id': '1816820318180747',
            'device_id': '0xa900811026800001',
            'module_id': 0,
            'server_name': 'ant-work',
        },
        {
            'db_column_name': 'rh_in2_percentage',
            'gateway_id': '1816820318180747',
            'device_id': '0xa900811026800001',
            'module_id': 1,
            'server_name': 'ant-work',
        }
    ]

    prepare_adapted_data_t_h(devs, cls, actual_time - 24 * 60 * 60 * 15, actual_time,
                             lat, lon, w)

It is necessary to set clients that can communicate with a remote server using the function setup_clients. Adaptation is performed on the basis of data measured during a given time interval. Time that defines end of a time interval is stored in the variable actual_time. Latitude and longitude that enable to get weather data according to a sensor position are stored in the variable lat and lon. The API key allows to get weather data from server and it is obtained by the function wunderground_api_key. The sensors used to gather data are stored in the variable devs. The function prepare_adapted_data_t_h creates a model using data gathered by devices up to the current time. To create the model data from the general model can also be used. This option can be enabled or disabled using the parameter enable or disable. The resulting model is stored in the directory adapted_data by default.

The example of a command which can be used to adapt a model to a certain room is below.

python examples2/0300_open_detector/t_h_init_adapted_model.py

Detector

The example of the use of a model for window opening detection follows.

"""Detector.
"""
from os.path import dirname, abspath, join
import sys
sys.path.append(abspath(join(dirname(__file__), '../..', '')))

from dm.ConnectionUtil import ConnectionUtil as cu
from dm.models.ModelsUtil import ModelsUtil
from dm.WundergroundCom import WundergroundCom
from dm.DateTimeUtil import DateTimeUtil


if __name__ == '__main__':
    cu.setup_logging()
    cls = cu.setup_clients()

    actual_time = int(DateTimeUtil.local_time_str_to_utc('2019/02/20 03:00:00').timestamp())
    lat = 49.1649894
    lon = 16.562262499999974

    w = WundergroundCom()
    w.api_key = cu.wunderground_api_key()

    devs = [
        {
            'db_column_name': 'temperature_in2_celsius',
            'gateway_id': '1816820318180747',
            'device_id': '0xa900811026800001',
            'module_id': 0,
            'server_name': 'ant-work',
        },
        {
            'db_column_name': 'rh_in2_percentage',
            'gateway_id': '1816820318180747',
            'device_id': '0xa900811026800001',
            'module_id': 1,
            'server_name': 'ant-work',
        }
    ]

    notification = ModelsUtil.estimate_open_t_h(devs, cls, lat, lon, actual_time, w)
    ModelsUtil.json_to_file(notification, 't_h_notification.doc.json', log_notification=True)

It is necessary to set clients that can communicate with a remote server using the function setup_clients. Detection is performed on the basis of data measured during a given time interval. Time that defines end of a time interval is stored in the variable actual_time. Latitude and longitude that enable to get weather data according to a sensor position are stored in the variable lat and lon. The API key allows to get weather data from server and it is obtained by the function wunderground_api_key. The sensors used to gather data are stored in the variable devs. Estimation if a window was open or not based on the humidity and the temperature at the given time is performed by the function estimate_open_t_h and the result is stored in the variable notification.

An output of a detection is a notification that contains basic information described in the section Notifications and information if a window was open or not which is stored in the key estimate_open.

{
    "data": {
        "estimate_open": false,
        "type": "t_h_open"
    },
    "device_id": "0xa900811026800001",
    "event": "env-notification-pre",
    "gateway_id": "1816820318180747",
    "raise": true,
    "raise_catch": false,
    "readable": "2019-02-20 03:00:00",
    "timestamp": 1550628000
}

The example of a command which can be used to the use of a model for window opening detection is below.

python examples2/0300_open_detector/t_h_detector.py