dm package

Subpackages

Submodules

dm.AttributeUtil module

Creates (additional) training set and testing set, keeps number of records in both sets in equilibrium.

class dm.AttributeUtil.AttributeUtil

Bases: object

static additional_training_set(no_event_records, func, row_selector, interval_selector, columns_map, print_each=10)

It additionally generates training data based on given time points.

Parameters
  • no_event_records – list of no events

  • func – function that computes the attributes

  • row_selector – selector of row

  • interval_selector – selector of interval

  • columns_map – dictionary containing mapping of attribute name in database to the name used in dataset

  • print_each – step in printing of log messages, other log messages will not be printed

Returns

list containing pairs of attribute name and value

static balance_set(training_set, additional_training_set)

It computes training data whereas opposite events are replaced.

It computes training data whereas opposite events are replaced with events from additional training set, the result is a balanced list.

Parameters
  • training_set – list of training data

  • additional_training_set – list of additional training data

Returns

list containing pairs of attribute name and value that is balanced

static cached_training_data(events, func, row_selector, interval_selector, event_type, file_path, columns_map, print_each=10)

It gets training data.

If method is called for the first time, training data will be computed and stored to file. Otherwise it only reads training data from the file.

Parameters
  • events – list of events

  • func – function that computes the attributes

  • row_selector – selector of row

  • interval_selector – selector of interval

  • event_type – type of event - open or close

  • file_path – path to a cached file

  • columns_map – dictionary containing mapping of attribute name in database to the name used in dataset

  • print_each – step in printing of log messages, other log messages will not be printed

Returns

pair of lists, the first list contains names of attributes and the second one contains pairs of attribute name and value

static testing_data(con, table_name, start, end, write_each, func, row_selector, interval_selector, event_type, columns_map, log_every_hour=3)

It generates testing data.

Parameters
  • con – connection to the database

  • table_name – name of table

  • start – start of interval from which testing data is generated

  • end – end of interval from which testing data is generated

  • write_each – step in writing of computed attributes to the output file

  • func – function that computes the attributes

  • row_selector – selector of row

  • interval_selector – selector of interval

  • event_type – type of event - open or close

  • columns_map – dictionary containing mapping of attribute name in database to the name used in dataset

  • log_every_hour – step in printing of log messages, other log messages will not be printed

Returns

list containing pairs of attribute name and value

static testing_data_with_write(con, table_name, start, end, write_each, func, row_selector, interval_selector, event_type, output_filename, columns_map, row_count=2048, log_every_hour=1)

It generates testing data, continuous writing to a file is optional.

Parameters
  • con – connection to the database

  • table_name – name of table

  • start – start of interval from which testing data is generated

  • end – end of interval from which testing data is generated

  • write_each – step in writing of computed attributes to the output file

  • func – function that computes the attributes

  • row_selector – selector of row

  • interval_selector – selector of interval

  • event_type – type of event - open or close

  • output_filename – output filename

  • columns_map – dictionary containing mapping of attribute name in database to the name used in dataset

  • row_count – number of attributes that are written at one time

  • log_every_hour – step in printing of log messages, other log messages will not be printed

Returns

list containing pairs of attribute name and value

static testing_one_row(func, timestamp, row_selector, interval_selector, event_type, c_mapper, end)

It computes one row with attributes.

Parameters
  • func – function that computes the attributes

  • timestamp – timestamp when an event occurred

  • row_selector – selector of row

  • interval_selector – selector of interval

  • event_type – type of event - open or close

  • c_mapper – dictionary containing mapping of attribute name in database to the name used in dataset

  • end – if the end of an event is available

Returns

list containing pairs of attribute name and value

static training_data(events, func, row_selector, interval_selector, event_type, columns_map, print_each=10)

It generates training data.

Parameters
  • events – list of events

  • func – function that computes the attributes

  • row_selector – selector of row

  • interval_selector – selector of interval

  • event_type – type of event - open or close

  • columns_map – dictionary containing mapping of attribute name in database to the name used in dataset

  • print_each – step in printing of log messages, other log messages will not be printed

Returns

pair of lists, the first list contains names of attributes and the second one contains pairs of attribute name and value

static training_data_without_opposite(events, func, row_selector, interval_selector, columns_map, print_each=10)

It computes training data without opposite.

Opposite event represents an event when nothing occurred.

Parameters
  • events – list of events

  • func – function that computes the attributes

  • row_selector – selector of row

  • interval_selector – selector of interval

  • columns_map – containing mapping of attribute name in database to the name used in dataset

  • print_each – step in printing of log messages, other log messages will not be printed

Returns

list containing pairs of attribute name and value

dm.BeeeOnClient module

REST API client for communication with BeeeOn server.

class dm.BeeeOnClient.BeeeOnClient(host, port)

Bases: object

Client for communication with server supporting BeeeOn api.

property api_key
device_info(gateway_id, device_id)

It gets information about device.

Parameters
  • gateway_id – identifier of gateway

  • device_id – identifier of device

Returns

json that contains information about device

history(gateway, device, sensor, start, end, interval=1, aggregation='avg')

It gets historical data from server.

Parameters
  • gateway – identifier of gateway

  • device – identifier of device

  • sensor – identifier of sensor

  • start – start of interval from which data is generated

  • end – end of interval from which data is generated

  • interval – time interval from which data are aggregated

  • aggregation – type of aggregation

Returns

json that contains historical data from given time interval

refresh_token()

It refreshes token.

Returns

token id

sensors_info(gateway_id, device_id)

It gets information about sensor.

Parameters
  • gateway_id – identifier of gateway

  • device_id – identifier of device

Returns

json that contains information about sensor

property token_id

dm.CSVUtil module

Creates a csv file from OrderedDict.

class dm.CSVUtil.CSVUtil

Bases: object

static create_csv_file(data: list, filename: str, enable_append=False)

It creates a CSV file.

Parameters
  • data – list of pairs containing attribute name and its value

  • filename – output filename

  • enable_append – if it is true, data is appended to the end of file

Returns

None

static create_csv_io(data: list)

It creates StringIO from given data.

Parameters

data – list of pairs containing attribute name and its value

Returns

StringIO containing given data

dm.ConnectionUtil module

Utils to create connections to db, REST client and utils for environment setting.

class dm.ConnectionUtil.ConnectionUtil

Bases: object

MAX_TESTABLE_EVENTS = 75
static ant_work_api_key(server_type, config_file='/etc/dp/config.ini')

It gets information about api key from configuration file.

Parameters
  • server_type – type of server - dm

  • config_file – path to configuration file

Returns

information about api key

static create_con(config_file='/etc/dp/config.ini')

It creates connection to the database.

Parameters

config_file – path to the configuration file

Returns

connection to the database

static estimate(key, config_file='/etc/dp/config.ini')

It gets information about estimation from configuration file.

Parameters
  • key – key for obtaining of data from information about estimation

  • config_file – path to configuration file

Returns

information about estimation obtained using the key

static get_value(key, config_file)

It gets value using given key from configuration file.

Parameters
  • key – key for obtaining of data from information

  • config_file – path to the configuration file

Returns

required data obtained using the key

static is_testable_system()

It gets information if system is travis for testing.

Returns

true if system is travis for testing, otherwise false

static open_detector(key, config_file='/etc/dp/config.ini')

It gets information about open detector from configuration file.

Parameters
  • key – key for obtaining of data from information about open detector

  • config_file – path to configuration file

Returns

information about open detector obtained using the key

static open_weather_api_key(config_file='/etc/dp/config.ini')

It gets information about OpenWeather.org api key from configuration file.

Parameters

config_file – path to configuration file

Returns

information about api key

static package(key, config_file='/etc/dp/config.ini')

It gets information about package from configuration file.

Parameters
  • key – key for obtaining of data from information about package

  • config_file – path to configuration file

Returns

information about package obtained using the key

static predictor(key, config_file='/etc/dp/config.ini')

It gets information about OpenWeather.org api key from configuration file.

Parameters
  • key – key for obtaining of data from information about predictor

  • config_file – path to configuration file

Returns

information about predictor obtained using the key

static rapid_miner(config_file='/etc/dp/config.ini')

It gets information about RapidMiner.

Parameters

config_file – path to the configuration file

Returns

information about RapidMiner

static rehivetech_api_key(server_type, config_file='/etc/dp/config.ini')

It gets information about api key from configuration file.

Parameters
  • server_type – type of server - dm or acontroller

  • config_file – path to configuration file

Returns

information about api key

static setup_clients()

It sets clients used for communication with servers.

Returns

dictionary of set clients

static setup_clients_logout(cls)

It log outs clients.

Parameters

cls – list of clients for communication with server

Returns

None

static setup_logging(log_level=10)

It sets loggers for logging.

Parameters

log_level – type of logging level

Returns

None

static wunderground_api_key(config_file='/etc/dp/config.ini')

It gets information about WunderGround.com api key from configuration file.

Parameters

config_file – path to configuration file

Returns

information about api key

dm.DBUtil module

Defines functions for working with database.

class dm.DBUtil.DBUtil

Bases: object

static check_timestamp_order(con, table_name)

It checks if a given table in database does not contain missing value.

Parameters
  • con – connection to the database

  • table_name – name of table

Returns

None

static create_table(conn, table_name)

It creates table containing measured values.

http://www.mysqltutorial.org/mysql-decimal/ :param conn: connection to the database :param table_name: name of table :return: None

static delete_from_time(con, table_name, delay)

It removes several last rows from a given table.

Parameters
  • con – connection to the database

  • table_name – name of table

  • delay – value used to define size of window from which rows are removed

Returns

None

static drop_table(conn, table_name)

It removes a given table.

Parameters
  • conn – connection to the database

  • table_name – name of table

Returns

None

static first_inserted_values(conn, table_name)

It gets the first inserted value into given table.

Parameters
  • conn – connection to the database

  • table_name – name of table

Returns

first inserted value

static insert_value(conn, task, enable_commit, table_name)

It inserts values into the table.

Parameters
  • conn – connection to the database

  • task – list of pairs that contain name of column and value

  • enable_commit – if true, result of a query is written into the database

  • table_name – name of table

Returns

None

static last_inserted_open_close_state(conn, table_name)

It gets the last inserted state of window (open or closed).

Parameters
  • conn – connection to the database

  • table_name – name of table

Returns

last inserted window state

static last_inserted_values(conn, table_name)

It gets the last inserted value into given table.

Parameters
  • conn – connection to the database

  • table_name – name of table

Returns

last inserted value

static measured_values_table_column_names()

It return list of available column names in table.

Returns

list of column names

static rows_count(conn, table_name)

It gets number of rows in given table.

Parameters
  • conn – connection to the database

  • table_name – name of table

Returns

number of rows in the table

static update_attribute(con, table_name, attribute, value, timestamp)

It updates value of a given attribute in given time.

Parameters
  • con – connection to the database

  • table_name – name of table

  • attribute – name of attribute

  • value – new value

  • timestamp – time that denotes a row that will be updated

Returns

None

dm.DateTimeUtil module

Converts various time formats (one to another).

class dm.DateTimeUtil.DateTimeUtil

Bases: object

static create_interval_str(start, end)

It converts given interval expressed using UTC timestamp to string form.

Parameters
  • start – start of interval

  • end – end of interval

Returns

interval expressed in string form

static local_time_str_to_utc(date_str, timezone='Europe/Prague', format='%Y/%m/%d %H:%M:%S')

It converts given local time in string form to UTC form (number).

Parameters
  • date_str – local time in string form

  • timezone – timezone of local time

  • format – output format of time

Returns

object containing local time in UTC form

static utc_timestamp_to_local_time(timestamp, timezone='Europe/Prague')

It converts given UTC timestamp to local time according to timezone.

Parameters
  • timestamp – UTC timestamp

  • timezone – timezone of local time

Returns

local time

static utc_timestamp_to_str(timestamp, format='%Y-%m-%d %H:%M:%S')

It converts UTC timestamp to local time in string form.

Parameters
  • timestamp – UTC timestamp

  • format – output format of time

Returns

local time in string form

dm.Differences module

dm.ExampleRunner module

Runs detector or more detectors.

class dm.ExampleRunner.ExampleRunner

Bases: object

static detector(before, after, script_name, process, enable_training)

It runs a process in RapidMiner.

Parameters
  • before – time shift before start of event

  • after – time shift after start of event

  • script_name – name of script

  • process – name of RapidMiner process

  • enable_training – if training phase should be performed

Returns

None

static feature_stats(script_name, enable_training)

It runs computing of statistics.

Parameters
  • script_name – name of script

  • enable_training – if training phase should be performed

Returns

None

dm.FilterUtil module

Defines various filters to select data that meets a required condition.

class dm.FilterUtil.FilterUtil

Bases: object

static attribute(events, attribute_name, value)

It gets events in which attribute contains given value.

Parameters
  • events – list of events

  • attribute_name – name of attribute

  • value – required value of attribute

Returns

list of events in which attribute contains given value

static attribute_exclude(events, attribute_name, value)

It gets events in which attribute does not contain given value.

Parameters
  • events – list of events

  • attribute_name – name of attribute

  • value – not required value of attribute

Returns

list of events in which attribute does not contain given value

static derivation_not_zero(events)

It gets events in which derivation is not zero.

Parameters

events – list of events

Returns

list of events in which derivation is not zero

static humidity(events, min_out_specific_humidity, min_diff, max_diff)

It gets events that fulfill the conditions.

It gets events in which specific humidity difference was in admissible range and outside specific humidity was not higher than given value.

Parameters
  • events – list of events

  • min_out_specific_humidity

  • min_diff

  • max_diff

Returns

list of events

static measured_values_not_empty(events, attributes)

It gets events that include valid measured data in given attributes.

Parameters
  • events – list of events

  • attributes – names of attributes

Returns

list of events that include valid measured data in given attributes

static min_length(events, length)

It gets events that lasted at least given time.

Parameters
  • events – list of events

  • length – minimal length of event

Returns

list of events that lasted at least given time

static min_max_time_interval(events, min_time, max_time)

It gets events that lasted admissible time.

Parameters
  • events – list of events

  • min_time – minimal length of event

  • max_time – maximal length of event

Returns

list of events that lasted admissible time

static min_time_interval(events, time_interval_length)

It gets events that lasted at least the required time.

Parameters
  • events – list of events

  • time_interval_length – length of time interval

Returns

list of events that lasted at least the required time

static min_timestamp(events, timestamp)

It gets events that started after given time.

Parameters
  • events – list of events

  • timestamp – timestamp from which events are considered

Returns

list of events that started after given time.

static only_valid_events(events)

It gets only valid events.

Parameters

events – list of events

Returns

list of valid events

static temperature_diff(events, min_value, max_value)

It gets events in which temperature difference was in admissible range.

Parameters
  • events – list of events

  • min_value – minimal difference of temperature

  • max_value – maximal difference of temperature

Returns

list of events in which temperature difference was in admissible range

static temperature_out_max(events, max_value)

It gets events in which temperature was not higher than given value.

Parameters
  • events – list of events

  • max_value – maximal temperature

Returns

list of events in which temperature was not higher than given value

dm.Graph module

Utils to create graph that can be available in html file.

class dm.Graph.Graph(path)

Bases: object

static db_to_simple_graph(event, column, color, label, number_output_records)

It generates a graph based on data in a database.

Parameters
  • event – an event

  • column – name of column (attribute)

  • color – color of curve

  • label – label for x-axis

  • number_output_records – number of time points in a graph

Returns

graph containing course of a given attribute

gen(data, output, scale_padding_min=0, scale_padding_max=0, g_type='line', min_value=None, max_value=None, global_range=False)

It generates a HTML page containing a graph.

Parameters
  • data – data containing information about events

  • output – filename

  • scale_padding_min – padding calculated using the minimal value of an attribute

  • scale_padding_max – padding calculated using the maximal value of an attribute

  • g_type – shape of a curve

  • min_value – minimal value of an attribute in a graph

  • max_value – maximal value of an attribute in a graph

  • global_range – if true, range is calculated for each graph from given data, otherwise the same range is used for all graphs

Returns

None

dm.GraphUtil module

Generates histogram, stacked barplot and grouped barplot.

class dm.GraphUtil.GraphUtil

Bases: object

static gen_duration_histogram(events, action, extensions, title, intervals, threshold)

It generates histogram of ventilation lengths.

Parameters
  • events – list of events

  • action – show|save - for histogram showing or saving

  • extensions – list of extensions used in case of action ‘save’

  • title – graph title

  • intervals – list of intervals in minutes for which number of events is calculated

  • threshold – value that is added to the interval in minutes or is subtracted from the interval to create value interval for a given column in a histogram

Returns

None

static gen_grouped_barplot(first_col, second_col, third_col, action, filename)

It generates grouped bar chart.

Parameters
  • first_col – deprecated

  • second_col – data (values of an attribute) used to plot graph

  • third_col – data (values of an attribute) used to plot graph

  • action – show|save - for graph showing or saving

  • filename – output filename

Returns

None

static gen_stacked_barplot(first_col, second_col, third_col)

It generates stacked bar graph.

Parameters
  • first_col – data (values of an attribute) used to plot graph

  • second_col – data (values of an attribute) used to plot graph

  • third_col – data (values of an attribute) used to plot graph

Returns

None

dm.HTTPClient module

REST api client with Bearer authentization.

class dm.HTTPClient.HTTPClient(host, port, end_point, method, verify_ssl_cert=True)

Bases: object

Simple HTTP client.

authorize(session_id)

It sets authentization.

Parameters

session_id – session identifier

Returns

None

body(body)

It sets HTTP body.

Parameters

body – HTTP body

Returns

None

perform()

It performs HTTP request.

It creates HTTP connection, performs HTTP request, processes HTTP response and closes the connection.

Returns

HTTP response and its content

dm.HeatMap module

Graph representing heat map.

class dm.HeatMap.HeatMap

Bases: object

static annotate_heatmap(im, data=None, valfmt='{x:.2f}', textcolors=['black', 'white'], threshold=None, **textkw)

It annotates a heatmap.

Parameters
  • im – the AxesImage to be labeled

  • data – data used to annotate; ff None, the image’s data is used

  • valfmt – the format of the annotations inside the heatmap; this should either use the string format method, e.g. “$ {x:.2f}”, or be a matplotlib.ticker.Formatter

  • textcolors – a list or array of two color specifications; the first is used for values below a threshold, the second for those above

  • threshold – value in data units according to which the colors from textcolors are applied; if None (the default), the middle of the colormap is used as separation

  • **textkw

Returns

static heatmap(data, row_labels, col_labels, ax=None, cbar_kw={}, cbarlabel='', **kwargs)

It create a heatmap from a numpy array and two lists of labels.

Parameters
  • data – a 2D numpy array of shape (N, M)

  • row_labels – a list or array of length N with the labels for the rows

  • col_labels – a list or array of length M with the labels for the columns

  • ax

    a matplotlib.axes.Axes instance to which the heatmap

    is plotted; if it is not provided, current axes are used or

    a new one is created

  • cbar_kw – a dictionary with arguments to matplotlib.Figure.colorbar().

  • cbarlabel – the label for the colorbar

  • **kwargs

Returns

dm.OpenWeatherOrg module

Gets information about weather from OpenWeather.org.

class dm.OpenWeatherOrg.OpenWeatherOrg(precision=2)

Bases: object

property api_key
weather_by_city_id(quantities, start, end, city_id: int)

It gets weather from city based on its identifier during certain time interval.

Parameters
  • quantities – list of required quantities

  • start – timestamp that denotes start of time interval

  • end – timestamp that denotes end of time interval

  • city_id – identifier of city

Returns

weather data

weather_by_city_name(quantities, start, end, name)

It gets weather from a given city during certain time interval.

Parameters
  • quantities – list of required quantities

  • start – timestamp that denotes start of time interval

  • end – timestamp that denotes end of time interval

  • name – name of city

Returns

weather data

weather_by_coordinates(quantities, start, end, lat, lon)

It gets weather from city based on its coordinates during certain time interval.

Parameters
  • quantities – list of required quantities

  • start – timestamp that denotes start of time interval

  • end – timestamp that denotes end of time interval

  • lat – latitude

  • lon – longitude

Returns

weather data

dm.Performance module

Calculates accuracy using standard approach and accuracy considering a tolerance interval.

class dm.Performance.Performance(filename)

Bases: object

simple()

It calculates standard accuracy.

Returns

confusion matrix including number of events and accuracy; list of dates and times of events when a window was closed but window opening was detected (false positive); structure containing information from confusion matrix and sum of false positives, false negatives, true positives and true negatives

with_delay(before, after)

It calculates accuracy considering a tolerance interval.

Parameters
  • before – number of seconds before start of an event

  • after – number of seconds after start of an event

Returns

confusion matrix including number of events and accuracy; list of dates and times of events when a window was closed but window opening was detected (false positive); structure containing information from confusion matrix and sum of false positives, false negatives, true positives and true negatives

dm.Performance.count(z, val)

Deprecated.

dm.PreProcessing module

Utils for date pre-processing.

class dm.PreProcessing.PreProcessing

Bases: object

OPEN_CLOSE_ATTR_NAME = 'open_close'
TIME_ATTR_NAME = 'measured_time'
TIME_STRING_ATTR_NAME = 'measured_time_str'
static check_start_end_interval(items: list, time_attribute: str) → None

It checks if lists of values have the same starting and end time.

Parameters
  • items – list of values

  • time_attribute – name of attributes that contains timestamp

Returns

None

static cut_interval(items: list, start: int, end: int, time_attribute: str) → list

It cuts data to have the same start and end of an interval (starting and end time).

Parameters
  • items – list of values

  • start – timestamp from which data is cut

  • end – timestamp to which data is cut

  • time_attribute – shift of start and end of time intervals

Returns

cut list of values

static db_name_maps(devices: list) → list

It gets a list of column names in database that is loaded from list of devices.

A column name is loaded from a file to assign a given value to the right column in database.

Parameters

devices – list of information about devices

Returns

list of column names

static download_data(clients: list, devices: list, start: int, end: int) → list

It downloads required data according to a list of devices.

Parameters
  • clients – list of clients

  • devices – list of information about devices

  • start – timestamp from which data is downloaded

  • end – timestamp to which data is downloaded

Returns

list of downloaded data

static extract_items(data, value_column)

It prepares data.

Parameters
  • data – dictionary of data

  • value_column – name of column

Returns

list of values and list of timestamps

static generate_data(values: list, value_attribute: str, time_attribute: str, precision: int = 7) → list

It modifies downloaded data so that an interval between two adjacent records was one second.

It modifies the data using linearization, it does not consider quantities or their course.

Parameters
  • values – list of values

  • value_attribute – name of attribute that contains data

  • time_attribute – ame of attribute that contains timestamp

  • precision – precision of calculation

Returns

list of values that contains modified downloaded data (with step of one second)

static generate_open_close(values: list, time_attribute_name: str, open_close_attribute_name: str, start: int, end: int, last_open_close_state: int) → list

It modifies downloaded data so that an interval between two adjacent records was one second.

It modifies the data using linearization, it does not consider quantities or their course.

Parameters
  • values – list of values

  • time_attribute_name – name of attribute that contains timestamp

  • open_close_attribute_name – name of attribute that contains state of window (open/closed)

  • start – timestamp that denotes start of an interval

  • end – timestamp that denotes end of an interval

  • last_open_close_state – last recorded state of a window

Returns

list of values that contains modified downloaded data (with step of one second)

static insert_values(conn, table_name, values, maps, write_each, precision)

It inserts values to a table.

Parameters
  • conn – connection to a database

  • table_name – name of table

  • values – data related to event

  • maps – list of column names

  • write_each – number of events - 1 between two adjacent events that are written to a table

  • precision – precision of inserted values

Returns

None

static join_items(items: list, time_attribute: str) → list

It joins several lists of values into one list.

The resulting list contains only one attribute that includes time because times are the same. It contains various values that were included in various lists.

THe method has to be called only if passed list of values was checked using the method check_start_end_interval().

Parameters
  • items – list of values

  • time_attribute – name of attribute that contains timestamp

Returns

list of values including all attributes and time

static ppm_filter(data, ppm_limit=2000)

It sets carbon dioxide concentration to None if its value is out of range.

Parameters
  • data – dictionary of data

  • ppm_limit – maximal admissible value of the concentration

Returns

dictionary of data where excessive concentration is set to None

static prepare(clients: list, devices: list, start: int, end: int, last_open_close_state: int, time_shift: int)

It prepares data.

Parameters
  • clients – list of clients

  • devices – list of information about devices

  • start – timestamp from which data is downloaded

  • end – timestamp to which data is downloaded

  • last_open_close_state – last recorded state of a window

  • time_shift – time shift that is subtracted or added to start or end of an interval respectively.

Returns

list of attribute names and list of values

static prepare_downloaded_data(clients: list, devices: list, start: int, end: int, time_shift: int, last_open_close_state) → list

It prepares downloaded data.

Parameters
  • clients – list of clients

  • devices – list of information about devices

  • start – timestamp that denotes start of an interval

  • end – timestamp that denotes end of an interval

  • time_shift – time shift that is subtracted or added to start or end of an interval respectively

  • last_open_close_state – last recorded state of a window

Returns

dictionary of prepared data

static prepare_value_conversion(value)

It converts relative humidity to absolute and specific humidity.

Parameters

value – data related to an event

Returns

data related to an event including absolute and specific humidity

static rename_all_attributes(items: list, devices: list) → list

It renames all attributes in a list that contains list of values.

It renames individual attributes that is related to a given module. Each downloaded value from list of values with a defined name of attributes by default. New names of attributes are derived from list of devices.

Parameters
  • items – list of values

  • devices – list of information about devices

Returns

list that contains a modified list of values including renamed attributes

static rename_attribute(values: list, old_attribute: str, new_attribute: str) → list

It renames an attribute in data from an original name to a new name.

Parameters
  • values – list of values

  • old_attribute – name of an original attribute

  • new_attribute – name of a new attribute

Returns

list of pairs of a renamed attribute and its value

static value_filter(data)

It checks if indoor relative humidity is in an admissible range.

If indoor relative humidity is out of range, it is logged.

Parameters

data – dictionary of data

Returns

dictionary of data

dm.SQLUtil module

Utils that contain simple SQL queries.

class dm.SQLUtil.SQLUtil

Bases: object

static select_interval(table_name: str, start: int, end: int, columns: str)

It selects given columns from a table on the basis of given time interval.

Parameters
  • table_name – name of table

  • start – timestamp that denotes start of a time interval

  • end – timestamp that denotes end of a time interval

  • columns – names of columns

Returns

SQL query

static select_interval_size(table_name: str, start: int, end: int, column: str)

It finds out number of records on the basis of given time interval.

Parameters
  • table_name – name of table

  • start – timestamp that denotes start of a time interval

  • end – timestamp that denotes end of a time interval

  • column – name of column

Returns

SQL query

static select_one_value(table_name: str, measured_time: int, columns: str)

It selects one record from a table on the basis of given timestamp.

Parameters
  • table_name – name of table

  • measured_time – timestamp that is used to select a record

  • columns – list of columns that are selected

Returns

SQL query

dm.Storage module

Storage that supports selection of events and their values or only selection of the values.

class dm.Storage.Storage(filename: str, no_event_time_shift: int, table_name: str)

Bases: object

static dw_columns_ordered(con, start, end, columns, table_name)

It gets several rows (interval) from a table and orders them.

Parameters
  • con – connection to a database

  • start – timestamp that denotes start of a time interval

  • end – timestamp that denotes end of a time interval

  • columns – list of columns

  • table_name – name of table

Returns

ordered rows (interval)

load_data(con, start_shift: int, end_shift: int, column: str)

It loads information about events. :param con: connection to a database :param start_shift: number of seconds added to start of event :param end_shift: number of seconds added to end of event :param column: name of column :return: dictionary of data

static one_row(con, table_name: str, columns: str, timestamp: int)

It gets one row (required columns) from a table.

Parameters
  • con – connection to a database

  • table_name – name of table

  • columns – list of columns

  • timestamp – timestamp that determines a required row

Returns

one row (required columns)

read_meta()

It reads information about events.

Returns

list of attributes describing events

static select_interval(con, start, end, column, table_name, without_none_value=True)

It gets several rows (interval) from a table.

Parameters
  • con – connection to a database

  • start – timestamp that denotes start of a time interval

  • end – timestamp that denotes end of a time interval

  • column – name of column

  • table_name – name of table

  • without_none_value – true if the result cannot contain null values

Returns

several rows (interval)

dm.ValueConversionUtil module

Converts relative humidity to absolute humidity, relative humidity to specific humidity and ppm to milligrams per cubic meter.

class dm.ValueConversionUtil.ValueConversionUtil

Bases: object

CO_MOLECULAR_WEIGHT = 44.0095
static ah_to_relative_percent(temp: float, ah: float) → float

It converts absolute humidity to relative humidity.

Parameters
  • temp – temperature

  • ah – absolute humidity

Returns

relative humidity

static co2_ppm_to_mg_m3(co2)

It converts ppm to milligrams per cubic meter.

Parameters

co2 – ppm

Returns

milligrams per cubic meter

static rh_to_absolute_g_m3(temp: float, rh: float) → float

It converts relative humidity to absolute humidity.

Parameters
  • temp – temperature

  • rh – relative humidity

Returns

absolute humidity

static rh_to_specific_g_kg(temp: float, rh: float) → float

It converts relative humidity to specific humidity.

Parameters
  • temp – temperature

  • rh – relative humidity

Returns

specific humidity

dm.ValueUtil module

Utils for detection of simple events.

class dm.ValueUtil.ValueUtil

Bases: object

static delays(events, delays_attr_name)

It gets list of delays.

Parameters
  • events – list of events

  • delays_attr_name – name of attribute that denotes delay

Returns

list of delays

static detect_sensor_delay(values, window_size, threshold)

It detects sensor delay.

Parameters
  • values – list of value

  • window_size – size of a window in seconds

  • threshold – threshold in seconds

Returns

index of value when a decrease occurred

static detect_window_action(values_count: int, actual_index: int)

It detects a window action.

Parameters
  • values_count – count of values

  • actual_index – actual index

Returns

window action

static events_duration(events, max_duration)

It gets durations of events that lasted at most given time.

Parameters
  • events – dictionary of events

  • max_duration – maximal duration of event

Returns

list of durations of events

static window_event_value(measured: dict, value_index: int, timestamp: int, precision: int)

It gets information about event when a window was open.

Parameters
  • measured – dictionary of measured values

  • value_index – index to a dictionary

  • timestamp – timestamp when an event occurred

  • precision – precision of output values

Returns

information about event when a window was open

static window_no_event_value(values: tuple, precision: int)

It gets information about event when a window was not open.

Parameters
  • values – measured values of quantities

  • precision – precision of output values

Returns

information about event when a window was not open

dm.WundergroundCom module

Weather data extraction from weather.com.

class dm.WundergroundCom.WundergroundCom(precision=1)

Bases: object

actual_weather(lat, lon)

It gets information about current weather in a locality.

Parameters
  • lat – latitude of a locality

  • lon – longitude of a locality

Returns

information about current weather

property api_key
weather_by_coordinates(quantities, start, end, lat, lon)

It gets information about weather in a locality at required time interval.

Parameters
  • quantities – deprecated

  • start – timestamp that denotes start of time interval

  • end – timestamp that denotes end of time interval

  • lat – latitude of a locality

  • lon – longitude of a locality

Returns

information about weather

Module contents