dm package¶
Subpackages¶
- dm.attrs package
- Submodules
- dm.attrs.AbstractPrepareAttr module
- dm.attrs.CO2VentilationLength module
- dm.attrs.DiffInLinear module
- dm.attrs.DifferenceBetweenRealLinear module
- dm.attrs.FirstDifferenceAttrA module
- dm.attrs.FirstDifferenceAttrB module
- dm.attrs.GrowthRate module
- dm.attrs.InLinear module
- dm.attrs.InOutDiff module
- dm.attrs.Regression module
- dm.attrs.SecondDifferenceAttr module
- dm.attrs.VentilationLength module
- Module contents
- dm.co2regression package
- dm.coefficients package
- dm.models package
- dm.selectors package
- Subpackages
- dm.selectors.from_server package
- dm.selectors.interval package
- dm.selectors.row package
- Submodules
- dm.selectors.row.AbstractTableRowSelector module
- dm.selectors.row.CachedDiffRowWithIntervalSelector module
- dm.selectors.row.CachedRowWithIntervalSelector module
- dm.selectors.row.LinearSimpleCachedRowSelector module
- dm.selectors.row.SimpleCachedRowSelector module
- dm.selectors.row.SimpleRowSelector module
- Module contents
- Submodules
- dm.selectors.AbstractSelector module
- Module contents
- Subpackages
Submodules¶
dm.AttributeUtil module¶
Creates (additional) training set and testing set, keeps number of records in both sets in equilibrium.
-
class
dm.AttributeUtil.
AttributeUtil
¶ Bases:
object
-
static
additional_training_set
(no_event_records, func, row_selector, interval_selector, columns_map, print_each=10)¶ It additionally generates training data based on given time points.
- Parameters
no_event_records – list of no events
func – function that computes the attributes
row_selector – selector of row
interval_selector – selector of interval
columns_map – dictionary containing mapping of attribute name in database to the name used in dataset
print_each – step in printing of log messages, other log messages will not be printed
- Returns
list containing pairs of attribute name and value
-
static
balance_set
(training_set, additional_training_set)¶ It computes training data whereas opposite events are replaced.
It computes training data whereas opposite events are replaced with events from additional training set, the result is a balanced list.
- Parameters
training_set – list of training data
additional_training_set – list of additional training data
- Returns
list containing pairs of attribute name and value that is balanced
-
static
cached_training_data
(events, func, row_selector, interval_selector, event_type, file_path, columns_map, print_each=10)¶ It gets training data.
If method is called for the first time, training data will be computed and stored to file. Otherwise it only reads training data from the file.
- Parameters
events – list of events
func – function that computes the attributes
row_selector – selector of row
interval_selector – selector of interval
event_type – type of event - open or close
file_path – path to a cached file
columns_map – dictionary containing mapping of attribute name in database to the name used in dataset
print_each – step in printing of log messages, other log messages will not be printed
- Returns
pair of lists, the first list contains names of attributes and the second one contains pairs of attribute name and value
-
static
testing_data
(con, table_name, start, end, write_each, func, row_selector, interval_selector, event_type, columns_map, log_every_hour=3)¶ It generates testing data.
- Parameters
con – connection to the database
table_name – name of table
start – start of interval from which testing data is generated
end – end of interval from which testing data is generated
write_each – step in writing of computed attributes to the output file
func – function that computes the attributes
row_selector – selector of row
interval_selector – selector of interval
event_type – type of event - open or close
columns_map – dictionary containing mapping of attribute name in database to the name used in dataset
log_every_hour – step in printing of log messages, other log messages will not be printed
- Returns
list containing pairs of attribute name and value
-
static
testing_data_with_write
(con, table_name, start, end, write_each, func, row_selector, interval_selector, event_type, output_filename, columns_map, row_count=2048, log_every_hour=1)¶ It generates testing data, continuous writing to a file is optional.
- Parameters
con – connection to the database
table_name – name of table
start – start of interval from which testing data is generated
end – end of interval from which testing data is generated
write_each – step in writing of computed attributes to the output file
func – function that computes the attributes
row_selector – selector of row
interval_selector – selector of interval
event_type – type of event - open or close
output_filename – output filename
columns_map – dictionary containing mapping of attribute name in database to the name used in dataset
row_count – number of attributes that are written at one time
log_every_hour – step in printing of log messages, other log messages will not be printed
- Returns
list containing pairs of attribute name and value
-
static
testing_one_row
(func, timestamp, row_selector, interval_selector, event_type, c_mapper, end)¶ It computes one row with attributes.
- Parameters
func – function that computes the attributes
timestamp – timestamp when an event occurred
row_selector – selector of row
interval_selector – selector of interval
event_type – type of event - open or close
c_mapper – dictionary containing mapping of attribute name in database to the name used in dataset
end – if the end of an event is available
- Returns
list containing pairs of attribute name and value
-
static
training_data
(events, func, row_selector, interval_selector, event_type, columns_map, print_each=10)¶ It generates training data.
- Parameters
events – list of events
func – function that computes the attributes
row_selector – selector of row
interval_selector – selector of interval
event_type – type of event - open or close
columns_map – dictionary containing mapping of attribute name in database to the name used in dataset
print_each – step in printing of log messages, other log messages will not be printed
- Returns
pair of lists, the first list contains names of attributes and the second one contains pairs of attribute name and value
-
static
training_data_without_opposite
(events, func, row_selector, interval_selector, columns_map, print_each=10)¶ It computes training data without opposite.
Opposite event represents an event when nothing occurred.
- Parameters
events – list of events
func – function that computes the attributes
row_selector – selector of row
interval_selector – selector of interval
columns_map – containing mapping of attribute name in database to the name used in dataset
print_each – step in printing of log messages, other log messages will not be printed
- Returns
list containing pairs of attribute name and value
-
static
dm.BeeeOnClient module¶
REST API client for communication with BeeeOn server.
-
class
dm.BeeeOnClient.
BeeeOnClient
(host, port)¶ Bases:
object
Client for communication with server supporting BeeeOn api.
-
property
api_key
¶
-
device_info
(gateway_id, device_id)¶ It gets information about device.
- Parameters
gateway_id – identifier of gateway
device_id – identifier of device
- Returns
json that contains information about device
-
history
(gateway, device, sensor, start, end, interval=1, aggregation='avg')¶ It gets historical data from server.
- Parameters
gateway – identifier of gateway
device – identifier of device
sensor – identifier of sensor
start – start of interval from which data is generated
end – end of interval from which data is generated
interval – time interval from which data are aggregated
aggregation – type of aggregation
- Returns
json that contains historical data from given time interval
-
refresh_token
()¶ It refreshes token.
- Returns
token id
-
sensors_info
(gateway_id, device_id)¶ It gets information about sensor.
- Parameters
gateway_id – identifier of gateway
device_id – identifier of device
- Returns
json that contains information about sensor
-
property
token_id
¶
-
property
dm.CSVUtil module¶
Creates a csv file from OrderedDict.
-
class
dm.CSVUtil.
CSVUtil
¶ Bases:
object
-
static
create_csv_file
(data: list, filename: str, enable_append=False)¶ It creates a CSV file.
- Parameters
data – list of pairs containing attribute name and its value
filename – output filename
enable_append – if it is true, data is appended to the end of file
- Returns
None
-
static
create_csv_io
(data: list)¶ It creates StringIO from given data.
- Parameters
data – list of pairs containing attribute name and its value
- Returns
StringIO containing given data
-
static
dm.ConnectionUtil module¶
Utils to create connections to db, REST client and utils for environment setting.
-
class
dm.ConnectionUtil.
ConnectionUtil
¶ Bases:
object
-
MAX_TESTABLE_EVENTS
= 75¶
-
static
ant_work_api_key
(server_type, config_file='/etc/dp/config.ini')¶ It gets information about api key from configuration file.
- Parameters
server_type – type of server - dm
config_file – path to configuration file
- Returns
information about api key
-
static
create_con
(config_file='/etc/dp/config.ini')¶ It creates connection to the database.
- Parameters
config_file – path to the configuration file
- Returns
connection to the database
-
static
estimate
(key, config_file='/etc/dp/config.ini')¶ It gets information about estimation from configuration file.
- Parameters
key – key for obtaining of data from information about estimation
config_file – path to configuration file
- Returns
information about estimation obtained using the key
-
static
get_value
(key, config_file)¶ It gets value using given key from configuration file.
- Parameters
key – key for obtaining of data from information
config_file – path to the configuration file
- Returns
required data obtained using the key
-
static
is_testable_system
()¶ It gets information if system is travis for testing.
- Returns
true if system is travis for testing, otherwise false
-
static
open_detector
(key, config_file='/etc/dp/config.ini')¶ It gets information about open detector from configuration file.
- Parameters
key – key for obtaining of data from information about open detector
config_file – path to configuration file
- Returns
information about open detector obtained using the key
-
static
open_weather_api_key
(config_file='/etc/dp/config.ini')¶ It gets information about OpenWeather.org api key from configuration file.
- Parameters
config_file – path to configuration file
- Returns
information about api key
-
static
package
(key, config_file='/etc/dp/config.ini')¶ It gets information about package from configuration file.
- Parameters
key – key for obtaining of data from information about package
config_file – path to configuration file
- Returns
information about package obtained using the key
-
static
predictor
(key, config_file='/etc/dp/config.ini')¶ It gets information about OpenWeather.org api key from configuration file.
- Parameters
key – key for obtaining of data from information about predictor
config_file – path to configuration file
- Returns
information about predictor obtained using the key
-
static
rapid_miner
(config_file='/etc/dp/config.ini')¶ It gets information about RapidMiner.
- Parameters
config_file – path to the configuration file
- Returns
information about RapidMiner
-
static
rehivetech_api_key
(server_type, config_file='/etc/dp/config.ini')¶ It gets information about api key from configuration file.
- Parameters
server_type – type of server - dm or acontroller
config_file – path to configuration file
- Returns
information about api key
-
static
setup_clients
()¶ It sets clients used for communication with servers.
- Returns
dictionary of set clients
-
static
setup_clients_logout
(cls)¶ It log outs clients.
- Parameters
cls – list of clients for communication with server
- Returns
None
-
static
setup_logging
(log_level=10)¶ It sets loggers for logging.
- Parameters
log_level – type of logging level
- Returns
None
-
static
wunderground_api_key
(config_file='/etc/dp/config.ini')¶ It gets information about WunderGround.com api key from configuration file.
- Parameters
config_file – path to configuration file
- Returns
information about api key
-
dm.DBUtil module¶
Defines functions for working with database.
-
class
dm.DBUtil.
DBUtil
¶ Bases:
object
-
static
check_timestamp_order
(con, table_name)¶ It checks if a given table in database does not contain missing value.
- Parameters
con – connection to the database
table_name – name of table
- Returns
None
-
static
create_table
(conn, table_name)¶ It creates table containing measured values.
http://www.mysqltutorial.org/mysql-decimal/ :param conn: connection to the database :param table_name: name of table :return: None
-
static
delete_from_time
(con, table_name, delay)¶ It removes several last rows from a given table.
- Parameters
con – connection to the database
table_name – name of table
delay – value used to define size of window from which rows are removed
- Returns
None
-
static
drop_table
(conn, table_name)¶ It removes a given table.
- Parameters
conn – connection to the database
table_name – name of table
- Returns
None
-
static
first_inserted_values
(conn, table_name)¶ It gets the first inserted value into given table.
- Parameters
conn – connection to the database
table_name – name of table
- Returns
first inserted value
-
static
insert_value
(conn, task, enable_commit, table_name)¶ It inserts values into the table.
- Parameters
conn – connection to the database
task – list of pairs that contain name of column and value
enable_commit – if true, result of a query is written into the database
table_name – name of table
- Returns
None
-
static
last_inserted_open_close_state
(conn, table_name)¶ It gets the last inserted state of window (open or closed).
- Parameters
conn – connection to the database
table_name – name of table
- Returns
last inserted window state
-
static
last_inserted_values
(conn, table_name)¶ It gets the last inserted value into given table.
- Parameters
conn – connection to the database
table_name – name of table
- Returns
last inserted value
-
static
measured_values_table_column_names
()¶ It return list of available column names in table.
- Returns
list of column names
-
static
rows_count
(conn, table_name)¶ It gets number of rows in given table.
- Parameters
conn – connection to the database
table_name – name of table
- Returns
number of rows in the table
-
static
update_attribute
(con, table_name, attribute, value, timestamp)¶ It updates value of a given attribute in given time.
- Parameters
con – connection to the database
table_name – name of table
attribute – name of attribute
value – new value
timestamp – time that denotes a row that will be updated
- Returns
None
-
static
dm.DateTimeUtil module¶
Converts various time formats (one to another).
-
class
dm.DateTimeUtil.
DateTimeUtil
¶ Bases:
object
-
static
create_interval_str
(start, end)¶ It converts given interval expressed using UTC timestamp to string form.
- Parameters
start – start of interval
end – end of interval
- Returns
interval expressed in string form
-
static
local_time_str_to_utc
(date_str, timezone='Europe/Prague', format='%Y/%m/%d %H:%M:%S')¶ It converts given local time in string form to UTC form (number).
- Parameters
date_str – local time in string form
timezone – timezone of local time
format – output format of time
- Returns
object containing local time in UTC form
-
static
utc_timestamp_to_local_time
(timestamp, timezone='Europe/Prague')¶ It converts given UTC timestamp to local time according to timezone.
- Parameters
timestamp – UTC timestamp
timezone – timezone of local time
- Returns
local time
-
static
utc_timestamp_to_str
(timestamp, format='%Y-%m-%d %H:%M:%S')¶ It converts UTC timestamp to local time in string form.
- Parameters
timestamp – UTC timestamp
format – output format of time
- Returns
local time in string form
-
static
dm.Differences module¶
dm.ExampleRunner module¶
Runs detector or more detectors.
-
class
dm.ExampleRunner.
ExampleRunner
¶ Bases:
object
-
static
detector
(before, after, script_name, process, enable_training)¶ It runs a process in RapidMiner.
- Parameters
before – time shift before start of event
after – time shift after start of event
script_name – name of script
process – name of RapidMiner process
enable_training – if training phase should be performed
- Returns
None
-
static
feature_stats
(script_name, enable_training)¶ It runs computing of statistics.
- Parameters
script_name – name of script
enable_training – if training phase should be performed
- Returns
None
-
static
dm.FilterUtil module¶
Defines various filters to select data that meets a required condition.
-
class
dm.FilterUtil.
FilterUtil
¶ Bases:
object
-
static
attribute
(events, attribute_name, value)¶ It gets events in which attribute contains given value.
- Parameters
events – list of events
attribute_name – name of attribute
value – required value of attribute
- Returns
list of events in which attribute contains given value
-
static
attribute_exclude
(events, attribute_name, value)¶ It gets events in which attribute does not contain given value.
- Parameters
events – list of events
attribute_name – name of attribute
value – not required value of attribute
- Returns
list of events in which attribute does not contain given value
-
static
derivation_not_zero
(events)¶ It gets events in which derivation is not zero.
- Parameters
events – list of events
- Returns
list of events in which derivation is not zero
-
static
humidity
(events, min_out_specific_humidity, min_diff, max_diff)¶ It gets events that fulfill the conditions.
It gets events in which specific humidity difference was in admissible range and outside specific humidity was not higher than given value.
- Parameters
events – list of events
min_out_specific_humidity –
min_diff –
max_diff –
- Returns
list of events
-
static
measured_values_not_empty
(events, attributes)¶ It gets events that include valid measured data in given attributes.
- Parameters
events – list of events
attributes – names of attributes
- Returns
list of events that include valid measured data in given attributes
-
static
min_length
(events, length)¶ It gets events that lasted at least given time.
- Parameters
events – list of events
length – minimal length of event
- Returns
list of events that lasted at least given time
-
static
min_max_time_interval
(events, min_time, max_time)¶ It gets events that lasted admissible time.
- Parameters
events – list of events
min_time – minimal length of event
max_time – maximal length of event
- Returns
list of events that lasted admissible time
-
static
min_time_interval
(events, time_interval_length)¶ It gets events that lasted at least the required time.
- Parameters
events – list of events
time_interval_length – length of time interval
- Returns
list of events that lasted at least the required time
-
static
min_timestamp
(events, timestamp)¶ It gets events that started after given time.
- Parameters
events – list of events
timestamp – timestamp from which events are considered
- Returns
list of events that started after given time.
-
static
only_valid_events
(events)¶ It gets only valid events.
- Parameters
events – list of events
- Returns
list of valid events
-
static
temperature_diff
(events, min_value, max_value)¶ It gets events in which temperature difference was in admissible range.
- Parameters
events – list of events
min_value – minimal difference of temperature
max_value – maximal difference of temperature
- Returns
list of events in which temperature difference was in admissible range
-
static
temperature_out_max
(events, max_value)¶ It gets events in which temperature was not higher than given value.
- Parameters
events – list of events
max_value – maximal temperature
- Returns
list of events in which temperature was not higher than given value
-
static
dm.Graph module¶
Utils to create graph that can be available in html file.
-
class
dm.Graph.
Graph
(path)¶ Bases:
object
-
static
db_to_simple_graph
(event, column, color, label, number_output_records)¶ It generates a graph based on data in a database.
- Parameters
event – an event
column – name of column (attribute)
color – color of curve
label – label for x-axis
number_output_records – number of time points in a graph
- Returns
graph containing course of a given attribute
-
gen
(data, output, scale_padding_min=0, scale_padding_max=0, g_type='line', min_value=None, max_value=None, global_range=False)¶ It generates a HTML page containing a graph.
- Parameters
data – data containing information about events
output – filename
scale_padding_min – padding calculated using the minimal value of an attribute
scale_padding_max – padding calculated using the maximal value of an attribute
g_type – shape of a curve
min_value – minimal value of an attribute in a graph
max_value – maximal value of an attribute in a graph
global_range – if true, range is calculated for each graph from given data, otherwise the same range is used for all graphs
- Returns
None
-
static
dm.GraphUtil module¶
Generates histogram, stacked barplot and grouped barplot.
-
class
dm.GraphUtil.
GraphUtil
¶ Bases:
object
-
static
gen_duration_histogram
(events, action, extensions, title, intervals, threshold)¶ It generates histogram of ventilation lengths.
- Parameters
events – list of events
action – show|save - for histogram showing or saving
extensions – list of extensions used in case of action ‘save’
title – graph title
intervals – list of intervals in minutes for which number of events is calculated
threshold – value that is added to the interval in minutes or is subtracted from the interval to create value interval for a given column in a histogram
- Returns
None
-
static
gen_grouped_barplot
(first_col, second_col, third_col, action, filename)¶ It generates grouped bar chart.
- Parameters
first_col – deprecated
second_col – data (values of an attribute) used to plot graph
third_col – data (values of an attribute) used to plot graph
action – show|save - for graph showing or saving
filename – output filename
- Returns
None
-
static
gen_stacked_barplot
(first_col, second_col, third_col)¶ It generates stacked bar graph.
- Parameters
first_col – data (values of an attribute) used to plot graph
second_col – data (values of an attribute) used to plot graph
third_col – data (values of an attribute) used to plot graph
- Returns
None
-
static
dm.HTTPClient module¶
REST api client with Bearer authentization.
-
class
dm.HTTPClient.
HTTPClient
(host, port, end_point, method, verify_ssl_cert=True)¶ Bases:
object
Simple HTTP client.
It sets authentization.
- Parameters
session_id – session identifier
- Returns
None
-
body
(body)¶ It sets HTTP body.
- Parameters
body – HTTP body
- Returns
None
-
perform
()¶ It performs HTTP request.
It creates HTTP connection, performs HTTP request, processes HTTP response and closes the connection.
- Returns
HTTP response and its content
dm.HeatMap module¶
Graph representing heat map.
-
class
dm.HeatMap.
HeatMap
¶ Bases:
object
-
static
annotate_heatmap
(im, data=None, valfmt='{x:.2f}', textcolors=['black', 'white'], threshold=None, **textkw)¶ It annotates a heatmap.
- Parameters
im – the AxesImage to be labeled
data – data used to annotate; ff None, the image’s data is used
valfmt – the format of the annotations inside the heatmap; this should either use the string format method, e.g. “$ {x:.2f}”, or be a
matplotlib.ticker.Formatter
textcolors – a list or array of two color specifications; the first is used for values below a threshold, the second for those above
threshold – value in data units according to which the colors from textcolors are applied; if None (the default), the middle of the colormap is used as separation
**textkw –
- Returns
-
static
heatmap
(data, row_labels, col_labels, ax=None, cbar_kw={}, cbarlabel='', **kwargs)¶ It create a heatmap from a numpy array and two lists of labels.
- Parameters
data – a 2D numpy array of shape (N, M)
row_labels – a list or array of length N with the labels for the rows
col_labels – a list or array of length M with the labels for the columns
ax –
- a matplotlib.axes.Axes instance to which the heatmap
is plotted; if it is not provided, current axes are used or
a new one is created
cbar_kw – a dictionary with arguments to
matplotlib.Figure.colorbar()
.cbarlabel – the label for the colorbar
**kwargs –
- Returns
-
static
dm.OpenWeatherOrg module¶
Gets information about weather from OpenWeather.org.
-
class
dm.OpenWeatherOrg.
OpenWeatherOrg
(precision=2)¶ Bases:
object
-
property
api_key
¶
-
weather_by_city_id
(quantities, start, end, city_id: int)¶ It gets weather from city based on its identifier during certain time interval.
- Parameters
quantities – list of required quantities
start – timestamp that denotes start of time interval
end – timestamp that denotes end of time interval
city_id – identifier of city
- Returns
weather data
-
weather_by_city_name
(quantities, start, end, name)¶ It gets weather from a given city during certain time interval.
- Parameters
quantities – list of required quantities
start – timestamp that denotes start of time interval
end – timestamp that denotes end of time interval
name – name of city
- Returns
weather data
-
weather_by_coordinates
(quantities, start, end, lat, lon)¶ It gets weather from city based on its coordinates during certain time interval.
- Parameters
quantities – list of required quantities
start – timestamp that denotes start of time interval
end – timestamp that denotes end of time interval
lat – latitude
lon – longitude
- Returns
weather data
-
property
dm.Performance module¶
Calculates accuracy using standard approach and accuracy considering a tolerance interval.
-
class
dm.Performance.
Performance
(filename)¶ Bases:
object
-
simple
()¶ It calculates standard accuracy.
- Returns
confusion matrix including number of events and accuracy; list of dates and times of events when a window was closed but window opening was detected (false positive); structure containing information from confusion matrix and sum of false positives, false negatives, true positives and true negatives
-
with_delay
(before, after)¶ It calculates accuracy considering a tolerance interval.
- Parameters
before – number of seconds before start of an event
after – number of seconds after start of an event
- Returns
confusion matrix including number of events and accuracy; list of dates and times of events when a window was closed but window opening was detected (false positive); structure containing information from confusion matrix and sum of false positives, false negatives, true positives and true negatives
-
-
dm.Performance.
count
(z, val)¶ Deprecated.
dm.PreProcessing module¶
Utils for date pre-processing.
-
class
dm.PreProcessing.
PreProcessing
¶ Bases:
object
-
OPEN_CLOSE_ATTR_NAME
= 'open_close'¶
-
TIME_ATTR_NAME
= 'measured_time'¶
-
TIME_STRING_ATTR_NAME
= 'measured_time_str'¶
-
static
check_start_end_interval
(items: list, time_attribute: str) → None¶ It checks if lists of values have the same starting and end time.
- Parameters
items – list of values
time_attribute – name of attributes that contains timestamp
- Returns
None
-
static
cut_interval
(items: list, start: int, end: int, time_attribute: str) → list¶ It cuts data to have the same start and end of an interval (starting and end time).
- Parameters
items – list of values
start – timestamp from which data is cut
end – timestamp to which data is cut
time_attribute – shift of start and end of time intervals
- Returns
cut list of values
-
static
db_name_maps
(devices: list) → list¶ It gets a list of column names in database that is loaded from list of devices.
A column name is loaded from a file to assign a given value to the right column in database.
- Parameters
devices – list of information about devices
- Returns
list of column names
-
static
download_data
(clients: list, devices: list, start: int, end: int) → list¶ It downloads required data according to a list of devices.
- Parameters
clients – list of clients
devices – list of information about devices
start – timestamp from which data is downloaded
end – timestamp to which data is downloaded
- Returns
list of downloaded data
-
static
extract_items
(data, value_column)¶ It prepares data.
- Parameters
data – dictionary of data
value_column – name of column
- Returns
list of values and list of timestamps
-
static
generate_data
(values: list, value_attribute: str, time_attribute: str, precision: int = 7) → list¶ It modifies downloaded data so that an interval between two adjacent records was one second.
It modifies the data using linearization, it does not consider quantities or their course.
- Parameters
values – list of values
value_attribute – name of attribute that contains data
time_attribute – ame of attribute that contains timestamp
precision – precision of calculation
- Returns
list of values that contains modified downloaded data (with step of one second)
-
static
generate_open_close
(values: list, time_attribute_name: str, open_close_attribute_name: str, start: int, end: int, last_open_close_state: int) → list¶ It modifies downloaded data so that an interval between two adjacent records was one second.
It modifies the data using linearization, it does not consider quantities or their course.
- Parameters
values – list of values
time_attribute_name – name of attribute that contains timestamp
open_close_attribute_name – name of attribute that contains state of window (open/closed)
start – timestamp that denotes start of an interval
end – timestamp that denotes end of an interval
last_open_close_state – last recorded state of a window
- Returns
list of values that contains modified downloaded data (with step of one second)
-
static
insert_values
(conn, table_name, values, maps, write_each, precision)¶ It inserts values to a table.
- Parameters
conn – connection to a database
table_name – name of table
values – data related to event
maps – list of column names
write_each – number of events - 1 between two adjacent events that are written to a table
precision – precision of inserted values
- Returns
None
-
static
join_items
(items: list, time_attribute: str) → list¶ It joins several lists of values into one list.
The resulting list contains only one attribute that includes time because times are the same. It contains various values that were included in various lists.
THe method has to be called only if passed list of values was checked using the method check_start_end_interval().
- Parameters
items – list of values
time_attribute – name of attribute that contains timestamp
- Returns
list of values including all attributes and time
-
static
ppm_filter
(data, ppm_limit=2000)¶ It sets carbon dioxide concentration to None if its value is out of range.
- Parameters
data – dictionary of data
ppm_limit – maximal admissible value of the concentration
- Returns
dictionary of data where excessive concentration is set to None
-
static
prepare
(clients: list, devices: list, start: int, end: int, last_open_close_state: int, time_shift: int)¶ It prepares data.
- Parameters
clients – list of clients
devices – list of information about devices
start – timestamp from which data is downloaded
end – timestamp to which data is downloaded
last_open_close_state – last recorded state of a window
time_shift – time shift that is subtracted or added to start or end of an interval respectively.
- Returns
list of attribute names and list of values
-
static
prepare_downloaded_data
(clients: list, devices: list, start: int, end: int, time_shift: int, last_open_close_state) → list¶ It prepares downloaded data.
- Parameters
clients – list of clients
devices – list of information about devices
start – timestamp that denotes start of an interval
end – timestamp that denotes end of an interval
time_shift – time shift that is subtracted or added to start or end of an interval respectively
last_open_close_state – last recorded state of a window
- Returns
dictionary of prepared data
-
static
prepare_value_conversion
(value)¶ It converts relative humidity to absolute and specific humidity.
- Parameters
value – data related to an event
- Returns
data related to an event including absolute and specific humidity
-
static
rename_all_attributes
(items: list, devices: list) → list¶ It renames all attributes in a list that contains list of values.
It renames individual attributes that is related to a given module. Each downloaded value from list of values with a defined name of attributes by default. New names of attributes are derived from list of devices.
- Parameters
items – list of values
devices – list of information about devices
- Returns
list that contains a modified list of values including renamed attributes
-
static
rename_attribute
(values: list, old_attribute: str, new_attribute: str) → list¶ It renames an attribute in data from an original name to a new name.
- Parameters
values – list of values
old_attribute – name of an original attribute
new_attribute – name of a new attribute
- Returns
list of pairs of a renamed attribute and its value
-
static
value_filter
(data)¶ It checks if indoor relative humidity is in an admissible range.
If indoor relative humidity is out of range, it is logged.
- Parameters
data – dictionary of data
- Returns
dictionary of data
-
dm.SQLUtil module¶
Utils that contain simple SQL queries.
-
class
dm.SQLUtil.
SQLUtil
¶ Bases:
object
-
static
select_interval
(table_name: str, start: int, end: int, columns: str)¶ It selects given columns from a table on the basis of given time interval.
- Parameters
table_name – name of table
start – timestamp that denotes start of a time interval
end – timestamp that denotes end of a time interval
columns – names of columns
- Returns
SQL query
-
static
select_interval_size
(table_name: str, start: int, end: int, column: str)¶ It finds out number of records on the basis of given time interval.
- Parameters
table_name – name of table
start – timestamp that denotes start of a time interval
end – timestamp that denotes end of a time interval
column – name of column
- Returns
SQL query
-
static
select_one_value
(table_name: str, measured_time: int, columns: str)¶ It selects one record from a table on the basis of given timestamp.
- Parameters
table_name – name of table
measured_time – timestamp that is used to select a record
columns – list of columns that are selected
- Returns
SQL query
-
static
dm.Storage module¶
Storage that supports selection of events and their values or only selection of the values.
-
class
dm.Storage.
Storage
(filename: str, no_event_time_shift: int, table_name: str)¶ Bases:
object
-
static
dw_columns_ordered
(con, start, end, columns, table_name)¶ It gets several rows (interval) from a table and orders them.
- Parameters
con – connection to a database
start – timestamp that denotes start of a time interval
end – timestamp that denotes end of a time interval
columns – list of columns
table_name – name of table
- Returns
ordered rows (interval)
-
load_data
(con, start_shift: int, end_shift: int, column: str)¶ It loads information about events. :param con: connection to a database :param start_shift: number of seconds added to start of event :param end_shift: number of seconds added to end of event :param column: name of column :return: dictionary of data
-
static
one_row
(con, table_name: str, columns: str, timestamp: int)¶ It gets one row (required columns) from a table.
- Parameters
con – connection to a database
table_name – name of table
columns – list of columns
timestamp – timestamp that determines a required row
- Returns
one row (required columns)
-
read_meta
()¶ It reads information about events.
- Returns
list of attributes describing events
-
static
select_interval
(con, start, end, column, table_name, without_none_value=True)¶ It gets several rows (interval) from a table.
- Parameters
con – connection to a database
start – timestamp that denotes start of a time interval
end – timestamp that denotes end of a time interval
column – name of column
table_name – name of table
without_none_value – true if the result cannot contain null values
- Returns
several rows (interval)
-
static
dm.ValueConversionUtil module¶
Converts relative humidity to absolute humidity, relative humidity to specific humidity and ppm to milligrams per cubic meter.
-
class
dm.ValueConversionUtil.
ValueConversionUtil
¶ Bases:
object
-
CO_MOLECULAR_WEIGHT
= 44.0095¶
-
static
ah_to_relative_percent
(temp: float, ah: float) → float¶ It converts absolute humidity to relative humidity.
- Parameters
temp – temperature
ah – absolute humidity
- Returns
relative humidity
-
static
co2_ppm_to_mg_m3
(co2)¶ It converts ppm to milligrams per cubic meter.
- Parameters
co2 – ppm
- Returns
milligrams per cubic meter
-
static
rh_to_absolute_g_m3
(temp: float, rh: float) → float¶ It converts relative humidity to absolute humidity.
- Parameters
temp – temperature
rh – relative humidity
- Returns
absolute humidity
-
static
rh_to_specific_g_kg
(temp: float, rh: float) → float¶ It converts relative humidity to specific humidity.
- Parameters
temp – temperature
rh – relative humidity
- Returns
specific humidity
-
dm.ValueUtil module¶
Utils for detection of simple events.
-
class
dm.ValueUtil.
ValueUtil
¶ Bases:
object
-
static
delays
(events, delays_attr_name)¶ It gets list of delays.
- Parameters
events – list of events
delays_attr_name – name of attribute that denotes delay
- Returns
list of delays
-
static
detect_sensor_delay
(values, window_size, threshold)¶ It detects sensor delay.
- Parameters
values – list of value
window_size – size of a window in seconds
threshold – threshold in seconds
- Returns
index of value when a decrease occurred
-
static
detect_window_action
(values_count: int, actual_index: int)¶ It detects a window action.
- Parameters
values_count – count of values
actual_index – actual index
- Returns
window action
-
static
events_duration
(events, max_duration)¶ It gets durations of events that lasted at most given time.
- Parameters
events – dictionary of events
max_duration – maximal duration of event
- Returns
list of durations of events
-
static
window_event_value
(measured: dict, value_index: int, timestamp: int, precision: int)¶ It gets information about event when a window was open.
- Parameters
measured – dictionary of measured values
value_index – index to a dictionary
timestamp – timestamp when an event occurred
precision – precision of output values
- Returns
information about event when a window was open
-
static
window_no_event_value
(values: tuple, precision: int)¶ It gets information about event when a window was not open.
- Parameters
values – measured values of quantities
precision – precision of output values
- Returns
information about event when a window was not open
-
static
dm.WundergroundCom module¶
Weather data extraction from weather.com.
-
class
dm.WundergroundCom.
WundergroundCom
(precision=1)¶ Bases:
object
-
actual_weather
(lat, lon)¶ It gets information about current weather in a locality.
- Parameters
lat – latitude of a locality
lon – longitude of a locality
- Returns
information about current weather
-
property
api_key
¶
-
weather_by_coordinates
(quantities, start, end, lat, lon)¶ It gets information about weather in a locality at required time interval.
- Parameters
quantities – deprecated
start – timestamp that denotes start of time interval
end – timestamp that denotes end of time interval
lat – latitude of a locality
lon – longitude of a locality
- Returns
information about weather
-