IoT devices within a commercial real-estate or residential building environment may be connected through networks, such as a building Automation and Control network (BACnet). systems and methods according to this disclosure provide automatic discovery of IoT devices and relationships in commercial real-estate and residential buildings and integration of the BACnet devices into the digital twin of the building. In some implementations, an IoT gateway is configured to translate the communication received from the BACnet to an IoT cloud platform and configured to normalize the data across the different security platforms into a consistent format which enables integration and interoperability of the different building system platforms that may otherwise be operating in isolation from each other. systems and methods according to the present disclosure provide edge based analytics and control of IoT BACnet devices based on defining conditions and rules, and provide integration of multiple building systems in the context of commercial real-estate and residential buildings.
|
16. A system for generating a contextually-aware digital twin of a first commercial real estate building comprising:
one or more processors; and
a memory storing computer-readable instructions that, when executed by the one or more processors, configure the one or more processors to:
generate information about relationships for devices based on training a machine learning algorithm, wherein the relationships are between physical entities of the first commercial real estate building;
insert the information about the relationships into a device library;
receive device data about an asset comprising one or more devices residing in one or more physical spaces associated with the first commercial real estate building;
store the device data about the one or more devices in the device library;
discover, based on a received identification of a discovered device connected to an associated network, whether the discovered device is stored in a graph representation of the first commercial real estate building;
access the device data from the device library, wherein the device data includes the information about the relationship;
when the device data from the device library is known about the discovered device,
identify device information about the discovered device from the accessed device data;
predict at least one relationship associated with the discovered device;
in response to the predicting, create one or more related entities and relationships associated with the discovered device; and
in response to the creating, automatically set up one or more nodes and relationships in the a contextually-aware digital twin based on the created one or more related entities and relationships; and
when the discovered device is not stored in the graph representation, modify the graph representation to include the discovered device with the identified device information.
1. A method of generating a contextually-aware digital twin of a first commercial real estate building comprising:
generating information about relationships for devices based on training a machine learning algorithm, wherein the relationships are between physical entities of the first commercial real estate building;
inserting the information about the relationships into a device library via a computing connectivity link;
receiving, by a computing device, device data about an asset comprising one or more devices residing in one or more physical spaces associated with the first commercial real estate building via a building communication link, wherein the building communication link is separate from the computing connectivity link;
storing, by the computing device, the device data about the one or more devices in the device library;
discovering, based on a received identification of a discovered device connected to an associated network, whether the discovered device is stored in a graph representation of the first commercial real estate building;
accessing the device data from the device library, wherein the device data includes the information about the relationships;
when the device data from the device library is known about the discovered device,
identifying device information about the discovered device from the accessed device data;
predicting at least one relationship associated with the discovered device;
in response to the predicting, automatically creating one or more related entities and one or more relationships associated with the discovered device; and
in response to the creating, automatically setting up one or more nodes and relationships of an asset in the a contextually-aware digital twin based on the created one or more related entities and relationships; and
when the discovered device is not stored in the graph representation, modifying the graph representation to include the discovered device with the identified device information.
21. A non-transitory machine-readable storage medium comprising machine-readable instructions for causing a processor to execute a method of generating a contextually-aware digital twin of a first commercial real estate building comprising:
generating information about relationships for devices based on training a machine learning algorithm, wherein the relationships are between physical entities of the first commercial real estate building;
inserting the information about the relationships into a device library;
receiving device data about an asset comprising one or more devices residing in one or more physical spaces associated with the first commercial real estate building;
storing the device data about the one or more devices in the device library;
discovering, based on a received identification of a discovered device connected to an associated network, whether the discovered device is stored in a graph representation of the first commercial real estate building;
accessing the device data from the device library, wherein the device data includes the information about the relationships;
when the device data from the device library is known about the discovered device, identifying device information about the discovered device from the accessed device data;
when the device data from the device library is known about the discovered device, predicting at least one relationship associated with the discovered device;
when the device data from the device library is known about the discovered device, in response to the predicting, creating one or more related entities and relationships associated with the discovered device;
when the device data from the device library is known about the discovered device, in response to the creating, automatically setting up one or more nodes and relationships in the a contextually-aware digital twin based on the created one or more related entities and relationships; and
when the discovered device is not stored in the graph representation, modifying the graph representation to include the discovered device with the identified device information.
2. The method of
3. The method of
including manufacturer information in the stored device data at the device library.
4. The method of
including model information in the stored device data at the device library.
5. The method of
obtaining a manufacturer identification of the discovered device; and
accessing the device information based on the manufacturer identification.
6. The method of
obtaining a model identification of the discovered device; and
accessing the device information based on the model identification.
7. The method of
determining at least one controller associated with at least one or more points of the discovered device.
8. The method of
training the machine learning algorithm with the received device data on the contextually-aware digital twin.
9. The method of
determining one or more points based on the device information.
10. The method of
11. The method of
12. The method of
executing the machine learning algorithm on the contextually-aware digital twin of the first commercial real estate building to provide a recommendation for a second commercial real estate building, wherein the first commercial real estate building and the second commercial real estate building are different.
13. The method of
collecting data across a plurality of commercial real estate buildings; and
training the machine learning algorithm on the collected data.
14. The method of
detecting at least one pattern from the collected data; and
generating a recommendation for a third commercial real estate building based on the detected at least one pattern.
15. The method of
17. The system of
predict a relationship between the discovered device and another entity of the graph representation of the first commercial real estate building; and
determine at least one controller associated with at least one or more points of the discovered device.
18. The system of
include manufacturer and model information in the stored device data at the device library;
obtain a manufacturer and model identification of the discovered device; and
access the device information based on the manufacturer and model identification.
19. The system of
train the machine learning algorithm with the received device data on the contextually-aware digital twin.
20. The system of
determine one or more points based on the device information.
22. The non-transitory machine-readable storage medium of
predicting a relationship between the discovered device and another entity of the graph representation of the first commercial real estate building; and
determining at least one controller associated with at least one or more points of the discovered device.
23. The non-transitory machine-readable storage medium of
including manufacturer and model information in the stored device data at the device library;
obtaining a manufacturer and model identification of the discovered device; and
accessing the device information based on the manufacturer and model identification.
24. The non-transitory machine-readable storage medium of
training the machine learning algorithm with the received device data on the contextually-aware digital twin.
|
|||||||||||||||||||||||||||
This application is a continuation of U.S. patent application Ser. No. 17/027,070 filed Sep. 21, 2020, which is a continuation of U.S. patent application Ser. No. 16/456,525 filed Jun. 28, 2019. The present application claims priority to and the benefit of the above-identified applications and the above-identified applications are incorporated by reference herein in their entirety.
Aspects described herein generally relate to building automation, IoT (Internet of Things) devices, and hardware and software related thereto. More specifically, one or more aspects describe herein provide intelligent commercial real-estate and residential building systems and automation.
IoT devices within a commercial real-estate building environment may be connected through networks with different wired or wireless communication protocols, such as Ethernet or Building Automation and Control network (BACnet), which is a wired communications protocol that allows the building automation and control systems to communicate back up to a network. The IoT devices within a commercial real-estate building environment may be connected to the Internet via an IoT Gateway. The IoT Gateway bridges IoT BACnet devices with the Internet by performing a communication protocol translation between the BACnet and Internet.
The following presents a simplified summary of various aspects described herein. This summary is not an extensive overview, and is not intended to identify required or critical elements or to delineate the scope of the claims. The following summary merely presents some concepts in a simplified form as an introductory prelude to the more detailed description provided below.
According to one aspect, the disclosure relates to a computer-implemented method of generating a contextually-aware digital twin of a commercial real estate or residential building. The method includes receiving information about an asset comprising one or more devices residing in one or more physical spaces associated with the commercial real-estate or residential building. The method includes determining based on an identification of the one or more devices of the asset, one or more points corresponding to a sensor or an actuator of the one or more devices. The method includes identifying, based on the one or more points, at least one controller that serves the asset via the one or more points. The at least one controller comprises a physical or logical entity. The method includes determining, one or more networks, and one or more locations associated with the asset. The method includes storing each of the asset, the one or more points, the at least one controller, the one or more networks, and the one or more locations as separate nodes of a graph representation of the commercial real estate building.
In some implementations, the method includes determining, based on an identification of a device connected to a network, whether the device is stored in the graph representation of the commercial real estate or residential building. In some implementations, the method includes, based on a determination that the device is not stored in the graph database of the commercial real estate building, identifying the device as a new device. In some implementations, the method includes determining one or more points corresponding to a sensor or an actuator of the new device. In some implementations, the method includes identifying, based on the one or more points of the new device, at least one controller, at least one location, and at least one network that is associated with the new device. In some implementations, the method includes storing the one or more points, the at least one controller, the at least one location, and the at least one network associated with the new device in the graph representation of the commercial real-estate building. In some implementations, the one or more devices of the asset are connected to a Building Automation Control (BAC) network. In some implementations, the one or more locations may include a physical or logical location associated with the one or more physical spaces of the commercial real estate building. The one or more locations comprise at least one of an identification of a floor, a longitude-latitude coordinate, a space, an elevation, a zone, a space zone, and an identification of an occupant of the one or more physical spaces. In some implementations, the graph representation of the building comprises a label property graph representation. In some implementations, the points comprise at least one of a setpoint value or a schedule. In some implementations, the identifying of the at least one controller associated with the new device is based on information generated by training a machine learning algorithm on one or more previously generated graph representations of the commercial real estate building.
According to certain aspects of the present disclosure, a system for generating a contextually-aware digital twin of a commercial real estate building is provided. The system includes one or more processors. The system includes a memory storing computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to receive information about an asset comprising one or more devices residing in one or more physical spaces associated with the commercial real estate building. The instructions cause the one or more processors to determine based on an identification of the one or more devices of the asset, one or more points corresponding to a sensor or an actuator of the one or more devices. The instructions cause the one or more processors to identify, based on the one or more points, at least one controller that serves the asset via the one or more points. The at least one controller is a physical or logical entity. The instructions cause the one or more processors to determine, one or more networks, and one or more locations associated with the asset. The instructions cause the one or more processors to store each of the asset, the one or more points, the at least one controller, the one or more networks, and the one or more locations as separate nodes of a graph representation of the commercial real estate building.
In some implementations, the instructions cause the one or more processors to, determine, based on an identification of a device connected to a network, whether the device is stored in the graph representation of the commercial real estate building. In some implementations, the instructions cause the one or more processors to, based on a determination that the device is not stored in the graph database of the commercial real estate building, identify the device as a new device. In some implementations, the instructions cause the one or more processors to determine one or more points corresponding to a sensor or an actuator of the new device. In some implementations, the instructions cause the one or more processors to identify, based on the one or more points of the new device, at least one controller, at least one location, and at least one network that is associated with the new device. In some implementations, the instructions cause the one or more processors to store the one or more points, the at least one controller, the at least one location, and the at least one network associated with the new device in the graph representation of the commercial real-estate building. In some implementations, the one or more devices of the asset are connected to a Building Automation Control (BAC) network. In some implementations, the one or more locations may include a physical or logical location associated with the one or more physical spaces of the commercial real estate building. The one or more locations comprise at least one of an identification of a floor, a longitude-latitude coordinate, a space, an elevation, a zone, a space zone, and an identification of an occupant of the one or more physical spaces. In some implementations, the graph representation of the building comprises a label property graph representation. In some implementations, the points comprise at least one of a setpoint value or a schedule. In some implementations, the identifying of the at least one controller associated with the new device is based on information generated by training a machine learning algorithm on one or more previously generated graph representations of the commercial real estate building.
According to certain aspects of the present disclosure, a non-transitory machine readable storage medium containing machine-readable instructions for causing a processor to execute a method of generating a contextually-aware digital twin of a commercial real estate building is provided. The instructions contained on the non-transitory machine readable storage medium cause the processor to perform a method that includes receiving information about an asset comprising one or more devices residing in one or more physical spaces associate with the commercial real estate building. The method includes determining based on an identification of the one or more devices of the asset, one or more points corresponding to a sensor or an actuator of the one or more devices. The method includes identifying, based on the one or more points, at least one controller that serves the asset via the one or more points. The at least one controller comprises a physical or logical entity. The method includes determining, one or more networks, and one or more locations associated with the asset. The method includes storing, each of the asset, the one or more points, the at least one controller, the one or more networks, and the one or more locations as separate nodes of a graph representation of the commercial real estate building.
These and additional aspects will be appreciated with the benefit of the disclosures discussed in further detail below.
A more complete understanding of aspects described herein and the advantages thereof may be acquired by referring to the following description in consideration of the accompanying drawings, in which like reference numbers indicate like features, and wherein:
In the following description of the various embodiments, reference is made to the accompanying drawings identified above and which form a part hereof, and in which is shown by way of illustration various embodiments in which aspects described herein may be practiced. It is to be understood that other embodiments may be utilized and structural and functional modifications may be made without departing from the scope described herein. Various aspects are capable of other embodiments and of being practiced or being carried out in various different ways.
It is to be understood that the phraseology and terminology used herein are for the purpose of description and should not be regarded as limiting. Rather, the phrases and terms used herein are to be given their broadest interpretation and meaning. The use of “including” and “comprising” and variations thereof is meant to encompass the items listed thereafter and equivalents thereof as well as additional items and equivalents thereof. The use of the terms “mounted,” “connected,” “coupled,” “positioned,” “engaged” and similar terms, is meant to include both direct and indirect mounting, connecting, coupling, positioning, and engaging.
One technique for creating digital twins of commercial real-estate buildings relies on modeling the relationships between different physical entities, such as the building, floors, suites, and spaces, as parent-child relationships which, however, do not accurately capture the many-to-many relationships between these entities. As an example, a variable air volume (VAV) box connected to a VAV controller may serve four different spaces within a building. In a parent-child relationship model, the VAV box is at the parent level, the VAV controller is a child of the VAV box, and the four spaces within the building are children of the VAV controller. As another example, there may be four VAV boxes serving a single open space. In this example, one parent-child representation may have the single open space at the parent level with the 4 VAV boxes as children. In another parent-child representation, each VAV box may be a parent node with the open space as its child node. Thus, modeling the relationships between different physical entities in a commercial building as parent-child relationships can become complex and convoluted. However, the many-to-many relationships between physical entities in a commercial building can be modeled and stored as a graph database. Some techniques use a RDF version of a graph database which stores the relationships between entities in subject-predicate-object format, and thus, cannot go as deep as, for example, a label property version of a graph database. Additionally, one technique does not represent controllers and devices as separate entities in the graph database.
IoT devices within a commercial real-estate building environment may be connected through networks with different wired or wireless communication protocols, such as Ethernet or Building Automation and Control network (BACnet) which is a wired communications protocol that allows the building automation and control systems to communicate back up to a network. The IoT devices within a commercial real-estate building environment may be connected to the Internet via an IoT Gateway. The IoT Gateway bridges IoT BACnet devices with the Internet by performing a communication protocol translation between the BACnet and Internet. An IoT Gateway may monitor BACnet devices by periodically polling these devices. While a pubset model has been developed so that IoT devices connected through a BACnet may publish in an asynchronous fashion, the pubset model is limited by the bandwidth of the BACnet, which has a specific baud rate and thus, there is only so much data that can be pumped onto the wire.
Some techniques use centralized queues for receiving data from devices or sending data to devices, which are a single point of failure, have bottlenecks, and do not prioritize messages well. Additionally, a centralized scheduler does not account for the device response time, the number of devices that can be queried within a time interval, or the aggregation of queries over time intervals (i.e. queries over 5 second, 10 second, 15 second intervals stacking on top of each other). Further, a centralized rules engine (i.e. if temperature exceeds value ‘y’, then change to mode value ‘z’) requires too much complexity to account for dynamic rules per device.
Systems and methods according to this disclosure provide a contextually-aware digital twin of a commercial building based on a graph database representation of entities and relationships that are important in the context of commercial buildings and spaces. Every commercial building is unique. Even though two commercial buildings may look alike, the connectivity between a device and its controller may be different between the buildings. Thus, the same device may have a different relationship to its controller in each building. Further, a device and its controller may be physically separate entities. A device such as a thermostat may be placed at a specific location because that may be the best location for the thermostat to accurately determine the temperature within a space. However, the thermostat may be communicating with a controller that is in a different location from the thermostat. As another example, a variable air volume (VAV) box is an essential part of an air conditioning system in any large industrial or commercial building. The air conditioning system is, in turn, part of a HVAC system, which encompasses heating, ventilation, air conditioning, and associated control systems. The VAV box may connect to a VAV controller that controls the opening and shutting of the VAV box dials in order to control the flow of air that is coming into a building. The VAV box may also include a heating element. If the air entering into the building is not at a high enough temperature, the VAV controller may turn on the heating element in the VAV box to raise the temperature of the air entering the building. In complex commercial buildings and structures, the VAV controller is not necessarily synonymous with the VAV box or device. Instead, the VAV box and the VAV controller may be physically separate entities. Additionally, the VAV box and the VAV controller may be manufactured by different entities. A customer may mix and match the parts from different manufacturers. Thus, modeling the VAV controller and VAV box as different entities in the digital twin of the building is also useful for inventory tracking. Therefore, decoupling the thermostat and its controller into separate entities in the graph database provides an accurate IoT model of the building resulting in accurate queries and analytics. Thus, implementations according to this disclosure decouple devices and controllers in terms of how the cloud platform interacts with them. Implementations may store document level data by using the label property graph version of the graph database which store all or some of the properties of the device and points, which may be the registers of a controller of a device. The digital twin of a building may reside in the cloud and may track every device that is supported by the platform. The digital twin may track points, assets, controllers and network and location, including service location and installed location. Digital twins of different buildings may be integrated. The graph database and actor model allows us to integrate with other digital twins like BIM, energy models, third-party APIs, third-party building software and applications, etc. Additionally, relationships between entities may be modeled based on an understanding of how the actual physical equipment gets connected. For example, a controller may be managed by another upstream controller, thus, the data model's representation of parent-child relationships can scale. Further, implementations according to this disclosure enable data from traditionally separate subsystems to be combined. Implementations enable search and filtering and navigation based on location, type, etc. (e.g. all assets serving a specific room—and query power meter, HVAC, lighting, etc.). An energy model is typically a static model that gets created upon initial design and uses approximations for how systems will perform with a defined set of inputs. In some implementations, an energy model may be calibrated for HVAC zones based on a representation of a building's digital twin. Implementations according to this disclosure may leverage machine learning in which the inputs may be varied to model how the building will perform in an infinite number of scenarios.
Systems and methods according to this disclosure provide automatic discovery of IoT devices and relationships in commercial real-estate buildings and integration of the devices into the digital twin of the building. Devices may be discovered on the network based on their IP addresses. For example, BACnet devices may be discovered via the “whois” command which returns the device identifiers of BACnet devices. For example, there may be multiple different manufacturers that make VAB boxes on the market and each manufacturer may manufacture several different models. This information may be recorded in a library of devices. When a new device is discovered on the network, it may be identified based on the information recorded in the device library. Additionally, the information in the library of devices may include relationships that may arise from an asset that includes a known device. The relationships for a device may be determined based on training a machine learning algorithm on the digital twins of various commercial real-estate buildings. If a device is known, then the relationships that come from that device may be predicted, enabling automatic creation of all or some of the related entities and relationships, thereby automatically setting up the nodes and relationships of an asset in the digital twin.
Systems and methods according to this disclosure provide an IoT Gateway configured to translate the communication received from the BACnet to an IoT cloud platform. Systems and methods according to this disclosure provide an IoT Gateway configured to provide normalization of big data generated by interconnected IoT in commercial real-estate buildings. A controller may have multiple registers in which the controller stores data. Each register may be identified as a separate point. However, different manufacturers may store specific types of data in the same register. Therefore, in order to enable seamless data collection and analytics, the data generated by the IoT devices and controllers may be normalized across various manufacturers and models before transmitting to a cloud platform for further processing, storage, and integration with other systems and platforms. The normalization may be performed at the API level. For example, a typical commercial real-estate building may comprise IoT devices and controllers related to multiple different security platforms that each store data in a specific manner or format. The IoT Gateway may be configured to normalize the data across the different security platforms into a consistent format which enables integration and interoperability of the different building system platforms that may otherwise be operating in isolation from each other. Additionally, the IoT Gateway may be configured to evaluate and filter the data received from the IoT devices before transmitting the data to a cloud platform for further processing and analytics, thus reducing the impact on network processing and bandwidth. Further, the IoT Gateway may be configured to perform edge based analytics and edge based control of IoT devices, including IoT BACnet devices, within a commercial real-estate building environment. In some implementations, the IoT Gateway may be configured to control the IoT devices based on evaluating incoming real-time data with rules. The IoT Gateway may be configured to perform queries and analytics based on edge computing which does not rely on network connectivity or the cloud computing resources. This enables real-time response to events without significant increase in bandwidth or processing power.
Systems and methods according to the present disclosure provide edge based analytics and control of IoT devices based on defining conditions and rules. For example, a rule may trigger an action based on receiving real-time data related to a supply air temperature that has exceeded a min/max alarm threshold. A windowing function may ensure that a specific number of consecutive real-time data points are received or observed before the rule is triggered. Once it is confirmed that the incoming data points sufficiently meet the relevant conditions set by the rule, automatic control of the related devices may be triggered. Representations of building systems and devices may be created using an actor model. In an actor model, a plurality of actors may receive message from other actors, perform computations, manage their state, and send messages to other actors. Actors may correspond to devices or groups of devices related to building systems, such as HVAC, lighting, etc. For example, a sensor may communicate a measurement, such as temperature, to an actor corresponding to the sensor. The measurement may be communicated to the actor as a message. Upon receiving the message, the actor updates the stored state based on the sensor measurement. The actor representing the sensor may also receive messages from other actors for access to the latest sensor measurement. The actor may also send messages to the physical sensor device to update the device configuration. An actor that triggers a rule may query connected devices and assets to find an optimum solution based on the current situation of the system. Once data related to various solutions is received, the best option may be determined based on pre-determined constraints and settings, such as energy usage, time required, etc. In some implementations, device analytics may be provided to model expected performance of a device based on a current state, and the device may be controlled based on computations in real-time using context aware data. By maintaining a stateful machine representation of the device, the exact behavior of a device may be modeled by training a data analytics machine learning algorithm on the real-time data of the input and output data points of the device. Implementations may provide precision command and control for an infinite number of stateful scenarios of the device instead of just a handful that are included in the device's onboard controller.
The actor model may be implemented by a distributed computing and non-blocking framework, such as Akka. The actor model and rules may be stored in a cloud platform. In addition to hosting an actor system in the cloud, implementations may also run the actor system at the edge which avoids reliance on cloud computing and enables better real-time response. The edge device analytics may be performed on an IoT gateway.
Systems and methods according to this disclosure provide integration of multiple building systems in the context of commercial real-estate buildings. Typically, there may be multiple different building software platforms supporting a commercial real-estate building that are isolated from each other. Implementations according to this disclosure provide integration of previously isolated software platforms and related building systems. Integration of various building systems enable discovery of new patterns and insights into the building systems. Integration of various building systems also enables the automation of interactions between devices across various building systems.
In addition to the devices 110a-e that are connected to the BACnet 112, the example building 102a includes a sixth device 110f and a seventh device 110g that are connected to a fourth controller 113d. The sixth and seventh devices 110f-g may communicate via an Ethernet protocol and may be referred to as “Ethernet devices.” The example building 102a includes an IoT Gateway 125 which serves as a connection point between the devices 110 and an external network, such as network 160. The BACnet devices 110a-d may connect to the IoT Gateway 125 via a BACnet Interface 112. The Ethernet devices may connect to the IoT Gateway via an Ethernet Interface 120. An eighth device 110h interfaces directly with the IoT Gateway 125. The devices 110 may leverage other networking protocols. The example building 102a may include additional devices 110 that may communicate via communications protocols, such as CoAP, DTLS MQTT, ModBus, AMQP, HTTP, HTTPS, and FTP, among others. The example building 102a may include additional devices 110 that may communicate via wireless protocols, such as IPv6, LPWAN, Zigbee, Bluetooth Low Energy, Z-Wave, RFID and NFC.
The IoT Gateway 125 may connect the devices 110 and the controllers 113 to the Intelligent Building Cloud Platform 165 over a network 160. The IoT Gateway 125 may be configured to bridge the BACnet devices 110 with the Internet by performing a communication translation between the BACnet and Internet communication protocols. The IoT Gateway 125 may communicate with an Intelligent Building Edge Application Platform 142. Additionally, the IoT Gateway 125 may be configured to evaluate and filter the data received from the IoT devices before transmitting the data to a cloud platform for further processing and analytics, thus reducing the impact on network processing and bandwidth. Further, the IoT Gateway 125 may be configured to perform edge based analytics and edge based control of IoT devices 110, including the BACnet devices 110, within the example building 102a. In some implementations, the IoT Gateway 125 may be configured to control the IoT devices 110 based on evaluating incoming real-time data with rules. The IoT Gateway 125 may be configured to perform queries and analytics based on edge computing which does not rely on network connectivity or the cloud computing resources. This enables real-time response to events without significant increase in bandwidth or processing power.
The Intelligent Building Edge Application Platform 142 include an Intelligent Building Edge Application Server 140 via a communication link 130. The Intelligent Building Edge Application Server 140 may receive data via the IoT Gateway 125 and send the data over the network 160 to the Intelligent Building Cloud Platform 165 for further processing and storage. The Intelligent Building Edge Application Server 140 may push one or more applications to configure the IoT Gateway 125 to perform various functions. Intelligent Building Edge Application Server 140 may be configured to normalize the data generated by the devices 110 and controller 113 may be normalized across various manufacturers and models before transmitting to a cloud platform, such as the Intelligent Building Cloud Application Platform 165, for further processing, storage, and integration with other systems and platforms. The normalization may be performed at the API level. The IoT Gateway 125 of different buildings may collect data specific for that building and send the collected data to the Intelligent Building Cloud Application Server which in turn process and analyze the data further for providing recommendations. For example, implementations may include training a machine learning algorithm on the data collected across various buildings to gain insights and patterns which may then in turn be analyzed for the purpose of providing recommendations on other buildings. Data from various buildings may be crowd sourced in such implementations. The Intelligent Building Edge Application Server 140 may be configured to create and manipulate a contextually-aware digital twin for each of the buildings 102 based on a graph database representation of entities and relationships that are important in the context of commercial buildings and spaces. The Intelligent Building Edge Application Server 140 may be configured to provide automatic discovery of devices located in the buildings 102, generate the various relationships arising from the discovered devices and integrate the discovered devices and relationships into the digital twins of the buildings 102.
The Intelligent Building Edge Application Server 140 may be configured to perform edge based analytics and control the devices 110. The Intelligent Building Edge Application Server 140 may be configured to generate and maintain an actor model comprising representations of building systems and devices. Actors may correspond to devices or groups of devices related to building systems, such as HVAC, lighting, etc. The Intelligent Building Edge Application Server 140 may store the generated digital twins and actor models of the buildings 102 in the Intelligent Building Cloud Platform 165. The Intelligent Building Edge Application Server 140 may also store the digital twins and the actor models of the buildings 102 at the edge as Building Edge Data 145. The Intelligent Building Edge Application Server 140 may be configured provide integration of multiple software platforms related to the building systems of the buildings 102 and to enable discovery of new patterns and insights into the building systems. The integration may enable the automation of interactions between the devices 110 across various building systems of the building 102a and across the buildings 102.
An example Building Edge Data 145a includes an Edge Digital Twin 150 and an Edge Actor Model 155. The Intelligent Building Edge Application Server 140 may use the Building Edge Data 145 for performing edge analytics and processing. The Intelligent Building Cloud Platform 165 includes an Intelligent Building Cloud Application Server 170 that may be configured to perform cloud analytics and processing. The Intelligent Building Cloud Platform 165 includes a Device Library 190 and Building Data 175. The Building Data 175 may store various information about the buildings 102. An example Building Data 175a includes a Historical Data 185, a Digital Twin 180, and an Actor Model 182. The Historical Data 185 may include data about one of the buildings 102 that is collected over a period of time. The Intelligent Building Cloud Application Server 170 may access the Historical Data 185, the Digital Twin 180, and the Actor Model 182 in order to perform analytics that provide additional insights into the devices and systems of the corresponding building 102a. The Intelligent Building Cloud Application Server 170 may store information about devices and controllers in the Device Library 190. The IoT Gateway 125 may include a Dell 500 box with a Linux operating system, and the Intelligent Building Edge Application Server 140 may include an IoTium orchestrator using an orchestration system, such as Kubernetes. The Intelligent Building Edge Application Server 140 may include software applications developed in JavaScript or TypeScript using a Backstack library, and an akka.js or scala.js framework. Additionally, PostgreSQL database system may be used for relational data, and a message broker such as Kafka, Aka Stream, RabbitMO, EventHub, etc. The Intelligent Building Cloud Application Platform 165 may include Azure data storage and analytics.
Although
A “thing” in the context of IoT (Internet of Things) may be a device configured to transmit or receive data through a network. Thus, a “thing” may be a sensor device transmitting sensor data to a controller and receiving control data from the controller. An IoT contextually-aware digital twin of a commercial real-estate building may be generated based on an IoT data model schema that defines various entities associated with a “thing”, such as a type, network, manufacturer, controller, point, asset, and location. A type entity may be a classification of a thing. A thing may be classified based on different categories, such as its class and subclass. A network entity may be a collection of “things” that are connected through wired or wireless protocols. The wired protocol may be a BACnet protocol. A manufacturer entity may be a maker or producer of a “thing”. A point may be a sensor or actuator of a device or controller that contains a digital or analog property. A point may be a configuration value, such as a setpoint value or a schedule. An asset entity may be physical equipment or an object that is located within a place, as understood by a person having skill in the art after review of the entirety disclosed herein. An asset may comprise multiple physical devices and/or points. An asset may have attributes, such as a serial number, or a manufacturer. Assets may be categorized into types and subtypes. Asset types may correspond to building systems, such as HVAC air, HVAC water, plumbing, fire protection, electrical, and fire alarm. Each asset type may include various subtypes. An asset type corresponding to the HVAC air system may include subtypes, such as the variable air volume (VAV) box/controller, fan powered box (FPB), fan coil unit (FCU), computer room air handler (CRAH), fan system, fan, air handling unit (AHU), heat recovery unit (HRU), and heat pump. An asset type corresponding to the HVAC water system may include subtypes, such as chilled water plant, pump, heat exchanger, unit heater, baseboard heater, duct heater, boiler, and expansion tank. An asset type corresponding to the plumbing system may include subtypes, such as pump, and grease trap. An asset type corresponding to the fire protection system may include subtypes, such as pump, and pre-action. An asset type corresponding to the electrical system may include subtypes, such as meter, switchboard, panel board, generator, transfer switch, uninterruptible power supply (UPS), lighting control panel, fire alarm control panel (FACP), and remote control panel. A controller entity may be a physical or logical entity that is associated with one or more points. A controller entity may serve one or more assets through the points of the one or more assets. A location entity may be a physical or logical identification of the whereabouts of a “thing,” such as longitude-latitude coordinates, elevation, address, floor name, logical, or physical zone. The location entities may be determined based on the physical location of an asset, as well as the areas of the building that the asset serves. For example, a VAV box residing at one location may serve many other areas of the building. Location entities may have their own digital twin that defines relationships between the physical and logical locations. The digital twin of the location entities may interface with the digital twin of the asset.
The IoT data model schema enables physical devices 110 to be decoupled from their corresponding physical controllers 113 and represented as separate entities in the digital twin of the building 102a. Even though the buildings 102 may look alike, the connectivity between a device, such as a sensor device, and its controller may be different across the buildings 102. Thus, the same device 110 may have a different relationship to its controller 113 in each of the buildings 102. Further, a device and its controller may be physically separate entities. A device, such as a thermostat residing at a physical location, may communicate with a controller that is placed in a different physical location from the thermostat. The IoT data model schema enables the thermostat and its controller to be represented as different entities in the digital twin of the building. According to the IoT data model schema, the sensors and actuators of the thermostat may be mapped to one or more points, and its physical controller may be mapped to a controller entity. As another example, in complex commercial buildings and structures, a VAV controller may not be synonymous with the VAV box or device that it controls. Instead, the VAV box and its VAV controller may be physically separate entities. Additionally, the VAV box and its VAV controller may have different manufacturers, so that a customer may mix and match the parts from different manufacturers. In this case, representing the VAV controller and VAV box as different entities in the digital twin is also useful for inventory tracking. Therefore, according the IoT data model schema, the sensors and actuators of the VAV box may be mapped to points and the VAV controller may be mapped to a controller entity. In some cases, a device may include sensors, actuators, and control logic within the same physical device. According to the IoT data model schema, the sensors, actuators, and control logic of the device may be mapped to points and a logical controller entity, and thus decoupled from each other and represented as separate entities in the digital twin of the building. Decoupling devices from their corresponding controllers and representing them as separate entities provides a more accurate digital twin of the building resulting in more accurate querying and analytics.
Referring back to
A plurality of relationships associated with the asset may be identified based on determining the points, controller entities, location entities, and network entities associated with the asset. The Device Library 190 in the Intelligent Building Cloud Platform 165 may include various information about the relationships between devices and controllers generated based on training a machine learning algorithm on the data of various building systems. The plurality of relationships associated with the asset may be determined based on the points, controller entities, location entities, and network entities associated with the asset. Therefore, at stage 310, the process 300 may include determining one or more points associated with the asset. At stage 315, the process 300 may include determining one or more controller entities associated with the asset. As discussed above, the controller entities may be logical controllers or physical controllers, such as the controllers 113. At stage 320, the process 300 may include determining a plurality of location entities for the asset. The location entities may be physical and logical locations associated with the asset. The location entities may be determined based on information about the actual locations of the devices and controllers associated with the asset, the areas of the building that the devices and controllers service (service areas), and other information from Building Information Modeling (BIM) data of the building. At stage 325, the process 300 may include determining the network entities associated with the asset. At stage 330, the process 300 may include determining the upstream and downstream utilities or data flow associated with the asset, thereby identifying the controllers and devices that are associated with the upstream and downstream data flow from the asset.
In some implementations, the Intelligent Building Edge Application Server 140 may be configured to provide an application and User Interface (UI) for adding an asset. The determined relationships associated with the asset may be validated in real-time based on asset templates. The asset templates may be based on training a machine learning algorithm on various building data and the digital twins of various buildings.
The process 300 may be repeated for each asset, resulting in a digital twin of the commercial real-estate building. The Intelligent Building Edge Application Server 140 may send the generated digital twin to the Intelligent Building Cloud Application Server 170 of the Intelligent Building Cloud Platform 165 over the network 160. The Intelligent Building Cloud Application Server 170 may store the digital twin as the Digital Twin 180 in the Building Data 175. The Intelligent Building Cloud Platform 165 may perform cloud analytics and processing based on integrating the data stored in the Digital Twin 180 of the different buildings 102. The Intelligent Building Cloud Application Server 170 may access the Historical Data 185, the Digital Twin 180, and the Actor Model 182 in order to perform analytics that provide additional insights into the devices and systems of the corresponding building 102a. The Intelligent Building Edge Application Server 140 may also store the digital twins of the buildings 102 at the edge as Building Edge Data 145. An example Building Edge Data 145a includes an Edge Digital Twin 150 and an Edge Actor Model 155. The Intelligent Building Edge Application Server 140 may use the Building Edge Data 145 for performing edge analytics and processing.
In some implementations, a User Interface (UI) may be provided for on-boarding building systems and devices. The Intelligent Building Edge Application Server 140 may be configured to provide an application and User Interface (UI) for generating a contextually-aware digital twin of a building. In order to access the application and UI, a user may connect a machine to a network of the building. The network may interface with a BACnet. The machine may be a laptop or an IoTium edge device. The user may provide information about a building, select a building from a list of buildings, and add all or some of the assets associated with the selected building to a database. The user may then run a command, such as a “whois” command, on the BACnet to discover the devices on the network. In response, the device identifiers for each of the devices associated with each of the assets may be obtained. The user may obtain additional details about the devices corresponding to the device identifiers by running a “whohas” command, on the BACnet. Based on the device identifiers, the points corresponding to the devices may be identified based on device information stored in, for example, the Device Library 190. The points may be added to the assets and saved to the database. The determined relationships associated with the asset may be validated in real-time based on asset templates. The asset templates may be based on training a machine learning algorithm on various building data and the digital twins of various buildings. The asset templates may be stored in a Document/NoSQL database that is integrated with the graph database representation of the building. In addition to the BACnet discovery discussed above, various information about the devices, such as the physical location, may also be retrieved from. dwg format files.
The graph representation of the building may track other static information about devices such as a manufacturer, serial number, model number, etc. Implementations may access the digital twin of a building to support user queries. For example, a user may, through an application on a user device, such as a smartphone, pull up data for the building or query information about the values of devices. Since the edge application is constantly updating the digital twin at the edge, it is possible to quickly and efficiently search for various information about the current state of the building systems.
Representations of building systems and devices may be created using an actor model, such as the Edge Actor Model 155. The actors may correspond to devices or groups of devices related to building systems, such as HVAC, lighting, etc.
An actor that triggers a rule may query connected devices and assets to find an optimum solution based on the current situation of the system. Once data related to various solutions is received, the best option may be determined based on pre-determined constraints and settings, such as energy usage, time required, etc. In some implementations, device analytics may be provided to model expected performance of a device based on a current state, and the device may be controlled based on computations in real-time using context aware data. By maintaining a stateful machine representation of the device, the exact behavior of a device may be modeled by training a data analytics machine learning algorithm on the real-time data of the input and output data points of the device. Implementations may provide precision command and control for an infinite number of stateful scenarios of the device instead of just a handful that are included in the device's onboard controller. The actor model may be implemented by a distributed computing and non-blocking framework, such as Akka. The actor model and rules may be stored as the Actor Model 182 in the Building Data 175 of the Intelligent Building Cloud Application Platform 165. In addition to hosting the Actor Model 182 in the cloud, implementations may also run the actor system at the edge, which avoids reliance on cloud computing and enables better real-time response.
At stage 505, the process 500 includes setting up the rules for each of the actors. The Intelligent Building Edge Application Server 140 and the IoT Gateway 125 may be configured to provide edge based analytics and control of the devices 110 and the controllers 113 based on rules. For example, a rule may be set to trigger an action based on receiving real-time data related to a supply air temperature that has exceeded a min/max alarm threshold. The rule may indicate that if the supply air temperature falls below a minimum threshold value or exceeded a maximum threshold value for more than five minutes, then actions must be taken to adjust air temperature to a preferred temperature set as a predetermined value. The rule may be stored in the Rules 815 of the actor 800 corresponding to the temperature sensor.
At stage 510, the process 500 includes monitoring the incoming real-time data from the temperature sensor device as an Incoming Message 880, and at stage 515, the process 500 includes processing the Incoming Message 880 and evaluating whether the real-time data meets the rule stored in the Rules 815. A windowing function may ensure that a specific number of consecutive real-time measurements are received from the temperature sensor in order for the rule in the Rules 815 to be triggered. Upon a determination that the supply air temperature has fallen below the minimum threshold value or exceeded the maximum threshold value for more than five minutes, the process 500 moves to stage 520.
At stage 520, the process 520 includes requesting other actors for information for adjusting the air temperature to the predetermined value. For example, other actors relevant for maintaining the air supply temperature may include devices related to a chiller system, an FPB, or other devices configured to effectuate local changes that may impact the air supply temperature. The air temperature actor may send the request to other actors via an Outgoing Message 855. The other actors may receive the Outgoing Message 855 and determine the actions their devices may take in order to achieve a solution, based on the current state of their devices. For example, the actor corresponding to the chiller may determine that operating the chiller for 5 min at 12 Kw is needed to adjust the air temperature to the preferred temperature. The actor corresponding to the FPB (Fan Powered Box) may determine that operating its devices for 2 min at 14 Kw is needed to adjust the air temperature to the preferred temperature. The actor corresponding for local changes may determine that operating its devices for 7 min at 9 Kw is needed to adjust the air temperature to the preferred temperature. Each actor sends this information as an Outgoing Message 855 to the air temperature actor which receives the information as Incoming Messages 880 and processes the messages. At stage 503 of the process 500, the air temperature actor may select a solution based on evaluating the information received from the other actors based on other considerations related to, for example, time, energy, and feasibility. At stage 545, the process 500 includes executing the solution by notifying the relevant assets. The air temperature actor may notify the relevant actor to take an action for executing the solution proposed by the actor. For example, the air temperature actor may send an Outgoing Message 855 to notify the chiller actor to operate the chiller for 5 minutes at 12 KW. The chiller actor receives the message from the air temperature actor as an Incoming Message 880. The chiller actor turns on the cooling system in order to decrease the air temperature and notifies the air temperature actor. At stage 555 of the process 500, the air temperature actor monitors the air temperature based on continuing to receive real-time temperature measurements. At stage 560, the process 500 includes determining whether the air temperature meets the criteria set by the rule stored in the Rule 815. Upon a determination that the air temperature has returned to the preferred temperature, at stage 565 of the process 500, the air temperature actor notifies the chiller to return to its normal operation.
Performing edge based device analytics (i.e. space temperature rate of change) may be used for controlling a device, for example, an air damper, without relying upon cloud compute. Performing edge device analytics relies on accessing the digital twin at the edge to perform a computation rather than relying on the digital twin stored in the cloud. For example, a device may have a data point called space temperature, which may have a function that checks for the rate of change of that value. If the rate of change exceeds a threshold consecutively for a given time period, the air damper of the device closes. This enables the device to act quickly to an urgent fire related condition without relying upon network connectivity to the cloud platform in order to provide a command back to the device.
In some implementations, the data collected from the devices may be stored in a time series database. If a user is attempting to change the temperature, or check on a current status of a VAV box, it is inefficient and expensive to query the cloud stored data. Thus, implementations provide information about the VAV box or other devices within the most recent period, such as 24 hours. If, based on the most recent data, it is observed that the present behavior of the device is not as expected, the cloud platform may be accessed for querying the cloud stored big data related to the device. This may happen based on both the end user triggering or requesting the query or automatically based on preset parameters and thresholds. In some implementations, artificial intelligence and machine learning may be used to analyze and identify anomalies at the same time as the data is being collected. In some implementations, a time bounding may be set of the last 30 sec or other predetermined time period of the data flowing in or being collected. Based on observed patterns in the data, an alert may be triggered which in turn triggers an action to be performed by various devices or systems. Using machine learning by training a machine learning algorithm on previously collected data enables implementations to predict anomalies in the device data as it is being collected. Implementations may access collected data and evaluate it to gain insights. The data that comes off of that could provide previously unknown insights that would not be possible to obtain using traditional methods. Additionally, implementations are able to maintain and manipulate the state machine for entire subsystems.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are described as example implementations of the following claims.
Knight, Robert, Patel, Nikunj, Szcodronski, Rick
| Patent | Priority | Assignee | Title |
| Patent | Priority | Assignee | Title |
| 11086298, | Apr 15 2019 | Rockwell Automation Technologies, Inc.; ROCKWELL AUTOMATION TECHNOLOGIES, INC | Smart gateway platform for industrial internet of things |
| 8229715, | Jun 17 2011 | GOOGLE LLC | System and methods facilitating collaboration in the design, analysis, and implementation of a structure |
| 8285521, | Sep 20 2011 | GOOGLE LLC | Certification controls for a structure design, analysis, and implementation system |
| 8516572, | Sep 20 2011 | GOOGLE LLC | User certification in a structure design, analysis, and implementation system |
| 8843352, | Aug 16 2011 | GOOGLE LLC | System and methods facilitating interfacing with a structure design and development process |
| 8954297, | Jan 02 2012 | GOOGLE LLC | Automated and intelligent structure design generation and exploration |
| 20150089399, | |||
| 20150234946, | |||
| 20150294543, | |||
| 20160370771, | |||
| 20170093915, | |||
| 20170284691, | |||
| 20170357738, | |||
| 20180232459, | |||
| 20180309818, | |||
| 20180357334, | |||
| 20180364654, | |||
| 20190014004, | |||
| 20190094827, | |||
| 20190107830, | |||
| 20190107831, | |||
| 20190107832, | |||
| 20190108013, | |||
| 20190109725, | |||
| 20190109907, | |||
| 20190158309, | |||
| 20190179282, | |||
| 20190199810, | |||
| 20190265661, | |||
| 20190302157, | |||
| 20200012265, | |||
| 20200227178, | |||
| 20200265329, | |||
| 20200285203, | |||
| 20200304375, | |||
| 20200313925, | |||
| CN102411345, | |||
| CN104238510, | |||
| CN109739099, | |||
| KR20170143354, | |||
| WO2019067631, |
| Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
| Jun 27 2019 | PATEL, NIKUNJ | COHESIONIB INC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 062922 | /0696 | |
| Jun 27 2019 | SZCODRONSKI, RICK | COHESIONIB INC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 062922 | /0696 | |
| Jul 26 2019 | KNIGHT, ROBERT | COHESIONIB INC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 062922 | /0696 | |
| Mar 08 2023 | cohesionIB, Inc. | (assignment on the face of the patent) | / |
| Date | Maintenance Fee Events |
| Mar 08 2023 | BIG: Entity status set to Undiscounted (note the period is included in the code). |
| Date | Maintenance Schedule |
| Jan 23 2027 | 4 years fee payment window open |
| Jul 23 2027 | 6 months grace period start (w surcharge) |
| Jan 23 2028 | patent expiry (for year 4) |
| Jan 23 2030 | 2 years to revive unintentionally abandoned end. (for year 4) |
| Jan 23 2031 | 8 years fee payment window open |
| Jul 23 2031 | 6 months grace period start (w surcharge) |
| Jan 23 2032 | patent expiry (for year 8) |
| Jan 23 2034 | 2 years to revive unintentionally abandoned end. (for year 8) |
| Jan 23 2035 | 12 years fee payment window open |
| Jul 23 2035 | 6 months grace period start (w surcharge) |
| Jan 23 2036 | patent expiry (for year 12) |
| Jan 23 2038 | 2 years to revive unintentionally abandoned end. (for year 12) |