A system and method for identification of a landing site for aerial vehicles are provided. In one embodiment, the method includes processing image data pertaining to a potential site area. The method further includes identifying existing building infrastructure and defining boundaries of the identified existing building infrastructure in the potential site area, based on the processed image data. The method also includes identifying existing road infrastructure adjacent to the boundaries of the identified existing building infrastructure and defining boundaries of the identified existing road infrastructure, based on the processed image data. The method further includes calculating a feasibility score of the existing building infrastructure by analyzing the identified existing building infrastructure and the identified existing road infrastructure. The method further includes outputting the identified existing building infrastructure as a potential landing site for landing aerial vehicles when the calculated feasibility score is above a predetermined threshold score.
|
1. A computer-implemented method for identification of a landing site for aerial vehicles, the method comprising:
processing image data pertaining to a potential site area;
identifying existing building infrastructure and defining boundaries of the identified existing building infrastructure in the potential site area, based on the processed image data;
identifying existing road infrastructure adjacent to the boundaries of the identified existing building infrastructure and defining boundaries of the identified existing road infrastructure, based on the processed image data;
determining an accessibility index of the identified existing road infrastructure based on a relative distance between the identified existing building infrastructure and the identified existing road infrastructure;
determining a connectedness index of the identified existing road infrastructure based on a distance of the identified existing road infrastructure from one or more of nearby transport stations;
determining a quality index of the identified existing road infrastructure based on a physical condition of the identified existing road infrastructure;
determining a traffic index of the identified existing road infrastructure based on an estimation of average traffic on the identified existing road infrastructure over one or more time periods;
calculating a connectivity score for the identified existing road infrastructure based on the determined accessibility index, the connectedness index, the quality index, and the traffic index thereof;
calculating a feasibility score of the existing building infrastructure based on the calculated connectivity score; and
outputting the identified existing building infrastructure as a potential landing site for landing aerial vehicles when the calculated feasibility score of the identified existing building infrastructure is above a predetermined threshold score.
13. An apparatus for identification of a landing site for aerial vehicles, the apparatus comprising:
one or more processors;
a memory communicatively coupled to the one or more processors, wherein the memory is configured with the one or more processors to:
process image data pertaining to a potential site area;
identify existing building infrastructure and define boundaries of the identified existing building infrastructure in the potential site area, based on the processed image data;
identify existing road infrastructure adjacent to the boundaries of the identified existing building infrastructure and define boundaries of the identified existing road infrastructure, based on the processed image data;
determine an accessibility index of the identified existing road infrastructure based on a relative distance between the identified existing building infrastructure and the identified existing road infrastructure;
determine a connectedness index of the identified existing road infrastructure based on a distance of the identified existing road infrastructure from one or more of nearby transport stations;
determine a quality index of the identified existing road infrastructure based on a physical condition of the identified existing road infrastructure;
determine a traffic index of the identified existing road infrastructure based on an estimation of average traffic on the identified existing road infrastructure over one or more time periods;
calculate a connectivity score for the identified existing road infrastructure based on the determined accessibility index, the connectedness index, the quality index, and the traffic index thereof;
calculate a feasibility score of the existing building infrastructure based on the calculated connectivity score; and
output the identified existing building infrastructure as a potential landing site for landing aerial vehicles when the calculated feasibility score of the identified existing building infrastructure is above a predetermined threshold score.
18. A system comprising:
one or more aerial vehicles; and
an apparatus in communication with the one or more aerial vehicles, the apparatus comprising:
one or more processors;
a memory communicatively coupled to the one or more processors, wherein the memory is configured with the one or more processors to:
process image data pertaining to a potential site area;
identify existing building infrastructure and define boundaries of the identified existing building infrastructure in the potential site area, based on the processed image data;
identify existing road infrastructure adjacent to the boundaries of the identified existing building infrastructure and define boundaries of the identified existing road infrastructure, based on the processed image data;
determine an accessibility index of the identified existing road infrastructure based on a relative distance between the identified existing building infrastructure and the identified existing road infrastructure;
determine a connectedness index of the identified existing road infrastructure based on a distance of the identified existing road infrastructure from one or more of nearby transport stations;
determine a quality index of the identified existing road infrastructure based on a physical condition of the identified existing road infrastructure;
determine a traffic index of the identified existing road infrastructure based on an estimation of average traffic on the identified existing road infrastructure over one or more time periods;
calculate a connectivity score for the identified existing road infrastructure based on the determined accessibility index, the connectedness index, the quality index, and the traffic index thereof;
calculate a feasibility score of the existing building infrastructure based on the calculated connectivity score; and
output the identified existing building infrastructure as a potential landing site for landing aerial vehicles when the calculated feasibility score of the identified existing building infrastructure is above a predetermined threshold score.
2. The method of
3. The method of
determining a height index based on an estimation of a height of the identified existing building infrastructure;
determining an obstacle index based on an identification of one or more possible obstacles proximal to the identified existing building infrastructure and relative positions thereof with respect to the identified existing building infrastructure for landing the aerial vehicles thereon; and
calculating an absence of obstacles score for the identified existing building infrastructure based on the determined height index and the obstacle index.
4. The method of
5. The method of
calculating an electricity availability score for the identified existing building infrastructure based on known uptime of electricity supply to the identified existing building infrastructure.
6. The method of
calculating an ease of flying score for the identified existing building infrastructure based on at least one of identification of no-fly zones and availability of potential emergency landing sites nearby the identified existing building infrastructure.
7. The method of
assigning respective weight coefficients to one or more of the calculated connectivity score, the absence of obstacles score, the electricity availability score, and the ease of flying score.
8. The method of
9. The method of
10. The method of
determining a height index based on an estimation of a height of the identified existing building infrastructure;
determining an obstacle index based on an identification of one or more possible obstacles proximal to the identified existing building infrastructure and relative positions thereof with respect to the identified existing building infrastructure for landing the aerial vehicles thereon; and
calculating an absence of obstacles score for the identified existing building infrastructure based on the determined height index and the obstacle index.
11. The method of
calculating an electricity availability score for the identified existing building infrastructure based on known uptime of electricity supply to the identified existing building infrastructure.
12. The method of
calculating an ease of flying score for the identified existing building infrastructure based on at least one of identification of no-fly zones and availability of potential emergency landing sites nearby the identified existing building infrastructure.
14. The apparatus of
15. The apparatus of
determine a height index based on an estimation of a height of the identified existing building infrastructure;
determine an obstacle index based on an identification of one or more possible obstacles proximal to the identified existing building infrastructure and relative positions thereof with respect to the identified existing building infrastructure for landing the aerial vehicles thereon; and
calculate an absence of obstacles score for the identified existing building infrastructure based on the determined height index and the obstacle index.
16. The apparatus of
17. The apparatus of
calculate an electricity availability score for the identified existing building infrastructure based on known uptime of electricity supply to the identified existing building infrastructure.
|
The present patent document claims the benefit of Indian Patent Application No. 201931034706, filed Aug. 28, 2019, which is hereby incorporated by reference.
The present disclosure generally relates to air mobility infrastructure, and more particularly relates to a system and a method for identification of landing sites for aerial vehicles.
Unmanned aerial vehicles (UAVs) include a variety of vehicles, from conventional fixed wing airplanes, to helicopters, to ornithopters (i.e., machines that fly like birds), and are used in a variety of roles. They may be remotely directed by a pilot on the ground or may be autonomous or semi-autonomous vehicles that fly missions using preprogrammed coordinates, GPS navigation, etc. UAVs may be equipped with cameras to provide imagery during flight, which may be used for navigational or other purposes, (e.g., identify a house address, etc.). UAVs may also be equipped with sensors to provide local weather and atmospheric conditions, radiation levels, and other conditions. UAVs may also include cargo bays, hooks, or other devices for carrying payloads. Newer generation UAVs may also provide significant payload capabilities. As a result, UAVs may also be used for delivering packages, groceries, mail, and other items. The use of UAVs for deliveries may reduce costs and increase speed and accuracy. However, the range provided by current UAV technology makes operations over a wide area; e.g., throughout a city, or even a portion of a city difficult to achieve, as the UAV may need to be recharged after completing a few trips or so.
Furthermore, it is widely acknowledged that there has been rapid increase in the number of vehicles on streets globally which has not seen proportional development in infrastructure. This has resulted in traffic congestion, in all major cities, at unprecedented levels. It is widely believed that flying vehicles may replace cars driven on roads, which would help in reducing the traffic congestions. This will also help to save time as trips that took hours on the ground may be reduced to minutes in the air, which in-turn would help to improve productivity and quality of life. With the increasing popularity of small UAVs or drones, and regulations increasingly supporting their commercial use, passenger drones and flying cars appear to be moving closer to reality. Despite the technological progress and many potential applications of the UAVs, there are various challenges to consider with respect to regulations, certifications, infrastructure, and air traffic management. In particular, infrastructure constraints including proper take-off and landing zones within the city, parking facility and battery charging bays poses as a big hurdle in realizing the potential of the industry.
It is a complex endeavor to construct take-off and landing pads for autonomous drones in regions that are already overcrowded. In order to discover potential infrastructure sites, human labor is required to gather and analyze survey data to identify feasible sites, which is time-consuming and expensive. For example, many businesses operating in the industry develop their own, limited take-off and landing infrastructure that is manually verified. Such infrastructure may be built ground up, from laying the foundation for take-off and landing pads, which may turn out to be quite expensive endeavor for the business.
In light of the above, there is a need of a viable solution to automate the identification and verification of a landing site for aerial vehicles which is valid across all geographical regions and is less labor intensive and is economical.
The scope of the present disclosure is defined solely by the appended claims and is not affected to any degree by the statements within this summary. The present embodiments may obviate one or more of the drawbacks or limitations in the related art.
The object of the present disclosure is achieved by a computer-implemented method for identification of landing sites for aerial vehicles. The method includes processing image data pertaining to a potential site area. The method further includes identifying existing building infrastructure and defining boundaries of the identified existing building infrastructure in the potential site area, based on the processed image data. The method also includes identifying existing road infrastructure adjacent to the boundaries of the identified existing building infrastructure and defining boundaries of the identified existing road infrastructure, based on the processed image data. The method further includes calculating a feasibility score of the existing building infrastructure by analyzing the identified existing building infrastructure and the identified existing road infrastructure. The method further includes outputting the identified existing building infrastructure as a potential landing site if the calculated feasibility score of the identified existing building infrastructure is above a predetermined threshold score.
In an embodiment, the feasibility score of the identified existing building infrastructure is a measure of the identified existing building infrastructure as the landing site to support landing of the aerial vehicles thereon.
In an embodiment, the method includes determining accessibility index of the identified existing road infrastructure based on relative distance between the identified existing building infrastructure and the identified existing road infrastructure. The method further includes determining connectedness index of the identified existing road infrastructure based on distance of the identified existing road infrastructure from one or more of nearby transport stations. Also, the method includes determining quality index of the identified existing road infrastructure based on physical condition of the identified existing road infrastructure. The method further includes determining traffic index of the identified existing road infrastructure based on estimation of average traffic on the identified existing road infrastructure over one or more time periods. The method further includes calculating a connectivity score for the identified existing road infrastructure based on the determined accessibility index, the connectedness index, the quality index, and the traffic index thereof.
In an embodiment, the method further includes determining a height index based on an estimation of height of the identified existing building infrastructure. The method further includes determining obstacle index based on identification of one or more possible obstacles proximal to the identified existing building infrastructure and relative positions thereof with respect to the identified existing building infrastructure for landing the aerial vehicles thereon. The method further includes calculating an absence of obstacles score for the identified existing building infrastructure based on the determined height index and the obstacle index.
In an embodiment, the height of the identified existing building infrastructure is estimated by utilizing one or more of information of camera parameters associated with the image data, altitude information associated with the image data, and depth analysis of the identified existing building infrastructure from the image data.
In an embodiment, the method further includes calculating an electricity availability score for the identified existing building infrastructure based on known uptime of electricity supply to the identified existing building infrastructure.
In an embodiment, the method further includes calculating an ease of flying score for the identified existing building infrastructure based on at least one of identification of no-fly zones and availability of potential emergency landing sites nearby the identified existing building infrastructure.
In an embodiment, the method further includes assigning respective weight coefficients to one or more of the calculated connectivity score, the absence of obstacles score, the electricity availability score, and the ease of flying score.
In an embodiment, the feasibility score is calculated as a cumulative sum of the calculated connectivity score, the absence of obstacles score, the electricity availability score, and the ease of flying score in consideration of the assigned one or more respective weight coefficients.
In an embodiment, the existing building infrastructure is outputted as the potential landing site utilizing real-time image data obtained from the aerial vehicle.
The object of the present disclosure is also achieved by an apparatus for identification of a landing site for aerial vehicles. The apparatus includes at least one memory configured to store computer program code instructions. The apparatus also includes at least one processor configured to execute the computer program code instructions to cause the acts of the method described above.
The object of the present disclosure is further achieved by a system including one or more aerial vehicles, and the apparatus for identification of a landing site for aerial vehicles as described above.
The object of the present disclosure is further achieved by a computer-program product, having machine-readable instructions stored therein, that when executed by a processor, cause the processor to perform acts of the method described above.
Still other aspects, features, and advantages of the disclosure are readily apparent from the following detailed description, simply by illustrating a number of particular embodiments and implementations. The disclosure is also capable of other and different embodiments, and its several details may be modified in various obvious respects, all without departing from the scope of the disclosure. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not as restrictive.
A more complete appreciation of the present disclosure and many of the attendant aspects thereof will be readily obtained as the same becomes better understood by reference to the following description when considered in connection with the accompanying drawings:
Various embodiments are described with reference to the drawings, wherein like reference numerals are used to refer the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for the purpose of explanation, numerous specific details are set forth in order to provide thorough understanding of one or more embodiments. It may be evident that such embodiments may be practiced without these specific details.
Examples of a method, a system, and a computer-program product for identification of landing sites for aerial vehicle are disclosed herein. In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments. It is apparent, however, to one skilled in the art that the embodiments may be practiced without these specific details or with an equivalent arrangement. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the embodiments.
As illustrated in
Further, as illustrated, the map processing platform 102 has connectivity over a communication network 106 to a services platform 108 (e.g., an OEM platform) that provides one or more services (e.g., services 108a and 108b), such as sensor data collection services. By way of example, the services 108a, 108b may also be other third-party services and include mapping services, navigation services, travel planning services, notification services, social networking services, content (e.g., audio, video, images, etc.) provisioning services, application services, storage services, contextual information determination services, location based services, information based services (e.g., weather, news), etc. In one embodiment, the services platform 108 uses the output (e.g., landing site identification) from the map processing platform 102 to provide services such as navigation, mapping, other location-based services, etc.
In one embodiment, content providers (e.g., content providers 110a, 110b) may provide content or data (e.g., including geographic data, parametric representations of mapped features, etc.) to the geographic database 104, the map processing platform 102, the services platform 108, the services 108a, 108b, and/or the UAV 101. The content provided may be any type of content, such as map content, textual content, audio content, video content, image content, etc. In one embodiment, the content providers 110a, 110b may provide content that may aid in the detecting and classifying landing sites or other related characteristics. In one embodiment, the content providers 110a, 110b may also store content associated with the geographic database 104, the map processing platform 102, the services platform 108, the services 108a, 108b, and/or the UAV 101. In another embodiment, the content providers 110a, 110b may manage access to a central repository of data, and offer a consistent, standard interface to data, such as a repository of the geographic database 104.
At act 302, image data pertaining to the potential site area is processed. For this purpose, the map processing platform 102 may be configured to retrieve information related to geographic data (including image data) of the potential site area from the geographic database 104. In one example, the sensor data ingestion module 202 may be used to retrieve UAV sensor data, and the geographic data module 204 may be used to retrieve geographic data for the potential site area. The geographic data may include satellite images of the potential site area. The images may be one or more of panchromatic images, multispectral images and RGB images. The images resolution may be as high as 30 centimeters (cm). The satellite images may be obtained from content providers 110a, 110b like Google Maps® or any other third-party source without any limitations. It is known that such satellite images are taken frequently and may be updated, and thus may be used to monitor infrastructure in the potential site area like buildings, roads, tree cover, etc. Further, herein, the UAV sensor data may be data regarding the potential site area, as collected by the UAV 101 travelling over the potential site area. The UAV sensor data may include aerial images taken by the UAV 101. Such aerial images may be one or more of RGB images (e.g., monocular images), depth images (e.g., from stereo camera or LIDAR), or the like. It may be appreciated that the UAV 101 may be fitted with the required sensors and cameras for the purpose. In one example, if the stereo camera and LIDAR may not be available with the UAV 101, the depth information about the potential site area may also be obtained from RGB images using monocular depth estimation techniques. These techniques are well known in the art. Some examples of such techniques are presented in research papers, for instance, “Digging into self-supervised monocular depth estimation” (Goddard et al.), “Deep ordinal regression network for monocular depth estimation” (H Fu et al.), and “Depth Estimation Using Monocular Camera” (Joglekar et al.). It may be appreciated that the UAV sensor data (including aerial images) may be taken more frequently to provide more up-to-date information. For example, a survey drone may be deployed for a few hours (depending on its battery capacity) to get real time data on human and vehicle movement at the potential site area. For this purpose, the UAV control module 212 may send command to the UAV 101 to capture the required images. It may be contemplated that such commands may be sent autonomously or may be sent by an operator or a pilot of the UAV, as required. Herein, as discussed, at first the satellite images are analyzed for initial feasibility, and then the aerial images from the UAV 101 are utilized for a detailed check.
The method 300 further includes a planning phase for recognition and validation of the landing site in the potential site area. The planning phase may include: (1) building and road analysis, (2) road analysis, (3) site candidate shortlisting, and (4) autonomous on-site survey. These processes may be carried out in order or concurrently to achieve the purpose. The details about the listed processes are provided in the subsequent paragraphs in context of the method of the present disclosure.
At act 304, existing building infrastructure and defining boundaries of the identified existing building infrastructure in the potential site area is identified, based on the processed image data. In the present system 100, the data processing module 206 may process the collected image data to identify potential building infrastructure (e.g., rooftops of the buildings) which may be suitable as a landing site. In particular, the data processing module 206 may utilize the collected images to identify the location and geometry of the building, dimensions of roof-tops, the planarity of the rooftops (e.g., whether a rooftop is flat or slanted) and height of the buildings.
Further, at act 306, existing road infrastructure and defining boundaries of the identified existing road infrastructure is identified, based on the processed image data. The road infrastructure is identified adjacent to the boundaries of the identified existing building infrastructure. The road infrastructure may include any one or more of public roads leading to the building from one or more nearby transportation stations, private roads providing accessed to the entry of the building, and the like.
Herein, the data processing module 206 may process the collected image data using image segmentation techniques (to obtain segmentation map) as known in the art. This segmentation map is obtained using deep convolutional neural networks (DCNNs), such as, for example, but not limited to, D-Linknet. To reduce computational complexity and preserve the spatial resolution of the input image, the input image is divided into a grid of smaller non-overlapping images. A segmentation map is obtained for each of these grid images. These maps are then stitched together to obtain the segmentation map of the original input image. In addition, the data processing module 206 may further make use of depth information or point clouds for segmentation. Examples of techniques implemented for the purpose may be found in the art, for instance, “PointSeg: Real-Time Semantic Segmentation Based on 3D LiDAR Point Cloud” (Y Wang et al.), “Semantic Segmentation of 3D LiDAR Data in Dynamic Scene Using Semi-supervised Learning” (J Mei et al.), “PointNet” (CR Qi et al.), “SqueezeSeg” (B Wu et al.), and the like.
Thereafter, the boundaries of the buildings may be defined using one of mask-based image segmentation technique and polygon-based instance segmentation technique. In the mask-based image segmentation technique, every pixel in the image is labelled as either building or background. Such technique may be implemented on Mask RCNN (Region-based convolutional neural networks) as known in the art. In the polygon-based instance segmentation technique, directly outputs the boundaries of individual buildings. Such technique may be implemented on PolyRNN (Polygon Recurrent Neural Network) as known in the art. The road analysis problem is formulated as binary semantic segmentation. It may be understood that semantic segmentation of building and road images is implemented using a standard encoder and decoder CNN architecture. Herein, the input is an RGB image and output is a segmentation map, which is an image of the same size. Further, herein, each pixel is assigned one of two labels, (e.g., either building or background). The details for implementation of such techniques have been described further in the subsequent paragraphs.
The method 300 further includes neighborhood analysis in relation to the identified existing building infrastructure and existing road infrastructure. The neighborhood analysis is required to determine feasibility of the exiting building infrastructure to facilitate landing of an aerial vehicle, (e.g., a UAV), thereon. This provides information about the neighborhood of identified buildings. For this purpose, both satellite images as well as aerial images and videos are utilized. The neighborhood analysis includes, but may not be limited to, last mile connectivity analysis, building height analysis, electricity availability analysis, and ease of flying analysis. It will be appreciated that the terms used should be given a reasonable interpretation in the context of the present disclosure.
Herein, the last mile connectivity analysis involves determining accessibility index of the identified existing road infrastructure based on relative distance between the identified existing building infrastructure and the identified existing road infrastructure. This is achieved using the road segmentation map (as obtained), the building locations, and other information about the camera parameters. It may be appreciated that the accessibility index may be inversely proportional to distance between the identified existing building infrastructure and the identified existing road infrastructure. Further, this involves determining connectedness index of the identified existing road infrastructure based on distance of the identified existing road infrastructure from one or more of nearby transport stations (as inferred earlier). The information about public transport information such as nearby bus, tram, and train connectivity may be obtained using third party sources of information such as Google Maps without any limitations. Again, it may be appreciated that the connectedness index may be inversely proportional to distance of the identified existing road infrastructure from one or more of nearby transport stations. Further, this involves determining quality index of the identified existing road infrastructure based on physical condition of the identified existing road infrastructure. For this purpose, the road is classified based on its type, (e.g., dirt, cement, or asphalt), which is a measure of the road quality. Such techniques are known in the art and thus have not been described herein for the brevity of the present disclosure. Further, this involves determining traffic index of the identified existing road infrastructure based on estimation of average traffic on the identified existing road infrastructure over one or more time periods. For instance, the average traffic metrics of the road is estimated by detecting vehicles on the road using algorithms such as You Only Look Twice (YOLT) or the like. Using the zone on the road closest to a potential landing site, the number of public transport vehicles passing through this zone is counted. Further, a connectivity score for the identified existing road infrastructure is calculated based on the determined accessibility index, the connectedness index, the quality index, and the traffic index thereof. It may be contemplated that the connectivity score may be a sum (or weighted sum) of the determined indexes herein.
Further, the building height analysis involves determining a height index based on an estimation of height of the identified existing building infrastructure (as inferred earlier). It may be contemplated that height of a building may be estimated from aerial images that are not perfectly overhead but taken at an angle to the building. Herein, knowledge of the camera parameters is utilized to calculate the height of the building. Building height may also be obtained as a function of the height from point cloud/depth image, altimeter reading from the UAV 101, and the like. This further involves determining obstacle index based on identification of one or more possible obstacles proximal to the identified existing building infrastructure and relative positions thereof with respect to the identified existing building infrastructure for landing the aerial vehicles thereon. The obstacles (e.g., poles, towers, trees, etc.) may be found using an object detection algorithm as known in the art. In some examples, a survey may be carried out by the UAV 101 which is equipped with LIDAR and three-dimensional scanners to obtain a 3D map of the region around the landing site, which in turn may be reviewed by an operator to identify obstacles. It may be appreciated that depth/point clouds are particularly useful here to obtain an exact 3D map of the region around the landing zone, which is required when the aerial vehicle cannot make a perfectly vertical takeoff/landing. This allows to have smaller landing zones in a complex environment. It may be appreciated that the obstacle index may be inversely proportional to number and/or size of possible obstacles identified. Further, an absence of obstacles score is calculated for the identified existing building infrastructure based on the determined height index and the obstacle index. It may be contemplated that the absence of obstacles score may be a sum (or weighted sum) of the determined indexes, (e.g., the height index and the obstacle index).
Further, the electricity availability analysis involves calculating an electricity availability score for the identified existing building infrastructure based on known uptime of electricity supply to the identified existing building infrastructure. For this purpose, connection to the electric grid and nearby electric towers are identified by detecting power transmission towers and lines from satellite images leading to the identified existing building infrastructure. Herein, the electricity availability score for the identified existing building infrastructure is directly proportional to the known uptime of electricity supply at the identified existing building infrastructure. It may be understood that reliable electrical supply is needed for providing changing infrastructure to the aerial vehicles using the landing site.
Further, the ease of flying analysis involves calculating an ease of flying score for the identified existing building infrastructure based on at least one of identification of no-fly zones and availability of potential emergency landing sites nearby the identified existing building infrastructure. Identification of no-fly zones, and airspace restrictions is carried out based on documents and charts issued by the regulatory body and third parties such as AirMap. Further, nearby airports and military bases are identified with object detection algorithm trained on a dataset such as xView. Further, emergency landing spots (like water bodies and flat land near potential landing sites) which may be used for emergency landings are identified.
At act 308, a feasibility score of the existing building infrastructure is calculated by analyzing the identified existing building infrastructure and the identified existing road infrastructure. Herein, the feasibility score is based on calculated connectivity score, the absence of obstacles score, the electricity availability score, and the ease of flying score (as described in the preceding paragraphs). In the present embodiments, the feasibility score of the identified existing building infrastructure is a measure of the identified existing building infrastructure as the landing site to support landing of the aerial vehicles thereon. For this purpose, respective weight coefficients are assigned to one or more of the calculated connectivity score, the absence of obstacles score, the electricity availability score, and the ease of flying score. It may be appreciated that the weights may be assigned based on various parameters and practical factors. For instance, weights may be assigned according to the specific requirements of a potential site. This accounts for the specific requirements of the aerial vehicles such as physical size, payload and takeoff and landing masses. Further, the feasibility score is calculated as a cumulative sum of the calculated connectivity score, the absence of obstacles score, the electricity availability score, and the ease of flying score in consideration of the assigned one or more respective weight coefficients. For example, the feasibility score is calculated as:
connectivity score, sc=fc(dp,dr,fp),wc
wherein, dp is distance to nearest public transport stop, dr is distance to nearest major road, fp=frequency of public transport, and wc is weight assigned thereto;
absence of obstacles score, so=fo(po,co),wo
wherein, po is position of obstacle with respect to the landing zone, co represents number/size of the predicted obstacles and wo is the weight assigned thereto;
electricity availability score, se=fe(ue),we
wherein, ue is uptime of electricity in this area (obtained from power company) and we is the weight assigned thereto; and
ease of flying score, sr is manually evaluated with weight assigned (wo), such that
feasibility score, s=Σ=c,e,o,rwisi
At act 310, the identified existing building infrastructure as a potential landing site is outputted if the calculated feasibility score of the identified existing building infrastructure is above a predetermined threshold score. That is, if the threshold score for this neighborhood is T; then the landing site is shortlisted if s>T. Herein, threshold score is calibrated based on well-known landing sites as well as human knowledge. All potential sites that have a score above the threshold score are then outputted.
In some examples, the method 300 includes an autonomous on-site survey. For this purpose, an autonomous drone is deployed to survey the shortlisted site. This survey collects fine-grained and high-resolution images of that site from a lower altitude. This survey data is evaluated with algorithms as well as manually.
According to an embodiment of the present disclosure, the method 300 also includes an operational phase providing real-time landing decision support. Herein, the data including the satellite images and aerial images (as collected from third-party sources or the like, as discussed above) may be used not just to identify the landing sites during the planning phase, but also during operation of aerial vehicle. That is, the existing building infrastructure is outputted as the potential landing site utilizing real-time image data obtained from the aerial vehicle.
For this purpose, the extracted segmentation maps, data about building height, obstacle information are stored on the aerial vehicle or made available over the network. This extracted information is geo-tagged (e.g., the extracted information is associated with geographical coordinates) in the planning phase and is available to the operator or the autonomous controller of the aerial vehicle. Such data may be utilized real-time for example, like in poor weather conditions the pilot of the aerial vehicle may make use of the existing 3D map for navigation around obstacles even if the aerial vehicle is not equipped with a depth sensor or the like. Further, such information may be utilized to identify near-by suitable landing sites in emergency situations. This data may also be used by the present systems to identify which landing site is suitable for an inbound drone. This may be done in real-time because all the infrastructural information is already available.
For the purposes of the present disclosure, the UAV 101 may be equipped with an imaging device, such as, one or more of a camera sensor, a LiDAR sensor (and/or RADAR sensor), a position sensor, a motion sensor, and the like. Further, the sensors may include a communication device to retrieve sensor data directly from other UAVs or air traffic infrastructure with connected communications capabilities (e.g., cellular or other wireless communications equipped vehicles) or from an Original Equipment Manufacturer (OEM) provider (e.g., UAV manufacturer) operating an OEM platform (e.g., a services platform 108a, 108b) that collects sensor data from UAVs manufactured by or otherwise associated with the OEM. The retrieval of the sensor data and/or the geographic data may occur in real-time or near real-time, continuously, periodically, according to a schedule, on demand, etc. In some examples, the sensor data ingestion module 202, for instance, may fuse data from these multiple different sensors to increase consistency thereof, like for detection of obstacles. In some embodiments, the sensor data ingestion module 202 may also pre-process the collected raw sensor data for training of the machine learning model via the machine learning module 208 for better prediction thereby. In the present embodiments, if the site is found to be suitable for landing, details of this site are added to a database, via the data publication module 210.
In an embodiment where the geographic data is used alone or in combination with the sensor data, the geographic data module 204 may retrieve requested geographic data for an area of interest by performing a location-based query of the geographic database 104 or equivalent. By way of example, the geographic data may include any attribute of the corresponding map location stored in the geographic database 104. The retrieved geographic data may include, but is not limited to, a functional class, information about infrastructure density, presence of obstacles, distance to a nearby transport stations, or a combination thereof.
In one embodiment, the communication network 106 of the present system 100 includes one or more networks such as a data network, a wireless network, a telephony network, or any combination thereof. It is contemplated that the data network may be any local area network (LAN), metropolitan area network (MAN), wide area network (WAN), a public data network (e.g., the Internet), short range wireless network, or any other suitable packet-switched network, such as a commercially owned, proprietary packet-switched network, e.g., a proprietary cable or fiber-optic network, and the like, or any combination thereof. In addition, the wireless network may be, for example, a cellular network and may employ various technologies including enhanced data rates for global evolution (EDGE), general packet radio service (GPRS), global system for mobile communications (GSM), Internet protocol multimedia subsystem (IMS), universal mobile telecommunications system (UMTS), etc., as well as any other suitable wireless medium, e.g., worldwide interoperability for microwave access (WiMAX), Long Term Evolution (LTE) networks, code division multiple access (CDMA), wideband code division multiple access (WCDMA), wireless fidelity (Wi-Fi), wireless LAN (WLAN), Bluetooth®, Internet Protocol (IP) data casting, satellite, mobile ad-hoc network (MANET), and the like, or any combination thereof.
By way of example, the map processing platform 102, services platform 108, services 108a, 108b, and/or content providers 110a, 110b communicate with each other and other components of the system 100 using well known, new or still developing protocols. In this context, a protocol includes a set of rules defining how the network nodes within the communication network 106 interact with each other based on information sent over the communication links. The protocols are effective at different layers of operation within each node, from generating and receiving physical signals of various types, to selecting a link for transferring those signals, to the format of information indicated by those signals, to identifying which software application executing on a computer system sends or receives the information. The conceptually different layers of protocols for exchanging information over a network are described in the Open Systems Interconnection (OSI) Reference Model.
Communications between the network nodes may be effected by exchanging discrete packets of data. Each packet may include (1) header information associated with a particular protocol, and (2) payload information that follows the header information and contains information that may be processed independently of that particular protocol. In some protocols, the packet includes (3) trailer information following the payload and indicating the end of the payload information. The header includes information such as the source of the packet, its destination, the length of the payload, and other properties used by the protocol. The data in the payload for the particular protocol may include a header and payload for a different protocol associated with a different, higher layer of the OSI Reference Model. The header for a particular protocol may indicate a type for the next protocol contained in its payload. The higher layer protocol is said to be encapsulated in the lower layer protocol. The headers included in a packet traversing multiple heterogeneous networks, such as the Internet, may include a physical (layer 1) header, a data-link (layer 2) header, an internetwork (layer 3) header and a transport (layer 4) header, and various application (layer 5, layer 6 and layer 7) headers as defined by the OSI Reference Model.
In one embodiment, the terminology applied to the representation of geographic features in the geographic database 104 may include “Node” which refers to a point that terminates a link. “Line segment” refers to a straight line connecting two points. “Link” (or “edge”) refers to a contiguous, non-branching string of one or more line segments terminating in a node at each end. “Shape point” refers to a point along a link between two nodes (e.g., used to alter a shape of the link without defining new nodes). “Oriented link” refers to a link that has a starting node (referred to as the “reference node”) and an ending node (referred to as the “non-reference node”). “Simple polygon” refers to an interior area of an outer boundary formed by a string of oriented links that begins and ends in one node. “Polygon” refers to an area bounded by an outer boundary and none or at least one interior boundary (e.g., a hole or island). In one embodiment, a polygon is constructed from one outer simple polygon and none or at least one inner simple polygon. A polygon is simple if it just includes one simple polygon, or complex if it has at least one inner simple polygon.
In one embodiment, the geographic database 104 follows certain conventions. For example, links do not cross themselves and do not cross each other except at a node. Also, there are no duplicated shape points, nodes, or links. Two links that connect each other have a common node. In the geographic database 104, overlapping geographic features are represented by overlapping polygons. When polygons overlap, the boundary of one polygon crosses the boundary of the other polygon. In the geographic database 104, the location at which the boundary of one polygon intersects they boundary of another polygon is represented by a node. In one embodiment, a node may be used to represent other locations along the boundary of a polygon than a location at which the boundary of the polygon intersects the boundary of another polygon. In one embodiment, a shape point is not used to represent a point at which the boundary of a polygon intersects the boundary of another polygon.
As shown, the geographic database 104 includes node data records 804, link data records 806, POI data records 808, obstacle records 810, other records 812, and indexes 814, for example. More, fewer, or different data records may be provided. In one embodiment, additional data records (not shown) may include cartographic (“carto”) data records, routing data, and maneuver data. In one embodiment, the indexes 814 may improve the speed of data retrieval operations in the geographic database 104. In one embodiment, the indexes 814 may be used to quickly locate data without having to search every row in the geographic database 104 every time it is accessed. For example, in one embodiment, the indexes 814 may be a spatial index of the polygon points associated with stored feature polygons. The obstacle records 810 store predicted/validated obstacles and other related road characteristics. The predicted data, for instance, may be stored as attributes or data records of an obstacle overlay, which fuses with the predicted attributes with map attributes or features.
In exemplary embodiments, the link data records 806 are links or segments representing paths, as may be used in the calculated route or recorded route information for determination of one or more personalized routes. The node data records 804 are end points corresponding to the respective links or segments of the road segment data records 806. The link data records 806 and the node data records 804 represent a road network, such as used by vehicles, cars, and/or other entities. Alternatively, the geographic database 104 may contain path segment and node data records or other data that represent pedestrian paths or areas in addition to or instead of the vehicle road record data, for example.
In one embodiment, the geographic database 104 may be maintained by the content provider 110a, 110b in association with the services platform 108 (e.g., a map developer). The map developer may collect geographic data to generate and enhance the geographic database 104. There may be different ways used by the map developer to collect data. These ways may include remote sensing, (e.g., aerial or satellite photography), obtaining data from other sources, (e.g., municipalities or respective geographic authorities), and the like.
The geographic database 104 may be a master geographic database stored in a format that facilitates updating, maintenance, and development. For example, the master geographic database or data in the master geographic database may be in an Oracle spatial format or other spatial format, such as for development or production purposes. The Oracle spatial format or development/production database may be compiled into a delivery format, such as a geographic data files (GDF) format. The data in the production and/or delivery formats may be compiled or further compiled to form geographic database products or databases, which may be used in end user navigation devices or systems.
For example, geographic data is compiled (such as into a platform specification format (PSF) format) to organize and/or configure the data for performing navigation-related functions and/or services, such as route calculation, route guidance, map display, speed calculation, distance and travel time functions, and other functions, by a navigation device, such as by the vehicle 103, for example. The navigation-related functions may correspond to vehicle navigation, pedestrian navigation, or other types of navigation. The compilation to produce the end user databases may be performed by a party or entity separate from the map developer. For example, a customer of the map developer, such as a navigation device developer or other end user device developer, may perform compilation on a received geographic database in a delivery format to produce one or more compiled navigation databases.
It is to be understood that the system and methods described herein may be implemented in various forms of hardware, software, firmware, special purpose processors, or a combination thereof. One or more of the present embodiments may take a form of a computer program product including program modules accessible from computer-usable or computer-readable medium storing program code for use by or in connection with one or more computers, processors, or instruction execution system. For the purpose of this description, a computer-usable or computer-readable medium may be any apparatus that may contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The medium may be electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation mediums in and of themselves as signal carriers are not included in the definition of physical computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, random access memory (RAM), a read only memory (ROM), a rigid magnetic disk and optical disk such as compact disk read-only memory (CD-ROM), compact disk read/write, and digital versatile disc (DVD). Both processors and program code for implementing each aspect of the technology may be centralized or distributed (or a combination thereof) as known to those skilled in the art.
In one embodiment, the computing system 900 includes a communication mechanism such as a bus 902 for passing information among the components of the computing system 900. The computing system 900 includes one or more processing units 904 and a memory unit 906. The memory unit 906 may be communicatively coupled to the one or more processing units 904. Hereinafter, the one or more processing units 904 are simply referred to as processor 904 and the memory unit 906 is simply referred to as memory 906. Herein, in particular, the processor 904 has connectivity to the bus 902 to execute instructions and process information stored in the memory 906. The processor 904 may include one or more processing cores with each core configured to perform independently. A multi-core processor enables multiprocessing within a single physical package. Examples of a multi-core processor include two, four, eight, or greater numbers of processing cores. Alternatively, or in addition, the processor 904 may include one or more microprocessors configured in tandem via the bus 902 to enable independent execution of instructions, pipelining, and multithreading. The processor 904 may also be accompanied with one or more specialized components to perform certain processing functions and tasks such as one or more digital signal processors (DSP) 908, or one or more application-specific integrated circuits (ASIC) 910. A DSP 908 may be configured to process real-world signals (e.g., sound) in real time independently of the processor 904. Similarly, an ASIC 910 may be configured to performed specialized functions not easily performed by a general purposed processor. Other specialized components to aid in performing the functions described herein include one or more field programmable gate arrays (FPGA) (not shown), one or more controllers (not shown), or one or more other special-purpose computer chips.
As used herein, the term “processor” refers to a computational element that is operable to respond to and processes instructions that drive the system. Optionally, the processor includes, but is not limited to, a microprocessor, a microcontroller, a complex instruction set computing (CISC) microprocessor, a reduced instruction set (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or any other type of processing circuit. Furthermore, the term “processor” may refer to one or more individual processors, processing devices and various elements associated with a processing device that may be shared by other processing devices. Additionally, the one or more individual processors, processing devices and elements are arranged in various architectures for responding to and processing the instructions that drive the system.
The processor 904 and accompanying components have connectivity to the memory 906 via the bus 902. The memory 906 includes both dynamic memory (e.g., RAM, magnetic disk, writable optical disk, etc.) and static memory (e.g., ROM, CD-ROM, etc.) for storing executable instructions that when executed perform the acts described herein for identification of a landing site for aerial vehicles. In particular, the memory 906 includes the instructions related to the map processing platform 102 (including one or more of the modules 202-212 and/or services platform 108, services 108a, 108b, and content providers 110a, 110b) to perform acts related to identification of a landing site for aerial vehicles. The memory 906 also stores the data associated with or generated by the execution of the acts.
The present disclosure provides system and method to identify potential sites for proper landing and take-off of aerial vehicles (VTOLs, delivery drones, etc.) with minimal manual effort. The present system and method may help with initial feasibility checks with low cost 3-channel satellite images such as those on Google Maps. If an area is suitable for UAV or VTOL landing, further feasibility checks are carried out using drone images and video. Herein, the neighborhood is surveyed to identify the potential landing site in consideration of factors such as transport connectivity, electrical supply, obstacles, emergency landing sites. This provides comprehensive information to decide if a landing site may be constructed. The present system and method utilize deep learning models to infer useful metrics of landing station infrastructure from aerial images. Polygon regression models are implemented to identify buildings from aerial images to obtain accurate geometry. This way the present disclosure may also be used to infer urban sprawl, scope for expansion of buildings, land encroachments, etc. It may be appreciated that the proposed method is scalable to large regions with minimal human effort.
The system and method of the present disclosure may be implemented for creating a network of landing pads/bays based on aerial imagery, ground data and artificial intelligence. Further, the present system and method may be implemented to evolve this network automatically based on new constructions/improvement to existing infrastructure. The proposed solution is designed to significantly reduce the effort required to solve the problem of identifying infrastructural sites for aerial vehicles and delivery drones. The key problems solved are detection of potential sites to land and take-off aerial vehicles, determining feasibility of these sites in terms of road connectivity and traffic conditions, detection of potential sites for construction of battery recharging infrastructure, and reducing the human effort significantly by automating the process.
It is to be understood that the elements and features recited in the appended claims may be combined in different ways to produce new claims that likewise fall within the scope of the present disclosure. Thus, whereas the dependent claims appended below depend from only a single independent or dependent claim, it is to be understood that these dependent claims may, alternatively, be made to depend in the alternative from any preceding or following claim, whether independent or dependent, and that such new combinations are to be understood as forming a part of the present specification.
While the present disclosure has been described in detail with reference to certain embodiments, it should be appreciated that the present disclosure is not limited to those embodiments. In view of the present disclosure, many modifications and variations would be present themselves, to those skilled in the art without departing from the scope of the various embodiments of the present disclosure, as described herein. The scope of the present disclosure is, therefore, indicated by the following claims rather than by the foregoing description. All changes, modifications, and variations coming within the meaning and range of equivalency of the claims are to be considered within their scope.
Sudhakaran, Vinay, Arrabotu, Sheetal Reddy, Sagar, Hrishikesh, Yeshwanth, Chandan
Patent | Priority | Assignee | Title |
11829447, | Sep 29 2020 | BEIJING BAIDU NETCOM SCIENCE TECHNOLOGY CO., LTD. | Resident area prediction method, apparatus, device, and storage medium |
Patent | Priority | Assignee | Title |
20140214222, | |||
20160299506, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Feb 17 2020 | ARRABOTU, SHEETAL REDDY | SIEMENS TECHNOLOGY AND SERVICES PVT LTD | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 056464 | /0416 | |
Feb 18 2020 | SAGAR, HRISHIKESH | SIEMENS TECHNOLOGY AND SERVICES PVT LTD | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 056464 | /0416 | |
Aug 19 2020 | Siemens Aktiengesellschaft | (assignment on the face of the patent) | / | |||
Feb 16 2021 | YESHWANTH, CHANDAN | SIEMENS TECHNOLOGY AND SERVICES PVT LTD | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 056464 | /0416 | |
Feb 18 2021 | SUDHAKARAN, VINAY | SIEMENS TECHNOLOGY AND SERVICES PVT LTD | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 056464 | /0416 | |
May 19 2021 | SIEMENS TECHNOLOGY AND SERVICES PVT LTD | Siemens Aktiengesellschaft | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 056464 | /0441 |
Date | Maintenance Fee Events |
Aug 19 2020 | BIG: Entity status set to Undiscounted (note the period is included in the code). |
Date | Maintenance Schedule |
Jul 19 2025 | 4 years fee payment window open |
Jan 19 2026 | 6 months grace period start (w surcharge) |
Jul 19 2026 | patent expiry (for year 4) |
Jul 19 2028 | 2 years to revive unintentionally abandoned end. (for year 4) |
Jul 19 2029 | 8 years fee payment window open |
Jan 19 2030 | 6 months grace period start (w surcharge) |
Jul 19 2030 | patent expiry (for year 8) |
Jul 19 2032 | 2 years to revive unintentionally abandoned end. (for year 8) |
Jul 19 2033 | 12 years fee payment window open |
Jan 19 2034 | 6 months grace period start (w surcharge) |
Jul 19 2034 | patent expiry (for year 12) |
Jul 19 2036 | 2 years to revive unintentionally abandoned end. (for year 12) |