The quality of service one may expect from a given network is often hard to determine in advance. Further, given agreements between network service providers and a variety of content providers, the complexity inherent in selecting a given available network is increasing for consumers. Embodiments described herein provide mechanisms for creating an augmented reality presentation of networks and the service quality of those networks. The presentation may display information regarding a variety of factors including content transit agreements with content providers, quality-of-service or prioritization guarantees for certain types of content, or low-level Wi-Fi signal quality. These presentations allow users to ascertain where in a given spatial area good quality is available for whichever services they intend to use.
|
1. A method performed by an augmented reality display device, the method comprising:
determining a location of the augmented reality display device;
using the determined location to retrieve a record of at least a first proximate network access point, wherein the retrieved record identifies (i) at least one first service benefit provided by a first content provider at the first network access point and (ii) a first region in which a signal from the first access point is expected to be available; and
displaying by the augmented reality display device at least a first graphical overlay registered to the first region, the first graphical overlay including an indication of the first service benefit.
18. An augmented reality display device comprising:
an augmented reality display;
a location determination module operative to determine a location of the augmented reality display device;
a network communication interface;
a processor;
a non-transitory computer readable medium storing instructions operative, when executed on the processor, to perform operations comprising:
using the determined location to retrieve a record of at least a first proximate network access point, wherein the retrieved record identifies (i) at least one service benefit provided by a first content provider at the first network access point and (ii) a first region in which a signal from the first access point is expected to be available; and
displaying on the augmented reality display at least a first graphical overlay registered to the first region, the first graphical overlay including an indication of the service benefit.
19. A method performed by an augmented reality display device, the method comprising:
determining a location of the augmented reality display device;
using the determined location to retrieve a record of at least a first proximate network access point, wherein the retrieved record identifies (i) at least one first service benefit relating to a networked streaming content service provided by a first content provider by way of the first network access point and (ii) a first region in which a signal from the first network access point is expected to be available, wherein the retrieved record further identifies at least one network-level attribute relating to network quality associated with the first network access point; and
displaying by the augmented reality display device a 3d perspective rendering of at least a first spatial volume registered to the first region, the first spatial volume including an indication of the first service benefit.
2. The method of
3. The method of
4. The method of
5. The method of
6. The method of
7. The method of
8. The method of
9. The method of
12. The method of
13. The method of
14. The method of
using the determined location to retrieve a record of at least a second proximate network access point, wherein the retrieved record identifies (i) at least one second service benefit provided by a second content provider at the second network access point and (ii) a second region in which a signal from the second access point is expected to be available; and
displaying by the augmented reality display device at least a second graphical overlay registered to the second region, the second graphical overlay including an indication of the second service benefit.
15. The method of
16. The method of
17. The method of
20. The method of
determining whether at least a portion of the first region is within a current field of view of the augmented reality display device, wherein the displaying of the first spatial volume is performed only after a determination that at least a portion of the first region is within the current field of view, and
wherein the at least one first service benefit comprises service benefit agreement information from a service benefit agreement between the first content provider and a network service provider.
|
The present application is a national stage application under 35 U.S.C. 371 of International Application No. PCT/US2016/061378, entitled SYSTEM AND METHOD FOR USING AUGMENTED REALITY TO VISUALIZE NETWORK SERVICE QUALITY, filed on Nov. 10, 2016, which claims benefit under 35 U.S.C. § 119(e) from, U.S. Provisional Patent Application Ser. No. 62/256,566, entitled “System and Method for Using Augmented Reality to Visualize Network Service Quality”, filed Nov. 17, 2015, the entirety of which is incorporated herein by reference.
Users increasingly access a variety of both paid and unpaid network services via a variety of network access point providers. For example, a library may provide a wireless network access point that a visitor can use to listen to streaming music, or a café may provide a wireless network access point that a customer can use to view streaming video. A variety of factors determine the overall quality of the experience that these consumers receive. These factors have effects at all layers of the networking stack. For example, at the physical layer, factors include Wi-Fi received signal strength (RSS) and channel crowding, or 4G signal strength. At the network layer, the factors include jitter, latency, and overall bandwidth. At the application layer, different service benefits may be available to customers of and/or visitors to particular establishments. With the variety of factors that can influence a user's online experience, it can be difficult for users to make informed decisions on where to go to find a network access point that meets their needs.
Systems and methods described herein provide users the ability to locate wireless network access points that may provide high-quality services desired by the users, whether based on performance, cost, or other factors. Some embodiments of the disclosed systems and methods provide the ability for users to locate these networks in context, based on their own current location and needs.
Exemplary embodiments described herein allow end-user consumers to discover locations where the end user may receive a desired level of service for a given content provider. This process is facilitated by an Augmented Reality (AR) presentation that allows the user to scan his or her spatial environment and to see graphical indicators of the quality of service that a user may expect in certain regions overlaid on the user's view of the real environment.
In some embodiments, different graphical indicators may be used for different indications for signal quality. For example, color may be used to indicate overall network quality (e.g. red indicates poor reception, while green indicates exceptional reception), while level of opacity may indicate cost (solid regions may indicate free service, while translucent regions may indicate that users have to pay). Additional graphical indicators, such as icons, may represent different services (e.g., a service logo on an area where a streaming video service has a partnering relationship with a network service provider, for instance, or a star rating for crowd-sourced ratings of particular networks). A variety of such mappings are available in different embodiments. The displayed information allows users to quickly scan the area around them to identify areas in which they are likely to receive improved quality of service.
In some embodiments, a method is performed by an augmented reality display device. In an exemplary method, the device determines its location. Using the determined location, the device retrieves a record of at least a first proximate network access point, wherein the retrieved record identifies (i) at least one service benefit provided by a first content provider at the first network access point and (ii) a first region in which a signal from the first access point is expected to be available. The device displays at least a first graphical overlay registered to the first region, with the first graphical overlay including an indication of the service benefit.
In some such embodiments, a step is performed of determining whether at least a portion of the first region is within a current field of view of the augmented reality display device, wherein the displaying of the first graphical overlay is performed only after a determination that at least a portion of the first region is within the current field of view.
In some such embodiments, the method includes selecting a color based on the service benefit, wherein the indication of the service benefit is provided by displaying the first graphical overlay with the selected color. In some embodiments, where the retrieved record further identifies a signal strength associated with the first network access point, the method further includes selecting a level of opacity based on the signal strength, wherein the first graphical overlay is displayed with the selected opacity. In some embodiments, the retrieved record further identifies a signal strength associated with the first network access point, the method further comprising selecting a color based on the signal strength, wherein the first graphical overlay is displayed with the selected color. In some embodiments, the retrieved record further identifies a bandwidth associated with the first network access point, the method further comprising selecting a level of opacity based on the bandwidth, wherein the first graphical overlay is displayed with the selected opacity. In some embodiments, the retrieved record further identifies a signal strength associated with the first network access point, the method further comprising selecting a level of color saturation based on the signal strength, wherein the first graphical overlay is displayed with the selected color saturation. In some embodiments, the retrieved record further identifies a bandwidth associated with the first network access point, the method further comprising selecting a level of color saturation based on the bandwidth, wherein the first graphical overlay is displayed with the selected color saturation.
In some embodiments, the graphical overlay includes a callout bubble identifying the service benefit. In some embodiments, the first graphical overlay is a spatial volume. In some embodiments, the first graphical overlay is a point cloud.
In exemplary embodiments, the service benefit identified in the record may be, for example, discounted or priority access to content provided by the first content provider.
In a further exemplary embodiment, the determined location of the device is used to retrieve a record of at least a second proximate network access point, wherein the retrieved record identifies (i) at least one second service benefit provided by a second content provider at the second network access point and (ii) a second region in which a signal from the second access point is expected to be available. The augmented reality display device then displays at least a second graphical overlay registered to the second region, where the second graphical overlay including an indication of the second service benefit.
Further described herein are exemplary augmented reality devices, including an augmented reality device comprising: an augmented reality display; a location determination module operative to determine a location of the augmented reality display device; a network communication interface; a processor; and a non-transitory computer readable medium storing instructions operative, when executed on the processor, to perform operations comprising: using the determined location to retrieve a record of at least a first proximate network access point, wherein the retrieved record identifies (i) at least one service benefit provided by a first content provider at the first network access point and (ii) a first region in which a signal from the first access point is expected to be available; and displaying on the augmented reality display at least a first graphical overlay registered to the first region, the first graphical overlay including an indication of the service benefit.
In some exemplary methods a graphic overlay is provided on an augmented reality display, where the graphic overlay indicates a location of a wireless network including a graphic indication of a service benefit available on the wireless network. The service benefit may be a benefit provided by a content provider, such as enhanced access, discounted access (including free access), or priority access to content provided by a content provider. The graphic indication of the service benefit may include a selected color associated with a content provider providing the benefit. The graphic indication of the service benefit may include a callout bubble identifying the benefit. The callout bubble may identify a content provider providing the benefit.
The graphic overlay may take various form. In various embodiments, the graphic overlay may be a point cloud, a cube or other polyhedron, or a region having a level of opacity or a level of color saturation, among other alternatives. In embodiments using a level of opacity, the level may be used to indicate one or more of a variety of factors. For example, the level of opacity or level of color saturation may be determined based on signal strength, on cost of access, on bandwidth, or on other factors.
In some embodiments, an augmented reality apparatus is provided that is operative to provide, on an augmented reality display, a graphic overlay indicating a location of a wireless network including a graphic indication of a service benefit available on the wireless network.
In some embodiments, an augmented reality network discovery method is provided. In some such methods, at least the following steps are performed: determining a location of a user; identifying a location proximate to the user at which access to at least one wireless network is available; identifying a network service provider associated with the wireless network; identifying at least one content provider providing a service benefit through the network service provider; and rendering a visual indication of the service benefit as an augmented reality overlay on the identified location. The identification of the location at which access to the wireless network is available includes checking a database for wireless access points near the location of the user. The determining of the location of a user may be performed using the global positioning system.
In some embodiments, a method is provided for visualizing an available network service and a quality of service thereof. In some such methods, at least the following steps are performed: determining a plurality of wireless network access points each having expected signal availability in a spatial volume within a determined present field of view of an augmented reality display device; determining wireless connectivity providers associated with each wireless network access point of the plurality of wireless network access points each having expected signal availability in a spatial volume within a determined present field of view; presenting, using the augmented reality display device, a plurality of visual indications of information services available via at least some of the wireless connectivity providers associated with each wireless network access point of the plurality of wireless network access points each having expected signal availability in a spatial volume within a determined present field of view of the augmented reality display device.
In some such embodiments, each visual indication is presented to appear to the user at a location where expected signal availability for the associated wireless connectivity provider is available. A determination may be made that a signal is expected to be available when signal strength is above a threshold.
In some embodiments, a method is provided for visualizing available network service and its quality of service. In some such methods, at least the following steps are performed: collecting data from service benefit agreements and partnerships between content providers and network service providers (NSPs); collecting data on network-level attributes in the physical environment of a location of interest; computing the overall service quality for a location of interest; and displaying an indication of the overall service quality as an augmented reality overlay on the region of interest.
In some embodiments, a method is provided comprising: capturing a perspective viewpoint, which may be the viewpoint of an augmented reality headset; receiving network information; and displaying a graphic overlay on the perspective viewpoint representing the network information. The graphic overlay may be displayed over the perspective viewpoint using an augmented reality head-mounted display of the augmented reality headset.
The perspective viewpoint may be captured using a camera, such as a forward-facing camera on an augmented reality display device. A network region in the viewpoint may be identified using a visual fiducial, a near-field communication (NFC) beacon, smart floor tiles, GPS positioning, a range finder, WiFi positioning, and/or other techniques.
In some such embodiments, the graphic overlay comprises a set of graphic indicators. The graphic indicators may be dynamically adjusted in response to a user request. The graphic indicators may be preconfigured by a user. The set of graphic indicators may comprise region shapes (cubes, clouds, spheres, etc.). In various embodiments, the set of graphic indicators may comprise region boundary lines, colors, opacity levels, message bubbles, and/or symbols.
In some such embodiments, the network information comprises user reviews, such as a review of a physical location corresponding to a respective network region. In some embodiments, the network information comprises information regarding one or more service benefit agreements between content providers and network service providers. In some embodiments, the network information comprises network-level attribute information, which may be collected via crowdsourcing. The network-level attribute information may include information such as measured bandwidth, an NSP ID, and/or received signal strength (RSS) information. In some embodiments, an overall service quality is calculated for each of the at least one network regions based on the network information.
In some embodiments, an apparatus is provided, with the apparatus comprising: a perspective acquisition device configured to capture a perspective viewpoint; a processor configured to receive network information; and a rendering module configured to generate a network visualization, and to project the network visualization on the perspective viewpoint.
Increasingly, factors other than low-level networking aspects may determine the quality that a consumer may expect to receive from a network service. For example, through partnerships with content providers, network service providers (NSPs) may promote or prioritize certain content, allowing the prioritized content to transit their networks at a higher priority (and thus with greater performance) than other content. In other cases, NSPs may allow use of certain content providers without counting this use against data plans or other caps. As one example, T-Mobile has partnerships with a variety of content providers to allow streaming music applications—which would normally use large amounts of bandwidth—to not count against users' data plans. In other cases, users who are customers of a given service may have free or discounted access on networks that may otherwise require payment (e.g. users of certain tiers of AT&T wireless may receive free access to AT&T hotspots). Such partnerships greatly affect the received customer service of consumers who use these networks and provide an incentive for them to return to places where these service benefit agreements are in effect.
It may be difficult for consumers to ascertain which places provide a desired level of service for content that the users wish to access. This is a complex decision process that hinges on knowledge of these service benefit agreements and knowledge of the physical location of the networks.
Currently, information used to assess the potential received quality for a given service is buried. The information is hidden on the websites of various content providers, service providers, and network providers. While the services a user does business with may attempt to keep the user informed of the possibilities (e.g., regular emails from Netflix about their partnerships with a given NSP), these attempts may be difficult for the user to keep track of. Further, when in a mobile setting, users may have no idea where the NSPs have a presence. Thus, it is desirable to provide users with an understanding of how various factors affecting service quality come together such that one NSP may provide a better customer experience than another. It is further desirable to enable users wishing to use a given service to physically locate a given NSP's network in order to take advantage of the service. While there are maps showing locations of wireless networks in the physical world, these maps do not generally provide information about advantages of services that these networks may provide.
Embodiments described herein allow end-user consumers to discover locations with a desired level of service for a given content provider. This process is facilitated by an AR presentation that allows the user to scan his or her spatial environment, and see graphical indicators of the quality of service that a user may expect in certain regions overlaid atop the reality view. Embodiments described herein involve several steps, and each may be performed in a variety of ways.
In an exemplary embodiment, data is assimilated from service benefit agreements and partnerships and used with an augmented reality presentation. This data may be collected from a variety of sources. In some embodiments, data may be provided by NSPs or content providers themselves. In some embodiments, data may be collected and provided by third party entities (such as the various “hotspot finder” applications and services known in the art). In some embodiments, data may be crowd-sourced, with multiple independent users collecting and providing data on service quality in various locations. This data assimilation may be periodically updated as service benefit agreements change.
Data may further be collected from network-level assessments of service quality. This data may include information such as overall signal strength in a given area, or bandwidth limitations on a given network. These network-level assessments may be used in some embodiments to supplement the service-level assessments. For example, if a given NSP has a partnership with a streaming video provider, but at a particular site the bandwidth is too limited to use the associated streaming video services effectively, the overall service quality will still be poor. In some embodiments, such network-level assessment data is collected via crowdsourcing.
In some embodiments, a metric of overall service quality is computed along a number of dimensions, based on data collected regarding network-level assessments and service benefits provided through service benefit agreements between NSPs and content providers. Information regarding the service quality may then be displayed in an augmented reality presentation on a mobile device.
The collection of data about service benefit agreements between content providers and NSPs may be performed in various ways. Collected data may be stored as a table with network providers and content providers along the two axes, and each intersection providing details about the service benefit agreements between that pair of entities. Table 1 illustrates an example:
TABLE 1
AT&T
T-Mobile
Boingo
Netflix
Prioritized service
—
—
Apple Music
—
Not charged against
—
data plan
Spotify
—
—
Ad-free
Amazon
—
—
Free access from
this network
A dash (“-”) in Table 1 indicates that no service benefit agreement exists for that pair. Cell contents indicate user-salient attributes of accessing a given service from a network service provider (NSP) along the top axis. While the cell contents here are provided in English, some embodiments may use a computer-readable taxonomy of symbols representing different service attributes. In some embodiments, collecting data about service benefit agreements between content providers and NSPs includes receiving data directly, e.g., from a NSP or a third party application. In some embodiments, data may be stored in a database of current service benefit agreements between content providers and NSPs. In some embodiments, a database may be represented as a table with two columns (“location” and “additional agreements”) listing any special service benefit agreements that exist at that specific location. In some embodiments, collecting service benefit agreement data includes receiving data via crowdsourcing. In some embodiments, crowd-sourcers may flag additional service benefit agreements about a given NSP in their collected data while using the given NSP, or while being in range of a given NSP. In some embodiments, content providers may have service benefit agreements with NSPs (e.g., AT&T or Google). In some embodiments, content providers may have a service benefit agreement with a given access-point provider (APP). For example, a store or coffee shop owner such as Starbucks may have such service benefit agreements.
In some embodiments, the collection of data regarding service benefit agreements may utilize databases associating SSIDs with NSPs and latitude/longitude locations in order to assist in determining a service benefit agreement with a given content provider. In some embodiments, the database may include network information (such as network-level attributes and service benefit agreement information) that is associated with a network region. In some embodiments, a service benefit agreement may be determined based on a given NSP and a given content provider (e.g., AT&T and Netflix). In some embodiments, an APP may have a service benefit agreement directly with a content provider for a single location, or in other cases a general service benefit agreement for all locations (e.g., Starbucks may have a Spotify agreement at the location in Table 2, or all Starbucks locations may have Spotify agreements). In some embodiments, a NSP database may associate general service benefit agreements between a given NSPs with content providers. In some embodiments, an APP database may associate general service benefit agreements between a given APP. In some embodiments, a content provider may associate location-specific service benefit agreements with NSPs and APPs.
A sequence diagram illustrating a process 100 for data collection is shown in
As multiple users 102 execute this process 100, the database 108 builds up a map of the networks, and network quality, in a region. For some embodiments, this process 100 is performed by a separate, stand-alone application. For other embodiments, this process 100 is performed during the viewing of collected data. That is, in some embodiments, a collection process 100 occurs simultaneously with a viewing process, resulting in a self-correcting update of a network attributes database as users 102 use an application.
Exemplary embodiments include a process 100 of collecting data on network-level attributes in a physical environment at various locations (sometimes represented by latitudes (lat) and longitudes (long)). This process may include the existence of certain networks at a given location (“at <lat, long> there exists an AT&T hotspot”) as well as potential performance assessments of the given network (“signal strength is weak for network SSID <att_hotspot> at <lat, long>” or “bandwidth max is 1.5 mbps for network SSID <att_hotspot>”). In some embodiments, this data may be collected in a crowd-sourced manner.
Table 2 illustrates an exemplary output of network-level data collection, with each row indicating a given reading collected through the process 100. Columns indicate observed SSID and various other attributes of the network quality:
TABLE 2
Measured
NSP/APP
Location
SSID
RSS
Bandwidth
ID
Lat = 34.771899
att_hotspot
−54
0.71
AT&T
Long = −86.732941
dBm
Mbps
Lat = 35.12258
starbucks
−62
8.5
Starbucks
Long = −91.83992
dBm
Mbps
Table 2 may be seeded with existing databases (such as from Place Lab), which maintain geospatial locations of SSIDs. See LaMarca, Anthony, et al., Place Lab: Device Positioning Using Radio Beacons in the Wild, P
In some embodiments, network metric data may be collected for given SSIDs via crowd-sourcing. In such embodiments, various readings of network metrics such as RSS and bandwidth may be made by a user connected to a given SSID/location combination (automatically or manually) and uploaded to a crowd-sourced database containing various crowd-sourced metric readings for the SSID/location combination. The various crowd-sourced metric readings may be used to create network-quality point clouds. For some embodiments, an intensity graphic (such as via a point cloud, a level of opacity, or a level of color saturation) may be used to display a network-level attribute for a network region. Such network-level attributes may include signal strength and bandwidth. For some embodiments, such an intensity graphic may be dynamically adjusted in response to a user request. In some embodiments, there may be an aggregation step to optimize a crowd-sourced database, for example, by eliminating duplicate measurements or nearby measurements, or representing coarser-grained changes in quality. In some embodiments, some measurements may be discarded if they do not meet certain network quality requirements, as described below. In embodiments where at least two networks have the same SSID name, a location may be associated with each network in order to distinguish the networks. There are a variety of established techniques for creating so-called “spatial indexes” for a database that contains geo-spatial information. For searches, a spatial index may be used instead of the SSID or specific latitude/longitude coordinates. For example, a technique called R-Tree is one of the most widely-used spatial indexing algorithms, and the technique allows efficient performance of queries (such as, “select all network measurements within 100 meters of user's current location”) when a database contains collections of specific coordinates. See Guttman, A., R-Trees: A Dynamic Index Structure for Spatial Searching, Proceedings of the 1984 ACM SIGMOD International Conference on the Management of Data (1984).
A computation may be performed to determine a metric of the overall service quality along a number of dimensions, based on the data collected regarding service benefit agreements and network-level quality. In some embodiments, a “natural join” operation is performed to combine data from the service agreements table (Table 1) with data from the network assessment table (Table 2), resulting in a collation of data about individual networks, the services available on them, and their performance at specific locations in the physical world.
For example, for service table S (Table 1) and network table N (Table 2), the result SN (where is the natural join operator) results in the following Table 3:
TABLE 3
Measured
Apple
Location
SSID
RSS
Bandwidth
NSP/APP
Netflix
Music
Spotify
Amazon
. . .
Lat = 34.771899
att_hotspot
−54
0.71 Mbps
AT&T
Prioritized
Ad-free
Long =
dBm
Access
−86.732941
Lat = 35.12258
starbucks
−62
8.5 Mbps
Starbucks
Prioritized
Long =
dBm
Access
−91.83992
. . .
For fast access, this data may be organized geospatially, allowing use of specific locations expressed as <latitude, longitude> as keys into the data. In this way, given a specific location, the database may contain details about nearby networks, service benefits available through such networks, and overall quality. In some embodiments, as described above, a given SSID may have numerous readings from various points within the network region. In such embodiments, the readings may be collected via crowd-sourcing, as described above. A network region is a geographic area served by a network. In some embodiments, based on service benefit agreements associated with certain SSIDs, heuristics may be used in determining overall network quality. For example, if an SSID has a service benefit agreement with a streaming video content provider, and if high quality streaming requires a bandwidth of 3.0 Mbps, a mobile device application may create a point cloud from the collected network metric readings of the SSID in which all points of the point cloud have a bandwidth of at least 3.0 Mbps, and all other points are discarded. Such embodiments may reduce clutter in the AR representation by effectively reducing the displayed region of the SSID from all areas in which the SSID is visible to only the areas of the SSID where the network is strong enough for the content associated with the network. In such embodiments, rather than a direct mapping of network quality measurements to graphical attributes, a more indirect presentation may be offered in which areas are shown as offering no access at all to the streaming video service (even though the SSID may be visible) unless the quality parameters for this category of service are met. In some embodiments, the visualization parameters of the mobile application are automatically configured based on the network quality requirements of a known content (e.g., content from a video streaming service provider or an audio streaming service provider). In other embodiments, a user 102 may configure the mobile application based on a custom set of network quality requirements.
In some embodiments, parameters such as signal strength and/or bit rate of a wireless network are measured at particular points (e.g. through crowd-sourcing), and the value of the parameters between those points may be determined using an interpolation and/or a curve fitting algorithm, such as Barnes interpolation, bilinear interpolation, bicubic interpolation, spline interpolation, or other techniques.
In exemplary embodiments, collected and processed data is displayed as an augmented reality overlay. A user's augmented reality device presents a visualization of this data based on a user's location in his or her environment. In some embodiments, the user's current location is collected from a mobile device application, and provided to the database as a key. In other embodiments, a spatial index as described above may be used as the key. Nearby network records are returned (e.g., rows from the database table above of networks which are physically proximate to the user are returned to the application). In some embodiments, a user may request a display of networks within a desired range of the user's current location.
In some embodiments, in order to represent differences in network quality over a small spatial region, individual measurements may be captured and—at least initially—stored in the database 108. This process allows a system to have information about changes in quality in small regions that may be presented to the user via the AR display. As described above, such databases may be formed via crowdsourcing. In alternative embodiments, each row of Table 3 may represent the network itself. Such information may be collected without crowdsourcing, but rather by using public databases of network locations (e.g., Wigle.net), which have estimated locations of all networks. In such embodiments, however, a user may be presented with the location of a given network in the AR display, along with its service benefit agreements, but without the more fine-grained information about quality fall-off in the spatial region around the network's estimated location.
In some embodiments, the records of either crowd-sourced databases or public databases of network locations are used to generate a graphical overlay on top of a camera view from a user's mobile device, such as a user's smartphone. In some embodiments, the visualization is sent to an external device, such as an AR head-mounted display (HMD) device, and the visualization provides a graphic overlay over the real world. This graphical overlay may be regenerated as the user's orientation changes (e.g., head tracking) so that from the perspective of the user, the graphical overlay remains registered spatially with the “reality” view from the user's camera or other device.
The data collected regarding network-level service quality may be made quite robust by integrating the network assessment into the usage of an application for display to a user. Because a system may determine the user's location during a display process, a system may perform a simultaneous network-level assessment that updates the data for that location. This, in effect, produces a self-correcting system for the network-level data, as usage of a system refines and improves its overall quality of data. As described above, when using crowd-sourced databases, some points may be discarded that do not meet network-quality parameters in order to reduce visual clutter in the AR visualization. In such embodiments, the network-quality parameters may be automatically assigned based on the content (e.g. streaming video, streaming audio, or other content) or manually set by the user.
Different visual properties of the augmented reality overlays may convey to a user various information regarding the respective network access point. In some embodiments, different overlay colors are used to represent information about the access points. In one example, different colors are used to represent different content providers from whom a service benefit is available at the respective network access point. A default color may be used for access points that are not associated with any service benefit. In another example, different colors are used to represent different expected signal strengths (e.g. as collected through crowd-sourcing). In a further example, different colors are used to represent different expected bandwidths (e.g. as measured through crowd-sourcing).
In some embodiments, different levels of overlay opacity are used to represent information about the access points. In one example, different levels of opacity are used to represent different expected signal strengths. In a further example, different levels of opacity are used to represent different expected bandwidths.
In some embodiments, different levels of color saturation are used to represent information about the access points. In one example, different levels of color saturation are used to represent different expected signal strengths. In a further example, different levels of color saturation are used to represent different expected bandwidths.
In the example of
As an example, an overlay associated with a network access point that provides a higher level of access to streaming video content from a particular content provider may be colored in purple. An overlay associated with a network access point that provides a higher level of access to streaming audio content from a particular content provider may be colored in green. With reference to
In some embodiments, the user may tap on or otherwise select the region to pull up additional information regarding the quality and/or service benefits of one or more network access points. In some embodiments, such additional information may be displayed by default.
Further illustrated in
In some embodiments, network parameters such as signal strength and/or bit rate are measured and mapped on a substantially two-dimensional plane. This may be done where, for example, network parameter measurements are carried out on a crowd-sourced basis by, for example, smartphones, which are typically in use between around 0.5 m and 2 m above the ground. Given a set of measurements of a network parameter at particular positions (e.g., particular x and y coordinates, where x and y may be latitude and longitude or other coordinates), estimated parameter values between points of measurement may be determined using interpolation, surface fitting, or other techniques. A two-dimensional region may be selected in which the estimated or measured parameter is above a threshold (e.g., where the network parameter is bit rate, the threshold may be 1 Mbps, 3 Mbps, or some other value). The two-dimensional above-threshold region may be converted to a three-dimensional display region by, for example, extruding the two-dimensional above-threshold region to a predetermined elevation (such as 3 m, or some other value). The three-dimensional display region is displayed to the user on an augmented reality device. A displayed color of the display region may be selected to correspond to a content provider that provides a service benefit (e.g. priority or free connectivity) on the associated network. For example, the display region may be colored purple to indicate that the network service provider has a service benefit agreement with a particular streaming video provider, or green to indicate that the network service provider has a service benefit agreement with a particular streaming audio provider. A displayed opacity of the display region may be selected to indicate another parameter, such as a cost of connectivity, signal strength, or bit rate.
In some embodiments, size and position of the displayed regions may be based on collected measurements (e.g., the crowd-sourced database). In such embodiments, the boundary of the region may represent the start of the region in which network quality passes some threshold which may be dependent on network quality metrics associated with the type of the service, as described above. In some embodiments, for very common networks, the region may represent the spatial areas in which network quality exceeds some threshold. This threshold may be set by the user or set autonomously by a system as a parameter. In some embodiments in areas with poor network density, it may be desirable to display even networks with relatively poor signal quality on the theory that such networks are preferable to no network at all.
Different network attributes may be mapped using various different graphical attributes. In embodiments described above, different service benefits are mapped to different overlay colors and performance is mapped to opacity, but any of the network attributes above may be mapped onto graphical attributes including color, opacity, texture, size, shape, ornamentation (graphical icons such as the star), and more. In some embodiments, a user may configure custom mapping settings via a user interface (UI) presented by an application. In some embodiments, additional options may allow the user to filter based on content provider (for example, only access points with a service benefit related to streaming video) or other attributes.
In exemplary embodiments, the graphical overlays retain the 3D perspective of the real environment. For example, overlays for signal areas that are farther away are rendered in a smaller size. In some embodiments, as the user moves or reorients the mobile device, the display shifts, revealing hidden detail or allowing “zooming” on a region by getting closer to it.
In some embodiments, collection of network-level quality data is not performed. A mobile device may use publicly available geo-coded databases of hotspots, along with collected service-level data, to create a visualization of sites with certain service benefits. However, the combination of service-level data with the network-level data may create a more compelling solution, as multiple aspects of the received customer service quality may be visualized.
In some embodiments, other data is collected as a part of the process, such as user ratings, reviews of the physical sites where the networks are located (e.g., whether this café is pleasant enough to watch a two-hour streaming movie), and so forth.
In exemplary embodiments, GPS and Wi-Fi positioning are used to detect location. In some embodiments, other mechanisms are used, such as NFC beacons (such as iBeacons), visual fiducial markers detected via camera, or other indoor positioning systems such as smart floor tiles.
In the scope of AR, a variety of end-user visualizations may be used, such as using different colors to denote signal quality, or including interactive icons in the presentation (to allow, for instance, the user to explore a region based on service availability).
Described embodiments allow easy visualization of a complex, multi-dimensional space of variables that may affect service quality and the selection of which network to use. Unlike approaches that merely visualize signal strength, exemplary embodiments disclosed herein collect and present in an intuitive way information on multiple factors that may impact service selection.
Exemplary visualizations disclosed herein occur in context, (e.g., overlaid atop a live view of the spatial region the user is in), but do not require extensive physical site maps or blueprints. In some cases, networks may be detected that are not in the user's line of sight (e.g., behind a building or some other obstruction). In such embodiments, it may still be desirable to render these networks, as the user may wish to know that the network is there and walk to it. For some embodiments, to reduce visual clatter, networks may be displayed if they meet user-set parameters of minimum signal strength and/or maximum distance away.
The augmented reality presentation allows a user to activate an application, and “look around” a region, seeing areas of high and low service quality rendered over the top of the live view of the area. This presentation is “live” in the sense that, as the user moves through the environment or reorients the mobile device, he or she views additional signal data from the new perspective. This process allows experimentation and exploration of various networks and services available.
In some alternative embodiments, the information collected regarding network-level quality, regions of expected signal availability, and the presence (or absence) of particular service benefits available through different access points may be displayed in a two-dimensional map format rather than an augmented reality display. The two-dimensional map display may be made accessible electronically for the benefit of users who may not have access to an augmented reality display. Such a map may be displayed using, for example, a desktop or laptop computer. Regions of the map in which a signal is expected to be available for a particular access point may be colored in using a color selected based on a service benefit available at that access point.
In some embodiments, a mobile device taking the form of a wireless transmit-receive unit (WTRU) may be used as an augmented reality display device to carry out the above-mentioned methods.
The transmit/receive element 1122 may be configured to transmit signals to, or receive signals from, a node over the air interface 1115. For example, in one embodiment, the transmit/receive element 1122 may be an antenna configured to transmit and/or receive RF signals. In another embodiment, the transmit/receive element 1122 may be an emitter/detector configured to transmit and/or receive IR, UV, or visible light signals, as examples. In yet another embodiment, the transmit/receive element 1122 may be configured to transmit and receive both RF and light signals. It will be appreciated that the transmit/receive element 1122 may be configured to transmit and/or receive any combination of wireless signals.
In addition, although the transmit/receive element 1122 is depicted in
The transceiver 1120 may be configured to modulate the signals that are to be transmitted by the transmit/receive element 1122 and to demodulate the signals that are received by the transmit/receive element 1122. As noted above, the WTRU 1102 may have multi-mode capabilities. Thus, the transceiver 1120 may include multiple transceivers for enabling the WTRU 1102 to communicate via multiple RATs, such as UTRA and IEEE 802.11, as examples.
The processor 1118 of the WTRU 1102 may be coupled to, and may receive user input data from, the audio transducers 1124, the keypad 1126, and/or the display/touchpad 1128 (e.g., a liquid crystal display (LCD) display unit or organic light-emitting diode (OLED) display unit). The processor 1118 may also output user data to the speaker/microphone 1124, the keypad 1126, and/or the display/touchpad 1128. In addition, the processor 1118 may access information from, and store data in, any type of suitable memory, such as the non-removable memory 1130 and/or the removable memory 1132. The non-removable memory 1130 may include random-access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 1132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments, the processor 1118 may access information from, and store data in, memory that is not physically located on the WTRU 1102, such as on a server or a home computer (not shown).
The processor 1118 may receive power from the power source 1134, and may be configured to distribute and/or control the power to the other components in the WTRU 1102. The power source 1134 may be any suitable device for powering the WTRU 1102. As examples, the power source 1134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), and the like), solar cells, fuel cells, and the like.
The processor 1118 may also be coupled to the GPS chipset 1136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 1102. In addition to, or in lieu of, the information from the GPS chipset 1136, the WTRU 1102 may receive location information over the air interface 1115 from a base station and/or determine its location based on the timing of the signals being received from two or more nearby base stations. It will be appreciated that the WTRU 1102 may acquire location information by way of any suitable location-determination method while remaining consistent with an embodiment.
The processor 1118 may further be coupled to other peripherals 1138, which may include one or more software and/or hardware modules that provide additional features, functionality and/or wired or wireless connectivity. For example, the peripherals 1138 may include an accelerometer, an e-compass, a satellite transceiver, a digital camera (for photographs or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, an AR HMD, and the like.
In some embodiments, systems and methods described herein may be implemented in a networked server, such as server 1202 illustrated in
The processor 1218 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Array (FPGAs) circuits, any other type of integrated circuit (IC), a state machine, and the like. The processor 1218 may perform signal coding, data processing, power control, input/output processing, and/or any other functionality that enables the server 1202 to operate in a wired or wireless environment. The processor 1218 may be coupled to the network interface 1220. While
The processor 1218 of the server 1202 may be coupled to, and may receive user input data from, the keypad 1226, and/or the display 1228 (e.g., a liquid crystal display (LCD) display unit or organic light-emitting diode (OLED) display unit). The processor 1218 may also output user data to the display/touchpad 1228. In addition, the processor 1218 may access information from, and store data in, any type of suitable memory, such as the non-removable memory 1230 and/or the removable memory 1232. The non-removable memory 1230 may include random-access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. In other embodiments, the processor 1218 may access information from, and store data in, memory that is not physically located at the server 1202, such as on a separate server (not shown).
The processor 1218 may receive power from the power source 1234, and may be configured to distribute and/or control the power to the other components in the server 1202. The power source 1234 may be any suitable device for powering the server 1202, such as a power supply connectable to a power outlet.
Although features and elements are described above in particular combinations, one of ordinary skill in the art will appreciate that each feature or element may be used alone or in any combination with the other features and elements. In addition, the methods described herein may be implemented in a computer program, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer-readable storage media include, but are not limited to, a read only memory (ROM), a random access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs). A processor in association with software may be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.
Patent | Priority | Assignee | Title |
10649963, | Apr 11 2014 | Twitter, Inc. | Multi-tenancy management within a distributed database |
Patent | Priority | Assignee | Title |
8086245, | Sep 12 2002 | AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE LIMITED | Advertising and controlling the advertisement of wireless hot spots |
8140364, | Jan 16 2008 | International Business Machines Corporation | Method and system for the bundling and pricing of wireless hotspots |
8978105, | Jul 25 2008 | TRAPEZE NETWORKS, INC | Affirming network relationships and resource access via related networks |
9148823, | Jul 05 2006 | Nokia Technologies Oy | Ensuring quality of service for private short-range wireless networks |
9589372, | Jan 21 2016 | International Business Machines Corporation | Augmented reality overlays based on an optically zoomed input |
20030229478, | |||
20050228860, | |||
20070162942, | |||
20100242079, | |||
20120122514, | |||
20140195606, | |||
20150052247, | |||
EP1639851, | |||
EP2950181, | |||
EP2983138, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Nov 10 2016 | PCMS Holdings, Inc. | (assignment on the face of the patent) | / | |||
Aug 08 2018 | EDWARDS, KEITH | PCMS HOLDINGS, INC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 046833 | /0523 | |
Oct 27 2022 | PCMS HOLDINGS, INC | InterDigital VC Holdings, Inc | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 062900 | /0645 |
Date | Maintenance Fee Events |
May 11 2018 | BIG: Entity status set to Undiscounted (note the period is included in the code). |
Mar 14 2023 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
Date | Maintenance Schedule |
Sep 24 2022 | 4 years fee payment window open |
Mar 24 2023 | 6 months grace period start (w surcharge) |
Sep 24 2023 | patent expiry (for year 4) |
Sep 24 2025 | 2 years to revive unintentionally abandoned end. (for year 4) |
Sep 24 2026 | 8 years fee payment window open |
Mar 24 2027 | 6 months grace period start (w surcharge) |
Sep 24 2027 | patent expiry (for year 8) |
Sep 24 2029 | 2 years to revive unintentionally abandoned end. (for year 8) |
Sep 24 2030 | 12 years fee payment window open |
Mar 24 2031 | 6 months grace period start (w surcharge) |
Sep 24 2031 | patent expiry (for year 12) |
Sep 24 2033 | 2 years to revive unintentionally abandoned end. (for year 12) |