A system for generating a user interface described herein can include a processor to detect a type of the system based on hardware components residing within the system or coupled to the system and determine a user interface manager to execute based on the type of the system. The processor can also execute the user interface manager to generate a user interface for the system, wherein the type of the user interface manager comprises a plurality of rules to indicate a layout of the user interface.
|
16. A system for generating a user interface comprising:
a processor; and
a storage device to store a plurality of instructions that, in response to being executed by a processor, cause the processor to:
detect a type of the system based on non-display hardware components residing within the system;
select a first user interface manager from a visual tree of user interface managers to execute based on the type of the system, wherein the visual tree resides in a user interface host and the user interface managers correspond to different system types; and
execute the selected first user interface manager to generate a first operating-system user interface for displaying multiple applications by the system, wherein the selected first user interface manager comprises a first plurality of rules to indicate a layout of the first operating-system user interface and to prevent windows corresponding to the multiple applications from overlapping;
detect a change in a hardware component coupled to the system;
based on the detected change, select a second user interface manager from the visual tree of user interface mangers;
execute the selected second user interface manager to generate a second operating-system user interface for the system, wherein the second user interface manager maintains a state of the first operating-system user interface, and the second user interface manager comprises a second plurality of rules to indicate a layout of the second operating-system user interface, the second plurality of rules including second layout rules that allow windows corresponding to the multiple applications to overlap; and
morph from the first operating-system user interface to the second operating-system user interface.
10. A method for generating a user interface, comprising:
detecting a type of a system based on non-display hardware components residing within the system;
determining a first user interface manager, from a plurality of user interface managers each configured for a different system type, to execute based on the type of the system, wherein the plurality of user interface managers include:
the first user interface manager that includes a first plurality of layout rules that indicate a first layout for an operating system user interface and prevent windows corresponding to multiple applications from overlapping; and
a second user interface manager that includes a second plurality of layout rules that indicate a second layout for the operating system user interface and allow the windows to overlap; and
executing the first user interface manager to generate a first visual appearance for an operating system of the of the system and a user interface for displaying the multiple applications and an application launcher by the system according to the first layout and prevent the windows corresponding to the multiple applications from overlapping, wherein the first user interface manager further comprises a plurality of rules to indicate a layout of the user interface including an area of a screen of the system that is to be occupied by the application launcher and the multiple applications;
detecting a change in a hardware component coupled to the system;
based on the detected change to the system, select the second user interface manager;
executing the second interface manager to generate a second visual appearance for the operating system; and
transition from the from the first visual appearance to the second visual appearance, wherein transitioning includes an animation.
1. A system for generating a user interface, comprising:
a processor; and
memory storing instructions that, when executed by the processor, cause the system to perform operations comprising:
based on hardware components residing within the system or coupled to the system, detect a type of the system to be one of a mobile system, a gaming console system, an augmented reality system, a tablet system, or a desktop system;
determine a first user interface manager, from a plurality of user interface managers each configured for a different system type, to execute based on the type of the system, wherein the plurality of user interface managers include:
the first user interface manager that includes a first plurality of layout rules that indicate a first layout for an operating system user interface and prevent windows corresponding to multiple applications from overlapping; and
a second user interface manager that includes a second plurality of layout rules that indicate a second layout for the operating system user interface and allow the windows to overlap;
execute the first user interface manager to generate a first visual appearance for an operating system of the of the system and a user interface for displaying the multiple applications executed by the operating system according to the first layout and prevent the windows corresponding to the multiple applications from overlapping, wherein the first layout of the user interface is a two-dimensional representation of the applications and system controls;
detect a change in a hardware component coupled to the system;
based on the detected change to the system, select the second user interface manager;
execute the second interface manager to generate a second visual appearance for the operating system; and
morph from the first visual appearance to the second visual appearance.
2. The system of
3. The system of
4. The system of
5. The system of
6. The system of
7. The system of
8. The system of
9. The system of
11. The method of
12. The method of
13. The method of
14. The method of
15. The method of
17. The system of
18. The system of
20. The system of
|
Pursuant to 35 U.S.C. § 371, this application is the United States National Stage Application of International Patent Application No. PCT/US2017/038032, filed Jun. 16, 2017.
Computer devices of any size can generate a user experience, which is typically fixed to the intended design. Each device can separately generate a user interface based on fixed application functions. For example, each device can separately generate a user interface by hard coding or using a fixed format for displaying applications.
The following presents a simplified summary in order to provide a basic understanding of some aspects described herein. This summary is not an extensive overview of the claimed subject matter. This summary is not intended to identify key or critical elements of the claimed subject matter nor delineate the scope of the claimed subject matter. This summary's sole purpose is to present some concepts of the claimed subject matter in a simplified form as a prelude to the more detailed description that is presented later.
An embodiment described herein includes a system for generating a user interface that includes a processor to detect a type of the system based on hardware components residing within the system or coupled to the system. The processor can also determine a user interface manager to execute based on the type of the system and execute the user interface manager to generate a user interface for the system, wherein the user interface manager comprises a plurality of rules to indicate a layout of the user interface.
In another embodiment described herein, a method for generating a user interface can include detecting a type of a system based on hardware components residing within the system or coupled to the system. The method can also include determining a user interface manager to execute based on the type of the system and executing the user interface manager to generate a user interface for the system, wherein the user interface manager comprises a plurality of rules to indicate a layout of the user interface.
In yet another embodiment described herein, a system for generating a user interface can include a processor and a storage device to store a plurality of instructions that, in response to being executed by the processor, cause the processor to detect a type of the system based on hardware components residing within the system or coupled to the system and select a user interface manager from a visual tree of user interface managers to execute based on the type of the system, wherein the visual tree is to reside in a user interface host. The plurality of instructions can also cause the processor to execute the selected user interface manager to generate a user interface for the system, wherein the selected user interface manager comprises a plurality of rules to indicate a layout of the user interface.
The following description and the annexed drawings set forth in detail certain illustrative aspects of the claimed subject matter. These aspects are indicative, however, of a few of the various ways in which the principles of the innovation may be employed and the claimed subject matter is intended to include all such aspects and their equivalents. Other advantages and novel features of the claimed subject matter will become apparent from the following detailed description of the innovation when considered in conjunction with the drawings.
The following detailed description may be better understood by referencing the accompanying drawings, which contain specific examples of numerous features of the disclosed subject matter.
User interfaces can be generated using various static, non-reusable techniques. For example, user interfaces for different devices can be generated using different sets of functions, different data paths, and different visual compositions. Accordingly, applications often include different code to generate a user interface for each type of device. These applications also have deep context about the device on which they are running and often map user interface controls directly to pixel coordinates on a display device. For example, the applications may specify pixel coordinates to display a user control element such as a text field, among others.
Techniques described herein provide a system for generating a user interface based on rules. In some examples, the rules can correspond to a particular type of device. For example, the rules for generating a user interface for a desktop computing device can differ from the rules for generating a user interface for a mobile device. In some embodiments, a system can detect a type of the system based on hardware components residing within the system and determine a type of user interface manager to execute based on the type of the system. The system can also execute the user interface manager to generate a user interface for the system, wherein the user interface manager comprises a plurality of rules to indicate a layout of the user interface.
The techniques described herein enable code for generating user interfaces to be shared while creating user interfaces for any number of different devices. For example, the techniques described herein can enable shared code to generate a user interface for a desktop device, a tablet device, a mobile device, a phone device, a gaming console device, and an augmented reality device, among others. The shared code can generate different user interfaces based on rules corresponding to each type of device. For example, code for displaying a user application with certain rules can be shared between any number of different user interface managers corresponding to different types of devices.
As a preliminary matter, some of the figures describe concepts in the context of one or more structural components, referred to as functionalities, modules, features, elements, etc. The various components shown in the figures can be implemented in any manner, for example, by software, hardware (e.g., discrete logic components, etc.), firmware, and so on, or any combination of these implementations. In one embodiment, the various components may reflect the use of corresponding components in an actual implementation. In other embodiments, any single component illustrated in the figures may be implemented by a number of actual components. The depiction of any two or more separate components in the figures may reflect different functions performed by a single actual component.
Other figures describe the concepts in flowchart form. In this form, certain operations are described as constituting distinct blocks performed in a certain order. Such implementations are exemplary and non-limiting. Certain blocks described herein can be grouped together and performed in a single operation, certain blocks can be broken apart into plural component blocks, and certain blocks can be performed in an order that differs from that which is illustrated herein, including a parallel manner of performing the blocks. The blocks shown in the flowcharts can be implemented by software, hardware, firmware, and the like, or any combination of these implementations. As used herein, hardware may include computer systems, discrete logic components, such as application specific integrated circuits (ASICs), and the like, as well as any combinations thereof.
As for terminology, the phrase “configured to” encompasses any way that any kind of structural component can be constructed to perform an identified operation. The structural component can be configured to perform an operation using software, hardware, firmware and the like, or any combinations thereof. For example, the phrase “configured to” can refer to a logic circuit structure of a hardware element that is to implement the associated functionality. The phrase “configured to” can also refer to a logic circuit structure of a hardware element that is to implement the coding design of associated functionality of firmware or software. The term “module” refers to a structural element that can be implemented using any suitable hardware (e.g., a processor, among others), software (e.g., an application, among others), firmware, or any combination of hardware, software, and firmware.
The term “logic” encompasses any functionality for performing a task. For instance, each operation illustrated in the flowcharts corresponds to logic for performing that operation. An operation can be performed using software, hardware, firmware, etc., or any combinations thereof.
As utilized herein, terms “component,” “system,” “client” and the like are intended to refer to a computer-related entity, either hardware, software (e.g., in execution), and/or firmware, or a combination thereof. For example, a component can be a process running on a processor, an object, an executable, a program, a function, a library, a subroutine, and/or a computer or a combination of software and hardware. By way of illustration, both an application running on a server and the server can be a component. One or more components can reside within a process and a component can be localized on one computer and/or distributed between two or more computers.
Furthermore, the claimed subject matter may be implemented as a method, apparatus, or article of manufacture using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof to control a computer to implement the disclosed subject matter. The term “article of manufacture” as used herein is intended to encompass a computer program accessible from any tangible, computer-readable device, or media.
Computer-readable storage media can include but are not limited to magnetic storage devices (e.g., hard disk, floppy disk, and magnetic strips, among others), optical disks (e.g., compact disk (CD), and digital versatile disk (DVD), among others), smart cards, and flash memory devices (e.g., card, stick, and key drive, among others). In contrast, computer-readable media generally (i.e., not storage media) may additionally include communication media such as transmission media for wireless signals and the like.
The system bus 108 couples system components including, but not limited to, the system memory 106 to the processing unit 104. The processing unit 104 can be any of various available processors. Dual microprocessors and other multiprocessor architectures also can be employed as the processing unit 104.
The system bus 108 can be any of several types of bus structure, including the memory bus or memory controller, a peripheral bus or external bus, and a local bus using any variety of available bus architectures known to those of ordinary skill in the art. The system memory 106 includes computer-readable storage media that includes volatile memory 110 and nonvolatile memory 112.
In some embodiments, a unified extensible firmware interface (UEFI) manager or a basic input/output system (BIOS), containing the basic routines to transfer information between elements within the computer 102, such as during start-up, is stored in nonvolatile memory 112. By way of illustration, and not limitation, nonvolatile memory 112 can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
Volatile memory 110 includes random access memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), SynchLink™ DRAM (SLDRAM), Rambus® direct RAM (RDRAM), direct Rambus® dynamic RAM (DRDRAM), and Rambus® dynamic RAM (RDRAM).
The computer 102 also includes other computer-readable media, such as removable/non-removable, volatile/non-volatile computer storage media.
In addition, disk storage 114 can include storage media separately or in combination with other storage media including, but not limited to, an optical disk drive such as a compact disk ROM device (CD-ROM), CD recordable drive (CD-R Drive), CD rewritable drive (CD-RW Drive) or a digital versatile disk ROM drive (DVD-ROM). To facilitate connection of the disk storage devices 114 to the system bus 108, a removable or non-removable interface is typically used such as interface 116.
It is to be appreciated that
System applications 120 take advantage of the management of resources by operating system 118 through program modules 122 and program data 124 stored either in system memory 106 or on disk storage 114. It is to be appreciated that the disclosed subject matter can be implemented with various operating systems or combinations of operating systems.
A user enters commands or information into the computer 102 through input devices 126. Input devices 126 include, but are not limited to, a pointing device, such as, a mouse, trackball, stylus, and the like, a keyboard, a microphone, a joystick, a satellite dish, a scanner, a TV tuner card, a digital camera, a digital video camera, a web camera, any suitable dial accessory (physical or virtual), and the like. In some examples, an input device can include Natural User Interface (NUI) devices. NUI refers to any interface technology that enables a user to interact with a device in a “natural” manner, free from artificial constraints imposed by input devices such as mice, keyboards, remote controls, and the like. In some examples, NUI devices include devices relying on speech recognition, touch and stylus recognition, gesture recognition both on screen and adjacent to the screen, air gestures, head and eye tracking, voice and speech, vision, touch, gestures, and machine intelligence. For example, NUI devices can include touch sensitive displays, voice and speech recognition, intention and goal understanding, and motion gesture detection using depth cameras such as stereoscopic camera systems, infrared camera systems, RGB camera systems and combinations of these. NUI devices can also include motion gesture detection using accelerometers or gyroscopes, facial recognition, three-dimensional (3D) displays, head, eye, and gaze tracking, immersive augmented reality and virtual reality systems, all of which provide a more natural interface. NUI devices can also include technologies for sensing brain activity using electric field sensing electrodes. For example, a NUI device may use Electroencephalography (EEG) and related methods to detect electrical activity of the brain. The input devices 126 connect to the processing unit 104 through the system bus 108 via interface ports 128. Interface ports 128 include, for example, a serial port, a parallel port, a game port, and a universal serial bus (USB).
Output devices 130 use some of the same type of ports as input devices 126. Thus, for example, a USB port may be used to provide input to the computer 102 and to output information from computer 102 to an output device 130.
Output adapter 132 is provided to illustrate that there are some output devices 130 like monitors, speakers, and printers, among other output devices 130, which are accessible via adapters. The output adapters 132 include, by way of illustration and not limitation, video and sound cards that provide a means of connection between the output device 130 and the system bus 108. It can be noted that other devices and systems of devices provide both input and output capabilities such as remote computing devices 134.
The computer 102 can be a server hosting various software applications in a networked environment using logical connections to one or more remote computers, such as remote computing devices 134. The remote computing devices 134 may be client systems configured with web browsers, PC applications, mobile phone applications, and the like. The remote computing devices 134 can be a personal computer, a server, a router, a network PC, a workstation, a microprocessor based appliance, a mobile phone, a peer device or other common network node and the like, and typically includes many or all of the elements described relative to the computer 102.
Remote computing devices 134 can be logically connected to the computer 102 through a network interface 136 and then connected via a communication connection 138, which may be wireless. Network interface 136 encompasses wireless communication networks such as local-area networks (LAN) and wide-area networks (WAN). LAN technologies include Fiber Distributed Data Interface (FDDI), Copper Distributed Data Interface (CDDI), Ethernet, Token Ring and the like. WAN technologies include, but are not limited to, point-to-point links, circuit switching networks like Integrated Services Digital Networks (ISDN) and variations thereon, packet switching networks, and Digital Subscriber Lines (DSL).
Communication connection 138 refers to the hardware/software employed to connect the network interface 136 to the bus 108. While communication connection 138 is shown for illustrative clarity inside computer 102, it can also be external to the computer 102. The hardware/software for connection to the network interface 136 may include, for exemplary purposes, internal and external technologies such as, mobile phone switches, modems including regular telephone grade modems, cable modems and DSL modems, ISDN adapters, and Ethernet cards.
The computer 102 can further include a radio 140. For example, the radio 140 can be a wireless local area network radio that may operate one or more wireless bands. For example, the radio 140 can operate on the industrial, scientific, and medical (ISM) radio band at 2.4 GHz or 5 GHz. In some examples, the radio 140 can operate on any suitable radio band at any radio frequency.
The computer 102 includes one or more modules 122, such as a system analyzer 142, a user interface selector 144, and a user interface generator 146. In some embodiments, the system analyzer 142 can detect a type of the system based on hardware components residing within the system or coupled to the system. For example, devices including display screens below a threshold size can be identified as mobile devices or devices coupled to gaming controllers can be identified as gaming consoles. In some embodiments, the user interface selector 144 can determine a type of user interface manager to execute based on the type of the system. In some examples, each type of device can be associated with a separate user interface manager. A user interface manager, as referred to herein, can include any suitable application that can generate a visual appearance for an operating system. In some embodiments, the user interface generator 146 can execute the user interface manager to generate a user interface for the system, wherein the type of the user interface manager comprises a plurality of rules to indicate a layout of the user interface. A layout, as referred to herein, can include a two dimensional representation of visible applications and system controls. The plurality of rules can include whether application windows can be overlapped, whether applications can be visible in a full screen mode, or a location on a display screen corresponding to various operating system menus and functions, among others. It is appreciated that the set of rules may also define a 3D user experience or even a non-visual user experience such as an audio only experience.
It is to be understood that the block diagram of
At block 202, a system analyzer 142 can detect a type of a system based on hardware components residing within the system or coupled to the system. For example, the type of the system can include an augmented reality device, a tablet device, a mobile device, a phone device, a gaming console device, or a desktop device, among others. In some embodiments, the system analyzer 142 can detect the type of the system based on any suitable hardware components such as a display screen, a processor, a graphics processor, a cellular radio, and the like. In some examples, the system analyzer 142 can determine that a system that includes a cellular radio is a phone device. In some embodiments, the system analyzer 142 can determine a type of the device based on a size of the display screen of the device. For example, devices that include a display screen that is smaller than a predetermined threshold size can be identified as mobile devices. In some embodiments, the system analyzer 142 can determine that devices that support three dimensional user interfaces are augmented reality devices and devices that connect to gaming controllers are gaming console devices. In some embodiments, the system analyzer 142 can detect a device type based on other hardware components such as power sources, graphics processors, memory devices, and the like. In each of these embodiments, system analyzer 142 can use a combination of predetermined conditions, such as the presence of a specific combination of modules 122 used for a mobile device or a different combination of modules 122 used for a desktop device, as well as the heuristics called out here to make a final determination of the type and configuration of the current system.
At block 204, a user interface selector 144 can determine a user interface manager to execute based on the type of the system. In some embodiments, the user interface managers (also referred to herein as composers) can generate a user interface based on the type of a system. For example, a user interface may enable overlapping windows on desktop devices, but not on mobile devices. Additionally, the user interface managers may include application switchers including a link to a digital assistant along with icons corresponding to applications that are being executed, among other features. The user interface selector 144 can determine the user interface manager to execute to produce a user experience corresponding to a device. In some embodiments, each type of device is associated with a particular user interface manager. For example, an augmented reality device may have a first user interface manager and a gaming console device may have a second user interface manager. As discussed in greater detail below, each user interface manager can indicate how a user interface is to be generated for applications being executed.
At block 206, a user interface generator 146 can execute the user interface manager to generate a user interface for the system, wherein the type of the user interface manager comprises a plurality of rules to indicate a layout of the user interface. The plurality of rules can indicate how to display applications being executed and other visual elements such as an application launcher, an application switcher, and a window list, among others, which may or may not be an independent application or a collection of user manager rules defining how to show, for example, a clock. An application launcher, as referred to herein, can include a list of executable applications installed on a system, a list of recently accessed applications installed on the system, recommended applications to be installed on the system, and the like. In some examples, the application launcher can include commands that can access programs, documents, and settings. These commands can include a search function based on locally stored applications and files, a list of documents available locally on a device or on a remote server, a control panel to configure components of a device, power function commands to alter the power state of the device, and the like. An application switcher, as referred to herein, can include a link to a digital assistant, a task view illustrating all open applications, a set of icons corresponding to applications being executed, and various icons corresponding to applications and hardware features that are enabled each time a device receives power.
In some embodiments, the plurality of rules can indicate an area of a screen that is to be occupied by the application launcher, application switcher, and windows corresponding to applications that are being executed. The plurality of rules may not rely on pixel to pixel mappings by the applications being executed. For example, user interface controls can be displayed in regions of a display screen based on the rules. In some examples, a text field may be displayed in the center of an application and the location of the text field can be determined by the user interface generator 146 based on the plurality of rules. For example, the location of the text field may depend upon whether application windows are overlapping one another, if more than one application window is visible, and the like. The location of user interface controls can also be adjusted based on a size and location of the application switcher. For example, the application switcher can be displayed along the top, bottom, left side, or right side of a display screen. Each type of user interface manager can determine a location of application windows in relation to the location of the application switcher. In some embodiments, the user interface generator 146 can display the user interface based on at least one display characteristic corresponding to the user interface manager. For example, a user interface manager for gaming console devices may display applications in a full screen mode with no frame or adornments. In some embodiments, the user interface manager 146 can also display an application switcher and desktop background. An application switcher, as described in greater detail below, can include any number of icons corresponding to hardware control applications, executing application, digital assistants, and the like. The desktop background can include any suitable image, any number of links to locally stored files, links to directories, and the like.
It is appreciated that user interface concepts such as an application launcher, application switcher, window adornments, action centers, and other instances of user interface experiences mentioned herein are exemplary in nature. The plurality of rules presented to user interface generator 146 may create any user experience desired. There is also no limitations on the complexity of such rules. Some rules may simply indicate when and how an element of text is to be placed on the screen while other rules may define a multi-stage 3D experience with numerous potential outcomes.
In one embodiment, the process flow diagram of
In some embodiments, the method 200 can include generating a user interface that includes at least one active application and at least one visible application. An active application can indicate an application accepting user input, while a visible application can be displayed but not accepting user input. In some embodiments, the method 200 can also include executing a second user interface manager, wherein the second user interface manager is to execute an action center and a notifications detector application. In some embodiments, the method 200 can also include managing at least two subordinate user interface managers, wherein each of the at least two subordinate user interface managers is to execute a separate application.
In some embodiments, the method 200 can also include detecting that an input device is coupled to a system and generating a new user interface manager based on the input device, wherein the input device is a gaming controller and the new user interface manager is to provide a gaming console user interface. Additionally, the method 200 can include storing a state of the user interface in response to transitioning from the user interface manager to the new user interface manager.
In some embodiments, a user interface manager can be implemented with various configurations. For example, the user interface manager can be implemented with an executed application that includes visuals and adornments, a persistent user interface in a main thread of a user interface manager, a persistent user interface in a non-main thread of a user interface manager, as a hosted shell experience for out of process applications, or a recursively composed user interface manager, among others. The various configurations of the user interface manager are illustrated and described below in relation to
In some embodiments, the user interface manager 300 can communicate with an underlying shell or user interface through a set of private application programming interfaces (APIs). These private APIs can allow the user interface manager to execute instructions corresponding to shell tasks associated with a shell. The shell task can include application activation and lifetime management, application or process locks, application resizing, application visibility changes, shutdown, and other shell features. In some embodiments, the user interface manager 300 can also reflect a current state of a shell. The state of a shell can include the active view, running applications, current screen resolution and other shell state information.
In some embodiments, the user interface manager 300 can present the visuals of user applications to the user in additional to the shell experience. These applications can paint into a buffer (represented by a DCOMP visual surface or other similar technology) using the framework of the application's choice. In such examples, the buffer does not paint to screen directly and by default does not get on screen by itself. Rather, the user interface manager 300 can select the visuals for an application and display the application. In some embodiments, the user interface manager 300 can also add adornments, such as a title bar or drop shadow, on the top level visual of a user application. In some embodiments, the user interface manager 300 can also indicate any per application window content such as splash screens, among others.
In one example, a user interface manager 300 can communicate with a Windows List Control 304 function to provide a list of applications being executed on a system. Based on the user interface rules applied by user interface manager 300 each application in the list can be associated with a windows user interface control 306 or 308, which is also associated with a title bar 310 or 312 and a windows content host 314 or 316. The resulting user interface rules enable, for example, a window that is dragged to have the contents of the window also moved because the render transform for the move is applied to the child visual that happens to be the application's composition visual in a coordinated fashion. In the example of
It is to be understood that the block diagram of
The various software components discussed herein may be stored on the tangible, computer-readable storage media 400, as indicated in
It is to be understood that any number of additional software components not shown in
As illustrated in
In one embodiment, user interface host 500 in
At a later point in time a user may do an action, such as detaching the keyboard, which causes system analyzer 406 to recognize that the keyboard is no longer active and user interface host 500 to determine that user interface manager 508 is no longer the correct experience to present to the user. Through the capabilities described here, user interface host 500 is enabled to cause user manager 508 to morph into user manager 510 which expresses a more tablet appropriate experience without impacting the user's ability to interact with the system. A tablet mode experience might replace the use of overlapping windows with a single full screen experience and user experiences which are optimized for touch usage among others. The process of morphing, as referred here in, implies a seamless visual transition from one visual state of the system to another visual state using animation techniques and the like all while maintaining the ability of componentized visual elements such as title bar 310 or title bar 312 to support user input or do other tasks. In one such embodiment, this is achieved by having a centralized repository of information about the state of top level application visuals and other key visual components being presented throughout the system. This state preserved in this repository is independent of the rules and information used by each user interface manager to present these top level application visuals and other elements. Part of this state information in the repository is the association which exists between these visual elements and the user interface manager responsible for presenting them on screen. When system analyzer 406 determines that the keyboard is no longer active and user interface host 500 responds by indicating that user interface manager 508 is to be replaced by user interface manager 510 the combination of information in this state repository and the loose coupling with user interface managers enable a quick and efficient replacement of user interface manager 508 with user interface manage 510. In addition, the use of user interface generator 410 to interpret rules for animation and other behaviors at the same time provides a seamless transition of visuals and input from user interface manager 508 to user interface manager 510. It is appreciated that the methods and technologies called out here, such as the use of a common state repository for top level application visuals, are examples only and other embodiments may employ different approaches and combinations of hardware and software to achieve similar effects.
In an alternate embodiment user interface host 500 is enabled to create environments in which user interface managers are recursively embedded in each other. Consider a traditional desktop computer which uses overlapping windows and includes a virtual desktop paradigm. Virtual desktops are a feature of many modern operating system user experiences which enables a user to create alternate desktops. The user is then able to switch between each of these desktops as a way to segregate different tasks. For example, a user may use one desktop for managing work related applications related to one project, a different desktop for managing work related applications for a separate project, and a different desktop for managing applications related to their personal life. Often such virtual desktop implementations allow for applying different customizations, such as a desktop background, on each virtual desktop. In some examples, they are mutually exclusive—meaning you cannot see both the work virtual desktop and personal virtual desktop at the same time. Returning to
In one embodiment, the fact that user interface manager 510 is an independent instance of the desktop experience, which itself is separated from user interface manager 508 and 512, allows for the user to simultaneously present any combination of user interface managers 508, 510, and 512 to the user. This enables, for example, a user to show and interact with both user interface manager 508 and user interface manager 510 simultaneously allowing the user to interact with both desktop experiences at the same time. One such embodiment of this experience allows for hosting user manager 510 inside of user manager 508 as a separate overlapped window. Similarly, user interface manager 512 containing personal information could be hosted in a window by user experience manager 508. In such a configuration, the connection between root visual 506 as shown in
In an alternate embodiment, system 400 may be enabled, through system analyzer and other appropriate hardware and software affordances, to create different associations of input devices 126 and output devices 130 as described in
In each of these examples described herein, an environment can be created in which user interface host 500 can show user experience 508 and 510 simultaneously in these configurations, which may not be different than that required to show user experience 508 and 510 in a mutually exclusive fashion. Through the use of user interface managers and user interface generator the burden of creating and managing the complexity of these scenarios has been significantly simplified effectively allowing for the re-use of an entire system user experience as a component in a much larger overall experience. It is further appreciated that the examples used herein are not limiting on any form or function. In an alternate embodiment user interface managers 508, 510, and 512 may represent different distinct experiences such as a desktop experience, a gaming console experience, and a media room entertainment experience instead of all being the same experience.
In one embodiment, a system for generating a user interface that includes a processor to detect a type of the system based on hardware components residing within the system or coupled to the system. The processor can also determine a user interface manager to execute based on the type of the system and execute the user interface manager to generate a user interface for the system, wherein the user interface manager comprises a plurality of rules to indicate a layout of the user interface.
Alternatively, or in addition, the user interface can include at least one active application and at least one visible application. Alternatively, or in addition, the type of the system comprises a mobile system, a gaming console system, an augmented reality system, a tablet system, or a desktop system. Alternatively, or in addition, the processor is to display the user interface based on at least one display characteristic corresponding to the user interface manager. Alternatively, or in addition, the user interface manager is to manage at least two subordinate user interface managers, wherein each of the at least two subordinate user interface managers is to execute a separate application. Alternatively, or in addition, the user interface manager is also to manage a shell application that is to support experiences. Alternatively, or in addition, the user interface manager is also to manage a shell experience host that is to support experiences. Alternatively, or in addition, the processor is to detect an input device is coupled to the system and generate a new user interface manager based on the input device. Alternatively, or in addition, the input device is a gaming controller and the new user interface manager is to provide a gaming console user interface. Alternatively, or in addition, the processor is to store a state of the user interface in response to transitioning from the user interface manager to the new user interface manager.
In another embodiment described herein, a method for generating a user interface can include detecting a type of a system based on hardware components residing within the system or coupled to the system. The method can also include determining a user interface manager to execute based on the type of the system and executing the user interface manager to generate a user interface for the system, wherein the user interface manager comprises a plurality of rules to indicate a layout of the user interface.
Alternatively, or in addition, the user interface can include at least one active application and at least one visible application. Alternatively, or in addition, the type of the system comprises a mobile system, a gaming console system, an augmented reality system, a tablet system, or a desktop system. Alternatively, or in addition, the method can include displaying the user interface based on at least one display characteristic corresponding to the user interface manager. Alternatively, or in addition, the user interface manager is to manage at least two subordinate user interface managers, wherein each of the at least two subordinate user interface managers is to execute a separate application. Alternatively, or in addition, the user interface manager is also to manage a shell application that is to support experiences. Alternatively, or in addition, the user interface manager is also to manage a shell experience host that is to support experiences. Alternatively, or in addition, the method can include detecting an input device is coupled to the system and generating a new user interface manager based on the input device. Alternatively, or in addition, the input device is a gaming controller and the new user interface manager is to provide a gaming console user interface. Alternatively, or in addition, the method can include storing a state of the user interface in response to transitioning from the user interface manager to the new user interface manager.
In yet another embodiment described herein, a system for generating a user interface can include a processor and a storage device to store a plurality of instructions that, in response to being executed by the processor, cause the processor to detect a type of the system based on hardware components residing within the system or coupled to the system and select a user interface manager from a visual tree of user interface managers to execute based on the type of the system, wherein the visual tree is to reside in a user interface host. The plurality of instructions can also cause the processor to execute the selected user interface manager to generate a user interface for the system, wherein the selected user interface manager comprises a plurality of rules to indicate a layout of the user interface.
Alternatively, or in addition, the user interface can include at least one active application and at least one visible application. Alternatively, or in addition, the type of the system comprises a mobile system, a gaming console system, an augmented reality system, a tablet system, or a desktop system. Alternatively, or in addition, the plurality of instructions can cause the processor to display the user interface based on at least one display characteristic corresponding to the user interface manager. Alternatively, or in addition, the user interface manager is to manage at least two subordinate user interface managers, wherein each of the at least two subordinate user interface managers is to execute a separate application. Alternatively, or in addition, the user interface manager is also to manage a shell application that is to support experiences. Alternatively, or in addition, the user interface manager is also to manage a shell experience host that is to support experiences. Alternatively, or in addition, the plurality of instructions can cause the processor to detect an input device is coupled to the system and generate a new user interface manager based on the input device. Alternatively, or in addition, the input device is a gaming controller and the new user interface manager is to provide a gaming console user interface. Alternatively, or in addition, the plurality of instructions can cause the processor to store a state of the user interface in response to transitioning from the user interface manager to the new user interface manager.
In particular and in regard to the various functions performed by the above described components, devices, circuits, systems and the like, the terms (including a reference to a “means”) used to describe such components are intended to correspond, unless otherwise indicated, to any component which performs the specified function of the described component, e.g., a functional equivalent, even though not structurally equivalent to the disclosed structure, which performs the function in the herein illustrated exemplary aspects of the claimed subject matter. In this regard, it will also be recognized that the innovation includes a system as well as a computer-readable storage media having computer-executable instructions for performing the acts and events of the various methods of the claimed subject matter.
There are multiple ways of implementing the claimed subject matter, e.g., an appropriate API, tool kit, driver code, operating system, control, standalone or downloadable software object, etc., which enables applications and services to use the techniques described herein. The claimed subject matter contemplates the use from the standpoint of an API (or other software object), as well as from a software or hardware object that operates according to the techniques set forth herein. Thus, various implementations of the claimed subject matter described herein may have aspects that are wholly in hardware, partly in hardware and partly in software, as well as in software.
The aforementioned systems have been described with respect to interaction between several components. It can be appreciated that such systems and components can include those components or specified sub-components, some of the specified components or sub-components, and additional components, and according to various permutations and combinations of the foregoing. Sub-components can also be implemented as components communicatively coupled to other components rather than included within parent components (hierarchical).
Additionally, it can be noted that one or more components may be combined into a single component providing aggregate functionality or divided into several separate sub-components, and any one or more middle layers, such as a management layer, may be provided to communicatively couple to such sub-components in order to provide integrated functionality. Any components described herein may also interact with one or more other components not specifically described herein but generally known by those of skill in the art.
In addition, while a particular feature of the claimed subject matter may have been disclosed with respect to one of several implementations, such feature may be combined with one or more other features of the other implementations as may be desired and advantageous for any given or particular application. Furthermore, to the extent that the terms “includes,” “including,” “has,” “contains,” variants thereof, and other similar words are used in either the detailed description or the claims, these terms are intended to be inclusive in a manner similar to the term “comprising” as an open transition word without precluding any additional or other elements.
Stauber, William Scott, Balasubramanian, Ramrajprabu, Mulye, Nishad, Davis, Darren R., Shipley, Kenton A., Derks, Patrick J., Radebaugh, Nathan T., Dykstra, Paul, Cross, Brian David, Karachale, Jan Harold
Patent | Priority | Assignee | Title |
Patent | Priority | Assignee | Title |
10025548, | Aug 09 2016 | International Business Machines Corporation | Automated display configuration |
10120659, | May 30 2014 | Apple Inc | Adaptive user interfaces |
5675755, | Jun 07 1995 | Sony Corporation; Sony Electronics, INC | Window system preventing overlap of multiple always-visible windows |
6188399, | May 08 1998 | Apple Inc | Multiple theme engine graphical user interface architecture |
7234111, | Sep 28 2001 | NTT DoCoMo, Inc | Dynamic adaptation of GUI presentations to heterogeneous device platforms |
7246326, | Jun 25 2001 | CERNER INNOVATION, INC | System and procedure for providing a user interface display |
7707514, | Nov 18 2005 | Apple Inc | Management of user interface elements in a display environment |
7895522, | Sep 28 2001 | NTT DoCoMo, Inc | Layout of platform specific graphical user interface widgets migrated between heterogeneous device platforms |
7907966, | Jul 19 2005 | Meta Platforms, Inc | System and method for cross-platform applications on a wireless phone |
8869027, | Aug 04 2006 | Apple Inc | Management and generation of dashboards |
9013510, | Jul 29 2011 | GOOGLE LLC | Systems and methods for rendering user interface elements in accordance with a device type |
9043714, | Jan 07 2011 | GOOGLE LLC | Adaptive user interface for widescreen devices |
9107076, | Jul 27 2012 | Sprint Communications Company L.P.; SPRINT COMMUNICATIONS COMPANY L P | Data fraud detection via device type identification |
9239668, | Jun 29 2012 | Intel Corporation | Provision of a user interface based on user interaction with a computing device |
9239713, | Mar 06 2013 | MOBILEFORCE SOFTWARE, INC | Platform independent rendering for native mobile applications |
9471201, | May 20 2014 | GOOGLE LLC | Laptop-to-tablet mode adaptation |
9513763, | Mar 20 2014 | Amazon Technologies, Inc | Adaptive user interfaces |
9621567, | Nov 29 2010 | BIOCATCH LTD | Device, system, and method of detecting hardware components |
9747005, | Mar 11 2013 | WORKDAY, INC | Adaptive user interface |
20010045961, | |||
20020039101, | |||
20020059514, | |||
20030070061, | |||
20040039862, | |||
20040230940, | |||
20040242322, | |||
20060132473, | |||
20060168388, | |||
20060284878, | |||
20070079236, | |||
20070118813, | |||
20080034314, | |||
20090024944, | |||
20090150773, | |||
20100138295, | |||
20110153612, | |||
20120017172, | |||
20120054634, | |||
20120204115, | |||
20130167111, | |||
20130191775, | |||
20140026086, | |||
20140040819, | |||
20140101343, | |||
20140157183, | |||
20150029228, | |||
20160132301, | |||
20160147402, | |||
20160209994, | |||
20170168782, | |||
20180260083, | |||
CN105493032, | |||
CN1387105, | |||
EP1884871, | |||
EP2584462, | |||
EP3465421, | |||
JP2014519079, | |||
KR20150014319, | |||
TW201044255, | |||
WO2008099776, | |||
WO2007036091, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Jun 16 2017 | Microsoft Technology Licensing, LLC | (assignment on the face of the patent) | / | |||
Jun 27 2017 | DERKS, PATRICK J | Microsoft Technology Licensing, LLC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 051011 | /0876 | |
Jun 28 2017 | RADEBAUGH, NATHAN T | Microsoft Technology Licensing, LLC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 051011 | /0876 | |
Jun 30 2017 | CROSS, BRIAN DAVID | Microsoft Technology Licensing, LLC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 051011 | /0876 | |
Jun 30 2017 | MULYE, NISHAD | Microsoft Technology Licensing, LLC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 051011 | /0876 | |
Jul 11 2017 | KARACHALE, JAN HAROLD | Microsoft Technology Licensing, LLC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 051011 | /0876 | |
Jul 12 2017 | DYKSTRA, PAUL | Microsoft Technology Licensing, LLC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 051011 | /0876 | |
Jul 13 2017 | SHIPLEY, KENTON A | Microsoft Technology Licensing, LLC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 051011 | /0876 | |
Jul 14 2017 | STAUBER, WILLIAM SCOTT | Microsoft Technology Licensing, LLC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 051011 | /0876 | |
Aug 01 2017 | BALASUBRAMANIAN, RAMRAJPRABU | Microsoft Technology Licensing, LLC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 051011 | /0876 | |
Aug 02 2017 | DAVIS, DARREN R | Microsoft Technology Licensing, LLC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 051011 | /0876 |
Date | Maintenance Fee Events |
Nov 14 2019 | BIG: Entity status set to Undiscounted (note the period is included in the code). |
Date | Maintenance Schedule |
Nov 07 2026 | 4 years fee payment window open |
May 07 2027 | 6 months grace period start (w surcharge) |
Nov 07 2027 | patent expiry (for year 4) |
Nov 07 2029 | 2 years to revive unintentionally abandoned end. (for year 4) |
Nov 07 2030 | 8 years fee payment window open |
May 07 2031 | 6 months grace period start (w surcharge) |
Nov 07 2031 | patent expiry (for year 8) |
Nov 07 2033 | 2 years to revive unintentionally abandoned end. (for year 8) |
Nov 07 2034 | 12 years fee payment window open |
May 07 2035 | 6 months grace period start (w surcharge) |
Nov 07 2035 | patent expiry (for year 12) |
Nov 07 2037 | 2 years to revive unintentionally abandoned end. (for year 12) |