A method for operating a voice trigger is provided. In some implementations, the method is performed at an electronic device including one or more processors and memory storing instructions for execution by the one or more processors. The method includes receiving a sound input. The sound input may correspond to a spoken word or phrase, or a portion thereof. The method includes determining whether at least a portion of the sound input corresponds to a predetermined type of sound, such as a human voice. The method includes, upon a determination that at least a portion of the sound input corresponds to the predetermined type, determining whether the sound input includes predetermined content, such as a predetermined trigger word or phrase. The method also includes, upon a determination that the sound input includes the predetermined content, initiating a speech-based service, such as a voice-based digital assistant.
|
9. A non-transitory computer readable storage medium storing one or more programs for execution by one or more processors of an electronic device, the one or more programs including instructions for:
determining whether the electronic device is in a predetermined orientation, wherein the predetermined orientation corresponds to a display of the electronic device being horizontal and facing down;
upon a determination that the electronic device is in the predetermined orientation, generating instructions to activate a predetermined mode of a voice trigger, wherein the predetermined mode is a standby mode; and
activating the predetermined mode of the voice trigger based on the instructions.
13. A method for operating a voice trigger, performed at an electronic device including one or more processors and memory storing instructions for execution by the one or more processors, the method comprising:
determining whether the electronic device is in a predetermined orientation, wherein the predetermined orientation corresponds to a display of the electronic device being horizontal and facing down;
upon a determination that the electronic device is in the predetermined orientation, generating instructions to activate a predetermined mode of a voice trigger, wherein the predetermined mode is a standby mode; and
activating the predetermined mode of the voice trigger based on the instructions.
14. An electronic device comprising:
one or more processors;
a memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
determining whether the electronic device is in a predetermined orientation, wherein the predetermined orientation corresponds to a display of the electronic device being horizontal and facing down;
upon a determination that the electronic device is in the predetermined orientation, generating instructions to activate a predetermined mode of a voice trigger, wherein the predetermined mode is a standby mode; and
activating the predetermined mode of the voice trigger based on the instructions.
7. A method for operating a voice trigger, performed at an electronic device including one or more processors and memory storing instructions for execution by the one or more processors, the method comprising:
operating a voice trigger in a first mode;
determining whether the electronic device is in an enclosed space by detecting that one or more of a microphone and a camera of the electronic device is occluded, wherein a trigger phrase associated with the voice trigger is adapted based on the determination of whether the electronic device is in the enclosed space;
upon a determination that the electronic device is in the enclosed space, generating instructions to switch the voice trigger to a second mode; and
switching the voice trigger to the second mode based on the instructions.
1. A non-transitory computer readable storage medium storing one or more programs for execution by one or more processors of an electronic device, the one or more programs including instructions for:
operating a voice trigger in a first mode;
determining whether the electronic device is in an at least partially enclosed space by detecting that one or more of a microphone and a camera of the electronic device is occluded, wherein a trigger phrase associated with the voice trigger is adapted based on the determination of whether the electronic device is in the at least partially enclosed space;
upon a determination that the electronic device is in the at least partially enclosed space, generating instructions to switch the voice trigger to a second mode; and
switching the voice trigger to the second mode based on the instructions.
8. An electronic device comprising:
one or more processors;
a memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
operating a voice trigger in a first mode;
determining whether the electronic device is in an enclosed space by detecting that one or more of a microphone and a camera of the electronic device is occluded, wherein a trigger phrase associated with the voice trigger is adapted based on the determination of whether the electronic device is in the enclosed space;
upon a determination that the electronic device is in the enclosed space, generating instructions to switch the voice trigger to a second mode; and
switching the voice trigger to the second mode based on the instructions.
2. The non-transitory computer readable storage medium of
3. The non-transitory computer readable storage medium of
4. The non-transitory computer readable storage medium of
5. The non-transitory computer readable storage medium of
6. The non-transitory computer readable storage medium of
10. The non-transitory computer readable storage medium of
determining whether the electronic device is in a second predetermined orientation, wherein the second predetermined orientation corresponds to the display of the electronic device being horizontal and facing up; and
upon a determination that the electronic device is in the second predetermined orientation, activating a second predetermined mode of the voice trigger, wherein the second predetermined mode is a listening mode.
11. The non-transitory computer readable storage medium of
12. The non-transitory computer readable storage medium of
17. The method of
18. The method of
19. The method of
22. The device of
23. The device of
24. The device of
25. The method of
determining whether the electronic device is in a second predetermined orientation, wherein the second predetermined orientation corresponds to the display of the electronic device being horizontal and facing up; and
upon a determination that the electronic device is in the second predetermined orientation, activating a second predetermined mode of the voice trigger, wherein the second predetermined mode is a listening mode.
26. The method of
27. The method of
28. The device of
determining whether the electronic device is in a second predetermined orientation, wherein the second predetermined orientation corresponds to the display of the electronic device being horizontal and facing up; and
upon a determination that the electronic device is in the second predetermined orientation, activating a second predetermined mode of the voice trigger, wherein the second predetermined mode is a listening mode.
29. The device of
30. The device of
|
This Application is a continuation of U.S. application Ser. No. 14/175,864, filed on Feb. 7, 2014, entitled VOICE TRIGGER FOR A DIGITAL ASSISTANT, which claims the benefit of U.S. Provisional Application No. 61/762,260, filed on Feb. 7, 2013, entitled VOICE TRIGGER FOR A DIGITAL ASSISTANT. The contents of each of these applications are hereby incorporated by reference in their entireties for all purposes.
The disclosed implementations relate generally to digital assistants, and more specifically, to a method and system for a voice trigger for a digital assistant.
Recently, voice-based digital assistants, such as Apple's SIRI, have been introduced into the marketplace to handle various tasks such as web searching and navigation. One advantage of such voice-based digital assistants is that users can interact with a device in a hands-free manner without handling or even looking at the device. Hands-free operation can be particularly beneficial when a person cannot or should not physically handle a device, such as when they are driving. However, to initiate the voice-based assistant, users typically must press a button or select an icon on a touch screen. This tactile input detracts from the hands-free experience. Accordingly, it would be advantageous to provide a method and system of activating a voice-based digital assistant (or other speech-based service) using a voice input or signal, and not a tactile input.
Activating a voice-based assistant using a voice input requires monitoring an audio channel to detect the voice input. This monitoring consumes electrical power, which is a limited resource on handheld or portable devices that rely on batteries and on which such voice-based digital assistants often run. Thus, it would be beneficial to provide an energy-efficient voice trigger that can be used to initiate voice- and/or speech-based services on a device.
Accordingly, there is a need for a low-power voice trigger that can provide “always-listening” voice trigger functionality without excessively consuming limited power resources.
The implementations described below provide systems and methods for initiating a voice-based assistant using a voice trigger at an electronic device. Interactions with a voice-based digital assistant (or other speech-based services, such as a speech-to-text transcription service) often begin when a user presses an affordance (e.g., a button or icon) on a device in order to activate the digital assistant, followed by the device providing some indication to the user that the digital assistant is active and listening, such as a light, a sound (e.g., a beep), or a vocalized output (e.g., “what can I do for you?”). As described herein, voice triggers can also be implemented so that they are activated in response to a specific, predetermined word, phrase, or sound, and without requiring a physical interaction by the user. For example, a user may be able to activate a SIRI digital assistant on an iPHONE (both provided by Apple Inc., the assignee of the present application) by reciting the phrase “Hey, SIRI.” In response, the device outputs a beep, sound, or speech output (e.g., “what can I do for you?”) indicating to the user that the listening mode is active. Accordingly, the user can initiate an interaction with the digital assistant without having to physically touch the device that provides the digital assistant functionality.
One technique for initiating a speech-based service with a voice trigger is to have the speech-based service continuously listen for a predetermined trigger word, phrase, or sound (any of which may be referred to herein as “the trigger sound”). However, continuously operating the speech-based service (e.g., the voice-based digital assistant) requires substantial audio processing and battery power. In order to reduce the power consumed by providing voice trigger functionality, several techniques may be employed. In some implementations, the main processor of an electronic device (i.e., an “application processor”) is kept in a low-power or un-powered state while one or more sound detectors that use less power (e.g., because they do not rely on the application processor) remain active. (When it is in a low-power or un-powered state, an application processor or any other processor, program, or module may be described as inactive or in a standby mode.) For example, a low-power sound detector is used to monitor an audio channel for a trigger sound even when the application processor is inactive. This sound detector is sometimes referred to herein as a trigger sound detector. In some implementations, it is configured to detect particular sounds, phonemes, and/or words. The trigger sound detector (including hardware and/or software components) is designed to recognize specific words, sound, or phrases, but is generally not capable of or optimized for providing full speech-to-text functionality, as such tasks require greater computational and power resources. Thus, in some implementations, the trigger sound detector recognizes whether a voice input includes a predefined pattern (e.g., a sonic pattern matching the words “Hey, SIRI”), but is not able to (or is not configured to) convert the voice input into text or recognize a significant amount of other words. Once the trigger sound has been detected, then, the digital assistant is brought out of a standby mode so that the user can provide a voice command.
In some implementations, the trigger sound detector is configured to detect several different trigger sounds, such as a set of words, phrases, sounds, and/or combinations thereof. The user can then use any of those sounds to initiate the speech-based service. In one example, a voice trigger is preconfigured to respond to the phrases “Hey, SIRI,” “Wake up, SIRI,” “Invoke my digital assistant,” or “Hello, HAL, do you read me, HAL?” In some implementations, the user must select one of the preconfigured trigger sounds as the sole trigger sound. In some implementations, the user selects a subset of the preconfigured trigger sounds, so that the user can initiate the speech-based service with different trigger sounds. In some implementations, all of the preconfigured trigger sounds remain valid trigger sounds.
In some implementations, another sound detector is used so that even the trigger sound detector can be kept in a low- or no-power mode for much of the time. For example, a different type of sound detector (e.g., one that uses less power than the trigger sound detector) is used to monitor an audio channel to determine whether the sound input corresponds to a certain type of sound. Sounds are categorized as different “types” based on certain identifiable characteristics of the sounds. For example, sounds that are of the type “human voice” have certain spectral content, periodicity, fundamental frequencies, etc. Other types of sounds (e.g., whistles, hand claps, etc.) have different characteristics. Sounds of different types are identified using audio and/or signal processing techniques, as described herein.
This sound detector is sometimes referred to herein as a “sound-type detector.” For example, if a predetermined trigger phrase is “Hey, SIRI”, the sound-type detector determines whether the input likely corresponds to human speech. If the trigger sound is a non-voiced sound, such as a whistle, the sound-type detector determines whether a sound input likely corresponds to a whistle. When the appropriate type of sound is detected, the sound-type detector initiates the trigger sound detector to further process and/or analyze the sound. And because the sound-type detector requires less power than the trigger sound detector (e.g., because it uses circuitry with lower power demands and/or more efficient audio processing algorithms than the trigger-sound detector), the voice trigger functionality consumes even less power than with a trigger sound detector alone.
In some implementations, yet another sound detector is used so that both the sound-type detector and the trigger sound detector described above can be kept in a low- or no-power mode for much of the time. For example, a sound detector that uses less power than the sound-type detector is used to monitor an audio channel to determine whether a sound input satisfies a predetermined condition, such as an amplitude (e.g., volume) threshold. This sound detector may be referred to herein as a noise detector. When the noise detector detects a sound that satisfies the predetermined threshold, the noise detector initiates the sound-type detector to further process and/or analyze the sound. And because the noise detector requires less power than either the sound-type detector or the trigger sound detector (e.g., because it uses circuitry with lower power demands and/or more efficient audio processing algorithms), the voice trigger functionality consumes even less power than the combination of the sound-type detector and the trigger sound detector without the noise detector.
In some implementations, any one or more of the sound detectors described above are operated according to a duty cycle, where they are cycled between “on” and “off’ states. This further helps to reduce power consumption of the voice trigger. For example, in some implementations, the noise detector is “on” (i.e., actively monitoring an audio channel) for 10 milliseconds, and “off” for the following 90 milliseconds. This way, the noise detector is “off’ 90% of the time, while still providing effectively continuous noise detection functionality. In some implementations, the on and off durations for the sound detectors are selected so that all of the detectors are be activated while the trigger sound is still being input. For example, for a trigger phrase of “Hey, SIRI,” the sound detectors may be configured so that no matter where in the duty cycle(s) the trigger phrase begins, the trigger sound detector is activated in time to analyze a sufficient amount of the input. For example, the trigger sound detector will be activated in time to receive, process, and analyze the sounds “ay SIRI,” which is enough to determine that the sound matches the trigger phrase. In some implementations, sound inputs are stored in memory as they are received and passed to an upstream detector so that a larger portion of the sound input can be analyzed. Accordingly, even if the trigger sound detector is not initiated until after a trigger phrase has been uttered, it can still analyze the entire recorded trigger phrase.
Some implementations provide a method for operating a voice trigger. The method is performed at an electronic device including one or more processors and memory storing instructions for execution by the one or more processors. The method includes receiving a sound input. The method further includes determining whether at least a portion of the sound input corresponds to a predetermined type of sound. The method further includes, upon a determination that at least a portion of the sound input corresponds to the predetermined type, determining whether the sound input includes predetermined content. The method further includes, upon a determination that the sound input includes the predetermined content, initiating a speech-based service. In some implementations, the speech-based service is a voice-based digital assistant. In some implementations, speech-based service is a dictation service.
In some implementations, determining whether the sound input corresponds to a predetermined type of sound is performed by a first sound detector, and determining whether the sound input includes predetermined content is performed by a second sound detector. In some implementations, the first sound detector consumes less power while operating than the second sound detector. In some implementations, the first sound detector performs frequency-domain analysis of the sound input. In some implementations, determining whether the sound input corresponds to the predetermined type of sound is performed upon a determination that the sound input satisfies a predetermined condition (e.g., as determined by a third sound detector, discussed below).
In some implementations, the first sound detector periodically monitors an audio channel according to a duty cycle. In some implementations, the duty cycle includes an on-time of about 20 milliseconds, and an off-time of about 100 milliseconds.
In some implementations, the predetermined type is a human voice and the predetermined content is one or more words. In some implementations, determining whether at least a portion of the sound input corresponds to the predetermined type of sound includes determining whether at least a portion of the sound input includes frequencies characteristic of a human voice.
In some implementations, the second sound detector is initiated in response to a determination by the first sound detector that the sound input corresponds to the predetermined type. In some implementations, the second sound detector is operated for at least a predetermined amount of time after a determination by the first sound detector that the sound input corresponds to the predetermined type. In some implementations, the predetermined amount of time corresponds to a duration of the predetermined content.
In some implementations, the predetermined content is one or more predetermined phonemes. In some implementations, the one or more predetermined phonemes constitute at least one word.
In some implementations, the method includes, prior to determining whether the sound input corresponds to a predetermined type of sound, determining whether the sound input satisfies a predetermined condition. In some implementations, the predetermined condition is an amplitude threshold. In some implementations, determining whether the sound input satisfies a predetermined condition is performed by a third sound detector, wherein the third sound detector consumes less power while operating than the first sound detector. In some implementations, the third sound detector periodically monitors an audio channel according to a duty cycle. In some implementations, the duty cycle includes an on-time of about 20 milliseconds, and an off-time of about 500 milliseconds. In some implementations, the third sound detector performs time-domain analysis of the sound input.
In some implementations, the method includes storing at least a portion of the sound input in memory, and providing the portion of the sound input to the speech-based service once the speech-based service is initiated. In some implementations, the portion of the sound input is stored in memory using direct memory access.
In some implementations, the method includes determining whether the sound input corresponds to a voice of a particular user. In some implementations, the speech-based service is initiated upon a determination that the sound input includes the predetermined content and that the sound input corresponds to the voice of the particular user. In some implementations, the speech-based service is initiated in a limited access mode upon a determination that the sound input includes the predetermined content and that the sound input does not correspond to the voice of the particular user. In some implementations, the method includes, upon a determination that the sound input corresponds to the voice of the particular user, outputting a voice prompt including a name of the particular user.
In some implementations, determining whether the sound input includes predetermined content includes comparing a representation of the sound input to a reference representation, and determining that the sound input includes the predetermined content when the representation of the sound input matches the reference representation. In some implementations, a match is determined if the representation of the sound input matches the reference representation to a predetermined confidence. In some implementations, the method includes receiving a plurality of sound inputs including the sound input; and iteratively adjusting the reference representation, using respective ones of the plurality of sound inputs, in response to determining that the respective sound inputs include the predetermined content.
In some implementations, the method includes determining whether the electronic device is in a predetermined orientation, and upon a determination that the electronic device is in the predetermined orientation, activating a predetermined mode of the voice trigger. In some implementations, the predetermined orientation corresponds to a display screen of the device being substantially horizontal and facing down, and the predetermined mode is a standby mode. In some implementations, the predetermined orientation corresponds to a display screen of the device being substantially horizontal and facing up, and the predetermined mode is a listening mode.
Some implementations provide a method for operating a voice trigger. The method is performed at an electronic device including one or more processors and memory storing instructions for execution by the one or more processors. The method includes operating a voice trigger in a first mode. The method further includes determining whether the electronic device is in a substantially enclosed space by detecting that one or more of a microphone and a camera of the electronic device is occluded. The method further includes, upon a determination that the electronic device is in a substantially enclosed space, switching the voice trigger to a second mode. In some implementations, the second mode is a standby mode.
Some implementations provide a method for operating a voice trigger. The method is performed at an electronic device including one or more processors and memory storing instructions for execution by the one or more processors. The method includes determining whether the electronic device is in a predetermined orientation, and, upon a determination that the electronic device is in the predetermined orientation, activating a predetermined mode of a voice trigger. In some implementations, the predetermined orientation corresponds to a display screen of the device being substantially horizontal and facing down, and the predetermined mode is a standby mode. In some implementations, the predetermined orientation corresponds to a display screen of the device being substantially horizontal and facing up, and the predetermined mode is a listening mode.
In accordance with some implementations, an electronic device includes a sound receiving unit configured to receive sound input; and a processing unit coupled to the sound receiving unit. The processing unit is configured to determine whether at least a portion of the sound input corresponds to a predetermined type of sound; upon a determination that at least a portion of the sound input corresponds to the predetermined type, determine whether the sound input includes predetermined content; and upon a determination that the sound input includes the predetermined content, initiate a speech-based service. In some implementations, the processing unit is further configured to, prior to determining whether the sound input corresponds to a predetermined type of sound, determine whether the sound input satisfies a predetermined condition. In some implementations, the processing unit is further configured to determine whether the sound input corresponds to a voice of a particular user.
In accordance with some implementations, an electronic device includes a voice trigger unit configured to operate a voice trigger in a first mode of a plurality of modes; and a processing unit coupled to the voice trigger unit. In some implementations, the processing unit is configured to: determine whether the electronic device is in a substantially enclosed space by detecting that one or more of a microphone and a camera of the electronic device is occluded; and upon a determination that the electronic device is in a substantially enclosed space, switch the voice trigger to a second mode. In some implementations, the processing unit is configured to determine whether the electronic device is in a predetermined orientation; and upon a determination that the electronic device is in the predetermined orientation, activate a predetermined mode of a voice trigger.
In accordance with some implementations, a computer-readable storage medium (e.g., a non-transitory computer readable storage medium) is provided, the computer-readable storage medium storing one or more programs for execution by one or more processors of an electronic device, the one or more programs including instructions for performing any of the methods described herein.
In accordance with some implementations, an electronic device (e.g., a portable electronic device) is provided that comprises means for performing any of the methods described herein.
In accordance with some implementations, an electronic device (e.g., a portable electronic device) is provided that comprises a processing unit configured to perform any of the methods described herein.
In accordance with some implementations, an electronic device (e.g., a portable electronic device) is provided that comprises one or more processors and memory storing one or more programs for execution by the one or more processors, the one or more programs including instructions for performing any of the methods described herein.
In accordance with some implementations, an information processing apparatus for use in an electronic device is provided, the information processing apparatus comprising means for performing any of the methods described herein.
Like reference numerals refer to corresponding parts throughout the drawings.
Specifically, once initiated, a digital assistant system is capable of accepting a user request at least partially in the form of a natural language command, request, statement, narrative, and/or inquiry. Typically, the user request seeks either an informational answer or performance of a task by the digital assistant system. A satisfactory response to the user request is generally either provision of the requested informational answer, performance of the requested task, or a combination of the two. For example, a user may ask the digital assistant system a question, such as “Where am I right now?” Based on the user's current location, the digital assistant may answer, “You are in Central Park near the west gate.” The user may also request the performance of a task, for example, by stating “Please invite my friends to my girlfriend's birthday party next week.” In response, the digital assistant may acknowledge the request by generating a voice output, “Yes, right away,” and then send a suitable calendar invite from the user's email address to each of the user' friends listed in the user's electronic address book or contact list. There are numerous other ways of interacting with a digital assistant to request information or performance of various tasks. In addition to providing verbal responses and taking programmed actions, the digital assistant can also provide responses in other visual or audio forms (e.g., as text, alerts, music, videos, animations, etc.).
As shown in
In some implementations, the DA server 106 includes a client-facing I/O interface 112, one or more processing modules 114, data and models 116, an I/O interface to external services 118, a photo and tag database 130, and a photo-tag module 132. The client-facing I/O interface facilitates the client-facing input and output processing for the digital assistant server 106. The one or more processing modules 114 utilize the data and models 116 to determine the user's intent based on natural language input and perform task execution based on the deduced user intent. Photo and tag database 130 stores fingerprints of digital photographs, and, optionally digital photographs themselves, as well as tags associated with the digital photographs. Photo-tag module 132 creates tags, stores tags in association with photographs and/or fingerprints, automatically tags photographs, and links tags to locations within photographs.
In some implementations, the DA server 106 communicates with external services 120 (e.g., navigation service(s) 122-1, messaging service(s) 122-2, information service(s) 122-3, calendar service 122-4, telephony service 122-5, photo service(s) 122-6, etc.) through the network(s) 110 for task completion or information acquisition. The I/O interface to the external services 118 facilitates such communications.
Examples of the user device 104 include, but are not limited to, a handheld computer, a personal digital assistant (PDA), a tablet computer, a laptop computer, a desktop computer, a cellular telephone, a smartphone, an enhanced general packet radio service (EGPRS) mobile phone, a media player, a navigation device, a game console, a television, a remote control, or a combination of any two or more of these data processing devices or any other suitable data processing devices. More details on the user device 104 are provided in reference to an exemplary user device 104 shown in
Examples of the communication network(s) 110 include local area networks (LAN) and wide area networks (WAN), e.g., the Internet. The communication network(s) 110 may be implemented using any known network protocol, including various wired or wireless protocols, such as Ethernet, Universal Serial Bus (USB), FIREWIRE, Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), code division multiple access (CDMA), time division multiple access (TDMA), Bluetooth, Wi-Fi, voice over Internet Protocol (VoIP), Wi-MAX, or any other suitable communication protocol.
The server system 108 can be implemented on at least one data processing apparatus and/or a distributed network of computers. In some implementations, the server system 108 also employs various virtual devices and/or services of third party service providers (e.g., third-party cloud service providers) to provide the underlying computing resources and/or infrastructure resources of the server system 108.
Although the digital assistant system shown in
For example, in some implementations, a motion sensor 210 (e.g., an accelerometer), a light sensor 212, a GPS receiver 213, a temperature sensor, and a proximity sensor 214 are coupled to the peripherals interface 206 to facilitate orientation, light, and proximity sensing functions. In some implementations, other sensors 216, such as a biometric sensor, barometer, and the like, are connected to the peripherals interface 206, to facilitate related functionalities.
In some implementations, the user device 104 includes a camera subsystem 220 coupled to the peripherals interface 206. In some implementations, an optical sensor 222 of the camera subsystem 220 facilitates camera functions, such as taking photographs and recording video clips. In some implementations, the user device 104 includes one or more wired and/or wireless communication subsystems 224 provide communication functions. The communication subsystems 224 typically includes various communication ports, radio frequency receivers and transmitters, and/or optical (e.g., infrared) receivers and transmitters. In some implementations, the user device 104 includes an audio subsystem 226 coupled to one or more speakers 228 and one or more microphones 230 to facilitate voice-enabled functions, such as voice recognition, voice replication, digital recording, and telephony functions. In some implementations, the audio subsystem 226 is coupled to a voice trigger system 400. In some implementations, the voice trigger system 400 and/or the audio subsystem 226 includes low-power audio circuitry and/or programs (i.e., including hardware and/or software) for receiving and/or analyzing sound inputs, including, for example, one or more analog-to-digital converters, digital signal processors (DSPs), sound detectors, memory buffers, codecs, and the like. In some implementations, the low-power audio circuitry (alone or in addition to other components of the user device 104) provides voice (or sound) trigger functionality for one or more aspects of the user device 104, such as a voice-based digital assistant or other speech-based service. In some implementations, the low-power audio circuitry provides voice trigger functionality even when other components of the user device 104 are shut down and/or in a standby mode, such as the processor(s) 204, I/O subsystem 240, memory 250, and the like. The voice trigger system 400 is described in further detail with respect to
In some implementations, an I/O subsystem 240 is also coupled to the peripheral interface 206. In some implementations, the user device 104 includes a touch screen 246, and the I/O subsystem 240 includes a touch screen controller 242 coupled to the touch screen 246. When the user device 104 includes the touch screen 246 and the touch screen controller 242, the touch screen 246 and the touch screen controller 242 are typically con figured to, for example, detect contact and movement or break thereof using any of a plurality of touch sensitivity technologies, such as capacitive, resistive, infrared, surface acoustic wave technologies, proximity sensor arrays, and the like. In some implementations, the user device 104 includes a display that does not include a touch-sensitive surface. In some implementations, the user device 104 includes a separate touch-sensitive surface. In some implementations, the user device 104 includes other input controller(s) 244. When the user device 104 includes the other input controller(s) 244, the other input controller(s) 244 are typically coupled to other input/control devices 248, such as one or more buttons, rocker switches, thumb-wheel, infrared port, USB port, and/or a pointer device such as a stylus.
The memory interface 202 is coupled to memory 250. In some implementations, memory 250 includes a non-transitory computer readable medium, such as high-speed random access memory and/or non-volatile memory (e.g., one or more magnetic disk storage devices, one or more flash memory devices, one or more optical storage devices, and/or other non-volatile solid-state memory devices).
In some implementations, memory 250 stores an operating system 252, a communications module 254, a graphical user interface module 256, a sensor processing module 258, a phone module 260, and applications 262, and a subset or superset thereof. The operating system 252 includes instructions for handling basic system services and for performing hardware dependent tasks. The communications module 254 facilitates communicating with one or more additional devices, one or more computers and/or one or more servers. The graphical user interface module 256 facilitates graphic user interface processing. The sensor processing module 258 facilitates sensor-related processing and functions (e.g., processing voice input received with the one or more microphones 228). The phone module 260 facilitates phone-related processes and functions. The application module 262 facilitates various functionalities of user applications, such as electronic-messaging, web browsing, media processing, navigation, imaging and/or other processes and functions. In some implementations, the user device 104 stores in memory 250 one or more software applications 270-1 and 270-2 each associated with at least one of the external service providers.
As described above, in some implementations, memory 250 also stores client-side digital assistant instructions (e.g., in a digital assistant client module 264) and various user data 266 (e.g., user-specific vocabulary data, preference data, and/or other data such as the user's electronic address book or contact list, to-do lists, shopping lists, etc.) to provide the client-side functionalities of the digital assistant.
In various implementations, the digital assistant client module 264 is capable of accepting voice input, text input, touch input, and/or gestural input through various user interfaces (e.g., the I/O subsystem 244) of the user device 104. The digital assistant client module 264 is also capable of providing output in audio, visual, and/or tactile forms. For example, output can be provided as voice, sound, alerts, text messages, menus, graphics, videos, animations, vibrations, and/or combinations of two or more of the above. During operation, the digital assistant client module 264 communicates with the digital assistant server (e.g., the digital assistant server 106,
In some implementations, the digital assistant client module 264 utilizes various sensors, subsystems and peripheral devices to gather additional information from the surrounding environment of the user device 104 to establish a context associated with a user input. In some implementations, the digital assistant client module 264 provides the context information or a subset thereof with the user input to the digital assistant server (e.g., the digital assistant server 106,
In some implementations, the context information that can accompany the user input includes sensor information, e.g., lighting, ambient noise, ambient temperature, images or videos of the surrounding environment, etc. In some implementations, the context information also includes the physical state of the device, e.g., device orientation, device location, device temperature, power level, speed, acceleration, motion patterns, cellular signals strength, etc. In some implementations, information related to the software state of the user device 106, e.g., running processes, installed programs, past and present network activities, background services, error logs, resources usage, etc., of the user device 104 is also provided to the digital assistant server (e.g., the digital assistant server 106,
In some implementations, the DA client module 264 selectively provides information (e.g., at least a portion of the user data 266) stored on the user device 104 in response to requests from the digital assistant server. In some implementations, the digital assistant client module 264 also elicits additional input from the user via a natural language dialogue or other user interfaces upon request by the digital assistant server 106 (
In some implementations, memory 250 may include additional instructions or fewer instructions. Furthermore, various functions of the user device 104 may be implemented in hardware and/or in firmware, including in one or more signal processing and/or application specific integrated circuits, and the user device 104, thus, need not include all modules and applications illustrated in
The digital assistant system 300 includes memory 302, one or more processors 304, an input/output (I/O) interface 306, and a network communications interface 308. These components communicate with one another over one or more communication buses or signal lines 310.
In some implementations, memory 302 includes a non-transitory computer readable medium, such as high-speed random access memory and/or a non-volatile computer readable storage medium (e.g., one or more magnetic disk storage devices, one or more flash memory devices, one or more optical storage devices, and/or other non-volatile solid-state memory devices).
The I/O interface 306 couples input/output devices 316 of the digital assistant system 300, such as displays, a keyboards, touch screens, and microphones, to the user interface module 322. The I/O interface 306, in conjunction with the user interface module 322, receives user inputs (e.g., voice input, keyboard inputs, touch inputs, etc.) and process them accordingly. In some implementations, when the digital assistant is implemented on a standalone user device, the digital assistant system 300 includes any of the components and I/O and communication interfaces described with respect to the user device 104 in
In some implementations, the network communications interface 308 includes wired communication port(s) 312 and/or wireless transmission and reception circuitry 314. The wired communication port(s) receive and send communication signals via one or more wired interfaces, e.g., Ethernet, Universal Serial Bus (USB), FIREWIRE, etc. The wireless circuitry 314 typically receives and sends RF signals and/or optical signals from/to communications networks and other communications devices. The wireless communications may use any of a plurality of communications standards, protocols and technologies, such as GSM, EDGE, CDMA, TDMA, Bluetooth, Wi-Fi, VoIP, Wi-MAX, or any other suitable communication protocol. The network communications interface 308 enables communication between the digital assistant system 300 with networks, such as the Internet, an intranet and/or a wireless network, such as a cellular telephone network, a wireless local area network (LAN) and/or a metropolitan area network (MAN), and other devices.
In some implementations, the non-transitory computer readable storage medium of memory 302 stores programs, modules, instructions, and data structures including all or a subset of: an operating system 318, a communications module 320, a user interface module 322, one or more applications 324, and a digital assistant module 326. The one or more processors 304 execute these programs, modules, and instructions, and reads/writes from/to the data structures.
The operating system 318 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, iOS, WINDOWS, or an embedded operating system such as VxWorks) includes various software components and/or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communications between various hardware, firmware, and software components.
The communications module 320 facilitates communications between the digital assistant system 300 with other devices over the network communications interface 308. For example, the communication module 320 may communicate with the communications module 254 of the device 104 shown in
In some implementations, the user interface module 322 receives commands and/or inputs from a user via the I/O interface 306 (e.g., from a keyboard, touch screen, and/or microphone), and provides user interface objects on a display.
The applications 324 include programs and/or modules that are configured to be executed by the one or more processors 304. For example, if the digital assistant system is implemented on a standalone user device, the applications 324 may include user applications, such as games, a calendar application, a navigation application, or an email application. If the digital assistant system 300 is implemented on a server farm, the applications 324 may include resource management applications, diagnostic applications, or scheduling applications, for example.
Memory 302 also stores the digital assistant module (or the server portion of a digital assistant) 326. In some implementations, the digital assistant module 326 includes the following sub-modules, or a subset or superset thereof: an input/output processing module 328, a speech-to-text (STT) processing module 330, a natural language processing module 332, a dialogue flow processing module 334, a task flow processing module 336, a service processing module 338, and a photo module 132. Each of these processing modules has access to one or more of the following data and models of the digital assistant 326, or a subset or superset thereof: ontology 360, vocabulary index 344, user data 348, categorization module 349, disambiguation module 350, task flow models 354, service models 356, photo tagging module 358, search module 360, and local tag/photo storage 362.
In some implementations, using the processing modules (e.g., the input/output processing module 328, the STT processing module 330, the natural language processing module 332, the dialogue flow processing module 334, the task flow processing module 336, and/or the service processing module 338), data, and models implemented in the digital assistant module 326, the digital assistant system 300 performs at least some of the following: identifying a user's intent expressed in a natural language input received from the user; actively eliciting and obtaining information needed to fully deduce the user's intent (e.g., by disambiguating words, names, intentions, etc.); determining the task flow for fulfilling the deduced intent; and executing the task flow to fulfill the deduced intent. In some implementations, the digital assistant also takes appropriate actions when a satisfactory response was not or could not be provided to the user for various reasons.
In some implementations, as discussed below, the digital assistant system 300 identifies, from a natural language input, a user's intent to tag a digital photograph, and processes the natural language input so as to tag the digital photograph with appropriate information. In some implementations, the digital assistant system 300 performs other tasks related to photographs as well, such as searching for digital photographs using natural language input, auto-tagging photographs, and the like.
As shown in
In some implementations, the speech-to-text processing module 330 receives speech input (e.g., a user utterance captured in a voice recording) through the I/O processing module 328. In some implementations, the speech-to-text processing module 330 uses various acoustic and language models to recognize the speech input as a sequence of phonemes, and ultimately, a sequence of words or tokens written in one or more languages. The speech-to-text processing module 330 is implemented using any suitable speech recognition techniques, acoustic models, and language models, such as Hidden Markov Models, Dynamic Time Warping (DTW)-based speech recognition, and other statistical and/or analytical techniques. In some implementations, the speech-to-text processing can be performed at least partially by a third party service or on the user's device. Once the speech-to-text processing module 330 obtains the result of the speech-to-text processing (e.g., a sequence of words or tokens), it passes the result to the natural language processing module 332 for intent deduction.
The natural language processing module 332 (“natural language processor”) of the digital assistant 326 takes the sequence of words or tokens (“token sequence”) generated by the speech-to-text processing module 330, and attempts to associate the token sequence with one or more “actionable intents” recognized by the digital assistant. As used herein, an “actionable intent” represents a task that can be performed by the digital assistant 326 and/or the digital assistant system 300 (
In some implementations, in addition to the sequence of words or tokens obtained from the speech-to-text processing module 330, the natural language processor 332 also receives context information associated with the user request (e.g., from the I/O processing module 328). The natural language processor 332 optionally uses the context information to clarify, supplement, and/or further define the information contained in the token sequence received from the speech-to-text processing module 330. The context information includes, for example, user preferences, hardware and/or software states of the user device, sensor information collected before, during, or shortly after the user request, prior interactions (e.g., dialogue) between the digital assistant and the user, and the like.
In some implementations, the natural language processing is based on an ontology 360. The ontology 360 is a hierarchical structure containing a plurality of nodes, each node representing either an “actionable intent” or a “property” relevant to one or more of the “actionable intents” or other “properties.” As noted above, an “actionable intent” represents a task that the digital assistant system 300 is capable of performing (e.g., a task that is “actionable” or can be acted on). A “property” represents a parameter associated with an actionable intent or a sub-aspect of another property. A linkage between an actionable intent node and a property node in the ontology 360 defines how a parameter represented by the property node pertains to the task represented by the actionable intent node.
In some implementations, the ontology 360 is made up of actionable intent nodes and property nodes. Within the ontology 360, each actionable intent node is linked to one or more property nodes either directly or through one or more intermediate property nodes. Similarly, each property node is linked to one or more actionable intent nodes either directly or through one or more intermediate property nodes. For example, the ontology 360 shown in
An actionable intent node, along with its linked concept nodes, may be described as a “domain.” In the present discussion, each domain is associated with a respective actionable intent, and refers to the group of nodes (and the relationships therebetween) associated with the particular actionable intent. For example, the ontology 360 shown in
While
In some implementations, the ontology 360 includes all the domains (and hence actionable intents) that the digital assistant is capable of understanding and acting upon. In some implementations, the ontology 360 may be modified, such as by adding or removing domains or nodes, or by modifying relationships between the nodes within the ontology 360.
In some implementations, nodes associated with multiple related actionable intents may be clustered under a “super domain” in the ontology 360. For example, a “travel” super-domain may include a cluster of property nodes and actionable intent nodes related to travels. The actionable intent nodes related to travels may include “airline reservation,” “hotel reservation,” “car rental,” “get directions,” “find points of interest,” and so on. The actionable intent nodes under the same super domain (e.g., the “travels” super domain) may have many property nodes in common. For example, the actionable intent nodes for “airline reservation,” “hotel reservation,” “car rental,” “get directions,” “find points of interest” may share one or more of the property nodes “start location,” “destination,” “departure date/time,” “arrival date/time,” and “party size.”
In some implementations, each node in the ontology 360 is associated with a set of words and/or phrases that are relevant to the property or actionable intent represented by the node. The respective set of words and/or phrases associated with each node is the so-called “vocabulary” associated with the node. The respective set of words and/or phrases associated with each node can be stored in the vocabulary index 344 (
In some implementations, the natural language processor 332 shown in
In some implementations, the digital assistant system 300 also stores names of specific entities in the vocabulary index 344, so that when one of these names is detected in the user request, the natural language processor 332 will be able to recognize that the name refers to a specific instance of a property or sub-property in the ontology. In some implementations, the names of specific entities are names of businesses, restaurants, people, movies, and the like. In some implementations, the digital assistant system 300 can search and identify specific entity names from other data sources, such as the user's address book or contact list, a movies database, a musicians database, and/or a restaurant database. In some implementations, when the natural language processor 332 identifies that a word in the token sequence is a name of a specific entity (such as a name in the user's address book or contact list), that word is given additional significance in selecting the actionable intent within the ontology for the user request.
For example, when the words “Mr. Santo” are recognized from the user request, and the last name “Santo” is found in the vocabulary index 344 as one of the contacts in the user's contact list, then it is likely that the user request corresponds to a “send a message” or “initiate a phone call” domain. For another example, when the words “ABC Café” are found in the user request, and the term “ABC Café” is found in the vocabulary index 344 as the name of a particular restaurant in the user's city, then it is likely that the user request corresponds to a “restaurant reservation” domain.
User data 348 includes user-specific information, such as user-specific vocabulary, user preferences, user address, user's default and secondary languages, user's contact list, and other short-term or long-term information for each user. The natural language processor 332 can use the user-specific information to supplement the information contained in the user input to further define the user intent. For example, for a user request “invite my friends to my birthday party,” the natural language processor 332 is able to access user data 348 to determine who the “friends” are and when and where the “birthday party” would be held, rather than requiring the user to provide such information explicitly in his/her request.
In some implementations, natural language processor 332 includes categorization module 349. In some implementations, the categorization module 349 determines whether each of the one or more terms in a text string (e.g., corresponding to a speech input associated with a digital photograph) is one of an entity, an activity, or a location, as discussed in greater detail below. In some implementations, the categorization module 349 classifies each term of the one or more terms as one of an entity, an activity, or a location.
Once the natural language processor 332 identifies an actionable intent (or domain) based on the user request, the natural language processor 332 generates a structured query to represent the identified actionable intent. In some implementations, the structured query includes parameters for one or more nodes within the domain for the actionable intent, and at least some of the parameters are populated with the specific information and requirements specified in the user request. For example, the user may say “Make me a dinner reservation at a sushi place at 7.” In this case, the natural language processor 332 may be able to correctly identify the actionable intent to be “restaurant reservation” based on the user input. According to the ontology, a structured query for a “restaurant reservation” domain may include parameters such as {Cuisine}, {Time}, {Date}, {Party Size}, and the like. Based on the information contained in the user's utterance, the natural language processor 332 may generate a partial structured query for the restaurant reservation domain, where the partial structured query includes the parameters {Cuisine=“Sushi”} and {Time=“7 pm”}. However, in this example, the user's utterance contains insufficient information to complete the structured query associated with the domain. Therefore, other necessary parameters such as {Party Size} and {Date} are not specified in the structured query based on the information currently available. In some implementations, the natural language processor 332 populates some parameters of the structured query with received context information. For example, if the user requested a sushi restaurant “near me,” the natural language processor 332 may populate a {location} parameter in the structured query with GPS coordinates from the user device 104.
In some implementations, the natural language processor 332 passes the structured query (including any completed parameters) to the task flow processing module 336 (“task flow processor”). The task flow processor 336 is configured to perform one or more of: receiving the structured query from the natural language processor 332, completing the structured query, and performing the actions required to “complete” the user's ultimate request. In some implementations, the various procedures necessary to complete these tasks are provided in task flow models 354. In some implementations, the task flow models 354 include procedures for obtaining additional information from the user, and task flows for performing actions associated with the actionable intent.
As described above, in order to complete a structured query, the task flow processor 336 may need to initiate additional dialogue with the user in order to obtain additional information, and/or disambiguate potentially ambiguous utterances. When such interactions are necessary, the task flow processor 336 invokes the dialogue processing module 334 (“dialogue processor”) to engage in a dialogue with the user. In some implementations, the dialogue processing module 334 determines how (and/or when) to ask the user for the additional information, and receives and processes the user responses. In some implementations, the questions are provided to and answers are received from the users through the I/O processing module 328. For example, the dialogue processing module 334 presents dialogue output to the user via audio and/or visual output, and receives input from the user via spoken or physical (e.g., touch gesture) responses. Continuing with the example above, when the task flow processor 336 invokes the dialogue processor 334 to determine the “party size” and “date” information for the structured query associated with the domain “restaurant reservation,” the dialogue processor 334 generates questions such as “For how many people?” and “On which day?” to pass to the user. Once answers are received from the user, the dialogue processing module 334 populates the structured query with the missing information, or passes the information to the task flow processor 336 to complete the missing information from the structured query.
In some cases, the task flow processor 336 may receive a structured query that has one or more ambiguous properties. For example, a structured query for the “send a message” domain may indicate that the intended recipient is “Bob,” and the user may have multiple contacts named “Bob.” The task flow processor 336 will request that the dialogue processor 334 disambiguate this property of the structured query. In turn, the dialogue processor 334 may ask the user “Which Bob?”, and display (or read) a list of contacts named “Bob” from which the user may choose.
In some implementations, dialogue processor 334 includes disambiguation module 350. In some implementations, disambiguation module 350 disambiguates one or more ambiguous terms (e.g., one or more ambiguous terms in a text string corresponding to a speech input associated with a digital photograph). In some implementations, disambiguation module 350 identifies that a first term of the one or more teens has multiple candidate meanings, prompts a user for additional information about the first term, receives the additional information from the user in response to the prompt and identifies the entity, activity, or location associated with the first term in accordance with the additional information.
In some implementations, disambiguation module 350 disambiguates pronouns. In such implementations, disambiguation module 350 identifies one of the one or more terms as a pronoun and determines a noun to which the pronoun refers. In some implementations, disambiguation module 350 determines a noun to which the pronoun refers by using a contact list associated with a user of the electronic device. Alternatively, or in addition, disambiguation module 350 determines a noun to which the pronoun refers as a name of an entity, an activity, or a location identified in a previous speech input associated with a previously tagged digital photograph. Alternatively, or in addition, disambiguation module 350 determines a noun to which the pronoun refers as a name of a person identified based on a previous speech input associated with a previously tagged digital photograph.
In some implementations, disambiguation module 350 accesses information obtained from one or more sensors (e.g., proximity sensor 214, light sensor 212, GPS receiver 213, temperature sensor 215, and motion sensor 210) of a handheld electronic device (e.g., user device 104) for determining a meaning of one or more of the terms. In some implementations, disambiguation module 350 identifies two terms each associated with one of an entity, an activity, or a location. For example, a first of the two terms refers to a person, and a second of the two terms refers to a location. In some implementations, disambiguation module 350 identifies three terms each associated with one of an entity, an activity, or a location.
Once the task flow processor 336 has completed the structured query for an actionable intent, the task flow processor 336 proceeds to perform the ultimate task associated with the actionable intent. Accordingly, the task flow processor 336 executes the steps and instructions in the task flow model according to the specific parameters contained in the structured query. For example, the task flow model for the actionable intent of “restaurant reservation” may include steps and instructions for contacting a restaurant and actually requesting a reservation for a particular party size at a particular time. For example, using a structured query such as: {restaurant reservation, restaurant=ABC Café, date=3/12/2012, time=7 pm, party size=5}, the task flow processor 336 may perform the steps of: (1) logging onto a server of the ABC Café or a restaurant reservation system that is configured to accept reservations for multiple restaurants, such as the ABC Café, (2) entering the date, time, and party size information in a form on the website, (3) submitting the form, and (4) making a calendar entry for the reservation in the user's calendar. In another example, described in greater detail below, the task flow processor 336 executes steps and instructions associated with tagging or searching for digital photographs in response to a voice input, e.g., in conjunction with photo module 132.
In some implementations, the task flow processor 336 employs the assistance of a service processing module 338 (“service processor”) to complete a task requested in the user input or to provide an informational answer requested in the user input. For example, the service processor 338 can act on behalf of the task flow processor 336 to make a phone call, set a calendar entry, invoke a map search, invoke or interact with other user applications installed on the user device, and invoke or interact with third party services (e.g. a restaurant reservation portal, a social networking website or service, a banking portal, etc.,). In some implementations, the protocols and application programming interfaces (API) required by each service can be specified by a respective service model among the service models 356. The service processor 338 accesses the appropriate service model for a service and generates requests for the service in accordance with the protocols and APIs required by the service according to the service model.
For example, if a restaurant has enabled an online reservation service, the restaurant can submit a service model specifying the necessary parameters for making a reservation and the APIs for communicating the values of the necessary parameters to the online reservation service. When requested by the task flow processor 336, the service processor 338 can establish a network connection with the online reservation service using the web address stored in the service models 356, and send the necessary parameters of the reservation (e.g., time, date, party size) to the online reservation interface in a format according to the API of the online reservation service.
In some implementations, the natural language processor 332, dialogue processor 334, and task flow processor 336 are used collectively and iteratively to deduce and define the user's intent, obtain information to further clarify and refine the user intent, and finally generate a response (e.g., provide an output to the user, or complete a task) to fulfill the user's intent.
In some implementations, after all of the tasks needed to fulfill the user's request have been performed, the digital assistant 326 formulates a confirmation response, and sends the response back to the user through the I/O processing module 328. If the user request seeks an informational answer, the confirmation response presents the requested information to the user. In some implementations, the digital assistant also requests the user to indicate whether the user is satisfied with the response produced by the digital assistant 326.
Attention is now directed to
In some implementations, the voice trigger system 400 includes a noise detector 402, a sound-type detector 404, a trigger sound detector 406, and a speech-based service 408, and an audio subsystem 226, each coupled to an audio bus 401. In some implementations, more or fewer of these modules are used. The sound detectors 402, 404, and 406 may be referred to as modules, and may include hardware (e.g., circuitry, memory, processors, etc.), software (e.g., programs, software-on-a-chip, firmware, etc.), and/or any combinations thereof for performing the functionality described herein. In some implementations, the sound detectors are communicatively, programmatically, physically, and/or operationally coupled to one another (e.g., via a communications bus), as illustrated in
In some implementations, the audio subsystem 226 includes a codec 410, an audio digital signal processor (DSP) 412, and a memory buffer 414. In some implementations, the audio subsystem 226 is coupled to one or more microphones 230 (
In some implementations, the speech-based service 408 is a voice-based digital assistant, and corresponds to one or more components or functionalities of the digital assistant system described above with reference to
In some implementations, the noise detector 402 monitors an audio channel to determine whether a sound input from the audio subsystem 226 satisfies a predetermined condition, such as an amplitude threshold. The audio channel corresponds to a stream of audio information received by one or more sound pickup devices, such as the one or more microphones 230 (
In some implementations, the predetermined condition is whether the sound input is above a certain volume for a predetermined amount of time. In some implementations, the noise detector uses time-domain analysis of the sound input, which requires relatively little computational and battery resources as compared to other types of analysis (e.g., as performed by the sound-type detector 404, the trigger word detector 406, and/or the speech-based service 408). In some implementations, other types of signal processing and/or audio analysis are used, including, for example, frequency-domain analysis. If the noise detector 402 determines that the sound input satisfies the predetermined condition, it initiates an upstream sound detector, such as the sound-type detector 404 (e.g., by providing a control signal to initiate one or more processing routines, and/or by providing power to the upstream sound detector). In some implementations, the upstream sound detector is initiated in response to other conditions being satisfied. For example, in some implementations, the upstream sound detector is initiated in response to determining that the device is not being stored in an enclosed space (e.g., based on a light detector detecting a threshold level of light).
The sound-type detector 404 monitors the audio channel to determine whether a sound input corresponds to a certain type of sound, such as sound that is characteristic of a human voice, whistle, clap, etc. The type of sound that the sound-type detector 404 is configured to recognize will correspond to the particular trigger sound(s) that the voice trigger is configured to recognize. In implementations where the trigger sound is a spoken word or phrase, the sound-type detector 404 includes a “voice activity detector” (VAD). In some implementations, the sound-type detector 404 uses frequency-domain analysis of the sound input. For example, the sound-type detector 404 generates a spectrogram of a received sound input (e.g., using a Fourier transform), and analyzes the spectral components of the sound input to determine whether the sound input is likely to correspond to a particular type or category of sounds (e.g., human speech). Thus, in implementations where the trigger sound is a spoken word or phrase, if the audio channel is picking up ambient sound (e.g., traffic noise) but not human speech, the VAD will not initiate the trigger sound detector 406.
In some implementations, the sound-type detector 404 remains active for as long as predetermined conditions of any downstream sound detector (e.g., the noise detector 402) are satisfied. For example, in some implementations, the sound-type detector 404 remains active as long as the sound input includes sound above a predetermined amplitude threshold (as determined by the noise detector 402), and is deactivated when the sound drops below the predetermined threshold. In some implementations, once initiated, the sound-type detector 404 remains active until a condition is met, such as the expiration of a timer (e.g., for 1, 2, 5, or 10 seconds, or any other appropriate duration), the expiration of a certain number of on/off cycles of the sound-type detector 404, or the occurrence of an event (e.g., the amplitude of the sound falls below a second threshold, as determined by the noise detector 402 and/or the sound-type detector 404).
As mentioned above, if the sound-type detector 404 determines that the sound input corresponds to a predetermined type of sound, it initiates an upstream sound detector (e.g., by providing a control signal to initiate one or more processing routines, and/or by providing power to the upstream sound detector), such as the trigger sound detector 406.
The trigger sound detector 406 is configured to determine whether a sound input includes at least part of certain predetermined content (e.g., at least part of the trigger word, phrase, or sound). In some implementations, the trigger sound detector 406 compares a representation of the sound input (an “input representation”) to one or more reference representations of the trigger word. If the input representation matches at least one of the one or more reference representations with an acceptable confidence, the trigger sound detector 406 initiates the speech-based service 408 (e.g., by providing a control signal to initiate one or more processing routines, and/or by providing power to the upstream sound detector). In some implementations, the input representation and the one or more reference representations are spectrograms (or mathematical representations thereof), which represent how the spectral density of a signal varies with time. In some implementations, the representations are other types of audio signatures or voiceprints. In some implementations, initiating the speech-based service 408 includes bringing one or more circuits, programs, and/or processors out of a standby mode, and invoking the sound-based service. The sound-based service is then ready to provide more comprehensive speech recognition, speech-to-text processing, and/or natural language processing.
In some implementations, the voice-trigger system 400 includes voice authentication functionality, so that it can determine if a sound input corresponds to a voice of a particular person, such as an owner/user of the device. For example, in some implementations, the sound-type detector 404 uses a voiceprinting technique to determine that the sound input was uttered by an authorized user. Voice authentication and voiceprinting are described in more detail in U.S. patent application Ser. No. 13/053,144, assigned to the assignee of the instant application, which is hereby incorporated by reference in its entirety. In some implementations, voice authentication is included in any of the sound detectors described herein (e.g., the noise detector 402, the sound-type detector 404, the trigger sound detector 406, and/or the speech-based service 408). In some implementations, voice authentication is implemented as a separate module from the sound detectors listed above (e.g., as voice authentication module 428,
In some implementations, the trigger sound detector 406 remains active for as long as conditions of any downstream sound detector(s) (e.g., the noise detector 402 and/or the sound-type detector 404) are satisfied. For example, in some implementations, the trigger sound detector 406 remains active as long as the sound input includes sound above a predetermined threshold (as detected by the noise detector 402). In some implementations, it remains active as long as the sound input includes sound of a certain type (as detected by the sound-type detector 404). In some implementations, it remains active as long as both the foregoing conditions are met.
In some implementations, once initiated, the trigger sound detector 406 remains active until a condition is met, such as the expiration of a timer (e.g., for 1, 2, 5, or 10 seconds, or any other appropriate duration), the expiration of a certain number of on/off cycles of the trigger sound detector 406, or the occurrence of an event (e.g., the amplitude of the sound falls below a second threshold).
In some implementations, when one sound detector initiates another detector, both sound detectors remain active. However, the sound detectors may be active or inactive at various times, and it is not necessary that all of the downstream (e.g., the lower power and/or sophistication) sound detectors be active (or that their respective conditions are met) in order for upstream sound detectors to be active. For example, in some implementations, after the noise detector 402 and the sound-type detector 404 determine that their respective conditions are met, and the trigger sound detector 406 is initiated, one or both of the noise detector 402 and the sound-type detector 404 are deactivated and/or enter a standby mode while the trigger sound detector 406 operates. In other implementations, both the noise detector 402 and the sound-type detector 404 (or one or the other) stay active while the trigger sound detector 406 operates. In various implementations, different combinations of the sound detectors are active at different times, and whether one is active or inactive may depend on the state of other sound detectors, or may be independent of the state of other sound detectors.
While
Moreover, different combinations of sound detectors may be used at different times. For example, the particular combination of sound detectors and how they interact may depend on one or more conditions, such as the context or operating state of a device. As a specific example, if a device is plugged in (and thus not relying exclusively on battery power), the trigger sound detector 406 is active, while the noise detector 402 and the sound-type detector 404 remain inactive. In another example, if the device is in a pocket or backpack, all sound detectors are inactive.
By cascading sound detectors as described above, where the detectors that require more power are invoked only when necessary by detectors that require lower power, power efficient voice triggering functionality can be provided. As described above, additional power efficiency is achieved by operating one or more of the sound detectors according to a duty cycle. For example, in some implementations, the noise detector 402 operates according to a duty cycle so that it performs effectively continuous noise detection, even though the noise detector is off for at least part of the time. In some implementations, the noise detector 402 is on for 10 milliseconds and off for 90 milliseconds. In some implementations, the noise detector 402 is on for 20 milliseconds and off for 500 milliseconds. Other on and off durations are also possible.
In some implementations, if the noise detector 402 detects a noise during its “on” interval, the noise detector 402 will remain on in order to further process and/or analyze the sound input. For example, the noise detector 402 may be configured to initiate an upstream sound detector if it detects sound above a predetermined amplitude for a predetermined amount of time (e.g., 100 milliseconds). Thus, if the noise detector 402 detects sound above a predetermined amplitude during its 10 millisecond “on” interval, it will not immediately enter the “off” interval. Instead, the noise detector 402 remains active and continues to process the sound input to determine whether it exceeds the threshold for the full predetermined duration (e.g., 100 milliseconds).
In some implementations, the sound-type detector 404 operates according to a duty cycle. In some implementations, the sound-type detector 404 is on for 20 milliseconds and off for 100 milliseconds. Other on and off durations are also possible. In some implementations, the sound-type detector 404 is able to determine whether a sound input corresponds to a predetermined type of sound within the “on” interval of its duty cycle. Thus, the sound-type detector 404 will initiate the trigger sound detector 406 (or any other upstream sound detector) if the sound-type detector 404 determines, during its “on” interval, that the sound is of a certain type. Alternatively, in some implementations, if the sound-type detector 404 detects, during the “on” interval, sound that may correspond to the predetermined type, the detector will not immediately enter the “off” interval. Instead, the sound-type detector 404 remains active and continues to process the sound input and determine whether it corresponds to the predetermined type of sound. In some implementations, if the sound detector determines that the predetermined type of sound has been detected, it initiates the trigger sound detector 406 to further process the sound input and determine if the trigger sound has been detected.
Similar to the noise detector 402 and the sound-type detector 404, in some implementations, the trigger sound detector 406 operates according to a duty cycle. In some implementations, the trigger sound detector 406 is on for 50 milliseconds and off for 50 milliseconds. Other on and off durations are also possible. If the trigger sound detector 406 detects, during its “on” interval, that there is sound that may correspond to a trigger sound, the detector will not immediately enter the “off” interval. Instead, the trigger sound detector 406 remains active and continues to process the sound input and determine whether it includes the trigger sound. In some implementations, if such a sound is detected, the trigger sound detector 406 remains active to process the audio for a predetermined duration, such as 1, 2, 5, or 10 seconds, or any other appropriate duration. In some implementations, the duration is selected based on the length of the particular trigger word or sound that it is configured to detect. For example, if the trigger phrase is “Hey, SIRI,” the trigger word detector is operated for about 2 seconds to determine whether the sound input includes that phrase.
In some implementations, some of the sound detectors are operated according to a duty cycle, while others operate continuously when active. For example, in some implementations, only the first sound detector is operated according to a duty cycle (e.g., the noise detector 402 in
In some implementations, the voice trigger includes noise, echo, and/or sound cancellation functionality (referred to collectively as noise cancellation). In some implementations, noise cancellation is performed by the audio subsystem 226 (e.g., by the audio DSP 412). Noise cancellation reduces or removes unwanted noise or sounds from the sound input prior to it being processed by the sound detectors. In some cases, the unwanted noise is background noise from the user's environment, such as a fan or the clicking from a keyboard. In some implementations, the unwanted noise is any sound above, below, or at predetermined amplitudes or frequencies. For example, in some implementations, sound above the typical human vocal range (e.g., 3,000 Hz) is filtered out or removed from the signal. In some implementations, multiple microphones (e.g., the microphones 230) are used to help determine what components of received sound should be reduced and/or removed. For example, in some implementations, the audio subsystem 226 uses beam forming techniques to identify sounds or portions of sound inputs that appear to originate from a single point in space (e.g., a user's mouth). The audio subsystem 226 then focuses on this sound by removing from the sound input sounds that are received equally by all microphones (e.g., ambient sound that does not appear to originate from any particular direction).
In some implementations, the DSP 412 is configured to cancel or remove from the sound input sounds that are being output by the device on which the digital assistant is operating. For example, if the audio subsystem 226 is outputting music, radio, a podcast, a voice output, or any other audio content (e.g., via the speaker 228), the DSP 412 removes any of the outputted sound that was picked up by a microphone and included in the sound input. Thus, the sound input is free of the outputted audio (or at least contains less of the outputted audio). Accordingly, the sound input that is provided to the sound detectors will be cleaner, and the triggers more accurate. Aspects of noise cancellation are described in more detail in U.S. Pat. No. 7,272,224, assigned to the assignee of the instant application, which is hereby incorporated by reference in its entirety.
In some implementations, different sound detectors require that the sound input be filtered and/or preprocessed in different ways. For example, in some implementations, the noise detector 402 is configured to analyze time-domain audio signal between 60 and 20,000 Hz, and the sound-type detector is configured to perform frequency-domain analysis of audio between 60 and 3,000 Hz. Thus, in some implementations, the audio DSP 412 (and/or other audio DSPs of the device 104) preprocesses received audio according to the respective needs of the sound detectors. In some implementations, on the other hand, the sound detectors are configured to filter and/or preprocess the audio from the audio subsystem 226 according to their specific needs. In such cases, the audio DSP 412 may still perform noise cancellation prior to providing the sound input to the sound detectors.
In some implementations, the context of the electronic device is used to help determine whether and how to operate the voice trigger. For example, it may be unlikely that users will invoke a speech-based service, such as a voice-based digital assistant, when the device is stored in their pocket, purse, or backpack. Also, it may be unlikely that users will invoke a speech-based service when they are at a loud rock concert. For some users, it is unlikely that they will invoke a speech-based service at certain times of the day (e.g., late at night). On the other hand, there are also contexts in which it is more likely that a user will invoke a speech-based service using a voice trigger. For example, some users will be more likely to use a voice trigger when they are driving, when they are alone, when they are at work, or the like. Various techniques are used to determine the context of a device. In various implementations, the device uses information from any one or more of the following components or information sources to determine the context of a device: GPS receivers, light sensors, microphones, proximity sensors, orientation sensors, inertial sensors, cameras, communications circuitry and/or antennas, charging and/or power circuitry, switch positions, temperature sensors, compasses, accelerometers, calendars, user preferences, etc.
The context of the device can then be used to adjust how and whether the voice trigger operates. For example, in certain contexts, the voice trigger will be deactivated (or operated in a different mode) as long as that context is maintained. For example, in some implementations, the voice trigger is deactivated when the phone is in a predetermined orientation (e.g., lying face-down on a surface), during predetermined time periods (e.g., between 10:00 PM and 8:00 AM), when the phone is in a “silent” or a “do not disturb” mode (e.g., based on a switch position, mode setting, or user preference), when the device is in a substantially enclosed space (e.g., a pocket, bag, purse, drawer, or glove box), when the device is near other devices that have a voice trigger and/or speech-based services (e.g., based on proximity sensors, acoustic/wireless/infrared communications), and the like. In some implementations, instead of being deactivated, the voice trigger system 400 is operated in a low-power mode (e.g., by operating the noise detector 402 according to a duty cycle with a 10 millisecond “on” interval and a 5 second “off” interval). In some implementations, an audio channel is monitored more infrequently when the voice trigger system 400 is operated in a low-power mode. In some implementations, a voice trigger uses a different sound detector or combination of sound detectors when it is in a low-power mode than when it is in a normal mode. (The voice trigger may be capable of numerous different modes or operating states, each of which may use a different amount of power, and different implementations will use them according to their specific designs.)
On the other hand, when the device is in some other contexts, the voice trigger will be activated (or operated in a different mode) so long as that context is maintained. For example, in some implementations, the voice trigger remains active while it is plugged into a power source, when the phone is in a predetermined orientation (e.g., lying face-up on a surface), during predetermined time periods (e.g., between 8:00 AM and 10:00 PM), when the device is travelling and/or in a car (e.g., based on GPS signals, BLUETOOTH connection or docking with a vehicle, etc.), and the like. Aspects of detect lining when a device is in a vehicle are described in more detail in U.S. Provisional Patent Application No. 61/657,744, assigned to the assignee of the instant application, which is hereby incorporated by reference in its entirety. Several specific examples of how to determine certain contexts are provided below. In various embodiments, different techniques and/or information sources are used to detect these and other contexts.
As noted above, whether or not the voice trigger system 400 is active (e.g., listening) can depend on the physical orientation of a device. In some implementations, the voice trigger is active when the device is placed “face-up” on a surface (e.g., with the display and/or touchscreen surface visible), and/or is inactive when it is “face-down.” This provides a user with an easy way to activate and/or deactivate the voice trigger without requiring manipulation of settings menus, switches, or buttons. In some implementations, the device detects whether it is face-up or face-down on a surface using light sensors (e.g., based on the difference in incident light on a front and a back face of the device 104), proximity sensors, magnetic sensors, accelerometers, gyroscopes, tilt sensors, cameras, and the like.
In some implementations, other operating modes, settings, parameters, or preferences are affected by the orientation and/or position of the device. In some implementations, the particular trigger sound, word, or phrase of the voice trigger is listening for depends on the orientation and/or position of the device. For example, in some implementations, the voice trigger listens for a first trigger word, phrase, or sound when the device is in one orientation (e.g., laying face-up on a surface), and a different trigger word, phrase, or sound when the device is in another orientation (e.g., laying face-down). In some implementations, the trigger phrase for a face-down orientation is longer and/or more complex than for a face-up orientation. Thus, a user can place a device face-down when they are around other people or in a noisy environment so that the voice trigger can still be operational while also reducing false accepts, which may be more frequent for shorter or simpler trigger words. As a specific example, a face-up trigger phrase may be “Hey, SIRI,” while a face-down trigger phrase may be “Hey, SIRI, this is Andrew, please wake up.” The longer trigger phrase also provides a larger voice sample for the sound detectors and/or voice authenticators to process and/or analyze, thus increasing the accuracy of the voice trigger and decreasing false accepts.
In some implementations, the device 104 detects whether it is in a vehicle (e.g., a car). A voice trigger is particularly beneficial for invoking a speech-based service when the user is in a vehicle, as it helps reduce the physical interactions that are necessary to operate the device and/or the speech based service. Indeed, one of the benefits of a voice-based digital assistant is that it can be used to perform tasks where looking at and touching a device would be impractical or unsafe. Thus, the voice trigger may be used when the device is in a vehicle so that the user does not have to touch the device in order to invoke the digital assistant. In some implementations, the device determines that it is in a vehicle by detecting that it has been connected to and/or paired with a vehicle, such as through BLUETOOTH communications (or other wireless communications) or through a docking connector or cable. In some implementations, the device determines that it is in a vehicle by determining the device's location and/or speed (e.g., using GPS receivers, accelerometers, and/or gyroscopes). If it is determined that the device is likely in a vehicle, because it is travelling above 20 miles per hour and is determined to be travelling along a road, for example, then the voice trigger remains active and/or in a high-power or more sensitive state.
In some implementations, the device detects whether the device is stored (e.g., in a pocket, purse, bag, a drawer, or the like) by determining whether it is in a substantially enclosed space. In some implementations, the device uses light sensors (e.g., dedicated ambient light sensors and/or cameras) to determine that it is stored. For example, in some implementations, the device is likely being stored if light sensors detect little or no light. In some implementations, the time of day and/or location of the device are also considered. For example, if the light sensors detect low light levels when high light levels would be expected (e.g., during the day), the device may be in storage and the voice trigger system 400 not needed. Thus, the voice trigger system 400 will be placed in a low-power or standby state.
In some implementations, the difference in light detected by sensors located on opposite faces of a device can be used to determine its position, and hence whether or not it is stored. Specifically, users are likely to attempt to activate a voice trigger when the device is resting on a table or surface rather than when it is being stored in a pocket or bag. But when a device is lying face-down (or face-up) on a surface such as a table or desk, one surface of the device will be occluded so that little or no light reaches that surface, while the other surface will be exposed to ambient light. Thus, if light sensors on the front and back face of a device detect significantly different light levels, the device determines that it is not being stored. On the other hand, if light sensors on opposite faces detect the same or similar light levels, the device determines that it is being stored in a substantially enclosed space. Also, if the light sensors both detect a low light level during the daytime (or when the device would expect the phone to be in a bright environment, the device determines with a greater confidence that it is being stored.
In some implementations, other techniques are used (instead of or in addition to light sensors) to determine whether the device is stored. For example, in some implementations, the device emits one or more sounds (e.g., tones, clicks, pings, etc.) from a speaker or transducer (e.g., speaker 228), and monitors one or more microphones or transducers (e.g., microphone 230) to detect echoes of the omitted sound(s). (In some implementations, the device emits inaudible signals, such as sound outside of the human hearing range.) From the echoes, the device determines characteristics of the surrounding environment. For example, a relatively large environment (e.g., a room or a vehicle) will reflect the sound differently than a relatively small, enclosed environment (e.g., a pocket, purse, bag, a drawer, or the like).
In some implementations, the voice trigger system 400 is operates differently if it is near other devices (such as other devices that have voice triggers and/or speech-based services) than if it is not near other devices. This may be useful, for example, to shut down or decrease the sensitivity of the voice trigger system 400 when many devices are close together so that if one person utters a trigger word, other surrounding devices are not triggered as well. In some implementations, a device determines proximity to other devices using RFID, near-field communications, infrared/acoustic signals, or the like.
As noted above, voice triggers are particularly useful when a device is being operated in a hands-free mode, such as when the user is driving. In such cases, users often use external audio systems, such as wired or wireless headsets, watches with speakers and/or microphones, a vehicle's built-in microphones and speakers, etc., to free themselves from having to hold a device near their face to make a call or dictate text inputs. For example, wireless headsets and vehicle audio systems may connect to an electronic device using BLUETOOTH communications, or any other appropriate wireless communication. However, it may be inefficient for a voice trigger to monitor audio received via a wireless audio accessory because of the power required to maintain an open audio channel with the wireless accessory. In particular, a wireless headset may hold enough charge in its battery to provide a few hours of continuous talk-time, and it is therefore preferable to reserve the battery for when the headset is needed for actual communication, instead of using it to simply monitor ambient audio and wait for a possible trigger sound. Moreover, wired external headset accessories may require significantly more power than on-board microphones alone, and keeping the headset microphone active will deplete the device's battery charge. This is especially true considering that the ambient audio received by the wireless or wired headset will typically consist mostly of silence or irrelevant sounds. Thus, in some implementations, the voice trigger system 400 monitors audio from the microphone 230 on the device even when the device is coupled to an external microphone (wired or wireless). Then, when the voice trigger detects the trigger word, the device initializes an active audio link with the external microphone in order to receive subsequent sound inputs (such as a command to a voice-based digital assistant) via the external microphone rather than the on-device microphone 230.
When certain conditions are met, though, an active communication link can be maintained between an external audio system 416 (which may be communicatively coupled to the device 104 via wires or wirelessly) and the device so that the voice trigger system 400 can listen for a trigger sound via the external audio system 416 instead of (or in addition to) the on-device microphone 230. For example, in some implementations, characteristics of the motion of the electronic device and/or the external audio system 416 (e.g., as determined by accelerometers, gyroscopes, etc. on the respective devices) are used to determine whether the voice trigger system 400 should monitor ambient sound using the on-device microphone 230 or an external microphone 418. Specifically, the difference between the motion of the device and the external audio system 416 provides information about whether the external audio system 416 is actually in use. For example, if both the device and a wireless headset are moving (or not moving) substantially identically, it may be determined that the headset is not in use or is not being worn. This may occur, for example, because both devices are near to each other and idle (e.g., sitting on a table or stored in a pocket, bag, purse, drawer, etc.). Accordingly, under these conditions, the voice trigger system 400 monitors the on-device microphone, because it is unlikely that the headset is actually being used. If there is a difference in motion between the wireless headset and the device, however, it is determined that the headset is being worn by a user. These conditions may occur, for example, because the device has been set down (e.g., on a surface or in a bag), while the headset is being worn on the user's head (which will likely move at least a small amount, even when the wearer is relatively still). Under these conditions, because it is likely that the headset is being worn, the voice trigger system 400 maintains an active communication link and monitors the microphone 418 of the headset instead of (or in addition to) the on-device microphone 230. And because this technique focuses on the difference in the motion of the device and the headset, motion that is common to both devices can be canceled out. This may be useful, for example, when a user is using a headset in a moving vehicle, where the device (e.g., a cellular phone) is resting in a cup holder, empty seat, or in the user's pocket, and the headset is worn on the user's head. Once the motion that is common to both devices is cancelled out (e.g., the vehicle's motion), the relative motion of the headset as compared to the device (if any) can be determined in order to determine whether the headset is likely in use (or, whether the headset is not being worn). While the above discussion refers to wireless headsets, similar techniques are applied to wired headsets as well.
Because people's voices vary greatly, it may be necessary or beneficial to tune a voice trigger to improve its accuracy in recognizing the voice of a particular user. Also, people's voices may change over time, for example, because of illnesses, natural voice changes relating to aging or hormonal changes, and the like. Thus, in some implementations, the voice trigger system 400 is able to adapt its voice and/or sound recognition profiles for a particular user or group of users.
As described above, sound detectors (e.g., the sound-type detector 404 and/or the trigger sound detector 406) may be configured to compare a representation of a sound input (e.g., the sound or utterance provided by a user) to one or more reference representations. For example, if an input representation matches the reference representation to a predetermined confidence level, the sound detector will determine that the sound input corresponds to a predetermined type of sound (e.g., the sound-type detector 404), or that the sound input includes predetermined content (e.g., the trigger sound detector 406). In order to tune the voice trigger system 400, in some implementations, the device adjusts the reference representation to which the input representation is compared. In some implementations, the reference representation is adjusted (or created) as part of a voice enrollment or “training” procedure, where a user outputs the trigger sound several times so that the device can adjust (or create) the reference representation. The device can then create a reference representation using that person's actual voice.
In some implementations, the device uses trigger sounds that are received under normal use conditions to adjust the reference representation. For example, after a successful voice triggering event (e.g., where the sound input was found to satisfy all of the triggering criteria) the device will use information from the sound input to adjust and/or tune the reference representation. In some implementations, only sound inputs that were determined to satisfy all or some of the triggering criteria with a certain confidence level are used to adjust the reference representation. Thus, when the voice trigger is less confident that a sound input corresponds to or includes a trigger sound, that voice input may be ignored for the purposes of adjusting the reference representation. On the other hand, in some implementations, sound inputs that satisfied the voice trigger system 400 to a lower confidence are used to adjust the reference representation.
In some implementations, the device 104 iteratively adjusts the reference representation (using these or other techniques) as more and more sound inputs are received so that slight changes in a user's voice over time can be accommodated. For example, in some implementations, the device 104 (and/or associated devices or services) adjusts the reference representation after each successful triggering event. In some implementations, the device 104 analyzes the sound input associated with each successful triggering event and determines if the reference representations should be adjusted based on that input (e.g., if certain conditions are met), and only adjusts the reference representation if it is appropriate to do so. In some implementations, the device 104 maintains a moving average of the reference representation over time.
In some implementations, the voice trigger system 400 detects sounds that do not satisfy one or more of the triggering criteria (e.g., as determined by one or more of the sound detectors), but that may actually be attempts by an authorized user to do so. For example, voice trigger system 400 may be configured to respond to a trigger phrase such as “Hey, SIRI”, but if a user's voice has changed (e.g., due to sickness, age, accent/inflection changes, etc.), the voice trigger system 400 may not recognize the user's attempt to activate the device. (This may also occur when the voice trigger system 400 has not been properly tuned for that user's particular voice, such as when the voice trigger system 400 is set to default conditions and/or the user has not performed an initialization or training procedure to customize the voice trigger system 400 for his or her voice.) If the voice trigger system 400 does not respond to the user's first attempt to active the voice trigger, the user is likely to repeat the trigger phrase. The device detects that these repeated sound inputs are similar to one another, and/or that they are similar to the trigger phrase (though not similar enough to cause the voice trigger system 400 to activate the speech-based service). If such conditions are met, the device determines that the sound inputs correspond to valid attempts to activate the voice trigger system 400. Accordingly, in some implementations, the voice trigger system 400 uses those received sound inputs to adjust one or more aspects of the voice trigger system 400 so that similar utterances by the user will be accepted as valid triggers in the future. In some implementations, these sound inputs are used to adapt the voice trigger system 400 only if a certain conditions or combinations of conditions are met. For example, in some implementations, the sound inputs are used to adapt the voice trigger system 400 when a predetermined number of sound inputs are received in succession (e.g., 2, 3, 4, 5, or any other appropriate number), when the sound inputs are sufficiently similar to the reference representation, when the sound inputs are sufficiently similar to each other, when the sound inputs are close together (e.g., when they are received within a predetermined time period and/or at or near a predetermined interval), and/or any combination of these or other conditions.
In some cases, the voice trigger system 400 may detect one or more sound inputs that do not satisfy one or more of the triggering criteria, followed by a manual initiation of the speech-based service (e.g., by pressing a button or icon). In some implementations, the voice trigger system 400 determines that, because speech-based service was initiated shortly after the sound inputs were received, the sound inputs actually corresponded to failed voice triggering attempts. Accordingly, the voice trigger system 400 uses those received sound inputs to adjust one or more aspects of the voice trigger system 400 so that utterances by the user will be accepted as valid triggers in the future, as described above.
While the adaptation techniques described above refer to adjusting a reference representation, other aspects of the trigger sound detecting techniques may be adjusted in the same or similar manner in addition to or instead of adjusting the reference representation. For example, in some implementations, the device adjusts how sound inputs are filtered and/or what filters are applied to sound inputs, such as to focus on and/or eliminate certain frequencies or ranges of frequencies of a sound input. In some implementations, the device adjusts an algorithm that is used to compare the input representation with the reference representation. For example, in some implementations, one or more terms of a mathematical function used to determine the difference between an input representation and a reference representation are changed, added, or removed, or a different mathematical function is substituted.
In some implementations, adaptation techniques such as those described above require more resources than the voice trigger system 400 is able to or is configured to provide. In particular, the sound detectors may not have, or have access to, the amount or the types of processors, data, or memory that are necessary to perform the iterative adaptation of a reference representation and/or a sound detection algorithm (or any other appropriate aspect of the voice trigger system 400). Thus, in some implementations, one or more of the above described adaptation techniques are performed by a more powerful processor, such as an application processor (e.g., the processor(s) 204), or by a different device (e.g., the server system 108). However, the voice trigger system 400 is designed to operate even when the application processor is in a standby mode. Thus, the sound inputs which are to be used to adapt the voice trigger system 400 are received when the application processor is not active and cannot process the sound input. Accordingly, in some implementations, the sound input is stored by the device so that it can be further processed and/or analyzed after it is received. In some implementations, the sound input is stored in the memory buffer 414 of the audio subsystem 226. In some implementations, the sound input is stored in system memory (e.g., memory 250,
In some implementations, the electronic device determines whether the sound input satisfies a predetermined condition (504). In some implementations, the electronic device applies time-domain analysis to the sound input to determine whether the sound input satisfies the predetermined condition. For example, the electronic device analyzes the sound input over a time period in order to determine whether the sound amplitude reaches a predetermined level. In some implementations, the threshold is satisfied if the amplitude (e.g., the volume) of the sound input meets and/or exceeds a predetermined threshold. In some implementations, it is satisfied if the sound input meets and/or exceeds a predetermined threshold for a predetermined amount of time. As discussed in more detail below, in some implementations, determining whether the sound input satisfies the predetermined condition (504) is performed by a third sound detector (e.g., the noise detector 402). (The third sound detector is used in this case to differentiate the sound detector from other sound detectors (e.g., the first and second sound detectors that are discussed below), and does not necessarily indicate any operational position or order of the sound detectors.)
The electronic device determines whether the sound input corresponds to a predetermined type of sound (506). As noted above, sounds are categorized as different “types” based on certain identifiable characteristics of the sounds. Determining whether the sound input corresponds to a predetermined type includes determining whether the sound input includes or exhibits the characteristics of a particular type. In some implementations, the predetermined type of sound is a human voice. In such implementations, determining whether the sound input corresponds to a human voice includes determining whether the sound input includes frequencies characteristic of a human voice (508). As discussed in more detail below, in some implementations, determining whether the sound input corresponds to a predetermined type of sound (506) is performed by a first sound detector (e.g., the sound-type detector 404).
Upon a determination that the sound input corresponds to the predetermined type of sound, the electronic device determines whether the sound input includes predetermined content (510). In some implementations, the predetermined content corresponds to one or more predetermined phonemes (512). In some implementations, the one or more predetermined phonemes constitute at least one word. In some implementations, the predetermined content is a sound (e.g., a whistle, click, or clap). In some implementations, as discussed below, determining whether the sound input includes predetermined content (510) is performed by a second sound detector (e.g., the trigger sound detector 406).
Upon a determination that the sound input includes the predetermined content, the electronic device initiates a speech-based service (514). In some implementations, the speech-based service is a voice-based digital assistant, as described in detail above. In some implementations, the speech-based service is a dictation service in which speech inputs are converted into text and included in and/or displayed in a text input field (e.g., of an email, text message, word processing or note-taking application, etc.). In implementations where the speech-based service is a voice-based digital assistant, once the voice-based digital assistant is initiated, a prompt is issued to the user (e.g., a sound or a speech prompt) indicating that the user may provide a voice input and/or command to the digital assistant. In some implementations, initiating the voice-based digital assistant includes activating an application processor (e.g., the processor(s) 204,
In some implementations, the electronic device determines whether the sound input corresponds to a voice of a particular user (516). For example, one or more voice authentication techniques are applied to the sound input to determine whether it corresponds to the voice of an authorized user of the device. Voice authentication techniques are described in greater detail above. In some implementations, voice authentication is performed by one of the sound detectors (e.g., the trigger sound detector 406). In some implementations, voice authentication is performed by a dedicated voice authentication module (including any appropriate hardware and/or software).
In some implementations, the sound-based service is initiated in response to a determination that the sound input includes the predetermined content and the sound input corresponds to the voice of the particular user. Thus, for example, the sound-based service (e.g., a voice-based digital assistant) will only be initiated when the trigger word or phrase is spoken by an authorized user. This reduces the possibility that the service can be invoked by an unauthorized user, and may be particularly useful when multiple electronic devices are in close proximity, as one user's utterance of a trigger sound will not activate another user's voice trigger.
In some implementations, where the speech-based service is a voice-based digital assistant, in response to determining that the sound input includes the predetermined content but does not correspond to the voice of the particular user, the voice-based digital assistant is initiated in a limited access mode. In some implementations, the limited access mode allows the digital assistant to access only a subset of the data, services, and/or functionality that the digital assistant can otherwise provide. In some implementations, the limited access mode corresponds to a write-only mode (e.g., so that an unauthorized user of the digital assistant cannot access data from calendars, task lists, contacts, photographs, emails, text messages, etc). In some implementations, the limited access mode corresponds to a sandboxed instance of a speech-based service, so that the speech-based service will not read from or write to a user's data, such as user data 266 on the device 104 (
In some implementations, in response to a determination that the sound input includes the predetermined content and the sound input corresponds to the voice of the particular user, the voice-based digital assistant outputs a prompt including a name of the particular user. For example, when a particular user is identified via voice authentication, the voice-based digital assistant may output a prompt such as “What can I help you with, Peter?”, instead of a more generic prompt such as a tone, beep, or non-personalized voice prompt.
As noted above, in some implementations, a first sound detector determines whether the sound input corresponds to a predetermined type of sound (at step 506), and a second sound detector determines whether the sound detector includes the predetermined content (at step 510). In some implementations, the first sound detector consumes less power while operating than the second sound detector, for example, because the first sound detector uses a less processor-intensive technique than the second sound detector. In some implementations, the first sound detector is the sound-type detector 404, and the second sound detector is the trigger sound detector 406, both of which are discussed above with respect to
In some implementations, the first and/or the sound detector performs frequency-domain analysis of the sound input. For example, these sound detectors perform a Laplace, Z-, or Fourier transform to generate a frequency spectrum or to determine the spectral density of the sound input or a portion thereof. In some implementations, the first sound detector is a voice-activity detector that is configured to determine whether the sound input includes frequencies that are characteristic of a human voice (or other features, aspects, or properties of the sound input that are characteristic of a human voice).
In some implementations, the second sound detector is off or inactive until the first sound detector detects a sound input of the predetermined type. Accordingly, in some implementations, the method 500 includes initiating the second sound detector in response to determining that the sound input corresponds to the predetermined type. (In other implementations, the second sound detector is initiated in response to other conditions, or is continuously operated regardless of a determination from the first sound detector.) In some implementations, initiating the second sound detector includes activating hardware and/or software (including, for example, circuits, processors, programs, memory, etc.).
In some implementations, the second sound detector is operated (e.g., is active and is monitoring an audio channel) for at least a predetermined amount of time after it is initiated. For example, when the first sound detector determines that the sound input corresponds to a predetermined type (e.g., includes a human voice), the second sound detector is operated in order to determine if the sound input also includes the predetermined content (e.g., the trigger word). In some implementations, the predetermined amount of time corresponds to a duration of the predetermined content. Thus, if the predetermined content is the phrase “Hey, SIRI,” the predetermined amount of time will be long enough to determine if that phrase was uttered (e.g., 1 or 2 seconds, or any another appropriate duration). If the predetermined content is longer, such as the phrase “Hey, SIRI, please wake up and help me out,” the predetermined time will be longer (e.g., 5 seconds, or another appropriate duration). In some implementations, the second sound detector operates as long as the first sound detector detects sound corresponding to the predetermined type. In such implementations, for example, as long as the first sound detector detects human speech in a sound input, the second sound detector will process the sound input to determine if it includes the predetermined content.
As noted above, in some implementations, a third sound detector (e.g., the noise detector 402) determines whether the sound input satisfies a predetermined condition (at step 504). In some implementations, the third sound detector consumes less power while operating than the first sound detector. In some implementations, the third sound detector periodically monitors an audio channel according to a duty cycle, as discussed above with respect to
Similar to the discussion above with respect to initiating the second sound detector (e.g., a trigger sound detector 406) in response to a determination by the first sound detector (e.g., the sound-type detector 404), in some implementations, the first sound detector is initiated in response to a determination by the third sound detector (e.g., the noise detector 402). For example, in some implementations, the sound-type detector 404 is initiated in response to a determination by the noise detector 402 that the sound input satisfies a predetermined condition (e.g., is above a certain volume for a sufficient duration). In some implementations, initiating the first sound detector includes activating hardware and/or software (including, for example, circuits, processors, programs, memory, etc.). In other implementations, the first sound detector is initiated in response to other conditions, or is continuously operated.
In some implementations, the device stores at least a portion of the sound input in memory (518). In some implementations, the memory is the buffer 414 of the audio subsystem 226 (
In various implementations, steps (516)-(520) are performed at different positions within the method 500. For example, in some implementations, one or more of steps (516)-(520) are performed between steps (502) and (504), between steps (510) and (514), or at any other appropriate position.
Upon a determination that the electronic device is in the predetermined orientation, the electronic device activates a predetermined mode of a voice trigger (604). In some implementations, the predetermined orientation corresponds to a display screen of the device being substantially horizontal and facing down, and the predetermined mode is a standby mode (606). For example, in some implementations, if a smartphone or tablet is placed on a table or desk so that the screen is facing down, the voice trigger is placed in a standby mode (e.g., turned off) to prevent inadvertent activation of the voice trigger.
On the other hand, in some implementations, the predetermined orientation corresponds to a display screen of the device being substantially horizontal and facing up, and the predetermined mode is a listening mode (608). Thus, for example, if a smartphone or tablet is placed on a table or desk so that the screen is facing up, the voice trigger is placed in a listening mode so that it can respond to the user when it detects the trigger.
The electronic device determines whether it is in a substantially enclosed space by detecting that one or more of a microphone and a camera of the electronic device is occluded (704). In some implementations, a substantially enclosed space includes a pocket, purse, bag, drawer, glovebox, briefcase, or the like.
As described above, in some implementations, a device detects that a microphone is occluded by emitting one or more sounds (e.g., tones, clicks, pings, etc.) from a speaker or transducer, and monitoring one or more microphones or transducers to detect echoes of the omitted sound(s). For example, a relatively large environment (e.g., a room or a vehicle) will reflect the sound differently than a relatively small, substantially enclosed environment (e.g., a purse or pocket). Thus, if the device detects that the microphone (or the speaker that emitted the sounds) is occluded based on the echoes (or lack thereof), the device determines that it is in a substantially enclosed space. In some implementations, the device detects that a microphone is occluded by detecting that the microphone is picking up a sound characteristic of an enclosed space. For example, when a device is in a pocket, the microphone may detect a characteristic rustling noise due to the microphone coming into contact or close proximity with the fabric of the pocket.
In some implementations, a device detects that a camera is occluded based on the level of light received by a sensor, or by determining whether it can achieve a focused image. For example, if a camera sensor detects a low level of light during a time when a high level of light would be expected (e.g., during daylight hours), then the device determines that the camera is occluded, and that the device is in a substantially enclosed space. As another example, the camera may attempt to achieve an in-focus image on its sensor. Usually, this will be difficult if the camera is in an extremely dark place (e.g., a pocket or backpack), or if it is too close to the object on which it is attempting to focus (e.g., the inside of a purse or backpack). Thus, if the camera is unable to achieve an in-focus image, it determines that the device is in a substantially enclosed space.
Upon a determination that the electronic device is in a substantially enclosed space, the electronic device switches the voice trigger to a second mode (706). In some implementations, the second mode is a standby mode (708). In some implementations, when in the standby mode, the voice trigger system 400 will continue to monitor ambient audio, but will not respond to received sounds regardless of whether they would otherwise trigger the voice trigger system 400. In some implementations, in the standby mode, the voice trigger system 400 is deactivated, and does not process audio to detect trigger sounds. In some implementations, the second mode includes operating one or more sound detectors of a voice trigger system 400 according to a different duty cycle than the first mode. In some implementations, the second mode includes operating a different combination of sound detectors than the first mode.
In some implementations, the second mode corresponds to a more sensitive monitoring mode, so that the voice trigger system 400 can detect and respond to a trigger sound even though it is in a substantially enclosed space.
In some implementations, once the voice trigger is switched to the second mode, the device periodically determines whether the electronic device is still in a substantially enclosed space by detecting whether one or more of a microphone and a camera of the electronic device is occluded (e.g., using any of the techniques described above with respect to step (704)). If the device remains in a substantially enclosed space, the voice trigger system 400 will be kept in the second mode. In some implementations, if the device is removed from a substantially enclosed space, the electronic device will return the voice trigger to the first mode.
In accordance with some implementations,
As shown in
The processing unit 806 is configured to: determine whether at least a portion of the sound input corresponds to a predetermined type of sound (e.g., with the sound type detecting unit 810); upon a determination that at least a portion of the sound input corresponds to the predetermined type, determine whether the sound input includes predetermined content (e.g., with the trigger sound detecting unit 812); and upon a determination that the sound input includes the predetermined content, initiate a speech-based service (e.g., with the service initiating unit 814).
In some implementations, the processing unit 806 is also configured to, prior to determining whether the sound input corresponds to a predetermined type of sound, determine whether the sound input satisfies a predetermined condition (e.g., with the noise detecting unit 808). In some implementations, the processing unit 806 is also configured to determine whether the sound input corresponds to a voice of a particular user (e.g., with the voice authenticating unit 816).
In accordance with some implementations,
As shown in
In some implementations, the processing unit 906 is configured to: determine whether the electronic device is in a substantially enclosed space by detecting that one or more of a microphone and a camera of the electronic device is occluded (e.g., with the environment detecting unit 908); and upon a determination that the electronic device is in a substantially enclosed space, switching the voice trigger to a second mode (e.g., with the mode switching unit 910).
In some implementations, the processing unit is configured to: determine whether the electronic device is in a predetermined orientation (e.g., with the environment detecting unit 908); and upon a determination that the electronic device is in the predetermined orientation, activate a predetermined mode of a voice trigger (e.g., with the mode switching unit 910).
In accordance with some implementations,
As shown in
The processing unit 1006 is configured to: determine whether the electronic device is in a substantially enclosed space by detecting that one or more of a microphone and a camera of the electronic device is occluded (e.g., with the environment detecting unit 1008); and upon a determination that the electronic device is in a substantially enclosed space, switching the voice trigger to a second mode (e.g., with the mode switching unit 1010).
The foregoing description, for purpose of explanation, has been described with reference to specific implementations. However, the illustrative discussions above are not intended to be exhaustive or to limit the disclosed implementations to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The implementations were chosen and described in order to best explain the principles and practical applications of the disclosed ideas, to thereby enable others skilled in the art to best utilize them with various modifications as are suited to the particular use contemplated.
It will be understood that, although the terms “first,” “second,” etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first sound detector could be termed a second sound detector, and, similarly, a second sound detector could be termed a first sound detector, without changing the meaning of the description, so long as all occurrences of the “first sound detector” are renamed consistently and all occurrences of the “second sound detector” are renamed consistently. The first sound detector and the second sound detector are both sound detectors, but they are not the same sound detector.
The terminology used herein is for the purpose of describing particular implementations only and is not intended to be limiting of the claims. As used in the description of the implementations and the appended claims, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and/or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
As used herein, the term “if” may be construed to mean “when” or “upon” or “in response to determining” or “in accordance with a determination” or “in response to detecting,” that a stated condition precedent is true, depending on the context. Similarly, the phrase “if it is determined [that a stated condition precedent is true]” or “if [a stated condition precedent is true]” or “when [a stated condition precedent is true]” may be construed to mean “upon determining” or “upon a determination that” or “in response to determining” or “in accordance with a determination” or “upon detecting” or “in response to detecting” that the stated condition precedent is true, depending on the context.
Tackin, Onur, Binder, Justin, Post, Samuel D., Gruber, Thomas R.
Patent | Priority | Assignee | Title |
11152001, | Dec 20 2018 | Synaptics Incorporated | Vision-based presence-aware voice-enabled device |
11217243, | Jul 17 2019 | GOOGLE LLC | Systems and methods to verify trigger keywords in acoustic-based digital assistant applications |
11461779, | Mar 23 2018 | Amazon Technologies, Inc | Multi-speechlet response |
11783825, | Apr 10 2015 | HONOR DEVICE CO , LTD | Speech recognition method, speech wakeup apparatus, speech recognition apparatus, and terminal |
11869504, | Jul 17 2019 | GOOGLE LLC | Systems and methods to verify trigger keywords in acoustic-based digital assistant applications |
Patent | Priority | Assignee | Title |
10037758, | Mar 31 2014 | Mitsubishi Electric Corporation | Device and method for understanding user intent |
10049663, | Jun 08 2016 | Apple Inc | Intelligent automated assistant for media exploration |
10074360, | Sep 30 2014 | Apple Inc. | Providing an indication of the suitability of speech recognition |
10096319, | Mar 13 2017 | Amazon Technologies, Inc | Voice-based determination of physical and emotional characteristics of users |
10102359, | Mar 21 2011 | Apple Inc. | Device access using voice authentication |
10170123, | May 30 2014 | Apple Inc | Intelligent assistant for home automation |
5764852, | Aug 16 1994 | International Business Machines Corporation | Method and apparatus for speech recognition for distinguishing non-speech audio input events from speech audio input events |
5907597, | Aug 05 1994 | SMART TONE, INC , A DELWARE CORPORATION | Method and system for the secure communication of data |
5983186, | Aug 21 1995 | Seiko Epson Corporation | Voice-activated interactive speech recognition device and method |
6070140, | Jun 05 1995 | Muse Green Investments LLC | Speech recognizer |
6397186, | Dec 22 1999 | AMBUSH INTERACTIVE, INC | Hands-free, voice-operated remote control transmitter |
6532447, | Jun 07 1999 | CLUSTER LLC | Apparatus and method of controlling a voice controlled operation |
6671672, | Mar 30 1999 | Nuance Communications | Voice authentication system having cognitive recall mechanism for password verification |
7475010, | Sep 03 2003 | PRJ HOLDING COMPANY, LLC | Adaptive and scalable method for resolving natural language ambiguities |
7475015, | Sep 05 2003 | Microsoft Technology Licensing, LLC | Semantic language modeling and confidence measurement |
7475063, | Apr 19 2006 | GOOGLE LLC | Augmenting queries with synonyms selected using language statistics |
7477238, | Aug 31 2004 | Malikie Innovations Limited | Handheld electronic device with text disambiguation |
7477240, | Sep 21 2001 | Lenovo PC International | Input apparatus, computer apparatus, method for identifying input object, method for identifying input object in keyboard, and computer program |
7478037, | Dec 31 1992 | Apple Inc | Assigning meanings to utterances in a speech recognition system |
7478091, | Apr 15 2002 | WRP IP MANAGEMENT, LLC | System and method for measuring image similarity based on semantic meaning |
7478129, | Apr 18 2000 | Method and apparatus for providing group interaction via communications networks | |
7479948, | Apr 25 2006 | LG Electronics Inc | Terminal and method for entering command in the terminal |
7479949, | Sep 06 2006 | Apple Inc. | Touch screen device, method, and graphical user interface for determining commands by applying heuristics |
7483832, | Dec 10 2001 | Cerence Operating Company | Method and system for customizing voice translation of text to speech |
7483894, | Jun 07 2006 | TAMIRAS PER PTE LTD , LLC | Methods and apparatus for entity search |
7487089, | Jun 05 2001 | Sensory, Incorporated | Biometric client-server security system and method |
7487093, | Apr 02 2002 | Canon Kabushiki Kaisha | Text structure for voice synthesis, voice synthesis method, voice synthesis apparatus, and computer program thereof |
7490034, | Apr 30 2002 | Microsoft Technology Licensing, LLC | Lexicon with sectionalized data and method of using the same |
7490039, | Dec 13 2001 | Cisco Technology, Inc. | Text to speech system and method having interactive spelling capabilities |
7493251, | May 30 2003 | Microsoft Technology Licensing, LLC | Using source-channel models for word segmentation |
7493560, | May 20 2002 | Oracle International Corporation | Definition links in online documentation |
7496498, | Mar 24 2003 | Microsoft Technology Licensing, LLC | Front-end architecture for a multi-lingual text-to-speech system |
7496512, | Apr 13 2004 | Microsoft Technology Licensing, LLC | Refining of segmental boundaries in speech waveforms using contextual-dependent models |
7499923, | Mar 05 2003 | Hewlett Packard Enterprise Development LP | Document clustering method and apparatus based on common information of documents |
7502738, | May 11 2007 | DIALECT, LLC | Systems and methods for responding to natural language speech utterance |
7505795, | Jul 07 2004 | Advanced Micro Devices, Inc. | Power save management with customized range for user configuration and tuning value based upon recent usage |
7508324, | Aug 06 2004 | Cerence Operating Company | Finger activated reduced keyboard and a method for performing text input |
7508373, | Jan 28 2005 | Microsoft Technology Licensing, LLC | Form factor and input method for language input |
7516123, | Apr 14 2005 | International Business Machines Corporation | Page rank for the semantic web query |
7519327, | Mar 05 2004 | Apple Inc | Athletic monitoring system and method |
7519398, | Apr 30 2004 | Casio Computer Col, Ltd. | Communication terminal apparatus and a communication processing program |
7522927, | Nov 03 1998 | Unwired Planet, LLC | Interface for wireless location information |
7523036, | Jun 01 2001 | Sony Corporation | Text-to-speech synthesis system |
7523108, | Jun 07 2006 | TAMIRAS PER PTE LTD , LLC | Methods and apparatus for searching with awareness of geography and languages |
7526466, | May 28 1998 | DATACLOUD TECHNOLOGIES, LLC | Method and system for analysis of intended meaning of natural language |
7526738, | Dec 20 1999 | Apple Inc. | User interface for providing consolidation and access |
7528713, | Sep 28 2006 | Scenera Mobile Technologies, LLC | Apparatus and method for providing a task reminder based on travel history |
7529671, | Mar 04 2003 | Microsoft Technology Licensing, LLC | Block synchronous decoding |
7529676, | Dec 05 2003 | RAKUTEN GROUP, INC | Audio device control device, audio device control method, and program |
7529677, | Jan 21 2005 | Exelis Inc | Methods and apparatus for remotely processing locally generated commands to control a local device |
7535997, | Jul 29 2002 | AT&T Intellectual Property I, L P | Systems and methods for silent message delivery |
7536029, | Sep 30 2004 | SAMSUNG ELECTRONICS CO , LTD | Apparatus and method performing audio-video sensor fusion for object localization, tracking, and separation |
7536565, | Jan 07 2005 | Apple Inc | Techniques for improved playlist processing on media devices |
7538685, | Jun 28 2005 | ARLINGTON TECHNOLOGIES, LLC | Use of auditory feedback and audio queues in the realization of a personal virtual assistant |
7539619, | Sep 05 2003 | ZAMA INNOVATIONS LLC | Speech-enabled language translation system and method enabling interactive user supervision of translation and speech recognition accuracy |
7539656, | Mar 06 2000 | AVOLIN, LLC | System and method for providing an intelligent multi-step dialog with a user |
7541940, | Feb 16 2006 | International Business Machines Corporation | Proximity-based task alerts |
7542967, | Jun 30 2005 | Microsoft Technology Licensing, LLC | Searching an index of media content |
7542971, | Feb 02 2004 | Fuji Xerox Co., Ltd.; FUJI XEROX CO , LTD ; Fuji Xerox | Systems and methods for collaborative note-taking |
7543232, | Oct 19 2004 | International Business Machines Corporation | Intelligent web based help system |
7546382, | May 28 2002 | International Business Machines Corporation | Methods and systems for authoring of mixed-initiative multi-modal interactions and related browsing mechanisms |
7546529, | Oct 05 1999 | Microsoft Technology Licensing, LLC | Method and system for providing alternatives for text derived from stochastic input sources |
7548895, | Jun 30 2006 | Microsoft Technology Licensing, LLC | Communication-prompted user assistance |
7552045, | Dec 18 2006 | Nokia Technologies Oy | Method, apparatus and computer program product for providing flexible text based language identification |
7552055, | Jan 10 2004 | Microsoft Technology Licensing, LLC | Dialog component re-use in recognition systems |
7555431, | Nov 12 1999 | Nuance Communications, Inc | Method for processing speech using dynamic grammars |
7555496, | Aug 12 1996 | Battelle Memorial Institute | Three-dimensional display of document set |
7558381, | Apr 22 1999 | AVAGO TECHNOLOGIES GENERAL IP SINGAPORE PTE LTD | Retrieval of deleted voice messages in voice messaging system |
7558730, | Nov 27 2001 | ADVANCED VOICE RECOGNITION SYSTEMS, INC | Speech recognition and transcription among users having heterogeneous protocols |
7559026, | Jun 20 2003 | Apple Inc | Video conferencing system having focus control |
7561069, | Nov 12 2003 | ELECTRONIC COMMUNICATION TECHNOLOGIES LLC | Notification systems and methods enabling a response to change particulars of delivery or pickup |
7562007, | Jun 16 2003 | Samsung Electronics Co., Ltd.; SAMSUNG ELECTRONICS, CO , LTD ; SAMSUNG ELECTRONICS CO , LTD | Method and apparatus for recognizing language input mode and method and apparatus for automatically switching language input modes using the same |
7562032, | Feb 21 2000 | Accenture Global Services Limited | Ordering items of playable content or other works |
7565104, | Jun 16 2004 | Wendell, Brown | Broadcast audio program guide |
7565380, | Mar 24 2005 | AVAGO TECHNOLOGIES GENERAL IP SINGAPORE PTE LTD | Memory optimized pattern searching |
7568151, | Jun 27 2002 | Microsoft Technology Licensing, LLC | Notification of activity around documents |
7571092, | Jul 29 2005 | Oracle America, Inc | Method and apparatus for on-demand localization of files |
7571106, | Apr 09 2007 | TAMIRAS PER PTE LTD , LLC | Methods and apparatus for freshness and completeness of information |
7577522, | Dec 05 2005 | GOOGLE LLC | Spatially associated personal reminder system and method |
7580551, | Jun 30 2003 | Research Foundation of State University of New York, The | Method and apparatus for analyzing and/or comparing handwritten and/or biometric samples |
7580576, | Jun 02 2005 | Microsoft Technology Licensing, LLC | Stroke localization and binding to electronic document |
7580839, | Jan 19 2006 | Kabushiki Kaisha Toshiba; Toshiba Digital Solutions Corporation | Apparatus and method for voice conversion using attribute information |
7584092, | Nov 15 2004 | Microsoft Technology Licensing, LLC | Unsupervised learning of paraphrase/translation alternations and selective application thereof |
7584093, | Apr 25 2005 | Microsoft Technology Licensing, LLC | Method and system for generating spelling suggestions |
7584278, | Dec 11 2000 | ServiceNow, Inc | Method and system for task based management of multiple network resources |
7584429, | Jul 01 2003 | CONVERSANT WIRELESS LICENSING S A R L | Method and device for operating a user-input area on an electronic display device |
7593868, | Jan 29 2003 | ZETA GLOBAL CORP | Systems and methods for providing contextual advertising information via a communication network |
7596269, | Apr 01 2004 | Kyocera Corporation | Triggering actions in response to optically or acoustically capturing keywords from a rendered document |
7596499, | Feb 02 2004 | Sovereign Peak Ventures, LLC | Multilingual text-to-speech system with limited resources |
7596606, | Mar 11 1999 | EASYWEB INNOVATIONS, LLC | Message publishing system for publishing messages from identified, authorized senders |
7596765, | May 23 2006 | SNAPTRACK, INC | Sound feedback on menu navigation |
7599918, | Dec 29 2005 | Microsoft Technology Licensing, LLC | Dynamic search with implicit user intention mining |
7603349, | Jul 29 2004 | R2 SOLUTIONS LLC | User interfaces for search systems using in-line contextual queries |
7603381, | Sep 30 2004 | Microsoft Technology Licensing, LLC | Contextual action publishing |
7606444, | Nov 29 2002 | Ricoh Company, Ltd. | Multimodal access of meeting recordings |
7606712, | Jun 28 2001 | Nuance Communications, Inc | Speech recognition interface for voice actuation of legacy systems |
7607083, | Dec 12 2000 | NEC Corporation | Test summarization using relevance measures and latent semantic analysis |
7609179, | Jan 08 2008 | ServiceNow, Inc; International Business Machines Corporation | Method for compressed data with reduced dictionary sizes by coding value prefixes |
7610258, | Jan 30 2004 | Microsoft Technology Licensing, LLC | System and method for exposing a child list |
7613264, | Jul 26 2005 | AVAGO TECHNOLOGIES GENERAL IP SINGAPORE PTE LTD | Flexible sampling-rate encoder |
7614008, | Jul 30 2004 | Apple Inc | Operation of a computer with touch screen interface |
7617094, | Feb 28 2003 | III Holdings 6, LLC | Methods, apparatus, and products for identifying a conversation |
7620407, | Mar 16 2003 | Qualcomm Incorporated | Handheld threading |
7620549, | Aug 10 2005 | DIALECT, LLC | System and method of supporting adaptive misrecognition in conversational speech |
7620894, | Oct 08 2003 | Apple Inc | Automatic, dynamic user interface configuration |
7623119, | Apr 21 2004 | RPX Corporation | Graphical functions by gestures |
7624007, | Nov 12 1999 | Nuance Communications, Inc | System and method for natural language processing of sentence based queries |
7627481, | Apr 19 2005 | Apple Inc | Adapting masking thresholds for encoding a low frequency transient signal in audio data |
7630900, | Dec 01 2004 | Microsoft Technology Licensing, LLC | Method and system for selecting grammars based on geographic information associated with a caller |
7630901, | Jun 29 2004 | Canon Kabushiki Kaisha | Multimodal input method |
7633076, | Sep 30 2005 | Apple Inc | Automated response to and sensing of user activity in portable devices |
7634409, | Aug 31 2005 | DIALECT, LLC | Dynamic speech sharpening |
7634413, | Feb 25 2005 | Apple Inc | Bitrate constrained variable bitrate audio encoding |
7634718, | Nov 30 2004 | Fujitsu Limited | Handwritten information input apparatus |
7634732, | Jun 26 2003 | Microsoft Technology Licensing, LLC | Persona menu |
7636657, | Dec 09 2004 | Microsoft Technology Licensing, LLC | Method and apparatus for automatic grammar generation from data entries |
7640158, | Nov 08 2005 | SOLVENTUM INTELLECTUAL PROPERTIES COMPANY | Automatic detection and application of editing patterns in draft documents |
7640160, | Aug 05 2005 | DIALECT, LLC | Systems and methods for responding to natural language speech utterance |
7643990, | Oct 23 2003 | Apple Inc | Global boundary-centric feature extraction and associated discontinuity metrics |
7647225, | Nov 12 1999 | Nuance Communications, Inc | Adjustable resource based speech recognition system |
7649454, | Sep 28 2006 | Scenera Mobile Technologies, LLC | System and method for providing a task reminder based on historical travel information |
7649877, | Oct 22 2001 | Apple Inc | Mobile device for sending text messages |
7653883, | Jul 30 2004 | Apple Inc | Proximity detector in handheld device |
7656393, | Mar 04 2005 | Apple Inc | Electronic device having display and surrounding touch sensitive bezel for user interface and control |
7657424, | Nov 12 1999 | Nuance Communications, Inc | System and method for processing sentence based queries |
7657430, | Jul 22 2004 | Sony Corporation | Speech processing apparatus, speech processing method, program, and recording medium |
7657828, | Jun 28 2000 | Nuance Communications, Inc | Method and apparatus for coupling a visual browser to a voice browser |
7657844, | Apr 30 2004 | GOOGLE LLC | Providing accessibility compliance within advanced componentry |
7657849, | Dec 23 2005 | Apple Inc | Unlocking a device by performing gestures on an unlock image |
7660715, | Jan 12 2004 | ARLINGTON TECHNOLOGIES, LLC | Transparent monitoring and intervention to improve automatic adaptation of speech models |
7663607, | May 06 2004 | Apple Inc | Multipoint touchscreen |
7664558, | Apr 01 2005 | Apple Inc | Efficient techniques for modifying audio playback rates |
7664638, | Feb 16 2001 | Nuance Communications, Inc | Tracking time using portable recorders and speech recognition |
7668710, | Dec 14 2001 | Intellectual Ventures I LLC | Determining voice recognition accuracy in a voice recognition system |
7669134, | May 02 2003 | APPLE COMPUTERS, INC | Method and apparatus for displaying information during an instant messaging session |
7672841, | Nov 12 1999 | Nuance Communications, Inc | Method for processing speech data for a distributed recognition system |
7672952, | Jul 13 2000 | RPX Corporation | System and method of semantic correlation of rich content |
7673238, | Jan 05 2006 | Apple Inc | Portable media device with video acceleration capabilities |
7673251, | Oct 02 2006 | Adobe Inc | Panel presentation |
7673340, | Jun 02 2004 | IGNITE ENTERPRISE SOFTWARE SOLUTIONS, LLC | System and method for analyzing system user behavior |
7676026, | Mar 08 2005 | Qualcomm Incorporated | Desktop telephony system |
7676365, | Dec 26 2000 | Microsoft Technology Licensing, LLC | Method and apparatus for constructing and using syllable-like unit language models |
7676463, | Nov 15 2005 | KLDiscovery Ontrack, LLC | Information exploration systems and method |
7679534, | Dec 04 1998 | Nuance Communications, Inc | Contextual prediction of user words and user actions |
7680649, | Jun 17 2002 | International Business Machines Corporation | System, method, program product, and networking use for recognizing words and their parts of speech in one or more natural languages |
7681126, | Oct 24 2006 | EDGETECH AMERICA, INC | Method for spell-checking location-bound words within a document |
7683886, | Sep 05 2006 | Malikie Innovations Limited | Disambiguated text message review function |
7683893, | Jan 20 2007 | LG Electronics Inc. | Controlling display in mobile terminal |
7684985, | Dec 10 2002 | WALOOMBA TECH LTD , L L C | Techniques for disambiguating speech input using multimodal interfaces |
7684990, | Apr 29 2005 | Nuance Communications, Inc | Method and apparatus for multiple value confirmation and correction in spoken dialog systems |
7684991, | Jan 05 2006 | Alpine Electronics, Inc | Digital audio file search method and apparatus using text-to-speech processing |
7689245, | Feb 07 2002 | Nuance Communications, Inc | System and method of ubiquitous language translation for wireless devices |
7689408, | Sep 01 2006 | Microsoft Technology Licensing, LLC | Identifying language of origin for words using estimates of normalized appearance frequency |
7689409, | Dec 17 2002 | France Telecom | Text language identification |
7689412, | Dec 05 2003 | Microsoft Technology Licensing, LLC | Synonymous collocation extraction using translation information |
7689421, | Jun 27 2007 | Microsoft Technology Licensing, LLC | Voice persona service for embedding text-to-speech features into software programs |
7693715, | Mar 10 2004 | Microsoft Technology Licensing, LLC | Generating large units of graphonemes with mutual information criterion for letter to sound conversion |
7693717, | Apr 12 2006 | CUSTOM SPEECH USA, INC | Session file modification with annotation using speech recognition or text to speech |
7693719, | Oct 29 2004 | Microsoft Technology Licensing, LLC | Providing personalized voice font for text-to-speech applications |
7693720, | Jul 15 2002 | DIALECT, LLC | Mobile systems and methods for responding to natural language speech utterance |
7698131, | Nov 12 1999 | Nuance Communications, Inc | Speech recognition system for client devices having differing computing capabilities |
7702500, | Nov 24 2004 | Method and apparatus for determining the meaning of natural language | |
7702508, | Nov 12 1999 | Nuance Communications, Inc | System and method for natural language processing of query answers |
7703091, | Dec 31 2002 | EMC IP HOLDING COMPANY LLC | Methods and apparatus for installing agents in a managed network |
7706510, | Mar 16 2005 | Malikie Innovations Limited | System and method for personalized text-to-voice synthesis |
7707026, | Mar 14 2005 | Fuji Xerox Co., Ltd. | Multilingual translation memory, translation method, and translation program |
7707027, | Apr 13 2006 | Microsoft Technology Licensing, LLC | Identification and rejection of meaningless input during natural language classification |
7707032, | Oct 20 2005 | National Cheng Kung University | Method and system for matching speech data |
7707221, | Apr 03 2002 | R2 SOLUTIONS LLC | Associating and linking compact disc metadata |
7707226, | Jan 29 2007 | Meta Platforms, Inc | Presentation of content items based on dynamic monitoring of real-time context |
7707267, | Feb 27 2001 | Microsoft Technology Licensing, LLC | Intent based processing |
7710262, | Sep 17 2004 | Nokia Corporation | Sensor data sharing |
7711129, | Mar 11 2004 | Apple Inc | Method and system for approximating graphic equalizers using dynamic filter order reduction |
7711550, | Apr 29 2003 | Microsoft Technology Licensing, LLC | Methods and system for recognizing names in a computer-generated document and for providing helpful actions associated with recognized names |
7711565, | Aug 17 2006 | WEST VIEW RESEARCH, LLC | “Smart” elevator system and method |
7711672, | May 28 1998 | DATACLOUD TECHNOLOGIES, LLC | Semantic network methods to disambiguate natural language meaning |
7712053, | Dec 04 1998 | Cerence Operating Company | Explicit character filtering of ambiguous text entry |
7716056, | Sep 27 2004 | Robert Bosch Corporation; Volkswagen of America | Method and system for interactive conversational dialogue for cognitively overloaded device users |
7716216, | Mar 31 2004 | GOOGLE LLC | Document ranking based on semantic distance between terms in a document |
7720674, | Jun 29 2004 | SAP SE | Systems and methods for processing natural language queries |
7720683, | Jun 13 2003 | Sensory, Inc | Method and apparatus of specifying and performing speech recognition operations |
7721226, | Feb 18 2004 | Microsoft Technology Licensing, LLC | Glom widget |
7721301, | Mar 31 2005 | Microsoft Technology Licensing, LLC | Processing files from a mobile device using voice commands |
7724242, | Aug 06 2004 | TOUCHTABLE, INC ; Qualcomm Incorporated | Touch driven method and apparatus to integrate and display multiple image layers forming alternate depictions of same subject matter |
7724696, | Mar 29 2006 | Amazon Technologies, Inc | Predictive reader power management |
7725307, | Nov 12 1999 | Nuance Communications, Inc | Query engine for processing voice based queries including semantic decoding |
7725318, | Jul 30 2004 | NICE SYSTEMS INC | System and method for improving the accuracy of audio searching |
7725320, | Nov 12 1999 | Nuance Communications, Inc | Internet based speech recognition system with dynamic grammars |
7725321, | Nov 12 1999 | Nuance Communications, Inc | Speech based query system using semantic decoding |
7725838, | Feb 22 1999 | Nokia Technologies Oy | Communication terminal having a predictive editor application |
7729904, | Nov 12 1999 | Nuance Communications, Inc | Partial speech processing device and method for use in distributed systems |
7729916, | Oct 02 1998 | Nuance Communications, Inc | Conversational computing via conversational virtual machine |
7734461, | Mar 03 2006 | Samsung Electronics Co., Ltd | Apparatus for providing voice dialogue service and method of operating the same |
7735012, | Nov 04 2004 | Apple Inc | Audio user interface for computing devices |
7739588, | Jun 27 2003 | Microsoft Technology Licensing, LLC | Leveraging markup language data for semantically labeling text strings and data and for providing actions based on semantically labeled text strings and data |
7742953, | Apr 01 2004 | Kyocera Corporation | Adding information or functionality to a rendered document via association with an electronic counterpart |
7743188, | Aug 12 1999 | HEWLETT-PACKARD DEVELOPMENT COMPANY, L P | Method and apparatus for accessing a contacts database and telephone services |
7747616, | Jan 10 2006 | Fujitsu Limited | File search method and system therefor |
7752152, | Mar 17 2006 | Microsoft Technology Licensing, LLC | Using predictive user models for language modeling on a personal device with user behavior models based on statistical modeling |
7756707, | Mar 26 2004 | Canon Kabushiki Kaisha | Signal processing apparatus and method |
7756708, | Apr 03 2006 | GOOGLE LLC | Automatic language model update |
7756868, | Feb 26 2004 | NHN Corporation | Method for providing search results list based on importance information and system thereof |
7756871, | Oct 13 2004 | MICRO FOCUS LLC | Article extraction |
7757173, | Jul 18 2003 | Apple Inc | Voice menu system |
7757176, | Apr 27 2007 | VAKIL, SANJAY; MUNTZ, AMY; ROMANIK, PHILIP, SANJ; MCEACHERN, WENDY; CARLSON, MAX; WYATT, GERBER & O ROURKE | Method and system for collective calendaring |
7757182, | Jun 25 2003 | Microsoft Technology Licensing, LLC | Taskbar media player |
7761296, | Apr 02 1999 | International Business Machines Corporation | System and method for rescoring N-best hypotheses of an automatic speech recognition system |
7763842, | Dec 30 2005 | HTC Corporation | Motion determination apparatus and method thereof |
7774202, | Jun 12 2006 | Lockheed Martin Corporation; LOCKHEED MARTIN COROPRATION | Speech activated control system and related methods |
7774204, | Sep 25 2003 | Sensory, Inc. | System and method for controlling the operation of a device by voice commands |
7774388, | Aug 31 2001 | Model of everything with UR-URL combination identity-identifier-addressing-indexing method, means, and apparatus | |
7777717, | Apr 05 2006 | Malikie Innovations Limited | Handheld electronic device and method for performing spell checking during text entry and for integrating the output from such spell checking into the output from disambiguation |
7778432, | Jun 06 2003 | GN RESOUND A S | Hearing aid wireless network |
7778595, | Mar 28 2000 | RPX Corporation | Method for managing media |
7778632, | Oct 28 2005 | Microsoft Technology Licensing, LLC | Multi-modal device capable of automated actions |
7778830, | May 19 2004 | Nuance Communications, Inc | Training speaker-dependent, phrase-based speech grammars using an unsupervised automated technique |
7779353, | May 19 2006 | Microsoft Technology Licensing, LLC | Error checking web documents |
7779356, | Nov 26 2003 | Microsoft Technology Licensing, LLC | Enhanced data tip system and method |
7779357, | Nov 04 2004 | Apple Inc. | Audio user interface for computing devices |
7783283, | Sep 12 2003 | Nokia Technologies Oy | Method and device for handling missed calls in a mobile communications environment |
7783486, | Nov 22 2002 | Response generator for mimicking human-computer natural language conversation | |
7788590, | Sep 26 2005 | Microsoft Technology Licensing, LLC | Lightweight reference user interface |
7788663, | Jun 08 2004 | KAHN, DAVID E | Method and system for device recruitment interoperability and assembling unified interoperating device constellation |
7796980, | Aug 11 2006 | T-MOBILE INNOVATIONS LLC | Remote mobile voice control of digital/personal video recorder |
7797265, | Feb 26 2007 | Siemens Corporation | Document clustering that applies a locality sensitive hashing function to a feature vector to obtain a limited set of candidate clusters |
7797269, | Mar 29 2007 | Nokia Technologies Oy | Method and apparatus using a context sensitive dictionary |
7797331, | Dec 20 2002 | Nokia Technologies Oy | Method and device for organizing user provided information with meta-information |
7797629, | Apr 05 2006 | Malikie Innovations Limited | Handheld electronic device and method for performing optimized spell checking during text entry by providing a sequentially ordered series of spell-check algorithms |
7801721, | Oct 02 2006 | GOOGLE LLC | Displaying original text in a user interface with translated text |
7801728, | Feb 26 2007 | Microsoft Technology Licensing, LLC | Document session replay for multimodal applications |
7801729, | Mar 13 2007 | Sensory, Inc | Using multiple attributes to create a voice search playlist |
7805299, | Mar 01 2004 | Method and apparatus for improving the transcription accuracy of speech recognition software | |
7809550, | Oct 08 2009 | System for reading chinese characters in seconds | |
7809565, | Mar 01 2003 | Method and apparatus for improving the transcription accuracy of speech recognition software | |
7809569, | Dec 22 2004 | SHADOW PROMPT TECHNOLOGY AG | Turn-taking confidence |
7809570, | Jun 03 2002 | DIALECT, LLC | Systems and methods for responding to natural language speech utterance |
7809610, | Apr 09 2007 | TAMIRAS PER PTE LTD , LLC | Methods and apparatus for freshness and completeness of information |
7809744, | Jun 19 2004 | International Business Machines Corporation | Method and system for approximate string matching |
7818165, | Apr 07 2005 | LinkedIn Corporation | Method and system for language identification |
7818176, | Feb 06 2007 | Nuance Communications, Inc; VB Assets, LLC | System and method for selecting and presenting advertisements based on natural language processing of voice-based input |
7818215, | Apr 19 2004 | Kyocera Corporation | Processing techniques for text capture from a rendered document |
7818291, | Feb 03 2006 | The General Electric Company | Data object access system and method using dedicated task object |
7818672, | Dec 30 2004 | Microsoft Technology Licensing, LLC | Floating action buttons |
7822608, | Feb 27 2007 | Microsoft Technology Licensing, LLC | Disambiguating a speech recognition grammar in a multimodal application |
7823123, | Mar 17 2006 | The MITRE Corporation | Semantic system for integrating software components |
7826945, | Jul 01 2005 | Bose Corporation | Automobile speech-recognition interface |
7827047, | Jun 24 2003 | AT&T Intellectual Property I, L P | Methods and systems for assisting scheduling with automation |
7831246, | Dec 08 2006 | AT&T MOBILITY II, LLC; Cingular Wireless II, LLC | Mobile merchant |
7831423, | May 25 2006 | 3M HEALTH INFORMATION SYSTEMS, INC | Replacing text representing a concept with an alternate written form of the concept |
7831426, | Nov 12 1999 | Nuance Communications, Inc | Network based interactive speech recognition system |
7831432, | Sep 29 2006 | International Business Machines Corporation | Audio menus describing media contents of media players |
7835504, | Mar 16 2003 | Qualcomm Incorporated | Telephone number parsing and linking |
7836437, | Feb 10 2006 | Microsoft Technology Licensing, LLC | Semantic annotations for virtual objects |
7840348, | Nov 04 2004 | LG Electronics Inc. | Output control method of voice guidance signal in navigation system |
7840400, | Mar 13 2001 | Intelligate, Ltd. | Dynamic natural language understanding |
7840447, | Oct 30 2007 | TAMIRAS PER PTE LTD , LLC | Pricing and auctioning of bundled items among multiple sellers and buyers |
7840581, | Feb 01 2008 | TAHOE RESEARCH, LTD | Method and system for improving the quality of deep metadata associated with media content |
7840912, | Jan 30 2006 | Apple Inc | Multi-touch gesture dictionary |
7844394, | Jul 18 2003 | LG Electronics, Inc. | Turn-by-turn navigation system and next direction guidance method using the same |
7848924, | Apr 17 2007 | WSOU Investments, LLC | Method, apparatus and computer program product for providing voice conversion using temporal dynamic features |
7848926, | Nov 22 2004 | National Institute of Advanced Industrial Science and Technology | System, method, and program for correcting misrecognized spoken words by selecting appropriate correction word from one or more competitive words |
7853444, | Oct 09 2005 | Kabushiki Kaisha Toshiba | Method and apparatus for training transliteration model and parsing statistic model, method and apparatus for transliteration |
7853445, | Dec 10 2004 | DECEPTION DISCOVERY TECHNOLOGIES, LLC | Method and system for the automatic recognition of deceptive language |
7853574, | Aug 26 2004 | International Business Machines Corporation | Method of generating a context-inferenced search query and of sorting a result of the query |
7853577, | Jun 09 2006 | PayPal, Inc | Shopping context engine |
7853664, | Jul 31 2000 | Apple Inc | Method and system for purchasing pre-recorded music |
7853900, | May 21 2007 | Amazon Technologies, Inc | Animations |
7865817, | Dec 29 2006 | Amazon Technologies, Inc | Invariant referencing in digital works |
7869998, | Apr 23 2002 | Microsoft Technology Licensing, LLC | Voice-enabled dialog system |
7869999, | Aug 11 2004 | Cerence Operating Company | Systems and methods for selecting from multiple phonectic transcriptions for text-to-speech synthesis |
7870118, | Jul 02 1999 | Telstra Corporation Limited | Search system |
7870133, | Jan 14 2008 | Infosys Limited | Method for semantic based storage and retrieval of information |
7873149, | Jun 01 2004 | RAKUTEN GROUP, INC | Systems and methods for gathering information |
7873519, | Nov 12 1999 | Nuance Communications, Inc | Natural language speech lattice containing semantic variants |
7873654, | Jan 24 2005 | PORTAL COMMUNICATIONS, LLC | Multimodal natural language query system for processing and analyzing voice and proximity-based queries |
7877705, | Jul 13 2001 | Universal Electronics Inc. | System and methods for interacting with a control environment |
7880730, | Feb 09 2004 | Nuance Communications, Inc | Keyboard system with automatic correction |
7881283, | Jul 13 2005 | Malikie Innovations Limited | Customizability of event notification on telephony-enabled devices |
7881936, | Dec 04 1998 | Cerence Operating Company | Multimodal disambiguation of speech recognition |
7885390, | Jul 01 2005 | Soleo Communications, Inc. | System and method for multi-modal personal communication services |
7885844, | Nov 16 2004 | Amazon Technologies, Inc. | Automatically generating task recommendations for human task performers |
7886233, | May 23 2005 | Nokia Corporation | Electronic text input involving word completion functionality for predicting word candidates for partial word inputs |
7889101, | Apr 14 2008 | Alpine Electronics, Inc | Method and apparatus for generating location based reminder message for navigation system |
7889184, | Jan 05 2007 | Apple Inc | Method, system and graphical user interface for displaying hyperlink information |
7889185, | Jan 05 2007 | Apple Inc | Method, system, and graphical user interface for activating hyperlinks |
7890330, | Dec 30 2005 | Alpine Electronics, Inc | Voice recording tool for creating database used in text to speech synthesis system |
7890652, | Apr 01 1996 | TVL LP | Information aggregation and synthesization system |
7895039, | Feb 04 2005 | VOCOLLECT, INC | Methods and systems for optimizing model adaptation for a speech recognition system |
7895531, | Aug 16 2004 | Microsoft Technology Licensing, LLC | Floating command object |
7899666, | May 04 2007 | EXPERT AI S P A | Method and system for automatically extracting relations between concepts included in text |
7904297, | May 31 2005 | Robert Bosch Corporation | Dialogue management using scripts and combined confidence scores |
7908287, | Dec 29 2005 | GOOGLE LLC | Dynamically autocompleting a data entry |
7912289, | May 01 2007 | Microsoft Technology Licensing, LLC | Image text replacement |
7912699, | Aug 23 2004 | Nuance Communications, Inc | System and method of lattice-based search for spoken utterance retrieval |
7912702, | Nov 12 1999 | Nuance Communications, Inc | Statistical language model trained with semantic variants |
7912720, | Jul 20 2005 | Nuance Communications, Inc | System and method for building emotional machines |
7912828, | Feb 23 2007 | Apple Inc. | Pattern searching methods and apparatuses |
7913185, | Oct 25 2001 | Adobe Inc | Graphical insertion of JavaScript pop-up menus |
7916979, | Jun 28 2002 | Microsoft Technology Licensing, LLC | Method and system for displaying and linking ink objects with recognized text and objects |
7917367, | Aug 05 2005 | DIALECT, LLC | Systems and methods for responding to natural language speech utterance |
7917497, | Sep 24 2001 | IAC Search & Media, Inc. | Natural language query processing |
7920678, | Mar 06 2000 | Avaya Inc. | Personal virtual assistant |
7920682, | Aug 21 2001 | Intellectual Ventures I LLC | Dynamic interactive voice interface |
7920857, | May 06 2002 | Malikie Innovations Limited | Event reminder method |
7925525, | Mar 25 2005 | Microsoft Technology Licensing, LLC | Smart reminders |
7925610, | Sep 22 1999 | GOOGLE LLC | Determining a meaning of a knowledge item using document-based information |
7929805, | Jan 31 2006 | The Penn State Research Foundation | Image-based CAPTCHA generation system |
7930168, | Oct 04 2005 | Robert Bosch GmbH | Natural language processing of disfluent sentences |
7930183, | Mar 29 2006 | ZHIGU HOLDINGS LIMITED | Automatic identification of dialog timing problems for an interactive speech dialog application using speech log data indicative of cases of barge-in and timing problems |
7930197, | Sep 28 2006 | Microsoft Technology Licensing, LLC | Personal data mining |
7936339, | Nov 01 2005 | LeapFrog Enterprises, Inc.; LEAPFROG ENTERPRISES, INC | Method and system for invoking computer functionality by interaction with dynamically generated interface regions of a writing surface |
7936861, | Jul 23 2004 | SBC KNOWLEDGE VENTURES L P | Announcement system and method of use |
7936863, | Sep 30 2004 | AVAYA Inc | Method and apparatus for providing communication tasks in a workflow |
7937075, | Oct 06 2006 | LYFT, INC | Mode changing of a mobile communications device and vehicle settings when the mobile communications device is in proximity to a vehicle |
7941009, | Apr 08 2003 | The Penn State Research Foundation | Real-time computerized annotation of pictures |
7945294, | Sep 24 2002 | Nuance Communications, Inc | Apparatus and method for providing hands-free operation of a device |
7945470, | Sep 29 2006 | Amazon Technologies, Inc. | Facilitating performance of submitted tasks by mobile task performers |
7949529, | Aug 29 2005 | DIALECT, LLC | Mobile systems and methods of supporting natural language human-machine interactions |
7949534, | Jul 03 2007 | ADVANCED VOICE RECOGNITION SYSTEMS, INC | Speech recognition and transcription among users having heterogeneous protocols |
7949752, | Oct 23 1998 | BYTEWEAVR, LLC | Network system extensible by users |
7953679, | Jul 22 2009 | MAJANDRO LLC | Scalable indexing for layout based document retrieval and ranking |
7957975, | Aug 09 2005 | Mobile Voice Control, LLC | Voice controlled wireless communication device system |
7958136, | Mar 18 2008 | GOOGLE LLC | Systems and methods for identifying similar documents |
7962179, | Sep 08 2006 | HTC Corporation | Handheld electronic device |
7974835, | Oct 01 2003 | Nuance Communications, Inc | Method, system, and apparatus for natural language mixed-initiative dialogue processing |
7974844, | Mar 24 2006 | Kabushiki Kaisha Toshiba; Toshiba Digital Solutions Corporation | Apparatus, method and computer program product for recognizing speech |
7974972, | Jun 07 2006 | TAMIRAS PER PTE LTD , LLC | Methods and apparatus for searching with awareness of geography and languages |
7975216, | Dec 07 1999 | Microsoft Technology Licensing, LLC | System and method for annotating an electronic document independently of its content |
7983478, | Aug 10 2007 | Microsoft Technology Licensing, LLC | Hidden markov model based handwriting/calligraphy generation |
7983915, | Apr 30 2007 | Sonic Foundry, Inc. | Audio content search engine |
7983917, | Aug 31 2005 | DIALECT, LLC | Dynamic speech sharpening |
7983919, | Aug 09 2007 | Nuance Communications, Inc | System and method for performing speech synthesis with a cache of phoneme sequences |
7983997, | Nov 02 2007 | FLORIDA INSTITUTE FOR HUMAN AND MACHINE COGNITION, INC | Interactive complex task teaching system that allows for natural language input, recognizes a user's intent, and automatically performs tasks in document object model (DOM) nodes |
7984062, | Apr 03 2002 | R2 SOLUTIONS LLC | Associating and linking compact disc metadata |
7986431, | Sep 30 2005 | Ricoh Company, Limited | Information processing apparatus, information processing method, and computer program product |
7987151, | Aug 10 2001 | GENERAL DYNAMICS MISSION SYSTEMS, INC | Apparatus and method for problem solving using intelligent agents |
7987244, | Dec 30 2004 | Nuance Communications, Inc | Network repository for voice fonts |
7991614, | Mar 20 2007 | Fujitsu Limited | Correction of matching results for speech recognition |
7992085, | Sep 26 2005 | Microsoft Technology Licensing, LLC | Lightweight reference user interface |
7996228, | Dec 22 2005 | Microsoft Technology Licensing, LLC | Voice initiated network operations |
7996589, | Apr 22 2005 | Microsoft Technology Licensing, LLC | Auto-suggest lists and handwritten input |
7996769, | Apr 05 2006 | Malikie Innovations Limited | Handheld electronic device and method for performing spell checking during text entry and for providing a spell-check learning feature |
7996792, | Sep 06 2006 | Apple Inc | Voicemail manager for portable multifunction device |
7999669, | Sep 28 2006 | Scenera Mobile Technologies, LLC | System and method for providing a task reminder based on historical travel information |
8000453, | Mar 06 2000 | AVAYA Inc | Personal virtual assistant |
8005664, | Apr 30 2007 | TACHYON TECHNOLOGIES PVT LTD | System, method to generate transliteration and method for generating decision tree to obtain transliteration |
8005679, | Oct 03 2001 | PROMPTU SYSTEMS CORPORATION | Global speech user interface |
8006180, | Jan 10 2006 | Microsoft Technology Licensing, LLC | Spell checking in network browser based applications |
8014308, | Sep 28 2006 | Microsoft Technology Licensing, LLC | Hardware architecture for cloud services |
8015006, | Jun 03 2002 | DIALECT, LLC | Systems and methods for processing natural language speech utterances with context-specific domain agents |
8015011, | Jan 30 2007 | Cerence Operating Company | Generating objectively evaluated sufficiently natural synthetic speech from text by using selective paraphrases |
8015144, | Feb 26 2008 | Microsoft Technology Licensing, LLC | Learning transportation modes from raw GPS data |
8018431, | Mar 29 2006 | Amazon Technologies, Inc | Page turner for handheld electronic book reader device |
8019271, | Dec 29 2006 | Nextel Communications, Inc. | Methods and systems for presenting information on mobile devices |
8019604, | Dec 21 2007 | Google Technology Holdings LLC | Method and apparatus for uniterm discovery and voice-to-voice search on mobile device |
8020104, | Apr 02 2001 | Microsoft Technology Licensing, LLC | Contextual responses based on automated learning techniques |
8024195, | Jun 27 2005 | Sensory, Inc. | Systems and methods of performing speech recognition using historical information |
8024415, | Mar 16 2000 | Microsoft Technology Licensing, LLC | Priorities generation and management |
8027836, | Nov 30 2006 | Cerence Operating Company | Phonetic decoding and concatentive speech synthesis |
8031943, | Jun 05 2003 | GOOGLE LLC | Automatic natural language translation of embedded text regions in images during information transfer |
8032383, | May 04 2007 | FONEWEB, INC | Speech controlled services and devices using internet |
8036901, | Oct 05 2007 | Sensory, Incorporated | Systems and methods of performing speech recognition using sensory inputs of human position |
8037034, | Mar 30 2006 | GLOBALFOUNDRIES Inc | Methods of creating a dictionary for data compression |
8041557, | Feb 24 2005 | FUJIFILM Business Innovation Corp | Word translation device, translation method, and computer readable medium |
8041570, | May 31 2005 | Robert Bosch Corporation | Dialogue management using scripts |
8041611, | Oct 30 2007 | TAMIRAS PER PTE LTD , LLC | Pricing and auctioning of bundled items among multiple sellers and buyers |
8042053, | Sep 24 2007 | Microsoft Technology Licensing, LLC | Method for making digital documents browseable |
8046363, | Apr 13 2006 | LG Electronics Inc. | System and method for clustering documents |
8046374, | May 06 2005 | CA, INC | Automatic training of a database intrusion detection system |
8050500, | Jul 06 2006 | Senapps, LLC | Recognition method and system |
8054180, | Dec 08 2008 | Amazon Technologies, Inc | Location aware reminders |
8055502, | Nov 28 2006 | General Motors LLC | Voice dialing using a rejection reference |
8055708, | Jun 01 2007 | Microsoft Technology Licensing, LLC | Multimedia spaces |
8056070, | Jan 10 2007 | GOOGLE LLC | System and method for modifying and updating a speech recognition program |
8060824, | Jan 05 2007 | Starz Entertainment, LLC | User interface for a multimedia service |
8064753, | Mar 05 2003 | HOT JAVA PRODUCTIONS, INC | Multi-feature media article and method for manufacture of same |
8065143, | Feb 22 2008 | Apple Inc. | Providing text input using speech data and non-speech data |
8065155, | Jun 10 1999 | WEST VIEW RESEARCH, LLC | Adaptive advertising apparatus and methods |
8065156, | Jun 10 1999 | WEST VIEW RESEARCH, LLC | Adaptive information presentation apparatus and methods |
8068604, | Dec 19 2008 | Computer Product Introductions Corporation | Method and system for event notifications |
8069046, | Aug 31 2005 | DIALECT, LLC | Dynamic speech sharpening |
8069422, | Jan 10 2005 | Samsung Electronics, Co., Ltd. | Contextual task recommendation system and method for determining user's context and suggesting tasks |
8073681, | Oct 16 2006 | Nuance Communications, Inc; VB Assets, LLC | System and method for a cooperative conversational voice user interface |
8073695, | Dec 09 1992 | Adrea, LLC | Electronic book with voice emulation features |
8077153, | Apr 19 2006 | Microsoft Technology Licensing, LLC | Precise selection techniques for multi-touch screens |
8078473, | Jun 10 1999 | WEST VIEW RESEARCH, LLC | Adaptive advertising apparatus and methods |
8082153, | Oct 02 1998 | Nuance Communications, Inc | Conversational computing via conversational virtual machine |
8082498, | May 27 2008 | AppFolio, Inc. | Systems and methods for automatic spell checking of dynamically generated web pages |
8090571, | Jun 29 2005 | International Business Machines Corporation | Method and system for building and contracting a linguistic dictionary |
8095364, | Jun 02 2004 | Cerence Operating Company | Multimodal disambiguation of speech recognition |
8099289, | Feb 13 2008 | Sensory, Inc | Voice interface and search for electronic devices including bluetooth headsets and remote systems |
8099395, | Jun 24 2004 | Oracle America, Inc | System level identity object |
8099418, | May 28 2007 | Panasonic Intellectual Property Corporation of America | Information search support method and information search support device |
8103510, | Dec 26 2003 | JVC Kenwood Corporation | Device control device, speech recognition device, agent device, on-vehicle device control device, navigation device, audio device, device control method, speech recognition method, agent processing method, on-vehicle device control method, navigation method, and audio device control method, and program |
8107401, | Sep 30 2004 | AVAYA Inc | Method and apparatus for providing a virtual assistant to a communication participant |
8112275, | Jun 03 2002 | DIALECT, LLC | System and method for user-specific speech recognition |
8112280, | Nov 19 2007 | Sensory, Inc. | Systems and methods of performing speech recognition with barge-in for use in a bluetooth system |
8117037, | Jun 10 1999 | WEST VIEW RESEARCH, LLC | Adaptive information presentation apparatus and methods |
8117542, | Aug 16 2004 | Microsoft Technology Licensing, LLC | User interface for displaying selectable software functionality controls that are contextually relevant to a selected object |
8121413, | Jun 29 2007 | NHN Corporation | Method and system for controlling browser by using image |
8121837, | Apr 24 2008 | Cerence Operating Company | Adjusting a speech engine for a mobile computing device based on background noise |
8122094, | Nov 05 2008 | RPX Corporation | Methods for performing an action relating to the scheduling of an event by performing one or more actions based on a response to a message |
8122353, | Nov 07 2008 | YAHOO ASSETS LLC | Composing a message in an online textbox using a non-latin script |
8130929, | May 25 2004 | GALILEO FINANCIAL TECHNOLOGIES, LLC | Methods for obtaining complex data in an interactive voice response system |
8131557, | Nov 27 2001 | Advanced Voice Recognition Systems, Inc, | Speech recognition and transcription among users having heterogeneous protocols |
8135115, | Nov 22 2006 | SECURUS TECHNOLOGIES, INC | System and method for multi-channel recording |
8138912, | Sep 28 2006 | Scenera Mobile Technologies, LLC | Apparatus and method for providing a task reminder based on travel history |
8140330, | Jun 13 2008 | Robert Bosch GmbH | System and method for detecting repeated patterns in dialog systems |
8140335, | Dec 11 2007 | VoiceBox Technologies Corporation | System and method for providing a natural language voice user interface in an integrated voice navigation services environment |
8140567, | Apr 13 2010 | Microsoft Technology Licensing, LLC | Measuring entity extraction complexity |
8145489, | Feb 06 2007 | Nuance Communications, Inc; VB Assets, LLC | System and method for selecting and presenting advertisements based on natural language processing of voice-based input |
8150694, | Aug 31 2005 | DIALECT, LLC | System and method for providing an acoustic grammar to dynamically sharpen speech interpretation |
8150700, | Apr 08 2008 | LG Electronics Inc. | Mobile terminal and menu control method thereof |
8155956, | Dec 18 2007 | Samsung Electronics Co., Ltd. | Voice query extension method and system |
8156005, | Sep 09 2008 | VIERI, RICCARDO, MR | Systems and methods of contextual advertising |
8160877, | Aug 06 2009 | The Boeing Company | Hierarchical real-time speaker recognition for biometric VoIP verification and targeting |
8160883, | Jan 10 2004 | Microsoft Technology Licensing, LLC | Focus tracking in dialogs |
8165321, | Mar 10 2009 | Apple Inc.; Apple Inc | Intelligent clip mixing |
8165886, | Oct 04 2007 | SAMSUNG ELECTRONICS CO , LTD | Speech interface system and method for control and interaction with applications on a computing system |
8166019, | Jul 21 2008 | T-MOBILE INNOVATIONS LLC | Providing suggested actions in response to textual communications |
8166032, | Apr 09 2009 | Marketchorus, Inc. | System and method for sentiment-based text classification and relevancy ranking |
8170790, | Sep 05 2006 | Garmin Switzerland GmbH | Apparatus for switching navigation device mode |
8175872, | Apr 14 2010 | GOOGLE LLC | Geotagged and weighted environmental audio for enhanced speech recognition accuracy |
8175876, | Mar 02 2001 | WIAV Solutions LLC | System and method for an endpoint detection of speech for improved speech recognition in noisy environments |
8179370, | Feb 09 2010 | GOOGLE LLC | Proximity based keystroke resolution |
8188856, | Sep 28 2006 | Scenera Mobile Technologies, LLC | System and method for providing a task reminder |
8190359, | Aug 31 2007 | PROXPRO, INC | Situation-aware personal information management for a mobile device |
8190596, | Nov 28 2007 | International Business Machines Corporation | Method for assembly of personalized enterprise information integrators over conjunctive queries |
8195467, | Feb 13 2008 | Sensory, Incorporated | Voice interface and search for electronic devices including bluetooth headsets and remote systems |
8195468, | Aug 29 2005 | DIALECT, LLC | Mobile systems and methods of supporting natural language human-machine interactions |
8200489, | Jan 29 2009 | The United States of America as represented by the Secretary of the Navy | Multi-resolution hidden markov model using class specific features |
8200495, | Feb 04 2005 | VOCOLLECT, INC | Methods and systems for considering information about an expected response when performing speech recognition |
8201109, | Mar 04 2008 | Apple Inc.; Apple Inc | Methods and graphical user interfaces for editing on a portable multifunction device |
8204238, | Jun 08 2007 | Sensory, Inc | Systems and methods of sonic communication |
8205788, | Dec 17 2003 | WEST VIEW RESEARCH, LLC | Chattel management apparatus and method |
8209183, | Jul 07 2011 | GOOGLE LLC | Systems and methods for correction of text from different input types, sources, and contexts |
8213911, | Jan 28 2009 | Virtual Hold Technology Solutions, LLC | Mobile communication device for establishing automated call back |
8219115, | May 12 2008 | GOOGLE LLC | Location based reminders |
8219406, | Mar 15 2007 | Microsoft Technology Licensing, LLC | Speech-centric multimodal user interface design in mobile technology |
8219407, | Dec 27 2007 | Apple Inc | Method for processing the output of a speech recognizer |
8219608, | Jun 20 2002 | Koninklijke Philips Electronics N.V. | Scalable architecture for web services |
8224649, | Jun 02 2004 | GENERAC HOLDINGS INC ; GENERAC POWER SYSTEMS, INC | Method and apparatus for remote command, control and diagnostics of systems using conversational or audio interface |
8224757, | Apr 15 2003 | SAP SE | Curriculum management system |
8228299, | Jan 27 2005 | The Reynolds and Reynolds Company | Transaction automation and archival system using electronic contract and disclosure units |
8233919, | Aug 09 2009 | III Holdings 1, LLC | Intelligently providing user-specific transportation-related information |
8234111, | Jun 14 2010 | GOOGLE LLC | Speech and noise models for speech recognition |
8239206, | Aug 06 2010 | GOOGLE LLC | Routing queries based on carrier phrase registration |
8239207, | Sep 05 2003 | ZAMA INNOVATIONS LLC | Speech-enabled language translation system and method enabling interactive user supervision of translation and speech recognition accuracy |
8244712, | Mar 18 2003 | Apple Inc | Localized viewing of file system names |
8250071, | Jun 30 2010 | Amazon Technologies, Inc. | Disambiguation of term meaning |
8254829, | May 09 2008 | Sprint Communications Company L.P. | Network media service with track delivery adapted to a user cadence |
8255216, | Oct 30 2006 | Microsoft Technology Licensing, LLC | Speech recognition of character sequences |
8255217, | Oct 16 2009 | Microsoft Technology Licensing, LLC | Systems and methods for creating and using geo-centric language models |
8260247, | Jul 21 2010 | BlackBerry Limited | Portable electronic device and method of operation |
8260617, | Apr 18 2005 | Microsoft Technology Licensing, LLC | Automating input when testing voice-enabled applications |
8270933, | Sep 26 2005 | LONGHORN AUTOMOTIVE GROUP LLC | Safety features for portable electronic device |
8271287, | Jan 14 2000 | QUARTERHILL INC ; WI-LAN INC | Voice command remote control system |
8275621, | Mar 31 2008 | Microsoft Technology Licensing, LLC | Determining text to speech pronunciation based on an utterance from a user |
8279171, | Jul 06 2006 | PANASONIC AUTOMOTIVE SYSTEMS CO , LTD | Voice input device |
8280438, | Sep 26 2005 | LONGHORN AUTOMOTIVE GROUP LLC | Safety features for portable electronic device |
8285546, | Jul 22 2004 | Nuance Communications, Inc. | Method and system for identifying and correcting accent-induced speech recognition difficulties |
8285551, | Jun 10 1999 | WEST VIEW RESEARCH, LLC | Network apparatus and methods for user information delivery |
8285553, | Jun 10 1999 | WEST VIEW RESEARCH, LLC | Computerized information presentation apparatus |
8290777, | Jun 12 2009 | Amazon Technologies, Inc. | Synchronizing the playing and displaying of digital content |
8290778, | Jun 10 1999 | WEST VIEW RESEARCH, LLC | Computerized information presentation apparatus |
8290781, | Jun 10 1999 | WEST VIEW RESEARCH, LLC | Computerized information presentation apparatus |
8296124, | Nov 21 2008 | GOOGLE LLC | Method and apparatus for detecting incorrectly translated text in a document |
8296145, | Nov 28 2006 | General Motors LLC | Voice dialing using a rejection reference |
8296146, | Jun 10 1999 | WEST VIEW RESEARCH, LLC | Computerized information presentation apparatus |
8296153, | Jun 10 1999 | WEST VIEW RESEARCH, LLC | Computerized information presentation methods |
8296380, | Apr 01 2010 | Kel & Partners LLC | Social media based messaging systems and methods |
8296383, | Oct 02 2008 | Apple Inc. | Electronic devices with voice command and contextual data processing capabilities |
8300776, | Jul 15 2009 | GOOGLE LLC | Highlighting of voice message transcripts |
8300801, | Jun 26 2008 | CenturyLink Intellectual Property LLC | System and method for telephone based noise cancellation |
8301456, | Jun 10 1999 | WEST VIEW RESEARCH, LLC | Electronic information access system and methods |
8311189, | Aug 08 2005 | Qualcomm Incorporated | Method and device for enabling message responses to incoming phone calls |
8311834, | Jun 10 1999 | WEST VIEW RESEARCH, LLC | Computerized information selection and download apparatus and methods |
8311835, | Aug 29 2003 | Microsoft Technology Licensing, LLC | Assisted multi-modal dialogue |
8311838, | Jan 13 2010 | Apple Inc. | Devices and methods for identifying a prompt corresponding to a voice input in a sequence of prompts |
8312017, | Feb 03 2005 | Apple Inc | Recommender system for identifying a new set of media items responsive to an input set of media items and knowledge base metrics |
8321786, | Jun 17 2004 | Apple Inc | Routine and interface for correcting electronic text |
8326627, | Dec 11 2007 | VoiceBox Technologies, Inc. | System and method for dynamically generating a recognition grammar in an integrated voice navigation services environment |
8332205, | Jan 09 2009 | Microsoft Technology Licensing, LLC | Mining transliterations for out-of-vocabulary query terms |
8332218, | Jun 13 2006 | Microsoft Technology Licensing, LLC | Context-based grammars for automated speech recognition |
8332224, | Aug 10 2005 | DIALECT, LLC | System and method of supporting adaptive misrecognition conversational speech |
8332748, | Oct 22 2009 | GOOGLE LLC | Multi-directional auto-complete menu |
8335689, | Oct 14 2009 | COGI INC | Method and system for efficient management of speech transcribers |
8340975, | Oct 04 2011 | Interactive speech recognition device and system for hands-free building control | |
8345665, | Oct 22 2001 | Apple Inc | Text to speech conversion of text messages from mobile communication devices |
8346563, | Apr 10 2012 | ARTIFICIAL SOLUTIONS IBERIA S L | System and methods for delivering advanced natural language interaction applications |
8352183, | Feb 04 2006 | Microsoft Technology Licensing, LLC | Maps for social networking and geo blogs |
8352268, | Sep 29 2008 | Apple Inc | Systems and methods for selective rate of speech and speech preferences for text to speech synthesis |
8352272, | Sep 29 2008 | Apple Inc | Systems and methods for text to speech synthesis |
8355919, | Sep 29 2008 | Apple Inc | Systems and methods for text normalization for text to speech synthesis |
8359234, | Jul 26 2007 | Braintexter, Inc | System to generate and set up an advertising campaign based on the insertion of advertising messages within an exchange of messages, and method to operate said system |
8370145, | Mar 29 2007 | Sovereign Peak Ventures, LLC | Device for extracting keywords in a conversation |
8370158, | Jun 10 1999 | WEST VIEW RESEARCH, LLC | Adaptive information presentation apparatus |
8371503, | Dec 17 2003 | WEST VIEW RESEARCH, LLC | Portable computerized wireless payment apparatus and methods |
8374871, | May 28 1999 | NANT HOLDINGS IP, LLC | Methods for creating a phrase thesaurus |
8375320, | Jun 22 2010 | Microsoft Technology Licensing, LLC | Context-based task generation |
8380504, | May 06 2010 | T-MOBILE INNOVATIONS LLC | Generation of voice profiles |
8380507, | Mar 09 2009 | Apple Inc | Systems and methods for determining the language to use for speech generated by a text to speech engine |
8381107, | Jan 13 2010 | Apple Inc. | Adaptive audio feedback system and method |
8381135, | Jul 30 2004 | Apple Inc | Proximity detector in handheld device |
8386485, | Jul 31 2008 | GEORGE MASON INTELLECTUAL PROPERTIES, INC | Case-based framework for collaborative semantic search |
8386926, | Oct 06 2011 | GOOGLE LLC | Network-based custom dictionary, auto-correction and text entry preferences |
8391844, | Jan 07 2007 | Apple Inc | Voicemail systems and methods |
8396714, | Sep 29 2008 | Apple Inc | Systems and methods for concatenation of words in text to speech synthesis |
8401163, | Oct 18 2005 | Callwave Communications, LLC | Methods and systems for call processing and for providing call progress status over a network |
8406745, | Jan 28 2010 | T-MOBILE INNOVATIONS LLC | Synchronization of voice mail greeting and email auto-reply by a wireless communication device |
8423288, | Nov 30 2009 | Apple Inc.; Apple Inc | Dynamic alerts for calendar events |
8428758, | Feb 16 2009 | Apple Inc. | Dynamic audio ducking |
8433572, | Apr 29 2005 | Microsoft Technology Licensing, LLC | Method and apparatus for multiple value confirmation and correction in spoken dialog system |
8433778, | Apr 22 2008 | CAVIUM INTERNATIONAL; MARVELL ASIA PTE, LTD | Device configuration |
8442821, | Jul 27 2012 | GOOGLE LLC | Multi-frame prediction for hybrid neural network/hidden Markov models |
8447612, | Jun 10 1999 | WEST VIEW RESEARCH, LLC | Computerized information presentation apparatus |
8452597, | Sep 30 2011 | GOOGLE LLC | Systems and methods for continual speech recognition and detection in mobile computing devices |
8457959, | Mar 01 2007 | ADAPX, INC | Systems and methods for implicitly interpreting semantically redundant communication modes |
8458115, | Jun 08 2010 | Microsoft Technology Licensing, LLC | Mining topic-related aspects from user generated content |
8458278, | May 02 2003 | Apple Inc. | Method and apparatus for displaying information during an instant messaging session |
8464150, | Jun 07 2008 | Apple Inc. | Automatic language identification for dynamic text processing |
8473289, | Aug 06 2010 | GOOGLE LLC | Disambiguating input based on context |
8479122, | Jul 30 2004 | Apple Inc | Gestures for touch sensitive input devices |
8484027, | Jun 12 2009 | 1082824 ONTARIO INC | Method for live remote narration of a digital book |
8489599, | Dec 02 2008 | Xerox Corporation | Context and activity-driven content delivery and interaction |
8498857, | May 19 2009 | General Atomics | System and method for rapid prototyping of existing speech recognition solutions in different languages |
8514197, | Sep 12 2008 | AT&T Intellectual Property I, L.P. | Multimodal portable communication interface for accessing video content |
8515750, | Jun 05 2012 | GOOGLE LLC | Realtime acoustic adaptation using stability measures |
8521513, | Mar 12 2010 | Microsoft Technology Licensing, LLC | Localization for interactive voice response systems |
8521531, | Aug 29 2012 | LG Electronics Inc. | Displaying additional data about outputted media data by a display device for a speech search command |
8527276, | Oct 25 2012 | GOOGLE LLC | Speech synthesis using deep neural networks |
8537033, | Mar 05 2002 | CORTLAND CAPITAL MARKET SERVICES LLC, AS ADMINISTRATIVE AGENT | Method for predicting a travel time for a traffic route |
8539342, | Oct 16 2008 | Adobe Inc | Read-order inference via content sorting |
8543375, | Apr 10 2007 | GOOGLE LLC | Multi-mode input method editor |
8543397, | Oct 11 2012 | GOOGLE LLC | Mobile device voice activation |
8543398, | Feb 29 2012 | GOOGLE LLC | Training an automatic speech recognition system using compressed word frequencies |
8560229, | Sep 15 2010 | GOOGLE LLC | Sensor based activity detection |
8571851, | Dec 31 2012 | GOOGLE LLC | Semantic interpretation using user gaze order |
8583416, | Dec 27 2007 | NANT HOLDINGS IP, LLC | Robust information extraction from utterances |
8583511, | May 19 2009 | Bradley Marshall, Hendrickson | Systems and methods for storing customer purchasing and preference data and enabling a customer to pre-register orders and events |
8589869, | Sep 07 2006 | Wolfram Alpha LLC | Methods and systems for determining a formula |
8589911, | Jul 26 2012 | GOOGLE LLC | Intent fulfillment |
8595004, | Dec 18 2007 | NEC Corporation | Pronunciation variation rule extraction apparatus, pronunciation variation rule extraction method, and pronunciation variation rule extraction program |
8600743, | Jan 06 2010 | Apple Inc. | Noise profile determination for voice-related feature |
8600746, | Sep 10 2012 | GOOGLE LLC | Speech recognition parameter adjustment |
8600930, | Jul 25 2008 | Sharp Kabushiki Kaisha | Information processing device and information processing method |
8606090, | Mar 17 2011 | Saturn Licensing LLC | Sport program chaptering |
8606568, | Oct 10 2012 | GOOGLE LLC | Evaluating pronouns in context |
8606576, | Nov 02 2012 | GOOGLE LLC | Communication log with extracted keywords from speech-to-text processing |
8620659, | Aug 10 2005 | DIALECT, LLC | System and method of supporting adaptive misrecognition in conversational speech |
8620662, | Nov 20 2007 | Apple Inc.; Apple Inc | Context-aware unit selection |
8626681, | Jan 04 2011 | GOOGLE LLC | Training a probabilistic spelling checker from structured data |
8638363, | Feb 18 2009 | Kyocera Corporation | Automatically capturing information, such as capturing information using a document-aware device |
8639516, | Jun 04 2010 | Apple Inc. | User-specific noise suppression for voice quality improvements |
8645137, | Mar 16 2000 | Apple Inc. | Fast, language-independent method for user authentication by voice |
8645138, | Dec 20 2012 | GOOGLE LLC | Two-pass decoding for speech recognition of search and action requests |
8654936, | Feb 24 2004 | AT&T Corp | Home control, monitoring and communication system using remote voice commands |
8655646, | Oct 28 2005 | Samsung Electronics Co., Ltd.; SAMSUNG ELECTRONICS CO , LTD | Apparatus and method for detecting named entity |
8655901, | Jun 23 2010 | GOOGLE LLC | Translation-based query pattern mining |
8660843, | Feb 04 2002 | Microsoft Technology Licensing, LLC | Management and prioritization of processing multiple requests |
8660849, | Jan 18 2010 | Apple Inc. | Prioritizing selection criteria by automated assistant |
8660970, | Apr 23 2009 | The Boeing Company | Passive learning and autonomously interactive system for leveraging user knowledge in networked environments |
8661112, | Dec 20 2002 | Nuance Communications, Inc | Customized interactive voice response menus |
8661340, | Sep 13 2007 | Apple Inc. | Input methods for device having multi-language environment |
8670979, | Jan 18 2010 | Apple Inc. | Active input elicitation by intelligent automated assistant |
8675084, | Sep 04 2009 | Apple Inc | Systems and methods for remote camera control |
8676904, | Oct 02 2008 | Apple Inc.; Apple Inc | Electronic devices with voice command and contextual data processing capabilities |
8677377, | Sep 08 2005 | Apple Inc | Method and apparatus for building an intelligent automated assistant |
8681950, | Mar 28 2012 | GENESYS CLOUD SERVICES, INC | System and method for fingerprinting datasets |
8682667, | Feb 25 2010 | Apple Inc. | User profiling for selecting user specific voice input processing information |
8687777, | Feb 03 2010 | Systems and methods for visual presentation and selection of IVR menu | |
8688446, | Feb 22 2008 | Apple Inc. | Providing text input using speech data and non-speech data |
8688453, | Feb 28 2011 | Microsoft Technology Licensing, LLC | Intent mining via analysis of utterances |
8695074, | Apr 26 2007 | Microsoft Technology Licensing, LLC | Pre-authenticated calling for voice applications |
8696364, | Mar 28 2007 | Breakthrough PerformanceTech, LLC | Systems and methods for computerized interactive training |
8706472, | Aug 11 2011 | Apple Inc.; Apple Inc | Method for disambiguating multiple readings in language conversion |
8706474, | Feb 23 2008 | Fair Isaac Corporation | Translation of entity names based on source document publication date, and frequency and co-occurrence of the entity names |
8706503, | Jan 18 2010 | Apple Inc. | Intent deduction based on previous user interactions with voice assistant |
8713119, | Oct 02 2008 | Apple Inc. | Electronic devices with voice command and contextual data processing capabilities |
8713418, | Apr 12 2004 | Kyocera Corporation | Adding value to a rendered document |
8719006, | Aug 27 2010 | Apple Inc. | Combined statistical and rule-based part-of-speech tagging for text-to-speech synthesis |
8719014, | Sep 27 2010 | Apple Inc.; Apple Inc | Electronic device with text error correction based on voice recognition data |
8719039, | Dec 05 2013 | GOOGLE LLC | Promoting voice actions to hotwords |
8731610, | Dec 13 2006 | Samsung Electronics Co., Ltd. | Method for adaptive user interface in mobile devices |
8731912, | Jan 16 2013 | GOOGLE LLC | Delaying audio notifications |
8731942, | Jan 18 2010 | Apple Inc | Maintaining context information between user interactions with a voice assistant |
8739208, | Feb 12 2009 | Digimarc Corporation | Media processing methods and arrangements |
8744852, | Oct 01 2004 | Apple Inc. | Spoken interfaces |
8760537, | Jul 05 2010 | Apple Inc.; Apple Inc | Capturing and rendering high dynamic range images |
8762145, | Nov 06 2009 | Kabushiki Kaisha Toshiba | Voice recognition apparatus |
8762156, | Sep 28 2011 | Apple Inc.; Apple Inc | Speech recognition repair using contextual information |
8762469, | Oct 02 2008 | Apple Inc. | Electronic devices with voice command and contextual data processing capabilities |
8768693, | May 31 2012 | R2 SOLUTIONS LLC | Automatic tag extraction from audio annotated photos |
8768702, | Sep 05 2008 | Apple Inc.; Apple Inc | Multi-tiered voice feedback in an electronic device |
8775154, | Sep 18 2008 | Conduent Business Services, LLC | Query translation through dictionary adaptation |
8775931, | Mar 30 2007 | BlackBerry Limited | Spell check function that applies a preference to a spell check algorithm based upon extensive user selection of spell check results generated by the algorithm, and associated handheld electronic device |
8781456, | Jan 07 2011 | Malikie Innovations Limited | System and method for controlling mobile communication devices |
8781841, | Jun 07 2011 | Cisco Technology, Inc. | Name recognition of virtual meeting participants |
8798255, | Mar 31 2009 | NICE LTD | Methods and apparatus for deep interaction analysis |
8798995, | Sep 23 2011 | Amazon Technologies, Inc.; Amazon Technologies, Inc | Key word determinations from voice data |
8799000, | Jan 18 2010 | Apple Inc. | Disambiguation based on active input elicitation by intelligent automated assistant |
8805690, | Aug 05 2010 | GOOGLE LLC | Audio notifications |
8812302, | Aug 02 2012 | GOOGLE LLC | Techniques for inserting diacritical marks to text input via a user device |
8838457, | Mar 07 2007 | Nuance Communications, Inc | Using results of unstructured language model based speech recognition to control a system-level function of a mobile communications facility |
8855915, | Nov 09 2011 | Hitachi, LTD | Navigation system, navigation apparatus, method and server |
8861925, | Jul 28 2010 | INTUIT INC.; INTUIT INC | Methods and systems for audio-visual synchronization |
8862252, | Jan 30 2009 | Apple Inc | Audio user interface for displayless electronic device |
8868409, | Jan 16 2014 | GOOGLE LLC | Evaluating transcriptions with a semantic parser |
8880405, | Mar 07 2007 | Nuance Communications, Inc | Application text entry in a mobile environment using a speech processing facility |
8886534, | Jan 28 2010 | HONDA MOTOR CO , LTD | Speech recognition apparatus, speech recognition method, and speech recognition robot |
8886540, | Mar 07 2007 | Nuance Communications, Inc | Using speech recognition results based on an unstructured language model in a mobile communication facility application |
8886541, | Feb 04 2010 | Saturn Licensing LLC | Remote controller with position actuatated voice transmission |
8892446, | Jan 18 2010 | Apple Inc. | Service orchestration for intelligent automated assistant |
8893023, | Oct 19 2007 | GOOGLE LLC | Method and system for predicting text |
8898568, | Sep 09 2008 | Apple Inc | Audio user interface |
8903716, | Jan 18 2010 | Apple Inc. | Personalized vocabulary for digital assistant |
8909693, | Aug 21 2009 | Avaya Inc. | Telephony discovery mashup and presence |
8930176, | Apr 01 2010 | Microsoft Technology Licensing, LLC | Interactive multilingual word-alignment techniques |
8930191, | Jan 18 2010 | Apple Inc | Paraphrasing of user requests and results by automated digital assistant |
8938394, | Jan 09 2014 | GOOGLE LLC | Audio triggers based on context |
8938450, | Feb 17 2012 | Bottlenose, Inc. | Natural language processing optimized for micro content |
8938688, | Dec 04 1998 | Cerence Operating Company | Contextual prediction of user words and user actions |
8942986, | Jan 18 2010 | Apple Inc. | Determining user intent based on ontologies of domains |
8943423, | Jul 07 2009 | International Business Machines Corporation | User interface indicators for changed user interface elements |
8972240, | May 19 2011 | Microsoft Technology Licensing, LLC | User-modifiable word lattice display for editing documents and search queries |
8972432, | Apr 23 2008 | GOOGLE LLC | Machine translation using information retrieval |
8972878, | Sep 21 2009 | Avaya Inc.; AVAYA Inc | Screen icon manipulation by context and frequency of Use |
8983383, | Sep 25 2012 | Amazon Technologies, Inc | Providing hands-free service to multiple devices |
8989713, | Jan 09 2008 | Microsoft Technology Licensing, LLC | Selection of a link in a received message for speaking reply, which is converted into text form for delivery |
8990235, | Mar 12 2009 | Kyocera Corporation | Automatically providing content associated with captured information, such as information captured in real-time |
8994660, | Aug 29 2011 | Apple Inc. | Text correction processing |
8996350, | Nov 02 2011 | AUTOFILE INC | System and method for automatic document management |
8996376, | Apr 05 2008 | Apple Inc. | Intelligent text-to-speech conversion |
8996381, | Sep 27 2011 | Sensory, Incorporated | Background speech recognition assistant |
8996639, | Oct 15 2013 | GOOGLE LLC | Predictive responses to incoming communications |
9009046, | Sep 27 2005 | Microsoft Technology Licensing, LLC | System and method for disambiguating multiple intents in a natural language dialog system |
9020804, | Jun 01 2007 | Xerox Corporation | Method for aligning sentences at the word level enforcing selective contiguity constraints |
9026425, | Aug 28 2012 | Xerox Corporation | Lexical and phrasal feature domain adaptation in statistical machine translation |
9031834, | Sep 04 2009 | Cerence Operating Company | Speech enhancement techniques on the power spectrum |
9037967, | Feb 18 2014 | KING FAHD UNIVERSITY OF PETROLEUM AND MINERALS | Arabic spell checking technique |
9043208, | Jul 18 2012 | International Business Machines Corporation | System, method and program product for providing automatic speech recognition (ASR) in a shared resource environment |
9043211, | May 09 2013 | DSP Group Ltd | Low power activation of a voice activated device |
9049255, | Feb 29 2008 | HUAWEI TECHNOLOGIES CO , LTD | Visual event notification on a handheld communications device |
9049295, | Aug 28 2012 | WEST TECHNOLOGY GROUP, LLC | Intelligent interactive voice response system for processing customer communications |
9053706, | Aug 06 2010 | GOOGLE LLC | Disambiguating input based on context |
9058811, | Feb 25 2011 | Kabushiki Kaisha Toshiba | Speech synthesis with fuzzy heteronym prediction using decision trees |
9063979, | Nov 01 2007 | PayPal, Inc | Analyzing event streams of user sessions |
9070366, | Dec 19 2012 | Amazon Technologies, Inc | Architecture for multi-domain utterance processing |
9071701, | Mar 29 2007 | Qualcomm Incorporated | Using wireless characteristic to trigger generation of position fix |
9076448, | Nov 12 1999 | Nuance Communications, Inc | Distributed real time speech recognition system |
9076450, | Sep 21 2012 | Amazon Technologies, Inc | Directed audio for speech recognition |
9081411, | May 10 2013 | SRI International | Rapid development of virtual personal assistant applications |
9081482, | Sep 18 2012 | GOOGLE LLC | Text input suggestion ranking |
9082402, | Dec 08 2011 | SRI International | Generic virtual personal assistant platform |
9098467, | Dec 19 2012 | Amazon Technologies, Inc | Accepting voice commands based on user identity |
9101279, | Feb 15 2006 | VIRTUAL VIDEO REALITY BY RITCHEY, LIMITED LIABILITY CORPORATION | Mobile user borne brain activity data and surrounding environment data correlation system |
9112984, | Mar 12 2013 | Cerence Operating Company | Methods and apparatus for detecting a voice command |
9117447, | Jan 18 2010 | Apple Inc. | Using event alert text as input to an automated assistant |
9123338, | Jun 01 2012 | GOOGLE LLC | Background audio identification for speech disambiguation |
9164983, | May 27 2011 | Robert Bosch GmbH | Broad-coverage normalization system for social media language |
9171541, | Nov 10 2009 | VOICEBOX TECHNOLOGIES, INC | System and method for hybrid processing in a natural language voice services environment |
9171546, | Mar 29 2011 | GOOGLE LLC | Performing functions based on commands in context of telephonic communication |
9190062, | Feb 25 2010 | Apple Inc. | User profiling for voice input processing |
9208153, | Dec 13 2013 | NORTONLIFELOCK INC | Filtering relevant event notifications in a file sharing and collaboration environment |
9218809, | Mar 16 2000 | Apple Inc. | Fast, language-independent method for user authentication by voice |
9218819, | Mar 01 2013 | GOOGLE LLC | Customizing actions based on contextual data and voice-based inputs |
9223537, | Apr 18 2012 | VERINT AMERICAS INC | Conversation user interface |
9255812, | Nov 07 2012 | Hitachi, Ltd. | Navigation system and navigation method |
9258604, | Nov 24 2014 | Meta Platforms, Inc | Commercial detection based on audio fingerprinting |
9262612, | Mar 21 2011 | Apple Inc.; Apple Inc | Device access using voice authentication |
9280535, | Mar 31 2011 | Infosys Limited | Natural language querying with cascaded conditional random fields |
9286910, | Mar 13 2014 | Amazon Technologies, Inc | System for resolving ambiguous queries based on user context |
9292487, | Aug 16 2012 | Amazon Technologies, Inc | Discriminative language model pruning |
9292489, | Jan 16 2013 | GOOGLE LLC | Sub-lexical language models with word level pronunciation lexicons |
9299344, | Mar 12 2013 | Intermec IP Corp. | Apparatus and method to classify sound to detect speech |
9300718, | Apr 09 2013 | Avaya Inc. | System and method for keyword-based notification and delivery of content |
9305543, | Apr 05 2008 | Apple Inc. | Intelligent text-to-speech conversion |
9305548, | May 27 2008 | Oracle International Corporation | System and method for an integrated, multi-modal, multi-device natural language voice services environment |
9311912, | Jul 22 2013 | Amazon Technologies, Inc | Cost efficient distributed text-to-speech processing |
9313317, | Aug 05 2010 | GOOGLE LLC | Audio notifications |
9318108, | Jan 18 2010 | Apple Inc.; Apple Inc | Intelligent automated assistant |
9325809, | Sep 07 2012 | Cisco Technology, Inc | Audio recall during voice conversations |
9330659, | Feb 25 2013 | Microsoft Technology Licensing, LLC | Facilitating development of a spoken natural language interface |
9330720, | Jan 03 2008 | Apple Inc. | Methods and apparatus for altering audio output signals |
9338493, | Jun 30 2014 | Apple Inc | Intelligent automated assistant for TV user interactions |
9349368, | Aug 05 2010 | GOOGLE LLC | Generating an audio notification based on detection of a triggering event |
9361084, | Nov 14 2013 | GOOGLE LLC | Methods and systems for installing and executing applications |
9367541, | Jan 20 2015 | Xerox Corporation | Terminological adaptation of statistical machine translation system through automatic generation of phrasal contexts for bilingual terms |
9377871, | Aug 01 2014 | Cerence Operating Company | System and methods for determining keyboard input in the presence of multiple contact points |
9378740, | Sep 30 2014 | Amazon Technologies, Inc | Command suggestions during automatic speech recognition |
9380155, | Nov 30 2000 | GOOGLE LLC | Forming speech recognition over a network and using speech recognition results based on determining that a network connection exists |
9383827, | Apr 07 2014 | GOOGLE LLC | Multi-modal command display |
9390726, | Dec 30 2013 | GOOGLE LLC | Supplementing speech commands with gestures |
9396722, | Jun 20 2013 | Electronics and Telecommunications Research Institute | Method and apparatus for detecting speech endpoint using weighted finite state transducer |
9401147, | Aug 06 2010 | GOOGLE LLC | Disambiguating input based on context |
9406224, | Jul 14 2005 | Zaxcom, Inc. | Systems and methods for remotely controlling local audio devices in a virtual wireless multitrack recording system |
9412392, | Oct 02 2008 | Apple Inc. | Electronic devices with voice command and contextual data processing capabilities |
9423266, | Jan 26 2012 | ARTAX, LLC | Navigational lane guidance |
9424840, | Aug 31 2012 | Amazon Technologies, Inc | Speech recognition platforms |
9436918, | Oct 07 2013 | Microsoft Technology Licensing, LLC | Smart selection of text spans |
9437186, | Jun 19 2013 | Amazon Technologies, Inc | Enhanced endpoint detection for speech recognition |
9437189, | May 29 2014 | GOOGLE LLC | Generating language models |
9454957, | Mar 05 2013 | Amazon Technologies, Inc | Named entity resolution in spoken language processing |
9465833, | Jul 31 2012 | Veveo, Inc | Disambiguating user intent in conversational interaction system for large corpus information retrieval |
9471566, | Apr 14 2005 | Oracle America, Inc.; Sun Microsystems, Inc | Method and apparatus for converting phonetic language input to written language output |
9484021, | Mar 30 2015 | Amazon Technologies, Inc | Disambiguation in speech recognition |
9495129, | Jun 29 2012 | Apple Inc. | Device, method, and user interface for voice-activated navigation and browsing of a document |
9501741, | Sep 08 2005 | Apple Inc. | Method and apparatus for building an intelligent automated assistant |
9502025, | Nov 10 2009 | VB Assets, LLC | System and method for providing a natural language content dedication service |
9508028, | Sep 24 2014 | Microsoft Technology Licensing, LLC | Converting text strings into number strings, such as via a touchscreen input |
9510044, | Dec 15 2010 | ROKU, INC | TV content segmentation, categorization and identification and time-aligned applications |
9524355, | Jan 22 2007 | EMC IP HOLDING COMPANY LLC | Methods for delivering task-related digital content based on task-oriented user activity |
9535906, | Jul 31 2008 | Apple Inc. | Mobile device having human language translation capability with positional feedback |
9536527, | Jun 30 2015 | Amazon Technologies, Inc | Reporting operational metrics in speech-based systems |
9547647, | Sep 19 2012 | Apple Inc. | Voice-based media searching |
9548050, | Jan 18 2010 | Apple Inc. | Intelligent automated assistant |
9569549, | May 25 2010 | Amazon Technologies, Inc. | Location based recommendation and tagging of media content items |
9575964, | Dec 08 2011 | SRI International | Generic virtual personal assistant platform |
9578173, | Jun 05 2015 | Apple Inc | Virtual assistant aided communication with 3rd party service in a communication session |
9607612, | May 20 2013 | Intel Corporation | Natural human-computer interaction for virtual personal assistant systems |
9620113, | Dec 11 2007 | VoiceBox Technologies Corporation | System and method for providing a natural language voice user interface |
9620126, | Aug 28 2012 | Kyocera Corporation | Electronic device, control method, and control program |
9626955, | Apr 05 2008 | Apple Inc. | Intelligent text-to-speech conversion |
9633004, | May 30 2014 | Apple Inc.; Apple Inc | Better resolution when referencing to concepts |
9633660, | Feb 25 2010 | Apple Inc. | User profiling for voice input processing |
9652453, | Apr 14 2014 | Xerox Corporation | Estimation of parameters for machine translation without in-domain parallel data |
9658746, | Jul 20 2012 | Wells Fargo Bank, National Association | Accessible reading mode techniques for electronic devices |
9665567, | Sep 21 2015 | International Business Machines Corporation | Suggesting emoji characters based on current contextual emotional state of user |
9668121, | Sep 30 2014 | Apple Inc. | Social reminders |
9672725, | Mar 25 2015 | Microsoft Technology Licensing, LLC | Proximity-based reminders |
9691378, | Nov 05 2015 | Amazon Technologies, Inc | Methods and devices for selectively ignoring captured audio data |
9697827, | Dec 11 2012 | Amazon Technologies, Inc | Error reduction in speech processing |
9720907, | Dec 08 2010 | Microsoft Technology Licensing, LLC | System and method for learning latent representations for natural language tasks |
9721566, | Mar 08 2015 | Apple Inc | Competing devices responding to voice triggers |
9734839, | Jun 20 2012 | Amazon Technologies, Inc | Routing natural language commands to the appropriate applications |
9741343, | Dec 19 2013 | Amazon Technologies, Inc | Voice interaction application selection |
9747083, | Jan 23 2017 | ESSENTIAL PRODUCTS, INC | Home device application programming interface |
9755605, | Sep 19 2013 | Amazon Technologies, Inc | Volume control |
9842584, | Mar 14 2013 | Amazon Technologies, Inc.; Amazon Technologies, Inc | Providing content on multiple devices |
9934785, | Nov 30 2016 | Spotify AB | Identification of taste attributes from an audio signal |
9948728, | May 30 2014 | Apple Inc. | Continuing an activity commenced on a source device with a companion device |
9966068, | Jun 08 2013 | Apple Inc | Interpreting and acting upon commands that involve sharing information with remote devices |
9967381, | Nov 03 2017 | RELAY, INC | Virtual telephony assistant |
9990176, | Jun 28 2016 | Amazon Technologies, Inc.; Amazon Technologies, Inc | Latency reduction for content playback |
20010024967, | |||
20050184958, | |||
20050216271, | |||
20060206724, | |||
20070004451, | |||
20070088556, | |||
20080267416, | |||
20090003115, | |||
20090005012, | |||
20090005891, | |||
20090006096, | |||
20090006097, | |||
20090006099, | |||
20090006100, | |||
20090006343, | |||
20090006345, | |||
20090006488, | |||
20090006671, | |||
20090007001, | |||
20090011709, | |||
20090012748, | |||
20090012775, | |||
20090018828, | |||
20090018829, | |||
20090018834, | |||
20090018835, | |||
20090018839, | |||
20090018840, | |||
20090022329, | |||
20090024595, | |||
20090028435, | |||
20090030685, | |||
20090030800, | |||
20090030978, | |||
20090043580, | |||
20090043583, | |||
20090043763, | |||
20090044094, | |||
20090048821, | |||
20090048841, | |||
20090048845, | |||
20090049067, | |||
20090055168, | |||
20090055175, | |||
20090055179, | |||
20090055186, | |||
20090055380, | |||
20090055381, | |||
20090055648, | |||
20090058823, | |||
20090058860, | |||
20090060351, | |||
20090060472, | |||
20090063974, | |||
20090064031, | |||
20090070097, | |||
20090070102, | |||
20090070109, | |||
20090070114, | |||
20090074214, | |||
20090076792, | |||
20090076796, | |||
20090076798, | |||
20090076819, | |||
20090076821, | |||
20090076825, | |||
20090077047, | |||
20090077165, | |||
20090077464, | |||
20090079622, | |||
20090083034, | |||
20090083035, | |||
20090083036, | |||
20090083037, | |||
20090083047, | |||
20090089058, | |||
20090091537, | |||
20090092239, | |||
20090092260, | |||
20090092261, | |||
20090092262, | |||
20090094029, | |||
20090094033, | |||
20090097634, | |||
20090097637, | |||
20090098903, | |||
20090100049, | |||
20090100454, | |||
20090104898, | |||
20090106026, | |||
20090106376, | |||
20090106397, | |||
20090112572, | |||
20090112576, | |||
20090112592, | |||
20090112596, | |||
20090112677, | |||
20090112892, | |||
20090119587, | |||
20090123021, | |||
20090123071, | |||
20090125477, | |||
20090125602, | |||
20090125813, | |||
20090125947, | |||
20090128505, | |||
20090132253, | |||
20090132255, | |||
20090137286, | |||
20090138263, | |||
20090138430, | |||
20090138736, | |||
20090138828, | |||
20090144036, | |||
20090144049, | |||
20090144428, | |||
20090144609, | |||
20090146848, | |||
20090150147, | |||
20090150156, | |||
20090152349, | |||
20090153288, | |||
20090154669, | |||
20090157382, | |||
20090157384, | |||
20090157401, | |||
20090158200, | |||
20090158323, | |||
20090158423, | |||
20090160761, | |||
20090160803, | |||
20090163243, | |||
20090164301, | |||
20090164441, | |||
20090164655, | |||
20090164937, | |||
20090167508, | |||
20090167509, | |||
20090171578, | |||
20090171662, | |||
20090171664, | |||
20090172108, | |||
20090172542, | |||
20090174667, | |||
20090174677, | |||
20090177300, | |||
20090177461, | |||
20090177966, | |||
20090182445, | |||
20090182549, | |||
20090182702, | |||
20090183070, | |||
20090187402, | |||
20090187577, | |||
20090187950, | |||
20090190774, | |||
20090191895, | |||
20090192782, | |||
20090192787, | |||
20090192798, | |||
20090198497, | |||
20090204402, | |||
20090204409, | |||
20090204478, | |||
20090204596, | |||
20090204601, | |||
20090204620, | |||
20090210230, | |||
20090210232, | |||
20090213134, | |||
20090215466, | |||
20090215503, | |||
20090216396, | |||
20090216528, | |||
20090216540, | |||
20090216704, | |||
20090219166, | |||
20090221274, | |||
20090222257, | |||
20090222270, | |||
20090222488, | |||
20090224867, | |||
20090228126, | |||
20090228273, | |||
20090228277, | |||
20090228281, | |||
20090228439, | |||
20090228792, | |||
20090228842, | |||
20090233264, | |||
20090234638, | |||
20090234651, | |||
20090234655, | |||
20090235280, | |||
20090239202, | |||
20090239552, | |||
20090240485, | |||
20090241054, | |||
20090241760, | |||
20090247237, | |||
20090248182, | |||
20090248395, | |||
20090248402, | |||
20090248420, | |||
20090248422, | |||
20090248456, | |||
20090249198, | |||
20090249247, | |||
20090252350, | |||
20090253457, | |||
20090253463, | |||
20090254339, | |||
20090254345, | |||
20090254819, | |||
20090254823, | |||
20090259472, | |||
20090259475, | |||
20090259969, | |||
20090265171, | |||
20090265368, | |||
20090271109, | |||
20090271175, | |||
20090271176, | |||
20090271178, | |||
20090271188, | |||
20090271189, | |||
20090274315, | |||
20090274376, | |||
20090278804, | |||
20090281789, | |||
20090284471, | |||
20090284482, | |||
20090286514, | |||
20090287583, | |||
20090290718, | |||
20090292987, | |||
20090296552, | |||
20090298474, | |||
20090298529, | |||
20090299745, | |||
20090299849, | |||
20090300391, | |||
20090300488, | |||
20090304198, | |||
20090305203, | |||
20090306967, | |||
20090306969, | |||
20090306979, | |||
20090306980, | |||
20090306981, | |||
20090306985, | |||
20090306988, | |||
20090306989, | |||
20090307162, | |||
20090307201, | |||
20090307584, | |||
20090307594, | |||
20090309352, | |||
20090313014, | |||
20090313020, | |||
20090313023, | |||
20090313026, | |||
20090313544, | |||
20090313564, | |||
20090316943, | |||
20090318119, | |||
20090318198, | |||
20090319257, | |||
20090319266, | |||
20090319342, | |||
20090326923, | |||
20090326936, | |||
20090326938, | |||
20090326949, | |||
20090327977, | |||
20100004918, | |||
20100004930, | |||
20100004931, | |||
20100005081, | |||
20100010803, | |||
20100010814, | |||
20100010948, | |||
20100013760, | |||
20100013796, | |||
20100017212, | |||
20100017382, | |||
20100019834, | |||
20100020035, | |||
20100023318, | |||
20100023320, | |||
20100023331, | |||
20100026526, | |||
20100030549, | |||
20100030562, | |||
20100030928, | |||
20100031143, | |||
20100036653, | |||
20100036655, | |||
20100036660, | |||
20100036829, | |||
20100036928, | |||
20100037183, | |||
20100042400, | |||
20100042576, | |||
20100046842, | |||
20100049498, | |||
20100049514, | |||
20100050064, | |||
20100054512, | |||
20100057435, | |||
20100057443, | |||
20100057457, | |||
20100057461, | |||
20100057643, | |||
20100058200, | |||
20100060646, | |||
20100063804, | |||
20100063825, | |||
20100063961, | |||
20100064113, | |||
20100064218, | |||
20100064226, | |||
20100066546, | |||
20100066684, | |||
20100067723, | |||
20100067867, | |||
20100070281, | |||
20100070517, | |||
20100070521, | |||
20100070899, | |||
20100071003, | |||
20100076760, | |||
20100076993, | |||
20100077350, | |||
20100079501, | |||
20100080398, | |||
20100080470, | |||
20100081456, | |||
20100081487, | |||
20100082286, | |||
20100082327, | |||
20100082328, | |||
20100082329, | |||
20100082333, | |||
20100082346, | |||
20100082347, | |||
20100082348, | |||
20100082349, | |||
20100082376, | |||
20100082567, | |||
20100082970, | |||
20100086152, | |||
20100086153, | |||
20100086156, | |||
20100088020, | |||
20100088093, | |||
20100088100, | |||
20100094632, | |||
20100098231, | |||
20100100212, | |||
20100100384, | |||
20100100385, | |||
20100100816, | |||
20100103776, | |||
20100106486, | |||
20100106498, | |||
20100106500, | |||
20100106503, | |||
20100114856, | |||
20100114887, | |||
20100121637, | |||
20100125456, | |||
20100125458, | |||
20100125460, | |||
20100125811, | |||
20100128701, | |||
20100131269, | |||
20100131273, | |||
20100131498, | |||
20100131899, | |||
20100138215, | |||
20100138224, | |||
20100138416, | |||
20100138680, | |||
20100138759, | |||
20100138798, | |||
20100142740, | |||
20100145694, | |||
20100145700, | |||
20100145707, | |||
20100146442, | |||
20100150321, | |||
20100153114, | |||
20100153115, | |||
20100153448, | |||
20100161311, | |||
20100161313, | |||
20100161337, | |||
20100161554, | |||
20100164897, | |||
20100169075, | |||
20100169093, | |||
20100169097, | |||
20100169098, | |||
20100171713, | |||
20100174544, | |||
20100175066, | |||
20100179932, | |||
20100179991, | |||
20100180218, | |||
20100185448, | |||
20100185949, | |||
20100191520, | |||
20100197359, | |||
20100199180, | |||
20100199215, | |||
20100204986, | |||
20100211199, | |||
20100211379, | |||
20100211644, | |||
20100216509, | |||
20100217604, | |||
20100222033, | |||
20100222098, | |||
20100223055, | |||
20100223056, | |||
20100223131, | |||
20100225599, | |||
20100225809, | |||
20100227642, | |||
20100228540, | |||
20100228549, | |||
20100228691, | |||
20100229082, | |||
20100229100, | |||
20100231474, | |||
20100235167, | |||
20100235341, | |||
20100235729, | |||
20100235732, | |||
20100235770, | |||
20100235780, | |||
20100241418, | |||
20100250542, | |||
20100250599, | |||
20100255858, | |||
20100257160, | |||
20100257478, | |||
20100262599, | |||
20100263015, | |||
20100268537, | |||
20100268539, | |||
20100269040, | |||
20100274753, | |||
20100277579, | |||
20100278320, | |||
20100278453, | |||
20100280983, | |||
20100281034, | |||
20100286984, | |||
20100286985, | |||
20100287514, | |||
20100290632, | |||
20100293460, | |||
20100295645, | |||
20100299133, | |||
20100299138, | |||
20100299142, | |||
20100302056, | |||
20100304342, | |||
20100304705, | |||
20100305807, | |||
20100305947, | |||
20100312547, | |||
20100312566, | |||
20100318366, | |||
20100318576, | |||
20100322438, | |||
20100324709, | |||
20100324895, | |||
20100324896, | |||
20100324905, | |||
20100325131, | |||
20100325158, | |||
20100325573, | |||
20100325588, | |||
20100330908, | |||
20100332003, | |||
20100332220, | |||
20100332224, | |||
20100332235, | |||
20100332236, | |||
20100332280, | |||
20100332348, | |||
20100332428, | |||
20100332976, | |||
20100333030, | |||
20100333163, | |||
20110002487, | |||
20110004475, | |||
20110006876, | |||
20110009107, | |||
20110010178, | |||
20110010644, | |||
20110015928, | |||
20110016150, | |||
20110016421, | |||
20110018695, | |||
20110021211, | |||
20110021213, | |||
20110022292, | |||
20110022388, | |||
20110022393, | |||
20110022394, | |||
20110022472, | |||
20110022952, | |||
20110029616, | |||
20110030067, | |||
20110033064, | |||
20110034183, | |||
20110035144, | |||
20110035434, | |||
20110038489, | |||
20110039584, | |||
20110040707, | |||
20110045841, | |||
20110047072, | |||
20110047149, | |||
20110047161, | |||
20110047266, | |||
20110047605, | |||
20110050591, | |||
20110050592, | |||
20110054647, | |||
20110054894, | |||
20110054901, | |||
20110055256, | |||
20110060584, | |||
20110060587, | |||
20110060589, | |||
20110060807, | |||
20110064387, | |||
20110065456, | |||
20110066366, | |||
20110066436, | |||
20110066468, | |||
20110066634, | |||
20110072033, | |||
20110072492, | |||
20110076994, | |||
20110077943, | |||
20110080260, | |||
20110081889, | |||
20110082688, | |||
20110083079, | |||
20110087491, | |||
20110087685, | |||
20110090078, | |||
20110092187, | |||
20110093261, | |||
20110093265, | |||
20110093271, | |||
20110093272, | |||
20110099000, | |||
20110103682, | |||
20110105097, | |||
20110106736, | |||
20110106892, | |||
20110110502, | |||
20110111724, | |||
20110112825, | |||
20110112827, | |||
20110112837, | |||
20110112838, | |||
20110112921, | |||
20110116610, | |||
20110119049, | |||
20110119051, | |||
20110119623, | |||
20110119715, | |||
20110123004, | |||
20110125498, | |||
20110125540, | |||
20110125701, | |||
20110130958, | |||
20110131036, | |||
20110131038, | |||
20110131045, | |||
20110137636, | |||
20110137664, | |||
20110141141, | |||
20110143726, | |||
20110143811, | |||
20110144857, | |||
20110144901, | |||
20110144973, | |||
20110144999, | |||
20110145718, | |||
20110151830, | |||
20110153209, | |||
20110153322, | |||
20110153324, | |||
20110153329, | |||
20110153330, | |||
20110153373, | |||
20110154193, | |||
20110157029, | |||
20110161072, | |||
20110161076, | |||
20110161079, | |||
20110161309, | |||
20110161852, | |||
20110166851, | |||
20110167350, | |||
20110175810, | |||
20110178804, | |||
20110179002, | |||
20110179372, | |||
20110183627, | |||
20110183650, | |||
20110184721, | |||
20110184730, | |||
20110184736, | |||
20110184737, | |||
20110184768, | |||
20110185288, | |||
20110191108, | |||
20110191271, | |||
20110191344, | |||
20110195758, | |||
20110196670, | |||
20110197128, | |||
20110199312, | |||
20110201385, | |||
20110201387, | |||
20110202526, | |||
20110205149, | |||
20110208511, | |||
20110208524, | |||
20110209088, | |||
20110212717, | |||
20110216093, | |||
20110218806, | |||
20110218855, | |||
20110219018, | |||
20110223893, | |||
20110224972, | |||
20110228913, | |||
20110231182, | |||
20110231184, | |||
20110231188, | |||
20110231218, | |||
20110231432, | |||
20110231474, | |||
20110238191, | |||
20110238407, | |||
20110238408, | |||
20110238676, | |||
20110239111, | |||
20110242007, | |||
20110244888, | |||
20110246471, | |||
20110249144, | |||
20110250570, | |||
20110257966, | |||
20110258188, | |||
20110260829, | |||
20110260861, | |||
20110264643, | |||
20110264999, | |||
20110274303, | |||
20110276595, | |||
20110276598, | |||
20110276944, | |||
20110279368, | |||
20110282663, | |||
20110282888, | |||
20110282906, | |||
20110283189, | |||
20110288852, | |||
20110288855, | |||
20110288861, | |||
20110288863, | |||
20110288866, | |||
20110298585, | |||
20110301943, | |||
20110302162, | |||
20110302645, | |||
20110306426, | |||
20110307241, | |||
20110307254, | |||
20110307491, | |||
20110307810, | |||
20110313775, | |||
20110313803, | |||
20110314003, | |||
20110314032, | |||
20110314404, | |||
20110314539, | |||
20110320187, | |||
20120002820, | |||
20120005602, | |||
20120008754, | |||
20120010886, | |||
20120011138, | |||
20120013609, | |||
20120015629, | |||
20120016658, | |||
20120016678, | |||
20120019400, | |||
20120020490, | |||
20120022787, | |||
20120022857, | |||
20120022860, | |||
20120022868, | |||
20120022869, | |||
20120022870, | |||
20120022872, | |||
20120022874, | |||
20120022876, | |||
20120022967, | |||
20120023088, | |||
20120023095, | |||
20120023462, | |||
20120029661, | |||
20120029910, | |||
20120034904, | |||
20120035907, | |||
20120035908, | |||
20120035924, | |||
20120035925, | |||
20120035926, | |||
20120035931, | |||
20120035932, | |||
20120035935, | |||
20120036556, | |||
20120039539, | |||
20120041752, | |||
20120042014, | |||
20120042343, | |||
20120053815, | |||
20120053829, | |||
20120053945, | |||
20120056815, | |||
20120059655, | |||
20120059813, | |||
20120062473, | |||
20120066212, | |||
20120066581, | |||
20120075054, | |||
20120077479, | |||
20120078611, | |||
20120078624, | |||
20120078627, | |||
20120078635, | |||
20120078747, | |||
20120082317, | |||
20120083286, | |||
20120084086, | |||
20120084634, | |||
20120088219, | |||
20120089331, | |||
20120101823, | |||
20120108166, | |||
20120108221, | |||
20120110456, | |||
20120116770, | |||
20120117499, | |||
20120124126, | |||
20120128322, | |||
20120130709, | |||
20120136572, | |||
20120136649, | |||
20120136855, | |||
20120136985, | |||
20120137367, | |||
20120149342, | |||
20120149394, | |||
20120150544, | |||
20120150580, | |||
20120158293, | |||
20120158399, | |||
20120158422, | |||
20120159380, | |||
20120163710, | |||
20120166196, | |||
20120166959, | |||
20120173222, | |||
20120173244, | |||
20120173464, | |||
20120174121, | |||
20120179457, | |||
20120179467, | |||
20120185237, | |||
20120185480, | |||
20120185781, | |||
20120191461, | |||
20120192096, | |||
20120197743, | |||
20120197995, | |||
20120197998, | |||
20120201362, | |||
20120209654, | |||
20120209853, | |||
20120209874, | |||
20120210266, | |||
20120214141, | |||
20120214517, | |||
20120215762, | |||
20120221339, | |||
20120221552, | |||
20120223889, | |||
20120223936, | |||
20120232885, | |||
20120232886, | |||
20120232906, | |||
20120233207, | |||
20120233266, | |||
20120239661, | |||
20120239761, | |||
20120242482, | |||
20120245719, | |||
20120245939, | |||
20120245941, | |||
20120245944, | |||
20120246064, | |||
20120250858, | |||
20120252367, | |||
20120252540, | |||
20120253785, | |||
20120253791, | |||
20120254143, | |||
20120254152, | |||
20120254290, | |||
20120259615, | |||
20120262296, | |||
20120265482, | |||
20120265528, | |||
20120265535, | |||
20120265806, | |||
20120271625, | |||
20120271634, | |||
20120271635, | |||
20120271640, | |||
20120271676, | |||
20120275377, | |||
20120278744, | |||
20120284015, | |||
20120284027, | |||
20120290291, | |||
20120290300, | |||
20120295708, | |||
20120296638, | |||
20120296649, | |||
20120296654, | |||
20120296891, | |||
20120297348, | |||
20120303369, | |||
20120303371, | |||
20120304124, | |||
20120309363, | |||
20120310642, | |||
20120310649, | |||
20120310652, | |||
20120310922, | |||
20120311478, | |||
20120311583, | |||
20120311584, | |||
20120311585, | |||
20120316862, | |||
20120316875, | |||
20120316878, | |||
20120317194, | |||
20120317498, | |||
20120321112, | |||
20120324391, | |||
20120327009, | |||
20120329529, | |||
20120330660, | |||
20120330661, | |||
20120330990, | |||
20130002716, | |||
20130005405, | |||
20130006633, | |||
20130006637, | |||
20130006638, | |||
20130007648, | |||
20130009858, | |||
20130010575, | |||
20130013313, | |||
20130013319, | |||
20130014026, | |||
20130018659, | |||
20130024576, | |||
20130027875, | |||
20130030787, | |||
20130030789, | |||
20130030804, | |||
20130030815, | |||
20130030913, | |||
20130030955, | |||
20130031162, | |||
20130031476, | |||
20130033643, | |||
20130035086, | |||
20130035942, | |||
20130035961, | |||
20130041647, | |||
20130041654, | |||
20130041661, | |||
20130041665, | |||
20130041667, | |||
20130041968, | |||
20130046544, | |||
20130050089, | |||
20130054550, | |||
20130054609, | |||
20130054613, | |||
20130054675, | |||
20130054706, | |||
20130055099, | |||
20130055147, | |||
20130063611, | |||
20130066832, | |||
20130067307, | |||
20130073286, | |||
20130073346, | |||
20130078930, | |||
20130080152, | |||
20130080162, | |||
20130080167, | |||
20130080177, | |||
20130080251, | |||
20130082967, | |||
20130085755, | |||
20130085761, | |||
20130090921, | |||
20130091090, | |||
20130095805, | |||
20130096909, | |||
20130096917, | |||
20130097566, | |||
20130097682, | |||
20130100268, | |||
20130103391, | |||
20130103405, | |||
20130106742, | |||
20130110505, | |||
20130110515, | |||
20130110518, | |||
20130110519, | |||
20130110520, | |||
20130110943, | |||
20130111330, | |||
20130111348, | |||
20130111365, | |||
20130111487, | |||
20130111581, | |||
20130115927, | |||
20130117022, | |||
20130124189, | |||
20130132084, | |||
20130132089, | |||
20130132871, | |||
20130141551, | |||
20130142317, | |||
20130142345, | |||
20130144594, | |||
20130144616, | |||
20130151339, | |||
20130152092, | |||
20130154811, | |||
20130157629, | |||
20130158977, | |||
20130159847, | |||
20130165232, | |||
20130166303, | |||
20130166442, | |||
20130170738, | |||
20130172022, | |||
20130173258, | |||
20130174034, | |||
20130176244, | |||
20130176592, | |||
20130179172, | |||
20130179440, | |||
20130183944, | |||
20130185059, | |||
20130185074, | |||
20130185081, | |||
20130185336, | |||
20130187850, | |||
20130187857, | |||
20130191117, | |||
20130197911, | |||
20130204813, | |||
20130204897, | |||
20130207898, | |||
20130218553, | |||
20130218560, | |||
20130219333, | |||
20130222249, | |||
20130225128, | |||
20130226935, | |||
20130231917, | |||
20130234947, | |||
20130235987, | |||
20130238326, | |||
20130238647, | |||
20130244615, | |||
20130246048, | |||
20130246050, | |||
20130246329, | |||
20130253911, | |||
20130253912, | |||
20130268263, | |||
20130275117, | |||
20130275138, | |||
20130275164, | |||
20130275199, | |||
20130275625, | |||
20130275875, | |||
20130275899, | |||
20130282709, | |||
20130283168, | |||
20130285913, | |||
20130289991, | |||
20130289993, | |||
20130289994, | |||
20130291015, | |||
20130297317, | |||
20130297319, | |||
20130297348, | |||
20130300645, | |||
20130300648, | |||
20130303106, | |||
20130304479, | |||
20130304758, | |||
20130304815, | |||
20130305119, | |||
20130307855, | |||
20130307997, | |||
20130308922, | |||
20130311184, | |||
20130311997, | |||
20130316746, | |||
20130322634, | |||
20130325436, | |||
20130325443, | |||
20130325447, | |||
20130325448, | |||
20130325480, | |||
20130325481, | |||
20130325484, | |||
20130325967, | |||
20130325970, | |||
20130325979, | |||
20130328809, | |||
20130329023, | |||
20130331127, | |||
20130332159, | |||
20130332162, | |||
20130332164, | |||
20130332168, | |||
20130332172, | |||
20130332400, | |||
20130339256, | |||
20130346068, | |||
20130346347, | |||
20140006012, | |||
20140006025, | |||
20140006027, | |||
20140006030, | |||
20140006153, | |||
20140012574, | |||
20140012580, | |||
20140012586, | |||
20140019116, | |||
20140019133, | |||
20140019460, | |||
20140028735, | |||
20140032453, | |||
20140033071, | |||
20140035823, | |||
20140039888, | |||
20140039893, | |||
20140039894, | |||
20140040274, | |||
20140040748, | |||
20140040801, | |||
20140040918, | |||
20140046934, | |||
20140047001, | |||
20140052680, | |||
20140052791, | |||
20140053082, | |||
20140053210, | |||
20140057610, | |||
20140059030, | |||
20140067361, | |||
20140067371, | |||
20140067402, | |||
20140068751, | |||
20140074454, | |||
20140074466, | |||
20140074470, | |||
20140074472, | |||
20140074483, | |||
20140074815, | |||
20140078065, | |||
20140079195, | |||
20140080428, | |||
20140081619, | |||
20140081633, | |||
20140082501, | |||
20140086458, | |||
20140087711, | |||
20140088961, | |||
20140095171, | |||
20140095172, | |||
20140095173, | |||
20140096209, | |||
20140098247, | |||
20140108017, | |||
20140114554, | |||
20140118155, | |||
20140122059, | |||
20140122086, | |||
20140122136, | |||
20140122153, | |||
20140134983, | |||
20140135036, | |||
20140136187, | |||
20140136195, | |||
20140136212, | |||
20140136946, | |||
20140136987, | |||
20140142922, | |||
20140142923, | |||
20140142935, | |||
20140143550, | |||
20140143721, | |||
20140146200, | |||
20140152577, | |||
20140153709, | |||
20140155031, | |||
20140156262, | |||
20140156279, | |||
20140157422, | |||
20140163951, | |||
20140163953, | |||
20140163954, | |||
20140163976, | |||
20140163977, | |||
20140163981, | |||
20140163995, | |||
20140164476, | |||
20140164532, | |||
20140164533, | |||
20140169795, | |||
20140172878, | |||
20140173460, | |||
20140180499, | |||
20140180689, | |||
20140180697, | |||
20140181865, | |||
20140188477, | |||
20140195226, | |||
20140195230, | |||
20140195233, | |||
20140195244, | |||
20140195251, | |||
20140195252, | |||
20140198048, | |||
20140203939, | |||
20140205076, | |||
20140207439, | |||
20140207446, | |||
20140207466, | |||
20140207468, | |||
20140207582, | |||
20140214429, | |||
20140214537, | |||
20140218372, | |||
20140222436, | |||
20140222678, | |||
20140223377, | |||
20140223481, | |||
20140230055, | |||
20140232656, | |||
20140236595, | |||
20140236986, | |||
20140237042, | |||
20140244248, | |||
20140244249, | |||
20140244254, | |||
20140244257, | |||
20140244258, | |||
20140244263, | |||
20140244266, | |||
20140244268, | |||
20140244271, | |||
20140244712, | |||
20140245140, | |||
20140247383, | |||
20140247926, | |||
20140249817, | |||
20140249821, | |||
20140250046, | |||
20140257809, | |||
20140257815, | |||
20140257902, | |||
20140258857, | |||
20140267022, | |||
20140267599, | |||
20140272821, | |||
20140274005, | |||
20140274203, | |||
20140274211, | |||
20140278343, | |||
20140278349, | |||
20140278379, | |||
20140278390, | |||
20140278391, | |||
20140278394, | |||
20140278406, | |||
20140278413, | |||
20140278429, | |||
20140278435, | |||
20140278436, | |||
20140278443, | |||
20140278513, | |||
20140280138, | |||
20140280292, | |||
20140280353, | |||
20140280450, | |||
20140281983, | |||
20140282003, | |||
20140282007, | |||
20140282045, | |||
20140282201, | |||
20140282203, | |||
20140282586, | |||
20140282743, | |||
20140288990, | |||
20140289508, | |||
20140297267, | |||
20140297281, | |||
20140297284, | |||
20140297288, | |||
20140304605, | |||
20140309996, | |||
20140310001, | |||
20140310002, | |||
20140316585, | |||
20140317030, | |||
20140317502, | |||
20140324884, | |||
20140330569, | |||
20140337048, | |||
20140337266, | |||
20140337371, | |||
20140337438, | |||
20140337751, | |||
20140337814, | |||
20140342762, | |||
20140344627, | |||
20140344687, | |||
20140350924, | |||
20140350933, | |||
20140351741, | |||
20140351760, | |||
20140358519, | |||
20140358523, | |||
20140361973, | |||
20140365209, | |||
20140365214, | |||
20140365216, | |||
20140365226, | |||
20140365227, | |||
20140365407, | |||
20140365880, | |||
20140365885, | |||
20140365895, | |||
20140365922, | |||
20140370817, | |||
20140370841, | |||
20140372112, | |||
20140372356, | |||
20140372931, | |||
20140379334, | |||
20140379341, | |||
20140380285, | |||
20150003797, | |||
20150006148, | |||
20150006157, | |||
20150006176, | |||
20150006178, | |||
20150006184, | |||
20150006199, | |||
20150012271, | |||
20150019219, | |||
20150019221, | |||
20150019974, | |||
20150031416, | |||
20150033219, | |||
20150033275, | |||
20150039292, | |||
20150039299, | |||
20150039305, | |||
20150040012, | |||
20150045003, | |||
20150045068, | |||
20150046537, | |||
20150050633, | |||
20150053779, | |||
20150058013, | |||
20150058018, | |||
20150058785, | |||
20150065200, | |||
20150066494, | |||
20150066496, | |||
20150066506, | |||
20150066516, | |||
20150067485, | |||
20150067822, | |||
20150071121, | |||
20150073788, | |||
20150073804, | |||
20150074524, | |||
20150074615, | |||
20150082229, | |||
20150088511, | |||
20150088514, | |||
20150088522, | |||
20150088523, | |||
20150095031, | |||
20150095278, | |||
20150100316, | |||
20150100537, | |||
20150100983, | |||
20150106093, | |||
20150113407, | |||
20150120723, | |||
20150121216, | |||
20150127348, | |||
20150127350, | |||
20150133109, | |||
20150134334, | |||
20150135085, | |||
20150135123, | |||
20150142420, | |||
20150142438, | |||
20150142447, | |||
20150142851, | |||
20150148013, | |||
20150149177, | |||
20150149182, | |||
20150149354, | |||
20150149469, | |||
20150154185, | |||
20150154976, | |||
20150160855, | |||
20150161370, | |||
20150161521, | |||
20150161989, | |||
20150162001, | |||
20150163558, | |||
20150169284, | |||
20150169336, | |||
20150170664, | |||
20150172463, | |||
20150178388, | |||
20150179176, | |||
20150185964, | |||
20150186012, | |||
20150186110, | |||
20150186154, | |||
20150186155, | |||
20150186156, | |||
20150186351, | |||
20150187355, | |||
20150187369, | |||
20150189362, | |||
20150193379, | |||
20150193391, | |||
20150193392, | |||
20150194152, | |||
20150195379, | |||
20150195606, | |||
20150199077, | |||
20150199960, | |||
20150199965, | |||
20150199967, | |||
20150201064, | |||
20150205858, | |||
20150208226, | |||
20150212791, | |||
20150213796, | |||
20150220507, | |||
20150221304, | |||
20150221307, | |||
20150227505, | |||
20150227633, | |||
20150228281, | |||
20150230095, | |||
20150234636, | |||
20150234800, | |||
20150242091, | |||
20150243278, | |||
20150243283, | |||
20150245154, | |||
20150248651, | |||
20150248886, | |||
20150254057, | |||
20150254058, | |||
20150254333, | |||
20150255071, | |||
20150256873, | |||
20150261496, | |||
20150261850, | |||
20150269139, | |||
20150277574, | |||
20150278370, | |||
20150279358, | |||
20150279360, | |||
20150281380, | |||
20150286627, | |||
20150287401, | |||
20150287409, | |||
20150288629, | |||
20150294086, | |||
20150294377, | |||
20150294516, | |||
20150295915, | |||
20150302855, | |||
20150302856, | |||
20150302857, | |||
20150309997, | |||
20150310858, | |||
20150310862, | |||
20150310879, | |||
20150312182, | |||
20150317069, | |||
20150317310, | |||
20150324041, | |||
20150324334, | |||
20150331664, | |||
20150331711, | |||
20150332667, | |||
20150339049, | |||
20150339391, | |||
20150340033, | |||
20150340040, | |||
20150340042, | |||
20150341717, | |||
20150347086, | |||
20150347381, | |||
20150347382, | |||
20150347385, | |||
20150347393, | |||
20150347733, | |||
20150347985, | |||
20150348547, | |||
20150348548, | |||
20150348549, | |||
20150348551, | |||
20150348554, | |||
20150350031, | |||
20150352999, | |||
20150355879, | |||
20150363587, | |||
20150364140, | |||
20150370531, | |||
20150370780, | |||
20150371639, | |||
20150371665, | |||
20150373183, | |||
20150379993, | |||
20150382047, | |||
20150382079, | |||
20160004690, | |||
20160005320, | |||
20160012038, | |||
20160014476, | |||
20160018900, | |||
20160019886, | |||
20160026258, | |||
20160027431, | |||
20160028666, | |||
20160029316, | |||
20160034042, | |||
20160034811, | |||
20160042735, | |||
20160042748, | |||
20160048666, | |||
20160055422, | |||
20160062605, | |||
20160063998, | |||
20160070581, | |||
20160071516, | |||
20160071521, | |||
20160072940, | |||
20160077794, | |||
20160078860, | |||
20160080165, | |||
20160086116, | |||
20160091967, | |||
20160092447, | |||
20160092766, | |||
20160093291, | |||
20160093298, | |||
20160093301, | |||
20160093304, | |||
20160094700, | |||
20160094979, | |||
20160098991, | |||
20160104486, | |||
20160111091, | |||
20160117386, | |||
20160118048, | |||
20160119338, | |||
20160125048, | |||
20160125071, | |||
20160132484, | |||
20160132488, | |||
20160133254, | |||
20160139662, | |||
20160140951, | |||
20160147725, | |||
20160148610, | |||
20160155442, | |||
20160155443, | |||
20160162456, | |||
20160163311, | |||
20160163312, | |||
20160170966, | |||
20160173578, | |||
20160173960, | |||
20160179462, | |||
20160180844, | |||
20160182410, | |||
20160182709, | |||
20160188181, | |||
20160188738, | |||
20160189717, | |||
20160198319, | |||
20160210981, | |||
20160212488, | |||
20160217784, | |||
20160224540, | |||
20160224774, | |||
20160225372, | |||
20160240187, | |||
20160247061, | |||
20160253312, | |||
20160253528, | |||
20160259623, | |||
20160259656, | |||
20160259779, | |||
20160260431, | |||
20160260433, | |||
20160260434, | |||
20160260436, | |||
20160266871, | |||
20160267904, | |||
20160275941, | |||
20160275947, | |||
20160282956, | |||
20160284005, | |||
20160284199, | |||
20160286045, | |||
20160293168, | |||
20160299685, | |||
20160299882, | |||
20160299883, | |||
20160307566, | |||
20160313906, | |||
20160314788, | |||
20160314792, | |||
20160321261, | |||
20160322045, | |||
20160322050, | |||
20160328205, | |||
20160328893, | |||
20160336007, | |||
20160336010, | |||
20160336024, | |||
20160337299, | |||
20160337301, | |||
20160342685, | |||
20160342781, | |||
20160351190, | |||
20160352567, | |||
20160357304, | |||
20160357728, | |||
20160357861, | |||
20160357870, | |||
20160358598, | |||
20160358600, | |||
20160358619, | |||
20160359771, | |||
20160360039, | |||
20160360336, | |||
20160360382, | |||
20160364378, | |||
20160365101, | |||
20160371250, | |||
20160372112, | |||
20160378747, | |||
20160379091, | |||
20160379626, | |||
20160379633, | |||
20160379641, | |||
20170004824, | |||
20170011303, | |||
20170011742, | |||
20170018271, | |||
20170019987, | |||
20170025124, | |||
20170031576, | |||
20170032783, | |||
20170032791, | |||
20170040002, | |||
20170055895, | |||
20170060853, | |||
20170068423, | |||
20170068513, | |||
20170068550, | |||
20170068670, | |||
20170076720, | |||
20170076721, | |||
20170083179, | |||
20170083285, | |||
20170090569, | |||
20170091168, | |||
20170092270, | |||
20170092278, | |||
20170102915, | |||
20170103749, | |||
20170105190, | |||
20170116177, | |||
20170116989, | |||
20170124190, | |||
20170125016, | |||
20170127124, | |||
20170131778, | |||
20170132199, | |||
20170140644, | |||
20170154033, | |||
20170154055, | |||
20170161018, | |||
20170161268, | |||
20170169818, | |||
20170169819, | |||
20170178619, | |||
20170178626, | |||
20170180499, | |||
20170185375, | |||
20170185581, | |||
20170186429, | |||
20170193083, | |||
20170199874, | |||
20170200066, | |||
20170221486, | |||
20170227935, | |||
20170228382, | |||
20170230709, | |||
20170242653, | |||
20170243468, | |||
20170256256, | |||
20170263247, | |||
20170263248, | |||
20170263249, | |||
20170264451, | |||
20170278514, | |||
20170285915, | |||
20170286397, | |||
20170295446, | |||
20170316775, | |||
20170316782, | |||
20170323637, | |||
20170345411, | |||
20170346949, | |||
20170352346, | |||
20170352350, | |||
20170357478, | |||
20170357632, | |||
20170357633, | |||
20170357637, | |||
20170357640, | |||
20170357716, | |||
20170358300, | |||
20170358301, | |||
20170358302, | |||
20170358303, | |||
20170358304, | |||
20170358305, | |||
20170371885, | |||
20180007060, | |||
20180007538, | |||
20180012596, | |||
20180033431, | |||
20180054505, | |||
20180060312, | |||
20180063624, | |||
20180067914, | |||
20180090143, | |||
20180107945, | |||
20180108346, | |||
20180130470, | |||
20180137856, | |||
20180137857, | |||
20180144748, | |||
20180190273, | |||
20180196683, | |||
20180213448, | |||
20180218735, | |||
20180232203, | |||
20180276197, | |||
20180308485, | |||
20180308486, | |||
20180322112, | |||
20180329677, | |||
20180329957, | |||
20180329982, | |||
20180330714, | |||
20180330722, | |||
20180330723, | |||
20180330730, | |||
20180330731, | |||
20180330737, | |||
20180332118, | |||
20180336275, | |||
20180336892, | |||
20180336894, | |||
20180336905, | |||
20180373487, | |||
20190014450, | |||
AU2015203483, | |||
CA2666438, | |||
CA2694314, | |||
CA2792412, | |||
CN101416471, | |||
CN101427244, | |||
CN101448340, | |||
CN101453498, | |||
CN101459722, | |||
CN101499156, | |||
CN101500041, | |||
CN101515952, | |||
CN101535983, | |||
CN101547396, | |||
CN101557432, | |||
CN101567167, | |||
CN101601088, | |||
CN101604521, | |||
CN101632316, | |||
CN101636736, | |||
CN101673544, | |||
CN101751387, | |||
CN101833286, | |||
CN101847405, | |||
CN101894547, | |||
CN101930789, | |||
CN101939740, | |||
CN101951553, | |||
CN102137193, | |||
CN102160043, | |||
CN102201235, | |||
CN102246136, | |||
CN102282609, | |||
CN102340590, | |||
CN102368256, | |||
CN102405463, | |||
CN102498457, | |||
CN102629246, | |||
CN102682769, | |||
CN102682771, | |||
CN102685295, | |||
CN102693725, | |||
CN102694909, | |||
CN102792320, | |||
CN102801853, | |||
CN102870065, | |||
CN102917004, | |||
CN102918493, | |||
CN103035240, | |||
CN103038728, | |||
CN103093334, | |||
CN103135916, | |||
CN103209369, | |||
CN103365279, | |||
CN103744761, | |||
CN103795850, | |||
CN103930945, | |||
CN104038621, | |||
CN104090652, | |||
CN104144377, | |||
CN104281259, | |||
CN104284257, | |||
CN104335234, | |||
CN104423625, | |||
CN104463552, | |||
CN104516522, | |||
CN104854583, | |||
CN104951077, | |||
CN105100356, | |||
CN105247511, | |||
CN105264524, | |||
CN105471705, | |||
CN107919123, | |||
CN202035047, | |||
CN202092650, | |||
DE102008024258, | |||
DE202016008226, | |||
EP1335620, | |||
EP1720375, | |||
EP1909263, | |||
EP2069895, | |||
EP2081185, | |||
EP2094032, | |||
EP2096840, | |||
EP2107553, | |||
EP2109295, | |||
EP2205010, | |||
EP2309491, | |||
EP2329348, | |||
EP2339576, | |||
EP2400373, | |||
EP2431842, | |||
EP2523188, | |||
EP2551784, | |||
EP2555536, | |||
EP2575128, | |||
EP2632129, | |||
EP2669889, | |||
EP2683175, | |||
EP2733598, | |||
EP2760015, | |||
EP2801890, | |||
EP2801972, | |||
EP2824564, | |||
EP2849177, | |||
EP2930715, | |||
EP2938022, | |||
EP2940556, | |||
EP2950307, | |||
EP2983065, | |||
EP3035329, | |||
EP3224708, | |||
EP3246916, | |||
EP3300074, | |||
GB2445436, | |||
JP10312194, | |||
JP11288296, | |||
JP200029661, | |||
JP2003255991, | |||
JP2003298687, | |||
JP200344090, | |||
JP2004056226, | |||
JP2004101901, | |||
JP2004310034, | |||
JP2007272773, | |||
JP2009069062, | |||
JP2009110300, | |||
JP2009116841, | |||
JP2009134409, | |||
JP2009140444, | |||
JP2009169470, | |||
JP2009186989, | |||
JP2009193448, | |||
JP2009193457, | |||
JP2009193532, | |||
JP2009205367, | |||
JP2009223840, | |||
JP20092850, | |||
JP2009294913, | |||
JP2009294946, | |||
JP200936999, | |||
JP200947920, | |||
JP2009503623, | |||
JP2009505142, | |||
JP2009543166, | |||
JP200998490, | |||
JP2010108378, | |||
JP2010146347, | |||
JP2010157207, | |||
JP2010166478, | |||
JP2010205111, | |||
JP2010224236, | |||
JP2010287063, | |||
JP2010518475, | |||
JP2010518526, | |||
JP2010535377, | |||
JP201066519, | |||
JP201078979, | |||
JP2011238022, | |||
JP2011250027, | |||
JP201133874, | |||
JP201141026, | |||
JP201145005, | |||
JP2011525045, | |||
JP201159659, | |||
JP201181541, | |||
JP2012014394, | |||
JP2012089020, | |||
JP2012116442, | |||
JP2012142744, | |||
JP2012147063, | |||
JP201233997, | |||
JP2012508530, | |||
JP2012518847, | |||
JP2013080476, | |||
JP2013134430, | |||
JP2013140520, | |||
JP2013156349, | |||
JP2013200423, | |||
JP2013205999, | |||
JP2013238936, | |||
JP201337688, | |||
JP2013511214, | |||
JP2013513315, | |||
JP2013517566, | |||
JP2013527947, | |||
JP2013528012, | |||
JP201365284, | |||
JP201373240, | |||
JP2014026629, | |||
JP2014077969, | |||
JP201410688, | |||
JP2014109889, | |||
JP2014124332, | |||
JP2014126600, | |||
JP2014145842, | |||
JP2014150323, | |||
JP2014222514, | |||
JP201460600, | |||
JP201472586, | |||
JP201518365, | |||
JP201541845, | |||
JP2015501022, | |||
JP2015519675, | |||
JP2015524974, | |||
JP2015526776, | |||
JP2015528140, | |||
JP2015528918, | |||
JP20158001, | |||
JP201594848, | |||
JP2016119615, | |||
JP2016151928, | |||
JP2016508007, | |||
JP201719331, | |||
JP2146099, | |||
JP3726448, | |||
JP4563106, | |||
JP9062293, | |||
KR100702645, | |||
KR100920267, | |||
KR101032792, | |||
KR101178310, | |||
KR101193668, | |||
KR101334342, | |||
KR1020060068393, | |||
KR1020060084455, | |||
KR1020090001716, | |||
KR1020090028464, | |||
KR1020090030117, | |||
KR1020090086805, | |||
KR1020090122944, | |||
KR1020090127961, | |||
KR1020090129192, | |||
KR1020100015958, | |||
KR1020100048571, | |||
KR1020100053149, | |||
KR1020100119519, | |||
KR1020110043644, | |||
KR1020110068490, | |||
KR1020110072847, | |||
KR1020110086492, | |||
KR1020110100620, | |||
KR1020110113414, | |||
KR1020110115134, | |||
KR1020120020164, | |||
KR1020120031722, | |||
KR1020120120316, | |||
KR1020120137435, | |||
KR1020120137440, | |||
KR1020120138826, | |||
KR1020120139827, | |||
KR1020130035983, | |||
KR1020130108563, | |||
KR1020130131252, | |||
KR1020130133629, | |||
KR1020140031283, | |||
KR1020140033574, | |||
KR1020140147557, | |||
KR1020150013631, | |||
KR1020150038375, | |||
KR1020150043512, | |||
KR1020160004351, | |||
KR1020160010523, | |||
KR1020160040279, | |||
KR1020170036805, | |||
RU2349970, | |||
RU2353068, | |||
RU2364917, | |||
TW200943903, | |||
TW201018258, | |||
TW201027515, | |||
TW201028996, | |||
TW201110108, | |||
TW201142823, | |||
TW201227715, | |||
TW201245989, | |||
TW201312548, | |||
TW348993, | |||
WO2009009240, | |||
WO2009016631, | |||
WO2009017280, | |||
WO2009034686, | |||
WO2009075912, | |||
WO2009104126, | |||
WO2009156438, | |||
WO2009156978, | |||
WO2010013369, | |||
WO2010054373, | |||
WO2010075623, | |||
WO2010100937, | |||
WO2010141802, | |||
WO2011057346, | |||
WO2011060106, | |||
WO2011088053, | |||
WO2011093025, | |||
WO2011116309, | |||
WO2011133543, | |||
WO2011150730, | |||
WO2011163350, | |||
WO2012019020, | |||
WO2012019637, | |||
WO2012129231, | |||
WO2012135157, | |||
WO2012154317, | |||
WO2012155079, | |||
WO2012167168, | |||
WO2013009578, | |||
WO2013022135, | |||
WO2013022223, | |||
WO2013048880, | |||
WO2013049358, | |||
WO2013163113, | |||
WO2013169842, | |||
WO2013173504, | |||
WO2013173511, | |||
WO2013176847, | |||
WO2013184953, | |||
WO2013184990, | |||
WO2014003138, | |||
WO2014021967, | |||
WO2014022148, | |||
WO2014028797, | |||
WO2014031505, | |||
WO2014047047, | |||
WO2014066352, | |||
WO2014070872, | |||
WO2014078965, | |||
WO2014096506, | |||
WO2014124332, | |||
WO2014137074, | |||
WO2014138604, | |||
WO2014143959, | |||
WO2014144579, | |||
WO2014159581, | |||
WO2014197336, | |||
WO2014200728, | |||
WO2014204659, | |||
WO2015018440, | |||
WO2015029379, | |||
WO2015030796, | |||
WO2015041892, | |||
WO2015084659, | |||
WO2015092943, | |||
WO2015094169, | |||
WO2015094369, | |||
WO2015099939, | |||
WO2015116151, | |||
WO2015151133, | |||
WO2015153310, | |||
WO2015157013, | |||
WO2015183401, | |||
WO2015183699, | |||
WO2015184186, | |||
WO2015200207, | |||
WO2016027933, | |||
WO2016028946, | |||
WO2016033257, | |||
WO2016054230, | |||
WO2016057268, | |||
WO2016075081, | |||
WO2016085775, | |||
WO2016100139, | |||
WO2016111881, | |||
WO2016144840, | |||
WO2016144982, | |||
WO2016175354, | |||
WO2016190950, | |||
WO2016209444, | |||
WO2017044260, | |||
WO2017044629, | |||
WO2017053311, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Dec 17 2018 | Apple Inc. | (assignment on the face of the patent) | / |
Date | Maintenance Fee Events |
Dec 17 2018 | BIG: Entity status set to Undiscounted (note the period is included in the code). |
Dec 27 2023 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
Date | Maintenance Schedule |
Jul 14 2023 | 4 years fee payment window open |
Jan 14 2024 | 6 months grace period start (w surcharge) |
Jul 14 2024 | patent expiry (for year 4) |
Jul 14 2026 | 2 years to revive unintentionally abandoned end. (for year 4) |
Jul 14 2027 | 8 years fee payment window open |
Jan 14 2028 | 6 months grace period start (w surcharge) |
Jul 14 2028 | patent expiry (for year 8) |
Jul 14 2030 | 2 years to revive unintentionally abandoned end. (for year 8) |
Jul 14 2031 | 12 years fee payment window open |
Jan 14 2032 | 6 months grace period start (w surcharge) |
Jul 14 2032 | patent expiry (for year 12) |
Jul 14 2034 | 2 years to revive unintentionally abandoned end. (for year 12) |