systems and methods for silent message delivery are provided. One such embodiment includes a message receiver configured to receive a text message and convert it to a corresponding speech message. A message router is connected to the message receiver and configured to route the speech message dependent on information included in the speech message in accordance with a messaging directory. Preferably, a voice Profile for Inter-system messaging (VPIM) agent is connected to the message router and configured to convert the speech message to a corresponding VPIM message and deliver it to a voice mail system. Methods are also provided for delivery of a text message as speech message to a voice mail system without initiating a call to a telephonic device.
|
7. A method for silent message delivery, comprising:
receiving a text message from a message source, the text message including a telephone number of an intended recipient for voice communications, transmission of said text message to a message receiver being initiated by a message sender and not said intended recipient, wherein the telephone of the intended recipient is entered in the text message by the message sender;
converting said text message to a corresponding speech message, the speech message including the telephone number for the intended recipient;
delivering said speech message to the voice mailbox of the intended recipient residing on a voice mail system without initiating a call to a telephonic device, wherein the intended recipient initiates a call to the voice mail system to access the speech message; wherein said converting of said text message comprises delivering said text message to a text-to-speech converter and receiving a corresponding speech message from said converter; wherein said delivering of said speech message comprises:
querying a messaging directory for contact information corresponding to said intended recipient of said speech message;
delivering said speech message to the voice mail system of said intended recipient without initiating a call to the telephonic device of said intended recipient if contact information corresponding to said intended recipient is obtained from said messaging directory, wherein said contact information is used to identify the voice mail system of said intended recipient;
delivering said speech message to a media server and causing said media server to deliver said speech message to the telephonic device of said intended recipient if contact information corresponding to said intended recipient is not obtained from said messaging directory; and
delivering a delivery confirmation message to said message source that indicates the delivery status of said speech message to the telephonic device of said intended recipient.
1. A system for silent message delivery, comprising:
a message receiver configured to receive a text message from a message source and convert said text message to a corresponding speech message, transmission of said text message to said message receiver being initiated by a message sender and not an intended recipient, said text message identifying said intended recipient and a telephone number of the intended recipient which is used to deliver contents of the text message as a voice message to the intended recipient, the telephone number of the intended recipient being entered in the text message by the message sender;
a message router in communication with said message receiver and configured to route said speech message dependent on information included within said speech message;
a voice mail format conversion agent in communication with said message router and configured to convert said speech message to a corresponding voice mail format message and deliver said voice mail format message to a voice mailbox of the intended recipient residing on a voice mail system, wherein said intended recipient initiates a call to the voice mail system to access the voice mail format message and the voice mail format conversion agent is co-located with the message receiver and the message router; a delivery confirmation notifier in communication with said voice mail format conversion agent and configured to deliver a delivery confirmation message to said message source, said message source comprising a text messaging device;
a voice agent in communication with said message router and configured to deliver said speech message to a media server and cause said media server to deliver said speech message to a telephonic device, wherein said voice agent is in communication with said delivery confirmation notifier; and
wherein said delivery confirmation message indicates the status of the delivery attempt of said voice mail format message to said voice mail system and said speech message to said telephonic device.
2. The system of
3. The system of
4. The system of
5. The system of
6. The system of
8. The method of
9. The method of
|
The present invention is generally related to telecommunications and more particularly to systems and methods for message delivery.
The telecommunications industry has advanced over time to offer many services that enhance or expand traditional voice telephone communications. Networked voice mail systems are a prime example of such advancements. Typically, networked voice mail systems provide all of the features of local answering machine systems along with a number of additional useful features. For example, some networked voice mail systems may submit a voice or text notification to the subscriber (e.g., to a cell phone or pager) that advises of the receipt of voice messages by the voice mail system. As another example, some networked voice mail systems may offer the subscriber the capability to review, reply to, and/or forward voice messages from any location using a telephone. As yet another example, some networked voice mail systems offer the capability to receive and manage messages for multiple subscribers within a common organization, such as a corporate office.
Existing networked voice mail systems are typically only accessible to submit or retrieve messages using voice telephone communications (e.g., a telephone connected to a public telephone system). Furthermore, the option to submit a message to a networked voice mail system is typically available only after a call is made to the subscriber's telephone but not answered. However, there are situations when the use of voice telephone communications to submit a message to a subscriber's networked voice mail system is impractical and/or inconvenient. For example, a message sender may wish to submit a message to a networked voice mail system without disrupting or disturbing the subscriber by making a call to the subscriber's telephone. As another example, a message sender may wish to thoroughly review a message, for example to verify the content, grammar, etc., before submitting the message to a subscriber's networked voice mail system. The needs that arise in these and other similar situations are not sufficiently satisfied by existing systems and methods for networked voice mail.
Therefore, it should be appreciated that there is a need for systems and methods for networked voice mail which address the above-mentioned shortcomings, as well as others, of existing systems and methods.
The present invention provides systems and methods for silent message delivery.
Briefly described, one embodiment of the system, among others, includes a message receiver unit configured to receive a text message and convert it to a corresponding speech message. A message router unit is connected to the message receiver unit and configured to route the speech message dependent on information included in the speech message. Preferably, a Voice Profile for Inter-system Messaging (VPIM) agent unit is connected to the message router unit and is configured to convert the speech message to a corresponding VPIM message and deliver it to a voice mail system.
The present invention can also be viewed as providing a computer program that includes logic configured to receive a text message from a message source. The computer program further includes logic configured to convert the text message to a corresponding speech message and logic configured to deliver the speech message to the voice mail system of an intended recipient without initiating a call to a telephonic device.
The present invention can also be viewed as providing methods for silent message delivery. In this regard, one embodiment of such a method, among others, can be broadly summarized by the following: receiving a text message from a message source, converting said text message to a corresponding speech message, and delivering said speech message to the voice mail system of an intended recipient without initiating a call to a telephonic device.
Other systems, methods, features, and advantages of the present invention will be or become apparent to one with skill in the art upon examination of the following drawings and detailed description. It is intended that all such additional systems, methods, features, and advantages be included within this description and be within the scope of the present invention.
The invention can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the present invention. Moreover, in the drawings, like reference numerals designate corresponding parts throughout the several views.
The invention now will be described more fully with reference to the accompanying drawings. The invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are intended to convey the scope of the invention to those skilled in the art. Furthermore, all “examples” given herein are intended to be non-limiting.
Referring now to
As depicted in
A portable text messaging device 108 may also interface to the telecommunications network 100 via the Internet 110. As depicted, the text messaging device typically submits text messages to the Internet 110 via a wireless link (e.g., RF communications) to an ISP 114. The ISP 114 in turn interfaces the wireless link of the text messaging device 108 to the Internet 110.
As shown in
Preferably, the messaging directory 118 is an information database of silent messaging service subscribers. The messaging directory 118 may also hold information about other types of subscribers, for example voice mail service subscribers. The subscriber information stored in the messaging directory 118 includes information that facilitates the silent delivery (i.e., without initiating a call to a telephone) of a submitted text message as a voice (i.e., speech) message to a subscriber. For example, the messaging directory may store voice mail system identifiers and voice mailbox identifiers of subscribers. This silent messaging delivery process is directly related to the silent message delivery system 102 and will be discussed in further detail subsequently.
The text-to-speech converter 120 converts a text message to a corresponding voice message that is stored as an audio message file in one or more of various file formats, for example WAV, 32K, or ADPCM, among others, which may be known in the art. Typically, a text-to-speech converter 120 is configured to receive a text message in one or more of various file formats. The text-to-speech converter 120 may be locally interfaced to the TCP/IP intranet 116, as depicted in
The voice mail system 122 may be a networked voice mail system or some similar system that provides voice mail services. Typically the voice mail system 122 provides a voice mail system subscriber with services such as storage and retrieval of voice messages. The voice mail system 122 also typically provides various other related services such as voice notification of a received message or forwarding of stored messages. The voice mail system 122 may be located remote to a subscriber's location, for example at a telecommunications service facility, or the voice mail system 122 may be located local to a subscriber's location, for example within a corporate office building where the subscriber works. Access to the voice mail system 122, for example to submit or retrieve messages, is typically made via voice telephone communications systems. For example, as depicted in
As briefly discussed above, the silent message delivery system 102 also interfaces with the TCP/IP intranet 116, typically via the application server 101 as depicted. The interface between the silent message delivery system 102 and the TCP/IP intranet 116 is also typically made by TCP/IP, although the interface may be accomplished by other known techniques in the art. As depicted in
Generally, the silent message delivery system 102 is configured to receive a text (i.e., written, typed, etc.) message and deliver a corresponding voice (i.e., speech) message. In that regard, reference is now made to
Following step 202, the operation of the silent message delivery system 102 proceeds to a step 204 in which the silent message delivery system 102 processes the received text message for delivery to an intended recipient as a corresponding voice (i.e., speech) message. Processing typically includes conversion of the received text message to a voice message by use of a text-to-speech converter 120 that is accessed, for example, via the TCP/IP intranet 116. The processing of step 204 typically also includes querying a messaging directory 118 in an attempt to obtain contact information about the intended recipient that may be stored in the messaging directory.
A step 206 follows the step 204 in which the silent message delivery system 102 delivers the processed voice message to the intended recipient. In this step 206, the silent message delivery system 102 may deliver the voice message to a voice mail system 122 if the intended recipient is a silent messaging subscriber who has contact information stored in the messaging directory 118. This delivery of the voice message to the voice mail system 122 is typically executed without initiating a call to the subscriber's telephonic device. The voice message may then be accessed by the intended recipient using a telephonic device 124 which may, for example, access the voice mail system 122 via a central office 126 in a telephonic communications system.
Alternately in step 206, the silent message delivery system 102 may deliver the voice message directly to the telephonic device 130 of the intended recipient if the recipient is not a silent messaging subscriber and, thus, does not have contact information stored in the messaging directory 118. Typically this delivery is implemented by the silent message delivery system 102 delivering the message to a media server 128. The silent message delivery system 102 instructs the media server to initiate a call to the intended recipient's telephonic device 130 and deliver the voice message to the intended recipient when the telephonic device 130 is answered, for example, by the intended recipient or a voice mail system. The telephonic device 130 may be accessed by the media server 128, for example, via one more central offices 132, 134 that are interconnected by a public switched telephone network 136.
The application server 101 also typically includes a user interface 310 which may include components such as a keyboard or display monitor. The user interface 310 typically provides a user of the application server 101 with means to submit and receive data from the application server 101. Additionally, the application server 101 includes a communications interface 312 which provides an interface to a telecommunications network and/or other elements that are not part of the application server 101. Thus, the silent message delivery system 102 may be executed by the application server 101 to receive a text (i.e., written, typed, etc.) message from a message source and deliver a corresponding voice (i.e., speech) message to an intended recipient, for example, according to the operation illustrated in the flowchart of
The message receiver 404 also interfaces to a text-to-speech converter 120. As discussed above with respect to
The message receiver 404 is linked to a message router 406 within the silent message delivery system 102. The message router 406 receives the converted voice message from the message receiver 404 and determines the appropriate delivery routing of the voice message to the intended recipient. The message router 406 is interfaced to a messaging directory 118. As discussed above with respect to
The message router 406 is linked to the VPIM agent 408, as depicted in
Although not depicted in
A step 504 follows the step 502 in which the text message received by the message receiver 404 is converted to a voice (i.e., speech) message. This conversion is typically made by a text-to-speech converter 120 that is interfaced to the message receiver 404. It is noted that the text-to-speech conversion of the text message could be implemented at other steps of the flowchart 500, in accordance with the invention—typically, the text-to-speech conversion is implemented prior to a subsequent conversion of the message, for example, to a VPIM message. In the step 504, the text message is typically converted to a corresponding audio message file (i.e., a voice message) that includes the telephone number of the intended recipient and may also include other information that was originally included in the text message.
Following the step 504 is a step 506 in which the message router 406 queries the messaging directory 118 for silent messaging system subscriber information related to the intended message recipient. The message router 406 uses contact information, such as the intended recipient's telephone number, that is obtained from the audio message file passed on to the message router 406 from the message receiver 404 to accomplish such queries. In the step 506, the message router 406 queries the messaging directory 118 for a subscriber record that corresponds to the contact information of the intended message recipient. The message router 406 obtains additional information from the record found in the messaging directory 118, such as the subscriber's voice mail system identifier and voice mailbox identifier, that is used to the route the audio message to the voice mail system 122 of the intended recipient.
Following step 506, in the step 508, the VPIM agent 408 converts the audio message file to a VPIM message. The VPIM message typically includes the voice message, the subscriber's voice mail system identifier, and the subscriber's voice mailbox identifier. An example of a VPIM message will be discussed below with respect to
Although not depicted in the flowchart of
The message receiver 404 also interfaces to a text-to-speech converter 120. As discussed above with respect to
The message receiver 404 is linked to a message router 606 within the silent message delivery system 602. The message router 606 receives the converted voice message from the message receiver 404 and determines the appropriate delivery routing of the voice message to the intended recipient. The message router 606 is interfaced to a messaging directory 118. As discussed above with respect to
The message router 606 is linked to the VPIM agent 408. As discussed above with respect to
The message router 606 is also linked to the voice agent 610. The voice agent 610 establishes a connection with a media server 128, for example by internet connectivity and/or TCP/IP, and delivers a voice message to the media server 128 that has been routed to it from the message router 606. The voice agent 610 also causes the voice message to be delivered via the media server 128 to the telephonic device of the intended recipient.
As depicted in
A step 704 follows the step 702 in which the text message received by the message receiver 404 is converted to a voice (i.e., speech) message. This conversion is typically made by a text-to-speech converter 120 that is interfaced to the message receiver 404. It is noted that the text-to-speech conversion of the text message could be implemented at other steps of the flowchart 700, in accordance with the invention—typically, the text-to-speech conversion is implemented prior to a subsequent conversion of the message, for example, to a VPIM message. In the step 704, the text message is typically converted to a corresponding audio message file (i.e., a voice message) that includes the telephone number of the intended recipient and may also include other information that was originally included in the text message.
Following the step 704 is a step 706 in which the message router 606 queries the messaging directory 118 to determine if the intended message recipient is a silent messaging system subscriber. The message router 606 uses contact information, such as the intended recipient's telephone number, that is obtained from the audio message file passed on to the message router 606 from the message receiver 404 to accomplish such queries. In the step 706, the message router 606 queries the messaging directory 118 for a subscriber record that corresponds to the contact information of the intended message recipient. If there is a corresponding record found in the messaging directory 118, the message router 606 obtains additional information from the record, such as the subscriber's voice mail system identifier and voice mailbox identifier, that is used to the route the audio message to the voice mail system 122 of the intended recipient.
In a step 708, which follows the step 706, the message router 606 determines the appropriate routing of the audio message file dependent on whether a subscriber record is found in the messaging directory 118. If a subscriber record is found that corresponds to the contact information of the intended message recipient, the message router 606 routes the audio message file to the VPIM agent 408, which corresponds with a progression to the step 710 of the flowchart 700. Alternately, if there is no subscriber record found that corresponds to the intended recipient, the message router 606 routes the audio message file to the voice agent 610, which corresponds with a progression to the step 714 of the flowchart 700.
In the step 710, the VPIM agent 408 converts the audio message file to a VPIM message. The VPIM message typically includes the voice message, the subscriber's voice mail system identifier, and the subscriber's voice mailbox identifier. An example of a VPIM message will be discussed below with respect to
In the step 714, the voice agent 610 selects an appropriate media server 128 and establishes a connection to it, for example, by internet connectivity and/or TCP/IP. The voice agent 610 then delivers the audio message file, including the intended recipient's telephone number, to the media server 128. Following the step 714, in the step 716, the voice agent 610 instructs the media server 128 to call the telephonic device of the intended message recipient and play the audio message file when the telephonic device is answered.
Following either the step 712 or the step 716 is a step 718 in which the delivery confirmation notifier 612 establishes a connection to the message source 104, for example by internet connectivity and/or TCP/IP, and delivers a delivery confirmation message to it. This delivery confirmation message typically indicates to the message sender that the sent text message has been delivered as an audio message to either the voice mail system 122 of the intended recipient or to the telephonic device of the intended recipient via the media server 128. However, a delivery confirmation message may also indicate other message delivery status information, such as, among other possibilities within the scope of the invention, an unsuccessful delivery of the message to the voice mail system 122 of the intended recipient. The delivery confirmation notifier 612 may, for example, deliver the delivery confirmation message as an e-mail in SMTP (Simple Message Transfer Protocol), however other types of messages may be delivered alternately.
The XML text message 800 also typically includes sender information 804 such as the message sender's name and address. The sender's address may also be a phone number, as shown, although other information, such as the sender's e-mail address, may be used alternately. The sender information 804 may be used, for example, by the delivery confirmation notifier of the silent message delivery system to send a delivery confirmation to the sender of the message 800. Finally, the XML text message 800 typically includes a message 806 that the message sender intends for the recipient to receive as a voice message on the recipient's voice mail system. It is noted that the XML text message 800 discussed in the foregoing is merely exemplary, and other variations of a text message defined by an XML document protocol or some other protocol may be sent from a message source to an embodiment of the silent message delivery system in accordance with the invention, as may be apparent to those skilled in the art.
The VPIM message 900 also typically includes message description information 908, such as the message file name 910 and the message content type 912, among other information. This message description information 908 typically describes the audio message file that is created by the VPIM agent of the silent message delivery system from the text message sent by the sender. Finally, the VPIM message 900 typically includes an actual audio message file 914, which is depicted by text in
It should be understood that any references made herein to Voice Profile for Inter-system Messaging, VPIM, or the like are merely exemplary of one type of voice mail format among others which may be known in the art and whose implementation are included within the scope of the present invention.
It is noted that embodiments of the silent message delivery system, such as those described above, may be implemented in hardware, software, firmware, or a combination thereof. For example, in some embodiments, the silent message delivery system may be implemented as an computer program or application in software or firmware that is stored in a memory and that is executed by a suitable instruction execution system. In other embodiments the silent message delivery system may be implemented, for example, with one or a combination of the following technologies which are known in the art: a discrete logic circuit(s) having logic gates for implementing logic functions upon data signals, an application specific integrated circuit (ASIC) having appropriate combinational logic gates, a programmable gate array(s) (PGA), a field programmable gate array (FPGA), etc.
It is further noted that any process descriptions or blocks in flow charts described above may represent modules, segments, and/or portions of computer program or application code which include one or more executable instructions for implementing specific logical functions or steps in the process. Alternate implementations are included within the scope of the invention in which functions may be executed out of order from that shown in the figures and/or discussed above, including substantially concurrently or in reverse order, depending at least in part on the functionality involved, as will be understood by those skilled in the art.
With regard to any block or network diagrams described above, although the flow of data or other elements may be depicted as unidirectional or bidirectional, such depictions are merely exemplary and not limiting. Variations of the flows depicted in the block or network diagrams are included within the scope of the invention. Furthermore, the functionality of some of the blocks may be implemented by a single combined block within the scope of the invention.
Moreover, embodiments of the silent message delivery system, such as those described above, may comprise an ordered listing of executable instructions for implementing logical functions which can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this disclosure, a “computer-readable medium” may be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer readable medium may be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium. More specific examples (i.e., a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM) (electronic), a read-only memory (ROM) (electronic), an erasable programmable read-only memory (EPROM or Flash memory) (electronic), an optical fiber (optical), and a portable compact disc read-only memory (CDROM) (optical). It is noted that the computer-readable medium may even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
Finally, it should be emphasized that the above-described embodiments of the present invention are merely possible examples of implementations set forth for a clear understanding of the principles of the invention. Many variations and modifications may be made to the above-described embodiment(s) of the invention without departing substantially from the spirit and principles of the invention. All such modifications and variations are intended to be included herein within the scope of this disclosure and the invention, and protected by the following claims.
Holt, Scott C., Adamczyk, Maria, McQuaide, Jr., A. Chester
Patent | Priority | Assignee | Title |
10043516, | Sep 23 2016 | Apple Inc | Intelligent automated assistant |
10049663, | Jun 08 2016 | Apple Inc | Intelligent automated assistant for media exploration |
10049668, | Dec 02 2015 | Apple Inc | Applying neural network language models to weighted finite state transducers for automatic speech recognition |
10049675, | Feb 25 2010 | Apple Inc. | User profiling for voice input processing |
10057736, | Jun 03 2011 | Apple Inc | Active transport based notifications |
10067938, | Jun 10 2016 | Apple Inc | Multilingual word prediction |
10074360, | Sep 30 2014 | Apple Inc. | Providing an indication of the suitability of speech recognition |
10078631, | May 30 2014 | Apple Inc. | Entropy-guided text prediction using combined word and character n-gram language models |
10079014, | Jun 08 2012 | Apple Inc. | Name recognition system |
10083688, | May 27 2015 | Apple Inc | Device voice control for selecting a displayed affordance |
10083690, | May 30 2014 | Apple Inc. | Better resolution when referencing to concepts |
10089072, | Jun 11 2016 | Apple Inc | Intelligent device arbitration and control |
10101822, | Jun 05 2015 | Apple Inc. | Language input correction |
10102359, | Mar 21 2011 | Apple Inc. | Device access using voice authentication |
10108612, | Jul 31 2008 | Apple Inc. | Mobile device having human language translation capability with positional feedback |
10127220, | Jun 04 2015 | Apple Inc | Language identification from short strings |
10127911, | Sep 30 2014 | Apple Inc. | Speaker identification and unsupervised speaker adaptation techniques |
10134385, | Mar 02 2012 | Apple Inc.; Apple Inc | Systems and methods for name pronunciation |
10169329, | May 30 2014 | Apple Inc. | Exemplar-based natural language processing |
10170123, | May 30 2014 | Apple Inc | Intelligent assistant for home automation |
10176167, | Jun 09 2013 | Apple Inc | System and method for inferring user intent from speech inputs |
10185542, | Jun 09 2013 | Apple Inc | Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant |
10186254, | Jun 07 2015 | Apple Inc | Context-based endpoint detection |
10192552, | Jun 10 2016 | Apple Inc | Digital assistant providing whispered speech |
10199051, | Feb 07 2013 | Apple Inc | Voice trigger for a digital assistant |
10223066, | Dec 23 2015 | Apple Inc | Proactive assistance based on dialog communication between devices |
10241644, | Jun 03 2011 | Apple Inc | Actionable reminder entries |
10241752, | Sep 30 2011 | Apple Inc | Interface for a virtual digital assistant |
10249300, | Jun 06 2016 | Apple Inc | Intelligent list reading |
10255907, | Jun 07 2015 | Apple Inc. | Automatic accent detection using acoustic models |
10269345, | Jun 11 2016 | Apple Inc | Intelligent task discovery |
10276170, | Jan 18 2010 | Apple Inc. | Intelligent automated assistant |
10283110, | Jul 02 2009 | Apple Inc. | Methods and apparatuses for automatic speech recognition |
10289433, | May 30 2014 | Apple Inc | Domain specific language for encoding assistant dialog |
10297253, | Jun 11 2016 | Apple Inc | Application integration with a digital assistant |
10303715, | May 16 2017 | Apple Inc | Intelligent automated assistant for media exploration |
10311144, | May 16 2017 | Apple Inc | Emoji word sense disambiguation |
10311871, | Mar 08 2015 | Apple Inc. | Competing devices responding to voice triggers |
10318871, | Sep 08 2005 | Apple Inc. | Method and apparatus for building an intelligent automated assistant |
10332518, | May 09 2017 | Apple Inc | User interface for correcting recognition errors |
10354011, | Jun 09 2016 | Apple Inc | Intelligent automated assistant in a home environment |
10354652, | Dec 02 2015 | Apple Inc. | Applying neural network language models to weighted finite state transducers for automatic speech recognition |
10356243, | Jun 05 2015 | Apple Inc. | Virtual assistant aided communication with 3rd party service in a communication session |
10366158, | Sep 29 2015 | Apple Inc | Efficient word encoding for recurrent neural network language models |
10381016, | Jan 03 2008 | Apple Inc. | Methods and apparatus for altering audio output signals |
10390213, | Sep 30 2014 | Apple Inc. | Social reminders |
10395654, | May 11 2017 | Apple Inc | Text normalization based on a data-driven learning network |
10403278, | May 16 2017 | Apple Inc | Methods and systems for phonetic matching in digital assistant services |
10403283, | Jun 01 2018 | Apple Inc. | Voice interaction at a primary device to access call functionality of a companion device |
10410637, | May 12 2017 | Apple Inc | User-specific acoustic models |
10417266, | May 09 2017 | Apple Inc | Context-aware ranking of intelligent response suggestions |
10417344, | May 30 2014 | Apple Inc. | Exemplar-based natural language processing |
10417405, | Mar 21 2011 | Apple Inc. | Device access using voice authentication |
10431204, | Sep 11 2014 | Apple Inc. | Method and apparatus for discovering trending terms in speech requests |
10438595, | Sep 30 2014 | Apple Inc. | Speaker identification and unsupervised speaker adaptation techniques |
10445429, | Sep 21 2017 | Apple Inc. | Natural language understanding using vocabularies with compressed serialized tries |
10446141, | Aug 28 2014 | Apple Inc. | Automatic speech recognition based on user feedback |
10446143, | Mar 14 2016 | Apple Inc | Identification of voice inputs providing credentials |
10453443, | Sep 30 2014 | Apple Inc. | Providing an indication of the suitability of speech recognition |
10474753, | Sep 07 2016 | Apple Inc | Language identification using recurrent neural networks |
10475446, | Jun 05 2009 | Apple Inc. | Using context information to facilitate processing of commands in a virtual assistant |
10482874, | May 15 2017 | Apple Inc | Hierarchical belief states for digital assistants |
10490187, | Jun 10 2016 | Apple Inc | Digital assistant providing automated status report |
10496705, | Jun 03 2018 | Apple Inc | Accelerated task performance |
10496753, | Jan 18 2010 | Apple Inc.; Apple Inc | Automatically adapting user interfaces for hands-free interaction |
10497365, | May 30 2014 | Apple Inc. | Multi-command single utterance input method |
10504518, | Jun 03 2018 | Apple Inc | Accelerated task performance |
10509862, | Jun 10 2016 | Apple Inc | Dynamic phrase expansion of language input |
10521466, | Jun 11 2016 | Apple Inc | Data driven natural language event detection and classification |
10529332, | Mar 08 2015 | Apple Inc. | Virtual assistant activation |
10552013, | Dec 02 2014 | Apple Inc. | Data detection |
10553209, | Jan 18 2010 | Apple Inc. | Systems and methods for hands-free notification summaries |
10553215, | Sep 23 2016 | Apple Inc. | Intelligent automated assistant |
10567477, | Mar 08 2015 | Apple Inc | Virtual assistant continuity |
10568032, | Apr 03 2007 | Apple Inc. | Method and system for operating a multi-function portable electronic device using voice-activation |
10580409, | Jun 11 2016 | Apple Inc. | Application integration with a digital assistant |
10592095, | May 23 2014 | Apple Inc. | Instantaneous speaking of content on touch devices |
10592604, | Mar 12 2018 | Apple Inc | Inverse text normalization for automatic speech recognition |
10593346, | Dec 22 2016 | Apple Inc | Rank-reduced token representation for automatic speech recognition |
10607140, | Jan 25 2010 | NEWVALUEXCHANGE LTD. | Apparatuses, methods and systems for a digital conversation management platform |
10607141, | Jan 25 2010 | NEWVALUEXCHANGE LTD. | Apparatuses, methods and systems for a digital conversation management platform |
10636424, | Nov 30 2017 | Apple Inc | Multi-turn canned dialog |
10643611, | Oct 02 2008 | Apple Inc. | Electronic devices with voice command and contextual data processing capabilities |
10657328, | Jun 02 2017 | Apple Inc | Multi-task recurrent neural network architecture for efficient morphology handling in neural language modeling |
10657961, | Jun 08 2013 | Apple Inc. | Interpreting and acting upon commands that involve sharing information with remote devices |
10657966, | May 30 2014 | Apple Inc. | Better resolution when referencing to concepts |
10659851, | Jun 30 2014 | Apple Inc. | Real-time digital assistant knowledge updates |
10671428, | Sep 08 2015 | Apple Inc | Distributed personal assistant |
10679605, | Jan 18 2010 | Apple Inc | Hands-free list-reading by intelligent automated assistant |
10681212, | Jun 05 2015 | Apple Inc. | Virtual assistant aided communication with 3rd party service in a communication session |
10684703, | Jun 01 2018 | Apple Inc | Attention aware virtual assistant dismissal |
10691473, | Nov 06 2015 | Apple Inc | Intelligent automated assistant in a messaging environment |
10692504, | Feb 25 2010 | Apple Inc. | User profiling for voice input processing |
10699717, | May 30 2014 | Apple Inc. | Intelligent assistant for home automation |
10705794, | Jan 18 2010 | Apple Inc | Automatically adapting user interfaces for hands-free interaction |
10706373, | Jun 03 2011 | Apple Inc. | Performing actions associated with task items that represent tasks to perform |
10706841, | Jan 18 2010 | Apple Inc. | Task flow identification based on user intent |
10714095, | May 30 2014 | Apple Inc. | Intelligent assistant for home automation |
10714117, | Feb 07 2013 | Apple Inc. | Voice trigger for a digital assistant |
10720160, | Jun 01 2018 | Apple Inc. | Voice interaction at a primary device to access call functionality of a companion device |
10726832, | May 11 2017 | Apple Inc | Maintaining privacy of personal information |
10733375, | Jan 31 2018 | Apple Inc | Knowledge-based framework for improving natural language understanding |
10733982, | Jan 08 2018 | Apple Inc | Multi-directional dialog |
10733993, | Jun 10 2016 | Apple Inc. | Intelligent digital assistant in a multi-tasking environment |
10741181, | May 09 2017 | Apple Inc. | User interface for correcting recognition errors |
10741185, | Jan 18 2010 | Apple Inc. | Intelligent automated assistant |
10747498, | Sep 08 2015 | Apple Inc | Zero latency digital assistant |
10748546, | May 16 2017 | Apple Inc. | Digital assistant services based on device capabilities |
10755051, | Sep 29 2017 | Apple Inc | Rule-based natural language processing |
10755703, | May 11 2017 | Apple Inc | Offline personal assistant |
10762293, | Dec 22 2010 | Apple Inc.; Apple Inc | Using parts-of-speech tagging and named entity recognition for spelling correction |
10769385, | Jun 09 2013 | Apple Inc. | System and method for inferring user intent from speech inputs |
10789041, | Sep 12 2014 | Apple Inc. | Dynamic thresholds for always listening speech trigger |
10789945, | May 12 2017 | Apple Inc | Low-latency intelligent automated assistant |
10789959, | Mar 02 2018 | Apple Inc | Training speaker recognition models for digital assistants |
10791176, | May 12 2017 | Apple Inc | Synchronization and task delegation of a digital assistant |
10791216, | Aug 06 2013 | Apple Inc | Auto-activating smart responses based on activities from remote devices |
10795541, | Jun 03 2011 | Apple Inc. | Intelligent organization of tasks items |
10810274, | May 15 2017 | Apple Inc | Optimizing dialogue policy decisions for digital assistants using implicit feedback |
10818288, | Mar 26 2018 | Apple Inc | Natural assistant interaction |
10839159, | Sep 28 2018 | Apple Inc | Named entity normalization in a spoken dialog system |
10847142, | May 11 2017 | Apple Inc. | Maintaining privacy of personal information |
10878809, | May 30 2014 | Apple Inc. | Multi-command single utterance input method |
10892996, | Jun 01 2018 | Apple Inc | Variable latency device coordination |
10904611, | Jun 30 2014 | Apple Inc. | Intelligent automated assistant for TV user interactions |
10909171, | May 16 2017 | Apple Inc. | Intelligent automated assistant for media exploration |
10909331, | Mar 30 2018 | Apple Inc | Implicit identification of translation payload with neural machine translation |
10928918, | May 07 2018 | Apple Inc | Raise to speak |
10930282, | Mar 08 2015 | Apple Inc. | Competing devices responding to voice triggers |
10942702, | Jun 11 2016 | Apple Inc. | Intelligent device arbitration and control |
10942703, | Dec 23 2015 | Apple Inc. | Proactive assistance based on dialog communication between devices |
10944859, | Jun 03 2018 | Apple Inc | Accelerated task performance |
10978090, | Feb 07 2013 | Apple Inc. | Voice trigger for a digital assistant |
10984326, | Jan 25 2010 | NEWVALUEXCHANGE LTD. | Apparatuses, methods and systems for a digital conversation management platform |
10984327, | Jan 25 2010 | NEW VALUEXCHANGE LTD. | Apparatuses, methods and systems for a digital conversation management platform |
10984780, | May 21 2018 | Apple Inc | Global semantic word embeddings using bi-directional recurrent neural networks |
10984798, | Jun 01 2018 | Apple Inc. | Voice interaction at a primary device to access call functionality of a companion device |
11009970, | Jun 01 2018 | Apple Inc. | Attention aware virtual assistant dismissal |
11010127, | Jun 29 2015 | Apple Inc. | Virtual assistant for media playback |
11010550, | Sep 29 2015 | Apple Inc | Unified language modeling framework for word prediction, auto-completion and auto-correction |
11010561, | Sep 27 2018 | Apple Inc | Sentiment prediction from textual data |
11023513, | Dec 20 2007 | Apple Inc. | Method and apparatus for searching using an active ontology |
11025565, | Jun 07 2015 | Apple Inc | Personalized prediction of responses for instant messaging |
11037565, | Jun 10 2016 | Apple Inc. | Intelligent digital assistant in a multi-tasking environment |
11048473, | Jun 09 2013 | Apple Inc. | Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant |
11069336, | Mar 02 2012 | Apple Inc. | Systems and methods for name pronunciation |
11069347, | Jun 08 2016 | Apple Inc. | Intelligent automated assistant for media exploration |
11080012, | Jun 05 2009 | Apple Inc. | Interface for a virtual digital assistant |
11087759, | Mar 08 2015 | Apple Inc. | Virtual assistant activation |
11120372, | Jun 03 2011 | Apple Inc. | Performing actions associated with task items that represent tasks to perform |
11127397, | May 27 2015 | Apple Inc. | Device voice control |
11133008, | May 30 2014 | Apple Inc. | Reducing the need for manual start/end-pointing and trigger phrases |
11140099, | May 21 2019 | Apple Inc | Providing message response suggestions |
11145294, | May 07 2018 | Apple Inc | Intelligent automated assistant for delivering content from user experiences |
11152002, | Jun 11 2016 | Apple Inc. | Application integration with a digital assistant |
11170166, | Sep 28 2018 | Apple Inc. | Neural typographical error modeling via generative adversarial networks |
11204787, | Jan 09 2017 | Apple Inc | Application integration with a digital assistant |
11217251, | May 06 2019 | Apple Inc | Spoken notifications |
11217255, | May 16 2017 | Apple Inc | Far-field extension for digital assistant services |
11227589, | Jun 06 2016 | Apple Inc. | Intelligent list reading |
11231904, | Mar 06 2015 | Apple Inc. | Reducing response latency of intelligent automated assistants |
11237797, | May 31 2019 | Apple Inc. | User activity shortcut suggestions |
11257504, | May 30 2014 | Apple Inc. | Intelligent assistant for home automation |
11269678, | May 15 2012 | Apple Inc. | Systems and methods for integrating third party services with a digital assistant |
11281993, | Dec 05 2016 | Apple Inc | Model and ensemble compression for metric learning |
11283918, | Aug 26 2010 | RingCentral, Inc | Method and system for automatic transmission of status information |
11289073, | May 31 2019 | Apple Inc | Device text to speech |
11301477, | May 12 2017 | Apple Inc | Feedback analysis of a digital assistant |
11307752, | May 06 2019 | Apple Inc | User configurable task triggers |
11314370, | Dec 06 2013 | Apple Inc. | Method for extracting salient dialog usage from live data |
11348573, | Mar 18 2019 | Apple Inc | Multimodality in digital assistant systems |
11348582, | Oct 02 2008 | Apple Inc. | Electronic devices with voice command and contextual data processing capabilities |
11350253, | Jun 03 2011 | Apple Inc. | Active transport based notifications |
11360641, | Jun 01 2019 | Apple Inc | Increasing the relevance of new available information |
11360739, | May 31 2019 | Apple Inc | User activity shortcut suggestions |
11386266, | Jun 01 2018 | Apple Inc | Text correction |
11405466, | May 12 2017 | Apple Inc. | Synchronization and task delegation of a digital assistant |
11410053, | Jan 25 2010 | NEWVALUEXCHANGE LTD. | Apparatuses, methods and systems for a digital conversation management platform |
11423886, | Jan 18 2010 | Apple Inc. | Task flow identification based on user intent |
11423908, | May 06 2019 | Apple Inc | Interpreting spoken requests |
11462215, | Sep 28 2018 | Apple Inc | Multi-modal inputs for voice commands |
11468282, | May 15 2015 | Apple Inc. | Virtual assistant in a communication session |
11475884, | May 06 2019 | Apple Inc | Reducing digital assistant latency when a language is incorrectly determined |
11475898, | Oct 26 2018 | Apple Inc | Low-latency multi-speaker speech recognition |
11488406, | Sep 25 2019 | Apple Inc | Text detection using global geometry estimators |
11495218, | Jun 01 2018 | Apple Inc | Virtual assistant operation in multi-device environments |
11496600, | May 31 2019 | Apple Inc | Remote execution of machine-learned models |
11500672, | Sep 08 2015 | Apple Inc. | Distributed personal assistant |
11526368, | Nov 06 2015 | Apple Inc. | Intelligent automated assistant in a messaging environment |
11556230, | Dec 02 2014 | Apple Inc. | Data detection |
11587559, | Sep 30 2015 | Apple Inc | Intelligent device identification |
11638059, | Jan 04 2019 | Apple Inc | Content playback on multiple devices |
11656884, | Jan 09 2017 | Apple Inc. | Application integration with a digital assistant |
7706511, | Oct 22 2001 | Apple Inc | System and method for sending text messages converted into speech through an internet connection |
8359234, | Jul 26 2007 | Braintexter, Inc | System to generate and set up an advertising campaign based on the insertion of advertising messages within an exchange of messages, and method to operate said system |
8661454, | Jan 26 2010 | Target Brands, Inc. | System and method for receiving and transmitting event information |
8892446, | Jan 18 2010 | Apple Inc. | Service orchestration for intelligent automated assistant |
8903716, | Jan 18 2010 | Apple Inc. | Personalized vocabulary for digital assistant |
8909545, | Jan 24 2008 | Braintexter, Inc. | System to generate and set up an advertising campaign based on the insertion of advertising messages within an exchange of messages, and method to operate said system |
8930191, | Jan 18 2010 | Apple Inc | Paraphrasing of user requests and results by automated digital assistant |
8942986, | Jan 18 2010 | Apple Inc. | Determining user intent based on ontologies of domains |
9117447, | Jan 18 2010 | Apple Inc. | Using event alert text as input to an automated assistant |
9262612, | Mar 21 2011 | Apple Inc.; Apple Inc | Device access using voice authentication |
9300784, | Jun 13 2013 | Apple Inc | System and method for emergency calls initiated by voice command |
9318108, | Jan 18 2010 | Apple Inc.; Apple Inc | Intelligent automated assistant |
9330720, | Jan 03 2008 | Apple Inc. | Methods and apparatus for altering audio output signals |
9338493, | Jun 30 2014 | Apple Inc | Intelligent automated assistant for TV user interactions |
9368114, | Mar 14 2013 | Apple Inc. | Context-sensitive handling of interruptions |
9430463, | May 30 2014 | Apple Inc | Exemplar-based natural language processing |
9483461, | Mar 06 2012 | Apple Inc.; Apple Inc | Handling speech synthesis of content for multiple languages |
9495129, | Jun 29 2012 | Apple Inc. | Device, method, and user interface for voice-activated navigation and browsing of a document |
9502031, | May 27 2014 | Apple Inc.; Apple Inc | Method for supporting dynamic grammars in WFST-based ASR |
9535906, | Jul 31 2008 | Apple Inc. | Mobile device having human language translation capability with positional feedback |
9548050, | Jan 18 2010 | Apple Inc. | Intelligent automated assistant |
9576574, | Sep 10 2012 | Apple Inc. | Context-sensitive handling of interruptions by intelligent digital assistant |
9582608, | Jun 07 2013 | Apple Inc | Unified ranking with entropy-weighted information for phrase-based semantic auto-completion |
9606986, | Sep 29 2014 | Apple Inc.; Apple Inc | Integrated word N-gram and class M-gram language models |
9620104, | Jun 07 2013 | Apple Inc | System and method for user-specified pronunciation of words for speech synthesis and recognition |
9620105, | May 15 2014 | Apple Inc. | Analyzing audio input for efficient speech and music recognition |
9626955, | Apr 05 2008 | Apple Inc. | Intelligent text-to-speech conversion |
9633004, | May 30 2014 | Apple Inc.; Apple Inc | Better resolution when referencing to concepts |
9633660, | Feb 25 2010 | Apple Inc. | User profiling for voice input processing |
9633674, | Jun 07 2013 | Apple Inc.; Apple Inc | System and method for detecting errors in interactions with a voice-based digital assistant |
9646609, | Sep 30 2014 | Apple Inc. | Caching apparatus for serving phonetic pronunciations |
9646614, | Mar 16 2000 | Apple Inc. | Fast, language-independent method for user authentication by voice |
9668024, | Jun 30 2014 | Apple Inc. | Intelligent automated assistant for TV user interactions |
9668121, | Sep 30 2014 | Apple Inc. | Social reminders |
9697820, | Sep 24 2015 | Apple Inc. | Unit-selection text-to-speech synthesis using concatenation-sensitive neural networks |
9697822, | Mar 15 2013 | Apple Inc. | System and method for updating an adaptive speech recognition model |
9711141, | Dec 09 2014 | Apple Inc. | Disambiguating heteronyms in speech synthesis |
9715875, | May 30 2014 | Apple Inc | Reducing the need for manual start/end-pointing and trigger phrases |
9721566, | Mar 08 2015 | Apple Inc | Competing devices responding to voice triggers |
9734193, | May 30 2014 | Apple Inc. | Determining domain salience ranking from ambiguous words in natural speech |
9760559, | May 30 2014 | Apple Inc | Predictive text input |
9785630, | May 30 2014 | Apple Inc. | Text prediction using combined word N-gram and unigram language models |
9798393, | Aug 29 2011 | Apple Inc. | Text correction processing |
9818400, | Sep 11 2014 | Apple Inc.; Apple Inc | Method and apparatus for discovering trending terms in speech requests |
9842101, | May 30 2014 | Apple Inc | Predictive conversion of language input |
9842105, | Apr 16 2015 | Apple Inc | Parsimonious continuous-space phrase representations for natural language processing |
9858925, | Jun 05 2009 | Apple Inc | Using context information to facilitate processing of commands in a virtual assistant |
9865248, | Apr 05 2008 | Apple Inc. | Intelligent text-to-speech conversion |
9865280, | Mar 06 2015 | Apple Inc | Structured dictation using intelligent automated assistants |
9886432, | Sep 30 2014 | Apple Inc. | Parsimonious handling of word inflection via categorical stem + suffix N-gram language models |
9886953, | Mar 08 2015 | Apple Inc | Virtual assistant activation |
9899019, | Mar 18 2015 | Apple Inc | Systems and methods for structured stem and suffix language models |
9922642, | Mar 15 2013 | Apple Inc. | Training an at least partial voice command system |
9934775, | May 26 2016 | Apple Inc | Unit-selection text-to-speech synthesis based on predicted concatenation parameters |
9953088, | May 14 2012 | Apple Inc. | Crowd sourcing information to fulfill user requests |
9959870, | Dec 11 2008 | Apple Inc | Speech recognition involving a mobile device |
9966060, | Jun 07 2013 | Apple Inc. | System and method for user-specified pronunciation of words for speech synthesis and recognition |
9966065, | May 30 2014 | Apple Inc. | Multi-command single utterance input method |
9966068, | Jun 08 2013 | Apple Inc | Interpreting and acting upon commands that involve sharing information with remote devices |
9971774, | Sep 19 2012 | Apple Inc. | Voice-based media searching |
9972304, | Jun 03 2016 | Apple Inc | Privacy preserving distributed evaluation framework for embedded personalized systems |
9986419, | Sep 30 2014 | Apple Inc. | Social reminders |
Patent | Priority | Assignee | Title |
6374223, | Jun 11 1999 | AVAYA Inc | Internet to voice mail messaging |
6411684, | Sep 16 1994 | AVAYA Inc | Network-based multimedia communications and directory system and method of operation |
6507643, | Mar 16 2000 | Breveon Incorporated | Speech recognition system and method for converting voice mail messages to electronic mail messages |
6765997, | Sep 13 1999 | MicroStrategy, Incorporated | System and method for the creation and automatic deployment of personalized, dynamic and interactive voice services, with the direct delivery of voice services to networked voice messaging systems |
6785266, | Mar 02 1998 | WEB TELEPHONY LLC | Internet controlled telephone system |
6895084, | Aug 24 1999 | Genesys Telecommunications Laboratories, Inc | System and method for generating voice pages with included audio files for use in a voice page delivery system |
6996212, | Jun 26 2002 | AT&T Intellectual Property I, L P | Voicemail system with subscriber specific storage folders |
7190950, | Jun 27 2002 | AT&T Intellectual Property I, L P | Storage of voicemail messages at an alternate storage location |
7212614, | Nov 09 2001 | AT&T Corp | Voice-messaging with attachments |
7369649, | Aug 15 2003 | AVAYA LLC | System and method for caller initiated voicemail annotation and its transmission over IP/SIP for flexible and efficient voice mail retrieval |
20030035412, | |||
20030135569, | |||
20050191994, | |||
20060013368, | |||
20080075244, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Jul 26 2002 | MCQUAIDE, JR , A CHESTER | Bellsouth Intellectual Property Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 013150 | /0894 | |
Jul 26 2002 | HOLT, SCOTT C | Bellsouth Intellectual Property Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 013150 | /0894 | |
Jul 29 2002 | AT&T Intellectual Property I, L.P. | (assignment on the face of the patent) | / | |||
Jul 29 2002 | ADAMCZYK, MARIA | Bellsouth Intellectual Property Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 013150 | /0894 | |
Apr 27 2007 | BellSouth Intellecutal Property Corporation | AT&T DELAWARE INTELLECTUAL PROPERTY, INC , F K A AT&T BLS INTELLECTUAL PROPERTY, INC , F K A AT&T INTELLECTUAL PROPERTY, INC | CHANGE OF NAME SEE DOCUMENT FOR DETAILS | 022189 | /0171 | |
Jan 30 2009 | AT&T Delaware Intellectual Property, Inc | AT&T Intellectual Property I, L P | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 022189 | /0131 |
Date | Maintenance Fee Events |
Apr 30 2009 | ASPN: Payor Number Assigned. |
Apr 30 2009 | RMPN: Payer Number De-assigned. |
Oct 04 2012 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
Oct 27 2016 | M1552: Payment of Maintenance Fee, 8th Year, Large Entity. |
Jan 04 2021 | REM: Maintenance Fee Reminder Mailed. |
Jun 21 2021 | EXP: Patent Expired for Failure to Pay Maintenance Fees. |
Date | Maintenance Schedule |
May 19 2012 | 4 years fee payment window open |
Nov 19 2012 | 6 months grace period start (w surcharge) |
May 19 2013 | patent expiry (for year 4) |
May 19 2015 | 2 years to revive unintentionally abandoned end. (for year 4) |
May 19 2016 | 8 years fee payment window open |
Nov 19 2016 | 6 months grace period start (w surcharge) |
May 19 2017 | patent expiry (for year 8) |
May 19 2019 | 2 years to revive unintentionally abandoned end. (for year 8) |
May 19 2020 | 12 years fee payment window open |
Nov 19 2020 | 6 months grace period start (w surcharge) |
May 19 2021 | patent expiry (for year 12) |
May 19 2023 | 2 years to revive unintentionally abandoned end. (for year 12) |