In accordance with an embodiment, described herein is a system and method for use of crowdsourced microphone or other information with a digital media content environment. A user may bring their media device with media application to a venue/performance, for example a theater, or a music concert, or other event which the user desires to attend. A microphone and/or other hardware components of the media device can be used to indicate a presence of the media device. The system can also, or alternatively, utilize an interaction information by the media device or the user thereof with other systems or environments. The media application can be configured so that, upon determining that the media device is present within a particular venue or attending a particular performance, the media server can enable one or more venue-aware media application features, to provide various types of functionalities that improve the user experience.
|
1. A system for use of crowdsourced microphone or other information with a digital media content environment, comprising:
a plurality of media devices associated with a plurality of users, wherein each media device of the plurality of media devices includes a media application, and a microphone or other hardware components that indicate a presence of the media device at a particular venue during a particular performance, which presence information is provided to a media server;
wherein the presence information indicative of each media device as being present at the particular venue during the particular performance is determined by one or more of:
receiving, by the microphone of the media device, audio corresponding to the particular performance;
matching a location data associated with the media device, with additional data indicative of a time and location of known performances;
determining a presence of other media devices that are registered as being in attendance at the particular performance;
receiving, from another system, a data indicative of a user attendance at the particular performance; or
receiving, by the media device, an interaction with the media application indicative of the user attendance at the particular performance;
wherein upon determining that the plurality of media devices are present at the particular venue during the particular performance, the media server communicates with each media device to provide operation of one or more venue-aware media application features, including:
modifying an original operation of the user interface to include a venue-aware interface; and
upon receipt, at the microphone of the media device, of an audio waveform associated with the particular performance, controlling the operation of the one or more venue-aware media application features in synchronization with the particular performance.
9. A method for use of crowdsourced microphone or other information with a digital media content environment, comprising:
identifying a plurality of media devices associated with a plurality of users, wherein each media device of the plurality of media devices includes a media application, and a microphone or other hardware components that indicate a presence of the media device at a particular venue during a particular performance, which presence information is provided to a media server;
wherein the presence information indicative of each media device as being present at the particular venue during the particular performance is determined by one or more of:
receiving, by the microphone of the media device, audio corresponding to the particular performance;
matching a location data associated with the media device, with additional data indicative of a time and location of known performances;
determining a presence of other media devices that are registered as being in attendance at the particular performance;
receiving, from another system, a data indicative of a user attendance at the particular performance; or
receiving, by the media device, an interaction with the media application indicative of the user attendance at the particular performance;
wherein upon determining that the plurality of media devices are present at the particular venue during the particular performance, the media server communicates with each media device to provide operation of one or more venue-aware media application features, including:
modifying an original operation of the user interface to include a venue-aware interface; and
upon receipt, at the microphone of the media device, of an audio waveform associated with the particular performance, controlling the operation of the one or more venue-aware media application features in synchronization with the particular performance.
17. A non-transitory computer readable storage medium, including instructions stored thereon which when read and executed by one or more computers cause the one or more computers to perform the steps comprising:
identifying a plurality of media devices associated with a plurality of users, wherein each media device of the plurality of media devices includes a media application, and a microphone or other hardware components that indicate a presence of the media device at a particular venue during a particular performance, which presence information is provided to a media server;
wherein the presence information indicative of each media device as being present at the particular venue during the particular performance is determined by one or more of:
receiving, by the microphone of the media device, audio corresponding to the particular performance;
matching a location data associated with the media device, with additional data indicative of a time and location of known performances;
determining a presence of other media devices that are registered as being in attendance at the particular performance;
receiving, from another system, a data indicative of a user attendance at the particular performance; or
receiving, by the media device, an interaction with the media application indicative of the user attendance at the particular performance;
wherein upon determining that the plurality of media devices are present at the particular venue during the particular performance, the media server communicates with each media device to provide operation of one or more venue-aware media application features, including:
modifying an original operation of the user interface to include a venue-aware interface; and
upon receipt, at the microphone of the media device, of an audio waveform associated with the particular performance, controlling the operation of the one or more venue-aware media application features in synchronization with the particular performance.
2. The system of
3. The system of
4. The system of
5. The system of
6. The system of
7. The system of
8. The system of
10. The method of
11. The method of
12. The method of
13. The method of
14. The method of
15. The method of
16. The method of
18. The non-transitory computer readable storage medium of
19. The non-transitory computer readable storage medium of
20. The non-transitory computer readable storage medium of
|
|||||||||||||||||||||||||||||||
This application claims the benefit of priority to U.S. Provisional Patent Application titled “SYSTEM AND METHOD FOR USE OF CROWDSOURCED MICROPHONE OR OTHER INFORMATION WITH A DIGITAL MEDIA CONTENT ENVIRONMENT”, Application No. 62/440,932, filed Dec. 30, 2016; which application is herein incorporated by reference.
A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
Embodiments of the invention are generally related to providing digital media content, and are particularly related to a system and method for use of crowdsourced microphone or other information with a digital media content environment.
Today's consumers enjoy the ability to access a tremendous amount of media content, such as music and videos, using a wide variety of media devices. Digital media content environments, for example media streaming services such as Spotify, enable users to stream music content that complements their lifestyle, while commuting, exercising, relaxing at home, or engaging in other types of activities. Such digital media content environments are well suited to offering additional lifestyle-oriented functionality that can provide an improved user experience. However, traditional digital media technologies are limited in the extent to which they can integrate with the user's physical environment.
In accordance with an embodiment, described herein is a system and method for use of crowdsourced microphone or other information with a digital media content environment. A user may bring their media device with media application to a venue/performance, for example a theater, or a music concert, or other event which the user desires to attend. A microphone and/or other hardware components of the media device can be used to indicate a presence of the media device. The system can also, or alternatively, utilize an interaction information by the media device or the user thereof with other systems or environments. The media application can be configured so that, upon determining that the media device is present within a particular venue or attending a particular performance, the media server can enable one or more venue-aware media application features, to provide various types of functionalities that improve the user experience.
As described above, digital media content environments, for example media streaming services such as Spotify, enable users to stream music content that complements their lifestyle, while commuting, exercising, relaxing at home, or engaging in other types of activities; and are well suited to offering additional lifestyle-oriented functionality that can provide an improved user experience. However, traditional digital media technologies are limited in the extent to which they can integrate with the user's physical environment.
In accordance with an embodiment, described herein is a system and method for use of crowdsourced microphone or other information with a digital media content environment. A user may bring their media device with media application to a venue/performance, for example a theater, or a music concert, or other event which the user desires to attend. A microphone and/or other hardware components of the media device can be used to indicate a presence of the media device. The system can also, or alternatively, utilize an interaction information by the media device or the user thereof with other systems or environments. The media application can be configured so that, upon determining that the media device is present within a particular venue or attending a particular performance, the media server can enable one or more venue-aware media application features, to provide various types of functionalities that improve the user experience.
Digital Media Content Environments
As illustrated in
In accordance with an embodiment, each of the media device and the media server can include, respectively, one or more physical device or computer hardware resources 104, 144, such as one or more processors (CPU), physical memory, network components, or other types of hardware resources.
Although, for purposes of illustration, a single client media device and media server are shown, in accordance with an embodiment a media server can support the simultaneous use of a plurality of client media devices. Similarly, in accordance with an embodiment, a client media device can access media content provided by a plurality of media servers, or switch between different media streams produced by one or more media servers.
In accordance with an embodiment, the media device can optionally include a touch-enabled or other type of display screen having a user interface 106, which is adapted to display media options, for example as an array of media tiles, thumbnails, or other format, and to determine a user interaction or input. Selecting a particular media option, for example a particular media tile or thumbnail, can be used as a command by a user and/or the media device, to the media server, to download, stream or otherwise access a corresponding particular media content item or stream of media content.
In accordance with an embodiment, the media device can also include a software media application 108, together with an in-memory client-side media content buffer 110, and a client-side data buffering logic or software component 112, which can be provided as software or program code that is executable by a computer system or other processing device, and which can be used to control the playback of media content received from the media server, for playing either at a requesting media device (i.e., controlling device) or at a controlled media device (i.e., controlled device), in the manner of a remote control.
In accordance with an embodiment, a connected media environment logic or software component 120, which can be provided as software or program code that is executable by a computer system or other processing device, can be provided at the media device, either as part of the media application, or separately, for example as a firmware, to enable the media device to participate within a connected media environment (e.g., a Spotify Connect environment) that enables a user to control the playback of media content at such controlled devices.
In accordance with an embodiment, the client-side data buffering logic, together with the media content buffer, enables a portion of media content items, or samples thereof, to be pre-buffered at a client media device. For example, while media options are being prepared for display on a user interface, e.g., as media tiles or thumbnails, their related media content can be pre-buffered at the same time, and cached by one or more media devices in their media content buffers, for prompt and efficient playback when required.
In accordance with an embodiment, the media server can include an operating system 146 or other processing environment which supports execution of a media server 150 that can be used, for example, to stream music, video, or other forms of media content to a client media device, or to a controlled device.
In accordance with an embodiment, the media server can provide a subscription-based media streaming service, for which a client media device or user can have an associated account and credentials, and which enable the user's media device to communicate with and receive content from the media server. A received media-access request from a client media device can include information such as, for example, a network address, which identifies a destination media device to which the media server should stream or otherwise provide media content, in response to processing the media-access request.
For example, a user may own several media devices, such as a smartphone and an audio speaker, which can play media content received from a media server. In accordance with an embodiment, identifying information provided with a media-access request can include an identifier, such as an IP address, MAC address, or device name, which identifies that the media-access request is intended for use with a particular destination device. This allows a user, for example, to use their smartphone as a controlling device, and their audio speaker as a controlled device to which media content should be sent. The media server can then send the requested media and/or forward the media-access request to the audio speaker, even though the request originated at the user's smartphone.
In accordance with an embodiment, one or more application interface(s) 148 can receive requests from client media devices, or from other systems, to retrieve media content from the media server. A context database 162 can store data associated with the presentation of media content by a client media device, including, for example, a current position within a media stream that is being presented by the media device, or a playlist associated with the media stream, or one or more previously-indicated user playback preferences. The media server can transmit context information associated with a media stream to a media device that is presenting that stream, so that the context information can be used by the device, and/or displayed to the user. The context database can be used to store a media device's current media state at the media server, and synchronize that state between devices, in a cloud-like manner. Alternatively, media state can be shared in a peer-to-peer manner, wherein each device is aware of its own current media state which is then synchronized with other devices as needed.
For example, in accordance with an embodiment, when the destination media device to which the media content is being streamed changes, say from a controlling device to a controlled device, or from a first controlled device to a second controlled device, then the media server can transmit context information associated with an active media content to the newly-appointed destination device, for use by that device in playing the media content.
In accordance with an embodiment, a media content database 164 can include media content, for example music, songs, videos, movies, or other media content, together with metadata describing that media content. The metadata can be used to enable users and client media devices to search within repositories of media content, to locate particular media content items.
In accordance with an embodiment, a server-side media content buffering logic or software component 180, which can be provided as software or program code that is executable by a computer system or other processing device, can be used to retrieve or otherwise access media content items, in response to requests from client media devices or other systems, and to populate a server-side media content buffer 181, at a media delivery component or streaming service 152, which can be similarly provided as software or program code that is executable by a computer system or other processing device, with streams 182, 184, 186 of corresponding media content data, which can then be returned to the requesting device or to a controlled device.
In accordance with an embodiment, a plurality of client media devices, media server systems, and/or controlled devices, can communicate with one another using a network, for example the Internet 190, a local area network, peer-to-peer connection, wireless or cellular network, or other form of network. For example, a user 192 can interact 194 with the user interface at a client media device, and issue requests to access media content, for example the playing of a selected music or video item at their device, or at a controlled device, or the streaming of a media channel or video stream to their device, or to a controlled device.
In accordance with an embodiment, the user's selection of a particular media option can be communicated 196 to the media server, via the server's application interface. The media server can populate its server-side media content buffer at the server 204, with corresponding media content, 206 including one or more streams of media content data, and can then communicate 208 the selected media content to the user's media device, or to a controlled device as appropriate, where it can be buffered in a client-side media content buffer for playing at the device.
In accordance with an embodiment, and as further described below, the system can include a server-side media gateway or access point 220, which can be provided as software or program code that is executable by a computer system or other processing device, or other process or component, which operates as a load balancer in providing access to one or more servers, for use in processing requests at those servers. The system can enable communication between a client media device and a server, via an access point at the server, and optionally the use of one or more routers, to allow requests from the client media device to be processed either at that server and/or at other servers.
For example, in a Spotify media content environment, Spotify clients operating on media devices can connect to various Spotify back-end processes via a Spotify “accesspoint”, which forwards client requests to other servers, such as sending one or more metadataproxy requests to one of several metadataproxy machines, on behalf of the client or end user.
Venue-Aware Media Application Features
In accordance with an embodiment, the system can include the use of crowdsourced microphone or other information with a digital media content environment, so that upon determining that the media device is present within a particular venue or attending a particular performance, the media server can enable one or more venue-aware media application features, to provide various types of functionalities that improve the user experience.
As illustrated in
In accordance with an embodiment, a microphone and/or other hardware components 252 of the media device can be used to indicate a presence 254 of the media device within the venue or at the performance.
In accordance with various embodiments, the determination of a media device's presence (and user attendance) at a particular venue or at the performance can be achieved in a number of ways, such as for example: matching a GPS location of the media device to a time and location of known performances; hearing, through the device's microphone, a musical performance taking place at the venue; cross-referencing the attendance of nearby (e.g., via Bluetooth, GPS, or ultrasound) user devices which have already registered their attendance; identifying a third-party social media posting referencing a user's attendance, such as a Snapchat or Instagram message/image that includes a metadata containing location coordinates; predicting a user's attendance based on a known ticket purchase by the user; and/or explicitly registering a user's attendance based on a direct interaction from the user.
For example, a media application (e.g., a Spotify app) can be configured to prompt the user with a question “Are you at the (band name) concert?”, and present “Yes”/“No” options to enable the user to set their particular attendance state.
In accordance with an embodiment, such presence information can be further determined by a venue/performance system 256 that is associated with a particular venue or performance, and can be provided as a combination of venue/performance/presence information 258 to a media server.
Alternatively, in accordance with an embodiment, the venue/performance/presence information can be communicated directly from the media device to the media server.
In accordance with an embodiment, the system can also, or alternatively, utilize an interaction information 262 by the media device (or by the user thereof) with other systems or environments 264, for example with social messaging environments at which the user has indicated their intent to attend a particular venue or performance.
Such interaction information 266 can then be similarly provided to the media server, either via those other systems or environments, or as described above directly from the media device to the media server.
In accordance with an embodiment, the media application can be configured so that, upon determining that the media device (and the user thereof) is present within a particular venue or attending a particular performance, the media server can enable 272 one or more venue-aware media application features 274.
In accordance with an embodiment, the venue-aware media application features can then be used to modify the user environment 276, for example by modifying the operation or appearance of the user interface to include a venue-aware interface 278, or to provide other types of functionalities that improve the user experience.
For example, a user may desire to attend a concert where a band is performing. Realizing where it is located, the media application (e.g., a Spotify app) on the user's media device (e.g., smartphone) can use device components such as its illuminated device screen, touch input interface, microphone, accelerometer, GPS radio, or other device components and features, to transform the media device into an interactive multimedia extension of the live concert experience, for the individual user and the crowd as a whole, to everyone's surprise and delight.
In accordance with an embodiment, the features and functionality can be created and customized in a way that is specific to, e.g., the band or the particular performance. In crafting such an interactive experience, the designer can draw from a toolbox of enhanced features which have been developed beforehand, and are made available to the designer, some examples of which enhanced features are described in further detail below.
As illustrated in
For example, as illustrated in
In accordance with an embodiment, when the users enter the venue/performance, their media devices can register its presence 320, and/or the system can also utilize interaction information 322 by the device (i.e., user), determine the presence of multiple media devices and users in attendance. The media server can then enable one or more venue-aware media application features on each of the media devices 324, to modify the user environments at those devices.
In accordance with an embodiment, in addition to establishing a user's attendance, the system can continue to use hardware on the mobile device to collect additional data, for example to continue to enhance and maintain the quality of the enhancements to the user experience.
For example, in accordance with an embodiment, the media device microphone can enable synchronization with the music playing. Rather than simply relying on an external metadata about the performance that says, e.g., song A will start at 10:35 PM, a live recorded waveform received via the microphone can be used to enable a precise synchronization if the band happens to begin the next song a few minutes late.
As another example, in accordance with an embodiment, the media device microphone can be used to assist the system to stay in more precise synchronization with the audio the user hears from the stage and speakers. In a large venue, users can perceive the amount of time the sound takes to travel. Instead of timing all of the media devices to a centralized clock, the waveform captured from the device microphone can more precisely control the elements of the enhanced user experience.
As another example, in accordance with an embodiment, the microphone can be used to determine an exact physical location within three-dimensional space in the venue. A strong single noise played from two or three single sources around the perimeter of the venue will reach devices at different locations within the venue at slightly different times. Using triangulation and time-of-flight, the system can determine where within the venue a device is currently being held, and update that location as users move around the venue.
As another example, in accordance with an embodiment, the media device camera allows precise synchronization to the lighting, to create a strobe effect, for example by incorporating both environmental lights and the screens of visiting devices, which is more effective than if such functionality was timed based solely on device clocks and mobile phone network time sources.
In accordance with an embodiment, when the media device (i.e., user) leaves the venue, the enhanced user experience stops, and the media application resumes its usual operation, for example presenting a “goodbye” interaction to the user, asking a final question, presenting a final piece of information, or soliciting a last third-party social media posting from the user.
As illustrated in
At step 332, the media server determines, upon receipt of information from one or more of the media device, media application, a venue/performance system, or other system or environment, that the media device (and user operating the media device) is attending a particular venue or performance.
At step 334, the operation of the media application (and user environment) is modified, including enabling venue-aware media application features, for at least one or a plurality of media devices (and users operating those media devices) located at the particular venue/performance, using information received from the media server.
At step 336, a determination is made, upon receipt of information from one or more of the media device, media application, a venue/performance system, or other system or environment, that the media device(s) (and user(s) operating those media device(s)) are no longer located at the particular venue/performance, and the usual operation of the media device and media application is restored.
Enhanced Features and Applications
As described above, in accordance with an embodiment, a designer of an interactive user experience can draw from a toolbox of enhanced features developed beforehand, and made available to the designer, some examples of which are illustrated below:
Lighting: in accordance with an embodiment, venue lighting may be attached to a programmed system to show patterns and effects to complement the music while the concert happens. The techniques described herein enable the illuminated screens of mobile devices in attendance to the landscape of light-producing devices at the event designer's disposal.
Audio playback: in accordance with an embodiment, the speakers on the media device can produce audio to complement the speakers installed at the venue. Some instruments, tracks, or effects might play from user devices instead of installed speakers.
Audio recording: in accordance with an embodiment, mastering a track of the live performance, the band's audio engineers can incorporate audio recorded from the audience not only from microphones installed throughout the venue, but also from the microphones attached to attendee's devices.
Singing and lyrics: in accordance with an embodiment, the media device screen can show lyrics that appear in time while the song plays in the venue. Users singing along can have their audio incorporated into the larger experience.
Touch: in accordance with an embodiment, the device touch display could suddenly produce audio for the user as part of the experience, either on the device, surrounding devices, or venue speakers, giving the user a musical instrument, like a drum, to tap on. Alternatively, input from the device touch panel could control light in the venue or a surrounding vicinity display.
Attendance: in accordance with an embodiment, display screens in the venue can list statistics about visitors, such as the number of attendees who are participating in the enhanced experience. Venue screens can display notable attendees, while screens of other attendees display large arrows, pointing them in the direction of the nearest superfan. Precise location positioning and six-axis orientation data from the accelerometer allows the arrows to keep pointing in the same direction as users move their devices.
Social media posts: in accordance with an embodiment, when identified at the venue and part of the experience, the user can be given additional capabilities to make special social media postings, including functionality, like a Snapchat or Snapchat-like photography filter, also only available to the user while he or she is at the concert.
Data mining: in accordance with an embodiment, after the event has concluded, organizers can access a list of who was in attendance complete with their user identity, and third-party social media identities.
The above examples of enhanced features are provided by way of example, to illustrate some of the types of features which the system can support. In accordance with various embodiments, additional and/or different enhanced features and applications can be supported.
Embodiments of the present invention may be conveniently implemented using one or more conventional general purpose or specialized digital computer, computing device, machine, or microprocessor, including one or more processors, memory and/or computer readable storage media programmed according to the teachings of the present disclosure. Appropriate software coding can readily be prepared by skilled programmers based on the teachings of the present disclosure, as will be apparent to those skilled in the software art.
In some embodiments, the present invention includes a computer program product which is a non-transitory storage medium or computer readable medium (media) having instructions stored thereon/in which can be used to program a computer to perform any of the processes of the present invention. Examples of the storage medium can include, but is not limited to, any type of disk including floppy disks, optical discs, DVD, CD-ROMs, microdrive, and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic or optical cards, nanosystems (including molecular memory ICs), or any type of media or device suitable for storing instructions and/or data.
The foregoing description of embodiments of the present invention has been provided for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations will be apparent to the practitioner skilled in the art.
For example, while the techniques described above generally illustrate examples such as a music streaming service such as Spotify, and streamed music or song content, the systems and techniques described herein can be similarly used with other types of media content environments, and other types of streamed data or media content.
The embodiments were chosen and described in order to best explain the principles of the invention and its practical application, thereby enabling others skilled in the art to understand the invention for various embodiments and with various modifications that are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the following claims and their equivalents.
| Patent | Priority | Assignee | Title |
| Patent | Priority | Assignee | Title |
| 6022223, | Oct 31 1995 | Brother Kogyo Kabushiki Kaisha; Xing, Inc. | Video/audio data supplying device |
| 6084168, | Jul 10 1996 | INTELLECTUAL VENTURES ASSETS 28 LLC | Musical compositions communication system, architecture and methodology |
| 8138930, | Jan 22 2008 | GOOGLE LLC | Advertising based on environmental conditions |
| 8929922, | Jun 03 2011 | AMG IP, LLC | Mobile device for venue-oriented communications |
| 9288594, | Dec 17 2012 | Amazon Technologies, Inc | Auditory environment recognition |
| 9412390, | Mar 15 2013 | SMULE, INC | Automatic estimation of latency for synchronization of recordings in vocal capture applications |
| 9696884, | Apr 25 2012 | RPX Corporation | Method and apparatus for generating personalized media streams |
| 20020193895, | |||
| 20080113325, | |||
| 20090094257, | |||
| 20100317371, | |||
| 20110144981, | |||
| 20110252322, | |||
| 20110302504, | |||
| 20120052838, | |||
| 20130290842, | |||
| 20130290843, | |||
| 20140108929, | |||
| 20150264533, | |||
| 20150278769, | |||
| 20150296288, | |||
| 20170180438, | |||
| EP2808870, | |||
| EP2927826, | |||
| EP3241357, | |||
| WO2016108087, |
| Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
| Dec 07 2017 | LIUSAARI, ARI | Spotify AB | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 044658 | /0858 | |
| Dec 28 2017 | Spotify AB | (assignment on the face of the patent) | / |
| Date | Maintenance Fee Events |
| Dec 28 2017 | BIG: Entity status set to Undiscounted (note the period is included in the code). |
| Jan 09 2024 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
| Date | Maintenance Schedule |
| Jul 28 2023 | 4 years fee payment window open |
| Jan 28 2024 | 6 months grace period start (w surcharge) |
| Jul 28 2024 | patent expiry (for year 4) |
| Jul 28 2026 | 2 years to revive unintentionally abandoned end. (for year 4) |
| Jul 28 2027 | 8 years fee payment window open |
| Jan 28 2028 | 6 months grace period start (w surcharge) |
| Jul 28 2028 | patent expiry (for year 8) |
| Jul 28 2030 | 2 years to revive unintentionally abandoned end. (for year 8) |
| Jul 28 2031 | 12 years fee payment window open |
| Jan 28 2032 | 6 months grace period start (w surcharge) |
| Jul 28 2032 | patent expiry (for year 12) |
| Jul 28 2034 | 2 years to revive unintentionally abandoned end. (for year 12) |