A system for automatically tuning a primary broadcast receiver using a secondary broadcast receiver to receive a first broadcast signal corresponding to a source of an origin program, one or more program transition detectors to monitor first broadcast signal tuned by the secondary broadcast receiver to determine and signal a transition from a non-original program content to origin program content while a user views or hears alternate programming from a primary broadcast receiver, and automatically tuning of the primary broadcast receiver to the first broadcast signal upon detection by the program transition detector that one or more of user-specific tune conditions have been met.
|
1. A system comprising:
a primary broadcast tuner and a secondary broadcast tuner in a broadcast television receiver, said primary broadcast tuner being under user control initially to view a first channel or signal source, said secondary broadcast tuner configured to monitor said first channel or signal source subsequent to said user selecting to surf alternate channels and signal sources using said primary broadcast tuner, said secondary broadcast receiver configured to detect an initial transition in content from an initial broadcast program to a sub-program, wherein said sub-program is content unrelated to said initial broadcast program transmitted from said first channel or signal source
a plurality of program transition detectors cooperative with said secondary broadcast tuner and configured to to detect a return from one or more sub-programs to said initial broadcast program, said detectors comprising:
an elapsed timer for a fixed period of time,
an elapsed timer for an integral multiple of a fixed period of time,
a volume change detector,
a vocabulary change detector monitoring closed captioning text,
a vocabulary change detector using voice recognition to monitor an audio program,
a V-chip ratings indicator change detector,
a scene-based program segment detector, and
a detector employing an interprogram dead frame detector;
a set of user-specified return from surfing automatic retuning conditions;
a tuner controller configured to, upon user activation, enable said secondary broadcast receiver and program transition detector to begin said monitoring, to allow surfing of channels and signal sources by a user, and to automatically tune said primary broadcast receiver to said first broadcast signal upon detection by said program transition detector that one or more of said user-specific tune conditions have been met, said tuner controller also configured to insert an alert tone or message into an audible output of the primary receiver upon said tuning action.
|
|||||||||||||||||||||||||||||||||||||||
none
None.
This invention was not developed in conjunction with any Federally-sponsored contract.
Not applicable
The following publicly available documents are incorporated by reference into the present disclosure, including illustrations therein, <dot> represents the period “.” character in a web site address:
1. Field of the Invention
This invention relates to automatic tuning of broadcast signal receivers.
2. Background of the Invention
Since the creation of the first broadcast television receiver (“TV”), many improvements and features have been made which enhance the entertainment value of TV use. These improvements have included addition of color to the image, higher resolution of the images, flattening of the screen, changes in technology for the display (e.g. from cathode ray tube to plasma and projection), and stereo and surround sound. Logical functions have been greatly improved and enhanced as well, such as addition of wireless remote controllers, clock and timer functions, automatic channel memory configuration, picture-in-picture display, etc. The TV has become a display for a number of other electronic systems, such as video tape players, digital versatile disk (“DVD”) players, satellite and cable programming decoders and receivers, video games, and even personal computers.
Today, a TV user can view or watch (including audio) a selection of programming sources often ranging from 8 to 12 “local” “over the air” broadcast channels in most areas, to hundreds of satellite and cable broadcast channels, as well as several local device playback (e.g. DVD, VCR, etc.) devices, all at their fingertips. The variety of shows and programs which are now available 24 hours a day 7 days a week is overwhelming to comprehend for many users, so many systems of menus and program guides have been developed to help a user sort through all of the options, including organizing channels by content type or theme (e.g. news, sports, history, fashion, music, home and garden, etc.), and providing channel guide filters and locks based on audience ratings for programs (e.g. family, teen, adult, etc.). A user can even watch multiple channels at once using the picture-in-picture (“PIP”) feature available on many TV sets, and can record parts of or entire programs while watching or not using a personal video recorder (“PVR”), such as a TiVo ™ device.
All of these options, however, have led to new user habits and frustration. “Channel surfing” is a habit of changing channels rapidly over time in an effort to avoid commercials, watch multiple programs or keep track of multiple live events (e.g. tracking scores in 2 or more simultaneous sports events), or avoid boredom. For example, consider a hypothetical, but very realistic situation where a user, “Mike”, is watching an Oakland Raiders National Football League (“NFL”)™ football game. During a commercial break or during the half-time pause, Mike channel surfs. Mike continuously has to manually check back to see if the Raiders game is back on. There is no product or feature available to Mike, either part of the TV set or the cable/satellite receiver, which can automatically notify Mike when the commercial or half-time break is finished. In fact, the most common feature which can assist Mike is the “channel return” key available on most remote controls for TVs or receivers, which essentially tunes the receiver back to the last channel Mike was watching. Some remote controls, however, update this “last channel memory” value even when Mike uses the channel up or channel down keys on the remote control, so pressing the channel return key would not actually return him to viewing the source of the football game after surfing to several other channels. Additionally, Mike must manually decide when to operate the channel return key, which often results in frustration and a portion of missed programming (e.g. the game starts again but Mike doesn't check back until several seconds or minutes later).
Although PIP can be of use or assistance in such a situation, PIP can obscure the channel currently being watched. Additionally, there is a time gap or delay between when Mike sees his primary program is back on and when he can press the right buttons on his remote to return and disable PIP. Furthermore, if Mike doesn't want to return when back from commercial, but rather when some known segment appears (such as sports highlights in local news), then the limitations of PIP are multiplied.
The following detailed description when taken in conjunction with the figures presented herein present a complete description of the present invention.
The present invention provides a user-configurable “surf return” function to an enhanced broadcast tuner, receiver or decoder for reception of video and audio signals such as, but not limited to, cable television broadcasts, satellite television broadcasts, over-the-air television broadcasts, multimedia broadcasts via computer networks, cable audio program broadcasts, satellite audio program broadcasts, and over-the-air audio program broadcasts.
According to a first aspect of the present invention, the surf return function allows detection of a sub-program such as commercials and automatically returns users to origin when the sub-program ends, thereby allowing the user to tune to alternate programming during the interim time period.
According to another aspect of the present invention, the surf return function provides a systematic way to help users avoid wasting time and energy by checking back repeatedly and periodically between multiple channels to ensure sub-program(s) has ended, etc.
According to another aspect of the present invention, the logic of the invention employs a clock or timer to allow the user to specify return to the original channel within a certain time period or at a certain time.
Using the invention, users or listeners do not need to hassle with “flipping” channels back and forth while waiting for a sub-program such as commercials to end, and broadcasters are allowed to give their customers the maximum ability to obtain entertainment. Additionally, users and listeners may elect to watch shows or listen to programs without being bombarded by constant interruptions and endless marketing tactics.
For example, a user may initiate the invention to allow auto-detection of commercial breaks while the user watches a football game on a first or original channel. When the commercial comes on, the user channel surfs, changing the channel or signal source to one or more alternate programs. When the advertisement ends or optionally when a certain time period has elapsed, the invention automatically tunes the tuner back to the original channel or channel source to return to watching the football game.
In a second example, a user may be interested in learning more about a specific news story that was reported (or has been previewed or announced to be upcoming) on his or her favorite news service. With the present invention, the user does not have to wait patiently through many other news reports and/or advertisements to view the story of interest, but instead can configure the invention to automatically tune the receiver to the news broadcast at the time the story airs. During the interim, the user may surf to other channels or other signal sources.
The present invention will be described in the following paragraphs in terms of one available embodiment of a television receiver/tuner/decoder suitable for receiving audio and video signals from “free” or “over-the-air” transmitters, as well as closed-circuit and subscription types of broadcast signals such as cable TV and satellite TV. It will be recognized that the present invention may alternately be realized to receive other types of audio and video broadcast signals including but not limited to over-the-air radio programs; satellite “radio” programs (e.g. XM™, Sirius™, etc.); and Internet or wireless handset video, audio and multimedia programming (e.g. digital movies, Macromedia's Flash™ objects, streaming audio, streaming video, audio objects, WAV files, MPG files, MP3 files, etc.).
Likewise, the invention may be realized in any suitable form including, but not limited to, enhancements of television tuners, radio tuners, satellite decoders, cable decoders, web browsers, wireless handsets, and pervasive computing devices (e.g. personal digital assistants, laptop computers, etc.). Additionally, the present invention is not restricted to automatically returning the tuning control to an original channel, but may also be configured to return to a non-broadcast signal source, such as a local DVD player or local VCR player. Throughout the remainder of this disclosure, we will refer to the various types of human operators of broadcast receivers, such as TV users, web browser users, radio listeners, etc., collectively as “users”.
Further, we will collectively refer to technologies and signal types for one-to-many transmissions (e.g. one station to many receivers, one web server to many web browsers, etc.), and point-to-point transmissions (e.g. peer-to-peer exchanges, two-party audio and/or video calls, etc.), as well as “live”, “real-time”, “on-demand” and “taped delayed” transmissions collectively as “broadcasts” for the purposes of the present disclosure.
Architecture of Enhanced Television Receiver/Tuner/Decoder
A TV embodiment preferably employs two tuners, a primary tuner (15) and a secondary tuner (14), as shown in
A secondary tuner/decoder (14) is incorporated in the system of the preferred embodiment. This secondary tuner (14) is used to monitor the content or program status of the original channel or signal source while the user “surfs” alternate channels and signal sources using the primary tuner (15). The secondary tuner may be integral to the TV, it may be an accessory, or it may be an external unit. Some examples of external or accessory secondary tuners which may be employed in some embodiments are the tuners in VCR's, audio/video/home theater tuners/controllers, or tuners in over-the-air digital television (“DTV”) set top boxes (“STB”).
Turning to
The controller is also adapted to control the secondary tuner/decoder (14) using commands (23), to receive status indicators and signals (24) from the secondary tuner/decoder (14), and to command (22) the signal selector to route a particular signal source to the secondary tuner to allow the proper monitoring of program transitions, according to the invention. Additional logical functions are preferably implemented and stored in the memory (21), as will be described in more detail in the following paragraphs, including one or more user preferences for control of the invention (27). In a preferred embodiment, the memory comprises at least in part a “writeable” or modifiable type, such as FLASH-ROM, or removable media such as a SIMM card, to allow for feature updates, feature downloads, etc.
Surf-Back Logical Operations
The primary control function (34) represents the traditional control functions of a TV tuner, with certain enhancements and modifications to integrate the secondary control (30) function. The primary control function is employed to allow the user to:
Additionally, the primary control function allows the user to be prompted for user surf-back settings using text and/or iconic menus displayed on the TV as commanded by the secondary control function (30).
A secondary audio program (“SAP”), closed captioning for the hearing impaired decoder (“CC”), V-Chip, and clock/timer (35) are employed in some embodiments of the invention to assist with program transition detection, as described in more detail in the following paragraphs.
The secondary control function (30) embodies the logical control processes to configure the secondary tuner to receive and monitor and audio, video, and/or sub-data (e.g. CC) of a selected channel and/or signal source while the user “surfs” using the primary tuner. The secondary control function also configures one or more program transition detectors, preferably according to the user's settings (27), and receives a status or signal in return when a transition has been detected. Additionally, and according to a preferred embodiment, the secondary control function receives a signal from the clock/timer (35), which allows a basic surf-back function (e.g. return automatically in 30 seconds) or a maximum surf time limitation (e.g. return when program transition is detected or within 180 seconds, whichever is earlier).
The surf-back user preference repository (27) is preferably stored in local memory, as well, where one or more user's may store their surf-back options and choices.
According to another aspect of the preferred embodiment, a plurality of program transition detectors are provided to allow the user one or more options for determining when a surf-back operation is to be executed. These detectors may be provided initially with the system (e.g. pre-configured), they may be downloaded via a network (e.g. modem, Internet, broadcast signal, etc.), they may be installed using removable media (e.g. floppy disks, removable memory cards, etc.), or a combination of these methods.
Turning to
Next, the invention receives from the user certain parameters (42) controlling when the surf-back is to be automatically executed. Typically, this would include enabling or selecting one or more of the available detector types, and optionally setting a time limit for automatic return. The user is prompted for these choices using text, audio, icons, or graphics as previously described, and user input may be made through the keypad or remote control. Alternatively, the user may pre-configure one or more options of the user surf-back settings to allow for full or partial indication of the conditions under which the user would like to automatically return to the origin program.
The invention then initializes (43) one or more of the program transition detectors (32), sets a timer, etc., according to the user's parameters (42). This step includes commanding the secondary tuner to tune or decode the channel and source of the origin channel for monitoring by the detectors (32), thereby freeing the user to command the primary tuner/decoder to alternate channels, frequencies, and/or signal sources while “surfing” for alternate programs.
Until the clock/timer expires, or until a detector indicates a program transition has occurred on the origin channel, the invention waits (44) while allowing the user to “surf” other channels. If the user cancels the surf-back function, the timer/clock expires, or a program transition is detected, the primary tuner/decoder is commanded (45) to tune to the original channel, frequency and/or source, thereby effecting the surf-back function and returning the user automatically to the origin program under the conditions he or she specified.
Program Transition Detection
A number of methods and systems for detecting or determining changes in program content are available for embodiment in the present invention, some of which are illustrated in
These are just a few examples of techniques and processes used to detect program transitions, especially in A/V signals and TV signals. The architecture of the present invention facilitates the incorporation of additional and new detectors as they become available, or as the user wishes to upgrade his or her system.
User Preferences
In this example, a number of user profiles (61) are defined, two of which are for user “Jack” and user “Jill” (62), which are names that are preferably set or chosen by each user. One or more detection methods (63) are logically grouped, such as Jack's first method, Method(1), which uses the volume level detection method during football 20 games. Jack's second method, Method(2), uses a timer method for 4 commercials (e.g. 30 seconds), or a detection in volume level change. Also according to a preferred embodiment, a priority (64) can be set for each enabled detection method to resolve or override the status between multiple detection methods. For Jack's Method(2), certain “required” conditions allow that rule to be used or enabled only during his favorite Wednesday evening shows, for example, while placing high priority to returning to the origin program within 4 commercials, or when the volume level returns to normal.
The Nth user in this set of example preferences is “Jill”, who is a news fanatic, and she watches many comedy shows. So, during commercial breaks from her favorite news channel or comedies, she likes to surf other channels, but wants to return if any important news stories are broadcast, or if a comedy returns from break. So, her preferences enable the closed caption text analysis, looking for the words related to breaking news or to broadcast of a comedy show.
It will be recognized by those skilled in the art that these are only example preferences, and that many other combinations and logical rules are possible within the scope of the present invention.
User Notification of Surf-Back Operation
In a typical embodiment of the invention, the primary tuner would be tuned to the origin program automatically without further notice to the user. However, in certain conditions, the user may wish to receive an audible alert tone upon execution of the surf-back function. This can be done by inserting (502) into the audible sound of the TV a tone, sound, or verbal message, by the secondary control function, especially with a higher volume level than that of the origin or main program. For example, the user can activate the surf-back function, surf away for a while, and then leave the room to get a snack. When the surf-back function executes (e.g. when his main program is back on), the invention emits through the TV speakers a loud “beep-beep-beep”, or verbal message “It's back on!”.
The present invention has been described using examples of specific embodiments. It will be recognized by those skilled in the art that these examples do not represent the extent of the invention, as the invention may be realized an a number of varieties for a number of system and signal types. Therefore, the extent of the invention is determined by the following claims.
Chen, Yen-Fu, Walker, Keith Raymond, Handy-Bosma, John Hans, Sun, Nancy T.
| Patent | Priority | Assignee | Title |
| 10356366, | May 31 2007 | Sony Interactive Entertainment LLC | System and method for taking control of a system during a commercial break |
| 10909161, | Dec 29 2016 | ARRIS ENTERPRISES LLC | System to build advertisement database from unreliable sources |
| 10945030, | Mar 30 2018 | Alphonso Inc. | Detection of potential commercial by detection and analysis of transitions in video content |
| 11172164, | May 31 2007 | Sony Interactive Entertainment LLC | System and method for taking control of a system during a commercial break |
| 11405916, | Nov 30 2015 | AT&T Intellectual Property I, L.P. | Method and apparatus for automated signal analysis and reporting among RF receiver devices |
| 11514914, | Feb 08 2019 | JPMORGAN CHASE BANK, N A | Systems and methods for an intelligent virtual assistant for meetings |
| 7773159, | Feb 15 2006 | GLOBALFOUNDRIES Inc | Switching system for signal monitoring and switch-back control |
| 7872692, | Feb 15 2006 | GLOBALFOUNDRIES Inc | Structure for switching system for signal monitoring and switch-back control |
| 8144259, | Feb 21 2006 | Canon Kabushiki Kaisha | Broadcast receiving apparatus and method for minimizing program interruption by an emergency warning signal |
| 8627400, | Jul 19 2007 | Canon Kabushiki Kaisha | Moving image reproducing apparatus and control method of moving image reproducing apparatus |
| 8830404, | Feb 21 2006 | Canon Kabushiki Kaisha | Broadcast receiving apparatus and method for controlling broadcast receiving apparatus |
| 9071852, | Sep 17 2010 | Samsung Electronics Co., Ltd. | Method for providing media-content related information, device, server, and computer-readable storage medium for executing the method |
| 9497512, | Nov 19 2008 | Canon Kabushiki Kaisha | Content player and method of controlling the same |
| Patent | Priority | Assignee | Title |
| 4857999, | Dec 20 1988 | PEAC Media Research, Inc. | Video monitoring system |
| 4918531, | Oct 25 1988 | RCA Licensing Corporation | Commercial message timer |
| 4979047, | Feb 21 1989 | RCA Licensing Corporation | Automatically activated commercial message timer |
| 5151788, | Jan 26 1988 | INTEGRATED CIRCUIT TECHNOLOGIES LTD , PANNELL KERR FORSTER BLDG , P O BOX 261, BRIDGETOWN, BARBADOS | Method and apparatus for identifying and eliminating specific material from video signals |
| 5343251, | May 13 1993 | PARETO PARTNERS LTD | Method and apparatus for classifying patterns of television programs and commercials based on discerning of broadcast audio and video signals |
| 5418621, | Sep 29 1987 | MagnaChip Semiconductor, Ltd | Circuit for detecting TV/radio broadcasting program, auto-tuning to channel selection and controlling VCR tape in recording operation |
| 5668917, | Jul 05 1994 | Apparatus and method for detection of unwanted broadcast information | |
| 5692093, | Jan 08 1993 | SRT, INC | Method and apparatus for eliminating television commercial messages |
| 5696866, | Aug 06 1993 | SRT, INC | Method and apparatus for eliminating television commercial messages |
| 5870151, | Aug 31 1993 | Method and equipment for the detection of undesired video scenes | |
| 5920360, | Jun 07 1996 | HEWLETT-PACKARD DEVELOPMENT COMPANY, L P | Method and system for detecting fade transitions in a video signal |
| 5973723, | Dec 12 1997 | Selective commercial detector and eliminator apparatus and method | |
| 5987210, | Jan 08 1993 | SRT, INC | Method and apparatus for eliminating television commercial messages |
| 5999688, | Jan 08 1993 | SRT, INC | Method and apparatus for controlling a video player to automatically locate a segment of a recorded program |
| 6100941, | Jul 28 1998 | U.S. Philips Corporation | Apparatus and method for locating a commercial disposed within a video data stream |
| 6144401, | May 29 1996 | Rovi Guides, Inc | Television schedule system with access control |
| 6449021, | Nov 30 1998 | Saturn Licensing LLC | Information processing apparatus, information processing method, and distribution media |
| 6469749, | Oct 13 1999 | Koninklijke Philips Electronics N V | Automatic signature-based spotting, learning and extracting of commercials and other video content |
| 6591420, | Aug 25 1999 | Warner Music Group, Inc. | Remote control system for audio and video content |
| 6593976, | Feb 14 2000 | Koninklijke Philips Electronics N V | Automatic return to input source when user-selected content reappears in input source |
| 6606747, | Mar 15 1996 | INDEX SYSTEMS, INC | System and method for grazing television channels from an electronic program guide |
| 6993245, | Nov 18 1999 | Interval Licensing LLC | Iterative, maximally probable, batch-mode commercial detection for audiovisual content |
| 7007234, | Jun 05 2002 | INTERDIGITAL MADISON PATENT HOLDINGS | Method and apparatus for performing a set up operation for a video system |
| 7043746, | Jan 06 2003 | Sovereign Peak Ventures, LLC | System and method for re-assuring delivery of television advertisements non-intrusively in real-time broadcast and time shift recording |
| 7089575, | Sep 04 2001 | Koninklijke Philips Electronics N.V. | Method of using transcript information to identify and learn commercial portions of a program |
| 7161637, | Nov 30 2001 | Bellsouth Intellectual Property Corporation | Program restart and commercial ending notification method and system |
| 7167632, | Mar 15 2000 | INTERDIGITAL MADISON PATENT HOLDINGS | Process for displaying broadcast and recorded transmissions possessing a common characteristic and associated receiver |
| 7170566, | Dec 21 2001 | UNILOC 2017 LLC | Family histogram based techniques for detection of commercials and other video content |
| 7190415, | Sep 06 2001 | Fujitsu Ten Limited | Digital broadcasting receiver |
| 7266701, | Sep 06 2001 | Universal Electronics, Inc. | System and method for enabling a remote control to automatically and dynamically set-up a V-chip |
| 7266832, | Jun 14 2001 | ARRIS ENTERPRISES LLC | Advertisement swapping using an aggregator for an interactive television system |
| 7280737, | Feb 23 2004 | WARNER BROS ENTERTAINMENT INC | Method and apparatus for discouraging commercial skipping |
| 7302160, | Jan 22 2002 | AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE LIMITED | Audio/video recorder with automatic commercial advancement prevention |
| 20030066077, | |||
| 20050240967, | |||
| 20080052739, | |||
| JP2143708, |
| Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
| Apr 15 2005 | WALKER, KEITH RAYMOND | International Business Machines Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 016219 | /0079 | |
| Apr 18 2005 | CHEN, YEN-FU | International Business Machines Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 016219 | /0079 | |
| Apr 18 2005 | SUN, NANCY T | International Business Machines Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 016219 | /0079 | |
| Apr 19 2005 | HANDY-BOSMA, JOHN HANS | International Business Machines Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 016219 | /0079 | |
| Apr 26 2005 | International Business Machines Corporation | (assignment on the face of the patent) | / | |||
| Sep 30 2019 | International Business Machines Corporation | DAEDALUS GROUP LLC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 051032 | /0784 | |
| Dec 30 2019 | International Business Machines Corporation | DAEDALUS GROUP, LLC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 051710 | /0445 | |
| Jan 28 2020 | DAEDALUS GROUP, LLC | Daedalus Blue LLC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 051737 | /0191 | |
| Nov 29 2021 | Daedalus Blue LLC | TERRACE LICENSING LLC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 058895 | /0322 | |
| Dec 20 2023 | TERRACE LICENSING LLC | Daedalus Blue LLC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 066158 | /0772 |
| Date | Maintenance Fee Events |
| Aug 28 2008 | ASPN: Payor Number Assigned. |
| Feb 27 2012 | REM: Maintenance Fee Reminder Mailed. |
| Jun 21 2012 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
| Jun 21 2012 | M1554: Surcharge for Late Payment, Large Entity. |
| Sep 30 2015 | M1552: Payment of Maintenance Fee, 8th Year, Large Entity. |
| Mar 02 2020 | REM: Maintenance Fee Reminder Mailed. |
| Mar 19 2020 | M1553: Payment of Maintenance Fee, 12th Year, Large Entity. |
| Mar 19 2020 | M1556: 11.5 yr surcharge- late pmt w/in 6 mo, Large Entity. |
| Date | Maintenance Schedule |
| Jul 15 2011 | 4 years fee payment window open |
| Jan 15 2012 | 6 months grace period start (w surcharge) |
| Jul 15 2012 | patent expiry (for year 4) |
| Jul 15 2014 | 2 years to revive unintentionally abandoned end. (for year 4) |
| Jul 15 2015 | 8 years fee payment window open |
| Jan 15 2016 | 6 months grace period start (w surcharge) |
| Jul 15 2016 | patent expiry (for year 8) |
| Jul 15 2018 | 2 years to revive unintentionally abandoned end. (for year 8) |
| Jul 15 2019 | 12 years fee payment window open |
| Jan 15 2020 | 6 months grace period start (w surcharge) |
| Jul 15 2020 | patent expiry (for year 12) |
| Jul 15 2022 | 2 years to revive unintentionally abandoned end. (for year 12) |