Methods and devices generate an exogenous natural second speech signal as an auditory stimulus to a user to enhance the fluency of persons who stutter. The natural speech signal is independent of the contemporaneous speech production of the stutterer and is provided by a voice gesture and can be a prolonged or sustained voice gesture sound such as a simple vowel, or consonant, or vowel trains and the like. The second speech signal can be transmitted in advance of a speaking event or speech production of the stutterer and/or concurrently with a speaking event, either intermittently or continuous during the speaking event. The devices of the instant invention are configured to provide the voice based speech signal such that it is audible to the user and does not require feedback of the user's own speech allowing the user to speak at a substantially normal pace with enhanced fluency. The device and methods can relay the signal based on a manual activation or automatically based on a detection of speech or a stuttering event on the part of the user.
|
59. A product for enhancing the fluency of a person who stutters, comprising an audio storage medium comprising an exogenously generated speech signal including at least one prolonged voice gesture sound having a duration of between about 2 seconds to 2 minutes and generated by a person other than the person using the product to enhance their speaking fluency, wherein in operation, said exogenous speech signal is adapted to be relayed to a user as an auditory stimulus that is output intermittently to the user to enhance the fluency of the user who stutters.
1. A method for enhancing the fluency of persons who stutter, comprising:
exogenously generating a speech signal of at least one prolonged voice gesture; producing speech defining a first speech signal corresponding to the patient speaking, the patient having a propensity to stutter during speech production; and intermittently delivering the exogenously generated speech signal to the patient temporally proximate to said producing step such that the exogenously generated speech signal is audible thereto to thereby enhance the fluency of the patient.
31. A device to enhance the fluency of persons who stutter, comprising:
an audio storage medium comprising at least one pre-determined exogenously generated auditory stimulus speech signal thereon, wherein the at least one predetermined auditory stimulus signal comprises at least one prolonged voice gesture; a speaker operably associated with said audio storage medium; a power source in communication with said audio storage medium and said speaker; and an activation switch operably associated with said power source; wherein said auditory stimulus speech signal is configured to be repeatedly output to a user at desired times corresponding to at least one of during an episodic stuttering event on the part of the user, in advance of the production of speech by the user, and intermittently during the production of speech of the user to thereby provide an auditory stimulus to the user who stutters to enhance the fluency of speech thereof.
2. A method according to
3. A method according to
4. A method according to
5. A method according to
6. A method according to
7. A method according to
8. A method according to
9. A method according to
10. A method according to
11. A method according to
12. A method according to
13. A method according to
14. A method according to
15. A method according to
16. A method according to
17. A method according to
18. A method according to
19. A method according to
20. A method according to
21. A method according to
22. A method according to
23. A method according to
24. A method according to
25. A method according to
26. A method according to
27. A method according to
28. A method according to
29. A method according to
30. A method according to
32. A device according to
33. A device according to
34. A device according to
35. A device according to
36. A device according to
37. A device according to
38. A device according to
39. A device according to
40. A device according to
41. A device according to
42. A device according to
43. A device according to
44. A device according to
45. A device according to
46. A device according to
47. A device according to
48. A device according to
49. A device according to
50. A device according to
51. A device according to
52. A device according to
53. A device according to
56. A device according to
57. A device according to
58. A device according to
60. A product according to
61. A product according to
62. A product according to
63. A product according to
64. A product according to
65. A product according to
|
The present invention relates to devices and methods for enhancing the fluency of persons who stutter.
Conventionally, stuttering has been treated by several different types of treatment, including psychiatric therapy, drug therapy, and the use of altered auditory feedback, generated by electrical signal processing devices, relayed to the person who stutters. These techniques can be generally characterized as either endogenous alterations of the speech signal output, such as prolonged or slowed speech, rhythmic speech, signing, and lipped speech, or exogenous dynamic alterations of the speech signal itself, both of which can successfully induce relatively fluent speech in people who stutter. See, e.g., O. Bloodstein, A Handbook on Stuttering (5th ed. Singular, San Diego, Calif., 1995).
It is believed that exogenous auditory alterations of speech, such as chorus reading, shadow speech, delayed auditory feedback, and frequency altered feedback, or a visual modality of treatment, such as visual choral speech, can generally produce more powerful and natural sounding reductions in stuttering than incongruous non-speech auditory inputs, such as masking noise and clicking, or visual inputs, such as flashing lights.
Two types of altered auditory feedback which have been used to treat stuttering include delayed auditory feedback ("DAF") and the introduction of a masking noise or masked auditory feedback ("MAF"). Generally described, DAF imposes a delay on the delivery of a feedback speech signal to a speaker/stutterer, while MAF serves to compete with a speaker's auditory feedback.
For example, M. E. Wingate, in Stuttering: theory and treatment, p. 237 (Irvington, 1976), describes a type of altered auditory feedback which can include DAF to provide emphasis on phonation, i.e., slowing speech down to extend syllable duration. However, this type of auditory feedback or fluency enhancement is conventionally thought to be achievable with or without the use of DAF as long as syllable prolongation was employed. See, e.g., W. H. Perkins, From Psychoanalysis to Discoordination, in H. H. Gregory (Ed.) Controversies about stuttering therapy, pp. 97-127 (University Press, 1979). See also Andrew Stuart et al., Fluent Speech, Fast Articulatory Rate, and Delayed Auditory Feedback: Creating a Crisis for A Scientific Revolution?, 82 Perceptual and Motor Skills, pp. 211-218 (1996).
Generally stated, the reduction in stuttering frequency under speech signal alterations has been attributed to entrained rhythm, distraction, modified vocalization, and rate reduction. Indeed, in the past, slowed speech rates were found to be an important factor in the reduction of stuttering. For example, in W. H. Perkins et al., Phone rate and the effective planning time hypothesis of stuttering, 29 Jnl. Of Speech and Hearing Research, 747-755 (1979), the authors reported that stuttering was virtually eliminated when speakers reduced speech by approximately 75%. However, other reports have found that rate reduction is neither necessary, nor sufficient, for fluency enhancement. See Kalinowski, et al., Stuttering amelioration at various auditory feedback delays and speech rates, European Journal of Disorders of Communication, 31, 259-269 (1996); Stuart et al., Fluent speech, fast articulatory rate, and delayed auditory feedback: Creating a crisis for a scientific revolution?, Perceptual and Motor Skills, 82, 211-218 (1996); MacLeod, et al., Effect of single and combined altered auditory feedback on stuttering frequency at two speech rates, Journal of Communication Disorders, 28, 217-228 (1995); Kalinowski et al., Effect of normal and fast articulatory rates on stuttering frequency, Journal of Fluency Disorders, 20, 293-302 (1995);. Hargrave et al, Effect of frequency altered feedback on stutterers' fluency at two speech rates, Journal of Speech and Hearing Research, 37, 1113-1119 (1994); and Kalinowski et al., The effects of alterations in auditory feedback on stuttering frequency, Language and Speech, 36, 1-16 (1993).
Recently, a portable therapeutic device and related stuttering enhancement treatment methods were described in U.S. Pat. No. 5,961,443 to Rastatter et al., the contents of which are hereby incorporated by reference as if recited in full herein. These devices and methods employ altered auditory feedback (auditory delay and/or frequency shift signals) to be delivered to a stutterer via a portably configured device. Despite the above, there remains a need to provide improved methods and devices for treating stuttering to enhance fluency in an effective easily implemented manner.
These and other objects are satisfied by the present invention by methods and devices which employ a "second" exogenously generated speech signal which is produced by a sound or sounds corresponding to spoken vocal utterances or natural speech (independent of the in situ uttered speech of the speaker/stutterer). The second exogenous speech signal can alternatively be generated by other than spoken speech so as to simulate natural speech sounds (such as generated electronically, mechanically, or electromechanically); these simulated sound(s) should be configured to simulate the voice gestures which trigger the auditory cortex of the speaker. The second speech signal of the instant invention can be used as an alternative to DAF or MAF, which typically manipulates, alters, or interferes or competes with the contemporaneous speech of the speaker himself (or herself). The second speech signal of the instant invention is an auditory stimulus which is a spoken speech signal (that is, a voice gesture associated with a vocal cord of a person). The second speech signal can be either stuttered or fluent, and/or coherent (a string of meaningful sounds forming words) or incoherent (the sound(s) having no understandable or meaningful content).
Preferably, the second speech signal comprises a prolonged uttered or spoken sound associated with a natural voice gesture such as a single syllabic vowel or consonant or a combination of vowels and/or consonants. The second speech signal of the instant invention can be relayed to the user such that it is intermittent, sustained for a determined period of time, or substantially continuous with the speech production of a user/patient undergoing treatment for stuttering.
Preferably, the second or exogenously generated auditory speech signal of the instant invention is generated exogenously by someone other than the speaker or patient/stutterer (or generated by a device which can substantially replicate a vocal tract output in order to trigger the auditory cortex of the speaker, as noted above). It is also preferred that the second speech signal be recorded and stored in advance of use such that it can be conveniently and reliably provided or audibly relayed to the speaker at a desirable time (and repeatable at appropriate times).
In one embodiment, the exogenously generated second speech signal is a spoken prolonged speech sound (such as the last sound in the word "sudden"). It is more preferred that the prolonged speech sound is a steady state single syllabic sound. It is still more preferred that the prolonged speech sound is a vocal tract output associated with producing a steady state vowel sound. The exogenously generated speech signal can be provided at the start of speech of a person or patient prone to stuttering and/or episodically during speech, such as when a person starts to stutter or is experiencing a stuttering event, or even just at intervals during fluent speech to inhibit the onset of a stuttering event.
The second speech signal can be provided as an arrangement of different voice gesture sounds, the output of which can be varied to alter the exogenously generated speech signal auditory stimulus provided to the patient, over time.
In preferred embodiments, the second or exogenously generated speech signal is pre-recorded and relayed to the user at a desired or appropriate times (either as activated by user input or automatically activated upon detection of a stuttering event). The volume and/or duty cycle of the output are preferably variable to allow a user to adjust the output to his or her needs. That is, in one embodiment, the user can increase or decrease the duration or frequency of the transmitted second speech signal from a continuum ranging from continuously outputting the signal during speech production or a desired output time period to intermittently outputting the signal at desired adjustable intervals during the desired output period.
The second speech signal can be held in and delivered by portable miniaturized devices such as ITE (in the ear), BTE (behind the ear) or OTE (over the ear) stuttering aid devices. Alternatively, the second speech signal auditory stimulus can be generated from stand-alone handheld devices with speakers (or provided as an audio medium such as a compact disk or tape, or downloadable computer code, or other computer readable program formats) or incorporated into communication devices having voice or microphone inputs (such as the handset or base of a telephone or wireless telephone body, two way headsets, and the like) or other devices such as writing implements and the like. In other embodiments, the second speech signal can be held in or incorporated into an audio chip or DSP incorporated into (wrist) watches, bracelets, lapel pins, necklaces or other proximately worn (within the audible range of the user) jewelry such as necklaces and earrings, or headbands, hats, and the like.
One aspect of the invention is a method for enhancing the fluency of persons who stutter, comprising the steps of (a) exogenously generating a speech signal (independent of the contemporaneous speech production of a patient); (b) producing speech by the patient having a propensity to stutter; and (c) delivering the exogenously generated speech signal to the patient temporally proximate to the producing step such that the exogenous speech signal is audible thereto.
In a preferred embodiment, the exogenously generated speech signal is stored or pre-recorded to be repeatedly played back and/or audibly transmitted to the patient at desired intervals or at appropriate times. It is also preferred that the exogenous or second speech signal be generated by a person other than the patient.
Another aspect of the present invention is directed to a device to enhance the fluency of persons who stutter. The device comprises an audio storage medium comprising at least one pre-recorded auditory stimulus speech signal thereon and a speaker operably associated with the audio storage medium to output the speech signal therefrom. The device also includes a power source in communication with the audio storage medium and speaker and an activation switch operably associated with the power source. The device is configured such that the auditory stimulus or second speech signal can be repeatedly output to a user at desired times corresponding to at least one of during an episodic stuttering event; in advance of a speaking event (the production of speech on the part of the user); and during a speaking event to thereby provide an auditory stimulus to the user/person who stutters to enhance the fluency of speech thereof.
In a preferred embodiment, the device includes a user input trigger switch operably associated with the speaker. The user input trigger switch is configured to accept user input to initiate a substantially immediate delivery of the auditory stimulus (second speech signal) such that it is audible to the user. The device can also include an intermittent output switch or button that can allow a user to determine the length, or repeating cycle of the transmitted output signal (to allow the user to vary the auditory stimulus). Similarly, the device can include a selectable signal button to allow the user to select which signal will be transmitted or to vary the output signal automatically over desired time periods.
In one embodiment, the device further includes a microphone and a signal processor configured to receive a signal generated by the user's speech. In this embodiment, the device can then automatically output the auditory stimulus speech signal to the user based on an analysis of a received signal associated with the user's speech, such that the auditory stimulus speech signal is provided substantially contemporaneously with the user's speech independent of (without) auditory feedback or manipulation of the user's contemporaneous speech itself. Advantageously, the auditory stimulus speech signal is delivered in a manner which allows the user to speak at a substantially normal speech pace.
The device can also be configured to identify the initiation of speech production on the part of the user and the termination of speech by the user by monitoring the signal received by the microphone and signal processor. The device can substantially continuously or intermittently output the auditory stimulus speech signal while the user is speaking (such as concurrent with or during the speech of the user).
In one embodiment, the device can also include a detector operably associated with the processor and receiver (microphone). The detector is configured to detect the onset of or an actual stuttering event, and, in operation, upon recognition of the initiation of an impending or actual stuttering event on the part of the user, the device can output the auditory stimulus speech signal to the user.
As noted above, the auditory stimulus speech signal can comprise a plurality of different natural speech prolonged sounds associated with voice gestures which are independent of the contemporaneous speech of the user and can be configured to be serially output to the user.
Advantageously, the exogenously generated or second spoken speech signal is a vocal communication, utterance, or speech sound(s) which is incongruent with the speech production of the stutterer/user. The present invention, thus, provides an auditory stimulus which can be an effective acoustic mechanism to enhance the fluency in persons who stutter while also allowing users to speak at a substantially normal pace and without requiring the use of DAF or MAF. The second stimulus speech signal can be meaningful or not meaningful and can be presented in incongruent text or spoken speech at normal or stuttered fluency or in steady state spoken speech signals having appropriate duration or prolonged or sustained voice gesture sounds.
The foregoing and other objects and aspects of the present invention are explained in detail in the specification set forth below.
The present invention will now be described more fully hereinafter with reference to the accompanying figures, in which preferred embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein.
Like numbers refer to like elements throughout. In the figures, layers, regions, or components may be exaggerated for clarity.
As shown in
The exogenously generated speech signals of the present invention can be thought of as "second" speech signals, wherein the first speech signal is typically associated with the actual speech of the speaker. The instant invention, unlike many conventional stuttering devices and treatments, uses the second exogenously generated speech signal as an auditory stimulus. That is, the second speech signal is a natural or spoken speech signal (a voice gesture associated with a vocal cord) not contemporaneously generated by or associated with the contemporaneous speech of the speaker himself/herself. The second speech signal is also not configured to interrupt (or delay or mask or otherwise feedback) the actual contemporaneously uttered speech of the user. Thus, the second speech signal of the present invention is independent and separate from the contemporaneous speech of the user and is provided as an auditory stimulus to allow the user to speak at a substantially normal pace with enhanced fluency. The second natural speech signal can be coherent or incoherent (i.e., the second exogenously generated natural speech signal can have comprehensible meaning to the user or it can have no meaning to the user, rather, the natural speech signal can be a voice gesture or a collection of voice gestures). In one embodiment, the second speech signal is provided to the patient/user such that it is in the same language as that of the primary language of the user. Alternatively, the second speech signal can be generated by speech spoken in a language which is different from the primary language of the user.
In one embodiment, the method also optionally comprises the step of recording or storing the voice of a person other than the patient to provide the exogenously generated second speech signal (Block 130). The recording or storing of the second speech signal is done in a manner which will allow the second speech signal to be reconstructed or played and transmitted repeatedly to the patient or user at the appropriate or desired times. In this way, the patient has a reliable speaking aid to assist in fluency whenever the need arises.
The second or exogenously generated speech signal can be either stuttered or fluent. The second speech signal can comprise a prolonged voice gesture or vocal spoken sound such as a prolonged single vowel or consonant or a combination of vowels and/or consonants, either alone, or in combination, as will be discussed further below. Further, the exogenous or second speech signal of the instant invention can be provided to the patient in an intermittent manner (such as with a 25-75% duty cycle, or combinations thereof) while the patient or user is speaking (i.e., such that it is intermittent during speech production on the part of the patient/user). Alternatively, the second speech signal can be provided such that the signal is sustained for a period of time, or such that the speech signal is substantially continuously transmitted to the user during speech production. Preferably, the second signal is delivered to the user such that it is either continuous with activation of the device, with the speech production of the user/patient, or with the onset or during a stuttering episode of the user/patient. The second speech signal 10s can also be provided both in advance of (and temporally proximate to) the output of speech as well as substantially continuously or intermittently while the speaker/user is speaking.
As noted above, it is preferred that the second or exogenously generated auditory speech signal is generated by someone other than the user or stutterer. The second speech signal may be able to be generated by a device, such as an elongated tube, which is configured so as to substantially replicate a voice or vocal tract or cord associated with the voice gesture sound of a person, so that, in operation, the replicated voiced speech signal can trigger the auditory cortex of the stutterer/user. Of course, the stutterer can record the appropriate (pre-determined and incongruous) prolonged second speech signal(s) in advance of use for later playback for use as the second speech signal. However, it may be more economical to "burn" or record large quantities of standardized second speech signals suitable for a wide audience. Thus, it is also preferred that the voiced base speech signal of the instant invention be generated and saved (recorded, "burned", and/or stored) in advance of use such that it can be conveniently and reliably played or output at desirable times.
It is also preferred that the exogenously generated second speech signal of the present invention is generated to include a prolonged spoken voice gesture (emphasizing a selected spoken sound). It is more preferred that the second speech signal include at least one spoken prolonged syllabic sound (such as the last sound in the word "sudden") or a sonorant or continuant sound. As used herein the term "prolonged" means to emphasize or sustain the voice gesture sound over normal speech patterns, and preferably means to sustain the voice gesture in substantially steady state form for about at least 2-30 seconds. It is even more preferred that the second speech signal includes a spoken simple sustained or steady state vowel in whatever appropriate language (whether a Romance language or other human spoken language). For example, in the English language, a simple sustained /a/, /i/, /e/, /o/, /u/, and /y/.
In another embodiment, the exogenously voiced speech signal includes trains of vowels such as a three-vowel train. For example, in the English language, a three vowel train representing the three corner of the vowel triangle /a-i-u/ or other vowel trains or serially uttered sustained vowel sounds. Similarly, the second speech signal can include consonant trains or serially uttered (preferably prolonged or sustained) consonant and/or vowels or combinations thereof or sonorant or continuant sounds. Preferably, the second speech signal is delivered to the user or stutterer such that it has a sustained duration of at least between about 5 seconds 2 minutes. More preferably, the second speech signal is transmitted such that it has a duration which is at least about 5-10 seconds and provided, as needed or desired, every 10-30 seconds to every 1-2 minutes (which can be repeated at the same time intervals or can be intermittently transmitted closer and further apart in time) during ongoing speech production such that the signal is relayed to the user intermittently throughout the speech production on the part of the user. It should also be noted that the second speech signal can be recorded as a single short signal (such as about a 1-5 second signal) which can then be looped to provide a longer length output second speech signal. For example, an exogenously generated speech signal having a 1 second (in duration) length can be electronically (such as by digital or analog means) looped 10 times to output a 10 second signal to the user.
The output or transmission of the second speech signal can be varied and/or timed or controlled by a timer incorporated into the device which times the transmission output of the second signal (such as based on the activation of the device or from the initially transmitted or output second speech signal). However, as noted above, the second speech signal can be otherwise provided such as substantially continuously (typically substantially overlapping with the duration of the speech production itself) or intermittently throughout (or provided as needed or desired during or proximate to) speech production of the user or patient responsive to the needs of the user. As such, the exogenously generated speech signal of the present invention can be provided just prior to or at the start of speech production of a speaker prone to stuttering and/or episodically during speech, such as when a person starts to stutter or is experiencing a stuttering event (either of which can be provided in several ways such as via a user input or activation button on the device). The device can also have a selectable duty cycle or timing function input to allow a user to select or vary the desired duration or output transmission cycle (not shown).
In one embodiment, the second speech signal can be provided as an arrangement of different spoken or voice gesture sounds to alter the exogenous voiced speech stimulus to the user, over time. For example, the enhanced fluency treatment can be performed by providing a first exogenous speech signal comprising a sustained steady state /a/ voice gesture sound (preferably relayed to the user proximate in time to either the start of speech production or for a first stuttering event) followed by a second different exogenous speech signal comprising a sustained /e/ (preferably for a subsequent stuttering event or perhaps a second speaking event or speech production temporally spaced apart from the start of the first speech production event or for a different speaking period), followed by the first exogenous signal (repeated) or a third different exogenous signal such as another sustained substantially steady state vowel or vowel train or a sustained consonant, and the like.
The methods and devices of the present invention may also provide exogenously generated second speech signals with a mixture of selectable natural speech signals, some of which may provide improved results for a particular type of stuttering disorder or for particular users as well as for other communicative disorders. For example, one may record the exogenously generated or second speech signals onto a compact disk (or tape) having multiple sound tracks, each providing a different second speech signal (different spoken utterances or voice gestures) relative to the others. Alternatively, a changeable storage medium such as an audio chip or DSP unit, and the like, can be used to provide selectable or changeable second speech signal and thus, selectable or changeable auditory stimulus.
Turning again to
The speech signal 10s can be captured and held by any number of suitable speech signal storage media 20, including, as non-limiting examples, processor circuits including digital signal processors such as DSP chips, audio cards, sound chips, general purpose computers, compact disks, tapes, computer program products (including those downloadable from an internet site), or other sound recording or audio storage mediums.
As shown in
The output of the low pass filter 72 can be input into a sample and hold circuit 74. As is well known in the art, the sampling rate should exceed twice the cutoff frequency of the low pass filter 72 to reduce the likelihood of introducing sampling errors. The sampled signals output by the sample and hold circuit 74 are then input into the A/D converter 76. The digital signal stream representing a desired sampling of data sufficient to allow the device 10" to determine that the user has commenced or terminated speech production is then fed into a controller 30' which is configured to analyze the digital stream to determine whether speech production has been initiated, or terminated or is continuing.
As shown, the controller 30' is in communication with the power source 27 and the speaker 25. In this embodiment, the device 10" also includes a speech signal chip 82 which stores the recorded audio second speech signal 10s. Of course, the controller 30' can be a DSP or other signal processor which can itself hold or store the audio speech signal therein. That is, the speech signal chip 82 does not need to be a separate component, but is merely illustrated as such for ease of description in the figures. The device 10" can also include an adjustable gain amplifier 86 to adjust the output of the signal 10s to a desired comfortable listening level.
During operation, the controller 30' analyzes the digital stream associated with the input signal from the receiver 70 to determine if the user has initiated speech (typically indicated by the analog or digital voice signal rising above a predetermined threshold level). If so, the controller 30' can proceed to automatically power the speaker 25 and output the speech signal 10s to the speaker 25. The controller 30' can continue to monitor samples of the digital stream to determine if speech is continuing to thereby continue to activate the speech signal. As noted above, the speech signal can be output intermittently during speech or substantially continuously with speech. Once the controller 30' determines that speech has terminated, the speech signal 10s can also be automatically terminated.
As is also shown in
In one embodiment, the device 10" can include a stuttering detector circuit 50. This detector circuit 50 is associated with the controller 30' and the digital data stream corresponding to the user's speech. The detector circuit 50 is configured such that during operation it identifies an irregular speech production pattern which can cause the controller 30' to immediately transmit the speech signal 10s to the user to enhance fluency. The device 10" may also increase the volume of the signal if a second speech signal is already being transmitted to the user, or may vary the speech signal transmitted to the user to a different second speech signal, as described above. Typical irregular speech patterns can be identified by prolongation of sounds (corresponding to part word or word prolongation), repetition of sounds (corresponding to part-word or word repetitions), and the like. Although shown as a separate circuit from the controller 30', the detector circuit 50 can also be incorporated into the controller 30' itself (as hardware, software or a combination of same). Examples of suitable means for identifying stuttering events are described in the following references: Howell et al., Development of a two-stage procedure for the automatic recognition of dysfluencies in the speech of children who stutter: II. ANN recognition of repetitions and prolongations with supplied word segment markers, Journal of Speech, Language, & Hearing Research. 40(5):1085-96, (October, 1997); Howell et al., Development of a two-stage procedure for the automatic recognition of dysfluencies in the speech of children who stutter: I. Psychometric procedures appropriate for selection of training material for lexical dysfluency classifiers, Journal of Speech, Language, & Hearing Research, 40(5):1073-84, (October, 1997); Howell, et al, Automatic recognition of repetitions and prolongations in stuttered speech, C. W. Starkweather and H. F. M. Peters (Eds), Proceedings of the First World Congress on Fluency Disorders, Vol. II (pp. 372-374), Nijmegen, The Netherlands: University Press Nijmegen. (1995); and Howell et al., Automatic stuttering frequency counts, W. Hulstijn, H. Peters and P. Van Lieshout (Eds.), Speech Production: Motor Control, Brain Research and Fluency Disorders, Amsterdam: Elsevier Science, 395-404 (1997). The contents of these references are hereby incorporated by reference as if recited in full herein.
As discussed above, the second speech signal can be held in and delivered by portable miniaturized devices such as ITE (in the ear), BTE (behind the ear) or OTE (over the ear) stuttering aid devices such as shown in
Alternatively, the auditory speech based stimulus of the instant invention can provided in a number of ways. In some embodiments, the audio stimulus can be generated from standalone handheld or wearable devices or provided as a compact disk (
Recently, consumer electronics companies have proposed wearable devices (featuring a body area network) on a jacket. This device also includes a headset which can allow a user to listen to a phone call and music using the same headphone or headset and is configured to allow a user to switch between the two modes with a remote control switching device. This technology may be suitable to integrate the second speech signal of the present invention into a similar device so as to be output as an alternative to or in addition to the outputs now allowed, music, second speech signal, and listening to a phone call. Thus, the second speech signal can be output from the headset upon activation of the output via a remote control unit in order to relay and output the second speech signal into the headset while the user is listening to a phone call via the same headset. See e.g., New Wired Clothing Comes With Personal Network, cnn.com/2000/TECH/computing/8/18/wired jacket.idg/index.html (posted on Aug. 18, 2000). The content of this document is hereby incorporated by reference as if recited in full herein.
Alternatively, the second speech signal audio-stimulus of the present invention can be incorporated into conventional consumer devices. For example, it is anticipated that the audio natural speech signal stimulus of the present invention can be incorporated into communication devices having voice or microphone inputs (such as the handset or base of a telephone or wireless telephone body) or other audio-prompter devices which can be easily accessed and used when a user will be typically expected to speak at various times during operation.
In other embodiments, the second speech signal 10s can be held in and provided by (wrist or other type) watches 220 (FIG. 7F), bracelets, lapel or shirt pins, necklaces 230 (
Some embodiments of the devices 10, 10', 10" of the present invention may employ external battery packs while others may employ internal battery power sources. Of course, extension cords and direct power cords and trickle chargers can also be employed. One example of a known BTE hearing aid with DSP and an external battery and processing pack is the PHOENIX produced by NICOLET Company of Madison, Wis.
As will be appreciated by one of skill in the art, the present invention may be embodied as methods, devices or computer executable programs. Accordingly, the present invention may take the form of a hardware embodiment or an embodiment combining software and hardware aspects.
The present invention is also described using flowchart illustrations and block diagrams. It will be understood that each block (of the flowchart illustrations and block diagrams), and combinations of blocks, can be implemented by computer program instructions. These program instructions may be provided to a processor circuit(s) within the mobile user terminal or system, such that the instructions which execute on the processor circuit(s) create means for implementing the functions specified in the block or blocks. The computer program instructions may be executed by the processor circuit(s) to cause a series of operational steps to be performed by the processor circuit(s) to produce a computer implemented process such that the instructions which execute on the processor circuit(s) provide steps for implementing the functions specified in the block or blocks.
Accordingly, the blocks support combinations of means for performing the specified functions, combinations of steps for performing the specified functions and program instruction means for performing the specified functions. It will also be understood that each block, and combinations of blocks, can be implemented by special purpose hardware-based systems which perform the specified functions or steps, or combinations of special purpose hardware and computer instructions.
Exogenous stuttered and normal speech signals were generated and compared for effectiveness. Incongruent speech signals were used in order to compare the inherently incongruent nature of exogenous stuttered speech to that of incongruent fluent speech (in incongruent speech, the second speech signal contains different phonemic material than that read aloud by the participants) to determine if fluency reduction is achieved and what components of the incongruent second speech signal might be responsible for the reduction in stuttering (or the enhancement in fluency). Thus, the natural classification scheme of vowels and consonants were examined in both dynamic and relatively static vocal tract positions. Experiment I involved meaningful speech: normal continuous speech, normal interrupted speech, stuttered continuous speech, and stuttered interrupted speech. Experiment II involved vowels and consonants: /a/, /a-i-u/, /s/, /s-sh-f/.
Ten normal-hearing adults who stutter (8 males, 2 females, mean age 27.9 years, SD 9.4) participated in both experiments. Participants did not present with any other speech and language disorders. All participants had a history of therapy but were currently not receiving any formal therapeutic intervention. Participants read different junior high-level passages of 300 syllables with similar theme and syntactic complexity in both experiments. The two experiments were counterbalanced while the experimental conditions and the passages were randomized. The participants were instructed throughout the experiment to read at a normal rate and not to use any controls to reduce or inhibit stuttering. In both experiments, participants listened to auditory feedback via supra-aural earphones at a comfortable listening level.
The first experiment required participants to listen to incongruous fluent or stuttered speech samples presented continuously or intermittently (50% duty cycle). Both speech samples were incongruent recorded text. The stuttered speech sample contained discrete stuttering acts on all words.
In the second experiment, participants listened to four continuous speech signals: a steady state neutral vowel /a/; a three vowel train representing the three corner of the vowel triangle /a-i-u/; a steady state consonant /s/; and a three consonant train /s-sh-f/. The consonants were selected as these could be presented in the absence of a vowel. Steady vowels and consonants and trains of each were used to represent different levels of proximity with the speech act. Participants also read a control passage with Non-altered Auditory Feedback (NAF). Stuttering episodes were calculated from the participants' videotape recorded passages. Stuttering was defined as part-word repetitions, part-word prolongations, and/or inaudible postural fixations.
The stimuli for these samples were recorded in a sound-treated room with a digital tape recorder (SONY model 8819). A normal fluent American English-speaking adult male produced the vowel, consonant, and fluent speech samples for both experiments. An American English speaking adult male who stutters produced the stuttered speech sample for the first experiment. Both speakers produced speech samples at normal vocal effort. The fluent speech samples used text at junior high level text passages with similar theme and syntactic complexity as those read by the participants of the experiments.
The recorded signals were then fed into a personal computer (Apple Power Macintosh 9600/300) via an APPLE sound input port. Sampling was performed at 44 kHz. Sound analysis software (SOUND EDIT version 2) was used to introduce silence, select the various stuttering moments, and loop the signals. Silent intervals randomly varied from two to five seconds. These were then recorded onto a compact disk that was used to deliver the signal via a compact disk player (SONY model CFD-S28). The signals were delivered binaurally via headphones (OPTIMUS model PRO.50MX) at an audible level comfortable to the participant. All participants spoke into a lapel microphone (RADIOSHACK model 33-3003) affixed at about 15 cm or less from their mouths with an approximate orientation of 0 azimuth and -120 altitude. The microphone output was fed into a video camera (SONY model CCD-TVR 75).
Mean stuttering frequency and standard errors for stuttering frequency as a function of auditory feedback condition for Experiment 1 is shown in
Means and standard errors for stuttering frequency (i.e., the number of stuttering episodes/300 syllables) as a function of auditory feedback for Experiment II are shown in FIG. 9. Error bars represent plus one standard error of the mean. In
This set of experiments provides empirical documentation that an exogenously generated stuttered incongruous voiced or spoken speech signals can induce or increase fluency in persons who stutter. Indeed, the results indicate that stuttering frequency can be reduced irrespective of whether the exogenous signal is based on stuttered or normal speech. Further, the use of an exogenously generated voiced speech signal comprising vowels may provide improved efficacy in enhancing fluency in those who stutter.
In view of the foregoing, it appears that stuttering may be a natural compensatory mechanism to an "involuntary block" at a central level, rather than a peripheral manifested problem. Stated differently, the person stutters in an attempt to generate an auditory release mechanism for an "involuntary block" in speech execution at the central level. The overt manifestations of stuttering are an attempt to compensate at the peripheral level for a loss of control at the central level, albeit via a conspicuous compensation. Thus, stuttering is hypothesized to be a form of compensation rather than a problem in itself. Stuttering can be analogized to the role of a fever in an infectious disease state. The lack of an appropriate fluency enhancing gesture is hypothesized to be the predominate etiological factor that is exhibited or manifested due to a lack of inhibition on the part of the auditory cortex in assimilating the appropriate plan for smooth execution of the speech act. Recent brain imaging procedures have employed choral speech condition to induce fluent speech in adults who stutter and have compared the brain images obtained to those attained during stuttering events/behaviors. See, e.g., Fox et al., A PET Study of the neural systems of stuttering, 382 Nature pp. 158-161 (1996); Wu et al., A positron emission tomograph [18F]deoxyglucose study of developmental stuttering, 6 Neuroreport pp. 501-505 (1995). A lack of activation in the auditory areas during the motor planning of stuttered speech was observed, but an essential normalization under the choral speech condition was noted, indicating fluency enhancing potential.
The foregoing is illustrative of the present invention and is not to be construed as limiting thereof. Although a few exemplary embodiments of this invention have been described, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiments without materially departing from the novel teachings and advantages of this invention. Accordingly, all such modifications are intended to be included within the scope of this invention as defined in the claims. In the claims, means-plus-function clauses, if used, are intended to cover the structures described herein as performing the recited function and not only structural equivalents but also equivalent structures. Therefore, it is to be understood that the foregoing is illustrative of the present invention and is not to be construed as limited to the specific embodiments disclosed, and that modifications to the disclosed embodiments, as well as other embodiments, are intended to be included within the scope of the appended claims. The invention is defined by the following claims, with equivalents of the claims to be included therein.
Rastatter, Michael, Stuart, Andrew, Kalinowski, Joseph
Patent | Priority | Assignee | Title |
10108824, | Jul 22 2010 | VOCOLLECT, Inc. | Method and system for correctly identifying specific RFID tags |
10257603, | Feb 07 2014 | Samsung Electronics Co., Ltd. | Wearable electronic system |
10299025, | Feb 07 2014 | Samsung Electronics Co., Ltd. | Wearable electronic system |
10537291, | Jul 16 2012 | VALCO ACQUISITION LLC | Medical procedure monitoring system |
11020062, | Jul 16 2012 | VALCO ACQUISITION LLC AS DESIGNEE OF WESLEY HOLDINGS, LTD | Medical procedure monitoring system |
11727949, | Aug 12 2019 | Massachusetts Institute of Technology | Methods and apparatus for reducing stuttering |
7031922, | Nov 20 2000 | East Carolina University | Methods and devices for enhancing fluency in persons who stutter employing visual speech gestures |
7292985, | Dec 02 2004 | Janus Development Group | Device and method for reducing stuttering |
7309315, | Sep 06 2002 | BRIGHTSTAR LEARNING LTD | Apparatus, method and computer program product to facilitate ordinary visual perception via an early perceptual-motor extraction of relational information from a light stimuli array to trigger an overall visual-sensory motor integration in a subject |
7421155, | Apr 01 2004 | Kyocera Corporation | Archive of text captures from rendered documents |
7437023, | Aug 18 2004 | Kyocera Corporation | Methods, systems and computer program products for data gathering in a digital and hard copy document environment |
7593605, | Apr 01 2004 | Kyocera Corporation | Data capture from rendered documents using handheld device |
7596269, | Apr 01 2004 | Kyocera Corporation | Triggering actions in response to optically or acoustically capturing keywords from a rendered document |
7599580, | Apr 01 2004 | Kyocera Corporation | Capturing text from rendered documents using supplemental information |
7599844, | Apr 01 2004 | Kyocera Corporation | Content access with handheld document data capture devices |
7606741, | Apr 01 2004 | Kyocera Corporation | Information gathering system and method |
7702624, | Apr 19 2004 | Kyocera Corporation | Processing techniques for visual capture data from a rendered document |
7706611, | Aug 23 2004 | Kyocera Corporation | Method and system for character recognition |
7707039, | Apr 01 2004 | Kyocera Corporation | Automatic modification of web pages |
7742953, | Apr 01 2004 | Kyocera Corporation | Adding information or functionality to a rendered document via association with an electronic counterpart |
7773767, | Feb 06 2006 | VOCOLLECT, INC | Headset terminal with rear stability strap |
7812860, | Apr 19 2004 | Kyocera Corporation | Handheld device for capturing text from both a document printed on paper and a document displayed on a dynamic display device |
7818215, | Apr 19 2004 | Kyocera Corporation | Processing techniques for text capture from a rendered document |
7831912, | Apr 01 2004 | Kyocera Corporation | Publishing techniques for adding value to a rendered document |
7885419, | Feb 06 2006 | VOCOLLECT, INC | Headset terminal with speech functionality |
7990556, | Dec 03 2004 | Kyocera Corporation | Association of a portable scanner with input/output and storage devices |
8005720, | Feb 15 2004 | Kyocera Corporation | Applying scanned information to identify content |
8019648, | Apr 01 2004 | Kyocera Corporation | Search engines and systems with handheld document data capture devices |
8081849, | Dec 03 2004 | Kyocera Corporation | Portable scanning and memory device |
8128422, | Jun 27 2002 | VOCOLLECT, Inc. | Voice-directed portable terminals for wireless communication systems |
8160287, | May 22 2009 | VOCOLLECT, Inc. | Headset with adjustable headband |
8179563, | Aug 23 2004 | Kyocera Corporation | Portable scanning device |
8214387, | Apr 01 2004 | Kyocera Corporation | Document enhancement system and method |
8261094, | Apr 19 2004 | Kyocera Corporation | Secure data gathering from rendered documents |
8275624, | Oct 16 2008 | Electronic speech aid and method for use thereof to treat hypokinetic dysarthria | |
8346620, | Jul 19 2004 | Kyocera Corporation | Automatic modification of web pages |
8386261, | Nov 14 2008 | VOCOLLECT, INC | Training/coaching system for a voice-enabled work environment |
8417185, | Dec 16 2005 | VOCOLLECT, INC | Wireless headset and method for robust voice data communication |
8418055, | Feb 18 2009 | Kyocera Corporation | Identifying a document by performing spectral analysis on the contents of the document |
8438659, | Nov 05 2009 | VOCOLLECT, Inc.; VOCOLLECT, INC | Portable computing device and headset interface |
8442331, | Apr 01 2004 | Kyocera Corporation | Capturing text from rendered documents using supplemental information |
8447066, | Mar 12 2009 | Kyocera Corporation | Performing actions based on capturing information from rendered documents, such as documents under copyright |
8489624, | May 17 2004 | Kyocera Corporation | Processing techniques for text capture from a rendered document |
8505090, | Apr 01 2004 | Kyocera Corporation | Archive of text captures from rendered documents |
8515816, | Apr 01 2004 | Kyocera Corporation | Aggregate analysis of text captures performed by multiple users from rendered documents |
8571873, | Apr 18 2011 | Nuance Communications, Inc | Systems and methods for reconstruction of a smooth speech signal from a stuttered speech signal |
8581700, | Feb 28 2006 | Panasonic Corporation | Wearable device |
8600196, | Sep 08 2006 | Kyocera Corporation | Optical scanners, such as hand-held optical scanners |
8600758, | Apr 18 2011 | Nuance Communications, Inc | Reconstruction of a smooth speech signal from a stuttered speech signal |
8620083, | Dec 03 2004 | Kyocera Corporation | Method and system for character recognition |
8620670, | Mar 14 2012 | International Business Machines Corporation | Automatic realtime speech impairment correction |
8638363, | Feb 18 2009 | Kyocera Corporation | Automatically capturing information, such as capturing information using a document-aware device |
8659397, | Jul 22 2010 | VOCOLLECT, Inc. | Method and system for correctly identifying specific RFID tags |
8682678, | Mar 14 2012 | International Business Machines Corporation | Automatic realtime speech impairment correction |
8713418, | Apr 12 2004 | Kyocera Corporation | Adding value to a rendered document |
8781228, | Apr 01 2004 | Kyocera Corporation | Triggering actions in response to optically or acoustically capturing keywords from a rendered document |
8799099, | May 17 2004 | Kyocera Corporation | Processing techniques for text capture from a rendered document |
8831365, | Apr 01 2004 | Kyocera Corporation | Capturing text from rendered documents using supplement information |
8842849, | Feb 06 2006 | VOCOLLECT, Inc. | Headset terminal with speech functionality |
8874504, | Dec 03 2004 | Kyocera Corporation | Processing techniques for visual capture data from a rendered document |
8892495, | Feb 01 1999 | Blanding Hovenweep, LLC; HOFFBERG FAMILY TRUST 1 | Adaptive pattern recognition based controller apparatus and method and human-interface therefore |
8933791, | Jul 22 2010 | VOCOLLECT, Inc. | Method and system for correctly identifying specific RFID tags |
8953886, | Aug 23 2004 | Kyocera Corporation | Method and system for character recognition |
8990235, | Mar 12 2009 | Kyocera Corporation | Automatically providing content associated with captured information, such as information captured in real-time |
9008447, | Mar 26 2004 | Kyocera Corporation | Method and system for character recognition |
9030699, | Dec 03 2004 | Kyocera Corporation | Association of a portable scanner with input/output and storage devices |
9075779, | Mar 12 2009 | Kyocera Corporation | Performing actions based on capturing information from rendered documents, such as documents under copyright |
9081799, | Dec 04 2009 | GOOGLE LLC | Using gestalt information to identify locations in printed information |
9116890, | Apr 01 2004 | Kyocera Corporation | Triggering actions in response to optically or acoustically capturing keywords from a rendered document |
9143638, | Apr 01 2004 | Kyocera Corporation | Data capture from rendered documents using handheld device |
9268852, | Apr 01 2004 | Kyocera Corporation | Search engines and systems with handheld document data capture devices |
9275051, | Jul 19 2004 | Kyocera Corporation | Automatic modification of web pages |
9323784, | Dec 09 2009 | Kyocera Corporation | Image search using text-based elements within the contents of images |
9381110, | Aug 17 2009 | Purdue Research Foundation | Method and system for training voice patterns |
9449205, | Jul 22 2010 | VOCOLLECT, Inc. | Method and system for correctly identifying specific RFID tags |
9514134, | Apr 01 2004 | Kyocera Corporation | Triggering actions in response to optically or acoustically capturing keywords from a rendered document |
9532897, | Aug 17 2009 | Purdue Research Foundation | Devices that train voice patterns and methods thereof |
9535563, | Feb 01 1999 | Blanding Hovenweep, LLC; HOFFBERG FAMILY TRUST 1 | Internet appliance system and method |
9633013, | Apr 01 2004 | Kyocera Corporation | Triggering actions in response to optically or acoustically capturing keywords from a rendered document |
9788799, | Jul 16 2012 | VALCO ACQUISITION LLC | Medical procedure monitoring system |
9826929, | Jan 18 2012 | AMPLIO LEARNING TECHNOLOGIES HOLDINGS LLC | Method and device for stuttering alleviation |
D567218, | Nov 16 2005 | VOCOLLECT, Inc. | Control panel for a headset |
D567219, | Nov 15 2005 | VOCOLLECT, Inc. | Headset |
D567799, | Nov 15 2005 | VOCOLLECT, Inc. | Headset |
D567806, | Nov 15 2005 | VOCOLLECT, Inc. | Headset |
D605629, | Sep 29 2008 | VOCOLLECT, Inc. | Headset |
D613267, | Sep 29 2008 | VOCOLLECT, Inc. | Headset |
D616419, | Sep 29 2008 | VOCOLLECT, Inc. | Headset |
D626949, | Feb 20 2008 | VOCOLLECT, INC | Body-worn mobile device |
D643013, | Aug 20 2010 | VOCOLLECT, INC | Body-worn mobile device |
D643400, | Aug 19 2010 | VOCOLLECT, INC | Body-worn mobile device |
Patent | Priority | Assignee | Title |
3349179, | |||
3566858, | |||
3773032, | |||
3920903, | |||
4336524, | Jul 17 1979 | MOTOROLA, INC A DE CORP | Video display pager receiver with memory |
4421488, | Mar 10 1980 | Aid for curing or mitigating stammering | |
4464119, | Nov 10 1981 | Method and device for correcting speech | |
4636866, | Dec 24 1982 | Seiko Epson Kabushiki Kaisha | Personal liquid crystal image display |
4685448, | Oct 11 1983 | VOCALTECH, INC | Vocal tactile feedback method and associated apparatus |
4695129, | May 26 1983 | U S PHILIPS CORPORATION A CORP OF DE | Viewer having head mounted display unit for cinerama pictures |
4784115, | Jun 16 1986 | NATIONAL MEDICAL EQUIPMENT, INC | Anti-stuttering device and method |
4856045, | Sep 08 1987 | NEC Corporation | Display terminal for a videophone |
4916441, | Sep 19 1988 | McKesson Information Solutions LLC | Portable handheld terminal |
4934773, | Jul 27 1987 | Microvision, Inc | Miniature video display system |
5003300, | Jul 27 1987 | Microvision, Inc | Head mounted display for miniature video display system |
5048077, | Jul 25 1988 | Microvision, Inc | Telephone handset with full-page visual display |
5106179, | May 17 1990 | Sony Corporation | Eyesight auxiliary liquid crystal device |
5111498, | Sep 25 1987 | Hinged-case sound and vision communications terminal, in particular a video-phone | |
5138312, | Jul 16 1987 | Casio Computer Co., Ltd. | Pager with a television function |
5189632, | Aug 20 1990 | Nokia Mobile Phones LTD | Portable personal computer and mobile telephone device |
5281957, | Nov 14 1984 | Schoolman Scientific Corp. | Portable computer and head mounted display |
5347400, | May 06 1993 | Optical system for virtual reality helmet | |
5478304, | Sep 23 1992 | Anti-sturrering device and method | |
5485318, | Oct 03 1994 | Motorola, Inc. | Dual image manifestation apparatus with integrated electro-optical package |
5485504, | Aug 07 1991 | DRNC HOLDINGS, INC | Hand-held radiotelephone with video transmission and display |
5561538, | Nov 17 1992 | Sharp Kabushiki Kaisha | Direct-view display apparatus |
5596451, | Jan 30 1995 | CITIZEN FINETECH MIYOTA CO , LTD | Miniature image generator including optics arrangement |
5794203, | Mar 22 1994 | Biofeedback system for speech disorders | |
5828427, | Jun 11 1990 | Reveo, Inc | Computer-based image display systems having direct and projection modes of viewing |
5940798, | Dec 31 1997 | Scientific Learning Corporation | Feedback modification for reducing stuttering |
5961443, | Apr 01 1997 | East Carolina University | Therapeutic device to ameliorate stuttering |
6073034, | Oct 31 1996 | Kopin Corporation | Wireless telephone display system |
6231500, | Mar 22 1994 | Electronic anti-stuttering device providing auditory feedback and disfluency-detecting biofeedback |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Sep 12 2000 | KALINOWSKI, JOSEPH | UNIVERSITY, EAST CAROLINA | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 011148 | /0101 | |
Sep 12 2000 | STUART, ANDREW | UNIVERSITY, EAST CAROLINA | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 011148 | /0101 | |
Sep 12 2000 | RASTATTER, MICHAEL | UNIVERSITY, EAST CAROLINA | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 011148 | /0101 | |
Sep 18 2000 | East Carolina University | (assignment on the face of the patent) | / |
Date | Maintenance Fee Events |
Dec 26 2007 | M2551: Payment of Maintenance Fee, 4th Yr, Small Entity. |
Dec 31 2007 | REM: Maintenance Fee Reminder Mailed. |
Dec 22 2011 | M2552: Payment of Maintenance Fee, 8th Yr, Small Entity. |
Dec 22 2015 | M2553: Payment of Maintenance Fee, 12th Yr, Small Entity. |
Date | Maintenance Schedule |
Jun 22 2007 | 4 years fee payment window open |
Dec 22 2007 | 6 months grace period start (w surcharge) |
Jun 22 2008 | patent expiry (for year 4) |
Jun 22 2010 | 2 years to revive unintentionally abandoned end. (for year 4) |
Jun 22 2011 | 8 years fee payment window open |
Dec 22 2011 | 6 months grace period start (w surcharge) |
Jun 22 2012 | patent expiry (for year 8) |
Jun 22 2014 | 2 years to revive unintentionally abandoned end. (for year 8) |
Jun 22 2015 | 12 years fee payment window open |
Dec 22 2015 | 6 months grace period start (w surcharge) |
Jun 22 2016 | patent expiry (for year 12) |
Jun 22 2018 | 2 years to revive unintentionally abandoned end. (for year 12) |