Aspects of the present invention involve novel three-dimensional display systems. Multi-view-window-based display systems are used to generate floating three-dimensional displays. In embodiments, an optical focusing element is positioned at a distance that is greater than or equal to the focal length of the optical focusing element from the viewing windows to form projected viewing windows through which a user can see a floating 3D image. In alternative embodiments, an optical focusing element is positioned at a distance that is less than the focal length of the optical focusing element from the viewing windows to form a rear-projection-like three-dimensional display system. In embodiments, a retro-reflective light diffusion screen is used in the multi-view-window-based display systems.
|
1. A system for generating an aerial three-dimensional image comprising:
a multi-view-window three-dimensional display system that generates a plurality of viewing windows; and
an optical focusing element that is positioned at a distance that is greater than a focal length of the optical focusing element from the plurality of viewing windows of the multi-view-window three-dimensional display system and that receives light from the multi-view-window three-dimensional display system and focuses the light to form a plurality of projected viewing windows,
the plurality of projected viewing windows being positioned such that a user can view an aerial three-dimensional image by viewing a first unique perspective image with a first eye at a first projected viewing window selected from the plurality of projected viewing windows and by viewing a second unique perspective image with a second eye at a second projected viewing window from the plurality of projected viewing windows.
16. A method for generating an aerial three-dimensional display system comprising:
generating a plurality of viewing windows using a multi-view-window three-dimensional display system;
using an optical focusing element that is positioned at a distance that is greater than a focal length of the optical focusing element from the plurality of viewing windows of the multi-view-window three-dimensional display system to receive light from the multi-view-window three-dimensional display system and focus the light to form a plurality of projected viewing windows, the plurality of projected viewing windows being positioned such that a user can view an aerial three-dimensional image by viewing a first unique perspective image with a first eye at a first projected viewing window selected from the plurality of projected viewing windows and by viewing a second unique perspective image with a second eye at a second projected viewing window from the plurality of projected viewing windows.
9. A system for generating an aerial three-dimensional image comprising:
a multi-view-window three-dimensional display system, comprising a plurality of projectors that project a plurality of projected images, that generates a plurality of viewing windows; and
an optical focusing element positioned at a distance that is less than a focal length of the optical focusing element from the plurality of viewing windows of the multi-view-window three-dimensional display system and that receives light from the multi-view-window three-dimensional display system and focuses the light to form a plurality of composite images, each composite image being formed by a plurality of image portions from the plurality of projected images and each composite image being viewable from a unique viewpoint area;
wherein an aerial three-dimensional image is viewable by a viewer seeing a first perspective image with a first eye at a first viewpoint area and seeing a second perspective image with a second eye at a viewpoint area.
21. A method for generating an aerial three-dimensional image comprising:
generating a plurality of viewing windows using a multi-view-window three-dimensional display system that comprises a plurality of projectors that project a plurality of projected images; and
using an optical focusing element positioned at a distance that is less than a focal length of the optical focusing element from the plurality of viewing windows of the multi-view-window three-dimensional display system to receive light from the multi-view-window three-dimensional display system and focus the light to form a plurality of composite images, each composite image being formed by a plurality of image portions from the plurality of projected images and each composite image being viewable from a unique viewpoint area;
wherein an aerial three-dimensional image is viewable by a viewer seeing a first perspective image with a first eye at a first viewpoint area and seeing a second perspective image with a second eye at a viewpoint area.
3. The system of
4. The system of
a screen comprising:
a two-dimensional retro-reflective surface configured to retro-reflect an image that passed through a diffusion layer to form a reflected image; and
the diffusion layer that receives the reflected image from the two-dimensional retro-reflective surface and diffuses the reflected image to form a viewing window corresponding to the image by diffusing the reflected image by a large diffusion angle in a first direction and diffusing the reflected image by a small diffusion angle in a second direction.
5. The system of
a transparent medium, positioned between the two-dimensional retro-reflective surface and the diffusion layer, that allows the image to be out of focus at the two-dimensional retro-reflective surface and wherein at least a portion of the out-of-focus image that was not retro-reflected by the two-dimensional retro-reflective surface is diffused by the diffusion layer.
6. The system of
a plurality of projectors, each projector having a unique position and configured to project an image with a perspective view onto the screen to form a viewing window corresponding to the projected image.
7. The system of
a screen comprising:
a lenticular layer that receives an image and focuses the image onto a light diffuser layer and that receives a diffused reflected image from the light diffuser layer and focuses the diffused reflected image to form a viewing window;
the light diffuser layer, position at the focal plane of the lenticular layer, that receives the image from the lenticular layer and that receives a reflected image from a two-dimensional retro-reflective surface and diffuses the reflected image to form the diffused reflected image; and
the two-dimensional retro-reflective surface that receives the image from the light diffuser layer and retro-reflects at least a portion of the image back to the light diffuser layer to form the reflected image.
8. The system of
a transparent medium, positioned between the two-dimensional retro-reflective surface and the light diffuser layer, that allows the image to be out of focus at the two-dimensional retro-reflective surface and wherein at least a portion of the out-of-focus image that was not retro-reflected by the two-dimensional retro-reflective surface is diffused by the light diffuser layer.
10. The system of
11. The system of
a screen comprising:
a two-dimensional retro-reflective surface configured to retro-reflect an image that passed through a diffusion layer to form a reflected image; and
the diffusion layer that receives the reflected image from the two-dimensional retro-reflective surface and diffuses the reflected image to form a viewing window corresponding to the image by diffusing the reflected image by a large diffusion angle in a first direction and diffusing the reflected image by a small diffusion angle in a second direction.
12. The system of
13. The system of
a transparent medium, positioned between the two-dimensional retro-reflective surface and the diffusion layer, that allows the image to be out of focus at the two-dimensional retro-reflective surface and wherein at least a portion of the out-of-focus image that was not retro-reflected by the two-dimensional retro-reflective surface is diffused by the diffusion layer.
14. The system of
a screen comprising:
a lenticular layer that receives an image and focuses the image onto a light diffuser layer and that receives a diffused reflected image from the light diffuser layer and focuses the diffused reflected image to form a viewing window;
the light diffuser layer, positioned at the focal plane of the lenticular layer, that receives the image from the lenticular layer and that receives a reflected image from a two-dimensional retro-reflective surface and diffuses the reflected image to form the diffused reflected image; and
the two-dimensional retro-reflective surface that receives the image from the light diffuser layer and retro-reflects at least a portion of the image back to the light diffuser layer to form the reflected image.
15. The system of
a transparent medium, positioned between the two-dimensional retro-reflective surface and the light diffuser layer, that allows the image to be out of focus at the two-dimensional retro-reflective surface and wherein at least a portion of the out-of-focus image that was not retro-reflected by the two-dimensional retro-reflective surface is diffused by the light diffuser layer.
18. The method of
a screen comprising:
a two-dimensional retro-reflective surface configured to retro-reflect an image that passed through a diffusion layer to form a reflected image; and
the diffusion layer that receives the reflected image from the two-dimensional retro-reflective surface and diffuses the reflected image to form a viewing window corresponding to the image by diffusing the reflected image by a large diffusion angle in a first direction and diffusing the reflected image by a small diffusion angle in a second direction.
19. The method of
a transparent medium, positioned between the two-dimensional retro-reflective surface and the diffusion layer, that allows the image to be out of focus at the two-dimensional retro-reflective surface and wherein at least a portion of the out-of-focus image that was not retro-reflected by the two-dimensional retro-reflective surface is diffused by the diffusion layer.
20. The method of
a screen comprising:
a lenticular layer that receives an image and focuses the image onto a light diffuser layer and that receives a diffused reflected image from the light diffuser layer and focuses the diffused reflected image to form a viewing window;
the light diffuser layer, positioned at the focal plane of the lenticular layer, that receives the image from the lenticular layer and that receives a reflected image from a two-dimensional retro-reflective surface and diffuses the reflected image to form the diffused reflected image; and
the two-dimensional retro-reflective surface that receives the image from the light diffuser layer and retro-reflects at least a portion of the image back to the light diffuser layer to form the reflected image.
22. The method of
23. The method of
a screen comprising:
a two-dimensional retro-reflective surface configured to retro-reflect an image that passed through a diffusion layer to form a reflected image; and
the diffusion layer that receives the reflected image from the two-dimensional retro-reflective surface and diffuses the reflected image to form a viewing window corresponding to the image by diffusing the reflected image by a large diffusion angle in a first direction and diffusing the reflected image by a small diffusion angle in a second direction.
24. The method of
|
This application is a continuation-in-part of and claims the benefit of co-pending and commonly assigned U.S. patent application Ser. No. 12/607,840, entitled “RETRO-REFLECTIVE LIGHT DIFFUSING DISPLAY SYSTEMS,” which was filed on Oct. 28, 2009, which is a continuation-in-part of and claims the benefit of co-pending and commonly assigned U.S. patent application Ser. No. 12/418,137, entitled “RETRO-REFLECTIVE LIGHT DIFFUSING DISPLAY SYSTEMS,” which was filed on Apr. 3, 2009. Each of the aforementioned applications is herein incorporated by reference in its entirety.
A. Technical Field
The present invention pertains generally to displays, and relates more particularly to autostereoscopic three-dimensional (3D) displays.
B. Background of the Invention
Three-dimensional movies and television are becoming increasingly more popular. With advances in technology, such as high-definition (HD) television, consumers desire more and better features. According to the “2008 3D Television Report” released in May 2008 by Insight Media, 3D may soon be an add-on feature of high-definition television. Many display manufacturers are developing their own 3D display technologies to meet these market demands.
In the current 3D market, traditional standard two-view stereo remains as the dominant implementation. For example, head mounted displays are widely used in the military training and research community, glass-based projection displays play a key role in the large form 3D display, such as the CAVED system and PowerWall systems of Mechdyne Corporation, as well as 3D cinemas. However, the requirement of wearing a helmet or glasses limits the use of the 3D technologies. As an alternative solution, autostereoscopic display technologies have attracted increasing attention. Autostereoscopic displays use special light directing devices to create separate viewing windows in the user's space, which allow the user to see 3D images without glasses. Since the designated viewing windows form a viewing space which is significantly larger than the size of the human eye, users can move their heads freely as long as their eyes are within the viewing space.
Current stereoscopic methods used to produce the viewing windows include parallel-barrier-based displays and lenticular-based displays. However, these autostereoscopic display technologies have significant limitation.
For example, parallel-barrier-based displays suffer from several limitations. First, because parallel-barrier displays use light blocking to produce viewing windows, only a small amount of light emitted from each pixel passes through the barrier window. Second, crosstalk between views can be significant. Crosstalk refers to the overlap of viewing areas, which results when one eye sees the image intended for the other eye. When the crosstalk is significant, the brain cannot perceive the stereo effect or cannot perceive it correctly. Third, the use of small apertures in parallel-barrier-based displays can cause diffraction. This problem becomes more acute as the display resolution increases. As the display resolution increases, the barrier aperture size must be decreased, which causes more severe diffraction effects. Fourth, parallel-barrier-based displays typically suffer from limited resolution. For a display with n views, the resolution of the individual view is essentially 1/n of the original display resolution. Because the views have to divide the resolution of the original display, a parallel-barrier-based display's resolution is limited by the original resolution of the display, which is also limited by diffraction as well as the display manufacturer's capability. Fifth, since each view only sees one pixel column out of n associated with one barrier window, there are many dark pixels lines in each view, which creates a “picket fence effect” in the monocular image. Finally, parallel-barrier-based displays typically suffer from having a limited number of viewing windows. In order to generate more viewing windows, the dark slits have to be wider while the slit windows remain unchanged. Obviously, it is impossible to infinitely increase the number of viewing windows without aggregating the artifacts such as reduced brightness and picket fence effect.
Although lenticular-based displays offer some improvements over parallel-barrier-based displays, the use of lenticular sheets also has important drawbacks. Lenticular-based displays offer higher resolution compared with barrier slits; however, it is more difficult and costly to make high quality lenticular sheets than to make simple black-white barriers. In fact, the quality of the display is directly related to the quality of the lenticular sheet used in the display. Aligning a lenticular sheet with a display also requires significant effort. Furthermore, lenticular-based displays also suffer from problems that plague parallel-barrier-based displays, such as crosstalk between view windows, dark line problem, limited resolution, and limit number of viewing windows.
Accordingly, what is needed are systems and methods that provide better displays, particularly better displays that can be used for autostereoscopic displays.
Aspects of the present invention involve the use of light diffusion and retro-reflectivity to generate novel display screens. In embodiments, a retro-reflective light diffusion screen can be used to generate autostereoscopic displays by generating a plurality of viewing windows. In embodiments, each viewing window depicts a perspective image view and a 3D image can be views by viewing one perspective image view from one viewing window with one eye and by viewing another perspective image view from another viewing window with the other eye.
In embodiments, a display screen system comprising a screen that has a two-dimensional retro-reflective surface and a diffusion surface. The diffusion surface is configured with a large diffusion angle in a first direction and a small diffusion angle in a second direction. For the display screen, the first direction is preferably the vertical direction and the second direction is preferably the horizontal direction. The diffusion surface is also configured to receive an image reflected from the two-dimensional retro-reflective surface and to diffuse the image to form a viewing window corresponding to the image.
In embodiments, a display screen system comprises a retro-reflector diffuser screen and at least one additional layer. In embodiments, the additional layer may be a transparent layer between the two-dimensional retro-reflector and the light shaping diffuser. In embodiments, the additional layer may be a lenticular layer that is positioned in front of the light shaping diffuser. In yet other embodiments, the retro-reflector diffuser screen comprises a lenticular layer, a light shaping diffuser, a transparent medium layer, and a two-dimensional retro-reflector.
The display screen system can also include a plurality of projectors. Each projector has a unique position and is configured to project an image with a unique perspective view onto the screen to form a unique viewing window corresponding to the projected image. Thus, the display system forms a plurality of viewing windows corresponding to a plurality of images projected by the plurality of projectors. A user can view a three-dimensional image by viewing a first perspective image with one eye at a first viewing window selected from the plurality of viewing windows and by viewing a second perspective image with the other eye at a second viewing window from the plurality of viewing windows.
In embodiments, the display screen system also comprises a beamsplitter positioned in an optical path between at least one of the plurality of projectors and the screen to direct a projected image to the screen and to direct the projected image reflected from the screen to a location to form a viewing window that is spatially separate from the at least one of the plurality of projectors. Such configurations remove the projector as an obstacle in the viewing window.
In embodiments, the display system also includes a second screen retro-reflective light diffusing screen positioned at an optically mirror-conjugated position relative to the first screen to increase the brightness of the image at a viewing window.
In embodiments, the display system includes a polarization-sensitive beamsplitter positioned in an optical path between at least one of the plurality of projectors and the screen to direct a projected image to the screen and a quarter-wave plate positioned in an optical path between the screen and the beamsplitter.
In embodiments, the display system includes a computing device communicatively coupled to the plurality of projectors to coordinate projection of images. The computing device may also include one or more datastores for storing images to be projected. It should be noted that the images to be projected may be still images, video images, or both.
Embodiments of the present invention also include methods for making an autostereoscopic display system according to teachings presented herein. For example, in embodiments, an autostereoscopic display system can be formed by positioning a retro-reflective light diffusing screen to receive projected images from a plurality of projectors. Each projector has a unique position and is configured to project an image with a unique perspective view onto the screen to form a viewing window corresponding to the projected image. A plurality of viewing windows are formed corresponding to the plurality of images projected by the plurality of projectors. The plurality of viewing windows are positioned such that a user can view a three-dimensional image by viewing a first perspective image with one eye at a first viewing window selected from the plurality of viewing windows and by viewing a second perspective image with another eye at a second viewing window.
Aspects of the present invention also include systems and methods for generating floating three-dimensional displays. In embodiments, an optical focusing element is positioned at a distance that is greater than or equal to the focal length of the optical focusing element from the viewing windows of a multi-view-window-base display system to form projected viewing windows from which a user can see a floating 3D image. In alternative embodiments, an optical focusing element is positioned at a distance that is less than the focal length of the optical focusing element from the viewing windows to form a rear-projection-like/light-field three-dimensional display system. In embodiments of the aerial three-dimensional displays, one of the embodiments of the retro-reflective light diffusion screen presented herein is used in the multi-view-window-based display systems.
Some features and advantages of the invention have been generally described in this summary section; however, additional features, advantages, and embodiments are presented herein or will be apparent to one of ordinary skill in the art in view of the drawings, specification, and claims hereof. Accordingly, it should be understood that the scope of the invention shall not be limited by the particular embodiments disclosed in this summary section.
Reference will be made to embodiments of the invention, examples of which may be illustrated in the accompanying figures. These figures are intended to be illustrative, not limiting, and are also not drawn to scale. Although the invention is generally described in the context of these embodiments, it should be understood that it is not intended to limit the scope of the invention to these particular embodiments.
In the following description, for purposes of explanation, specific details are set forth in order to provide an understanding of the invention. It will be apparent, however, to one skilled in the art that the invention can be practiced without these details. Furthermore, one skilled in the art will recognize that embodiments of the present invention, described below, may be implemented in a variety of ways. Accordingly, the examples described below are illustrative of specific embodiments of the invention and are meant to avoid obscuring the invention.
Components, or modules, shown in block diagrams are illustrative of exemplary embodiments of the invention and are meant to avoid obscuring the invention. It shall also be understood that throughout this discussion that components may be described as separate functional units, which may comprise sub-units, but those skilled in the art will recognize that various components, or portions thereof, may be divided into separate components or may be integrated together, including integrated within a single system or component.
Furthermore, connections between components within the figures are not intended to be limited to direct connections. Rather, data between these components may be modified, re-formatted, or otherwise changed by intermediary components. Also, additional or fewer connections may be used. It shall also be noted that the terms “coupled” or “communicatively coupled” shall be understood to include direct connections, indirect connections through one or more intermediary devices, and wireless connections.
Reference in the specification to “one embodiment,” “preferred embodiment,” “an embodiment,” or “embodiments” means that a particular feature, structure, characteristic, or function described in connection with the embodiment is included in at least one embodiment of the invention and may be in more than one embodiment. The appearances of the phrases “in one embodiment,” “in an embodiment,” or “in embodiments” in various places in the specification are not necessarily all referring to the same embodiment or embodiments. It shall also be understood that the terms “image” or “images” as used herein shall mean still images, video images, or both.
A. Overview
Most stereoscopic viewing techniques rely on binocular parallax cues to generate a three-dimensional (3D) image. Binocular parallax refers to seeing a stereo image pair (i.e., two different perspectives) of the same 3D object or objects. A head mounted display is one of such viewing device, in which the user sees two separate stereo images through two separated optical paths. Anaglyphic glass, shutter glass, and polarized glass are widely used to separate the stereo images displayed on a single screen or printed on the paper.
Autostereoscopic display techniques also rely on parallax depth cues. However, instead of separating the stereo views with glass in front of the user's eyes, most of autostereoscopic displays try to control the light path in display space to generate viewing windows in the user's space. In each viewing window, only one monocular image can be observed. The viewing windows further form viewing zones. Since the size of the viewing zone is significantly larger than the size of a human eye, the user can freely move his/her head within the viewing zone. Most of the existing autostereoscopic display systems are based on either parallel-barrier techniques or lenticular techniques.
1. Parallel-Barrier-Based Display
The fundamental design using a parallel barrier is to place strips to block light emitted from the display. A black-white mask grid is typically placed in front of the display. Each small white grid acts as a pinhole which maps pixels into the viewing space and allows a user to see the pixels from certain viewing angle (i.e., creating a viewing window), while the black grid blocks the neighbor pixels from that viewing angle. As a result, in a left viewing window, the user sees one set of pixels on the display, and in a right viewing window, the user sees the other set of pixels on the display. The left viewing window and the right viewing window together form one viewing zone. More than one viewing zone can be formed. Also, by increasing the resolution of the display and associating more pixels to each barrier slits, multiple viewing windows can be created within a single viewing zone.
As previously noted, parallel-barrier-based displays have several drawbacks. These drawbacks include: reduced brightness, crosstalk between views, diffraction effect caused by a small window, limited resolution, picket fence effect in the monocular image, image flipping artifact when crossing a viewing zone, and a limited number of viewing windows.
First, because parallel-barrier-based displays use a light block strategy, only a small amount of light emitted from each pixel passes through the barrier window. Thus, the brightness of the viewed image is reduced. Attempts to reduce the brightness problem have included placing the barrier mask behind the display. The light is modulated by the barrier before reaching the display pixels.
Second, crosstalk between views can be significant. Crosstalk refers to the overlap of viewing areas, which can result in one eye seeing the image intended for the other eye. When the crosstalk is significant, the brain cannot perceive the stereo effect or cannot perceive it correctly. Attempts to minimize the crosstalk artifact involve choosing a barrier pitch for the barrier mask or placing the barrier behind the display.
Third, the use of small apertures in parallel-barrier-based displays can cause diffraction. This problem becomes more acute as the display resolution increases. As the display resolution increases, the barrier aperture size must be decreased, which causes more severe diffraction effects. Thus, the aperture size of the barrier is limited by the diffraction limit.
Fourth, parallel-barrier-based displays typically suffer from limited resolution. For a display with n views, the resolution of the individual view is essentially 1/n of the original display resolution. Because the views have to divide the resolution of the original display, a parallel-barrier-based display's resolution is limited by the original resolution of the display, which is also limited by diffraction as well as the resolution that can be achieved by the display manufacturer.
Fifth, since each view only sees one pixel column out of n associated with one barrier window, there are many dark pixels lines in each view. For a two-view case, the dark line is a single pixel width, interlacing in these bright pixels. The dark pixel lines create a “picket fence effect” in the monocular image. Attempts to reduce the picket fence artifact involve placing the barrier lines at an angle with respect to the pixel column of the display.
Sixth, these displays can suffer from image flipping artifact. The imaging flipping artifact is caused by the improper alignment of a user's eyes with the viewing windows. In the two-view display systems, a user will see a flipped set of stereo images if the user's left eye is in a right viewing window and the user's right eye is in a left viewing window. For a multiple-view display system, such flipping artifacts happen when a user crosses a viewing zone.
Finally, parallel-barrier-based displays typically suffer from having a limited number of viewing windows. In order to generate more viewing windows, the dark slits have to be wider while the slit windows remain unchanged. It is impossible to increase the number of viewing windows without increasing artifacts, such as reduced brightness and picket fence effect.
2. Lenticular-Based Displays
As noted above, one drawback of parallel-barrier-based displays is low brightness due to the narrow vertical slits of the barrier. One solution to this problem is to use lenses to improve the light gathering. One form of lens that is used is a lenticular sheet, which contains a set of cylindrical lenses placed side-by-side. To use a lenticular sheet for displaying 3D images, the sheet is vertically aligned with a 2D display. Similar to parallel-barrier displays, if aligning two sets of pixels (e.g., left eye pixels and right eye pixels), two viewing windows can be created to form one viewing zone. Multiple viewing zones (with L & R viewing windows) can also be created. If aligning multiple pixels with each lens, multiple viewing windows can be created in one viewing zone. Since the lenses in the lenticular sheet are in a cylindrical shape, only horizontal parallax is created. In addition to an improvement over parallel-barrier-based displays with respect to light-gathering ability, the lenticular sheet also can offer higher resolution comparing with barrier slits.
Although lenticular-based displays have better brightness and resolution possibilities as compared to parallel-barrier-based displays, lenticular-based displays present their own disadvantages. First, the quality of the display is dependent upon having a high quality lenticular sheet. However, it is significantly more difficult and costly, particularly as compared to barrier masks, to manufacture high quality lenticular sheets. Additionally, aligning a lenticular sheet with a display also requires significant effort. Furthermore, lenticular-based displays also suffer from problems that plague parallel-barrier-based displays, such as crosstalk between view windows, picket fence problem, limited resolution, image flipping, and limited number of viewing windows.
3. Light Diffusion Based Displays
A third type of multiple-view 3D display uses light shaping diffuser (LSD) technology. Examples of devices utilizing light shaping diffusers include a light-field display called HoloVizo™ developed by Holografika of Budapest, Hungary and a 3D light-field display developed by the Institute for Creative Technologies at the University of Southern California. The light shaping diffuser is a one-dimensional light diffuser which is used for optical communication device and LCD display backlighting. When illuminated by a projector, the light shaping diffuser has small diffusion in the horizontal direction and large diffusion in the vertical direction
Holografika takes advantage of the one-dimensional diffusion property in its HoloVizo™ display. The display uses a number of projectors that illuminate a LSD screen. In the horizontal cross-section view, a viewer can only see one very thin slit of images from each projector, assuming that the screen diffuses light in the vertical direction only. To generate one viewing perspective, these thin slits from different projectors have to be mosaiced together. Therefore, the display requires many projectors to work together. For the HoloVizo™ display system, as many as 50 projectors are used for the prototype. Assuming a horizontal diffusion angle of 1 degree, it requires about 60 projectors to generate a 60 degree field-of-view multiple-view display.
The light-field display developed by the Institute for Creative Technologies at the University of Southern California consists of a high-speed video projector, a spinning mirror covered by a holographic diffuser, and field-programmable-gate-array circuitry to decode specially rendered Digital Visual Interface (DVI) video signals. The high-speed video projector and a spinning mirror covered by a holographic diffuser generate a 360 degree view. The light-field display developed by the Institute for Creative Technologies at the University of Southern California has similar properties as the HoloVizo™ display.
Although these displays do not share all of the drawbacks of the parallel-barrier-based and lenticular-based displays, there are some inherent problems with these approaches. For example, there can be crosstalk problem, difficult system calibration, and extra-cost for rendering.
First, for the HoloVizo™ display, there can be crosstalk problems. The crosstalk artifact can be introduced by the light diffuser. An ideal diffuser should be a perfect low pass filter (i.e., perfect rectangular). However, the actually material has a Gaussian-like diffusion pattern. As a result, there can be crosstalk between two neighbor thin slits, which could significantly blur the images or generate a zigzag image.
Second, both systems require extremely specialized and complex components and require complex system calibration. For example, since the images in the HoloVizo™ display are mosaiced from image segments of many projectors, these projectors must be properly aligned and calibrated. The light-field display developed by the Institute for Creative Technologies at the University of Southern California also requires specialized equipment and specialized set-up.
Finally, there are extra costs for image rendering. For the HoloVizo™ display, the rendering costs are caused, at least in part, by its special image mosaic requirement. The images displayed on the projectors are not the perspective images of a regular 3D movie or the images rendered by the standard OpenGL software. Thus, extra steps have to be taken to generate proper images for each projector. And, for the light-field display developed by the Institute for Creative Technologies at the University of Southern California, the system requires specially rendered DVI video signals.
B. Embodiments of Autostereoscopic Display
Embodiments of the present invention include using light shape diffusion with retro-reflection. Such configurations have several benefits over prior autostereoscopic displays. First, such systems can use full resolution of the projectors and can display bright images. Systems of the present invention generate each view image with only one projector and do not require mosaicing the views from multiple projectors when the user views images at the designated viewing location. Additionally, the images do not require specially rendered video signals. As a result, the special post-rendering processes mentioned above are not necessary. Compared with parallel-barrier-based and lenticular-based displays, the present systems do not suffer from the same resolution limits, are typically much brighter, and are theoretically able to create a large number of viewing windows. Additional benefits shall be apparent to one skilled in the art.
C. Retro-reflective Vertical Light Diffusion Screen (RRVLD)
In embodiments, an autostereoscopic display of the present invention comprises two layers. The first layer comprises a one-dimensional (1D) light diffusion material that has a small diffusion angle in one direction and a large diffusion angle in the other direction.
As showing in
The second layer of the screen is a retro-reflective material, which reflects a light ray back in the incident direction.
As depicted in
D. Embodiments of Display Systems
1. General Display System Embodiments
W is the width of the slit;
Da is the aperture size of the projection lens;
Zp is the distance from the projector to the screen; and
It shall be noted that an advantage of having a large vertical diffusion angle is that the viewing window is extended. Without an extended viewing window, the viewing window would coincide with the projector lens, thereby making it not possible for an individual to view the reflected image. By extending the viewing window in a vertical direction, a user can view the image in the viewing window either above or below the projector 410.
The display system depicted in
One skilled in the art shall recognize that there are several advantages to a display system of the kind depicted in
Second, the display screen can be configured into different shapes. Due to the retro-reflective property of the material, the screen shape could take arbitrary forms, such as regular planar, cylindrical shape, spherical shape, or almost any irregular shape. These shape variations do not affect the refocusing property of the retro-reflective screen.
Third, the display system is easily scalable. For example, more viewing windows can be generated by simply adding more projectors.
Fourth, the display system does not have the resolution limitations of prior solutions. Even though all the images are projected on the same screen, each image is only seen in the designated viewing window; therefore, the resolution can be as high as the resolution of the projector.
Fifth, the display system does not suffer from the picket fence effect. Because the user perceives one full resolution image from a single projector at each viewing window, there is no picket fence effect in the image.
Sixth, the display system does not suffer from an image flipping effect. The flipping effect occurs when a user moves his head across the viewing zones and perceives a right image in his left eye and a left image in his right eye. The display system does not have repeated viewing zones with specific stereo pair images and therefore does not have image flipping problems. Rather, each viewing window displays a perspective view image and any pair of images forms a 3D view. For example, in embodiments, the viewing windows may have a progression of perspective view images, wherein any two images form a 3D view.
Finally, the display system can potentially have an infinity number of viewing windows. Although theoretically the display system can generate an infinity number of viewing windows, the number of viewing windows that can be generated depends upon the horizontal diffusion angle of the diffusion material, the distance from the projector to the screen and the size of the projector.
2. Compact Design Embodiments
3. Dual Screen Display System Embodiments
Although the embodiment depicted in
4. Polarization Managed Display System Embodiments
It shall be noted that the configurations shown in
5. Additional Layer Embodiments
a) Transparent Layer Embodiments
When an image is projected onto a retro-reflective surface, the vast majority of the light is retro-reflected back to the image source. However, because retro-reflectors are not perfect, some light is diffused or reflected in other directions. This diffused or errantly reflected light can create unwanted images, known as ghost images.
An anti-reflection coating on the reflection surface would reduce the ghost images from the diffusion. However, this solution can be very expensive. As an alternative solution, presented herein is a transparent medium gap between the diffusor and the retro-reflective material. The transparent space allows the ghost image to blur on the retro-reflective material, while keeping the image on the diffuser sharp because of the focusing ability of the retro-reflective material. Furthermore, the front diffusor layer diffuses the already blurred ghost image, which further dims the ghost image. As a result, a user will see a darker and blurred ghost image, if at all.
As discussed above, when the light diffusion material is placed in front of the retro-reflective material, light is reflected back along the incident direction by the retro-reflective layer and is diffused into a fan shape by the light diffusing material. The resulting reflected light rays create a viewing window. Each viewing window displays an image of a projector. Accordingly, multi-projectors combined with the retro-reflective light diffusion screen can produce multiple viewing windows. If those windows display images from different perspectives, the multi-projector and screen system forms a three-dimensional display system. That is, by using the projectors to display a set of images captured from multiple perspectives, a user can see a three-dimensional image via the viewing windows that are formed. However, if there is imperfection in the screen, ghost images may be formed.
In embodiments of the light diffuser and retro-reflective screen system as discussed above, an image is focused on the diffuser and the retro-reflective material. Both the retro-reflected image on the diffuser and the ghost image on the retro-reflective material are sharp. The diffused or misdirected light of the ghost image is in focus. If the ghost image is sufficiently bright, it can be seen in viewing windows and would interfere with a user's ability to perceive the proper image for that view window. When the ghost image is significant, the user may not be able to perceive a stereo view.
In embodiments, to reduce the effects of ghost images, a third layer is introduced between the retro-reflective material and the light shaping diffuser. This layer allows the diffused light to further diffuse, which causes it to be dimmed and significantly blurred while still allowing the retro-reflected image to remain sharp.
In a projector system, the projector is focused on the first layer—the diffuser 910. Therefore, the image on the diffuser is sharp. However, the image on the retro-reflective material 915 is significantly blurred because space exists due to the transparent medium 920 for the image to diffuse before reaching the retro-reflector 915. As illustrated in
Although most of the light will be properly retro-reflected, some light rays (e.g., 940-x) are imperfectly reflected or diffused and travel in various directions. These light rays will be further diffused when passing through the diffuser 910. Therefore, the ghost images will be additionally blurred and darkened after passing through the diffuser 910. For example, light ray 940-1 travels through diffuser 910 and is further diffused by the diffuser 910 to create even more dispersed, and therefore dimmer, light rays 945.
Since the image is focused on the light diffuser 910, after the image is properly diffused by the diffuser 910 and forms a view window 1010 by being diffused vertically but only diffused slightly in the horizontal direction, in the view window 1010, a user will see a sharp image. And, the user will see less, if any, of the greatly diffused ghost images.
It shall be noted that the triple layer screen embodiments may also be utilized in the embodiments discussed above with respect to the dual layer screen.
b) Lenticular Layer Embodiments
A lenticular sheet contains many cylindrical lens (one-dimensional lens) arranged side-by-side. Since the lenses are one dimensional, a lenticular sheet has focusing ability in only one direction. Lenticular sheets have been used in both transparent-type and reflection-type three-dimensional displays. For example, a lenticular reflection screen (LRS) contains two layers: a lenticular sheet and a regular diffusion surface. In the vertical direction, the light is diffused in all the directions, and in the horizontal direction, the light is first diffused and then refocused back to the projector by the one-dimensional lenses.
Although lenticular reflection screens have been used in three-dimensional displays, lenticular reflection screens have significant limitations. When projecting images on a lenticular reflection screen, a set of projector form not only a main viewing zone but also side viewing zones, similar to a transparent-type configuration projection system. If additional projectors are placed outside of the main viewing zone in a side viewing zone, crosstalk between the projected images can occur, which limits the field of view of the lenticular sheet system.
As illustrated in
When multiple projectors 1135 are used to form multiple viewing windows 1140 in the main viewing zone 1120, in each side viewing zone a corresponding set of the viewing windows (e.g., a set of viewing windows 1145L in side viewing zone 1125L and a set of viewing windows 1145R in side viewing zone 1125R) are also formed.
If another projector 1160 is placed inside a side viewing zone 1125L to project images onto the screen 1105, light from the projector 1160 forms a corresponding main viewing window in side viewing zone 1125L and side viewing windows in main viewing zone 1120 and side viewing zone 1125R, and these viewing windows could overlap with the viewing windows formed by the set of projectors 1135. For example, the light from the projector 1160 in the side viewing zone 1125L may interfere with light from the projector 1130 in the set of projectors 1135 in the main viewing zone 1120. Accordingly, crosstalk between the reflected images exists. If the crosstalk is significant, it can impede the ability of a user to perceive stereo images.
Because a projector or set of projectors cannot placed outside of the main viewing zone of a main projector set without creating crosstalk issues, a typical LRS-based system can be very limited because its field of view for the main view area is limited.
Depicted in
Such a screen 1205 as the one depicted in
In addition to the reducing the crosstalk issue related to side viewing zones, the retro-reflective light diffusion screen with the lenticular layer addresses another crosstalk problem due to the diffusion profile of the diffuser.
Because the light follows a Gaussian distribution, there is no well defined energy cutoff boundary. In embodiments, the view window may be defined as the region at which the intensity of the light is above a threshold value or percentage, such as, by way of example and not limitation, 50% of the maximum. Because of the distribution, some energy may leak to the neighbor viewing windows as depicted in
It shall be noted that the retro-reflective light diffusion screen with a lenticular layer may also be utilized in the embodiments discussed above with respect to the dual layer screen.
c) Lenticular Layer and Transparent Layer Embodiments
E. Aerial Three-Dimensional Image Displays
Aspects of the present invention include embodiments that produce aerial displays of images; that is, images that appear to be floating or suspended in the air. The images may be still images or video images. Embodiments of an aerial three-dimensional display include a multi-view-window-based display and one or more optical focusing elements to focus the light from the multi-view-window-based display to a different location.
1. Viewing Window Aerial Three-Dimensional Image Displays
Consider, by way of illustration and not limitation, the embodiment depicted in
The optical focusing element 1620 is positioned relative to the multi-view-window-based display system 1615 such that the multi-view windows 1610 of the display 1615 are focused by the optical element 1620 to another location to form projected viewing windows 1640. A viewer 1645 properly positioned to view the projected viewing windows 1640 will see a three-dimensional object 1630. Since the displayed image 1630 appears to be presented as on a virtual screen 1625, the three-dimensional image 1630 will appear as if floating in air.
As shown in
The relation of the screen 1805 and the virtual screen 1825 is governed by the lens law. The position and size of the virtual screen 1825 can be expressed according to the following equations:
Similarly, the relation of the viewing windows 1810 and the projected viewing windows 1840 is also governed by the lens law.
As shown in
2. Parallel Projected Viewing Windows Aerial Three-Dimensional Image Displays
Consider, by way of illustration and not limitation, the embodiment depicted in
The optical focusing element 2320 is positioned relative to the multi-view-window-based display system such that the viewing windows 2310 of the display system are at the focal plane of the optical element 2320. When the projectors/viewing windows 2310 are placed at the focal plane of the imaging element 2320, the projected viewing windows are located at infinity or may be considered to be effectively at infinity. In this case, all the rays from the same viewing window are parallel with each other. Therefore, on the display, the displayed images are a set of parallel projection images 2340. In embodiments, the screen 2305 position may be the same as discussed with respect to the other embodiments, above.
For viewing window based displays, a viewer sees a pair of stereo images from two viewing windows when he/she aligns his/her eyes with the viewing windows. The observed stereo images are mosaiced images from multiple image sources when he/she stands in front of or behind the viewing windows. For the parallel projection embodiment discussed above, the viewing window is at infinity, which means that a viewer is always in front of the viewing windows. Therefore, he or she will see a pair of mosaiced images from a plurality of the projectors. By viewing a pair of stereo images, one with each eye, a viewer can see a floating three-dimensional image.
3. Light-field Aerial Three-Dimensional Image Displays
Embodiments of a prior section used multi-view-window-based display systems with one or more optical focusing elements to create aerial three-dimensional image display systems; that is, systems the allowed a user to see three-dimensional images as if they were floating. Those systems produced viewing windows, each with a perspective image. When a viewer viewed one image in one viewing window with one eye and another image in another viewing window with his/her other eye, a floating three-dimensional image appeared to the viewer. In alternative embodiments presented in this section, an aerial three-dimensional image display system is formed using a multi-view-window-based display system with one or more optical focusing elements. However, the embodiments depicted in this section are configured to produce a light-field aerial three-dimensional image display, not a viewing window aerial three-dimensional image display. As will be explained in more detail below, the embodiments in this section essentially produce a rear projection system, and as such, a viewer sees a floating three-dimensional image, not by looking a two different viewing windows that each have a perspective image, but rather, by seeing an image that is formed as a mosaic from a set of projectors/viewing windows. Such a rear projection system is sometime also referred to as a “light-field” display system.
This principle of rear projection/light-field display can be applied to embodiments of the present invention to generate an aerial three-dimensional display system. Consider, by way of illustration and not limitation, the embodiment depicted in
As shown in
The optical focusing element 2520 is positioned relative to the multi-view-window-based display system 2515 such that the multi-view windows 2510 formed by the multi-view-window-based display system 2515 effectively form rear projectors, the light of which are captured by the optical element 2520 to form a virtual screen 2525 at another location. A viewer 2545 properly positioned to view the virtual screen images 2525 will see a floating three-dimensional image 2530. Since the displayed image 2530 appears to be presented as on a virtual screen 2525, the three-dimensional image 2530 will appear as if floating in air.
In addition to converting a viewing window type display system to a rear-projection/light-field type display system, the embodiments of this section differ from the embodiments of the preceding sections in that they can produce systems that are more compact, spatially. Only the physical screen (e.g., 2505) and the virtual screen (e.g., 2525) are outside the focal point of the optical focusing element. The physical screen and the virtual screen are optically conjugate with each other. The viewing windows (e.g., 2510) are located inside the focal point. As a result, the floating display system becomes a rear-projection/light-field type display system, and it can be more compact than embodiments of the prior sections.
It should be noted that, similar to the embodiments of the prior sections, the size and location of the virtual screen and the virtual projection windows can be changed by changing the location of the actual screen and the viewing windows.
The virtual screen 2625 is optically conjugate with the physical screen 2605 and the virtual projection window 2640 is optically conjugate with the viewing window 2610 at the projection point. By adding more projectors, more viewing windows, as well as their images (the virtual projection windows) are formed.
Turning now to
The embodiment depicted in
In the aforementioned embodiments, the viewing windows can theoretically be positioned from the left focal point to right infinity. The virtual projection windows will range from left infinity, when the viewing windows are close to the left focal point, to the right focal point, when the viewing windows are close to the right infinity.
F. Display System Embodiments
One skilled in the art shall recognize that system depicted in
It shall be noted that the present invention may be implemented using an instruction-execution/computing device or system capable of processing data, including without limitation, a general-purpose computer and a specific computer, such as one intended for data or image processing. The present invention may also be implemented with other computing devices and systems. Furthermore, aspects of the present invention may be implemented in a wide variety of ways including software, hardware, firmware, or combinations thereof. For example, the functions to practice various aspects of the present invention may be performed by components that are implemented in a wide variety of ways including discrete logic components, one or more application specific integrated circuits (ASICs), and/or program-controlled processors. It shall be noted that the manner in which these items are implemented is not critical to the present invention.
An addressable memory 3306, coupled to processor 3302, may be used to store data and software instructions to be executed by processor 3302. Memory 3306 may be, for example, firmware, read only memory (ROM), flash memory, non-volatile random access memory (NVRAM), random access memory (RAM), or any combination thereof. In one embodiment, memory 3306 stores a number of software objects, otherwise known as services, utilities, components, or modules. One skilled in the art will also recognize that storage 3304 and memory 3306 may be the same items and function in both capacities. In an embodiment, one or more of the software components or modules may be stored in memory 3304, 3306 and executed by processor 3302.
In an embodiment, computing system 3300 provides the ability to communicate with other devices, other networks, or both. Computing system 3300 may include one or more network interfaces or adapters 3312, 3314 to communicatively couple computing system 3300 to other networks and devices. For example, computing system 3300 may include a network interface 3312, a communications port 3314, or both, each of which are communicatively coupled to processor 3302, and which may be used to couple computing system 3300 to other computer systems, networks, and devices.
In an embodiment, computing system 3300 may include one or more output devices 3308, coupled to processor 3302, to facilitate displaying graphics and text. Output devices 3308 may include, but are not limited to, a projector, a display, LCD screen, CRT monitor, printer, touch screen, or other device for displaying information. Computing system 3300 may also include a graphics adapter (not shown) to assist in displaying information or images on output device 3308.
One or more input devices 3310, coupled to processor 3302, may be used to facilitate user input. Input device 3310 may include, but are not limited to, a pointing device, such as a mouse, trackball, or touchpad, and may also include a keyboard or keypad to input data or instructions into computing system 3300.
In an embodiment, computing system 3300 may receive input, whether through communications port 3314, network interface 3312, stored data in memory 3304/3306, or through an input device 3310, from a scanner, copier, facsimile machine, or other computing device.
One skilled in the art will recognize no computing system is critical to the practice of the present invention. One skilled in the art will also recognize that a number of the elements described above may be physically and/or functionally separated into sub-modules or combined together.
It shall be noted that embodiments of the present invention may further relate to computer products with a computer-readable medium that have computer code thereon for performing various computer-implemented operations. The media and computer code may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind known or available to those having skill in the relevant arts. Examples of computer-readable media include, but are not limited to: magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and holographic devices; magneto-optical media; and hardware devices that are specially configured to store or to store and execute program code, such as application specific integrated circuits (ASICs), programmable logic devices (PLDs), flash memory devices, and ROM and RAM devices. Examples of computer code include machine code, such as produced by a compiler, and files containing higher level code that are executed by a computer using an interpreter. Embodiments of the present invention may be implemented in whole or in part as machine-executable instructions that may be in program modules that are executed by a computer. Examples of program modules include libraries, programs, routines, objects, components, and data structures. In distributed computing environments, program modules may be physically located in settings that are local, remote, or both.
While the invention is susceptible to various modifications and alternative forms, specific examples thereof have been shown in the drawings and are herein described in detail. It should be understood, however, that the invention is not to be limited to the particular forms disclosed, but to the contrary, the invention is to cover all modifications, equivalents, and alternatives falling within the scope of the appended claims.
Patent | Priority | Assignee | Title |
11009640, | Aug 11 2017 | 8259402 CANADA INC | Transmissive aerial image display |
11624934, | Nov 02 2017 | InterDigital Madison Patent Holdings, SAS | Method and system for aperture expansion in light field displays |
11917121, | Jun 28 2019 | InterDigital Madison Patent Holdings, SAS | Optical method and system for light field (LF) displays based on tunable liquid crystal (LC) diffusers |
11991343, | Jun 07 2019 | InterDigital Madison Patent Holdings, SAS | Optical method and system for light field displays based on distributed apertures |
8581929, | Jun 05 2012 | Display of light field image data using a spatial light modulator at a focal length corresponding to a selected focus depth | |
8862285, | Feb 15 2013 | Disney Enterprises, Inc. | Aerial display system with floating pixels |
8994728, | Jun 05 2012 | Display of light field image data using a spatial light modulator at a focal length corresponding to a selected focus depth | |
9169030, | Feb 15 2013 | Disney Enterprises, Inc. | Aerial display system with floating pixels |
9667951, | Feb 18 2014 | SYNAMEDIA LIMITED | Three-dimensional television calibration |
9881529, | Jun 12 2015 | Innolux Corporation | Display device and operating method thereof |
Patent | Priority | Assignee | Title |
5621572, | Aug 24 1994 | Fergason Patent Properties LLC | Optical system for a head mounted display using a retro-reflector and method of displaying an image |
5671992, | Apr 28 1993 | XENOTECH RESEARCH PTY LTD | Stereoscopic display unit |
5993003, | Mar 27 1997 | Northrop Grumman Systems Corporation | Autostereo projection system |
6055013, | Feb 04 1997 | Sharp Kabushiki Kaisha | Autostereoscopic display |
6075557, | Apr 17 1997 | Sharp Kabushiki Kaisha | Image tracking system and method and observer tracking autostereoscopic display |
6224214, | Mar 27 1997 | Northrop Grumman Systems Corporation | Autostereo projection system |
6381068, | Mar 19 1999 | 3M Innovative Properties Company | Reflective projection screen and projection system |
6543899, | Dec 05 2000 | Eastman Kodak Company | Auto-stereoscopic viewing system using mounted projection |
6752498, | May 14 2001 | CARESTREAM HEALTH, INC | Adaptive autostereoscopic display system |
6843564, | Jan 04 2002 | VIRRILLI, JAMES | Three-dimensional image projection employing retro-reflective screens |
6963224, | Aug 08 2003 | Olympus Corporation | Reflexive optical screen, and viewing system incorporating the same |
6999071, | May 19 2000 | Method and apparatus for displaying 3d images | |
7167308, | Mar 09 2005 | Rockwell Collins, Inc | System for eliminating secondary images in rear projection systems |
7224526, | Dec 08 1999 | VIRRILLI, JAMES | Three-dimensional free space image projection employing Fresnel lenses |
7239293, | Jan 21 1998 | Intellectual Ventures Holding 81 LLC | Autostereoscopic display |
7251077, | Dec 15 2000 | De La Rue International Limited | Diffuse reflector and method |
7425070, | May 13 2005 | Microsoft Technology Licensing, LLC | Three-dimensional (3D) image projection |
20070064098, | |||
20070146358, | |||
20080204663, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Mar 22 2010 | XIAO, JING | EPSON RESEARCH AND DEVELOPMENT, INC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 024137 | /0306 | |
Mar 24 2010 | GAO, CHUNYU | EPSON RESEARCH AND DEVELOPMENT, INC | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 024137 | /0306 | |
Mar 25 2010 | Seiko Epson Corporation | (assignment on the face of the patent) | / | |||
Apr 02 2010 | EPSON RESEARCH AND DEVELOPMENT, INC | Seiko Epson Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 024311 | /0953 |
Date | Maintenance Fee Events |
Dec 04 2013 | ASPN: Payor Number Assigned. |
Mar 30 2016 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
Apr 02 2020 | M1552: Payment of Maintenance Fee, 8th Year, Large Entity. |
Jun 03 2024 | REM: Maintenance Fee Reminder Mailed. |
Date | Maintenance Schedule |
Oct 16 2015 | 4 years fee payment window open |
Apr 16 2016 | 6 months grace period start (w surcharge) |
Oct 16 2016 | patent expiry (for year 4) |
Oct 16 2018 | 2 years to revive unintentionally abandoned end. (for year 4) |
Oct 16 2019 | 8 years fee payment window open |
Apr 16 2020 | 6 months grace period start (w surcharge) |
Oct 16 2020 | patent expiry (for year 8) |
Oct 16 2022 | 2 years to revive unintentionally abandoned end. (for year 8) |
Oct 16 2023 | 12 years fee payment window open |
Apr 16 2024 | 6 months grace period start (w surcharge) |
Oct 16 2024 | patent expiry (for year 12) |
Oct 16 2026 | 2 years to revive unintentionally abandoned end. (for year 12) |