There is described an image sensor based indicia reading terminal comprising a variable setting imaging lens having a first setting at which the terminal has a first plane of optimum focus and a second setting at which the terminal has a second plane of optimum focus. According to one embodiment, a first predetermined picture size where picture size is determined according to a number of pixels subject to read out, can be associated to the first lens setting and a second picture size can be associated to the second lens setting such that the terminal with the lens setting set to the first setting reads out a frame of a first picture size and with the lens setting set to a second setting reads out a frame of a second picture size. In addition to or in place of the picture size operational parameters, different operational parameters can be associated to the respective first and second lens settings. Additionally or alternatively, different processes for determining an operational parameter and/or different algorithms for determining an operational parameter can be associated to each of the first and second lens settings.
|
9. An indicia reading terminal comprising:
an image sensor including an image sensor array having a plurality of pixels;
a variable setting lens assembly for focusing light on the image sensor array, the variable setting lens assembly having a first lens setting at which the terminal has a first plane of optimum focus and a second lens setting at which the terminal has a second plane of optimum focus;
wherein the indicia reading terminal is operative in a first operator selectable operating state and a second operator selectable operating state;
wherein the indicia reading terminal during an operator initiated read attempt in the first operator selectable operating state is operative to vary the lens setting of the lens assembly between a first lens setting and a second lens setting, wherein the indicia reading terminal is further operative in the first operator selectable operating state so that frames exposed with the lens assembly at the first lens setting have a first picture size and further so that frames exposed with the lens assembly at the second lens setting have a second picture size different from the first picture size;
wherein the indicia reading terminal during an operator initiated read attempt in the second operator selectable operating state is operative to vary the lens setting of the lens assembly between a first lens setting and a second lens setting, wherein the indicia reading terminal is further operative in the second operator selectable operating state so that frames exposed with the lens assembly at the first lens setting and frames exposed with the lens assembly at the second lens setting have a common picture size.
10. An indicia reading terminal comprising:
an image sensor including an image sensor array having a plurality of pixels;
a variable setting lens assembly for focusing light on the image sensor array, the variable setting lens assembly having a first lens setting at which the terminal has a first plane of optimum focus and a second lens setting at which the terminal has a second plane of optimum focus;
wherein the indicia reading terminal is operative to expose a first frame with the lens assembly at the first lens setting and is further operative to expose a second frame with the lens assembly at the second lens setting;
wherein the indicia reading terminal is further operative to attempt to decode a decodable indicia by processing at least one of the first frame and second frame;
wherein the indicia reading terminal is operative so that during an operator initiated read attempt the terminal exposes a first plurality of frames with the lens assembly at the first lens setting and a second plurality of frames with the lens assembly at the second lens setting, wherein the terminal is further operative so that during the operator initiated read attempt the terminal switches a lens setting of the lens assembly from the first lens setting to the second lens setting and back to the first lens setting so that there are exposed a plurality of frames with an alternating pattern of lens setting associations;
wherein the indicia reading terminal is further operative so that when executing a process for determining an operational parameter to be applied for a next frame with the lens setting at the first lens setting, the terminal preferentially utilizes a prior frame having as an associated lens setting the first lens setting.
1. An indicia reading terminal comprising:
an image sensor including an image sensor array having a plurality of pixels;
a variable setting lens assembly for focusing light on the image sensor array, the variable setting lens assembly having a first lens setting at which the terminal has a first plane of optimum focus and a second lens setting at which the terminal has a second plane of optimum focus, the second plane of optimum focus being farther than the first plane of optimum focus;
wherein the indicia reading terminal is operative to expose a first frame with the lens assembly at the first lens setting and is further operative to expose a second frame with the lens assembly at the second lens setting and is further operative to expose a third frame with the lens assembly at the first lens setting, the indicia reading terminal being operative to expose the second frame subsequently to exposing the first frame, the indicia reading terminal being operative to expose the third frame subsequently to exposing the second frame;
wherein the indicia reading terminal is further operative to attempt to decode a decodable indicia by processing each of the first frame the second frame and the third frame;
wherein the indicia reading terminal is further operative so that there is associated with the first lens setting a first picture size and further so that there is associated with the second lens setting a second picture size so that a frame exposed with the lens assembly at the first lens setting has a first picture size and further so that a frame exposed with the lens assembly at the second lens setting has a second picture size, wherein the first picture size is larger than the second picture size, wherein the first frame and the third frame have pixel values corresponding to a first set of pixels of the image sensor array, wherein the second frame has pixel values corresponding to a second set of pixels of the image sensor array, the first set of pixels including the pixels of the second set of pixels, wherein the indicia reading terminal is further operative so that responsively to a trigger signal activation the indicia reading terminal captures the first and second and third frames, the second frame being captured subsequently to a capture of the first frame, the third frame being captured subsequently to a capture of the second frame.
13. An indicia reading terminal comprising:
an image sensor including an image sensor array having a plurality of pixels;
a variable setting lens assembly for focusing light on the image sensor array, the variable setting lens assembly having a first lens setting at which the terminal has a first plane of optimum focus and a second lens setting at which the terminal has a second plane of optimum focus and a third lens setting at which the terminal has a third plane of optimum focus, the third plane of optimum focus being farther than the second plane of optimum focus, the second plane of optimum focus being farther than the first plane of optimum focus;
wherein the indicia reading terminal is operative to expose a first frame with the lens assembly at the first lens setting and is further operative to expose a second frame with the lens assembly at the second lens setting and is further operative to expose a third frame with the lens assembly at the third lens setting;
wherein the indicia reading terminal is further operative to attempt to decode a decodable indicia by processing each of the first frame the second frame and the third frame;
wherein the indicia reading terminal is further operative so that there is associated with the first lens setting a first picture size defined by a first set of pixel values corresponding to the first set of pixels of the image sensor array and further so that there is associated with the second lens setting a second picture size defined by a second set of pixel values of the image sensor array and further so that there is associated with the third lens setting a third picture size defined by a third set of pixel values corresponding to a third set of pixels of the image sensor array, the third picture size being larger than the first picture size and the second picture size being larger than the first picture size, wherein the indicia reading terminal is operative so that a frame exposed with the lens assembly at the first lens setting has the first picture size and further so that a frame exposed with the lens assembly at the second lens setting has the second picture size and further being operative so that a frame exposed with the lens assembly at the third lens setting has the third picture size, wherein the first set of pixels includes pixels of each row the second set of pixels, and wherein the second set of pixels includes pixels of each row of the third set of pixels, wherein the indicia reading terminal is operative to that responsively to activation of a trigger signal the indicia reading terminal captures the first second and third frames.
2. The indicia reading terminal as set forth in
3. The indicia reading terminal as set forth in
4. The indicia reading terminal as set forth in
5. The indicia reading terminal as set forth in
6. The indicia reading terminal as set forth in
7. The indicia reading terminal as set forth in
8. The indicia reading terminal as set forth in
11. The indicia reading terminal as set forth in
12. The indicia reading terminal as set forth in
14. The indicia reading terminal as set forth in
15. The indicia reading terminal as set forth in
16. The indicia reading terminal as set forth in
17. The indicia reading terminal as set forth in
18. The indicia reading terminal as set forth in
19. The indicia reading terminal as set forth in
|
|||||||||||||||||||||||||||
The present invention relates in general to optical based registers, and particularly is related to an image sensor based indicia reading terminal.
Indicia reading terminals for reading decodable indicia are available in multiple varieties. For example, minimally featured indicia reading terminals devoid of a keyboard and display are common in point of sale applications. Indicia reading terminals devoid of a keyboard and display are available in the recognizable gun style form factor having a handle and trigger button (trigger) that can be actuated by an index finger. Indicia reading terminals having keyboards and displays are also available. Keyboards and display equipped indicia reading terminals are commonly used in shipping and warehouse applications, and are available in form factors incorporating a display and keyboard. In a keyboard and display equipped indicia reading terminal, a trigger button for actuating the output of decoded messages is typically provided in such locations as to enable actuation by a thumb of an operator. Indicia reading terminals in a form devoid of a keyboard and display or in a keyboard and display equipped form are commonly used in a variety of data collection applications including point of sale applications, shipping applications, warehousing applications, security check point applications, and patient care applications. Some indicia reading terminals are adapted to read bar code symbols including one or more of one dimensional (1D) bar codes, stacked 1D bar codes, and two dimensional (2D) bar codes. Other indicia reading terminals are adapted to read OCR characters while still other indicia reading terminals are equipped to read both bar code symbols and OCR characters.
There is described an image sensor based indicia reading terminal comprising a variable setting imaging lens having a first setting at which the terminal has a first plane of optimum focus and a second setting at which the terminal has a second plane of optimum focus. According to one embodiment, a first predetermined picture size where picture size is determined according to a number of pixels subject to read out, can be associated to the first lens setting and a second picture size can be associated to the second lens setting such that the terminal with the lens setting set to the first setting reads out a frame of a first picture size and with the lens setting set to a second setting reads out a frame of a second picture size. In addition to or in place of the picture size operational parameters, different operational parameters can be associated to the respective first and second lens settings. Additionally or alternatively, different processes for determining an operational parameter and/or different algorithms for determining an operational parameter can be associated to each of the first and second lens settings.
The features described herein can be better understood with reference to the drawings described below. The drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention. In the drawings, like numerals are used to indicate like parts throughout the various views.
There is described an image sensor based indicia reading terminal 1000 having a variable setting imaging lens assembly that includes a first setting at which the terminal has a first plane of optimum focus (best focus distance) and a second setting at which the terminal has a second plane of optimum focus. Such settings, therefore, can be regarded as plane of optimum focus (or best focus distance) lens settings. Referring to
In addition, as will be described herein, terminal 1000 can be operative to have a first functionality wherein during an operator initiated read attempt, a lens setting of a lens assembly is varied and a second functionality wherein a lens setting remains fixed during an operator activated read attempt, but which can be subject to adjustment intermediate of first and second operator activated read attempts in response to an operator input command input to terminal 1000 between first and second operator activated read attempts.
Terminal 1000 can be operative so that at a first time during an operator activated read attempt, an actual present reading distance of terminal 1000 may be spaced apart significantly from a present plane of optimum focus distance lens setting of terminal 1000. Nevertheless, at a second time during an operator activated read attempt, an actual present terminal to target distance, D, of terminal 1000 can closely correspond to and be in proximity with a plane of optimum focus lens setting of terminal 1000. Operating in accordance with the first functionality, a varying plane of optimum focus lens setting may converge on an actual terminal to target distance during an operator activated read attempt in which a succession of frames are exposed, read out, captured, and subject to processing. Operating according to the second functionality, terminal 1000, if portable, can be moved by an operator during an operator activated read attempt into a position at which a current terminal to target distance closely corresponds to and is proximate to the plane of optimum focus setting. Also, according to the second functionality, a substrate bearing a decodable indicia can be brought into proximity with terminal 1000 during an operator activated read attempt.
According to one embodiment, a first picture size can be associated to the first lens setting and a second picture size can be associated to the second lens setting such that the terminal with the lens setting set to the first lens setting reads out a frame of a first picture size and with the lens setting set to a second setting reads out a frame of a second picture size.
In addition to or in place of the picture size operational parameter, different operational parameters can be associated to the respective first and second lens settings.
One such operational parameter is frame exposure level; another operational parameter is amplifier gain; another operational parameter is initial exposure period; another operational parameter is initial amplifier gain.
Additionally or alternatively, different processes for determining an operational parameter and/or different algorithms for determining an operational parameter can be associated to each of the first and second lens settings.
For example, a first process and/or algorithm for controlling exposure level can be active when the lens setting is at the first lens setting and a second process and/or algorithm for controlling exposure level can be active when the lens setting is at the second lens setting. Also, a first process and/or algorithm for controlling amplifier gain can be active when the lens setting is at the first lens setting and a second process and/or algorithm for controlling amplifier gain can be active when the lens setting is at the second lens setting.
In development of a variable lens setting indicia reading terminal described herein, it was determined that decoding of frames exposed with the terminal highly out of focus (i.e., with the terminal at a terminal to target distance significantly spaced apart from a distance corresponding to the plane of optimum focus distance of the present lens setting) will normally fail as a result of the terminal being out of focus irrespective of whether operational controls are optimized for the actual current reading distance of the terminal. Accordingly, in one embodiment, an indicia reading terminal is operative so that operational parameters of the terminal are optimized for reading distances corresponding to the plane of optimum focus of the current setting of the terminal's lens assembly.
In one embodiment, a lens setting of a terminal's lens assembly can be controlled on an open loop basis, i.e., without reference to either a ranging parameter or a focus determination or another determined condition. In such an embodiment, the terminal can be operative so that operational parameters of the terminal are optimized for reading distances corresponding to the plane of optimum focus of the current setting of the terminal's lens assembly irrespective of whether the plane of optimum focus distance of the current lens setting is proximate to the current actual terminal to target distance. Operative as described, a likelihood of successful reading and expected read time can be improved. In another embodiment, terminal 1000 can incorporate range detection functionality and a lens setting of a lens assembly of terminal 1000 can be set according to the detected range.
A hardware platform for support of operations described herein with reference to an image sensor based indicia reading terminal is shown and described with reference to
Indicia reading terminal 1000 can include an image sensor 1032 comprising a multiple pixel image sensor array 1033 having pixels arranged in rows and columns of pixels, associated column circuitry 1034 and row circuitry 1035. Associated with the image sensor 1032 can be amplifier circuitry 1036 (amplifier), and an analog to digital converter 1037 which converts image information in the form of analog signals read out of image sensor array 1033 into image information in the form of digital signals. Image sensor 1032 can also have an associated timing and control circuit 1038 for use in controlling e.g., the exposure period of image sensor 1032, gain applied to the amplifier 1036. The noted circuit components 1032, 1036, 1037, and 1038 can be packaged into a common image sensor integrated circuit 1040. In one example, image sensor integrated circuit 1040 can be provided e.g., by an MT9V022 (752×480 pixel array) or an MT9V023 (752×480 pixel array) image sensor integrated circuit available from Micron Technology, Inc. In one example, image sensor integrated circuit 1040 can incorporate a Bayer pattern filter, so that defined at the image sensor array are red pixels at red pixel positions, green pixels at green pixel positions, and blue pixels at blue pixel positions. Frames that are provided utilizing such an image sensor array incorporating a Bayer pattern can include red pixel values at red pixel positions, green pixel values at green pixel positions, and blue pixel values at blue pixel positions. In an embodiment incorporating a Bayer pattern image sensor array, CPU 1060 prior to subjecting a frame to further processing can interpolate pixel values at frame pixel positions intermediate of green pixel positions utilizing green pixel values for development of a monochrome frame of image data. Alternatively, CPU 1060 prior to subjecting a frame for further processing can interpolate pixel values intermediate of red pixel positions utilizing red pixel values. CPU 1060 can alternatively prior to subjecting a frame for further processing can interpolate pixel values intermediate of blue pixel positions utilizing blue pixel values.
In the course of operation of terminal 1000, image signals can be read out of image sensor 1032, converted, and stored into a system memory such as RAM 1080. A memory 1085 of terminal 1000 can include RAM 1080, a nonvolatile memory such as EPROM 1082 and a storage memory device 1084 such as may be provided by a flash memory or a hard drive memory. In one embodiment, terminal 1000 can include CPU 1060 which can be adapted to read out image data stored in memory 1080 and subject such image data to various image processing algorithms. Terminal 1000 can include a direct memory access unit (DMA) 1070 for routing image information read out from image sensor 1032 that has been subject to conversion to RAM 1080. In another embodiment, terminal 1000 can employ a system bus providing for bus arbitration mechanism (e.g., a PCI bus) thus eliminating the need for a central DMA controller. A skilled artisan would appreciate that other embodiments of the system bus architecture and/or direct memory access components providing for efficient data transfer between the image sensor 1032 and RAM 1080 are within the scope and the spirit of the invention.
Referring to further aspects of terminal 1000, lens assembly 200 can be adapted for focusing an image of a decodable indicia 15 located within a field of view 1240 on a substrate, T, onto image sensor array 1033. Imaging light rays can be transmitted about imaging axis 25. Lens assembly 200 can be adapted to be capable of multiple focal lengths and multiple best focus distances.
Terminal 1000 can also include an illumination pattern light source bank 1204 and associated light shaping optics 1205 for generating an illumination pattern 1260 substantially corresponding to a field of view 1240 of terminal 1000. The combination of bank 1204 and optics 1205 can be regarded as an illumination pattern generator 1206. Terminal 1000 can also include an aiming pattern light source bank 1208 and associated light shaping optics 1209 for generating an aiming pattern 1270 on a target, T, which can be provided by a substrate. The combination of bank 1208 and optics 1209 can be regarded as an aiming pattern generator 1210. In use, terminal 1000 can be oriented by an operator with respect to a target, T, (e.g., a substrate, a piece of paper, a package) bearing decodable indicia 15 in such manner that aiming pattern 1270 is projected on a decodable indicia 15. In the example of
In the embodiment of
Referring to
Terminal 1000 can also include a number of peripheral devices including trigger 3408 which may be used to make active a trigger signal for activating frame readout and/or certain decoding processes. Terminal 1000 can be adapted so that activation of trigger 3408 activates a trigger signal and initiates a decode attempt. Specifically, terminal 1000 can be operative so that in response to activation of a trigger signal, a succession of frames can be read out and captured by way of read out of image information from image sensor array 1033 (typically in the form of analog signals) and then storage of the image information after conversion into memory 1080 (which can buffer one or more of the succession of frames at a given time). CPU 1060 can be operational to subject one or more of the succession of frames to a decode attempt. For attempting to decode a bar code symbol, e.g., a one dimensional bar code symbol, CPU 1060 can process image data of a frame corresponding to a line of pixel positions (e.g., a row, a column, or a diagonal set of pixel positions) to determine a spatial pattern of dark and light cells and can convert each light and dark cell pattern determined into a character or character string via table lookup.
Where a decodable indicia representation is a 2D bar code symbology, a decode attempt can comprise the steps of locating a finder pattern using a feature detection algorithm, locating matrix lines intersecting the finder pattern according to a predetermined relationship with the finder pattern, determining a pattern of dark and light cells along the matrix lines, and converting each light pattern into a character or character string via table lookup.
Terminal 1000 can include various interface circuits for coupling various of the peripheral devices to system address/data bus (system bus) 1500, for communication with CPU 1060 also coupled to system bus 1500. Terminal 1000 can include interface circuit 1028 for coupling image sensor timing and control circuit 1038 to system bus 1500, interface circuit 1118 for coupling electrical power input unit 55 to system bus 1500, interface circuit 1218 for coupling illumination light source bank control circuit 1220 to system bus 1500, interface circuit 1224 for coupling aiming light source bank control circuit 1222 to system bus 1500, and interface circuit 1402 for coupling trigger 3408 to system bus 1500. Terminal 1000 can also include a display 3420 coupled to system bus 1500 and in communication with CPU 1060, via interface 1418, as well as pointer mechanism 3410 in communication with CPU 1060 via interface 2409 connected to system bus 1500.
A succession of frames of image data that can be captured and subject to the described processing can be full frames (including pixel values corresponding to each pixel of image sensor array 1033 or a maximum number of pixels read out from array 1033) during operation of terminal 1000. A succession of frames of image data that can be captured and subject to the described processing can also be “windowed frames” comprising pixel values corresponding to less than a full frame of pixels of image sensor array 1033. A succession of frames of image data that can be captured and subject to the described processing can also comprise a combination of full frames and windowed frames. A full frame can be captured by selectively addressing for read out pixels of image sensor 1032 having image sensor array 1033 corresponding to the full frame. A windowed frame can be captured by selectively addressing for read out pixels of image sensor 1032 having image sensor array 1033 corresponding to the windowed frame. In one embodiment, a number of pixels subject to addressing and read out determine a picture size of a frame. Accordingly, a full frame can be regarded as having a first relatively larger picture size and a windowed frame can be regarded as having a relatively smaller picture size relative to a picture size of a full frame. A picture size of a windowed frame can vary depending on the number of pixels subject to addressing and relevant for capture of a windowed frame.
Terminal 1000 can capture frames of image data at a rate known as a frame rate. A typical frame rate is 60 frames per second (FPS) which translates to a frame time (frame period) of 16.6 ms. Another typical frame rate is 30 frames per second (FPS) which translates to a frame time (frame period) of 33.3 ms per frame. A frame rate of terminal 1000 can be increased (and frame time decreased) by decreasing of a frame picture size.
A physical form view of terminal 1000 in one embodiment is shown in
Referring to terminal 1000, terminal 1000 can be operative to move a lens setting of lens assembly 200 between at least a first plane of optimum focus setting (best focus distance setting) and a second plane of optimum focus setting. Indicia reading terminal 1000 can be operative to move a lens setting of the lens assembly between at least first and second different planes of optimum focus settings, and can further be operative to expose a first frame of image data with the lens assembly at the first plane of optimum focus setting and a second frame of image data with the lens assembly at the second plane of optimum focus setting, and the terminal can further be configured so that the terminal is operative to subject each of the first and second frames of image data to a decode attempt for decoding of a decodable indicia. The second frame can be a successive frame in relation to the first frame or a non-successive subsequent frame in relation to the first frame. Also, the first and second frames of image data can be exposed, read out, captured, and processed during a common operator activated read attempt, or alternatively, separate operator activated read attempts.
Terminal 1000 can be operative so that terminal 1000, when an operator activated read attempt is activated by actuation of trigger 3408, can capture a succession of frames and subject one or more of the frames to a decode attempt until a time that the operator activated read attempt is deactivated, e.g., by release of trigger 3408 or a successful decode or a timeout condition being satisfied. In another aspect, terminal 1000 according to a first functionality set forth herein can be operative to move a lens setting of the lens assembly between at least the first and the second lens settings of the lens assembly during a time that the terminal executes an operator activated read attempt of the terminal.
Terminal 1000 according to a second functionality set forth herein can be operative to maintain a lens setting of the terminal at a certain lens setting during a time that the terminal executes an operator activated read attempt of the terminal. In addition, terminal 1000 can be operative to move a lens setting in accordance with an operator input command input subsequent to a first operator activated read attempt and prior to a second operator activated read attempt. Terminal 1000 can be further operative so that a first frame and a second frame utilized for a decode attempt are frames exposed during separate first and second separate operator activated read attempts of the terminal.
A timing diagram further illustrating operation of terminal 1000 in one embodiment is shown in
Referring to the timing diagram of
Referring to processing periods 3520, 3522, 3524, 3526, 3528, and 3530, the noted processing periods can represent processing periods during which time CPU 1060 of terminal 1000 processes stored (e.g., buffered) frames representing a substrate that can bear decodable indicia. Such processing can include processing for attempting to decode a decodable indicia as described herein.
With further reference to the timing diagram of
Referring to the timing diagram of
Referring to signal 3508, signal 3508 can be established at an energy level corresponding to the selected lens setting. In one embodiment, a selection of a configuration in which the second described functionality is active can be regarded as a selected lens setting. Referring to the timing diagram of
Mode, configuration, or setting selections described herein described as being made with use of a user (operator) interface comprising a display and pointer mechanism of terminal 1000 or terminal 1000 can also be made with use of another user interface, e.g., terminal 1000 can be operative to be reprogrammed by reading of programming bar code symbols, and a user interface can be regarded as being provided by the providing of terminal 1000 to be reprogrammed by reading of programming bar code symbols.
Various configurations of terminal 1000 are summarized in Table A. Table A summarizes operation of terminal 1000 in accordance with various configurations including those summarized in connection with
TABLE A
CON-
FIG-
URA-
FRAME
FRAME
FRAME
FRAME
FRAME
FRAME
FRAME
FRAME
FRAME
FRAME
TION
1
2
3
4
5
6
7
8
9
10
1
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
. . .
FAR
FAR
FAR
FAR
FAR
FAR
FAR
FAR
FAR
2
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
. . .
INTER-
INTER-
INTER-
INTER-
INTER-
INTER-
INTER-
INTER-
INTER-
MEDIATE
MEDIATE
MEDIATE
MEDIATE
MEDIATE
MEDIATE
MEDIATE
MEDIATE
MEDIATE
3
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
. . .
NEAR
NEAR
NEAR
NEAR
NEAR
NEAR
NEAR
NEAR
NEAR
4
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
. . .
NEAR
NEAR
NEAR
NEAR
NEAR
NEAR
NEAR
NEAR
NEAR
CONTACT
CONTACT
CONTACT
CONTACT
CONTACT
CONTACT
CONTACT
CONTACT
CONTACT
5
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
. . .
NEAR
NEAR
INTER-
FAR
INTER-
INTER-
INTER-
INTER-
INTER-
CONTACT
MEDIATE
MEDIATE
MEDIATE
MEDIATE
MEDIATE
MEDIATE
6
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
. . .
NEAR
NEAR
INTER-
FAR
INTER-
NEAR
NEAR
NEAR
INTER-
CONTACT
MEDIATE
MEDIATE
CONTACT
MEDIATE
7
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
. . .
NEAR
NEAR
INTER-
FAR
NEAR
NEAR
INTER-
FAR
NEAR
CONTACT
MEDIATE
CONTACT
MEDIATE
CONTACT
8
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
. . .
INTER-
FAR
INTER-
NEAR
NEAR
NEAR
INTER-
FAR
INTER-
MEDIATE
MEDIATE
CONTACT
MEDIATE
MEDIATE
9
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
SETTING:
. . .
NEAR
INTER-
FAR
INTER-
NEAR
INTER-
FAR
INTER-
NEAR
MEDIATE
MEDIATE
MEDIATE
MEDIATE
Referring now to Table B, various association profiles of terminal 1000 are shown and described. With reference to Table B, terminal 1000 can be operative to have a profile of the set of profiles A*B*C*D*E* where “*” is a wild card designator designating any one of the subscripts associated with the character as shown in Table B. Terminal 1000 can be operative to always operate in a single profile of the set of profiles A*B*C*D*E* or else can be operative so that a particular active profile among a set of possible candidate profiles is operator selectable. Candidate profiles of the set of profiles A*B*C*D*E* can comprise a plurality of sub-profiles. A selected profile can include a selected sub-profile of each of several categories. In the described example, selection of a sub-profile in the A* category establishes a particular association (or lack of association) between a lens setting and a picture size. Selection of a particular profile in the B* sub-profile category establishes a particular association (or lack of association) between a lens setting and an exposure level. Selection of a particular profile in the C* sub-profile category establishes a particular association (or lack of association) between a lens setting and an amplifier gain control, i.e., the gain of amplifier 1036, that can amplify image signals. As indicated by the D* sub-profiles and E* sub-profiles, terminal 1000 can be operative so that a lens setting of lens assembly 200 can be linked to one or more controls of additional or alternative categories.
For allowing operator selection of a profile, terminal 1000 can be operative to have a selector feature as is described with reference to button 5502 of
TABLE B
Assoc.
NEAR
Profile
FAR
INTERMEDIATE
NEAR
CONTACT
COMMENTS
A0
Picture Size:
Picture Size:
Picture Size:
Picture Size:
Picture size is unchanged and
Full Frame:
Full Frame:
Full Frame:
Full Frame:
set to a full frame picture size
irrespective of lens setting.
A1
Picture Size:
Picture Size:
Picture Size:
Picture Size:
Picture size is increased for closer
Window
Window
Window
Window
plane of optimum focus (best focus
Coordinates
Coordinates
Coordinates
Coordinates
distance) lens settings.
(1, 240; 752,
(200, 200; 552,
(100, 100; 652,
(1, 1; 752,
240; 1, 241;
200; 200, 280;
100; 100, 380;
1; 1, 480;
752, 241)
552, 280)
652, 380)
752, 480) (full frame)
A2
Picture Size:
Picture Size:
Picture Size:
Picture Size:
Picture size remains at a setting
Window
Window
Window
Window
corresponding to a specific window
Coordinates
Coordinates
Coordinates
Coordinates
position and picture size less than
(1, 240; 752,
(1, 240; 752,
(1, 240; 752,
(1, 240; 752,
a full frame picture size irrespective
240; 1, 241;
240; 1, 241;
240; 1, 241;
240; 1, 241;
of lens setting.
752, 241)
752, 241)
752, 241)
752, 241)
B0
Exposure:
Exposure:
Exposure:
Exposure:
Exposure level remains constant
E = EB
E = EB
E = EB
E = EB
irrespective of lens setting.
B1
Exposure:
Exposure:
Exposure:
Exposure:
Exposure increases with longer
E = EF, EF > EB
E = EB
E = EN, EN < EB
E = ENC, ENC < EN
best focus distance lens settings.
B2
Exposure:
Exposure:
Exposure:
Exposure:
Each lens setting has a specific
Eo = EF, EF > EB
Eo = EB
Eo = EN,
Eo = ENC,
exposure control process.
Ei = fE1 (Ei−1, Wi−1)
Ei = fE1 (Ei−1, Wi−1)
EN < EB
ENC < EN
Ei = fE1 (Ei−1, Wi−1)
Ei = fE1 (Ei−1), Wi−1)
B3
Exposure:
Exposure:
Exposure:
Exposure:
Each lens setting has an
Eo = EF, EF > EB
Eo = EB
Eo = EN,
Eo = ENC,
associated specific exposure control
Ei = fE1 (Ei−1, Wi−1)
Ei = fE2 (Ei−1, Wi−1)
EN < EB
ENC < EN
process and algorithm.
Ei = fE3 (Ei−1, Wi−1)
Ei = fE4 (Wi−1)
C0
AMPLIFIER
AMPLIFIER
AMPLIFIER
AMPLIFIER
Gain unchanged regardless of lens
GAIN: G = GB
GAIN: G = GB
GAIN: G = GB
GAIN: G = GB
settings.
C1
AMPLIFIER
AMPLIFIER
AMPLIFIER
AMPLIFIER
Gain is decreased for nearer best
GAIN: G = GF,
GAIN: G = GB
GAIN: G = GN,
GAIN: G = GNC,
focus distance lens settings.
GF > GB
GN < GB
GNC < GN
C2
AMPLIFIER
AMPLIFIER
AMPLIFIER
AMPLIFIER
Each lens setting has an associated
GAIN: Go = GF,
GAIN: Go = GB
GAIN: Go = GN,
GAIN: Go = GNC,
specific gain control process.
GF > GB
Gi = fG1 (Gi−1, Wi−1)
GN < GB
GNC < GN
Gi = fG1 (Gi−1, Wi−1)
Gi = fG1 (Gi−1, Wi−1)
Gi = fG1 (Gi−1, Wi-1)
C3
AMPLIFIER
AMPLIFIER
AMPLIFIER
AMPLIFIER
Each lens setting has an associated
GAIN: Go = GF,
GAIN: Go = GB
GAIN: Go = GN,
GAIN: Go = GNC,
specific gain control process and
GF > GB
Gi = fG2 (Gi−1, Wi−1)
GN < GB
GNC < GN
algorithm.
Gi = fG1 (Gi−1, Wi−1)
Gi = fG3 (Gi−1, Wi−1)
Gi = fG4 (Wi−1)
As seen from Table B, selection of a certain sub-profile associates a particular control (e.g., an operational parameter, a process for determining an operational parameter, an algorithm for determining an operational parameter) to a particular lens setting so that in operation of terminal 1000, the particular control is active when the particular lens setting is active. If a particular picture size is associated to a particular lens setting, a frame having particularly specified window coordinates (defining a boundary for a windowed frame to be read out) can be addressed for read out when there is a read out of a frame with the lens setting active. If a particular process for determining an operational parameter is associated to a particular lens setting, the particular process for determining an operational parameter will be active when there is being determined an operative parameter for a scan having the associated lens setting. If a particular exposure period is associated to a particular lens setting, the terminal will control the exposure period to the particular level when there is association with the particular lens setting. If a particular gain is associated to a particular lens setting, the terminal can control the gain to the associated level when applying gain to frame image signals of a frame exposed with that particular lens setting active. In the examples of Table B, a window position is specified in terms of outer boundary coordinates. It will be understood that other formulas can be utilized for specified window positions (e.g., a window dimension in combination with an offset).
Additional features are now described with reference to particular sub-profiles that can be selected by an operator. Referring to sub-profile A1, with sub-profile A1 active, a picture size (expressed in terms of window coordinates defining a boundary for a windowed frame to be read out (according to the described formatting, pixels within the boundary can be subject to readout, and readout can be avoided for pixels outside the boundary) is associated with and automatically varies with a lens setting. In general, as seen with reference to sub-profile A1, picture size in terms of number of pixels subject to read out is increased when a lens setting of lens assembly 200 includes a nearer plane of optimum focus and is reduced when a lens setting of lens assembly 200 is at a longer plane of optimum focus. The availability of sub-profile A2 is in recognition of the fact that at longer reading distances, a smaller picture size will be sufficient to encompass a complete representation of a decodable indicia such as a bar code. By contrast, referring to sub-profile A0, with sub-profile A0 active a picture size of terminal 1000 remains at a constant size (a full frame) irrespective of the lens setting of the terminal. With sub-profile A2 active, the picture size which remains constant (at a full frame setting) irrespective of the lens setting can be regarded as being disassociated with the lens setting. With sub-profile A2 active, the picture size also remains constant, but at a constant setting that defines a specific windowed frame position.
Turning now to sub-profiles B0, B1, B2, and B3, the sub-profiles B0, B1, B2, and B3 relate to the control of an exposure level (exposure period) of terminal 1000. In the development of terminal 1000, it was determined that it might be useful in some applications to vary a control of exposure with the present lens setting.
Referring to sub-profile B1, a longer exposure period for a terminal can be utilized at a farther plane of optimum focus lens settings irrespective of the actual terminal to target distance and shorter exposure periods can be applied at shorter plane of optimum focus lens settings irrespective of the actual terminal to target distance. Thus, with longer plane of optimum focus lens settings, longer exposure periods can be utilized for optimization of performance of terminal at terminal to target distances corresponding to the current lens setting. By contrast, referring to sub-profile B0, with sub-profile B0 active, an exposure period applied to image sensor 1032 is constant at a baseline level E=EB irrespective of an active lens setting.
Referring to sub-profiles B2 and B3, with sub-profile B2 active, exposure period parameters applied to image sensor 1032 of the terminal 1000 are not constant but rather are determined according to an algorithm. With sub-profile B2 active, an applied exposure period value can have an initial value, E0 established at a predetermined level with subsequent values Ei, (the value for each frame after the initial frame) determined on a frame by frame basis. As seen from Table B, an initial exposure period, E0, which can refer to the exposure period for a frame during a certain operator activated read attempt can vary depending on the lens setting. That is, referring to sub-profile B2, an initial applied exposure period parameter value at the far lens setting is E0=EF, at the immediate lens setting, E0=EB, at the near lens setting E0=EN and at the near contact setting E0=ENC, as is indicated in Table B.
Referring to subsequent exposure periods for subsequent frames after an initial frame, the subsequent exposure periods can be determined according to function f1. Referring to function fE1, function fE1 is a function of the applied exposure period, E=Ei-1, for the most recent frame and the white level, Wi-1, for the most recent frame available for processing by CPU 1060. A white level for a frame may be calculated as an average of pixel values at pixel positions of a frame of image data. In one embodiment, an exposure parameter value, E, for a next frame may be determined by increasing the applied exposure, applied during the most recent exposure period by a predetermined amount if Wi-1 is below a target value, and decreasing that amount by a predetermined amount if the parameter Wi-1 is above a predetermined target value. In one embodiment, with reference to sub-profile B2, the most recent frame for purposes of determining an exposure period for a next frame may not be regarded as the most recent overall frame available for processing associated with the present operator activated read attempt, but rather the most recent frame exposed with the present lens setting active during the present operator activated read attempt. Otherwise, the most recently applied parameter value (e.g., exposure value) may not be the overall most recently applied parameter value during a present read attempt, but rather the most recently applied parameter value with the present lens setting active during the present read attempt. Thus referring to Configuration 6 in Table A, the most recent frame relative to frame 9 for purposes of calculation of an exposure period may not be regarded as being frame 8, but rather frame 5, the most recent frame available for processing with the present lens setting active. Also, the most recently applied gain parameter value may be regarded as not being the gain applied for frame 8, but rather for frame 5. It is seen, therefore, that terminal 1000 can be operative when determining an operator parameter for a certain frame having an associated certain lens setting can preferentially utilize frames having the same lens setting. In one example of such preferential utilization as set forth herein in the described example, terminal 1000 can discard (ignore) data of frames having associated lens settings that are different from the certain lens setting. Referring still to the example of sub-profile B2, the same algorithm expressed as the function f=fE1 may be active for determination of an applied exposure period irrespective of the current lens setting. However, the applied exposure at each lens setting may be quite different, first, because the initial exposure period, E0 is different for each lens setting with sub-profile B2 active, and second because different processes may be active for determining a subsequently applied i.e., after an initial frame exposure parameter for each particular lens setting.
Different programs may be run by CPU 1060 for determination of an energization level to apply to a light emitting light source with each respective lens setting active. Terminal 1000 can be operative so that CPU 1060 executes a different processing thread for each exposure period determination process it is currently executing.
Referring now to sub-profile B3, sub-profile B3 is similar to sub-profile B2 except that in addition to running different exposure parameter determination processes for each respective lens setting, the algorithms by which the processes are run are differentiated between various lens settings. Referring to the example of sub-profile B3, the active algorithm with the first lens setting active is expressed by the function f=fE1. The active algorithm with the intermediate lens setting active is expressed by the function f=fE2, the active algorithm corresponding to the near optimum focus distance lens setting is expressed by the function f=fE3 and the active algorithm corresponding to the near plane of optimum focus setting is expressed by the function f=fE4, where fE4 depends only on a white level of a most recent frame but not on an applied exposure period of a most recent frame available for processing associated to the present operator activated read attempt. In such an embodiment, an exposure period parameter can be determined by looking up a value from a lookup table, rather than incrementing or decrementing a recently applied value.
Turning now to sub-profiles C0, C1, C2, and C3, the sub-profiles C0, C1, C2, and C3 relate to the control of an amplifier gain for input to amplifier 1036 for amplification of frame image signals. In the development of terminal 1000, it was determined that it might be useful in some applications to vary a control of an amplifier gain with the current lens setting.
Referring to sub-profile C1, a higher gain level for a terminal amplifier gain can be applied at farther lens settings irrespective of the actual terminal to target distance and lower gain levels can be applied at shorter plane of optimum focus lens settings irrespective of the actual terminal to target distance. Thus, with farther plane of optimum focus lens settings there can be applied additional amplification of frame image signals. By contrast, referring to sub-profile C0, with sub-profile C0 active, gain applied to frame image signals is constant at the baseline level G=GB for each possible lens setting. When sub-profile C0 is active, the amplifier gain (since it remains the same irrespective of lens settings), can be regarded as being disassociated from the lens setting.
Referring to sub-profiles C2 and C3, with sub-profile C2 active, the gain applied is determined according to an algorithm. With sub-profile C2 active, terminal 1000 can have an initial value G0 and subsequent values Gi (the gain for each subsequent set of frame image signals) determined on a frame by frame basis. As seen from Table B, the initial applied gain value G0 can vary depending on the lens setting. That is, referring to sub-profile C2, an initial applied gain level at the far lens setting is G0=GF, at the immediate lens setting, G0=GB, at the near lens setting G0=GN and at the near contact setting G0=GNC as is indicated in Table B. Referring to subsequent gain levels for subsequent scans after an initial frame, the subsequent gain levels can be determined according to function fG1. Referring to function fG1, function fG1, is a function of the applied gain level for the previous scan G=Gi-1 and the white level, the white level of the most recent frame available for proofing by CPU 1060. In one embodiment, with reference to sub-profile C2, the most recent frame may not be regarded as the most recent overall frame captured during a present operator activated read attempt, but rather the most recent frame captured during the present operator activated read attempt with the present lens setting active. Thus referring to configuration 6 in Table A, the most recent frame relative to frame 9 for purposes of calculation of a gain level may be regarded not as being frame 8, but rather frame 5, the most recent scan with the present lens setting active. Similarly, the most recently applied gain G=Gi-1 may be regarded not as the overall most recently applied gain, but the most recently applied gain with the present lens setting active. Thus, in the example of Configuration 6, the gain of frame 6, not frame 8, can be regarded as the most recently applied gain. Referring still to the example of sub-profile C2, the same algorithm expressed as the function f=fG1 may be active for determination of an applied gain irrespective of the current lens setting. However, the applied gain at each lens setting may be quite different, first, because the initial energization level, G0 is different for each lens setting with sub-profile C2 active, and second because difference processes may be active for determining a subsequently applied i.e., after an initial gain level for each particular lens setting.
Different programs may be run by CPU 1060 for determination of a gain level to apply to a light emitting light source with each respective lens setting active. Terminal 1000 can be operative so that CPU 1060 executes a different processing thread for each gain level determination process it is currently executing.
Referring now to sub-profile C3, sub-profile C3 is similar to sub-profile C2 except that in addition to running different gain determination processes for each respective lens setting, the algorithms by which the processes are run are differentiated between various lens settings. The gain determination algorithms active with the far lens setting active can be f=fG1. The active algorithm with the intermediate lens setting active can be expressed by the function f=fG2. The gain determination algorithm corresponding to the near optimum focus distance lens setting can be expressed by the function f=fG3 and the algorithm active corresponding to the near contact optimum focus setting if the function f=fG4 where fG4 depends only on a white level of a most recent frame but not on a most recently applied gain. In such an embodiment, a gain for a present frame can be determined by reference to a lookup table without reference to a previously applied gain.
A small sample of systems methods and apparatus that are described herein is as follows:
an image sensor including an image sensor array having a plurality of pixels;
a variable setting lens assembly for focusing light on the image sensor array, the variable setting lens assembly having a first lens setting at which the terminal has a first plane of optimum focus and a second lens setting at which the terminal has a second plane of optimum focus;
wherein the indicia reading terminal is operative to expose a first frame with the lens assembly at the first lens setting and is further operative to expose a second frame with the lens assembly at the second lens setting;
wherein the indicia reading terminal is further operative to attempt to decode a decodable indicia by processing at least one of the first frame and second frame;
wherein the indicia reading terminal is further operative so that there is associated with the first lens setting a first picture size and further so that there is associated with the second lens setting a second picture size so that a frame exposed with the lens assembly at the first lens setting has a first picture size and further so that a frame exposed with the lens assembly at the second lens setting has a second picture size.
an image sensor including an image sensor array having a plurality of pixels;
a variable setting lens assembly for focusing light on the image sensor array, the variable setting lens assembly having a first lens setting at which the terminal has a first plane of optimum focus and a second lens setting at which the terminal has a second plane of optimum focus;
wherein the indicia reading terminal is operative to expose a first frame with the lens assembly at the first lens setting and is further operative to expose a second frame with the lens assembly at the second lens setting;
wherein the indicia reading terminal is further operative to attempt to decode a decodable indicia by processing at least one of the first frame and second frame;
wherein there is associated with the first lens setting a first at least one control, and wherein there is associated with the second lens setting a second at least one control, the first at least one control selected from the group consisting of an operational parameter and a process for determining an operational parameter, the second at least one control selected from the group consisting of an operational parameter and a process for determining an operational parameter.
an image sensor including an image sensor array having a plurality of pixels;
a variable setting lens assembly for focusing light on the image sensor array, the variable setting lens assembly having a first lens setting at which the terminal has a first plane of optimum focus and a second lens setting at which the terminal has a second plane of optimum focus;
wherein the indicia reading terminal is operative in a first operator selectable operating state and a second operator selectable operating state;
wherein the indicia reading terminal during an operator initiated read attempt in the first operator selectable operating state is operative to vary the lens setting of the lens assembly between a first lens setting and a second lens setting, wherein the indicia reading terminal is further operative in the first operator selectable operating state so that frames exposed with the lens assembly at the first lens setting have a first picture size and further so that frames exposed with the lens assembly at the second lens setting have a second picture size different from the first picture size;
wherein the indicia reading terminal during an operator initiated read attempt in the second operator selectable operating state is operative to vary the lens setting of the lens assembly between a first lens setting and a second lens setting, wherein the indicia reading terminal is further operative in the second operator selectable operating state so that frames exposed with the lens assembly at the first lens setting and frames exposed with the lens assembly at the second lens setting have a common picture size.
an image sensor including an image sensor array having a plurality of pixels;
a variable setting lens assembly for focusing light on the image sensor array, the variable setting lens assembly having a first lens setting at which the terminal has a first plane of optimum focus and a second lens setting at which the terminal has a second plane of optimum focus;
wherein the indicia reading terminal is operative to expose a first frame with the lens assembly at the first lens setting and is further operative to expose a second frame with the lens assembly at the second lens setting;
wherein the indicia reading terminal is further operative to attempt to decode a decodable indicia by processing at least one of the first frame and second frame;
wherein the indicia reading terminal is operative so that during an operator initiated read attempt the terminal exposes a first plurality of frames with the lens assembly at the first lens setting and a second plurality of frames with the lens assembly at the second lens setting, wherein the terminal is further operative so that during the operator initiated read attempt the terminal switches a lens setting of the lens assembly from the first lens setting to the second lens setting and back to the first lens setting so that there are exposed a plurality of frames with an alternating pattern of lens setting associations;
wherein the indicia reading terminal is further operative so that when executing a process for determining an operational parameter to be applied for a next frame with the lens setting at the first lens setting, the terminal preferentially utilizes a prior frame having as an associated lens setting the first lens setting.
While the present invention has been described with reference to a number of specific embodiments, it will be understood that the true spirit and scope of the invention should be determined only with respect to claims that can be supported by the present specification. Further, while in numerous cases herein wherein systems and apparatuses and methods are described as having a certain number of elements it will be understood that such systems, apparatuses and methods can be practiced with fewer than the mentioned certain number of elements. Also, while a number of particular embodiments have been described, it will be understood that features and aspects that have been described with reference to each particular embodiment can be used with each remaining particularly described embodiment.
Havens, William H., Meier, Timothy P.
| Patent | Priority | Assignee | Title |
| Patent | Priority | Assignee | Title |
| 5378883, | Jul 19 1991 | Omniplanar Inc. | Omnidirectional wide range hand held bar code reader |
| 5576529, | Oct 08 1986 | INERMEC IP CORP ; Intermec IP CORP | Hand-held optically readable information set reader focus with operation over a range of distances |
| 5756981, | Feb 27 1992 | Symbol Technologies, Inc | Optical scanner for reading and decoding one- and-two-dimensional symbologies at variable depths of field including memory efficient high speed image processing means and high accuracy image analysis means |
| 5811828, | Sep 17 1991 | Intermec IP CORP | Portable reader system having an adjustable optical focusing means for reading optical information over a substantial range of distances |
| 5814803, | Dec 23 1994 | PSC SCANNING, INC | Image reader with multi-focus lens |
| 6123261, | May 05 1997 | Symbol Technologies, LLC | Optical scanner and image reader for reading images and decoding optical information including one and two dimensional symbologies at variable depth of field |
| 6611060, | Oct 04 1999 | TOSHIBA MEMORY CORPORATION | Semiconductor device having a damascene type wiring layer |
| 7227117, | May 30 2006 | Symbol Technologies, LLC | High speed auto-exposure control |
| 7240844, | Nov 13 2003 | Metrologic Instruments, Inc. | Hand-suportable imaging-based bar code symbol reader employing an automatic light exposure measurement and illumination control subsystem for measuring illumination exposure on CMOS image sensing array and controlling LED illumination array driver circuitry |
| 7762466, | Dec 18 2008 | Symbol Technologies, Inc. | Two position zoom lens assembly for an imaging-based bar code reader |
| 7918398, | Jun 04 2007 | HAND HELD PRODUCTS, INC | Indicia reading terminal having multiple setting imaging lens |
| 20040118928, | |||
| 20060011724, | |||
| 20060202036, | |||
| 20060202038, | |||
| 20070063048, | |||
| 20070069030, | |||
| 20070102520, | |||
| 20070164115, | |||
| 20080245872, | |||
| 20090072037, | |||
| 20090108071, |
| Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
| Aug 12 2009 | Hand Held Products, Inc. | (assignment on the face of the patent) | / |
| Date | Maintenance Fee Events |
| Date | Maintenance Schedule |
| Apr 03 2015 | 4 years fee payment window open |
| Oct 03 2015 | 6 months grace period start (w surcharge) |
| Apr 03 2016 | patent expiry (for year 4) |
| Apr 03 2018 | 2 years to revive unintentionally abandoned end. (for year 4) |
| Apr 03 2019 | 8 years fee payment window open |
| Oct 03 2019 | 6 months grace period start (w surcharge) |
| Apr 03 2020 | patent expiry (for year 8) |
| Apr 03 2022 | 2 years to revive unintentionally abandoned end. (for year 8) |
| Apr 03 2023 | 12 years fee payment window open |
| Oct 03 2023 | 6 months grace period start (w surcharge) |
| Apr 03 2024 | patent expiry (for year 12) |
| Apr 03 2026 | 2 years to revive unintentionally abandoned end. (for year 12) |