The disclosure provides a method and a system for multi-lens module alignment, adapted to perform alignment on a multi-lens module having at least a first lens and a second lens during its manufacturing stage, where the method includes the following steps. A calibration object is first captured by using the first lens and the second lens to accordingly generate intrinsic parameters and external parameters of the first lens and the second lens, where the external parameters of the first lens and the second lens are associated with a common reference coordinate system. A target object is next captured by using the first lens and the second lens, and captured images of the target object are processed by using the intrinsic parameters and the external parameters to generate rectified images. The first lens and the second lens are adjusted and aligned according to the rectified images.
|
1. A method for multi-lens module alignment, adapted to perform alignment on a multi-lens module comprising a first lens and a second lens during manufacture, wherein the method comprises the following steps:
capturing a calibration object by using the first lens and the second lens to accordingly generate a first calibration parameter set corresponding to the first lens and a second calibration parameter set corresponding to the second lens, wherein the first calibration parameter set comprises an intrinsic parameter and an external parameter of the first lens, wherein the second calibration parameter set comprises an intrinsic parameter and an external parameter of the second lens, and wherein the external parameter of the first lens and the external parameter of the second lens are associated with a common reference coordinate system;
capturing a target object by using the first lens and the second lens, and processing images of the target object respectively captured by the first lens and the second lens by using the first calibration parameter set and the second calibration parameter set to generate rectified images; and
adjusting and aligning the first lens and the second lens according to the rectified images.
12. A system for multi-lens module alignment, adapted to perform alignment on a multi-lens module comprising a first lens and a second lens during manufacture, wherein the system comprises:
a calibration object;
a target object;
an alignment platform, configured to place the multi-lens module to be adjusted and aligned and comprising a lens adjusting mechanism configured to adjust the first lens and the second lens;
an image processing device, connected to the first lens and the second lens, configured to process images of the calibration object respectively captured by the first lens and the second lens to accordingly generate a first calibration parameter set corresponding to the first lens and a second calibration parameter set corresponding to the second lens, and configured to process images of the target object respectively captured by the first lens and the second lens by using the first calibration parameter set and the second calibration parameter set to generate rectified images, wherein the first calibration parameter set comprises an intrinsic parameter and an external parameter of the first lens, wherein the second calibration parameter set comprises an intrinsic parameter and an external parameter of the second lens, and wherein the external parameter of the first lens and the external parameter of the second lens are associated with a common reference coordinate system; and
a display, connected to the image processing device, and configured to display the rectified images as references to adjust and align the first lens and the second lens.
2. The method according to
setting each of the rectified images as a synthetic target image; and
continuously adjusting and aligning the first lens and the second lens and capturing the target object by using the first lens and the second lens until images respectively captured by the first lens and the second lens match the synthetic target images.
3. The method according to
for the first lens:
capturing the target object by using the first lens to generate a first target object image;
determining whether the first target object image matches the first synthetic target image; and
if not, adjusting the first lens and capturing the target object by using the adjusted first lens to re-generate the first target object image; and
for the second lens:
capturing the target object by using the second lens to generate a second target image;
determining whether the second target object image matches the second synthetic target image; and
if not, adjusting the second lens and capturing the target object by using the adjusted second lens to re-generate the second target object image.
4. The method according to
rotating the first lens with respect to a lens center thereof so as to adjust an optical axis of the first lens; and
rotating the second lens with respect to a lens center thereof so as to adjust an optical axis of the second lens.
5. The method according to
aligning the lens center of the first lens and the lens center of the second lens.
6. The method according to
determining whether features for comparison in the first rectified image and the second rectified image satisfy a determination standard; and
if no, aligning the first lens and the second lens to re-capture the target object, and processing images of the target object respectively captured by the first lens and the second lens by using the first calibration parameter set and the second calibration parameter set to re-generate the first rectified image and the second rectified image.
7. The method according to
translating at least one of the first lens and the second lens so as to align a lens center of the first lens and a lens center of the second lens.
8. The method according to
moving at least one of the first lens and the second lens in a back-and-forth direction.
9. The method according to
moving at least one of the first lens and the second lens in a vertical direction.
10. The method according to
moving at least one of the first lens and the second lens in a horizontal direction according to an object distance of the target object.
11. The method according to
adjusting an optical axis of the first lens and an optical axis of the second lens.
13. The system according to
14. The system according to
15. The system according to
16. The system according to
17. The system according to
18. The system according to
19. The system according to
20. The system according to
|
This application claims the priority benefit of Taiwan application serial no. 105119829, filed on Jun. 24, 2016. The entirety of the above-mentioned patent application is hereby incorporated by reference herein and made a part of this specification.
The disclosure relates to a method and a system for multi-lens module alignment, in particular, to a method and a system for multi-lens module alignment during manufacture.
With development in technology, various smart image capturing devices, such as tablet computers, personal digital assistants and smart phones, have become indispensable tools for people nowadays. Camera lenses equipped in high-end smart mobile image capturing devices provide same or better specifications than those of traditional consumer cameras, and some even provide three-dimensional image capturing features or near-equivalent pixel qualities to those of digital single lens reflex cameras.
Ideally, two optical axes of a dual-lens module are supposed to be parallel to each other, and two lens centers are supposed to be perpendicular to a plane formed by the two optical axes. However, the optical axes and the lens centers are all invisible, and thus it is difficult for an inspector to ensure that the optical axes and the lens centers are precisely positioned in an assembly stage during manufacture.
Accordingly, a method and a system for multi-lens module alignment are proposed in the disclosure to ensure precision disposition of optical axes and lens centers of a multi-lens module in its assembly stage during manufacture.
According to one of the exemplary embodiments, the method for multi-lens module alignment is adapted to perform alignment on a multi-lens module having at least a first lens and a second lens during manufacture. The method includes the following steps. A calibration object is first captured by using the first lens and the second lens to accordingly generate a first calibration parameter set corresponding to the first lens and a second calibration parameter set corresponding to the second lens, where the first calibration parameter set includes an intrinsic parameter and an external parameter of the first lens, and the second calibration set includes an intrinsic parameter and an external parameter of the second lens, and the external parameters of the first lens and the second lens are associated with a common reference coordinate system. A target object is next captured by using the first lens and the second lens, and captured images of the target object are processed by using the first calibration parameter set and the second calibration parameter set to generate rectified images. The first lens and the second lens are adjusted and aligned according to the rectified images.
According to one of the exemplary embodiments, the system for multi-lens module alignment includes a calibration object, a target object, an alignment platform, an image processing device, and a display. The alignment platform including a lens adjusting mechanism is configured to place the multi-lens module to be adjusted and aligned, where the multi-lens module at least includes a first lens and a second lens. The image processing device is configured to generate a first calibration parameter set corresponding to the first lens and a second calibration parameter set corresponding to the second lens according to images of the calibration object respectively captured by the first lens and the second lens and also configured to process images of the target object respectively captured by the first lens and the second lens by using the first calibration parameter set and the second calibration parameter set to generate rectified images, where the first calibration parameter set includes an intrinsic parameter and an external parameter of the first lens, and the second calibration set includes an intrinsic parameter and an external parameter of the second lens, and the external parameters of the first lens and the second lens are associated with a common reference coordinate system. The display is connected to the image processing device and configured to display the rectified images provided as references to adjust and align the first lens and the second lens.
In summary, in the proposed disclosure, prior to assembly and gluing of a multi-lens module, images for comparison and references are generated by using intrinsic parameters and external parameters of the lenses so as to determine whether optical axes and lens centers of the multi-lens module are precisely located based on image visualization, image recognition and comparison, and thus such images could be provided as an indication of whether to adjust and align the lenses in the assembly stage during manufacture.
In order to make the aforementioned features and advantages of the present disclosure comprehensible, preferred embodiments accompanied with figures are described in detail below. It is to be understood that both the foregoing general description and the following detailed description are exemplary, and are intended to provide further explanation of the disclosure as claimed.
It should be understood, however, that this summary may not contain all of the aspect and embodiments of the present disclosure and is therefore not meant to be limiting or restrictive in any manner. Also the present disclosure would include improvements and modifications which are obvious to one skilled in the art.
The accompanying drawings are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments of the disclosure and, together with the description, serve to explain the principles of the disclosure.
To make the above features and advantages of the application more comprehensible, several embodiments accompanied with drawings are described in detail as follows.
Some embodiments of the disclosure will now be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all embodiments of the application are shown. Indeed, various embodiments of the disclosure may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. Like reference numerals refer to like elements throughout.
Referring to
The alignment platform 110 is configured to place the multi-lens module DL for alignment and includes a lens adjusting mechanism 115. The lens adjusting mechanism 115 could include a rotation mechanism and a translation mechanism configured to rotate and translate the first lens L1 and the second lens L2 by piezoelectric or mechanical control to adjust and align the multi-lens module DL.
The image processing device 120 could be an industrial computer, a personal computer, a laptop computer, a tabular computer, and so forth. The disclosure is not limited in this regard. The image processing device 120 would obtain images detected by sensing elements of the first lens L1 and the second lens L2 through wired or wireless transmission and perform image processing. The sensing elements could be charge coupled device (CCD) elements, complementary metal-oxide semiconductor (COMS) elements.
It should be understood by the person skilled in the art that the image processing device 120 would include a processor and a data storage device. The processor could be a central processing unit (CPI), a digital signal processor (DSP), a programmable controller, an application specific integrated circuit (ASIC), a programmable logic device (PLD), other similar devices or a combination of above-mentioned devices. The data storage device could be, one or a combination of a stationary or mobile random access memory (RAM), a read-only memory (ROM), a flash memory, a hard drive, other similar devices or a combination of above-mentioned devices.
The display 130 would obtain images detected by sensing elements of the first lens L1 and the second lens L2 as well as images processed by the image processing device 120 through wired or wireless transmission and display the obtained images for the user. In the present exemplary embodiment, the display 130 could be, for example, a liquid crystal display (LCD), a light-emitting diode (LED) display, a field emission display (FED) or other types of displays. Moreover, in another exemplary embodiment, the display 130 and the image processing device 120 could be integrated into an image processing device with a screen. The disclosure is not limited in this regard.
The calibration object C and the target object T would be respectively placed in front of the multi-lens module DL in different stages of the alignment process. In the present exemplary embodiment, the calibration object C could be a calibration board with a check pattern or other patterns easy to be recognized. The target object T could be an object which is the same as or different from the calibration object C. The disclosure is not limited in this regard.
Detailed steps of multi-lens module alignment during manufacture would be illustrated along with each component of the system 100 hereafter. As a pre-processing step of the following exemplary embodiments, the multi-lens module DL to be aligned has been placed on the adjustment platform 110.
Referring to both
In general, the intrinsic parameters would be used to describe a transformation between camera coordinates and image coordinates. That is, the camera coordinates may be projected onto a projective plane according to the pinhole imaging principle. The intrinsic parameters include focal length, image center, principal point, and distortion coefficients, which define internal geometric and optical characteristics of the first lens L1 and the second lens L2.
On the other hand, the external parameters would be used to describe a transformation between the world coordinates and the camera coordinates. That is, the positions and view directions of the first lens L1 and the second lens L2 in the 3D coordinates. The external parameters include rotation matrix and translation vector, which are related to disposition positions and view directions of the first lens L1 and the second lens L2. Since the external parameters describe the transformation between the camera coordinates of the first lens L1 and the second lens L2, the external parameters of the first lens L1 and the second lens L2 would correspond to a common reference coordinate system. In the present exemplary embodiment, such common reference coordinate system would be the world coordinate system. In another exemplary embodiment, it could also be another common reference coordinate system that corresponds to the external parameters of both the first lens L1 and the second lens L2.
Next, the target object T is captured by using the first lens L1 and the second lens L2 (Step S206) to respectively generate images of the target object T. The image processing device 120 would process the two captured images of the target object T by using the first calibration parameter set and the second calibration parameter set to generate rectified images (Step S208). Herein, the image processing device 120 would perform homography computation on the images of the target object T according to the first calibration parameter set and the second calibration parameter set to generate the rectified images. The purpose of the step is to estimate images of the target object T captured by the two lenses in an ideal condition (i.e. their two optical axes being parallel to each other and their lens centers being perpendicular to a plane formed by the two optical axes) according to the internal geometry and optical characteristics of the two lenses as well as the relative relationship between the two lenses. Such ideal images would be provided as references to adjust and align the lenses in the follow-up step.
Next, the display 130 would display the rectified images so as to provide the inspector of the multi-lens module DL to adjust and align the first lens L1 and the second lens through the lens adjusting mechanism 115 (Step S210). The adjustment and the alignment herein is to let the two optical axes be parallel to each other and their lens centers be perpendicular to a plane formed by the two optical axes through rotation and translation. More detailed explanation will be given later on.
Moreover, in another exemplary embodiment, the image processing device 120 would compute the required adjustment for the first lens L1 and the second lens L2 according to the rectified images based on image recognition and comparison, and then transmit a control command to the alignment platform 110 so as to control the lens adjusting mechanism 115 to perform alignment on the first lens L1 and the second lens L2.
Referring to
In the present embodiment, the display 130 would concurrently display the images of the target object T and the synthetic target images so as to provide the inspector of the multi-lens module DL to determine whether the images of the target object T match the synthetic target images (Step S310). Moreover, Step S310 of determining whether the images of the target object T match the synthetic target images could also be performed by the image processing device 120 based on image recognition and comparison. The disclosure is not limited in this regard.
When the images of the target object T do not match the synthetic target images, the first lens L1 and/or the second lens L2 would be rotated by using the lens adjusting mechanism 115 to adjust the optical axis of the first lens L1 and/or the optical axis of the second lens L2 (Step S312). Next, the target object T would be captured by the adjusted first lens L1 and the adjusted second lens L2 (Step S314), and the flow returns to Step S310 until the images of the target object T captured by the adjusted first lens L1 and the adjusted second lens L2 respectively match their synthetic target images. On the other hand, when the images of the target object T match the synthetic target images, the flow of multi-lens module DL alignment on optical axes would be completed. If the lens centers of the first lens L1 and the second lens L2 have not yet been adjusted prior to the flow of optical axis alignment as illustrated in
To be specific,
Referring first to
Next, referring to
Revisiting
Referring to
In the present embodiment, the display 130 would concurrently display the first rectified image and the second rectified image so as to provide the inspector of the multi-lens module DL to determine whether features for comparison in the first rectified image and the second rectified image satisfy a determination standard (Step S410). Similarly, Step S410 of determining whether the features for comparison in the first rectified image and the second rectified image satisfy a determination standard could also be performed by the image processing device 120 based on image recognition and comparison. The disclosure is not limited in this regard.
When the features for comparison in the first rectified image and the second rectified image do not satisfy the determination standard, the first lens L1 and/or the second lens L2 would be translated by the lens adjusting mechanism 115 so as to adjust the lens center of the first lens L1 and the lens center of the second lens L2 (Step S412). The features for comparison could be sizes, horizontal lines, parallax of the two rectified images, and whether the lens centers of the first lens L1 and the second lens L2 are shifted back and forth, left and right, and up and down. Next, the flow returns to Step S406 until the features for comparison in the first rectified image and the second rectified image generated through image capturing by the adjusted first lens L1 and the second lens L2 and rectification satisfy the determination standard.
On the other hand, when the features for comparison in the first rectified image and the second rectified image satisfy the determination standard, the flow of multi-lens module DL alignment on lens centers would be completed. If the optical axes of the first lens L1 and the second lens L2 have not yet been adjusted prior to the flow of lens center alignment as illustrated in
To be specific,
Referring first to
Next, referring to
Next, whether a horizontal line d1 of the first rectified image RT1 is aligned to a horizontal line d2 of the second rectified image RT2, or whether same feature points respectively in the first rectified image RT1 and the second rectified image RT2 lie on a same horizontal line would be determined so as to determine whether the horizontal lines in the two rectified images are aligned. If they are not aligned, the first lens L1 and the second lens L2 would be translated by the lens adjusting mechanism 115 in a vertical direction until the horizontal lines d1 and d2 are aligned.
Next, whether an actual parallax (x1-x2) between the first rectified image RT1 and the second rectified image RT2 satisfies an expected parallax could be determined according to an object distance D of the target object T based on Eq.(1):
where d denotes the expected parallax, F denotes a focus length of the main lens, B denotes the distance between two lens centers (i.e. the length of a baseline), and D denotes the distance between an object and the lens. When the actual parallax does not satisfy the expected parallax, the first lens L1 and the second lens L2 would be adjusted by the lens adjusting mechanism 115 in a horizontal direction. That is, the length of the baseline would be adjusted, and the actual parallax (x1-x2) between a first rectified image RT1′ and a second rectified image RT2′ that satisfies the expected parallax would be thus generated.
Revisiting
In summary, in the method and system for multi-lens module alignment proposed in the disclosure, prior to assembly and gluing of a multi-lens module, images for comparison and references are generated by using intrinsic parameters and external parameters of the lenses so as to determine whether optical axes and lens centers of the multi-lens module are precisely located based on image visualization, image recognition and comparison, and thereby determine whether to adjust the lenses through rotation or translation. Hence, the disclosure ensures precision disposition of optical axes and lens centers of a multi-lens module provides high-quality assembly and testing during manufacture.
No element, act, or instruction used in the detailed description of disclosed embodiments of the present application should be construed as absolutely critical or essential to the present disclosure unless explicitly described as such. Also, as used herein, each of the indefinite articles “a” and “an” could include more than one item. If only one item is intended, the terms “a single” or similar languages would be used. Furthermore, the terms “any of” followed by a listing of a plurality of items and/or a plurality of categories of items, as used herein, are intended to include “any of”, “any combination of”, “any multiple of”, and/or “any combination of multiples of the items and/or the categories of items, individually or in conjunction with other items and/or other categories of items. Further, as used herein, the term “set” is intended to include any number of items, including zero. Further, as used herein, the term “number” is intended to include any number, including zero.
It will be apparent to those skilled in the art that various modifications and variations can be made to the structure of the disclosed embodiments without departing from the scope or spirit of the disclosure. In view of the foregoing, it is intended that the disclosure cover modifications and variations of this disclosure provided they fall within the scope of the following claims and their equivalents.
Wei, Shou-Te, Huang, Yu-Wei, Huang, Chi-Ying, Wang, Yu-Chih, Yang, Fu-Kui
Patent | Priority | Assignee | Title |
Patent | Priority | Assignee | Title |
9113083, | Apr 30 2014 | ALTEK SEMICONDUCTOR CORPORATION | Image adjusting system with multiple lens modules and method thereof |
20150092089, | |||
20150109418, | |||
CN104730802, | |||
CN202631858, | |||
TW201312080, | |||
TW201541961, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Aug 26 2016 | WANG, YU-CHIH | ALTEK SEMICONDUCTOR CORP | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 039692 | /0887 | |
Aug 26 2016 | WEI, SHOU-TE | ALTEK SEMICONDUCTOR CORP | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 039692 | /0887 | |
Aug 26 2016 | HUANG, YU-WEI | ALTEK SEMICONDUCTOR CORP | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 039692 | /0887 | |
Aug 26 2016 | YANG, FU-KUI | ALTEK SEMICONDUCTOR CORP | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 039692 | /0887 | |
Aug 26 2016 | HUANG, CHI-YING | ALTEK SEMICONDUCTOR CORP | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 039692 | /0887 | |
Sep 06 2016 | Altek Semiconductor Corp. | (assignment on the face of the patent) | / |
Date | Maintenance Fee Events |
Nov 30 2021 | M2551: Payment of Maintenance Fee, 4th Yr, Small Entity. |
Date | Maintenance Schedule |
Jun 12 2021 | 4 years fee payment window open |
Dec 12 2021 | 6 months grace period start (w surcharge) |
Jun 12 2022 | patent expiry (for year 4) |
Jun 12 2024 | 2 years to revive unintentionally abandoned end. (for year 4) |
Jun 12 2025 | 8 years fee payment window open |
Dec 12 2025 | 6 months grace period start (w surcharge) |
Jun 12 2026 | patent expiry (for year 8) |
Jun 12 2028 | 2 years to revive unintentionally abandoned end. (for year 8) |
Jun 12 2029 | 12 years fee payment window open |
Dec 12 2029 | 6 months grace period start (w surcharge) |
Jun 12 2030 | patent expiry (for year 12) |
Jun 12 2032 | 2 years to revive unintentionally abandoned end. (for year 12) |