Title of Invention	SYSTEM AND METHOD FOR RECONSTRUCTING A THREE-DIMENSIONAL SURFACE
Abstract	The present invention provides a system and for reconstruction of a three-dimensional surface (14). The proposed system (10) comprises a line scan camera (30) and includes a dynamic beam projection means (12) adapted for synthesizing an inverse curve (152) corresponding to each strip, of successive strips of said three-dimensional surface (14), such that when a radiation beam (20) having a beam profile (34) in the form of said inverse curve (152) is projected by said dynamic beam projection means (12) on said strip of said three-dimensional surface (14), the projection (28) of the radiation beam (20) appears as a straight line from the position of said line scan camera (30). The dynamic beam projections means (12) thus obtains synthesized inverse curves (152) for successive strips of said three-dimensional surface (14). The proposed system (10) further includes reconstruction means (32) adapted for generating a virtual straight line (158) across said synthesized inverse curves (152) and obtaining a depth curve (156) based upon a correspondence between points on said inverse curve (152) and said straight lines (154) seen by said line scan camera (30). The reconstruction means (32) then transforms depth curves (156) obtained for successive strips of said three-dimensional surface (14) perspective projections and stacks said transformed depth curves (154) to obtain a three-dimensional surface map.

Title of Invention

SYSTEM AND METHOD FOR RECONSTRUCTING A THREE-DIMENSIONAL SURFACE

Abstract

The present invention provides a system and for reconstruction of a three-dimensional surface (14). The proposed system (10) comprises a line scan camera (30) and includes a dynamic beam projection means (12) adapted for synthesizing an inverse curve (152) corresponding to each strip, of successive strips of said three-dimensional surface (14), such that when a radiation beam (20) having a beam profile (34) in the form of said inverse curve (152) is projected by said dynamic beam projection means (12) on said strip of said three-dimensional surface (14), the projection (28) of the radiation beam (20) appears as a straight line from the position of said line scan camera (30). The dynamic beam projections means (12) thus obtains synthesized inverse curves (152) for successive strips of said three-dimensional surface (14). The proposed system (10) further includes reconstruction means (32) adapted for generating a virtual straight line (158) across said synthesized inverse curves (152) and obtaining a depth curve (156) based upon a correspondence between points on said inverse curve (152) and said straight lines (154) seen by said line scan camera (30). The reconstruction means (32) then transforms depth curves (156) obtained for successive strips of said three-dimensional surface (14) perspective projections and stacks said transformed depth curves (154) to obtain a three-dimensional surface map.

Full Text	Description System and method for reconstructing a three-dimensional surface The present invention relates to a system and method for reconstructing a three-dimensional surface. The problem of reconstructing a three-dimensional surface can be solved by using a pair of cameras (for example, as disclosed in U.S Patent No. 6950121). The system described in U.S Patent No. 6950121 is fast but suffers from drawbacks in resolution for contrast starved regions and images where detecting feature points for registration becomes difficult. Also, such a system is prohibitively coarse for fine machine vision inspection purposes. Other systems following a similar philosophy have been used (for example, as disclosed in U.S Patent No. 7142281). Depth estimation can also be done using 'Depth from Defocus' systems which also provide low resolution data. Also, there are studies comparing 'Depth from defocus' and stereo imaging and there is some ambiguity as to whether they are really different. The above problem has also been solved using high speed acquisition system and a triangulated laser beam. Various solutions have been implemented to address each issue at different levels of success. All prior-art. inspection systems require a high speed area scan camera and a laser. The laser line obtained from a beam shaper or triangulation using optical assemblies is made incident on the inspection surface (Downing et al, 5054918). A camera scans the laser profile as the object under inspection is made to pass under the incident laser line. The distortion in the laser pattern provides depth information which is subsequently converted to 3D surface structure data. In a conventional system, a light sheet emitter emits a sheet of light, preferably laser triangulated beam on a three- dimensional surface. The light sheet forms a projected curve which is read by the camera to form a curve in the image. The light sheet scans over the entire surface to obtain information which can be later compared for inspection purposes. The fundamental drawback with such a system is that it requires a huge acquisition time and transfers a large amount of unnecessary data for a mapping. Our system will circumvent such problems and simultaneously provide a device which is high speed, efficient (in terms of information transfer and processing) and cost effective. An adaptation of this technique is the digital fringe projection which involves projecting multiple fringes thereby reducing time of 3D data acquisition. However, technological impediments in fringe projection reduce phase measurement accuracy ('Digital fringe projection in 3D shape measurement: An error analysis' Notni, G.H: Conference "Optical Measurement Systems for Industrial Inspection" 2003 ) The object of the present invention is to provide an improved system and method for reconstructing a three-dimensional surface. The above object is achieved by a method according to claim 1 and a system according to claim 4. The underlying idea of the present invention is to provide a system that will be able to capture 3D surface data as well as intensity of radiation beam at a high speed. The Line scan camera operates fast, hence iterations are performed quicker. In one embodiment, synthesizing said inverse curve for each strip of the three-dimensional surface further comprises: - projecting a radiation beam having an initial beam profile on that strip, - capturing an image of a projection of said beam on said strip by said line scan camera, and obtaining a resultant profile from the image of the projection of said radiation beam on said strip, and - iteratively modifying the beam profile of the radiation beam to arrive at said inverse curve that defines a final beam profile for said radiation beam for which the resultant profile obtained from said image of the projection of said radiation beam is a straight line, based upon a feedback relationship between the beam profile of the radiation beam and the resultant profile obtained from the image of the projection of the radiation beam on said strip. This involves mathematical computations that can advantageously be carried out with appropriate computer software. In a further embodiment, obtaining a resultant profile from the image of the projection of said radiation beam on said strip further comprises: - obtaining an intermediate profile from the image of the projection of said radiation beam on said strip captured by said line scan camera, and - spline fitting discontinuous portions of said intermediate profile to obtain a continuous resultant profile. In a preferred embodiment, said dynamic beam projection means includes a programmable liquid crystal display (LCD) projector. In one embodiment, said radiation beam (20) comprises a laser beam. Using a laser beam is useful in inspection of irregularities dark surfaces having very little contrast in intensity. In another embodiment, said radiation beam (20) comprises an x-ray beam. An x-ray beam has lower wavelength and is useful in measuring surface irregularities of extremely small dimensions. In one form of the present invention, the proposed system further comprises a rotatable mirror adapted for reflecting the projection of the radiation beam on a strip of said three-dimensional surface toward said line scan camera such that rotation of said rotatable mirror enables scanning of successive strips of said three-dimensional surface. The present invention is further described hereinafter with reference to illustrated embodiments shown in the accompanying drawings, in which: FIG 1 is a schematic diagram of a system for reconstructing a three-dimensional surface, FIG 2 is a schematic drawing illustrating successive deformations of the radiation beam profile and the corresponding resultant profile of the projection of the beam in the image captured by the line scan camera, FIG 3 is a flowchart illustrating a method for synthesizing inverse curves according to one embodiment of the present invention, FIGS 4A, 4B and 4C respectively illustrate successive deformations of the radiation beam profile, the corresponding intermediate profiles of the projection of the beam as seen by a line scan cameras and resultant profiles after spline fitting of discontinuous portions in the intermediate profiles, FIG 5 is a schematic diagram of an exemplary arrangement of a rotatable mirror with a line scan camera for scanning the 3D surface, FIG 6 is a schematic drawing illustrating correspondence between points on the synthesized inverse curve and points on the straight line seen by the line scan camera, and FIG 7A and 7B respectively illustrate a virtual straight line drawn on the inverse curves of successive strips and the corresponding depth curve. Embodiments of the present invention provide an efficient and low cost three-dimensional camera system for obtaining intensity as well as high resolution three-dimensional surface information at realistic times. The surface can be arbitrary, having no regularity constraints. The proposed system integrates a dynamic beam projection beams, for example a programmable liquid crystal display (LCD) projector or a digital light processing (DLP™) projector, a line scan camera, and an adaptive beam reshaping algorithm. Conventional systems project a straight light sheet, capture the profiles using area scan cameras and use them for 3D reconstruction. The drawback of such systems is the unnecessary time consumed for acquisition of the whole image and high data transfer rates. Such drawbacks decrease the speed of execution and increase the cost of the system or limit the resolution. The proposed system is conceptualized inverse to the above paradigm. The objective here is to synthesize a curve which on incident on a section of the 3D surface appears as a straight line when seen from the line scan camera. Once the line scan camera sees the straight line, the synthesized curve now represents information about the surface. This process is repeated for different strips on the surface and combined to form a representative shape descriptor for the particular surface. The purpose of the dynamic beam projection means is to iteratively structure the incident light in a feedback loop with the line scan camera. Referring to FIG 1, a system 10 is shown for reconstructing a three-dimensional surface 14, which may be any arbitrarily curved surface. The system 10 includes a line scan camera 30 and dynamic beam projection means 12, for example a programmable LCD or DLP projector 16, which is coupled to a computing device 18 (for example, a PC) that executes the algorithm for synthesizing inverse curves for each of successive strips of the three-dimensional surface 14. In operation, the projector 16 projects a radiation beam 20 (in this case, a light beam) on a strip of the surface 14. The surface 14 is a three-dimensional surface, and hence, if the radiation beam 20 has a beam profile 22 (i.e, beam cross- section) in the form of a straight line, the projection 26 of the beam 20 on the strip of the 3D surface 14 would appear as a curved line to the line scan camera 30. The idea of the present invention is to modify or re-shape the beam profile of the beam 20 by synthesizing an inverse curve, such that when a radiation beam having a beam profile in the form of this inverse curve is projected on that strip of the 3D surface 14, the projection 28 as seen from the position of the line scan camera 30 is a straight line. The synthesized inverse curve profile 34 is stored. Similarly, inverse curves are synthesized and stored for successive strips of the 3D surface 14. In operation, a radiation beam 20 having an initial beam profile is projected on to the strip from the projector 16. An image of the projection of the beam 20 is captured by the line scan camera 30, and a resultant profile obtained from the image of that projection as captured by the line scan camera. The beam profile of the radiation beam 20 is then iteratively modified using feedback based mechanism between the projector 16 and the line scan camera 30, which reshapes the projected radiation beam 20 from the projector 16 such that the image of the projection of the radiation beam as captured by the line scan camera has a resultant profile which is a straight line. The beam profile projected by the projector which gives a resultant profile (captured by the line scan camera) as a straight line is in the form of a curve which is referred to herein as an inverse curve for that particular strip of the 3D surface. FIG 2 shows the successive deformations of the beam profiles vky (x) projected by the projector and the corresponding resultant profiles fky (x) obtained from the line scan camera for a particular strip of the 3D surface as the iterations progress. The arrow 40 indicates the direction of iterations. The iterations stop when the line scan camera sees a continuous and straight resultant profile 154. This deformed curve beam profile is stored as an inverse curve 152 for a particular strip of the 3D surface. The entire surface profiling is completed when all strips are traversed and their corresponding inverse curves are stored. It should be noted that the line scan camera would not be able to image complete profile of the projection of the radiation beam on a given strip of the 3D surface, and would see only a part of the profile. So, in the illustrated embodiment, an intermediate profile is obtained from the image of the projection of the beam as captured by the line scan camera, and then, discontinuous portions of this intermediate profile are spline fitted to obtain the resultant profile. This is explained referring to FIGS 4A, 4B and 4C. FIG 4A shows the deformation of the beam profile vky(x) along a direction of iteration indicated by the arrow 40. FIG 4B shows the corresponding intermediate profiles 72 from the images of the projection of the beam having the corresponding beam profiles as shown in FIG 4A. As can be seen, the intermediate profiles are discontinuous, since the line scan camera can capture only portions of the projection of the beam on a given strip of the 3D surface. FIG 4C shows the resultant profiles fky(x) obtained by spine fitting of discontinuous portions in the intermediate profiles shown in FIG 4B. The beam-reshaping algorithm is carried out based on a feedback relationship between the beam profile vky(x) shown in FIG 4A and the resultant profile fky(x) obtained from the line scan camera as shown in FIG 4C. FIG 3 is a flowchart showing an exemplary beam re-shaping algorithm 50 for synthesizing inverse curves for successive strips of the 3D surface. The algorithm 50 begins at block 52 when a target strip is selected on the 3D surface. Let *y' be a strip in consideration at a certain point of time. The strip at 'y' denotes the position along the scanning axis. For example, if the 3D surface is placed on a conveyor, as the object travels, the conveyor belt movement represents the scanning axis; as the machine part traverses, the 'y' value (or position) increments. The iteration carried for each 'y' is as follows. The iterative beam re-shaping algorithm is carried out as explained in FIG 2. The objective profile is what is sought after. In this case, the objective function is Φy(x) = h/2 , where 'h' is the height in pixels of the scanning area. At block 54, a radiation beam having an initial beam profile is projected on to the selected strip of the 3D surface. The initial beam profile is v°y (x), where 'x' runs along the line of pixels, represented by the line scan camera (for a standard line scan camera available, x would be a vector from 1:640 or 1:1024). At block 56, a discontinuous intermediate profile of the projection of the beam on the surface is captured by the line scan camera. The resultant profile obtained after spline fitting discontinuous portions of the intermediate profile is f°y (x). This information is fed to the iterative beam re-shaping algorithm (block 58). In the iterative algorithm (block 58), the new profile to be fed to the projector is iterated by The profile v1y (x) when projected on to the 3D surface would give a resultant profile as f1y (x). This iteration is carried out for every time step 't' as generalized as . The iterations are performed until the quantity is minimized. The parameter 'k' is a relaxation parameter enhancing the convergence rates and incorporates the mapping between the physical space and the image plane. Next, at block 60, the synthesized inverse curve corresponding to the beam profile vfinaly (x) is stored. At block 62, the next adjacent strip of the 3D surface is selected as the target strip and the above steps (blocks 52 to 60) are repeated. The algorithm 50 ends at block 64 by storing all the inverse curves that are synthesized. FIG 5 shows an exemplary embodiment illustrating the mechanics of the line scan camera 30. A rotating mirror (rotatable from position 106 to position 108) sweeps for a range of angles (represented by numerals 102 and 104) to enable vertical scan over a 3D test surface 14. The projector 16 projects a radiation beam 20 at a particular designated strip which is the focus strip as deflected by the scanning mirror at any instant of time such that the line scan sensor 30 is focused on the strip. In alternate embodiments, instead of light beams, other forms of radiation can be used. For example, the radiation beam may comprise laser beams. Using a laser beam is useful in inspection of irregularities dark surfaces having very little contrast in intensity. Still alternately, the radiation beam may comprise x-ray beams. X-ray beams have lower wavelength and is useful in measuring surface irregularities of extremely small dimensions. Referring back to FIG 1, the synthesized inverse curves are fed to the reconstruction means 32 to reconstruct the 3D surface. Herein, as shown in FIG 6, a correspondence is established between points on each inverse curve and points on the straight line of the resultant profile obtained from the line scan camera. As shown, the inverse curve 152 has points 160 in a regular fashion marked on it. The corresponding points on the straight line 154 seen by the line scan camera are noted; hence correspondences in points between the inverse curves 152 and the seen straight line 152 are determined. For different strips along the 'y' direction, different inverse curves 152 and the corresponding straight lines 154 seen by the line scan camera are shown in FIG 7A and FIG 7B respectively. Herein, inverse curves 152 obtained from scanning different strips with the corresponding points in the straight lines 154 are shown The next task involves obtaining the actual depth map from the inverse curves. It is shown that drawing a virtual straight line 158 on the set of inverse curves 152 (shown in FIG 7A). establishes the depth curve 156 (shown in FIG 7B) using, the point correspondence information explained with respect to FIG 6. Depth curves 156 are obtained at various 'y' strips and transformed geometrically to account for perspective projection. They are then stacked to form a true 3D surface map. The present invention is advantageous in a number of ways. The proposed system will be able to capture 3D surface data as well as intensity of light at a high speed. The line scan camera operates fast, hence iterations are performed quicker. The system also provides high resolution 3D surface data. The LCD or DLP projector provides dynamic structured lighting and this can be used at the invisible spectrum of light without actually making the user uncomfortable (as in case of face detection, etc). Moreover, the proposed system uses simple components to achieve the said performance and is cost effective. Other systems using stereo cameras fail to provide dense structure information due to lack of feature point distribution; they fail in the case of ill-contrasted images. The proposed method ensures effective 3D reconstruction in all scenarios of shape, texture and intensity contrast. The proposed system may be used in innumerous applications like face recognition, surface inspection, embossment inspection for sorting, virtualization of 3D objects etc. Summarizing, the present invention provides a system and for reconstruction of a three-dimensional surface. The proposed system comprises a line scan camera and includes a dynamic beam projection means adapted for synthesizing an inverse curve corresponding to each strip, of successive strips of said three-dimensional surface, such that when a radiation beam having a beam profile in the form of said inverse curve is projected by said dynamic beam projection means on said strip of said three-dimensional surface, the projection of the radiation beam appears as a straight line from the position of the line scan camera. The dynamic beam projections means thus obtains synthesized inverse curves for successive strips of said three-dimensional surface. The proposed system further includes reconstruction means adapted for generating a virtual straight line across said synthesized inverse curves and obtaining a depth curve based upon a correspondence between points on said inverse curve and said straight lines seen by said line scan camera. The reconstruction means then transforms depth curves obtained for successive strips of said three-dimensional surface perspective projections and stacks said transformed depth curves to obtain a three-dimensional surface map. Although the invention has been described with reference to specific embodiments, this description is not meant to be construed in a limiting sense. Various modifications of the disclosed embodiments, as well as alternate embodiments of the invention, will become apparent to persons skilled in the art upon reference to the description of the invention. It is therefore contemplated that such modifications can be made without departing from the spirit or scope of the present invention as defined. We claim, 1. A method for reconstruction of a three-dimensional surface (14), comprising: - synthesizing an inverse curve (152) corresponding to each strip, of successive strips of said three- dimensional surface (14), such that when a radiation beam (20) having a beam profile (34) in the form of said inverse curve (152) is projected on said strip of said three-dimensional surface (14), the projection (28) of the radiation beam (20) appears as a straight line (154) from the position of a line scan camera (30), - obtaining synthesized inverse curves (152) for successive strips of said three-dimensional surface (14), - generating a virtual straight line (158) across said synthesized inverse curves (152) and obtaining a depth curve (156) based upon a correspondence between points on said inverse curve (152) and said straight lines (154) seen by said line scan camera (30), and transforming depth curves (156) obtained for successive strips of said three-dimensional surface (14) perspective projections and stacking said transformed depth curves (154) to obtain a three- dimensional surface map. 2. The method according to claim 1, wherein synthesizing said inverse curve for each strip of the three- dimensional surface further comprises: - projecting a radiation beam having an initial beam profile on that strip, - capturing an image of a projection of said beam on said strip by said line scan camera, and obtaining a resultant profile from the image of the projection of said radiation beam on said strip, and - iteratively modifying the beam profile of the radiation beam to arrive at said inverse curve that defines a final beam profile for said radiation beam for which the resultant profile obtained from said image of the projection of said radiation beam is a straight line, based upon a feedback relationship between the beam profile of the radiation beam and the resultant profile obtained from the image of the projection of the radiation beam on said strip. The method according to claim 2, wherein obtaining a resultant profile from the image of the projection of said radiation beam on said strip further comprises: - obtaining an intermediate profile from the image of the projection of said radiation beam on said strip captured by said line scan camera, and - spline fitting discontinuous portions of said intermediate profile to obtain a continuous resultant profile. A system (10) for reconstruction of a three- dimensional surface (14), comprising: - a line scan camera (30) , - dynamic beam projection means (12) adapted for synthesizing an inverse curve (152) corresponding to each strip, of successive strips of said three- dimensional surface (14), such that when a radiation beam (20) having a beam profile (34) in the form of said inverse curve (152) is projected by said dynamic beam projection means (12) on said strip of said three-dimensional surface (14), the projection (28) of the radiation beam (20) appears as a straight line (154) from the position of said line scan camera (30), said dynamic beam projections means (12) adapted for obtaining synthesized inverse curves (152) for successive strips of said three- dimensional surface (14), - reconstruction means (32) adapted for generating a virtual straight line (158) across said synthesized inverse curves (152) and obtaining a depth curve (156) based upon a correspondence between points on said inverse curve (152) and said straight lines (154) seen by said line scan camera (30), and for transforming depth curves (156) obtained for successive strips of said three-dimensional surface (14) perspective projections and stacking said transformed depth curves (154) to obtain a three- dimensional surface map. 5. The system (10) according to claim 4, wherein said dynamic beam projection means (12) includes a programmable liquid crystal display (LCD) projector. 6. The system (10) according to claim 4, wherein said radiation beam (20) comprises a laser beam. 7. The system \l0) according to claim 4, wherein said radiation beam comprises an x-ray beam. 8. The system (10) according to any of claims 4 to 7, further comprising a rotatable mirror (106,108) adapted for reflecting the projection of the radiation beam on a strip of said three-dimensional surface (14) toward said line scan camera (30) such that rotation of said rotatable mirror enables scanning of successive strips of said three-dimensional surface (14) . 9. A system or method substantially as herein above described in the specification with reference to the accompanying drawings. The present invention provides a system and for reconstruction of a three-dimensional surface (14). The proposed system (10) comprises a line scan camera (30) and includes a dynamic beam projection means (12) adapted for synthesizing an inverse curve (152) corresponding to each strip, of successive strips of said three-dimensional surface (14), such that when a radiation beam (20) having a beam profile (34) in the form of said inverse curve (152) is projected by said dynamic beam projection means (12) on said strip of said three-dimensional surface (14), the projection (28) of the radiation beam (20) appears as a straight line from the position of said line scan camera (30). The dynamic beam projections means (12) thus obtains synthesized inverse curves (152) for successive strips of said three-dimensional surface (14). The proposed system (10) further includes reconstruction means (32) adapted for generating a virtual straight line (158) across said synthesized inverse curves (152) and obtaining a depth curve (156) based upon a correspondence between points on said inverse curve (152) and said straight lines (154) seen by said line scan camera (30). The reconstruction means (32) then transforms depth curves (156) obtained for successive strips of said three-dimensional surface (14) perspective projections and stacks said transformed depth curves (154) to obtain a three-dimensional surface map.

Full Text

Description
System and method for reconstructing a three-dimensional
surface
The present invention relates to a system and method for
reconstructing a three-dimensional surface.
The problem of reconstructing a three-dimensional surface can
be solved by using a pair of cameras (for example, as
disclosed in U.S Patent No. 6950121). The system described in
U.S Patent No. 6950121 is fast but suffers from drawbacks in
resolution for contrast starved regions and images where
detecting feature points for registration becomes difficult.
Also, such a system is prohibitively coarse for fine machine
vision inspection purposes. Other systems following a similar
philosophy have been used (for example, as disclosed in U.S
Patent No. 7142281). Depth estimation can also be done using
'Depth from Defocus' systems which also provide low
resolution data. Also, there are studies comparing 'Depth
from defocus' and stereo imaging and there is some ambiguity
as to whether they are really different.
The above problem has also been solved using high speed
acquisition system and a triangulated laser beam. Various
solutions have been implemented to address each issue at
different levels of success. All prior-art. inspection systems
require a high speed area scan camera and a laser. The laser
line obtained from a beam shaper or triangulation using
optical assemblies is made incident on the inspection surface
(Downing et al, 5054918). A camera scans the laser profile as
the object under inspection is made to pass under the
incident laser line. The distortion in the laser pattern
provides depth information which is subsequently converted to
3D surface structure data.
In a conventional system, a light sheet emitter emits a sheet
of light, preferably laser triangulated beam on a three-

dimensional surface. The light sheet forms a projected curve
which is read by the camera to form a curve in the image. The
light sheet scans over the entire surface to obtain
information which can be later compared for inspection
purposes. The fundamental drawback with such a system is that
it requires a huge acquisition time and transfers a large
amount of unnecessary data for a mapping. Our system will
circumvent such problems and simultaneously provide a device
which is high speed, efficient (in terms of information
transfer and processing) and cost effective.
An adaptation of this technique is the digital fringe
projection which involves projecting multiple fringes thereby
reducing time of 3D data acquisition. However, technological
impediments in fringe projection reduce phase measurement
accuracy ('Digital fringe projection in 3D shape measurement:
An error analysis' Notni, G.H: Conference "Optical
Measurement Systems for Industrial Inspection" 2003 )
The object of the present invention is to provide an improved
system and method for reconstructing a three-dimensional
surface.
The above object is achieved by a method according to claim 1
and a system according to claim 4.
The underlying idea of the present invention is to provide a
system that will be able to capture 3D surface data as well
as intensity of radiation beam at a high speed. The Line scan
camera operates fast, hence iterations are performed quicker.
In one embodiment, synthesizing said inverse curve for each
strip of the three-dimensional surface further comprises:
- projecting a radiation beam having an initial beam profile
on that strip,
- capturing an image of a projection of said beam on said
strip by said line scan camera, and obtaining a resultant

profile from the image of the projection of said radiation
beam on said strip, and
- iteratively modifying the beam profile of the radiation
beam to arrive at said inverse curve that defines a final
beam profile for said radiation beam for which the resultant
profile obtained from said image of the projection of said
radiation beam is a straight line, based upon a feedback
relationship between the beam profile of the radiation beam
and the resultant profile obtained from the image of the
projection of the radiation beam on said strip.
This involves mathematical computations that can
advantageously be carried out with appropriate computer
software.
In a further embodiment, obtaining a resultant profile from
the image of the projection of said radiation beam on said
strip further comprises:
- obtaining an intermediate profile from the image of the
projection of said radiation beam on said strip captured by
said line scan camera, and
- spline fitting discontinuous portions of said intermediate
profile to obtain a continuous resultant profile.
In a preferred embodiment, said dynamic beam projection means
includes a programmable liquid crystal display (LCD)
projector.
In one embodiment, said radiation beam (20) comprises a laser
beam. Using a laser beam is useful in inspection of
irregularities dark surfaces having very little contrast in
intensity.
In another embodiment, said radiation beam (20) comprises an
x-ray beam. An x-ray beam has lower wavelength and is useful
in measuring surface irregularities of extremely small
dimensions.

In one form of the present invention, the proposed system
further comprises a rotatable mirror adapted for reflecting
the projection of the radiation beam on a strip of said
three-dimensional surface toward said line scan camera such
that rotation of said rotatable mirror enables scanning of
successive strips of said three-dimensional surface.
The present invention is further described hereinafter with
reference to illustrated embodiments shown in the
accompanying drawings, in which:
FIG 1 is a schematic diagram of a system for reconstructing a
three-dimensional surface,
FIG 2 is a schematic drawing illustrating successive
deformations of the radiation beam profile and the
corresponding resultant profile of the projection of the beam
in the image captured by the line scan camera,
FIG 3 is a flowchart illustrating a method for synthesizing
inverse curves according to one embodiment of the present
invention,
FIGS 4A, 4B and 4C respectively illustrate successive
deformations of the radiation beam profile, the corresponding
intermediate profiles of the projection of the beam as seen
by a line scan cameras and resultant profiles after spline
fitting of discontinuous portions in the intermediate
profiles,
FIG 5 is a schematic diagram of an exemplary arrangement of a
rotatable mirror with a line scan camera for scanning the 3D
surface,
FIG 6 is a schematic drawing illustrating correspondence
between points on the synthesized inverse curve and points on
the straight line seen by the line scan camera, and

FIG 7A and 7B respectively illustrate a virtual straight line
drawn on the inverse curves of successive strips and the
corresponding depth curve.
Embodiments of the present invention provide an efficient and
low cost three-dimensional camera system for obtaining
intensity as well as high resolution three-dimensional
surface information at realistic times. The surface can be
arbitrary, having no regularity constraints.
The proposed system integrates a dynamic beam projection
beams, for example a programmable liquid crystal display
(LCD) projector or a digital light processing (DLP™)
projector, a line scan camera, and an adaptive beam reshaping
algorithm. Conventional systems project a straight light
sheet, capture the profiles using area scan cameras and use
them for 3D reconstruction. The drawback of such systems is
the unnecessary time consumed for acquisition of the whole
image and high data transfer rates. Such drawbacks decrease
the speed of execution and increase the cost of the system or
limit the resolution. The proposed system is conceptualized
inverse to the above paradigm. The objective here is to
synthesize a curve which on incident on a section of the 3D
surface appears as a straight line when seen from the line
scan camera. Once the line scan camera sees the straight
line, the synthesized curve now represents information about
the surface. This process is repeated for different strips on
the surface and combined to form a representative shape
descriptor for the particular surface. The purpose of the
dynamic beam projection means is to iteratively structure the
incident light in a feedback loop with the line scan camera.
Referring to FIG 1, a system 10 is shown for reconstructing a
three-dimensional surface 14, which may be any arbitrarily
curved surface. The system 10 includes a line scan camera 30
and dynamic beam projection means 12, for example a
programmable LCD or DLP projector 16, which is coupled to a
computing device 18 (for example, a PC) that executes the

algorithm for synthesizing inverse curves for each of
successive strips of the three-dimensional surface 14. In
operation, the projector 16 projects a radiation beam 20 (in
this case, a light beam) on a strip of the surface 14. The
surface 14 is a three-dimensional surface, and hence, if the
radiation beam 20 has a beam profile 22 (i.e, beam cross-
section) in the form of a straight line, the projection 26 of
the beam 20 on the strip of the 3D surface 14 would appear as
a curved line to the line scan camera 30. The idea of the
present invention is to modify or re-shape the beam profile
of the beam 20 by synthesizing an inverse curve, such that
when a radiation beam having a beam profile in the form of
this inverse curve is projected on that strip of the 3D
surface 14, the projection 28 as seen from the position of
the line scan camera 30 is a straight line. The synthesized
inverse curve profile 34 is stored. Similarly, inverse curves
are synthesized and stored for successive strips of the 3D
surface 14.
In operation, a radiation beam 20 having an initial beam
profile is projected on to the strip from the projector 16.
An image of the projection of the beam 20 is captured by the
line scan camera 30, and a resultant profile obtained from
the image of that projection as captured by the line scan
camera. The beam profile of the radiation beam 20 is then
iteratively modified using feedback based mechanism between
the projector 16 and the line scan camera 30, which reshapes
the projected radiation beam 20 from the projector 16 such
that the image of the projection of the radiation beam as
captured by the line scan camera has a resultant profile
which is a straight line. The beam profile projected by the
projector which gives a resultant profile (captured by the
line scan camera) as a straight line is in the form of a
curve which is referred to herein as an inverse curve for
that particular strip of the 3D surface.
FIG 2 shows the successive deformations of the beam profiles
vky (x) projected by the projector and the corresponding

resultant profiles fky (x) obtained from the line scan camera
for a particular strip of the 3D surface as the iterations
progress. The arrow 40 indicates the direction of iterations.
The iterations stop when the line scan camera sees a
continuous and straight resultant profile 154. This deformed
curve beam profile is stored as an inverse curve 152 for a
particular strip of the 3D surface. The entire surface
profiling is completed when all strips are traversed and
their corresponding inverse curves are stored.
It should be noted that the line scan camera would not be
able to image complete profile of the projection of the
radiation beam on a given strip of the 3D surface, and would
see only a part of the profile. So, in the illustrated
embodiment, an intermediate profile is obtained from the
image of the projection of the beam as captured by the line
scan camera, and then, discontinuous portions of this
intermediate profile are spline fitted to obtain the
resultant profile. This is explained referring to FIGS 4A, 4B
and 4C. FIG 4A shows the deformation of the beam profile
vky(x) along a direction of iteration indicated by the arrow
40. FIG 4B shows the corresponding intermediate profiles 72
from the images of the projection of the beam having the
corresponding beam profiles as shown in FIG 4A. As can be
seen, the intermediate profiles are discontinuous, since the
line scan camera can capture only portions of the projection
of the beam on a given strip of the 3D surface. FIG 4C shows
the resultant profiles fky(x) obtained by spine fitting of
discontinuous portions in the intermediate profiles shown in
FIG 4B. The beam-reshaping algorithm is carried out based on
a feedback relationship between the beam profile vky(x) shown
in FIG 4A and the resultant profile fky(x) obtained from the
line scan camera as shown in FIG 4C.
FIG 3 is a flowchart showing an exemplary beam re-shaping
algorithm 50 for synthesizing inverse curves for successive
strips of the 3D surface. The algorithm 50 begins at block 52
when a target strip is selected on the 3D surface. Let *y' be

a strip in consideration at a certain point of time. The
strip at 'y' denotes the position along the scanning axis.
For example, if the 3D surface is placed on a conveyor, as
the object travels, the conveyor belt movement represents the
scanning axis; as the machine part traverses, the 'y' value
(or position) increments. The iteration carried for each 'y'
is as follows. The iterative beam re-shaping algorithm is
carried out as explained in FIG 2. The objective profile is
what is sought after. In this case, the objective function is
Φy(x) = h/2 , where 'h' is the height in pixels of the
scanning area.
At block 54, a radiation beam having an initial beam profile
is projected on to the selected strip of the 3D surface. The
initial beam profile is v°y (x), where 'x' runs along the line
of pixels, represented by the line scan camera (for a
standard line scan camera available, x would be a vector from
1:640 or 1:1024). At block 56, a discontinuous intermediate
profile of the projection of the beam on the surface is
captured by the line scan camera. The resultant profile
obtained after spline fitting discontinuous portions of the
intermediate profile is f°y (x). This information is fed to
the iterative beam re-shaping algorithm (block 58).
In the iterative algorithm (block 58), the new profile to be
fed to the projector is iterated by
The profile v1y (x) when projected on to the 3D surface would
give a resultant profile as f1y (x). This iteration is carried
out for every time step 't' as generalized as
. The iterations are performed until
the quantity is minimized. The parameter 'k' is
a relaxation parameter enhancing the convergence rates and
incorporates the mapping between the physical space and the
image plane.
Next, at block 60, the synthesized inverse curve
corresponding to the beam profile vfinaly (x) is stored. At

block 62, the next adjacent strip of the 3D surface is
selected as the target strip and the above steps (blocks 52
to 60) are repeated. The algorithm 50 ends at block 64 by
storing all the inverse curves that are synthesized.
FIG 5 shows an exemplary embodiment illustrating the
mechanics of the line scan camera 30. A rotating mirror
(rotatable from position 106 to position 108) sweeps for a
range of angles (represented by numerals 102 and 104) to
enable vertical scan over a 3D test surface 14. The projector
16 projects a radiation beam 20 at a particular designated
strip which is the focus strip as deflected by the scanning
mirror at any instant of time such that the line scan sensor
30 is focused on the strip.
In alternate embodiments, instead of light beams, other forms
of radiation can be used. For example, the radiation beam may
comprise laser beams. Using a laser beam is useful in
inspection of irregularities dark surfaces having very little
contrast in intensity. Still alternately, the radiation beam
may comprise x-ray beams. X-ray beams have lower wavelength
and is useful in measuring surface irregularities of
extremely small dimensions.
Referring back to FIG 1, the synthesized inverse curves are
fed to the reconstruction means 32 to reconstruct the 3D
surface. Herein, as shown in FIG 6, a correspondence is
established between points on each inverse curve and points
on the straight line of the resultant profile obtained from
the line scan camera. As shown, the inverse curve 152 has
points 160 in a regular fashion marked on it. The
corresponding points on the straight line 154 seen by the
line scan camera are noted; hence correspondences in points
between the inverse curves 152 and the seen straight line 152
are determined.
For different strips along the 'y' direction, different
inverse curves 152 and the corresponding straight lines 154

seen by the line scan camera are shown in FIG 7A and FIG 7B
respectively. Herein, inverse curves 152 obtained from
scanning different strips with the corresponding points in
the straight lines 154 are shown
The next task involves obtaining the actual depth map from
the inverse curves. It is shown that drawing a virtual
straight line 158 on the set of inverse curves 152 (shown in
FIG 7A). establishes the depth curve 156 (shown in FIG 7B)
using, the point correspondence information explained with
respect to FIG 6. Depth curves 156 are obtained at various
'y' strips and transformed geometrically to account for
perspective projection. They are then stacked to form a true
3D surface map.
The present invention is advantageous in a number of ways.
The proposed system will be able to capture 3D surface data
as well as intensity of light at a high speed. The line scan
camera operates fast, hence iterations are performed quicker.
The system also provides high resolution 3D surface data. The
LCD or DLP projector provides dynamic structured lighting and
this can be used at the invisible spectrum of light without
actually making the user uncomfortable (as in case of face
detection, etc). Moreover, the proposed system uses simple
components to achieve the said performance and is cost
effective. Other systems using stereo cameras fail to provide
dense structure information due to lack of feature point
distribution; they fail in the case of ill-contrasted images.
The proposed method ensures effective 3D reconstruction in
all scenarios of shape, texture and intensity contrast. The
proposed system may be used in innumerous applications like
face recognition, surface inspection, embossment inspection
for sorting, virtualization of 3D objects etc.
Summarizing, the present invention provides a system and for
reconstruction of a three-dimensional surface. The proposed
system comprises a line scan camera and includes a dynamic
beam projection means adapted for synthesizing an inverse

curve corresponding to each strip, of successive strips of
said three-dimensional surface, such that when a radiation
beam having a beam profile in the form of said inverse curve
is projected by said dynamic beam projection means on said
strip of said three-dimensional surface, the projection of
the radiation beam appears as a straight line from the
position of the line scan camera. The dynamic beam
projections means thus obtains synthesized inverse curves for
successive strips of said three-dimensional surface. The
proposed system further includes reconstruction means adapted
for generating a virtual straight line across said
synthesized inverse curves and obtaining a depth curve based
upon a correspondence between points on said inverse curve
and said straight lines seen by said line scan camera. The
reconstruction means then transforms depth curves obtained
for successive strips of said three-dimensional surface
perspective projections and stacks said transformed depth
curves to obtain a three-dimensional surface map.
Although the invention has been described with reference to
specific embodiments, this description is not meant to be
construed in a limiting sense. Various modifications of the
disclosed embodiments, as well as alternate embodiments of
the invention, will become apparent to persons skilled in the
art upon reference to the description of the invention. It is
therefore contemplated that such modifications can be made
without departing from the spirit or scope of the present
invention as defined.

We claim,
1. A method for reconstruction of a three-dimensional
surface (14), comprising:
- synthesizing an inverse curve (152) corresponding to
each strip, of successive strips of said three-
dimensional surface (14), such that when a radiation
beam (20) having a beam profile (34) in the form of
said inverse curve (152) is projected on said strip
of said three-dimensional surface (14), the
projection (28) of the radiation beam (20) appears
as a straight line (154) from the position of a line
scan camera (30),
- obtaining synthesized inverse curves (152) for
successive strips of said three-dimensional surface
(14),
- generating a virtual straight line (158) across said
synthesized inverse curves (152) and obtaining a
depth curve (156) based upon a correspondence
between points on said inverse curve (152) and said
straight lines (154) seen by said line scan camera
(30), and
transforming depth curves (156) obtained for
successive strips of said three-dimensional surface
(14) perspective projections and stacking said
transformed depth curves (154) to obtain a three-
dimensional surface map.
2. The method according to claim 1, wherein synthesizing
said inverse curve for each strip of the three-
dimensional surface further comprises:
- projecting a radiation beam having an initial beam
profile on that strip,
- capturing an image of a projection of said beam on
said strip by said line scan camera, and obtaining a
resultant profile from the image of the projection
of said radiation beam on said strip, and

- iteratively modifying the beam profile of the
radiation beam to arrive at said inverse curve that
defines a final beam profile for said radiation beam
for which the resultant profile obtained from said
image of the projection of said radiation beam is a
straight line, based upon a feedback relationship
between the beam profile of the radiation beam and
the resultant profile obtained from the image of the
projection of the radiation beam on said strip.
The method according to claim 2, wherein obtaining a
resultant profile from the image of the projection of
said radiation beam on said strip further comprises:
- obtaining an intermediate profile from the image of
the projection of said radiation beam on said strip
captured by said line scan camera, and
- spline fitting discontinuous portions of said
intermediate profile to obtain a continuous
resultant profile.
A system (10) for reconstruction of a three-
dimensional surface (14), comprising:
- a line scan camera (30) ,
- dynamic beam projection means (12) adapted for
synthesizing an inverse curve (152) corresponding to
each strip, of successive strips of said three-
dimensional surface (14), such that when a radiation
beam (20) having a beam profile (34) in the form of
said inverse curve (152) is projected by said
dynamic beam projection means (12) on said strip of
said three-dimensional surface (14), the projection
(28) of the radiation beam (20) appears as a
straight line (154) from the position of said line
scan camera (30), said dynamic beam projections
means (12) adapted for obtaining synthesized inverse
curves (152) for successive strips of said three-
dimensional surface (14),

- reconstruction means (32) adapted for generating a
virtual straight line (158) across said synthesized
inverse curves (152) and obtaining a depth curve
(156) based upon a correspondence between points on
said inverse curve (152) and said straight lines
(154) seen by said line scan camera (30), and for
transforming depth curves (156) obtained for
successive strips of said three-dimensional surface
(14) perspective projections and stacking said
transformed depth curves (154) to obtain a three-
dimensional surface map.
5. The system (10) according to claim 4, wherein said
dynamic beam projection means (12) includes a
programmable liquid crystal display (LCD) projector.
6. The system (10) according to claim 4, wherein said
radiation beam (20) comprises a laser beam.
7. The system \l0) according to claim 4, wherein said
radiation beam comprises an x-ray beam.
8. The system (10) according to any of claims 4 to 7,
further comprising a rotatable mirror (106,108)
adapted for reflecting the projection of the radiation
beam on a strip of said three-dimensional surface (14)
toward said line scan camera (30) such that rotation
of said rotatable mirror enables scanning of
successive strips of said three-dimensional surface
(14) .
9. A system or method substantially as herein above
described in the specification with reference to the
accompanying drawings.

The present invention provides a system and for
reconstruction of a three-dimensional surface (14). The proposed system (10) comprises a line scan camera (30) and includes a dynamic beam projection means (12) adapted for
synthesizing an inverse curve (152) corresponding to each strip, of successive strips of said three-dimensional surface
(14), such that when a radiation beam (20) having a beam profile (34) in the form of said inverse curve (152) is projected by said dynamic beam projection means (12) on said
strip of said three-dimensional surface (14), the projection (28) of the radiation beam (20) appears as a straight line
from the position of said line scan camera (30). The dynamic beam projections means (12) thus obtains synthesized inverse
curves (152) for successive strips of said three-dimensional surface (14). The proposed system (10) further includes reconstruction means (32) adapted for generating a virtual straight line (158) across said synthesized inverse curves
(152) and obtaining a depth curve (156) based upon a correspondence between points on said inverse curve (152) and said straight lines (154) seen by said line scan camera (30).
The reconstruction means (32) then transforms depth curves (156) obtained for successive strips of said three-dimensional surface (14) perspective projections and stacks
said transformed depth curves (154) to obtain a three-dimensional surface map.

Documents:

http://ipindiaonline.gov.in/patentsearch/GrantedSearch/viewdoc.aspx?id=PDEGIQCV73eqStEQpQOsCg==&loc=wDBSZCsAt7zoiVrqcFJsRw==

« Previous Patent

Next Patent »

Patent Number

272960

Indian Patent Application Number

414/KOL/2009

PG Journal Number

20/2016

Publication Date

13-May-2016

Grant Date

05-May-2016

Date of Filing

06-Mar-2009

Name of Patentee

SIEMENS INFORMATION SYSTEMS LTD.

Applicant Address

43, SHANTIPAQLLY EM BYPASS-RASHBEHARI CONNECTOR KOLKATA

Inventors:

#	Inventor's Name	Inventor's Address
1	VARUN AKUR VENKATESAN	#286, SECTOR 5, HSR LAYOUT, 560102 BANGALORE
2	GARIMELLA PADMA MADHURI	22/26(1), EAST END C MAIN, JAYANAGAR 9TH, 560069 BANGALORE
3	VENKATESH BAGARIA	1179, FIRST FLOOT, 11TH CROSS, 22B MAIN, HSR 560034

PCT International Classification Number

G01B11/25; G01B11/24

PCT International Application Number

N/A

PCT International Filing date

PCT Conventions:

#	PCT Application Number	Date of Convention	Priority Country
1			NA