Light pen whole space coordinate measuring system based on a tracking turntable

WANG Sen， LIU Shugui， MAO Qing

(State Key Laboratory of Precision Measuring Technology and Instruments， Tianjin University， Tianjin 300072, China)

Abstract： Light pen coordinate measuring system (LPCMS) is a kind of portable coordinate measuring technique based on vision metrology. In classical LPCMS, the measuring range is limited to the camera’s field of view. To overcome this defect, a new LPCMS is designed in this paper to fulfil whole space coordinate measurement. The camera is installed on a turntable instead of a tripod, so that the camera can rotate to track the movement of the light pen. The new system can be applied to large scale onsite measurement, and therefore it notably extends the application of LPCMS. To guarantee the accuracy of the new system, a method to calibrate the parameters of the tracking turntable is also proposed. Fixing the light pen at a stationary position, and changing the azimuth angles of the turntable’s two shafts, so that the camera can capture the images of the light pen from different view angles. According to the invariant spatial relationship between the camera and the pedestal of the tracking turntable, a system of nonlinear equations can be established to solve the parameters of the turntable. Experimental results show that the whole space coordinate measuring accuracy of the new system can reach 0.25 mm within 10 m. It can be concluded that the newly designed system can significantly expand the measuring range of LPCMS without losing too much accuracy.

Key words： light pen； whole space coordinate measuring system； tracking turntable； vision metrology； parameter calibration； homogeneous coordinate transformation matrix

0 Introduction

Coordinate measuring is a very important technique in manufacturing industry and reverse engineering[1-3]. With the continuous development of the manufacturing industry, coordinate measuring technique is increasingly applied to on-site measurement of workpieces. To meet this demand, several kinds of portable coordinate measuring systems have been designed in the last three decades[4-7]. Among these systems, light pen coordinate measuring system is indispensable for its ability to measure the hidden features of workpieces[8-9].

Light pen coordinate measuring system (LPCMS) is developed on the basis of vision metrology principle. It has the advantages of good portability, high measuring accuracy and low environmental requirements, which makes it competent for on-site measurement.

When three dimensional object space is mapped to two dimensional image space through the camera’s perspective projection model, the depth information is lost[10]. Therefore, it is impossible to recover the three dimensional information of an object just from one single image. LPCMS utilizes control point technique to provide additional constraints to the perspective projection model, so that it is able to measure three dimensional coordinates from the image of the light pen.

Despite classical LPCMS has many good features, it has its own limitation. The most salient one is that its measuring range is limited to the camera’s field of view, whose shape is a rectangular pyramid. In practical application, the position of the camera needs to change frequently according to the positon of the measured object, so that the object is guaranteed in the camera’s field of view all the time. Once the camera is moved, the reference coordinate system of LPCMS changes. Therefore these measuring results cannot be unified in the same reference coordinate system. In other words, it is impossible for the classical LPCMS to fulfil whole space coordinate measurement.

To overcome the defect of the classical LPCMS, a new light pen whole space coordinate measuring system based on a tracking turntable is designed in this paper. By mounting it to a tracking turntable, the camera can rotate driven by the turntable to track the movement the light pen[11-13]. Thus the measuring range of the newly designed system is no longer confined to the camera’s field of view, and therefore can realize whole space coordinate measurement. This new system can be applied to on-site measurement of large workpieces such as ship hull, train body and plane shell, which exceeds the measuring range of the classical LPCMS.

In the newly designed light pen whole space coordinate measuring system, the positon of the camera is constantly changing with the movement of the light pen during the measuring process. Therefore, the reference coordinate system of the new system cannot be fixed to the camera any more. Instead, the coordinate system which is fasten to the pedestal of the turntable can serve as the new reference coordinate system. To transform the final measuring results to the coordinates in the new reference coordinate system, the coordinate transformation matrix between the camera and the turntable’s pedestal must be determined. For this purpose, the parameters of the tracking turntable must be carefully calibrated to guarantee the measuring accuracy of the new system in the whole space.

To calibrate the parameters of the tracking turntable, a method that utilizes a stationary light pen is also proposed in this paper. By changing the azimuth angles of the turntable’s two rotation shafts, the camera can capture the image of the stationary light pen from different view directions. A series of nonlinear equations can be established according to the invariance of the coordinate transformation matrix between the light pen and the turntable’s pedestal. The parameters of the tracking turntable can be solved from this equation system by nonlinear least squares algorithm.

1 Detailed system description

1.1 System configuration

Fig.1 shows the configuration of the light pen whole space coordinate measuring system based on a tracking turntable designed in this paper. From Fig.1, it can be seen that the system mainly consists of 4 components: a light pen, a laptop, an industrial camera and a tracking turntable. In addition to these, there are some wires and accessories as well.

Fig.1 Configuration of light pen whole space coordinate measuring system based on a tracking turntable

Fig.2 shows the structure of the light pen adopted in this paper. It uses luminous infrared LEDs as control points, and there are 13 control points distributing on the light pen. The relative spatial positions between these control points need to be accurately calibrated in advance to provide accurate additional constraints to the camera’s perspective projection model. Besides the control points, there is a contact probe at the end of the light pen as well. The positon of the center of the probe tip also needs to be calibrated in advance to obtain its position relative to all the control points.

The laptop is used to run the measuring software. Besides the mathematical calculation of the whole measuring system, the measuring software is also in charge of the communication with the light pen and the turntable. According to actual requirements, the laptop can control the rotation angles of the turntable’s two shafts through RS232 serial communication, and control the luminescence time and the electric current of the infrared LEDs through wireless communication.

The industrial camera is used to capture the image of the light pen. Both the intrinsic parameter matrix of the camera and the distortion factors of the lens need to be calibrated in advance to guarantee the accuracy of the camera’s imaging model. To track the movement of the light pen, the camera is installed on a tracking turntable instead of a tripod.

Fig.3 gives the structure of the tracking turntable. The turntable consists of a stationary pedestal, two precision rotation shafts and two high precision angular encoders. The camera is installed on a rectangular metal frame which can rotate driven by these two shafts. The vertical shaft can drive the camera conduct left and right deflection in the horizontal plane, and it is calledAaxis in this paper. The horizontal shaft can drive the camera conduct up and down pitching in the vertical plane, and it is calledBaxis in this paper. Under ideal condition, theAaxis and theBaxis of the turntable are strictly perpendicularly intersecting. However, because of the existence of manufacture error and assembly error, they are neither perpendicular nor intersecting. In actual application, theAaxis and theBaxis are treated as two skew lines in different planes, which has no special geometric relationship.

Fig.3 Structure of tracking turntable

1.2 Measurement principle

The new whole space coordinate measuring system designed in this paper is developed on the basis of traditional LPCMS, so it is necessary to give a detailed introduction of the measurement principle of classical LPCMS first.

For convenience of description, two coordinate systems are established in LPCMS. They are fixed to the light pen and the camera, respectively, and they can be called the light pen coordinate system and the camera coordinate system severally. The light pen coordinate system can be denoted asOl-xlylzl, and the camera coordinate system can be denoted asOc-xcyczc.

(1)

whereAl,cis the 4×4 homogeneous coordinate transformation matrix betweenOl-xlylzlandOc-xcyczc;Rl,cis the 3×3 orthogonal rotation matrix, andtl,cis the three dimensional translation vector.

(2)

whereKis the intrinsic parameter matrix of the camera;λis an arbitrary scale factor that is nonzero;fuandfvare the effective focal lengths along the two directions of the imaging plane;cuandcvare the pixel coordinates of the principal point, andsis a parameter that reflects the non-perpendicularity of the two axes of the imaging plane.

What needs to be emphasized here is that the lens distortion is not taken into consideration, because it is not the main issue of this paper. In practice, the lens distortions in the homogeneous pixel coordinates need to be compensated first to get the undistorted pixel coordinates, so that the linear perspective projection model in Eq.(2) is satisfied.

When LPCMS is used to measure the coordinates of an object, the camera needs to be located at a proper stationary position, so that the whole object is in the camera’s field of view. The operator holds the light pen and makes the probe tip contact the surface of the object. According to the image of the light pen captured by the camera, the coordinates of the point touched by the light pen can be measured.

According to Eqs.(1) and (2), the equation satisfying for each control point can be expressed as

(3)

After eliminating the scale factorλ, two nonlinear equations can be derived from Eq.(3) for each control point. The unknown variables in these equations are the rotation matrixRl,cand the translation vectortl,cbetweenOl-xlylzlandOc-xcyczc. Merging these equations together, if the number of the control points is big enough, an overdetermined nonlinear equation system can be established.Rl,candtl,ccan be solved from this equation system by various nonlinear least squares algorithms. This is a well-known pose estimation issue in vision metrology, which is named as perspective-n-point (PnP) problem.

For the reason that the position of the camera is fixed during the whole measuring process, the camera coordinate systemOc-xcyczccan serve as the reference coordinate system of LPCMS. The final measuring result of LPCMS is the coordinates of the measured point in the reference coordinate systemOc-xcyczc.

(4)

From Eq.(4), it can be seen that how to determine the homogeneous transformation matrixAc,tbetweenOc-xcyczcandOt-xtytztis very important for the new system. For the reason that theAaxis and theBaxis of the turntable are treated as skew lines in different planes, this transformation process can be decomposed into 4 steps, and two intermediate coordinate systems need to be established to make each step simple and clear. The two intermediate coordinate systems are fixed to and rotate with theAaxis and theBaxis of the tracking turntable, respectively, and they can be denoted as theA-axis coordinate systemOA-xAyAzAand theB-axis coordinate systemOB-xByBzBseverally.

The two intermediate coordinate systems are established when theAaxis and theBaxis are both at zero position. The way to establishOA-xAyAzAis: take theAaxis as thexaxisxA; take the common perpendicular of theAaxis and theBaxis as thezaxiszA; take the intersection of theAaxis and the common perpendicular as the originOA; theyaxisyAcan be determined according to the right hand rule. The way to establishOB-xByBzBis: take theBaxis as theyaxisyB; take the common perpendicular of theAaxis and theBaxis as thezaxiszB; take the intersection of theBaxis and the common perpendicular as the originOB; thexaxisxBcan be determined according to the right hand rule.

From the structure of the turntable shown in Fig.3, it can be inferred thatOA-xAyAzAonly rotates with theAaxis andOB-xByBzBrotates with theAaxis and theBaxis at the same time. Suppose the azimuth angles of theAaxis and theBaxis can be denoted asθandφ, respectively, and the two intermediate coordinate systems can be further denoted asOA-xAyAzA(θ) andOB-xByBzB(θ,φ) when the two shafts of the turntable are at any azimuth angles. When theAaxis is at zero position, theA-axis coordinate system can be denoted asOA-xAyAzA(θ=0), and it can serve as the reference coordinate system of the newly designed measuring system. In other words, the coordinate systemsOA-xAyAzA(θ=0) is exactly the turntable coordinate systemOt-xtytzt.

After finishing the establishment of the relevant coordinate systems, the 4 steps to deduce the expression form ofAc,tare shown as follows.

(5)

Because there are no special restrictions on the way that the camera is installed on the turntable, the matrixA1represents a general three dimensional rigid body transformation. The orthogonal rotation matrixR1has only 3 degrees of freedom, and it can be parameterized with 3 rotation angles around the coordinate axes, which can be denoted as (α,β,γ). The translation vectort1also has 3 degrees of freedom, and it is composed of 3 translational components along the coordinate axes, which can be denoted as (r,s,t). These 6 parameters describes how the camera is installed on the tracking turntable, and therefore they are called the assembly parameters of the camera in this paper[15-16].

Fig.4 Rotation of OB-xByBzB(θ,φ) around B axis

(6)

where

(7)

Fig.5 Spatial relationship of two shafts of turntable

(8)

where

(9)

The two parametersdandεare determined by the structure of the tracking turntable, and therefore they are called the structure parameters of the tracking turntable in this paper.

Fig.6 Rotation of OA-xAyAzA(θ) around A axis

(10)

where

(11)

By establishing two intermedia coordinate systemsOA-xAyAzAandOB-xByBzB, the 4 steps of coordinate transformation all become straight for ward. According to the coordinate transformation chain listed in Eqs.(5), (6), (8) and (10), the homogeneous coordinate transformation matrix between the camera coordinate systemOc-xcyczcand the turntable coordinate systemOt-xtytztcan be calculated from the product of the transformation matrix in each step:

Ac,t=A4A3A2A1.

(12)

From the expression forms of the 4 matrixes in Eqs.(5), (7), (9) and (11), it can be seen that the matrixAc,tcontains 10 variables in total. Among them, the two azimuth anglesθandφof theAaxis and theBaxis are directly given by the readings of the two high precision angular encoders installed on the tracking turntable. The rest 8 variables (α,β,γ,r,s,t,d,ε) are the parameters of the tracking turntable. As is mentioned in Steps 1) and 3), they can be divided into two parts: the assembly parameters of the camera (α,β,γ,r,s,t), and the structure parameters of the tracking turntabledandε. The values of these parameters need to be determined by the calibration of the tracking turntable. This will be introduced in the Section 2.

To sum up, the measurement principle of the newly designed light pen whole space coordinate measuring system based on a tracking turntable can be summarized as

(13)

The matrixAl,cis calculated from the image of the light pen by solving PnP problem, and the matrixAc,tis calculated according to Eq.(12).

1.3 Tracking equation of turntable

In the newly designed light pen whole space coordinate measuring system, the camera can track the movement of the light pen automatically. The general principle of the tracking process is to make the light pen always located at the center of the camera’s field of view approximately. As long as there is at least one control point can be seen by the camera, the rotation angles of the two shafts of the turntable can be calculated according to the current positon of the light pen in the image.

(14)

The objective of the tracking process is to make the centroid of the control points coincide with the principal point of the camera’s imaging plane. According to the pinhole imaging model, the rotation angles of the turntable’s two shafts, which can be denoted as Δθand Δφ, can be calculated by

(15)

where (cu,cv,fu,fv) are the intrinsic parameters of the camera as shown in Eq.(3).

(16)

Once the movement of the light pen makes the condition in Eq.(16) satisfied, the two shafts need to rotate by the angles calculated according to Eq.(15), so that the moving light pen is always in the center of the camera’s field of view approximately. Therefore, the equation in Eq.(15) is called the tracking equation of the turntable.

2 System calibration

In order to calculate the matrixAc,tin Eq.(13) accurately, the parameters (α,β,γ,r,s,t,d,ε) of the tracking turntable must be calibrated as well. In this paper, a method is proposed to calibrate the parameters of the tracking turntable.

2.1 Calibration procedure

The overall idea of the proposed calibration method is utilizing the camera to capture the image of the stationary light pen from different view angles. The calibration procedure is as follows:

1) Fix the pedestal of the tracking turntable on a stable platform and make sure that its position remains unchanged during the whole calibration process.

2) Set the azimuth angles of theAaxis and theBaxis of the turntable both to zero, i.e.θ=0°，φ=0°.

3) As shown in Fig.7(a), locate the light pen at a proper distance from the camera and make it at the center of the camera’s field of view. Fix the current position of the light pen and make sure its position remains unchanged during the whole calibration process as well.

Fig.7 Schematic diagram of calibration procedure of turntable’s parameters

4) As shown in Fig.7(b), let theAaxis deflect to the right and theBaxis pitch down respectively to change the view angle of the camera, so that the light pen becomes at the top left corner of the camera’s field of view. Record the current azimuth angles of theAaxis and theBaxis.

5) As shown in Figs.7(c)-(e), similar to the operation in Step 4), change the azimuth angles of theAaxis and theBaxis sequentially, so that the light pen becomes at the other three corners of the camera’s field of view, respectively. Record the azimuth angles of theAaxis and theBaxis that correspond to these 3 situations.

6) Through the operations in Steps 4) and 5), the variation ranges of the two azimuth angles during the whole calibration process are determined, which can be denoted asθ∈[θb,θe] andφ∈[φb,φe]. According to these two ranges, a two dimensional subdivision calibration grid is generated by segmenting both the two ranges into several equal intervals.

7) Let the two shafts of the tracking turntable rotate automatically to scan line by line according to the generated calibration grid, so that the azimuth angles of theAaxis and theBaxis can traverse every node of the calibration grid. At each node of the grid, use the camera to capture the image of the stationary light pen from different view angles.

8) With the captured light pen images, a series of nonlinear equations can be established according to the invariable coordinate transformation between the light pen and the pedestal of the turntable. The parameters of the tracking turntable can be solved from the established equation system with nonlinear least squares method.

2.2 Equation establishment

For the reason that the positions of the light pen and the pedestal of the tracking turntable both remain unchanged during the whole calibration process, the homogeneous coordinate transformation matrix between the light pen coordinate systemOl-xlylzland the turntable coordinate systemOt-xtytztis constant, which can be denoted asAl,t. According to this invariant relation, a series of nonlinear equations can be established.

Suppose that the numbers of intervals in the azimuth angles of theAaxis and theBaxis arepandq, respectively, then the azimuth angles of the two shafts at each node of the calibration grid can be denoted as (θi,φk), (i=1,2,…,p;k=1,2,…,q). They can be calculated by

(17)

(18)

wherei1,i2=1,2,…,p;k1,k2=1,2,…,q;i1≠i2;k1≠k2.

From Eq.(18), 12 nonlinear equations which take the 8 parameters of the tracking turntable as unknown variables can be deduced. The concrete expression forms of these equations are quite complicated. For convenience, they are expressed in a simplified form as

(19)

whereh=1,2,…,12. When all the nodes of the calibration grid are taken into consideration at the same time, an overdetermined nonlinear equation system can be established. The parameters of the tracking turntable can be solved from this equation system by nonlinear least squares algorithm.

2.3 Solving algorithm

1) Give the initial solution vectorX(0). Set the threshold value of the convergence conditionσ1andσ2. Set the iteration index tol=0.

2) In thel-th iteration, the current solution vector isX(l). Calculate the value vectorG(X(l)) and the Jacobian matrixJ(X(l)) according to this solution vector.

3) Solve the Gauss-Newton equation to obtain the iteration direction at the current solution vectorX(l):Z(l)=-(J(X(l))TJ(X(l)))-1J(X(l))G(X(l)).

4) Calculate the step length along iteration directionZ(k)by using one dimensional line search, which can be formulized as the optimization problem:a(l)=argmin‖G(X(l)+λZ(l))‖2. This problem can be solved with the rational extremum method.

5) With the calculated iteration direction and step length, update the solution vector according to the iterative criterion:X(l+1)=X(l)+a(l)Z(l).

6) Check the threshold condition and judge whether the iteration process converges: if ‖d(l)Z(l)‖≤σ1‖X(l)‖ or ‖G(X(l+1))-G(X(l))‖≤σ2‖G(X(l))‖, the iteration terminates and the solution vectorX(l+1)is the final solution; otherwise go back to Step 2) and continue the iteration with the updated solution vectorX(l+1). At the same time, update the iteration indexl=l+1.

3 Results and discussion

To verify the precision of the proposed calibration method, the calibration experiments are repeated 10 times. The calibration results of the parameters of the turntable are listed in Table 1. In the camera’s assembly parameters, the repeatability of the three rotation angles (α,β,γ) is no more than 0.02°, and the repeatability of the three translational components (r,s,t) is no more than 0.05 mm. In the structure parameters of the turntable, the repeatability of the distancedbetween the two shafts can reach 0.023 8 mm, and the repeatability of the non-perpendicularityεbetween the two shafts can reach 0.001 19°. From Table 1, it can be concluded that the proposed method to calibrate the parameters of the tracking turntable has good repeatability.

To verify the measuring accuracy of the light pen whole space coordinate measuring system based on a tracking turntable, the system is used to measure a standard gauge block whose length is 1 000 mm. In order to highlight the superiority of the newly designed whole space coordinate measuring system, the gauge block is placed in many different directions around the turntable, and located at several different distances from the turntable. Because the camera can rotate driven by the turntable to track the movement of the light pen, the new coordinate measuring system can complete all these measurements through setting up it only once. This is impossible for the classical LPCMS. Table 2 shows the results of the measuring experiments at 7 different distances, and the measurements are repeated 10 times at each distance. It can be seen that the accuracy of the newly designed system can reach 0.25 mm within 10 m, which can satisfy the accuracy requirement of most in situ measurement of large workpieces such as ship hull, plane shell, etc.

Table 1 Results of 10 calibration experiments (PAR, AVE, STD, REP are abbreviations of parameter,

From the measuring results in Table 2, it can be seen that the worst accuracy appears when the measuring distance is 1.5 m. The reason is that the major error source at this distance is the rotation of the tracking turntable. The smaller the measuring distance is, the more widely the two shafts of the turntable rotate. Although calibrated, the parameters of the turntable inevitably involve some errors. The measuring errors will magnify rapidly with the enlargement of the rotation angles. As the measuring distance increases, the major error source gradually becomes the decrease of the spatial resolution of the camera. According to the perspective projection model, in the premise that the image resolution is unchanged, the spatial resolution decreases with the enlargement of distance. As a result, for the 1 000 mm standard gauge block, the measuring accuracy first decreases rapidly and then gradually increases with the enlargement of the measuring distance within a certain distance range.

Table 2 Measuring results of 1 000 mm gauge block (DIS, AVE, RAN, STD are abbreviations of distance,

4 Conclusions

In this paper, a new kind of light pen whole space coordinate measuring system based on a tracking turntable is designed. By introducing a turntable to the classical LPCMS, the camera can track the movement of the light pen during the whole measuring process. Benefiting from this, the measuring range is no longer confined to the view field of the camera. As a result, the newly designed system can accomplish whole space coordinate measuring, and therefore can be applied to large scale on-site measurement of workpieces such as ship hull, train body, plane shell, etc. To guarantee the accuracy of the new system, a method to calibrate the parameters of the tracking turntable is also proposed. After calibration, the real-time homogeneous coordinate transformation between the camera and the pedestal of the turntable can be accurately determined according to the turntable’s parameters and the azimuth angles of the two shafts. After experimental verification, the whole space coordinate measuring accuracy can reach 0.25 mm within 10 m. It can be concluded that the newly designed light pen whole space coordinate measuring system significantly expands the measuring range of classical LPCMS without losing too much accuracy. The research significance of the new system is extending the application scope of classical LPCMS and providing a brand new solution to large scale on-site coordinate measurement.