Development of 3D Scanning System for Robotic Plasma Processing of Medical Products with Complex Geometries

Abstract—This paper describes the development of an intelligent automated control system of a robot manipulator for plasma treatment of medical implants with complex shapes.The two-layer coatings from the Ti wire and hydroxyapatite powders are applied on the surface of Ti medical implants by microplasma spraying to increase the biocompatibility of implants.The coating process requires precise control of a number of parameters,particularly the plasma spray distance and plasma jet traverse velocity.Thus,the development of the robotic plasma surface treatment involves automated path planning.The key idea of the proposed intelligent automatic control system is the use of data of preliminary three-dimensional (3D) scanning of the processed implant by the robot manipulator.The segmentation algorithm of the point cloud from laser scanning of the surface is developed.This methodology is suitable for robotic 3D scanning systems with both non-contact laser distance sensors and video cameras,used in additive manufacturing and medicine.

1.Introduction

At present,the application of robots in the area of manufacturing is fast growing[1],[2].A modern robot manipulator can be considered as a means of allowing setting spatial positions and orientations of an arbitrary tool with high precision and accuracy.With the addition of a distance sensor or a vision system element (camera or/and projector),the robot manipulator can provide an excellent system of surface scanning.Currently,industrial robot based three-dimensional (3D) scanners are used in quality control systems,automatic packing lines,and assembly systems[2]-[6].The application of robots based scanners has been steadily expanding,especially in medical metrology[7],[8].

Robots are currently used in manufacturing processes,such as plasma cutting,plasma surface treatment in the form of plasma spraying of powders or wires coatings[9]-[11].The plasma processing requires the accurate setting of a number of parameters (the distance from the plasma system nozzle to the surface of a workpiece,the nozzle movement speed,the angle between the plasma jet and the surface being treated,etc.[9]) during the entire processing time.Exceeding these parameters beyond the permissible limits can lead not only to defective products,but also to an accident (a short circuit).In cases when the robot program is generated according to a given geometrical model of a processed workpiece or part,the deviation of the shape of the real object from the model often leads to the violation of process parameters with all its undesirable consequences.This problem is particularly acute in the case of objects with complex shapes,when small relative errors of geometric parameters and object positioning cause large deviations of the distances between the tools mounted on the manipulator and the object surface.The optimum solution to these problems is pre-scanning the surface of an object.It should be noted that the methods associated with the construction of 3D-scanning systems attached to a robot are of considerable interest to both practitioners and theoreticians.The research related to these topics is conducted by both research and development units of companies,specializing in the production of control and measuring devices,and the academic community[3]-[8].

Our motivation in this research study is the development of an intelligent robotic system for plasma processing of complex shapes based on preliminary 3D scanning of the treated surface by the robot.In this study,we address both the above-described scientific problem,as well as the practical challenges which authors met with the use of the Kawasaki industrial robot (Kawasaki Robotics,Japan) for the plasma surface treatment[11]-[13].We especially focus on the application of a two-layer biocompatible coating of Ti/hydroxyapatite (HA) to medical implants[11].To ensure the reliable secondary orthopedic implants setting,it is necessary to make their surface porous and rough.One of the methods to develop a suitable biocompatible coating on the implants’ surface is to spray the porous Ti coating,followed by the application of an outer layer of HA[4].HA is the calcium phosphate mineral of the apatite group.In medical implant manufacturing,synthetic HA is used as implants coating,promoting a new bone formation.The main commercially viable method of HA coatings is plasma spraying[14]-[16].The porous structure of the coating,developed by the plasma spraying method,promotes the effective ingrowth of bone tissue into the pores of the implant significantly,which in turn contributes to its reliable fixation to the bone.The plasma coating of surfaces of complex configurations,such as the medical implants of endoprostheses,presents a challenge for the implementation of the plasma spraying technology.It requires automated manipulations of the plasma jet and/or the substrate along with robotic control for the appropriate treatment of a surface[9],[10].The deviation from the desirable spraying angle to the surface required spraying distance and a predetermined constant speed of movement of the plasma source along the surface can compromise coating properties.

The purpose of this study is to develop a 3D scanning scheme based on contactless distance sensors installed on an industrial robot manipulator and to design the segmentation algorithm of the point cloud obtained at the stage of rough scanning of the surface.The brief review of existing modern 3D scanning methods and an analysis of their applicability to the robotic plasma treatment of medical implants have been carried out.This is used to develop an intelligent automated control system of a robot manipulator for plasma treatment of medical implants with complex shapes.

2.Experiment

The research has been carried out at a pilot production site established in the East Kazakhstan State Technical University with an industrial complex for plasma processing of materials on the basis of Kawasaki RS-010LA,an industrial robot (Kawasaki Robotics,Japan).The robot consists of movable parts with six degrees of mobility for moving the equipment installed therein according to a predetermined trajectory.The robot’s arm is equipped with “MPN-004” microplasmatron produced by E.O.Paton Electric Welding Institude(Ukraine) for microplasma deposition of wires or powders coatings.Fig.1 shows the robotic processes of microplasma spraying of a wire (Fig.1 (a)) or powders (Fig.1 (b)) and a set of a hip joint endoprosthesis with a two-layer microplasma sprayed coating (Fig.1 (c)).

Fig.1.Robotic pilot production site processes of microplasma spraying of (a) Ti wire,(b) microplasma spraying of HA powders,and (c) a set of a hip joint endoprosthesis with a two-layer microplasma sprayed coating.

The Kawasaki RS-010LA robot manipulator characteristics are as follows:The positioning accuracy is 0.06 mm;the maximal linear speed is 13100 mm/s;the engagement zone is 1925 mm;the working load capacity is 10 kg.Kawasaki robots are controlled by the Advanced Superior (AS) software system.

The application of the Ti/HA two-layer coating to the surface of the implant material has been carried out (Fig.1).In order to enhance the integration of the implant to the bone,an optimal coating needs to be applied to the surface of the implant.It has been shown by previous research that the optimum porous Ti coating has the thickness from 200 μm to 300 μm and a pore size of about 150 μm.A layer of HA should be applied over the porous Ti coating.The ingrowth of bone into the Ti sub-layer will ensure the essential strength of the implant’s integration to the bone,and the HA coating will provide the necessary biocompatibility.When selecting the thickness of applied layers,it was taken into account that the microplasma application of biocompatible coatings with a thickness of fewer than 200 μm and a large pore size of about 150 μm can result in the formation of through-thickness porosity.This can cause an adverse contact and interaction of the implant metal with the internal environment.In addition,a Ti sublayer with well-designed surface morphology will increase the adhesion strength of the HA layer to the surface of the implant.

Grade 5 electro low interstitial elements (ELI) Ti alloy samples were used as substrates for microplasma spraying.The reference standard composition of such grade in wt.% of the Ti base is as follows:Fe is 0.25% to 0.40%,N 0.05%,O 0.13% to 0.20%,Al 5.50% to 6.75%,C 0.08%,V 3.50% to 4.50%,and H 0.015%.As the coating material,the HA powder of a fission-fragment shape with large size difference across the particle axes was used.The powder was represented by particles with diameters from 40 μm to 80 μm.The phase composition of the HA powder is an all-crystalline combination of Ca10(PO4)6(OH)2with Ca/P-1.67.For the application of Ti coating,a wire of 0.3 mm in diameter was used from Ti of BT-1-00 (GOST 19807-91) Grade.The composition of such grade in wt.% (max) of the Ti base is as follows:Al is 0.3%,Fe 0.15%,Si 0.08%,С 0.05%,N 0.03%,Н 0.003%,and O 0.12%.

The parameters for applying two-layer coatings from the Ti wire and HA powders to the Ti alloy substrates are presented in Table 1.

3.Results and Discussion

3.1.Development of Robotic 3D Scanning System—Point Scanning Methods

At present,the most widely used 3D scanning systems are based on a one-dimensional (1D)distance meter.There are two ways to scan an object—the first method is to move the sensor along the surface of the scanned object and the second way is to move the scanned object.As an example,consider the method proposed by Reyeset al.[17].In order to scan solid 3D objects,[17]made use of a very simple method.It consists of taking distance measurement from a solid 3D object that is subjected to a controlled rotation.After each revolution of the object,the distance measurement needs to be taken at different heights of it.In this manner,the outline of the object can be obtained in every revolution by gathering the distance measurements and corresponding rotation degrees.The outline distances are obtained by the difference between a fixed distance and distance measurements.

Table 1:Microplasma spraying parameters

It should be noted that there are intermediate methods of scanning,that is,methods in which both the sensor and the scanned object are mobile.Rahayem and Kjellander used this type of setup for their research as described in [18].Their setup used a turntable and a motor-driven potter-wheel on which the object is placed upon.This is popular in automated 3D scanning and significantly reduces the need of the scanner being mobile.If the setup is properly calibrated,the scanner only needs to be moved vertical or in an arc alongside the object.Scanning technologies based on 1D distance sensors can be classified as the point technology.The point scanning technology is similar to a physical probe in which it uses a single point as the reference,repeated many times.This is the slowest approach as it involved lots of physical movements by the sensor (or the scanned object).The optical scanning technology provides two other types of scanning methods:

1) Area,which is technically difficult.This group of methods is based on technologies of machine vision.

2) Stripe,the second system is soon found to be faster than point probing as it used a band of many points to pass over the object in one time.It is accurate and matched both requirements of speed and precision.

Speaking about the technology of machine vision,we mean primarily the systems of stereo vision.Stereo vision is a technique aimed at extracting depth information of a scene from two camera images[19].The position difference in pixel in two images produces the depth output.Stereo vision algorithms search for a correspondence between the pixels of the right and left images.In the theory of machine vision,this problem is formulated as the problem of constructing the so-called disparity map[19].

Fig.2.Proposed scheme of the 3D-scanning system based on laser sensors installed on the manipulator of the industrial robot Kawasaki RS10LA.

In order to develop a robotic 3D scanning system,we proposed a 3D scanning scheme based on laser distance sensors installed on the Kawasaki RS10L robot manipulator (Fig.2).In this scheme,the interrelation of components of the system,such as the robot manipulator,sensor (3D scanner),E40 controller,and personal computer (PC),is clearly represented.For serial data transmission (for communications between the robot controller and PC),the Recommended Standard-232 (RS-232) was used.

The system can scan an object based on the measurement of the distance from the surface of an object in a discrete set of points forming a network with the specified geometric parameters imposed on the surface of the object.A 3D model of the scanned object is built using the interpolation procedures of grid points.Thus,the creation of a system of 3D scanning requires the construction of a data collection system involved in the formation of tasks for the robot manipulator.The system consists of consecutive moving according to scanning points,at each point performing actions necessary for scanning,and also storing the measurement results in the memory.As precision distance sensors usually have a limited operation range,and some of them require the orientation of the axis of the sensor in a direction perpendicular to the area of an object subjected to scanning,it is necessary to implement a complete system consisting of “rough” and “fine”scanning subsystems.The system of “rough” scanning,which uses the position sensors with a wide range of measurable distances and is insensitive to the orientation relative to the surface,is used for setting “fine”scanning sensors in the operating position and prevents emergency situations during the process of scanning.Thus,in order to implement the scanning system,we must develop the hardware and software for the data collection system briefly described above.The system development should include research on testing the algorithms of setting the distance sensors in the operating position and the study of the system performance characteristics in test situations with known geometry.

The main difference between our current methods of robotic 3D scanning from the methods described in previous researchers’ work[3]-[8]is that we propose to generate a robot program by a 3D model of the object to be processed.This 3D model is obtained by scanning the object using an industrial robot as a key component of the scanning system.The basic idea of our method is the development of a combined system for scanning with splitting the scanning process into two phases:A rough scanning phase and a refining phase.The rough scanning vision system uses a single camera mounted on the manipulator and a fixed structured light projector.During the rough scan phase,photography of an “illuminated” object from several points of space is performed (with the known orientation of the principal optical axis of the camera).Using the images obtained in the shooting process,the software of the scanning system produces a segmentation of the object surface and builds an approximate 3D model of the object.According to the segmentation results,a set of reference points is selected on the surface with known spatial coordinates.This will enable us to fairly and accurately construct the 3D model of the object.After selecting the reference points,the software generates the program of the manipulator.The program defines how the manipulator successively passes the reference points,performing surface scanning at each of the points.

The vision system is built on the original algorithm,implemented in three stages of processing of the image obtained by the camera:1) Building a function module of the intensity gradient;2) construction of a set of lines of this function level (the structuring of the system of level lines radically simplifies the task of finding the correlation between the lines obtained in processing the two photos taken at different camera positions);3) calculation of the spatial coordinates of the scene points,whose images lie on these lines.

For accurate scanning,we use a point scanning system based on the industrial robot manipulator Kawasaki RS010LA (Fig.2).We use a laser triangulation sensor LD2 (Russia) as a distance sensor.

3.2.Theoretical Foundation of 3D Scanning by Robot Manipulator

Data is collected in two coordinate systems.One is the Cartesian coordinates of the end effector,and the second is Euler’s angles,e.g.,rotational angles aboutX,Y,andZaxes,which relate to the tool orientation.Fig.3 presents the robot’s coordinate system.

The work coordinate system homogeneous vector (xn,yn,zn,1) defines the world coordinate system homogeneous vector (X,Y,Z,1) by means of transform (1):

According to Fig.3,the homogeneous transformation matrixAcan be described as

where

Andrx,ry,andrzare the coordinates values of the work coordinates system origin;α1,α2,andα3are the rotational angels aboutX,Y,andZaxes,respectively.

Fig.3.Robot coordinate system.

Thus,if we organize the measurement in such a way that the laser device measures the distance along theznaxis then to build the 3D image of the scanned object,in our case we only need the last column from the result matrix (2).The robot working frame (including the laser measurement head) moves along the given path during 3D scanning.The path and the trajectory planning for this application is an important and interesting research and development topic.

3.3.Application System Concept

For the given application,the data synchronization was achieved by the application of supplementary software.PC is connected with the robot controller via RS232 serial interface (Fig.2).In this case,it is possible to operate the process of the measurement using a PC software via the user interface,or automatically via the specialized software application.The sequence of events for the system is briefly described below:

-Sending the robot to the position according to the planned trajectory;

-sending the request of the measurement to the sensor;

-getting the results of the distance measurement;

-reading current coordinates of the manipulator;

-transferring received data to the data base;

-waiting for the next request/stop.

3.4.Advantages of Use of Laser Sensor

There are at least two main advantages in using the laser instead of any other signal as follows:

1) The laser provides a concentrated narrow beam with a very low deviation.It allows measuring the distance to very tiny objects,such as the depth of very narrow holes.This is not achievable by ultrasonic or radio beams.

2) The laser beam is monochromatic.This simplifies measurement processing significantly.The laser has a high intensity which is nearly not diffusing,so it is not as much attenuated by the environment as other mediums.Therefore it allows performing long-distance measurement.

There are four most common measuring principles implemented in laser range finders and laser scanners.These include the time of flight method,triangulation method,interferometer method,and phase shift method[20].Our choice in this study is the triangulation method,as most precious in relatively short distances.Binocular triangulation laser sensors intend for the use in automation systems and the non-contact measurement of various geometric,such as the thickness,straightness,internal and external diameters,scanning the complex geometries,and the determination of the position of objects.In addition,the sensor is able to receive positioning signals from machines for the implementation of 3D scanning systems.Because the two sensors are symmetrically positioned relatively to the laser beam,the binocular sensor is able to work with a deeper relief without the loss of signal.

3.5.Development of Segmentation Algorithm of Point Cloud

Segmentation of the surface,in turn,is necessary for determining geometrically homogeneous regions of the surface and determining the reference points within the region in which precision point scanning will be performed.The greatest difficulty in constructing an adequate analytical 3D model is the surface with sharp kinks.This is due to the technique developed is mainly for smooth surface approximation.Thus,the surface segmentation technique plays an important role in the proposed surface treatment technology.This will modify the surface scanning to accommodate all types of the surfaces.For the segmentation of the surface,we use a variation of the region’s growing algorithm.The proposed procedure for automatic segmentation of the surface is based on a local analysis of the Gaussian and mean curvature of the surface estimated by constructing a nonparametric analytic model.In the proposed technique,a cloud of points is interpreted as a map height,i.e.,it is assumed that a discrete function of two variablesZk,m(Xk,Ym) is given on some grid in theX-Yplane.At each point,the surface is modeled by a second-order polynomial in local coordinates.Calculation of the coefficients of the approximating polynomial identified by the method of least square is carried out by considering the selected number of neighboring points lying within a circle of a given radius.The main advantage of the non-parametric approach is its generality:This approach does not introduce any assumptions about the local geometry of the surface and,moreover,does not offer an analytical model of the surface.We consider the approximation of a surface in a local neighborhood of an arbitrary pointP(xk,ym) of the form (3):

whereu=xi−xk,v=xj−ym,andc0=zk,m.

This expression can be regarded as an expansion in the Taylor series of a continuous smooth functionZ(u,v).The derivations ofZare defined as (4) to (9):

Calculation of coefficientsc0,c1,…,c5of the local approximation of the form (3) is performed by regression analysis methods based on using the values of the coordinates of the points adjacent toP.More precisely,we give a certain value of the radiusr(chosen empirically) and consider the set of points defined by

wheredi,jis the distance between the point with the coordinates (xi,yj) and the pointP,determined according to (11)

The coefficients of the first quadratic form are calculated from the coefficientsc1,c2,…,c5of the approximate polynomial (3) by (12) to (14).

The matrix of the first quadratic form is defined as

Also,the coefficients of the second quadratic form are related to the coefficients of the approximate polynomial by (15) to (17)

And the matrix of the second quadratic form is defined as

In turn,the coefficients of the first and second quadratic forms make it possible to calculate the mean curvatureHand the Gaussian curvatureKof the surface at the pointPas below:

Thus,the values of the local curvatureKandHcan be automatically obtained for each point as a function of the coefficients of the approximate polynomial.In addition,such curvature is invariant to the accepted frame of reference,which is a very important property when analyzing the shape of the surface.

4.Conclusion

Based on the analysis of contemporary 3D scanning methods and parameters of the existing technological equipment,and taking into account the current problems of plasma processing of medical implants with complex shapes,a 3D scanning system based on laser triangulation distance sensors installed on the manipulator of the industrial robot Kawasaki RS10L has been developed.The methodology,established in 3D scanning and setting up the robot used in microplasma spraying,accordingly provides a generic support system in the coating of objects with complex shapes.

The segmentation algorithm of the point cloud obtained at the stage of rough scanning of the surface has been designed to create an intelligent automated control system of a robot manipulator for plasma treatment of medical implants with complex shapes.This segmentation algorithm is applied to the processing of noisy data and suitable for robotic 3D scanning systems with both non-contact laser distance sensors and video cameras.

The results of the research are of significance for a wide range of researchers developing automated processes for plasma surface treatment and robotic 3D scanning systems used in additive manufacturing and medicine.