PNN-SVM Approach of Ti-Based Powder’s Properties Evaluation for Biomedical Implants Production

Ivan Izonin,Roman TkachenkoMichal Gregus,Zoia Duriagina3 and Nataliya Shakhovska

1Lviv Polytechnic National University,Lviv,79013,Ukraine

2Comenius University in Bratislava,Bratislava,82005,Slovak Republic

3The John Paul II Catholic University of Lublin,Lublin,20-708,Poland

Abstract:The advent of additive technologies has provided a significant breakthrough in the production of medical implants.It has reduced costs,increased productivity and accuracy of the implant manufacturing process.However,there are problems associated with assessing defects in the microstructure,mechanical and technological properties of alloys,both during their production by powder metallurgy and in the process of 3D printing.Thus traditional research methods of alloys properties demand considerable human,material,and time resources.At the same time,artificial intelligence tools create opportunities for intelligent evaluation of the conformity for the microstructure,phase composition, and properties of titanium powder’s alloys.It provides new possibilities for the efficient production of biocompatible implants for various functional purposes.However, the accuracy of the methods and models used should be as high as possible.In this paper we designed a hybrid PNN-SVM (Probabilistic Neural Network-Support Vector Machine)high-precision approach for the intelligent evaluation of alloy properties for additive manufacturing of biomedical implants.We have proposed a new approach for extending the dimensionality of input data space by the outputs of the summation layer of the modified PNN topology.Subsequent classification based on the expanded dataset is performed using SVM.We conducted experimental modeling of the proposed approach using a data set on the properties of titanium alloys Ti-6Al-4V and Ti-Al-V-Zr.We have demonstrated a significant increase in the accuracy of the PNN-SVM scheme compared to the single classifiers that form it and other machine learning methods.

Keywords: PNN; SVM; hybrid systems; classification accuracy; medical implants;additive manufacturing;3D printing;titanium alloys

1 Introduction

Modern biomedical engineering has come a long way in evolution[1].Precious metals were used as the first biocompatible materials.Then there were self-dissolving sutures-hair,animal tendons,silk threads, and copper wire.Then Leonardo da Vinci proposed the first contact lenses made of water.The discovery of the phenomenon of osseointegration of titanium in bone tissue[2]has identified the rapid development of new biocompatible materials and technologies to improve the quality of human life.

The classic requirements for biocompatible materials are as follows.First, it is necessary to maintain parity between their tribological characteristics,strength,Young’s modulus,and resistance to destruction.The perfection of a design and duration of reliable functioning of implants depends on it.Second,no toxic chemical reaction products should form on the interface between the implant and living tissues or cells.In this case, the organism’s immune, hematological and regenerative response will be the markers that indicate the material’s real biocompatibility.At present, among the metalbased biomaterials for the manufacture of implants for various functional purposes use alloys of ironchromium,cobalt-chromium,and titanium-based alloys[1,2].The chemical composition,phase state,and properties of biocompatible titanium alloys gradually improve[3,4].Alloys with the memory effect of the shape for the Ti–Ni system(nitinol)and the Ti–Nb–X(O,Sn)[5],Ti–Mo–X(Ga,Ge)[6],Ti–Ta–X(Sn,Zr)[7],Ti–Cr–Al[8]have been widely used.

Nevertheless,there is a danger of possible cytotoxicity of the body to bioinert titanium implants.One of the most effective methods of preventing adverse organism reactions to the rejection of implants is applying protective nanostructured coatings with high adhesive strength, predetermined phase composition,and parameters of the surface microgeometry.In addition,there is another technological approach for regulating the microstructure, mechanical and biocompatible properties of Ti-alloys.It can be achieved by conducting surface plastic deformation (SPD) or local thermomechanical treatment using highly concentrated laser or plasma radiation streams.

In the last decade, additive technologies have been gaining momentum for the manufacture of implants from biocompatible polymers and powdered titanium alloys.It allows abandoning organ transplantation.Still, it requires improving existing biocompatible materials and developing new technologies in virology,surgery,anesthesiology,and resuscitation.

Investigations by Market Research Future show an increase in the global market for additive technologies for medicine and pharmacology in 2018–2023 at 18% [9].The indisputable advantages of 3D printing include high accuracy,the ability to reproduce bone tissue,skin,blood vessels,internal organs, accelerate the provision of medical care, the manufacture of drugs, a large market of new innovative materials [10–12].However, the quality of alloys derived from titanium powders must be high, ensuring the high quality of the finished implant.When sintering such alloys in additive manufacturing,it is necessary to ensure their best biocompatibility with the human body.Traditional methods of studying the properties of sintered alloys require considerable human,material,and time resources[13].It requires searching for alternative approaches to assessing the conformity of alloys’microstructure and other properties for the efficient production of biocompatible implants.The use of artificial intelligence tools can increase the effectiveness of this problem [14].However, ensuring the high accuracy of such methods is an important issue.The use of the above approach will provide an opportunity to replace traditional methods of alloys investigation with faster, cheaper, and more accurate.

This paper aims to design a hybrid intelligent approach to solve applied problems of effective additive manufacturing of biomedical implants,mainly to evaluate defects in the microstructure and properties of titanium powder’s alloys before its sintering additive technologies.

The main contribution of this paper,as well as its practical value,can be summarized as follows:

· we proposed a new approach for intelligent evaluation of the titanium alloy’s properties based on a set of characteristics of its powders.It reduces material,time,and human resources when studying the quality of biomedical implants obtained through additive technologies;

· we proposed a new approach for expanding the input space of the initial set of tabular data by adding the output signals of the summation layer of Probabilistic Neural Network(a collection of probabilities belonging to each class,which together form a complete system of events).This approach is based on the well-known Cover’s theorem and allows to increase the likelihood of correct classification by the chosen machine learning method;

· we designed a hybrid PNN-SVM (Probabilistic Neural Network-Support Vector Machine)system to improve the efficiency of solving classification tasks for additive manufacturing of biomedical implants.We have demonstrated a significant increase in the accuracy of the proposed approach compared to the single classifiers that form it.

The rest of the paper is organized as follows.In Section 2, we conducted the literature review.Section 3 describes the proposed hybrid approach.Results of the experimental investigation of the proposed approach and optimal parameters selection are presented in Section 4.In Section 5, we introduced the comparison of the designed approach with existing machine-learning-based (MLbased)methods.

2 Related Works

In recent years,machine learning technologies[15,16]are increasingly penetrating various industries to accelerate or reduce the cost of a number of processes there.This also applies to the additive production of biomedical implants.In this chapter,the authors review and analyze the latest research in this area.

Paper [17] is devoted to reviewing the current state, trends, and prospects of modern additive manufacturing.The authors identified three main problems in this case.Among them are limited material resources, the appearance of various kinds of defects, and unsatisfactory product quality obtained by additive technologies.The authors outlined the prospects and possible areas of application of artificial intelligence to increase the efficiency of additive manufacturing.

In[18],many tasks for additive manufacturing are outlined,which can be solved more effectively using machine learning tools.The authors consider artificial neural networks and Support Vector Machine (SVM) the most used supervised machine learning algorithms to achieve this goal.The analysis carried out in the paper shows that these methods can effectively solve several applied problems of additive production,such as biomimetic designs,generating lattice structures with predefined mechanical properties, process monitoring, parameter optimization, identification of printability of components in material extrusion;and many others.However,this work,as the previous one,is purely theoretical.

In [19], the authors proposed ML-based methods for the monitoring process in additive manufacturing.The main aim of this method is to ensure the quality of the product, which is made by 3D printing.The authors used the data on the quality of the obtained sample collected by an accelerometer.Then a binary SVM-based classifier was used to predict the faulty component.The disadvantage of this method is the relatively high error of its operation.However, the accuracy of the proposed method,even about 79%,provides the ability to preserve raw materials during printed specimens.

The authors of [20] proposed a new method for constructing a process map during the manufacture of biomedical parts.It is based on using a standard SVM classifier and does not contain novelties in terms of new technical solutions.However,the use of SVM,in this case,provides a high practical value to the proposed method.It reduces the number of experimental studies required to optimize the parameters for obtaining the specimens with the required properties.In addition, the authors outlined some physical interpretations of the use of SVM in solving the stated task.SVM’s decision function can serve as a semi-quantitative indicator for the porosity density of specimens made by additive manufacturing.

Paper [21] is devoted to the problems of size control of medical specimens manufactured by 3D printing.The authors used several machine learning methods to control the specimen’s width,thickness,and length in the process of additive production.The conducted experimental investigation has shown much higher accuracy of the applied methods’work compared to linear models.In addition,the authors identified the best machine learning methods that are best suited to control the above three parameters for laser-sintered specimens.

In[22],the authors consider the prediction mechanical properties task in metal additive manufacturing.Such manufacturing provides an opportunity for easy construction of metal designs of various geometrical forms.However,the problems of cooling and heating with each new layer of the structure cause a change in the product’s mechanical properties.It may affect its performance.The authors used wavelet transforms and convolutional neural networks to predict the mechanical properties for metal additive manufacturing.Experimental studies on the use of this hybrid approach have confirmed the satisfactory accuracy of the proposed method compared to other machine learning methods.The disadvantage of this study is the small amount of noisy data that can significantly distort a particular machine learning(ML)model.

In[23],authors developed a machine learning approach to predict the mechanical deformation in Ti-based alloys.This approach has shown high efficiency compared to,for example,the“d-electron design method”[24].In addition,it does not require massive calculations,even when included in the model of multi-component alloys.However,the data set for modeling contained only eight attributes,limiting the model’s scope.It can consist of many more factors that affect the mechanical deformation of Ti-based alloys.

The authors in[13]conduct performance evaluation of the different machine learning algorithms to solve the problem of the titanium powder alloys’properties.This study shows that the highest accuracy of 78%was obtained when using an SVM-based classifier.In addition,it provides the highspeed implementation of the training procedure.Other machine learning algorithms, including the ensemble ones,AdaBoost algorithm,shows much worse results.

Paper [25] presents an attempt to improve the results of classical machine learning algorithms in solving the stated task.The authors used classical Probabilistic Neural Network (PNN) to solve the Ti-based powder’s alloy properties evaluation task in additive manufacturing.The choice of such Artificial Neural Networ(ANN)is justified by the small amount of data set that was processed and the lack of training procedures for this ANN type.

An important parameter of this ANN,which affects the accuracy of PNN,is the smooth factor.Selection of the optimal value of the smooth factor by a complete search on a small interval allowed reducing the error in solving the task.In particular, the authors obtained 80% accuracy while maintaining high performance compared to, for example, ensemble methods.However, the error of operation is relatively high.It makes it impossible to apply the proposed approach for solving applied tasks in the production of biomedical implants.This situation necessitates the search for new models and methods to solve Ti-based powder’s alloy properties evaluation task for high-precision additive manufacturing.

3 Materials and Methods

This section describes the hybrid PNN-SVM approach developed by the authors to classify titanium alloys sintered in the 3D printing process.It is based on several steps:

· The preliminary processing of the existing data set (a mixture of spherical and non-spherical Ti-6Al-4V and Ti-Al-V-Zr powders)by a modified version of PNN;

· An adding the output signals of the summation layer of this ANN to the initial data set;

· An applying an SVM-based classifier to determine one of the four classes of properties of sintered titanium alloys.

Each component of the proposed approach in more detail is considered in the next subsection.

3.1 Probabilistic Neural Network and its Modification

Probabilistic Neural Networks(PNN)belong to the computational intelligence tools,for which the learning procedure is not provided,and the application is quite simple[26].Effective operation of such ANN is possible with short data samples.Under conditions of large data sets,it is made significant time delays in the operation mode.

It is known [27] that for PNN networks, the solution of classification tasks is carried out by calculating the probability densities for each class and is based on the Kernel Density Estimation method.Bell-shaped(Gaussian)functions are often used as kernel functions.The output layer of the elements of the PNN network is connected to the radial elements belonging to its class.It summarizes the outputs of all radial elements belonging to its class(Fig.1).

Therefore,output signal values are formed in proportion to kernel estimation of the probabilities belonging to the respective classes.

Traditionally[28],the functioning of PNN networks is described by relations:

whereP(k)is the probability that the input vector belongs to thekthclass(k=for this taskkmax=4);andGu,ikis a Gaussian function:

Eu,ikis the Euclidean distance from the current input vector toikth-vector of thekthclass;σis a smooth factor;Nkis the number of sampling vectors of thekthclass.

The construction of the PNN network based on the relation(1)considers the non-uniformity of the representation of individual classes in the sample.Still,there is some inaccuracy in the probabilistic assessment of the belonging of current input vectors to a particular class.In this case,the condition regarding the existence of a complete system of events belonging to the input vector to one of thekmax classes is not formally fulfilled.

Figure 1:PNN topology

One more variant of the formula for estimation of belonging of the current input vector to thekthclass is demonstrated below:

The application of formula (3) provides a more precise probabilistic estimate.It confirms the complete system of events, where the input vector belongs to one of the classes, and the sum of the corresponding probabilities is equal to one[29].

3.2 SVM

The SVM-based classifier is often used for solving applied tasks in various fields of industry,medicine,economics,and science.Among its advantages should be noted high speed,high accuracy,and ease of implementation.In addition,all major software packages related to data mining already contain an excellent implementation.

The SVM method is based on the possibility of constructing a separating surface using a small number of points (support points), which belong to the critical area for the separation of objects.All other points that uniquely belong to one of the specified classes are used only during the implementation of the optimization algorithm.The advantage of this method is that it builds an optimal hypersurface or hyperplane by maximizing the width of the margin between objects.This provides resistance to outliers.

If we talk about the mathematical interpretation of this linear classifier, then the method is reduced to the quadratic optimization problem with linear constraints.This approach provides a straightforward solution,which is a significant advantage over,for example,artificial neural networks of the iterative type(where the random initial initialization of the scales with each new run of the ANN returns a new result).

If it is necessary to consider nonlinearities in the data,this method uses the principle of expanding the space of input data.For this purpose,various functional transformations implemented in the form of special kernel functions are used.Among the most famous are the linear kernel,polynomial,rbfkernel and sigmoid kernel.

A Gaussian kernel with a radial-base function(rbf-kernel)can be written as:

whereγis the kernel parameter that should be optimized.

The sigmoidal kernel can be written as follows:

whereγ,β0are the kernel parameters to be optimized.

Due to their great popularity,these two kernels will be used in the experimental modeling of the developed hybrid system.

3.3 Hybrid PNN-SVM Approach

The hybrid approach to designing the technology for evaluating Ti-based alloy’s properties is based on the consistent use of PNN and SVM.Its main idea is that PNN is used as a tool of preprocessing data for further use of the results of its work by other classifiers.In essence,the designed hybrid system uses the outputs of the summation layer but not the output layer of the PNN topology from Fig.1.

The processed data set based on one of the PNN realization variants ((1) or (3)) in the form of output signals of the summation layer of PNN from Fig.1 are added to each relevant vector of the initial training and test data samples.Thus, there is an expansion of the input data space by the probabilities of belonging to each class.The theoretical justification for this approach is Cover’s theorem on image separation.The proposed scheme should increase the likelihood of the correct answer when solving the classification task based on the machine learning algorithm, mainly to evaluate the properties of titanium alloys.

In the next step,an SVM-based classifier processes the obtained in such way extended data set.The output signals of this classifier will be the results for solving the stated task.

A simplified flowchart of the proposed approach is shown in Fig.2.

The main steps of the algorithmic implementation of the proposed approach(training procedure)can be summarized as follows:

· to divide the data to train and test datasets;

· to obtained the output signals from the summation layer of the PNN(probabilities of belonging of each observation from the training data set to a specific class of the task);

· To extend the training dataset by adding additional columns with the probabilities obtained in the previous step;

· To train the SVM with chosen kernel.

Figure 2:Proposed hybrid PNN-SVM approach

The proposed approach has several advantages:

· It is easy to implement and apply;

· It does not require significant computing and energy resources(because PNN does not require training procedures);

· It provides a high speed in the case of processing small data sets(because PNN does not require training);

· It provides for setting only one parameter for PNN and basic SVM parameters;

· It provides an increase in accuracy compared to every single method from the proposed hybrid scheme.

4 Modeling and Results

4.1 Dataset Description

In this paper, we investigated the properties of titanium alloy of the Ti-6Al-4V and Ti-Al-VZr systems [23].We took into account both powders of spherical and non-spherical shapes (Fig.3).The first one is often used for additive production due to its exact shape and predictable properties.However,obtaining it is a very resource-intensive and,therefore,expensive task.Different structures characterize the second form of powder materials,but the meager cost of receiving them.

The authors have collected in [25] and processed a data set containing 480 observations in this paper.For each of them, 20 features were obtained.All attributes are grouped into five categories:investigated material(4 features)average values of polydispersity,%(4 features),the content of growths on the surface of particles,%(4 features),diameter range of the powder particles(5 attributes),and the form of powder particles(3 features).Each of the obtained observations was assigned to one of the four classes(Fig.4).Detailed descriptions of the collection and preprocessing procedures for the Ti-based alloys dataset can be found in[25].

The obtained data set was randomly divided into training and test data sets (80% and 20%).The task was to effectively solve the classification task for predicting the properties of the mixture of titanium alloy powders that form high-quality or low-quality material in the application of additive technologies to manufacture biomedical implants.

Figure 3:Fragments of images showing the morphology of the structure of titanium alloys powders:(a)spherical shape;(b)non-spherical shape

Figure 4:Classes of titanium alloys after sintering using additive technologies

4.2 Selection of PNN Implementation Algorithm

For choosing the proper PNN implementation to be used in the developed hybrid approach,we calculated different performance indicators (Total accuracy, Precision, Recall, and F1-measure) for the first algorithm based on (1) and for the second algorithm based on (3).The calculations were based on the smooth factor(σ)optimal value selection for both algorithms using exhaustive search[30]in the range from 0.01 to 10 with steps of 0.01.The results of this experiment for all four classes using F1-measure are shown in Fig.5.

As shown in Fig.5,F1-measure for the first two classes is very close(at optimal values of smooth factor).However,Algorithm 1 shows better results.The situation is entirely different when recognizing classes 3 and 4.In this case, Algorithm 2 shows significantly better results.In the context of the problem,where classes 3 and 4 denote an alloy with a possible defect and a defective alloy,respectively,their correct recognition is a priority.In case of incorrect identification of alloys of these classes,all biocompatible products made through 3D printing can be defective and even harm their owner in case of application.

Tab.1 summarizes the average values of the results of this experiment(Accuracy,Precision,Recall,and F-measure)for all four classes.

Figure 5: The value of F1-measure for both algorithms of the PNN implementation when changing the smooth factor value(blue line indicates Algorithm 1,orange-Algorithm 2):(a)for the class#1;(b)for the class#2;(c)for the class#3;(d)for the class#4

Table 1: Performance indicators for both variants of the PNN realization

Tab.1 shows that three of the four performance indicators show a higher efficiency of PNN implementation according to the second algorithm.

Given that the total number of correct results is more significant for the second PNN algorithm,and most of the metrics showed its higher accuracy,it should be used in the practical application of ANN to build a hybrid approach.

4.3 Selection of the Optimal SVM Kernel

We can consider SVM as a nonlinear generalization of some linear classifier based on expanding the dimension of the initial data using special kernel functions.The use of the latter provides the possibility of constructing separating surfaces of various shapes.The basic idea of using kernels (as in the case of using PNN in this paper) is to increase classification accuracy by mapping the data into a higher dimension space.That is why selecting the optimal kernel is essential for the practical implementation of the developed hybrid approach.

The experimental modeling was performed using two kernel functions (rbfandsigmoid) under otherwise identical conditions.The implementation of SVM from thescikit-learnlibrary is taken as a basis[31].The training and test data set was previously expanded with the outputs of the second PNN algorithm,which describes the complete system of events.The results are summarized in Tab.2.

Table 2: Performance indicators for SVM with rbf-and sigmoid kernels

As shown in Tab.2, SVM withrbfkernel offers significantly higher performance on all performance indicators than SVM with thesigmoidkernel.That is why therbfkernel will be used as the optimal value for the SVM operation in the proposed hybrid approach.

4.4 Results

As a result of the performed experimental studies concerning the selection of optimal parameters of work of the hybrid scheme,the following results of its work summarized in Tab.3 are received.

Table 3: The results of the working of hybrid PNN-SVM approach

As shown in Tab.3,overfitting is not observed due to the increase in the number of independent attributes.The number of additional features is equal to the number of classes, and in our case,only four variables were added.In addition,the values of all performance indicators are high,which indicates the possibility of applying the proposed approach in practice.

5 Comparison and Discussion

The comparison of the designed hybrid PNN-SVM approach was performed using several existing machine learning algorithms and neural networks without training.The results of the comparison are shown in Tab.4.

Table 4: Comparison with other ML-based classifiers

Tab.4 clearly shows that the classical machine learning algorithms provide classification accuracy close to 80%.The same results were obtained in [13].From the point of view of the stated task, the received accuracy is satisfactory.However, the practical use of existing methods with the possibility of a 20% error is somewhat complicated.It is due to significant material losses in manufacturing a biomedical implant from defective material,as well as high risks to human health and even life.

Much better results were obtained using the classical Probabilistic Neural Network (based on Algorithm 1).The accuracy here reaches 88%in the application mode.It is another argument in favor of using this computational intelligence tool in practice.If we compare these results with the results obtained in [25], it should be noted that the accuracy in our case is significantly higher (by about 8%).It is due to careful selection in this paper of the smooth factor, which dramatically affects the results of PNN.Immensely better results on almost all performance indicators were obtained for PNN,implemented according to the algorithmNo_2 described in this paper.In particular,compared to the basic version[23],we got 2%greater classification accuracy.

The best results were obtained for the designed hybrid PNN-SVM approach.There are two points.First,the hybrid PNN-SVM approach showed significantly higher accuracy(up to 25%)in comparison with the primary SVM classifier with both kernels.It is explained by using an additional procedure to expand the dimensional of the input attributes by the output signals of the PNN’s summation layer,which form a complete system of events.In addition,the developed system also showed higher accuracy than the prior and modified versions of PNN.It should be noted here the further use of the classifier based on SVM withrbfkernel.

As a result,the designed hybrid PNN-SVM approach provided the highest accuracy of solving Tibased alloy’s properties evaluation task in additive manufacturing of biomedical implants.Its value,reaching 91%,allows the use of the results of this system to predict the properties of the sintered alloy in the manufacture of medical implants by 3D printing

6 Conclusions

This paper describes the hybrid PNN-SVM approach designed by the authors to solve the problem of estimating microstructure defects and properties of titanium powder alloys before sintering through additive technologies.This task is important for the additive manufacturing of biomedical implants.The use of such systems will significantly reduce material, time, and human resources for the manufacture of biomedical implants.

The hybrid PNN-SVM approach is based on new approaches to expanding a given set of tabular data input space.It is implemented by adding probabilities of belonging to each specific class,forming a complete system of events to the initial dataset.Such probabilities are the output signals of the summation layer of the Probabilistic Neural Network modified in this paper.Further operation of the system is based on the use of an SVM-based classifier withrbfkernel.

Experimental modeling on an actual data set showed that:

· The modified PNN demonstrates significantly higher accuracy compared to the base one;

· The developed PNN-SVM approach demonstrates a significant increase in the accuracy of work in comparison with the basic classifiers that form it(PNN and SVM);

· The obtained accuracy of the developed approach (91%) provides the possibility of its use to solve applied problems of implant production.

Among the disadvantages of the proposed hybrid PNN-SVM approach,the possible significant time delays in processing large data sets should be noted.It is due to the use of PNN in the proposed scheme,which in the case of large data sets can become cumbersome and slow,which will affect the operation of the entire hybrid approach.In this case,it is necessary to modify the developed system by additional application of clustering[32–34].Replacing the existing sampling vectors with coordinate vectors of the centers of data clusters will significantly increase performance without a significant loss of classification accuracy.In addition,further research should focus on studying other options to expand the space of the initial data,particularly with the additional application of Ito decomposition.This approach will provide the ability to model the relationships between the input attributes of the initial data set and the probabilities of belonging to a particular class(PNN outputs),which should also increase the accuracy of solving the stated task.

Acknowledgement:Authors would like to thanks Dr.Ihor Lemishka for the data collection and Liliia Ryvak for the excellent technical support.

Funding Statement:The National Research Foundation of Ukraine funds this study from the state budget of Ukraine within the project “Decision support system for modeling the spread of viral infections”(103/01/0025).

Conflicts of Interest:The authors declare that they have no conflicts of interest to report regarding the present study.