Design and Study of Virtual Interventional Surgical System with Force Feedback

ZHANG Chao，BAI Dongming，CHEN Bai

College of Mechanical and Electrical Engineering，Nanjing University of Aeronautics and Astronautics，Nanjing 210016，P.R.China

Abstract: A virtual interventional surgical system with force feedback is designed to provide practice before complicated interventional operation and assistance during operation. The collision detection，vessel deformation calculating and virtual force computing of the virtual system are implemented by using skeleton-spring model as the physical modeling foundation，which is based on the mass-spring model and easy to construct with high computing efficiency. In order to increase the real-time performance，the central plane of the vessel model is extracted and then simplified to complete the skeleton filling. The initiative bending kinematics of the virtual catheter is analyzed so as to provide the virtual system with higher fidelity. The experimental results show that the virtual system can well simulate the vessel deformation and force feedback within an interventional surgery，which gives the virtual system better immersion.

Key words: virtual reality;interventional surgery;skeleton-filling;force feedback

0 Introduction

In interventional surgeries， surgical instruments are led to lesion with the guidance of medical imaging devices for further diagnosis and treatment.Interventional surgery has several advantages such as good curative effects，little trauma，quick recovery and low cost[1].

However，due to the extremely complicated surgical procedures and high risks，doctors can master interventional surgical skills only after long-time training and practice to prevent severe consequences[2].

Virtual surgery is designed as an interventional surgical simulation technique aiming to provide operators with vivid surgical environment compared with low-fidelity and poor training effect of traditional surgical operations[3]. Virtual surgery is a typical application of virtual reality（VR）technology in medicine. In virtual surgeries，doctors can acquire complete cognition of virtual models and receive realistic，precise and reliable training practice with force feedback devices[4].

Virtual surgery technique has been studied and some outstanding achievements have been made.Brown et al. built a virtual surgical 3D display system of 1 mm vessel and nerve suture based on graphic workstations[5]；Forschungszentrum Karlsruh，a Germany institute developed a virtual surgical system for gallbladder operations[6]. Tang et al.developed a virtual laparoscopic surgery system for training new surgeons[7].PHILIPS developed MEDICAL，a virtual surgical system with force feedback[8]. In Japan，Suzuki et al. proposed a virtual surgical system for live tissue operations based on the sphere-filled model[9]. Liu et al. proposed a method for obtaining the model of the surgical instruments from point cloud data to keep the geometric characteristics of the original model and improve the real-time property for the simulation[10]. Domestic research on virtual surgery is still in the process of exploration by now. Virtual surgical systems with force feedback are mostly developed based on existing force feedback devices.

In this article，a virtual surgical simulation system with force feedback is designed to assist in presurgical training practice and real-time surgeries，which can provide the virtual sugery system with higher computing efficiency and higher fidelity.

1 Design of Virtual Surgical System

Virtual reality technology has three basic features：immersion，interaction and imagination[10].As a typical application of VR in medicine，design of virtual surgeries should meet the demands of environment reality，real-time performance and high precision.

A complete virtual surgical system covers multiple disciplines：3D reconstruction of tissues and organs，human interaction，result display and the calculation modules[10]，as shown in Fig.1.

The reconstruction of tissues and organs includes the geometric model construction and physical model constructions. The physical construction is to reproduce the biomechanical characters of soft tissues. Human interaction includes the system’s receiving human behavior and making corresponding reaction ，human’s perception of virtual environment changes and the sensory stimuli. Result display is aimed to show calculation results in the form of a series of images on display devices.

Fig.1 Framework of virtual surgical system

2 Key Technology of Virtual Surgical System

2.1 Physical modeling of flexible body

In a virtual surgery，visual immersion and the fidelity provided by the virtual surgical system are affected by the physical modeling of flexible body.The geometric model，the physical model of the virtual object and relevant calculation method can be built through the modeling process，which provides the basis for the following surgical operation，collision detection and tissue deformation. Several constraints are then added to the basic geometric model in the physical modeling process to make the model conformed to the object’s real physical properties，including the mass，weight，inertia，surface roughness and the deformation characteristic[11].

Viscoelasticity is the most typical physical characteristic of soft tissue organ，which describes the soft tissue viscoelastic deformation under external forces. Due to the continuity of soft tissues’mass distribution and force distribution，the object is often discretized in modeling so that the complex problem can be discretized into several small parts to be solved separately. The physical model of soft tissues is the reverse combination of these small parts[12].

The mass-spring model ，the finite element method and the meshless method are the three most representative methods in the physical modeling of soft tissues. The mass-spring model is easier to implement，less complex in computing and more adaptive to soft tissues’topological structure changes，compared with the finite element method and the meshless method. Due to these features，the massspring model is widely used in real-time simulation of virtual surgeries. The mass-spring model is used in the physical modeling in this article.

The skeleton-spring model is constructed by a series of skeleton balls and the springs among them（See Fig.2）. A single skeleton ball’s movement and rotation is characterize by mass m，radius r，fixed coordinate system p. The damping spring between two skeleton balls（including linear spring and angle spring）is characterize by the length λ or torsion angle α，stiffness coefficients k and damping coefficient c.

Fig.2 Skeleton-spring model

Flexible body is in the form of triangular mesh after geometric modeling. Even a simple geometric structure contains a large number of nodes，so it takes huge computation to calculate each node’s displacement to simulate the deformation. However，by using the skeleton filling method，it will significantly simplify the calculation and give the flexible body’s surface model some volume character if skeleton balls’quantity and size are picked properly.

As is shown in Fig.3，there are two cases when filling the flexible body with skeleton balls ：（1）If a node is in the position of the shortest distance from the skeleton ball center，it can be connected to the skeleton ball center through a spring.Flexible deformation happens when the node moves with the skeleton ball’s movement and rotation.（2）If a node is in the position that the spring between the node and the skeleton ball is shortest，it can be connected to its projection on the spring through a spring. Flexible deformation happens when the node moves with the skeleton balls.

Fig.3 Skeleton filling

The Kinematics analysis of skeleton-spring model is as follows. The motion model is shown in Fig.4.

Fig.4 Kinematic model

While an external force is applied to a skeleton ball，it can be resolved into Faand Ft，where Fais in the direction of the spring，Ftis the torsion force applied to the skeleton ball.

where Fkis the elastic force of the linear spring，Fcthe damping force，kathe stiffness coefficient of the spring，c the damping coefficient，Δλ the deformation of the spring，τ the torque，Ftthe current length of the spring，ktthe coefficient of the angle spring and α the twisting angle of the angle spring.

According to Eqs.（1）—（3），the deformation of the spring，also，the movement of the skeleton ball can be expressed as

According to Eqs.（4）and（5），the twisting angle，also，the rotation angle of the skeleton ball can be expressed as

2.2 Collision detection

Collision detection is fundamental to the virtual reality system. Real time performance，stability and precision of collision detection affect the subsequent processing of the VR system directly.It has a significantly impact on the performance of the whole surgical simulation[13].

Hierarchical bounding volume algorithm is a classical algorithm of collision detection，which mainly includes the axis - aligned bounding boxes（AABB），oriented bounding boxes（OBB），sphere bounding box（SBB）. Sphere bounding box is the simplest in construction and collision detection.Combining the skeleton-spring model，a simplified algorithm based on sphere bounding box is adopted in this article.

As is shown in Fig.5，rvis the vessel skeleton ball radius，and rcis the catheter end skeleton ball radius. By calculating the distance d between the catheter skeleton ball center and the vessel skeleton ball center and comparing d with rv+rc，it can be estimated that whether the catheter end has collided with the vessel wall. If d ＜rv+rc，the collision happens.

Fig.5 Collision detection

In practice，each skeleton ball of the catheter end needs to be compared with the vessel skeleton ball.It takes huge calculation when the vessel is relatively long，which will result in poor real-timing performance. Considering that skeleton balls can be arrayed artificially when they are filled into the vessel and the limitations of catheter movement，the vessel can be subdivided along the axial line first. According to the position of catheter head，collision detection can be carried out in a smaller region，which reduces unnecessarily the calculation.

2.3 Virtual force feedback

Force/haptic feedback is different from the oneway feedback of visual and audio feedback.

With force feedback devices，operators can manipulate objects in virtual environment and receive the reactive force，which creates better immersive experience in virtual surgical systems.

At present，the finite element model and the mass spring model are two main computational models of force feedback[14]. As the skeleton-spring model is built on the mass spring model，the feedback force of mass spring model is calculated as follows.

Fig.6 Feedback force calculating model

As is shown in Fig.6，the contact force between the catheter end and the vessel wall is resolved into positive pressure FNperpendicular to the vessel wall and friction force Ffparallel with vessel wall. Let the collision depth between the catheter head and the vessel wall be D，collision velocity v，the friction coefficient between the catheter and the vessel μ，elastic coefficient k，damping coefficient c，therefore

where t is constant. The resultant force of the two，that is，the feedback force F on the catheter head，can be written as

The result shows that the feedback force depends on the collision depth，the elastic coefficient and the damping coefficient. The elastic coefficient and the damping coefficient depend on the material used.

3 Software Implementations of Virtual System

3.1 Pre-process of system

The off-line processing of the system includes geometric and physical modeling of the catheter and the vessel. Geometric modeling is proceeded in 3D modeling software，while physical model is constructed by filling the geometric model with skeleton balls.

As is shown in Fig.7，models of the catheter and the vessel of bifurcation is built in Pro/E and saved to 3ds format. The diameter of the vessel is 20 mm. The thickness of the vessel wall is 2 mm.The diameter of the catheter is 10 mm and the length is 100 mm.

Fig.7 Geometric model

Due to the complex geometric structure of the vessel，there will be a large number of data of vertex exported by 3D modeling software. It is necessary to simplify the vessel geometric model by using the triangular mesh processing software MESH. As is shown in Fig.8，the vessel model contains 2 690 vertexes，5 384 triangular facets before processing.And after processing，there are 866 vertexes and 1 742 triangular facets. The numbers shrink by a third，which indicates that it significantly reduced the subsequent work and improved the real-time performance.

Fig.8 Simplified vessel model

Unlike filling the catheter model with skeleton balls，it is rather difficult to build the vessel model.In this article，the central plane of the vessel is extracted. Skeleton balls are then created according to nodes on the central plane. Fig.9 shows the central plane and its vertex extracted from the vessel model. According to measured data，the force on the vessel in the situation of collision and puncture is 0.5～3.5 N[15]. In order to make the deformation and collision force conform to the real vessel，filling balls with the mass of 0.002 kg，the damping coefficient of 0.3 N/（m·s-1），elastic coefficient of 100.0 N/m，and the stiffness coefficient of 500.0 N/m are used in this article. The result of the catheter and vessel is shown in Fig.10.

Fig.9 Central plane and vertex of the vessel model

Fig.10 Skeleton filling of catheter and vessel

3.2 Program flow and software implementation

The software for the virtual system is developed on MFC in Microsoft Visual Studio 2008，with graph-base OpenGL and VR open source software libraries CHAI3D. Program flowchart is shown in Fig.11.

Fig.11 Program flowchart

In the virtual surgical system，the vessel deformation happens as a consequence of the collision.Unlike the vessel’s passive deformation，catheter’s deformation is active. Operators make the catheter bend by operating force feedback haptic devices.Therefore，it is necessary to analyze the catheter bending.

The catheter bending is shown in Fig.12.

Fig.12 Catheter bending

Considering the miniaturization and limited working space of the catheter，in this article，on the assumption that the catheter bends along an iso-curvature curve，the length of the central axis remains unchanged and skeleton balls are arrayed uniformly，let the bending angle is y，so

where β is the angle between the connecting lines of the（i-1）th skeleton ball and the ith skeleton ball to the first skeleton ball，n the number of skeleton balls，s the distance between the（i-1）th skeleton ball and the ith skeleton ball，R the radius of the catheter，and L the length of the catheter.

Let the distance between the ith skeleton ball and the first skeleton ball be Di，it can be inferred that

When the bend happens，the ith skeleton ball’s offset dxi，dyican be expressed as

The user interface of the virtual system is shown in Fig.13. It includes the display area，force feedback area and the manipulating area. The system also reserves software interface for master-slave control.

Fig.13 User interface of the virtual system

4 Experiments and Analysis

The force feedback device Novint Falcon is used in the experiment. Novint Falcon is a 3-DOF haptic master device，which has the maximum output force of 9 N and a refresh rate at 1 000 times per second. Virtual catheters can be simulated as being pushed，bended and twisted in vessels through Falcon’s handles. Dynamic deformation of vessel happens when catheter collides with the vessel wall，which produces the force applied to the Falcon handles.

To prevent the vessel from moving in experiment，the vessel is fixed at one end，which constrains skeleton ball’s DOF of the vessel end.Fig.14 shows the vessels in different situations：（a）Before the collision；（b）the collision depth of 0.5 mm；（c）the collision depth of 1.0 mm.

Fig.14 Vessel deformation

While the catheter is operated，data of forces on Falcon are collected and exported as graphs in MATLAB as is shown in Fig.15. Here，s is the bending direction of the catheter and y is the pushing direction. As can be seen，the feedback forces are in agreement with Ref.[15]and are mainly in x-axis and y-axis directions. In addition，it can be seen that the force changes around the time of 25 s，which means the vessel is punctured at that time.

Fig.15 Feedback force export

5 Conclusions

A virtual interventional surgical system with force feedback is developed using the skeleton -spring model as the physical modeling foundation. It has achieved the VR functions with simpler models and less calculation. The experiment shows that the virtual system can well simulate the vessel deformation and force feedback within an interventional surgery，which gives the virtual system better immersion. The system can work for any pre-surgical training and surgical assistance to increase the success rate.

AcknowledgementThis work was supported by National High Technology Development Program of China(No.51575256).

AuthorsMr. ZHANG Chao received his M.S. degree from Nanjing University of Aeronautics and Astronautics（NUAA）in 2017. His research interests include continuum robots and surgical robots.

Mr. BAI Dongming received his M.S. degree from Shanghai University in 2015. He is currently a Ph.D. candidate at College of Mechanical and Electrical Engineering，NUAA. His research interests include continuum robots and surgical robots.

Prof. CHEN Bai received his M.S. and Ph.D. degrees from Zhejiang University in 2000 and 2005，respectively. He is currently a professor at College of Mechanical and Electrical Engineering，NUAA. His research interests include design and control of surgical robots，cable-driven robots and spermlike swimming micro robots.

Author contributionsProf. CHEN Bai contributed to the background of the study，designed the study and prepared the manuscript. Mr. ZHANG Chao and Mr. BAI Dongming contributed to the discussion and simulation analysis as well as all the drafts. Prof. CHEN Bai reviewed and edited the manuscript.All authors read and approved the paper.

Competing interestsThe authors declare no competing interests.