Investigation on mobile robot navigation based on Kinect sensor

ZOU Yong-wei, WU Bin

(National Key Laboratory of Precision Testing Techniques and Instrument， Tianjin University， Tianjin 300072, China)

0 Introduction

The intelligent mobile robot can move purposefully and complete the corresponding tasks in complex environments including in water, on the ground, in the air and even in outer space without human intervention and any provision of the environments. In order to complete tasks in such complex environment independently, autonomous mobile robot navigation must be implemented[1]. Navigation in mobile robotics involves three key questions: where am I, where am I going and how do I get there. The corresponding terms in robotics are self-localization, map-building, map-interpretation and path planning[2]. One of the most important problems in this field is simultaneous localization and mapping (SLAM)[3], for knowing where a robot autonomously explores and mapping surrounding environment with its sensors while localizing itself.

The use of SLAM is constrained by currently available sensors. The used devices are mostly sonar sensors[4-5], cameras[6-8]and laser scanners[9]. However, the methods are often device-specific due to the different sensing modalities and capabilities of each device. Sonar sensors have low resolution, small detection range and are vulnerable to the interference of environment. Suitable laser sensors cost a lot and usually only produce a 2D view of the environment. Cameras are much cheaper, but have a limited resolution and require complex algorithms to determine depth values from the camera images.

Fig.1 shows the structure of the Kinect sensor. The Kinect sensor is a depth camera by Microsoft and Prime Sense for the Xbox360 video game console. Besides being able to provide a RGB-video stream like conventional camera, it is capable of recording distance of objects placed in front of it, which is achieved through the addition of the IR projector and receiver. With a price of less than US$ 150, the Kinect sensor is very affordable. Although Kinect sensor has the lowest price, it does not mean that the functionality is poor. The Kinect sensor is still able to provide 3D scan of the surrounding, which is similar to the lidar sensor. Besides, the sensor has a view field of 57° horizontally and 43° vertically. The optimal operating range of the sensor is between 0.8-3.5 m, although several researchers claim that it can be extended up to 0.4-6 m. The Kinect’s depth sensor is able to provide depth resolution of 1 cm and spatialx/yresolution of 3 mm at 2 m distance from the sensor. The maximal stable transfer rate of the frame is up to 30 Hz, depending on the driver or software used[10].

Fig.1 Structure of Kinect sensor

The main objective for this paper is to build a cost effective solution that can achieve navigation within an unknown environment using low cost Kinect sensor while maintaining the collected information and the built map from a remotely connected workstation. Also, the Turtlebot’s mapping ability in the different indoor environment with complex structure is tested.

This paper is organized as follows, section 1 presents the design and implantation details of Turtlebot system. Section 2 explains one of the slam question solution-Gmapping. The detailed Gmapping algorithm is presented. Section 3 provides details on the experimental results. Section 4 concludes this paper’s work.

1 Design and implementation

Fig.2 illustrates the system architecture, which consists of a Turtlebot robot system and a workstation system.

Fig.2 System architecture

1.1 Robot operating system

Robot operating system (ROS)[11]is a free and open source framework that has developed out of a novel joint effort midst of industry and the scholarly field. ROS is an open source programming stage intended to help another generation of individual robots. Individual robots move around in a human environment and interact with humans during individuals’ utilization. These robots that are frequently turned off and on again are called as administration robots or multipurpose controllers.

Fig.3 shows that a ROS framework is as an algorithm diagram comprising of a set of nodes corresponding with each other over edges. ROS contains instruments for assessing the diagram and checking what is the topic said by node or theme.

Fig.3 Illustration of hierarchical structure of ROS

Fig.4 shows the process of communication between nodes. One of the essential apparatuses of ROS, rostopic, permits a command line client to see what is constantly said in regards to a subject, how often times messages are, how the nodes distribute and so forth.

Fig.4 Process of communication between nodes

1.2 Turtlebot Ⅱ

Turtlebot Ⅱ[12], shown in Fig.5, is built on the success of the original Turtlebot. It is an open robotics platform that consists of a motorized wheeled base that is the Yujin Kuboki hexa-base, Microsoft Kinect sensor set on the top of the base that provides depth information, video input and an audiometer.

Fig.5 Turtlebot Ⅱ

Some of the main features for this robot other than its low cost, is that it includes wheel encoders, integrated gyroscope, big batteries, bump sensors, cliff sensors and wheel drop sensors. The speed of the robot can reach up to 50 cm/s.

The robot includes a lithium battery providing the electrical power for the robot itself and the connected Kinect sensor.

1.3 Kinect’s depth data to 2D scan

The use of 1D or 2D sensors, such as sonar and 2D laser scanners, for navigation and mapping is believed to not be efficient due to the possibility of missing or misinterpreting an obstacle’s location[13].

Fig.6 shows a scenario representing these problems. The sensor fixed at a certain height on the robot is unable to see the obstacle that is located below the visible range. Another possibility is that the sensor may mistakenly interpret an object’s location if it is non-uniform in shape (e.g.,Lshape).

Fig.6 Robot equipped with 1D or 2D sensor mistakenly determine object’s location

A 3D vision sensor is believed to be able to solve this limitation since the object’s size and shape can be obtained. The Kinect is chosen in this project because of it has 3D depth sensor, relatively fast sampling rate and is available at a low price. Firstly, the 11-bit raw data (an array of 640×480 elements) is retrieved from the Kinect and converted into real depth. Then, theXandYcoordinates that correspond to each depth pixel (Zcoordinate) are calculated by

(1)

(2)

whereiandjare the pixel’s row and column number in theZ-array respectively;ωandhare the width and the height of theZ-array;Mis the multiplier constant of NUI camera depth image to skeleton.

In order to remove the floor that is visible in the 3D data, the pixels located below the height of Kinect are replaced with an infinity value indicating that the data is invalid. The process is repeated for the pixels above the height of the robot since they are not considered as obstacles. Thus, the resultingX,YandZarray only contain the 3D obstacles with which the robot can collide. To convert the data into 2D obstacles (i.e., 2D scan), the minimum element in each column of theZ-array is selected to represent the entire column such that

(3)

wherejis the respective column number.

1.4 Client PC

The client PC is installed with Ubuntu 14.04 and connected to the robot and the Kinect sensor, the Kinect is mounted on top of the Turtlebot Ⅱ. This PC collects the whole sensor’s information such as:

1) Depth information received from the Microsoft Kinect sensor;

2) Images and video stream received from the Kinect;

3) Odometry information received from the Turtlebot Ⅱ.

ROS is installed on this PC, here we describe the application/handlers running on the client PC itself:

1) Turtlebot handler (hardware driver): this is the main driver to connect the components of the robot through a USB cable.

2) OpenNI handler: the driver can make the PC obtain depth information, video and audio streams which is collected by the Microsoft Kinect.

3) Depth_to_laserscan application: this application converts the Kinect’s depth information into a horizontal “line of site” laser scan data, which is used later in the main application to detect and build the grid map.

4) Map server: this application is used to collect the built grid map from the main application and send a copy of the map to the workstation for viewing/saving.

5) Gmapping application: this is the main ROS application that processes all of the sensor’s information (laser scan, odometry) to build the grid map on the map server.

1.5 Workstation PC

This PC is installed with Ubuntu 14.04 and ROS indigo, which is a remote machine that controls the client PC, views the maps, saves the grid maps and views the video stream. All of this are done through the below components:

1) ROSCORE application: this is the main ROS application which should run on the ROS master node. All other ROS commands and nodes will communicate to the ROS master node to get a list of all the available nodes in the ROS ecosystem, so it acts as a database listing all the nodes and approaches to reach them.

2) Tele-operating application: this is an application that connects to the ROS ecosystem and sends control commands to the robot to coordinate its movement. This can be done through the keyboard or a joystick connected to the workstation PC.

3) Visualization tool for ROS (RVIZ): this is a 3D visualizer for displaying sensor data and stating information from ROS. Using RVIZ, the current configuration can be visualized on a virtual model of the robot. You can also display live representations of sensor values coming over ROS topics including camera data, infrared distance measurements, sonar data and so on, as shown in Fig.7.

Fig.7 RVIZ visualizer

4) Map_saver: this is an application that saves the current built grid map to a local file on the workstation.

1.6 Network

In order to transmit the data that client PC achieved to the workstation PC, the network between each other need configure. Fig.8 shows the detail progress of the configuration. Clock synchronization is important for ROS. Chrony has been found to be the best ntp client over lossy wireless. In case of robot behaves strange when messages are sent from PC application (like RVIZ, rqt, or ROS node running in PC), clock synchronization is necessary.

Fig.8 Progress of configuration

2 Simultaneous localization and mapping

Under normal circumstances, robot system learning needs to solve three tasks: mapping, localization and path planning. Fig.9 describes mapping, localization, path planning and comprehensive discovery in overlapping region.

Fig.9 Combinatorial problem of mapping, localization and path planning

In order to solve a complex computational problem about building a map under an unknown environment and at the same time keeping track of the robot’s location within the same built map, there are several algorithms known for solving it. Gmapping could be one of the popular approximate solution methods.

The Gmapping algorithm is developed based on the Rao-Blackwellized Particle Filter (RBPF) introduced by Murphy and Doucet[14-15]. The RBPF is proposed to solve grid-based SLAM problems and it requires odometry information and the sensor’s observations. The main idea of the RBPF is to estimate the trajectory of the robot,x1∶t=x1,…,xtand the map,mgives the observations,z1∶t=z1,…,ztand the odometry data,u1∶t-1=u1,…,ut-1. This joint posterior is denoted asp(x1∶t,m|z1∶t,u1∶t-1) and can be factorized into Eq.(4) through Rao-Blackwellization technique

p(x1∶t,m|z1∶t,u1∶t-1)=p(m|x1∶t,z1∶t)·

p(x1∶t|z1∶t,u1∶t-1).

(4)

The factorization simplifies the computations such that it allows the process to be carried out in two steps. First, the trajectory of the robot can be estimated by the odometry data and the observations. Then, the map (p(m|z1∶t,u1∶t-1)) can be computed sincex1∶tandz1∶tare known.

The RBPF uses a particle filter to estimate the posteriorp(x1∶t|z1∶t,u1∶t-1), where each particle represents a potential trajectory. An individual map is also computed for each of the particles, since the map is highly dependent on the robot’s trajectory. Finally, the particle with the highest probability will be chosen and the associated map will be output by the algorithm.

The Gmapping method also features an adaptive technique that selectively carries out resampling operations for the particle. A careful resampling step is crucial to avoid the filter from removing good samples (i.e., particles that contain trajectory and map data). The approach takes into account the measure of the dispersion of the importance weights,Neff, which indicates how well the particle set approximate the posterior trajectory. The formulation can be denoted as

(5)

3 Experiment

The regular environment is the National Key Laboratory of Precision Testing Techniques and Instrument. Throughout the tests, the Turtlebot robot was moving at a rate of 0.2 m/s for both the straight and angular movement. All of the maps have been generated by using logged data after optimizing the Gmapping parameters.

3.1 Experiment 1

Task：SLAM map building with Turtlebot Ⅱ in large indoor area with a lot of obstacles.

Localization: National Key Laboratory of Precision Testing Techniques and Instrument.

Duration: about 1.5 h.

Experiment 1 shows a map that was built in a Key Laboratory which contains a lot of experiment instruments. A lot of attempts were taken to acquire the best map for Key Laboratory. Some of the huge instrument were distributed around the Key Laboratory so that the Turtlebot robot can not go to the edge of the laboratory. The main purpose of this experiment is to test the mapping ability of the mobile robot in an unknown environment, including a lot of barrier which could affect the Turtlebot’s movements. The real environment of Key Laboratory is shown in the Fig.10.

Fig.10 Structure of Key Laboratory

The final built map, Fig.11, can respect the key laboratory indoor encironment. The information shown in the built map is area in the labarotory that the turtlebot can reach. Howerver, the edge of the labaratory failed to express in the map, which indicates that no sensor data was available for these regions. This is due to the obastacle that mobile robot can not aviod .

Fig.11 Map of key laboratory

3.2 Experiment 2

Task：SLAM map building with Turtlebot Ⅱ in normal indoor environment.

Localization: normal apartment.

Duration: about 1 h.

Experiment 2 shows a map built in a normal apartment. In contrast to first experiment, the structure of the indoor environment is an open field so that the Turtlebot could go to the edge and corner of the indoor area. The experiment can test the Turtlebot’s mapping ability in a static and open space. Fig.12 shows the real picture of indoor environment. The structure of indoor environment has been demonstrated in Fig.13. The edge of the indoor environment can clearly express in the built map. This is due to Kinect sensor can achieve available depth data from these regions.

Fig.12 Real picture of indoor environment

Fig.13 Map of indoor environment

4 Conclusion

In conclusion, unknown environment map building and autonomous navigation using the affordable Kinect sensor are achieved. The system is able to pass the depth of the obstacle from the environment to the laptop running ROS, drive the robot using tele-operation via keyboard, build a 2D map of the environment and navigate autonomously. This autonomous system simplifies the robot navigation by providing a visual representation of the surrounding instead of the typical numerical values. The findings here also show that different indoor environment could affect the Turtlebot’s map quality. In general, it can be said that the Kinect sensor is suitable for unknown environment map building and autonomous robot navigation.

[1] Gao Y, Zou D, Hu D W, et al. Path planning for mobile robot in unknown environment and exploration algorithm. Electronic Design Engineering, 2014, 22(3): 1-3.

[2] Siegwart R, Nourbakhsh I R. Introduction to autonomous mobile robots. Massachusetts: MIT Press, 2004.

[3] Smith R, Self M, Cheeseman P. Estimating uncertain spatial relationships in robotics. In: Proceedings of IEEE International Conference on Robotics and Automation Proceedings, 2003: 435-461.

[4] Choi J, Ahn S, Wan K C. Robust sonar feature detection for the SLAM of mobile robot. In: Proceedings of IEEE International Conference on Intelligent Robots and Systems, IEEE, 2005: 3415-3420.

[5] Mallios A, Ridao P, Ribas D, et al. EKF-SLAM for AUV navigation under probabilistic sonar scan-matching. In: Proceedings of the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2010), Taipei, 2010: 4404-4411.

[6] Mei C, Sibley G, Cummins M, et al. RSLAM: a system for large-scale mapping in constant-time using stereo. International Journal of Computer Vision, 2011, 94(2): 198-214.

[7] Marks T K, Howard A, Bajracharya M, et al. Gamma-SLAM: using stereo vision and variance grid maps for SLAM in unstructured environments. In: Proceedings of IEEE International Conference on Robotics and Automation, IEEE, 2008: 3717-3724.

[8] Mcdonald J, Kaess M, Cadena C, et al. Real-time 6-DOF multi-session visual SLAM over large-scale environments. Robotics & Autonomous Systems, 2013, 61(10): 1144-1158.

[9] Grisetti G, Stachniss C, Burgard W. Improved techniques for grid mapping with Rao-Blackwellized particle filters. IEEE Transactions on Robotics, 2007, 23(1): 34-46.

[10] Biswal S. Tech News. 2011-02-12[2017-12-23]. http:∥www.techibuzz.com/xbox-360-kinect-review/.

[11] ROS.org. ROS Wiki. 2012-09-21[2017-12-30]. http:∥www.ros.org/wiki.

[12] Xia H J. Research of robot SLAM technology based on ROS. Shenyang: Northeasten University, 2013.

[13] Kamarudin K, Mamduh S M, Shakaff A Y M, et al. Method to convert Kinect’s 3D depth data to a 2D map for indoor SLAM. In: Proceedings of the 9th IEEE Colloquium on Signal Processing and Its Applications, Kuala Lumpur, Malaysia, 2013.

[14] Murphy K P. Bayesian map learning in dynamic environments. In: Proceedings of the Neural Information Processing Systems, Denver, CO, 1999: 1015-1021.

[15] Doucet A, Freitas N D, Murphy K P, et al. Rao-blackwellised particle filtering for dynamic bayesian networks. In: Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, Stanford, CA, 2000: 176-183.