An inexpensive underwater mine countermeasures simulator with real-time 3D after action review

Robert STONE*,Timothy SNELL,Neil COOKE

aHuman Interface Technologies Team,University of Birmingham,UK

bUniversity of Birmingham,UK

Available online 11 July 2016

An inexpensive underwater mine countermeasures simulator with real-time 3D after action review

Robert STONEa,*,Timothy SNELLb,1,Neil COOKEa

aHuman Interface Technologies Team,University of Birmingham,UK

bUniversity of Birmingham,UK

Available online 11 July 2016

This paper presents the results of a concept capability demonstration pilot study,the aim of which was to investigate how inexpensive gaming software and hardware technologies could be exploited in the development and evaluation of a simulator prototype for training Royal Navy mine clearance divers,specifically focusing on the detection and accurate reporting of the location and condition of underwater ordnance.The simulator was constructed using the Blender open source 3D modelling toolkit and game engine,and featured not only an interactive 3D editor for underwater scenario generation by instructors,but also a real-time,3DAfterAction Review(AAR)system for formative assessment and feedback. The simulated scenarios and AAR architecture were based on early human factors observations and briefings conducted at the UK’s Defence Diving School(DDS),an organisation that provides basic military diving training for all Royal Navy and Army(Royal Engineers)divers.An experimental pilot study was undertaken to determine whether or not basic navigational and mine detection components of diver performance could be improved as a result of exposing participants to the AAR system,delivered between simulated diving scenarios.The results suggest that the provision ofAAR was accompanied by significant performance improvements in the positive identification of simulated underwater ordnance (in contrast to non-ordnance objects)and on participants’description of their location,their immediate in-water or seabed context and their structural condition.Only marginal improvements were found with participants’navigational performance in terms of their deviation accuracies from a pre-programmed expert search path.Overall,this project contributes to the growing corpus of evidence supporting the development of simulators that demonstrate the value of exploiting open source gaming software and the significance of adopting established games design techniques in delivering highly engaging scenarios to defence training communities.

©2016 The Authors.Production and hosting by Elsevier B.V.on behalf of China Ordnance Society.This is an open access article under the CC BY-NC-ND license(http://creativecommons.org/licenses/by-nc-nd/4.0/).

Simulation;Virtual reality;Diver training;Ordnance disposal;Mine clearance

1.Introduction

As NATO’s armed forces continue to play a key role in international conflicts,their sophisticated air,land and maritime assets are being met regularly by an enemy that is becoming more and more innovative in its ability to disrupt operations through the use of concealed explosive ordnance.The threat posed by land-based Improvised Explosive Devices(IEDs)is well known and documented(e.g.in Reference[1]).Considerable financial resources are being expended to develop sophis-ticated counter-IED(C-IED)technologies that enable ground forces to stay ahead of what is a rapidly changing arena.Less well documented,at least from a public awareness perspective, are the threats posed by subsea mines or underwater IEDs (UWIEDs),despite the fact that new technologies in this field are evolving at a consistently and alarmingly rapid rate.

One of the key issues with IEDs,be they deployed on land or at sea,is that they are“fairly easy to build but difficult to detect and clear”[2].However,the damage that can potentially be caused by a relatively inexpensive UWIED to multi-million dollar surface vessels and submarines is all too obvious.As long ago as 1988,the USS frigate,the Samuel B.Roberts, collided with an Iranian-laid modified Russian mine and,whilst the vessel was not sunk,the damage caused by this$1500 munition amounted to nearly$90 million[3].UWIEDs now pose significant threats to naval forces and commercial vesselsacross the globe–a single device deployed from the smallest of vessels is capable of disrupting entire shipping lanes[4].

http://dx.doi.org/10.1016/j.dt.2016.06.001

2214-9147/©2016 The Authors.Production and hosting by Elsevier B.V.on behalf of China Ordnance Society.This is an open access article under the CC BY-NC-ND license(http://creativecommons.org/licenses/by-nc-nd/4.0/).

It has become apparent that the vast array of naval mines that were previously only available to the countries in which they were developed,such as China,Russia and the US,are now being obtained by increasingly unstable countries,many of which are located in strategically critical areas.A prime example of this is Iran,which,in 2010,was believed to possess the fourth largest known stockpile of remotely-controlled,drifting or moored contact naval mines in the world,and therefore posed a significant threat to force disruption in the Gulf ofAden and Gulf of Oman.This is still the case today,with more advanced mine technology,including pressure,acoustic and rocket-propelled devices,being sourced from the likes of Russia and China[2,5].Yet,only 4.7 percent of the US Navy’s 275 warships are dedicated to mine warfare[6].

Indicative of the seriousness of the threat is the extent to which significant research and development resources are being expended by many NATO member countries,a good number of which have,for many years,deployed mine countermeasures vessels(MCMVs)to the Straits of Hormuz and elsewhere. These vessels are typically equipped with mine clearance diving teams and“mine neutralisation vehicles”,taking the form of remotely operated submersible vehicles(ROVs),such as Atlas Elektronik’s SeaFox.However,NATO navies are gradually moving away from dedicated MCMVs to more modular counter-UWIED packages,including autonomous unmanned underwater vehicles(AUVs or UUVs),such as the British Talisman and US KnifeFish systems(a more capable vehicle than the previously-deployed Woods Hole REMUS system,which successfully detected mines during Operation Iraqi Freedom in 2003).However,until AUVs reach a level of sustained reliability in the UWIED arena(e.g.in Reference[7]) and become mainstream detection and clearance technologies, these functions will,for the next 5 years at least,continue to be delivered by divers,ROVs,and even specially-trained dolphins and sea lions,as used in the US Navy Marine Mammal Program.

2.Project origins

The motivation for conducting the present underwater mine clearance training research,with an emphasis on subsea“situational awareness”(e.g.in Reference[8]),came about as a result of a number of“serious games”,or,more correctly, games-based simulation projects that were delivered as part the UK’s Human Factors Integration Defence Technology Centre programme between 2003 and 2012.In addition,an educational project based on the UK’s first artificial reef–the deliberately scuttled hull of the ex-Royal Navy(RN)frigate HMS Scylla–provided early confirmation that some of the current-generation gaming toolkits were capable of delivering credible and believable underwater scenarios.In brief(and with further details available in the form of case study summaries in Reference[9]), these projects were:

1)Virtual Scylla([9]–Case Study 23[10];):two interactive 3D demonstrators developed in collaboration with the National Marine Aquarium(NMA).The first demonstrator enabled end users to explore external and limited internal spaces of a three-dimensional model of the wreck of the Scylla as if controlling an ROV(using a typical gamepad control device).The second artificial life demonstrator(e.g.in Reference[11])enabled end users to interact with a simulated ecosystem as might be found on the Scylla Reef,observing behavioural changes over time as sea temperature was varied.

2)SubSafe([9]–Case Studies 8 and 9):SubSafe was a prototype interactive 3D spatial awareness training tool, designed to supplement legacy training media currently in use by Royal Navy submarine qualification(SMQ) instructors.Navigating decks and compartments in a“first-person”mouse-and-keyboard game style,SMQ trainees were afforded access to a detailed virtual British submarine,comprising over 30 compartments and 500 objects–including major safety-critical items(e.g.fire extinguishers,hose units,high-pressure air valves,and emergency breathing system masks).A statistical analysis of knowledge transfer data,collated over a year of experimental trials undertaken in collaboration with the RN’s Submarine School,revealed that use of the simulation during classroom training significantly improved the final“walkthrough”performance of trainees(onboard an actual submarine)when compared to those of a control group.

3)Submarine Rescue([9]–Case Study 10):This project set out to assess the potential for games-based technologies to support early situational awareness training in submarine rescue,for both the current UK Submarine Rescue System(based on the LR5 submersible)and the future NATO Submarine Rescue System,NSRS.The demonstration featured a virtual disabled Kilo Class submarine with which the trainee submersible pilot had to rendezvous and dock,using a combination of direct viewing through the simulated submersible’s main viewport and simulated closed-circuit TV views,based on virtual cameras mounted on the external hull of the virtual rescue system.Turbidity,underwater lighting and backscatter effects,together with viewport dome distortions were also simulated.

4)Subsea Visualisation Concept Display([9]–Case Study 13):During the course of the Virtual Scylla Project,considerable support in the form of bottom profiling and side-scan sonar data was provided by the Royal Navy’s hydrographic teams.These data supported the planning of activities during the short windows of opportunity available to conduct ROV surveys on the actual wreck. Whilst presenting the Virtual Scylla Project at an NMA public event,a naval representative expressed interest in developing the simulation effort further,to address the visualisation of seabed topography and artefacts,using bathymetric data collected by the Navy’s hydrographic fleet.The resulting multi-window display concept,comprising seabed topographical representations supplemented with chart textures and simulated ROV views inreal time,has also generated interest on the part of maritime heritage organisations for the mapping of seabed sites and artefacts and the planning of expeditions to those sites.

5)EODSim:([9]–Case Studies 15 to 17):EODSim(Explosive Ordnance Disposal Simulation)was a C-IED technology research project,designed to investigate the application of innovative simulation-based training solutions for IED search,threat assessment and disposal,in close collaboration with the UK explosive ordnance specialists and instructors.Developed over a period of two years,an interactive 3D classroom tool was developed based around a homeland(urban)scenario with virtual features of relevance to the training of threat awareness, intervention planning and the formulation of IED“render-safe procedures”.Following an initial concept demonstrator review with defence IED specialists,the deliveredprototypeEODSimsystemfeaturedan improved user interface to support pre-lesson configuration of a range of virtual threat scenarios based on a 3D representation of a typical British town(with shops, school,railway station,car park,etc.)and virtual EOD assets(support vehicles,remotely controlled vehicles, Army avatars,etc.).

All of these projects demonstrated that it was possible to develop credible,real-time and editable virtual reality(VR) scenarios of acceptable fidelity and to use said scenarios to deliver(a)convincing proof-of-concept demonstrators to potential stakeholders and(b)engaging education and training systems suitable for experimental evaluation in military and civilian classrooms.During demonstrations of some of the scenarios described above,the principal author and his team were approached by members of theTechnology BasedTraining Unit (TBTU),an important stakeholder in the field of synthetic training for the Royal Navy,based at HMS Collingwood in Fareham, UK.Two demonstrator projects resulted from the discussions with the TBTU.One project focused on developing a low-cost classroom VR simulator to help foster early decision-based training in navigation activities,otherwise known as“Rules of the Road”[12];the second related to the focus of the present paper,namely situational awareness training for divers conducting mine clearance or mine countermeasures(MCM)training at the UK’s Defence Diving School(DDS).

3.Defence diving school early briefings

The DDS is located at the UK shore base HMS Excellence on Horsea Island,originally off the northern shore of Portsmouth Harbour but today connected to the mainland as a result of reclamation.Since 1995,the school has provided diving training for numerous branches of the UK’s Armed Forces, including basic military diving training for all RN and Army (Royal Engineers)clearance and MCM divers(Fig.1).

Early dive training observations and briefings from DDS project stakeholders identified a range of concerns,many of which suggested a strong potential for technology-based classroom training to help illustrate some of the search and identificationproblemsassociatedwithdegradedvisionand situational or spatial awareness(an old,but still highly relevant article on which is provided in Reference[13]).When conducting mine identification and clearance activities on the seabed,in water columns and especially under ships’hulls,the MCM diver must be able to cope with these conditions and numerous other physical constraints to discern between harmless artefacts and any possible explosive ordnance.The diver’s task is further complicated by the need to follow specific underwater search patterns and there is,therefore,a crucial need for a strong sense of orientation in an environment where visual cues can be sparse.The outcome of such scenarios is of particular concern as the potential implications from false positives and,more importantly,false negatives can be severe.

An additional issue was also raised relating to what was labelled as“psychological fear”on the part of some trainees when they experienced under-hull diving for the first time.This was described as a combination of(a)a realisation on the part of the diver that,depending on vessel type,s/he is actually swimming under some 8000 to 22,000 tonnes of steel,(b) spatial disorientation brought about by exhaled gas bubbles moving no further than the immediate hull section(depending on what part of the vessel the diver is inspecting),and(c)a similar psychological effect to that experienced by some astronauts during the start of their very first extra-vehicular activity (EVA)when the“call of the abyss”–a fear of falling and spatialdisorientation–can occur as a result of the Earth appearing underneath below their body’s location(e.g.Reference[14];p 373).The DDS instructors claimed that,as a result of one or more of these experiences,some clearance divers have actually left the service after their first operational dive in the vicinity of an RN vessel.

Fig.1.Trainee mine clearance divers undergoing pre-dive briefing and final dive checks at the UK’s Defence Diving School.

Another,related concern that became apparent during briefings with DDS instructors and specialists was the lack of training provided covering specific tasks and specific vessel types. Vessel hull inspection was cited as an excellent example of a task that could not,at that time,be performed at Horsea Island, yet divers with minimum experience could quite easily be expected to conduct such inspections on and around a variety of RN assets,including completely new platforms,such as the Astute class of submarine,the Type 45 Destroyer or the new Queen Elizabeth Carrier.It was felt that vessel-and task-specific training using appropriate simulation technologies might help to overcome some of the psychological issues described above,especially when inspecting a large,unfamiliar hull for the first time.

Another,more concerning issue,however was the apparent absence of a consistent method for divers to report back on any underwater artefacts they may have discovered.DDS instructors described incidences where there had been a lack of consistency from divers reporting on the same underwater object and that the combination of poor visibility,crude methods of measurement and no clear or concise reporting methods or metrics could easily lead to an under-or overestimation of the dimensions,shape,condition and water column or seabed location of such objects.Related to these issues was the fact that no adequate means existed whereby instructors were able to review and discuss a trainee diver’s performance after each dive, thereby providing a strong formative assessment process.These issues became highly relevant in the design of what became known as the MCMSim(Mine Countermeasures Simulation) project,the virtual environments and scenarios for which are described briefly later,particularly with regard to features such as the fidelity of the underwater ordnance models,the subsea rendering effects,the methodology for capturing trainee reports from simulated dives and,importantly,a real-time,graphical After Action Review process.

4.After action review

Very broadly,the termAfterAction Review(AAR)refers to processes that support meaningful and interactive instructor–student reviews of training outcomes with the aim of improving subsequent individual or group performance through reflective learning,or to“correct training deficiencies”[15].In the case of recent simulation-based training applications,AARs may be generated by embedded software algorithms that log and subsequently present summaries of pre-defined behaviours exhibited during a specific training session.However,the concept of AAR is not new.A number of research reviews make reference to“after-combat interviews”used duringWorldWar II,or“performance critiques”developed in the 1970s(e.g.Reference [16]).Nevertheless,the power of the AAR process is now well established and,if designed appropriately,enables both trainer and trainee to understand and reflect upon what happened during a training session,how it happened and why it happened. AARisfindingsignificantapplicationwithinthose organisations whose personnel work in high-risk environments (e.g.the military,nuclear industry,fire-fighting,etc.),or where human error can produce unacceptable consequences,such as in the medical and surgical domain(e.g.Reference[17]).

Numerous attempts to develop generic and comprehensive AAR systems for simulation-based training systems have been evident since the early 2000s(e.g.[18]).Development attempts have employed a variety of technologies and techniques, ranging from discussion groups,closed-circuit TV and audio capture of participants’comments,low-cost and embedded video capture and game/simulation benchmarking tools(such asFRAPS:http://www.fraps.com/)tosophisticateddata capture systems,capable of logging a variety of end user activities,including movement paths and dwell times,reaction times and key decision points.However,a generic,cross-platform, cross-softwareAAR capability is still elusive.Some researchers even question the possibility of a generic tool being a realistic goal,given the speed with which games-based simulations and real-time engines are developing,although some initiatives, such as the Dismounted Infantry Virtual After Action Review System,or DIVAARS project,developed by theArmy Research Institute for the Behavioral and Social Sciences,show significant potential[19,20].Indeed,many of the functionalities demonstrated by DIVAARS provided early inspiration for the MCMSim AAR system described later,including the ability to view scenes during the AAR in 3D,mapping the trainee’s simulated motion paths,bookmarking event times and views, and presenting tabular and graphical data summaries.

An earlier example of AAR technology,the Tactical Simulation System(TACSIM)After Action Review User System (TAARUS)described in Reference[21],also influenced the design process for MCMSim.Despite not being associated with an interactive 3D or virtual environment-based training simulator,certain key aspects of TAARUS in its role as an AAR system supporting military training exercises are worthy of mention here.For example,the designers of the system paid particular attention to how theAAR data should be presented to the users,and included the overlay of key information on maps, highlighting the impact of a particular decision.This,coupled with an“instant query”facility,helped to provide a high-level picture of intelligence activities occurring within the exercise. The development of TAARUS also focused on encouraging widespread uptake and utilisation at minimal cost.To this end, public domain software was used for development and early steps were taken to ensure that the system could be utilised on common computing equipment(indeed,the AAR software was made freely available,even to non-military users).

5.The MCMSim system

Of course,the use of VR techniques to train civilian and military divers or ROV operators to undertake MCM or underwater incident interventions is by no means a new concept. Recent gaming set-ups,as with many contemporary VR applications,focus on exploiting the latest inVR wearable technolo-gies,such as the“swimming with sharks”demonstration shown at the 2014 Scuba Show,held in Long Beach,California(using 360-degree spherical images and video)and Vertigo Games’World of Diving[22],both of which are designed to work with the Oculus Rift DK1 and DK2 Head-Mounted Displays.There are also various training and research products available from organisations such as Imetrix[23],Forum Energy Technologies,Marine Simulation LLC,Schilling,Vortex Dynamics and Szenaris(used to train German Navy mine clearance divers), and from other developments spearheaded by international academies and institutes(e.g.References[24–26]).However, most of the commercially available simulation tools are quite expensive,require annual investments to maintain licences and some fail to provide end users with comprehensive trainee performance review techniques.Furthermore,very few seem to have been the focus of any form of experimental evaluation relating to enhanced situational awareness or performance enhancements in laboratory environments,let alone real-world diving settings.It should be stressed here that the system developed as part of the project described herein was not designed to be a competitor for commercially available simulation products.Rather,MCMSim was designed to demonstrate how a freely-available modelling and rendering software toolkit could be exploited in a short period of time to produce a reasonably comprehensive,freely-distributable VR training package,complete with a flexible scenario generation interface and a realtime interactive AAR facility for meaningful end user performance recording and review.

MCMSim was developed using Blender(v2.49),a popular and free open source 3D modelling and animation tool and game engine.Blender’s interface simultaneously supports concurrent 3D modelling and simulation/game development and, as such,suits the rapid development and evaluation of proofof-concept applications such as that described here.The Blender Game Engine utilises the Bullet real-time physics engine for rigid and soft body simulation and collision detection.The embedded Python(v7)language interpreter supports the definition of game logic using visual programming blocks and scripting.Python affords the use of bindings(modules)to add custom functionality;this is essential in MCMSim because of the requirement for AAR functions involving data storage and parsing,and specific control mechanisms related to the use of multiple input devices.

The MCMSim workstation took the form of a single Hewlett-Packard laptop(HP Pavilion dv6565cl,with Intel Core 2 Duo T7300 processor and an NVIDIA GeForce 8600M GS), an Xbox wireless game controller and a pair of 19-inch Samsung SyncMaster LCD displays(one for the participant,a second for the instructor/experimenter).The Xbox controller was configured for egocentric(first-person)view or camera control,such that the left thumbstick governed translational movements(forward,backward,left and right),whilst the right thumbstick controlled rotational(left,right,up and down) movements.The two triggers on the Xbox device governed ascent and descent through the virtual underwater scene and other buttons were allocated to discrete functions,such as light (torch)control and object designation/selection.Sennheiser HD 448 noise-cancelling headphones were used to convey underwater sounds to the participants in a brief pilot experiment using MCMSim,described later.

5.1.MCMSim rendering environment

Three categories of assets were developed for the virtual environment.The player assets included a basic diver avatar and a remotely operated vehicle(ROV),the 3D model of which was based on the commercially-available VideoRay system.Seascape assets included a variety of rocks and vegetation.Animation effects for vegetation simulated simple swaying caused by underwater currents.Man-made assets included wreck debris, explosive ordnance,a crashed military helicopter and a selection of“distraction objects”such as jetsam,oil barrels and wooden boxes.Multiple versions of man-made objects were necessary to support representations within the virtual environment which vary in terms of degree of silting,deterioration and position in the water column.The geometry for many assets was sourced from third-party online resources,such as Turbosquid.com,and modified as required.

In order to enhance the credibility of the simulated scenarios (described below),attention also had to be paid to how best to represent the realistic positions and orientations of underwater ordnance,depending on such factors as whether the weapon had been deployed from a surface vessel,submarine or airborne platform,or whether it is moored to some form of anchorage or baseplate.The major determinant ultimately affecting the resting“posture”of the mine is the design of the mine itself,as underwater ordnance is designed to come to rest in such a way so as to maximise the functionality of the onboard weapon(s).A self-propelled torpedo-shaped ordnance will typically come to a rest in a parallel fashion with the sea bed,resulting in the potential for high levels of silting[27].When the same ordnance is launched from an aerial platform such as a helicopter, there is a much greater variation in how it reaches the sea bed, with the potential for a much more random final resting posture. Space precludes a review of the different mines represented in the MCMSim project.However,in summary,and based on information freely available from the Internet and specific defence and international stabilisation forums[28],a number of mine types and geometries were selected for modelling in 3D. Some mines,including the Chinese submarine-deployed EM-52(also known as T-1 an unguided rocket-propelled bottom mine)and the Iranian SADAF-01(a moored,contact trigger-type device)had to be modelled from scratch using 3dsmax.However,where possible,use was made of existing underwaterweapons,suchasthoseavailablefrom Turbosquid.com(such as the Mark VI,the Manta,the Mark 48 Torpedo and a generic missile munition,shown in Fig.2).

The simulation of the seabed was based partly on video footage obtained from DDS training exercises in Horsea Island’s 1-km wide saltwater lake.Additional features and“challenges”were added as the project progressed and further information was forthcoming from UWIED specialists.To match the diver’s views,graphical shaders developed in the OpenGL Shading Language GLSL were used for distance fogging and underwater“caustics”(focused and defocusedlight patterns brought about by reflection and refraction and seabed undulations).

Fig.2.3D Underwater mine and weapon models sourced from Turbosquid.com(clockwise from top left:the Mk VI,the Mk 48 Torpedo,a generic missile and the Manta).

5.2.MCMSim experimental setup

The training flow adopted during early MCMSim evaluations and later experimental sessions with participants consisted of three steps.Firstly,participants were briefed about the scenario,and this was followed by a short scene-setting vignette during which they were required to undertake simulated abovewater preparations(such as taking position in a line of divers“treading water”next to the hull of the vessel concerned)prior to the dive itself.Once located in their allocated surface dive team position,the participant indicated his/her readiness to the instructor who would then initialise the virtual underwater scenario(described below).Underwater images were presented to the participant via one of the two available 19-inch LCD screens,with a restricted field of view effect generated using a simple simulated diving mask template as part of the display (Fig.3).

Once in the underwater scene,participants were required to navigate throughout the local environment and identify any objects that they felt posed a potential damage or destructive threat to the vessel.They were required to distinguish between the objects which were not threats and those that were,as well as having to deal with several other sources of visual noise in the simulated scene,such as viewpoint occlusion caused by the mask,underwater debris and vegetation.

The instructor/experimenter was provided with the second 19-inch screen which,together with a standard keyboard-andmouse interface,performed two functions.Firstly it allowed him/her to set up a dive scenario by selecting a range of 3D objects(natural and man-made)from a library and to locate each on the simulated seabed(Fig.4).The interface also allowed the instructor/experimenter to set turbidity and visibility conditions and,importantly(from an After Action Review perspective)to define an“ideal”search path through the scenario.Such a feature also supports the execution of repeat or remedial training sessions,as might become necessary to reinforce specific training procedures.For the purposes of this project and the experimentation described later,the“ideal”search path(from which deviations by participants could be measured)was based on a“Jackstay J”pattern.This pattern is adopted when individual or pairs of divers are deployed in conditions of limited visibility,when the search target object is small,or when a search pattern has to be conducted twice(on each side of the jackstay–the search line or tether),so as to avoid missing the target,which may be obscured by largerunderwater objects.Every five(simulation/video)frames,the MCMSim system logs participant performance by recording and time-stamping his/her navigation progress(as represented by the virtual diver’s position and orientation).These data were used to calculate the normalised root mean square deviation from the pre-set ideal search path.Other search performance parameters were also logged,including the dwell times associated with all underwater objects and objects identified(correctly or incorrectly)as ordnance.

Fig.3.Simplified simulated scuba mask limiting the simulation user’s field of view.

Fig.4.Instructor’s scenario set-up interface,showing the 3D artefact and debris database(left-hand screen)and seabed rock density setting(right-hand screen).

In each scenario,participants were given a number of challenges based on instructional material provided by the instructors at the Defence Diving School.Again,it should be emphasised here that the demonstrator was not designed with the ultimate aim of training divers in specific skills relating to diving or the use of specialist underwater equipment.Rather, the focus of the research was to address issues relating to the use of interactive technologies to deliver situational awareness training during the early stages of mine clearance diver courses. The participants’overall objective was to ensure that the area underneath and in close proximity to the two vessels was clear of any explosive ordnance,with the challenge being varied in difficulty dependent upon specific water conditions such as visibility and turbidity.Blender’s physics capabilities were exploited and,where necessary,enhanced,to support the generation of turbidity,bubble and leakage effects via particle settings,including size randomisation and the specification of Brownian forces.

5.3.MCMSim scenarios

Two scenarios representative of those that could be faced by MCM Divers in service were implemented.The first scenario was based on responses to an actual event that occurred during the sea trials for the UK’s Astute class of nuclear powered attack submarine(SSN).On 22 October 2010,HMS Astute ran aground off the Isle of Skye in Scotland.The official incident report concluded that the grounding came about as a result of“non-adherence to correct procedures...combined with significant lack of appreciation by the officer of the watch(OOW) of the proximity of danger”[29].At the time,there was a significant concern that sensitive areas of the vessel may have been damaged,with the possible risk of a radiation leakage.An underwater search for hull damage and SSN debris on the immediate sea bed was undertaken.A virtual scenario based on the location of HMS Astute at the Kyle of Lochalsh,close to the Skye Bridge was developed using a combination of websourced and custom-built 3D assets(Fig.5).

The second scenario was fictional in nature and was based on UK’s latest generation of aircraft carrier,the Queen Elizabeth (QE)Class(Fig.6).Once again the scenario began with an incident dive team in position alongside the hull of an early 3D model of a QE Class vessel.However,this time,the underwatertasks involved a hull inspection and a thorough search of the surrounding seabed for possible ordnance.

Fig.5.Virtual Astute Class Submarine in Blender,showing the incident dive team and support boat alongside.

Fig.6.Virtual Queen Elizabeth Class Carrier in Blender and incident dive team alongside.

A final scenario was developed to demonstrate the power of the Blender toolkit in rendering underwater scenes(caustics, point lighting,etc.)and to show effects such as ordnance leakage and dynamic turbidity and silting caused by underwater disturbances,such as structural collapse.In this case,the focus of the demonstration was the 3D representation of a VideoRay ROV,as shown in Fig.7.Basic umbilical handling effects for the ROV demonstration exploited Blender’s Bullet physics engine.

5.4.The MCMSim after action review system

The AAR mode,once executed,displays a summary screen containing high-level performance records,including total time in simulation,dwell times per object found,the number of correct and incorrect objects identified(“hits”),objects missed and“hit percentages”.Once the contents of this screen have been reviewed with the participant,the instructor/experimenter can commence a more detailed replay sequence.Moving forwards and backwards in time using the mouse scroll wheel,and with a subset of the performance indices displayed at the bottom of the screen(simulator time,“hit items”–objects identified–and correct/incorrect hits;see Fig.8),the experimenter is able to highlight important“mission”features to the participant(through the use of a mouse-controlled digital“highlighter pen”function,also shown in Fig.8),discussing their individual strategies and making suggestions for improvement in subsequent simulated dives.

To support as detailed an AAR session as possible,the instructor/experimenter also has the ability to select different camera views,visualise a trace of the participant’s search path in the scenario(shown in Fig.9)and to use a mouse-controlled digital“marker pen”to highlight issues with the search strategies,dwell behaviours and ordnance/non-ordnance decisions of the participants.Three of these camera views are centred on the virtual diver and provide front,rear and side perspectives.A fourth camera view(“Free Cam”)gives the experimenter freedom of movement in the simulated underwater view–movement that is governed by using the W-A-S-D key combination.

5.5.Dive report chart

Fig.7.Virtual VideoRay ROV and subsea debris scenarios in Blender.

Fig.8.AAR review screen.

To support the recording and reporting of participants’performances in the scenarios described earlier,a“Dive Report Chart”was developed,to be used in addition to the softwarebased recording system developed forAAR purposes.The contents of the chart were based on the observations undertaken and briefings provided at the outset of the project by the Defence Diving School.The rational for developing such a chart was that,in real-world diving scenarios,the dive team members are unable to take recording material with them to help report back on what they find during the underwater surveys.They have to report their findings by recalling their experiences from memory.Consequently,it was decided to develop a chart that could be used immediately after a simulated diving session,enabling participants to record the findings of their dive using a collection of representative symbols, images and text descriptors.The final version of the chart is shown in Fig.10.

1)Location of discovered artefact(in terms of angular sector and range from own ship)

Fig.9.Example of instructor/experimenter’s rear-perspective AAR review screen with search path.

2)General shape of artefact(in terms of major form and cross section)

3)Presence of specific or unique identifying features(such as fins,vanes,protrusions)

4)Approximate dimension of artefact(major and minor axes)

5)Presence or absence of tether

6)Presence or absence of text,numerals or other markings

7)Approximate depth of artefact(hull,water column, seabed)

8)Conditionofartefact(e.g.deterioration,damage, leakage)

随着高校的扩招及扩建,伴随着信息技术的飞速发展,校园网络应用业务越来越多,网络负荷越来越重,逐渐超出了原有网络的承受力,多数校园网络需要改造及扩容[1-4]．校园网的更新需要考虑多方面的因素:原有的网络体系结构是否合适,能否进一步扩充;从资金及技术2个方面考虑更新方案是否可行等．因此,如何对现有网络进行升级改造,改造后的网络能否满足实际应用需求,值得深入探讨．

9)Local seabed“quality”(foliage–dense to clear;seabed material–sand to large stones/rocks)

10)Degree of artefact silting or coverage(for the three finest seabed material classes–1/5 to 5/5)

11)Condition of artefact on,or embedded in seabed(e.g. deterioration,damage,leakage)

6.MCMSim AAR–pilot study experiment

A short experimental pilot study was designed to determine whether or not certain aspects of diver performance and situational awareness can be improved as a result of providing the simulation-based,instructor-ledAAR facility developed for the MCMSim system described herein.The main hypothesis adopted stated that,in comparison to participants who receive no form of AAR,those who undergo a structured post-task AAR of the form developed for the MCMSim would subsequently demonstrate significant improvements in simulated diving performance in terms of minimised deviations of search patterns from a pre-programmed“ideal”,identification of correct and incorrect objects and the reporting accuracy or detail of correctly identified objects.

Fig.10.“Dive report chart”designed for experimental trials and AAR debriefs.

6.1.Participants

Volunteer participants consisted of staff and Final Year students from the University of Birmingham’s School of Electrical Electronic and Systems Engineering.This“convenience”sample[30]was chosen for two key reasons.The first was the fact that the Pilot Study Experiment was the concluding element of the 6-month,Year 4 Master of Engineering(MEng) project at the University of Birmingham.The second and more important reason was the huge logistical challenge faced by the University team in trying to organise a formal experiment using trainee divers at the Defence Diving School.If the outcomes of the present pilot study were positive,then it might be possible to consider developing a more extensive trial with MoD diving personnel,along the lines of that undertaken with submariner trainees for the SubSafe project described earlier Reference[9] and in References[31,32].The sample,then,represented the most pragmatic way of carrying out an experimental evaluation within the timescales available,avoiding,amongst other things, the lengthy ethical approval process that MoD experimentation typically demands.Ethical approval for the experiment was obtained by following University of Birmingham procedures and every attempt was made to avoid experimenter bias.The participant cohort,then,consisted of 20 individuals(15 males and 5 females)between the ages of 21 and 51,with the majority of participants aged between 21 and 23.None of the participants had prior diving experience diving,or experience of identifying explosive ordnance,or simulation-based AAR.The group exhibited a mix of prior experiences with use of gamesbased technologies,ranging from complete novices to expert gamers.

6.2.Design

The pilot study adopted an independent samples,betweenparticipants design.The main independent variable was exposure to MCMSim with and without AAR between scenarios. The dependent variable was defined as the participant’s performance in the simulator on the basis of three performance factors:

1)Search PathAccuracy:every five frames during simulator run-time,the virtual location of the participant was recorded by the MCMSim system.These data were then used to calculate the normalised root mean squared deviation from an expert search path in the simulation. The expert search path was kept constant for each simulation and was produced by recording every five frames of an expert user’s location in the simulation.

2)Correct and Incorrect Items Identified:the participant was required to press a button on the simulator’s input controller(an Xbox gamepad)and verbally signify to the experimenter that an object had been identified.The numerical value of the correct and incorrect items was recorded in each simulation.

3)Reporting of Correct Items(Accuracy):here the user was required to complete the“Dive Report Chart”shown in Fig.10 after each scenario.Participants’reports were then scored against an ideal report of the objects.

6.3.Procedure

Participants were divided randomly and equally into the two experimental groups–those who were to perform two MCMSim scenarios with an interveningAAR session(“Sim+”) and those who were to have no such AAR session(“Sim-”). Both groups were required to undertake the simulated scenarios in the same order;the Astute Class submarine scenario followed by the QE Class carrier scenario,as described earlier.All participants undertook a brief familiarisation session with the MCMSim to become competent with the Xbox controller interface before performing the experiments.TheAAR delivered by the experimenter followed,as closely as possible,the formal procedures contained within the USArmy’s Leader’s GuideTo After Action Reviews[15],so as to ensure no variation in participant influence on the part of the experimenter.Finally,participants were also required to take part in a basic usability study,which was aimed at identifying any issues with the system in terms of the user interface and any aspects of the visual and audio features of the simulation.A customised 5-point Likert Scale questionnaire was developed which combined elements from the(at the time)recommended usability approach adopted by the UK’s Ministry of Defence[33]and the simulation-specific guidelines contained within[9].Changes in performance factors between the first and second scenarios for each group were analysed using a Student’s t-test,a test used frequently with small,independently-collected samples drawn from a normally distributed population when the population standard deviation is unknown.This analysis was one-tailed (based on the expectation that AAR would significantly improve performance between scenarios),with the corresponding p-value analysed,and significance identified at values below 0.25.

6.4.Results summary

6.4.1.Search path accuracy

The results of the analysis of the normalised root mean squared deviation from an expert search path in the simulation (Table 1)demonstrated that,in the Sim-(noAAR)condition,a decrease in mean accuracy occurs between scenarios with a relatively large standard error of the mean(this provides an estimate of the variability between samples),in contrast to the Sim+(withAAR)condition,where a small increase in the mean accuracy with a lower standard error was found.The t-test results for these data indicate that these findings are only marginally significant at the 2.5%level,returning a p-value of 0.03. This is,perhaps unsurprising,as there may be many other performance and personal interactive styles at play when tryingto navigate in six degrees of freedom through an underwater scene using a gamepad control device and a two-dimensional display where the field of view has been further constrained by the use of a scuba mask template(as shown in Fig.3).

Table 1 Mean and standard errors for changes in search path accuracy between scenarios for each group.

Table 2 Mean and standard errors for changes in correct and incorrect items identified between scenarios for each group.

6.4.2.Item identification

The results,as summarised in Table 2,demonstrated a large increase in the mean number of correct items identified in the Sim+(with AAR)condition in contrast to the Sim-(no AAR) condition,with similar standard error of the mean values in both groups.With regard to the number of incorrect items identified,it was clear to see that the Sim+condition resulted in a much lower mean,and observations made during the experiments indicated that,in all but one case for the Sim+group,a zero false positive rate was obtained.The t-test results for these data were highly significant,delivering a p-value of 0.000001 with respect to the data relating to the increase in correctly identified items,and 0.001 for the decrease in incorrect items identified.These results support the hypothesis that providing AAR after the first exposure to a low-cost training simulator such as MCMSim may well deliver a significant increase in object identification performance and a similarly significant decrease in incorrect identifications during simulation-based training.

6.4.3.Reporting accuracy

As can be seen for Table 3,there is a huge difference in percentage reporting scores between the Sim+and Simgroups.The t-test results confirm this,delivering a p-value of 0.017.These results support the hypothesis that providingAAR after the first exposure to a low-cost training simulator such as MCMSim may well deliver a significant increase in participants’reporting scores relating to details of objects detected and the nature of their immediate subsea contexts.

During the experiments,when asked if they felt they understood the task better afterAAR,all members of the Sim+group indicated that they did.Similarly,Sim+group members felt that,after AAR,they knew how to“perform better”and understood the good and bad aspects of their performance.Of course, these observations do not hold any statistical significance in this experiment.However,they are in line with other systems suchas the DIVAARS system[19]where such questions attracted similar responses.These responses were not evident in the Sim-group.

Table 3 Mean and standard errors for changes in reporting scores between scenarios for each group.

6.4.4.Usability results

The results of the Likert Scale Usability Questionnaire uncovered no significant problems reported by the participants relating to the MCMSim user interface and,in general,all of the users reported that they found it easy to use,easy to learn and presented no inconsistencies.Some participants remarked that they felt that certain visual features of the simulated task compromised their performance,as no visual confirmations were given once an identification had been made.Of course,in the real world of underwater ordnance identification,no visual confirmation is provided at the time of detection.The simulator’s auditory system also provided mixed results.Despite the fact that all users confirmed that the sound effects aided in their sense of“immersion”within the simulation,some felt that the sounds compromised their ability to perform the task.In particular,some participants expressed frustration at the constant and repetitive breathing sounds.As with the visual identification comments above,this,too,would be a natural sensory element of diving in the real world,so it would be interesting to note if similar criticisms were raised by Royal Navy divers undertaking early training using the MCMSim.

7.Discussion&conclusions

Surprisingly little work has been reported addressing why the provision of AAR can be so effective in simulation-based training(and even less research is available that tackles the issue of providing real-time,fully interactive forms of AAR). What happens to the user of a simulation system that makes him/her perform in a different way after exposure to an AAR process?In essence,it seems that a well-designedAAR system facilitates the discovery,on the part of the simulation user,of a range of new items of knowledge–about objects,contexts, processes and so on,that may not be explicit when performing the task for the first time.

There are three central themes that seem to reoccur in much of the(limited)reported work in this field.These are(a)appreciation on the part of the user as to the extent to which their own performance was successful,(b)specifically understanding which parts of a task performance were successful and unsuccessful and their impact upon the final outcome,and(c)how to construct new strategies from expert knowledge in order to perform better in the future.Such themes are often central to theories related to cognitive processes of learning,one in particular being the constructionist approach to learning(e.g.Reference[34]),which argues that knowledge acquisition is a process of continuous self-construction,based on interactions between personal experiences and ideas.Central to the effectiveness of such a theory is the ability to formulate plans and test their outcomes;it is this formulation of plans where AAR appears to be capable of enhancing the performance of individuals in virtual environments.Peachley[35]also highlights the importance of constructionist approach to learning in virtual environments,stressing the potential of VEs to be a test bed for the formulation of more effective strategies for the real world,as is the ultimate aim of the work presented in the present paper.By analysing the quantitative effects that AAR has on performance in a simulation,the results presented here have taken a small step forward from previously reported work, especially given the fact that subjective results have dominated most studies ofAAR to date.AAR attempts to address the often sporadic nature of strategy formulation in such environments, by providing the user with knowledge of success or failure evident in previous performances.Combining this with expert knowledge allows the user to formulate strategies that are far more effective for subsequent performances and may even help to guarantee a strong positive transfer of knowledge and skills from the virtual to the real and to minimise skill fade over time.

Whilst the research reported in the present paper only took the form of a pilot study,the impact of the results has been quite significant and has supported the further development of underwater scenarios for both visualisation and training purposes. Since the execution of the work described here,there have been numerous developments in the capabilities of games engines in supporting credible underwater scenarios,with physics engines supporting high-fidelity scenes featuring,for example,very fine particles,in both suspended and dynamic states,accurate underwater fogging effects and the ability to simulate backscatter from diver-held or vehicle-mounted light sources.The ability to embed software capable of capturing key elements of end user behaviours supporting real-time AARs–motion paths,dwell points,object interactions and own-view camera angles(i.e. viewing frustum)–has also been taken further in other defence and healthcare projects,for assessing the impact of interface technologies on end users’navigation and interaction strategies. These include the development of a deployment activity recording system for the UK’s CUTLASS bomb disposal telerobot VR trainer[9]and a“tracking and behaviour capture system”for assessing patient interactions with large-scale virtual restorative environments deployed in hospitals(e.g[36]).

Finally,a criticism that is often levelled by the vendors of commercial virtual reality or“serious games”software packages(particularly those selling into the defence simulation market)is that a significant weakness of the more Open Source, freely available or lower-cost toolkits and engines is that they are devoid of packages and processes by which credible AARs can be developed.One of the major outcomes of the present study is the finding that the development of a relevant and usable AAR does not require dedicated software packages. Indeed,not only can such an AAR subsystem be defined early on in the development of aVR project–by undertaking detailed and iterative observations and briefing sessions with instructors–such an early AAR definition can form the“backbone”upon which all of the other interactive and procedural elements of the final simulation can be developed.

Funding

This study was undertaken as part of a University of BirminghamYear 4(MEng)FinalYear Project,with funding made available from internal resources at the University of Birmingham’s School of Electronic,Electrical&Systems Engineering.

Ethical Statement

The experiments described in this paper were subject to a standardStudentProjectEthicalReviewconductedby members of the University of Birmingham’s Department of Electronics,Electrical&Systems Engineering Ethical Committee.Ethical approval was approved in advance of the experimental studies being conducted and all participants were anonymised.

Acknowledgement

The University of Birmingham co-authors(Stone and Cooke)would like to express their sincere thanks for the early briefings and material assets provided by the Defence Diving School at HMS Excellence,Portsmouth.

References

[1]Checchia M.Counter-improvised explosive devices:a global review (March,April&May 2012).Civil-Military Fusion Centre Report,June Issue,;2012[accessed 23.05.15].

[2]Weinberger S.Iran’s naval mines:the IED of the seas.Popular Mechanics. October 9 Online Issue,; 2012[accessed 23.05.15].

[3]Peniston B.No higher honor:saving the USS Samuel B.Roberts in the Persian Gulf.US Naval Institute Press;2006.

[4]Haesler JM.Sea devils,submersibles and underwater IEDs:asymmetric undersea threads to assured access in an operational environment.US Naval War College Report,Newport,Rhode Island,USA;2010.

[5]GlobalSecurity.org.IranianNavy,;2011[accessed 23.05.15].

[6]Freedberg SJ.Sowing the sea with mines.Breaking Defense,;2015[accessed 23.05.15].

[7]Ashton D.Unmanned maritime systems autonomy.Presentation delivered at the 10th International Mine Warfare Technology Symposium, Monterey,USA,; 2012[accessed 23.05.15].

[8]Endsley MR.Designing for situation awareness:an approach to user-centred design.second ed.CRC Press;2012.

[9]Stone RJ.Human factors guidance for designers of interactive 3D and games-basedtrainingsystems(secondedition).HumanFactors Integration Defence Technology Centre Publication,;2012[accessed 23.05.15].

[10]Stone RJ,Guest R.Virtual Scylla:interactive 3D and artificial life for marine virtual heritage.In Henderson,J(ed)Beyond Boundaries,IKUWA 3,Proceedings of the 3rd International Congress on Underwater Archaeology.Römisch-Germanische Kommission/NAS Publications, Frankfurt:485–491;2012.

[11]Kim KJ,Cho SB.A comprehensive overview of the applications of artificial life.Artif Life 2006;12:153–82.

[12]Cooke NJ,Stone RJ.RORSIM-a warship collision avoidance 3D simulation designed to complement existing junior warfare officer training.Virtual Real 2013;17(3):169–79.

[13]Luria M,Kinney J.Underwater vision.Science 1970;167(3924):1454–61.

[14]Hale W,Lane H,Lulla K,Chapline G,editors.Wings in orbit:scientific and engineering legacies of the space shuttle,1971 to 2010.1st ed.NASA Publication SP-2010-3249;2011.

[15]U.S.Army.Leader’s guide to after action reviews(AAR).US Army Combined Arms Center–Training,Fort Leavenworth,Kansas 66027; 2011.

[16]Morrison JE,Meliza LL.Foundations of the after action review process. United States Army Research Institute for the Behavioural and Social Sciences Special Report,Volume 42;1999.

[17]Clark RE,Pugh CM,Yates KA,Sullivan M.The use of cognitive task analysis and simulators for after action review of medical events in Iraq.Technical Report 5-21-2008 developed under contract W81XWH-04-C-0093 from U.S Army Medical Research and Materiel Command,CenterforCognitive Technology,RossierSchoolof Education,University of Southern California;2008.

[18]Martin GA,Daly J,Thurston CL.An after action review engine fortraininginmultipleareas.EngPsycholCognErgonomics 2011;6781:598–607.

[19]Knerr BW,Lampton DR,Martin GA,Washburn DA,Cope D.Developing an after action review system for virtual dismounted infantry simulations. In Proceedings of the 2002 Interservice/Industry Training,Simulation& Education Conference(I/ITSEC);2002.

[20]Thurston CL,Martin GA.A generic after action review capability for game-based training.Eng Psychol Cogn Ergonomics 2011;6781:628–34.

[21]Smith R,Allen G.After action review in military training simulations.In Proceedings of the 26th Winter Simulation Conference(WSC‘94), Society for Computer Simulation International,San Diego,USA: 845–849;1994.

[22]Vertigo Games.World of Diving,;2013[accessed 23.05.15].

[23]Fletcher B,Harris S.Development of a virtual environment-based training System for ROV pilots.In Proceedings of OCEANS‘96;Prospects for the 21st Century,Fort Lauderdale,USA,MTS/IEEE,Volume 1:65–71;1996.

[24]Hamzah MSM,Zakaria M,Abd Jalil MFI,Zamli KZ.3D virtual simulation software for underwater application.In Proceedings of the Second International Conference on Underwater System Technology: Theory and Applications,Bali,Indonesia;2008.

[25]Ge X,Guo W,Li Z.Design of ROV training simulator.Appl Mech Mater 2012;130–134:170–4.

[26]Kawamoto LT Jr,Slaets AFF,Rodrigues SCM.Development and validation of a virtual environment to evaluate the capacity in executing the safety procedures in recreational scuba diving.Games Health 2012;1(2):104–14.

[27]Chu PC,Gilles AF,Fan CW,Lan J,Fleischer P.Hydrodynamics of falling cylinder in water column.Adv Fluid Mech 2002;4:163–81.

[28]CISR.Iraq ordnance identification guide.James Madison University Center for International Stabilization&Recovery Report,sponsored by the Office of the Secretary and Deputy Secretary of Defense for Special Operations/Low-Intensity&Interdependent Capabilities(SO/LIC)and fundedbytheDefenseSecurityCooperationAgency(DSCA), ; 2004.[accessed 23.05.15].

[29]MoD.Report of the service inquiry into the grounding of HMSAstute on 22 Oct 2010.Ministry of Defence(MoD)publication;01 January.2011, ;2011[accessed 23.05.15].

[30]Robson C.Real world research.2nd ed.Oxford.:Blackwell Publishing; 2002.

[31]Stone RJ,Caird-Daley A,Bessell K.SubSafe:a games-based training system for submarine safety and spatial awareness.Virtual Real 2009;13(1):3–12.

[32]Stone RJ,Caird-Daley A,Bessell K.Human factors evaluation of a submarine spatial awareness training tool.In Proceedings of the Human Performance at Sea 2010 Conference,Glasgow:231–241;2010.

[33]Ravden SJ,Johnson GI.Evaluating the usability of human-computer interfaces.Ellis Horwood;1989.

[34]Piaget J.The principles of genetic epistemology:selected works. Routledge 1998;1972.

[35]Peachley A.Researching learning in virtual worlds.Springer;2010.

[36]Stone RJ,Hannigan FP.Applications of virtual environments:an overview,Chapter 36.In:Hale KS,Stanney KM,editors.Handbook of virtual environments:design,implementation and applications.CRC Press;2014.p.881–955[Chapter 36].

Peer review under responsibility of China Ordnance Society.

.

E-mail address:r.j.stone@bham.ac.uk(R.STONE).

1Formerly a student at the University of Birmingham,now with Jaguar Land Rover.

12 March 2016;revised 14 June 2016;accepted 15 June 2016

Based on the earlier briefings by instructors and specialists at the Defence Diving School,it was concluded that the AAR system developed for the present simulation should support a range of interactive activities on the part of the instructor/experimenter,thereby allowing him/her to be able to provide the participants with the best quality feedback possible on their performance.It should be noted that the AAR facility developed during this project was not automated as has been seen with other systems mentioned previously–TAARUS and DIVAARS,for example.Instead,and following consultation with DDS subject matter experts,the present solution was designed to support the instructor in helping to convey their own task-specific knowledge,thus enhancing the overall AAR process.