China’s EarthLab—Forefront of Earth System Simulation Research

Zhaoyang CHAI ,He ZHANG* ,Mulan ZHANG ,Xiao TANG ,Weipeng ZHENG ,Jiang ZHU ,Guangqing ZHOU,Junji CAO,and Qingcun ZENG

1Institute of Atmospheric Physics,Chinese Academy of Sciences,Beijing 100029,China

2University of Chinese Academy of Sciences,Beijing 100049,China

ABSTRACT This article introduces“EarthLab”,a major new Earth system numerical simulation facility developed in China.EarthLab is a numerical simulation system for a physical climate system,an environmental system,an ecological system,a solid earth system,and a space weather system as a whole with a high-performance scientific computing platform.EarthLab consists of five key elements—namely:a global earth numerical simulation system,a regional high-precision simulation system,a supercomputing support and management system,a database,data assimilation and visualization system,and a high-performance computing system for earth sciences.EarthLab helps to study the atmosphere,hydrosphere,cryosphere,lithosphere,and biosphere,as well as their interactions,to improve the accuracy of predictions by integrating simulations and observations,and to provide a scientific foundation for major issues such as national disaster prevention and mitigation.The construction and operation of EarthLab will involve close cooperation with joint contributions and shared benefits.

Key words:EarthLab,earth system numerical simulation,high-performance computing platform

1.Introduction

China’s Earth system numerical simulation facility,named“EarthLab”,is a numerical simulation system for the main earth systems with matching software and hardware.EarthLab is a major new earth system science (ESS) facility that is comparable to its international counterparts around the world,such as the Community Earth System Model (CESM) produced by the National Center for Atmospheric Research (NCAR).It is the first comprehensive virtual earth laboratory in China for simulating the physical climate system,environmental system,ecological system,solid earth system,and space weather system as a whole with a high-performance scientific computing platform.Combined with traditional theories and observations,it enables deeper studies of the physical and biological components of the Earth—namely,the atmosphere,lithosphere,hydrosphere,and biosphere,as well as their interactions.EarthLab consists of five key elements—namely a global earth numerical simulation system,a regional high-precision simulation system,a supercomputing support and management system,a database,data assimilation and visualization system,and a high-performance computing (HPC) system for the earth sciences.Fiugre 1 shows a schematic diagram of EarthLab’s framework and content.

The Institute of Atmospheric Physics (IAP),Chinese Academy of Sciences (CAS),started developing EarthLab in 2018,working in tandem with Tsinghua University.EarthLab is located in the Miyun Economic Development Zone,Eastern Cluster of Huairou Science City,Beijing.The first phase of EarthLab,including construction and installation of its computing systems and infrastructure,is expected to be preliminarily completed in 2021,at which point it will be available for trialing,followed by inspection and approval.EarthLab is expected to become fully operational in 2022.IAP will manage Earth-Lab in the aspects of scientific research planning,daily operation,staff hiring,finance management,education of graduate students,and scientific research services.More details on EarthLab can be found at the official website (http://earthlab.iap.ac.cn).

Fig.1.Schematic diagram of EarthLab’s framework and content.

2.Earth system science

2.1.Background

ESS,emerging in the 1980s following requirements for a new“science of the Earth”,aims to study the physics,chemistry,and life processes of the various spheres of the earth system and to clarify how these aspects evolve over time,as well as to elucidate upon the mechanisms by which they interact (Steffen et al.,2020).Since the earth system is extremely large and complex,traditional theories and observations are too limited to meet the overall requirements of the scientific research community (Heavens et al.,2013).Numerical simulations that combine theories,observations,and HPC have become the core method in this regard when researching the earth system (Wang et al.,2014;Wang and Yuan,2020).This method,known as the earth system model (ESM),refers to the coupled atmosphere-ocean-land-sea ice model which comprises the physical climate model,plus biogeochemical components to represent,for example,the carbon cycle (Zeng et al.,2008;Flato,2011;Randall et al.,2019).Earth labs,as a combination of an ESM and its supporting facilities,facilitate various controllable scientific experiments in the virtual digital laboratory,i.e.,the supercomputer,just like in other physics and chemistry disciplines,providing unprecedented opportunities for new findings and breakthroughs.

2.2.Context

Realizing the significance of earth labs,developed countries such as the United States and Japan,along with some European countries,have built specialized numerical simulation facilities for the earth system.NCAR,with its world-leading earth system numerical simulator,has developed an excellent ESM (CESM,https://www.cesm.ucar.edu/).Japan has performed a set of mesoscale resolving simulations with a world-leading 1.9 km horizontal resolution global atmospheric model MSSG-A (Takahashi et al.,2011) and has achieved the world's largest weather data assimilation experiment with its customized Earth Simulator (also including the successive K computer and supercomputer Fugaku),greatly enhancing its international influence (Sato,2004;Yashiro et al.,2020).At present,the European Commission is building a“digital twin”of planet Earth (Bauer et al.,2021).

The current lack of a numerical simulation facility dedicated to earth system science greatly restricts the development of ESMs and the simulation of climate change in China.As a rough estimate,China’s previous versions of global climate models,which participate in the IPCC’s Fifth Assessment Reports,lag behind those of developed countries by 10 years(Zhou et al.,2014).In particular,the topographic environment in China is extremely complex,and thus weather,climate,and environmental disasters occur frequently and seriously with grave losses of life and property.Consequently,a global earth system simulation system,as well as a high-precision regional environmental simulation system,are urgently needed to better predict climate and environment variability,to prevent and mitigate natural disasters more effectively,and to formulate relevant national plans more scientifically.Furthermore,EarthLab will enhance international collaborations on earth system simulation research.

2.3.Scientific objectives

EarthLab is committed to the following scientific objectives:

(1) Use of an ESM to explore the physical,chemical,and biological processes of the atmosphere,hydrosphere,cryosphere,lithosphere,and biosphere,and their interactions,and to probe the effects of these interactions on the whole earth system.

(2) Use of a separate space model to investigate the interplanetary evolution of solar wind and to carry out space weather forecast experiments,compensating for the scarcity of solar–terrestrial observations.Use of a separate ice-sheet model to predict the evolution of ice sheets under the influence of climate change,and to explore the feedback of ice sheets to regional and global climate,as well as their effects on the global gravity and crustal stress fields.Use of a separate solid earth model to investigate long-term evolution of the upper and lower mantle and the lithosphere,to solve medium to longterm lower crustal deformation problems,to solve short-term middle-upper crustal deformation issues,and to simulate surficial geological processes.

(3) Use of regional models for research on weather forecasting,air pollution forecasting and early warning,agricultural drought forecasting,and climate change risk assessment over China,as well as providing a scientific foundation for major issues such as national disaster prevention and mitigation and improving the atmospheric environment.

(4) Use of a regional atmospheric model with two-way coupling to a global model for seasonal prediction over China,especially in high-terrain regions.

(5) Development of databases integrating simulations and observation data to improve the accuracy of predictions,and to achieve quantitative descriptions and simulations of complex processes of the earth system at mesoscale resolutions.

(6) Further development of the ESM and its components and integration of the separate models to participate CMIP7.

3.Key elements

Key features of EarthLab are given with respect to five elements as follows:

(1) The ESM will feature horizontal resolutions of 10–25 km on a global scale,which enables the earth spheres to be simulated well,including the atmosphere,the oceans (including sea ice),the land surface,vegetation (e.g.,dynamic vegetation evolution),aerosol and atmospheric chemical processes,and biogeochemical processes on land and sea (e.g.,carbon cycle),and their interactions.Initially and separately,there will be a development and application of a solar-terrestrial space environment model,an ice sheet model,and a solid earth model.

(2) The spatial resolution of the regional high-precision environmental simulation system will reach 3 km over the whole of China and 1 km over some key areas of the country,which is geared towards directly tackling key regional environmental problems,such as weather forecasting,air pollution forecasting and early warning,agricultural drought forecasting,and climate risk forecasting.

(3) There will be a dedicated support and management system to facilitate earth system modeling,which will provide support for program development,debugging,and performance tuning,as well as for user-oriented model testing,evaluation,and adjustment.

(4) EarthLab will include a data assimilation system and a related database,with a cutting-edge visualization system in support of the numerical simulation of the earth system in China.

(5) The high-performance computing system will have 15 PFLOPS of peak performance and 80 PB of storage capacity,which will provide fundamental support for the entire EarthLab facility.

3.1.Global earth numerical simulation system

The ESM is one of the key software systems of EarthLab,which is currently the CAS-ESM version 2 (Zhang et al.,2020),and has been used for CMIP6 simulations.It consists of several component models—namely,the Atmospheric General Circulation Model developed in IAP (IAP-AGCM version 5),the LASG/IAP Climate system Ocean Model (LICOM version 2),the Beijing Normal University/IAP Common Land Model (CoLM),the Los Alamos Sea-Ice Model (CICE version 4),and the IAP Aerosol and Atmospheric Chemistry Model (IAP-AACM),using the CESM Coupler 7 (CPL7) to integrate all the components above.It also comprises the IAP Dynamic Global Vegetation Model (IAP-DGVM) and the IAP fire model that are incorporated into the land model,and the IAP ocean biogeochemistry model that is incorporated into the ocean model.In addition,the mesoscale Weather Research and Forecasts (WRF) model is coupled to atmospheric model IAP AGCM through CPL7 with two-way interactions to enable regionally refined climate simulation or one-way downscaling (He et al.,2013).

IAP-AGCM 5 is a grid-point model with a terrain-following coordinate (Zhang et al.,2013).It has three configurations of horizontal resolution with 1.4° × 1.4°,0.5° × 0.5°,and 0.31° × 0.23° (lon × lat).The default vertical configuration has 35 levels with a model top at 2.2 hPa while the high-top version has 69/91 levels with a model top at 0.01 hPa (Chai et al.,2021a,b).LICOM 2.0 (Liu et al.,2012) uses η-coordinates with horizontal resolutions of～1.0° × 1.0° globally,increasing to 1.0° × 0.5° (lon × lat) between 10°S and 10°N.It has 30 vertical layers with a grid spacing of 10 m in the upper 150 m.Additionally,the Ocean Biogeochemistry General Circulation Model (OBGCM) was developed and embedded into LICOM to represent the natural carbon cycle and the uptake and storage of anthropogenic COin the ocean (Xu et al.,2013).CoLM (Dai et al.,2003),with its advanced performance,was well-known as the Community Land Model (CLM)incorporated into the Community Climate System Model (CCSM).It has 15 vertical soil layers resolving down to 42.1 m deep and five snow layers,using the same horizontal resolution with IAP-AGCM.The latest version of CoLM contains the biogeochemical processes in the soil and provides the lower boundary of greenhouse gases for the global climate model,including carbon dioxide,methane,and nitrous oxide,which is under development (e.g.,Li et al.,2020).IAP-DGVM has 12 kinds of natural land vegetations classified by physical,phylogenetic,phenological parameters,and bioclimatic limitations (Zeng et al.,2014),giving an advanced performance in the simulation of vegetation distributions and carbon fluxes(Zhu et al.,2018).The IAP fire model is developed based on a fire parameterization,which is used to simulate the fire burned regions,fire seasonality,and interannual variability as well as to further characterize the carbon cycle and ecological processes (Li et al.,2012).Both the IAP-DGVM and the IAP fire model are embedded within the CoLM.The IAPAACM is developed to calculate gaseous chemistry,aqueous chemistry,heterogeneous chemistry,dry and wet deposition,which is coupled with IAP-AGCM through a coupler via two-way interactions (Chen et al.,2015;Wei et al.,2019).

For different research objectives,the above component models are coupled and integrated into two versions of the ESM as follows:

(1) A medium-resolution ESM,which consists of all component models described above.The horizontal resolution is 140 km for the atmospheric model,and 100 km for the ocean model,while the resolutions for the other component models are consistent with the atmosphere or ocean model.In reference to the experimental design for climate change in CMIP6,a series of numerical simulations have been completed (e.g.,Zhang et al.,2020;Dong et al.,2021;Jin et al.,2021).

(2) A high-resolution climate system model,which consists of three sub-systems:atmosphere,ocean (including sea ice),and land surface.The horizontal resolution is 25 km for the atmosphere and land surface models,and 10 km for the ocean model.This version will also contribute to the simulations in the next generation of CMIP,such as HighResMIP,and will be directly used for short-term climate prediction.

The ESM also comprises a solar-terrestrial space environment model,an ice sheet model,and a solid earth model,which are not coupled with the other seven component models during the current phase but will be integrated into the ESM during the second phase.The solar-terrestrial space environment model is an integration of a kinetic model and a three-dimensional magnetohydrodynamics model,which can be used to simulate the evolution of the solar wind from the Sun to the Earth.The ice sheet model can simulate the ice sheet flow at the continental scale based on satellite data and results from the atmosphere and ocean via solving equations governed by ice dynamics using the finite element method.The solid earth model is a multi-scale and multi-field system containing the mantle convection and plate motion module,the global viscoelastic and elastic module,the sea-storm tide and tsunami module,the landscape evolution module,and the regional porous elastic module.All three separated component models are under development.Figure 2 shows a schematic diagram of the ESM numerical simulation system.Most of the component models will gain further improvement and development based on the existing self-developed models.This project will build a software platform that integrates the latest scientific understanding of the earth system,facilitates understanding of the basic laws of physics,chemistry,and biogeochemistry for the various spheres of the earth system and their interaction mechanisms,while serving research and applications in earth system science.

3.2.Regional high-precision simulation system

The regional high-precision simulation system is targeted at key environmental issues that impose major impacts on the national economy and people’s livelihoods,such as weather forecasting,atmospheric pollution prediction and warnings,agricultural drought prediction,and climate risk evaluation.The system consists of four sub-systems—namely,a regional cloud-resolving weather forecasting sub-system,a regional high-resolution air pollution modeling sub-system,a drought modeling sub-system for grain crops of major agricultural production areas in China and the world,and a regional high-precision long-term climate change risk modeling sub-system.Through model development and software system engineering,regional refinements in the simulation of these key environmental elements will be realized.The spatial resolution will reach 1–3 km over the whole of China,greatly enhancing the current level of high-resolution simulation technology and capacity catering for scientific research and application needs in terms of the regional environment.The relationships among the four sub-systems are shown in Fig.3.

(1) The Regional Cloud-Resolving Weather Forecasting sub-system is to provide accurate and precise forecasting products and improve the forecasting accuracy of heavy rainfall,strong convection,typhoons,and other high-impact weather based on the Weather Research and Forecasting Model (WRF).It has an advanced multi-source observation data assimilation module,which integrates various conventional or irregular meteorological observations to establish the assimilation analysis field.

(2) The Regional High-Resolution Air Pollution Modeling sub-system is a new-generation air quality modeling system based on Global Nested Air Quality Prediction Model System (GNAQPMS,Wang et al.,2017).It includes an aerosol microdynamic module,a secondary organic aerosol module,a combined moisture absorption and growth of aerosol component module,and a visibility module.It involves advanced modeling techniques,such as emissions inversion,ensemble forecasting,and source apportionment.

(3) The Regional High-Precision Long-term Climate Change Risk Modeling sub-system is designed to analyze the trends of global climate change,the trends and thresholds of East Asian climate,and the impacts of climate change on socioeconomics over China and East Asia.It uses climate change data,energy systems,and a human greenhouse gas emission assessment application module to calculate the greenhouse gas emissions generated by human socio-economic and energy activities as well as the surface characteristics of the climate change risk assessment area.

Fig.2.Schematic diagram of the ESM software platform.

Fig.3.The regional high-precision environmental simulation system.

(4) The Drought Modeling subsystem for Grain Crops in Major Agricultural Regions enables the simulations of the global agricultural remote sensing drought index and the formation of agricultural drought.It can automatically extract the global farmland distribution,including paddies,drylands,and irrigated water areas.It includes a global database of six major cereal crops (rice,maize,wheat,sorghum,rice,and barley) and a growth modeling module.It enables the simulation of food-crop growth in China and major agricultural areas worldwide under conditions of real-time meteorological observations and dynamic analysis data.

3.3.Supercomputing support and management system

The supercomputing support and management system has support functions for the development and transplantation of the underlying code,debugging and tuning,and optimization of the computing environment,as well as management functions for user-oriented model testing,evaluation,and tuning.Based on the common C-Coupler,the flux interactive ensemble coupled platform,and the physical scheme parameter tuning platform (all with independent R&D in China),the system utilizes domestic and foreign advanced coupling techniques,quantitative uncertainty analysis methods,and data analysis techniques,as well as adopting a means of modern software engineering with the intent of building an integrated supercomputing support and management system.The system addresses the bottlenecks encountered by China’s ESMs in terms of parallelization,componentization,coupler techniques,and large data analysis,in addition to supporting full-process scientific research activities covering the ESM design,development,debugging,verification,release,and application with flexibility and efficiency.The specific content of construction includes three sub-systems of computing and encapsulation,model resources,and model services.The hierarchical relationships among the sub-systems and the parallel relationships among the sub-systems are shown in Fig.4.

3.4.Database,data assimilation and visualization system

The database,data assimilation,and visualization system provides data support,assimilation tools,model initial and forcing data,database management,and scientific data visualization functions for the whole of EarthLab,and underpins the earth system numerical simulation.It consists of six sub-systems as follows:data assimilation,observation and simulation experiments,satellite key parameter retrieval,data merging,supporting database,and visualization,as shown in Fig.5.

Fig.4.The hierarchical relationships among the three sub-systems and the parallel relationships among the sub-systems in the supercomputing support and management system.

Fig.5.Schematic diagram of the database,data assimilation,and visualization system.

This system acquires massive amounts of routine observational data,radar data,and meteorological satellite remote sensing data for China and develops a data assimilation system by employing an advanced variational and ensemble data assimilation,and builds a supporting database and analysis dataset shared by the whole of EarthLab.The system provides high-precision,high-resolution input data,and validation data for global-scale earth system modeling and regional high-precision environmental simulations,and serves as an efficient visualization tool that supports visual analysis of big data for EarthLab.

The observation and simulation experiment sub-system,will establish the classified quality identifications of the observations through the real-time monitoring and evaluation of the instruments on in-orbit satellites including FY-3 and FY-4,and generate real-time parameters of the earth and space environment that affect the observations by the in-orbit detectors based on all kinds of existing observations and simulations.The sub-system will also build a simulation system based on the requirements for designing new payloads of satellites.It helps to obtain optimal parameters of in-orbit instruments through simulation performance tests,in order to provide a basis and essential parameters for the product algorithm,assist in the joint debugging of the ground system,and offer scientific references,and a basis for such issues as the constellation layout and payload update of the subsequently launched satellites.

The satellite key parameter retrieval subsystem will use FY-series geostationary and polar orbital meteorological satellite data to establish a real-time data processing and retrieval system with rigorous quality control of the observations and appropriate retrieval algorithms.The system can efficiently and regularly receive,process,and retrieve key atmospheric parameters with high accuracy and high resolution,such as temperature,humidity,and cloud parameters,providing real-time satellite data for verification of the simulations.Moreover,it will reprocess the long-term sequence data of historical satellites to obtain consistent and reliable data (level 1),which can be used to obtain key atmospheric parameters that have significant effects on weather and climate,such as cloud,aerosol,and atmospheric precipitation.The acquisition of these parameters can provide input data for the earth system model,deepen our understanding of weather processes,and offer observational facts for testing and improving ESMs.

The data merging subsystem will collect real-time observational data derived from automatic observation stations,radars,satellites,and model data from numerical forecasts to yield multi-source,high-quality,and high-precision meteorological grid products using data fusion and assimilation technology.All of the data will be preprocessed by the other sub-systems before getting into the data merging subsystem.The product will contain land surface temperature,wind,humidity,precipitation,and solar radiation data to drive land models and 3-D cloud fields for the hot start of mesoscale atmosphere models,and so on.

3.5.High-performance computing system for earth sciences

As one of the core contents of the EarthLab hardware as a whole,the HPC system for earth sciences will provide a high-performance and extensible hardware platform specially oriented to perform ESM numerical simulations and regional high-precision environmental simulations,and support the efficient operation and data exchange for various models.Further,it provides external services via high-speed networks,and necessary hardware support for the simulation support and management system,database,and data assimilation and visualization system.The system consists of five sub-systems as follows:model calculation,data storage,network switching,support and management,and infrastructure.The framework is shown in Fig.6.Upon completion,it will become a world-leading earth system simulation computer.This system has a high-speed network connected with the China Science and Technology Network (CSTNET) and the China Education and Research Network (CERNET),which enables open sharing of the EarthLab computing resources to users at home and abroad.

4.Future prospects

The Chinese name for EarthLab is“Huan”,a place as vast as the earth where people live and upon whose land they depend.The name was voted for by children in an outreach activity in 2017 organized by IAP (http://english.iap.cas.cn/home/News/201705/t20170523_233814.html) and it is hoped that EarthLab,once completed,will also become a science outreach base for children and students to get to know the earth sciences.The construction and operation of EarthLab will involve close cooperation with joint contributions and shared benefits.Stakeholders including the China Meteorological Administration,the Ministry of Natural Resources,the Ministry of Ecology and Environment,the Ministry of Water Resources,the Ministry of Emergency Management,the Ministry of Land and Resources,the Ministry of Agriculture and Rural Affairs,and the National Food and Strategic Reserves Administration have expressed interest in using EarthLab for hazard prediction and mitigation.

EarthLab is also open to universities and research institutes all over the world.IAP will oversee the all-around management,scientific research planning,daily operation,staff,finance,graduate students,and scientific research services of Earth-Lab,and establish an operation and management mechanism that highlights“open,shared,circulated,and cooperative”development.The goals are:1) to have mobile posts and open funding,hire part-time staff,and adopt other open management models;2) to appeal to top-notch talents and outstanding teams at home and abroad to carry out research and maximize the technical advantages of EarthLab;and 3) to support major scientific discoveries and diplomatic negotiations on major national climate and environment issues,as well as to formulate relevant plans with the power of science and technology.

Fig.6.Framework diagram of the high-performance computing system for earth sciences.

To make better use of EarthLab’s resources including software and hardware,a user committee will be set up when the construction of EarthLab is completed.The committee is responsible for releasing project planning with topics of interest every year.Both domestic and overseas scientists who are interested in the topics can apply for the projects.By reviewing proposals,the committee decides which projects could be funded.The selected applicants would then share EarthLab’s resources.

Along with other earth simulators around the world,the development and construction of EarthLab will advance not only the understanding of the Earth’s spheres and their interactions,and Earth’s past,present,and future,but also contribute to the progress of computational mathematics,high-performance computing,technology,and other broader fields.

.The authors sincerely thank the anonymous reviewers and professor Jianhua LU for their comments and suggestions to improve this manuscript.The authors would like to acknowledge all EarthLab team members for their tremendous efforts in system development.This work was supported by the National Key Scientific and Technological Infrastructure project“Earth System Numerical Simulation Facility”(EarthLab) and the National Major Research High-Performance Computing Program of China (Grant No.2016YFB0200800).