UltraStar: A Lightweight Simulator of Ultra-Dense LEO Satellite Constellation Networking for 6G

Xiaoyu Liu,, Ting Ma,, Zhixuan Tang,, Xiaohan Qin,, Haibo Zhou,Senior , and Xuemin (Sherman) Shen,

Abstract—The mega-constellation network has gained significant attention recently due to its great potential in providing ubiquitous and high-capacity connectivity in sixth-generation(6G) wireless communication systems.However, the high dynamics of network topology and large scale of mega-constellation pose new challenges to the constellation simulation and performance evaluation.In this paper, we introduce UltraStar, a lightweight network simulator, which aims to facilitate the complicated simulation for the emerging mega-constellation of unprecedented scale.Particularly, a systematic and extensible architecture is proposed,where the joint requirement for network simulation, quantitative evaluation, data statistics and visualization is fully considered.For characterizing the network, we make lightweight abstractions of physical entities and models, which contain basic representatives of networking nodes, structures and protocol stacks.Then, to consider the high dynamics of Walker constellations, we give a two-stage topology maintenance method for constellation initialization and orbit prediction.Further, based on the discrete event simulation (DES) theory, a new set of discrete events is specifically designed for basic network processes, so as to maintain network state changes over time.Finally, taking the first-generation Starlink of 11 927 low earth orbit (LEO) satellites as an example,we use UltraStar to fully evaluate its network performance for different deployment stages, such as characteristics of constellation topology, performance of end-to-end service and effects of network-wide traffic interaction.The simulation results not only demonstrate its superior performance, but also verify the effectiveness of UltraStar.

I.INTRODUCTION

THE current terrestrial networks face significant challenges to support seamless coverage, massive connectivity, and diverse applications to meet exponentially growing data traffic [1].Fortunately, recent developments in ultralarge-scale low earth orbit (LEO) satellite constellations may make up for the shortcomings of traditional terrestrial networks [2]–[5].An ultra-dense (UD) constellation formed by thousands of LEO satellites can supply ubiquitous coverage and provide reliable and low-latency services all over the world [6], [7].Unlike traditional geosynchronous earth orbit(GEO) satellites, LEO satellites orbit the earth at an orbital altitude of less than 2000 kilometers.Such a low orbital altitude means that the communication delay between satellite and ground station is relatively low.Moreover, the faster speed of light in a vacuum and the avoidance of long and winding terrestrial optical fiber paths will make up for the connection overhead between satellite and ground station,which will enable the LEO satellite network to provide lower end-to-end communication delays for long-distance communications than terrestrial optical fiber communication networks.Differently from the existing satellite network [8]–[10], which can only provide limited access, the emerging LEO satellite constellation can not only expand the network coverage to the most remote areas with broadband speed and low latency, but also compete with the terrestrial network for traditional traffic services in the current market.In order to achieve global coverage and provide sufficient access bandwidth for a larger target user group, the number of satellites of the new system is unprecedented.Therefore, the name“mega-constellation”is created.New technologies such as the miniaturization of satellites and reusable rockets also make these emerging LEO satellite constellations possible [11], [12].In recent years,more and more LEO satellites are launched into space.Since SpaceX [13] announced its project Starlink in 2014, it has developed into an LEO constellation system with the fastest transmission frequency and the largest number of satellites in orbit.Several other commercial enterprises have also announced their constellation projects, such as OneWeb [14]and Amazon [15].

Driven by the above exciting prospects of the future megaconstellation, researches on constellation simulation, performance evaluation and network optimization are urgently requisite.Fundamentally different from the traditional terrestrial network, the high mobility of a large number of satellites result in highly dynamic constellation topology and intermittent connectivity.Routers and switches in terrestrial wired networks are generally static, and even in terrestrial mobile cellular networks, the dynamics and connectivity of mobile nodes are far less complex than those of LEO satellites in the mega-constellation network.A large number of LEO satellites move rapidly relative to the earth.At the same time, relative movement between satellites is also taking place[16]–[18].Numerous heterogeneous links, including intersatellite links (ISLs), inter-orbital links (IOLs) and user data links (UDLs), will bring frequent establishments of new links and interruptions of old links, which means that routes over the mega-constellation network need to be updated in time to adapt the special high dynamics.Therefore, the quantitative analysis of mega-constellations on topology characteristics and network performance faces several challenging problems.First, the maintenance of highly dynamic topology for ultralarge-scale constellation networks is unprecedented.Simulating the small-scale satellites over time is usually implemented by means of orbit analysis tools (e.g., STK [19]).But they are time-consuming and resource-intensive, or even impractical,when maintaining the ultra-large-scale network.Additionally,these tools only focus on characteristics of topology architecture but do not have the ability to implement the network simulation (e.g., protocol-level simulation).Second, there is a lack of systematic and effective tools for network simulation of ultra-large-scale LEO satellite constellations.Most of today’s open-source projects are primarily designed for satellite positioning, precision navigation, and Earth observation,without the ability to simulate LEO constellations.For example, SNS3 [20] is a high-fidelity ns3-based satellite communication simulator.But limited to the static simulation configuration of one geostationary satellite, the current version of SNS3 cannot support the simulation of LEO constellations.

In order to facilitate the complicated simulation for the emerging mega-constellation of unprecedented scale, we develop a lightweight network simulator named UltraStar.Taking the first-generation Starlink of 11 927 LEO satellites as an example, we use UltraStar to fully evaluate its network performance for different deployment stages, such as characteristics of constellation topology, performance of end-to-end service and effects of network-wide traffic interaction.

We summarize our contributions as follows:

1) A Systematic Simulation Framework for Mega-Constellations:We develop UltraStar, a lightweight network simulator,which aims to facilitate the complicated simulation for the emerging mega-constellation.Particularly, a systematic and extensible software architecture is proposed, where the joint requirement for software simulation, quantitative evaluation,data statistics and visualization is fully considered.Based on the design principle of polymorphic module, its extensibility can extend to more scenarios of 6G oriented space-air-ground integrated networks.

2) Lightweight Simulator Module Design:For characterizing the satellite network, we first make lightweight abstractions of physical entities and models, where the basic representatives of networking nodes, structures, and protocol stacks are developed respectively.To further account for the high dynamics of mega-constellations, we give a two-stage topology maintenance method, where the complete constellation is first initialized with the real configuration parameters, and then the satellite trajectories are updated over time through the orbit prediction interface.Finally, based on the DES theory, a new set of discrete events is specifically designed for basic network processes such as packet processing and discrete movement, so as to maintain network state changes over time.

3) First Trial for Mega-Constellation Analysis:To the best of our knowledge, this is the first trial to simulate the ultralarge-scale LEO constellation consisting of more than 10 000 satellites.Taking the first-generation Starlink as an example,we first provide a complete visualization of this mega-constellation for establishing a good intuition of the emerging network.Then, to demonstrate the great potential of the constellation, basic network characteristics at different deployment stages are fully evaluated.Finally, in the simulated scenario of network-wide business, extensive simulation results are further presented to provide insight into network dynamics,including round-trip time (RTT) fluctuations, path structure changes and link utilization fluctuations.As a preliminary study of the mega-constellation network, interesting and enlightening conclusions have been given in our work.

The remainder of this paper is organized as follows.In Section II, we present the related work.Section III describes the architecture of UltraStar and the mechanism of discrete event simulation.The detailed module design and constellation mobility formulation are presented in Section IV.Then, in Sections V and VI, the evaluation results for constellation characteristics and end-to-end connection performance in different phases are given.Finally, Section VII concludes the paper.

II.RELATED WORK

Some existing works have preliminarily analyzed smaller LEO satellite constellation networks in terms of low latency,large capacity and high bandwidth.In [21], it mainly focused on reconstructing LEO satellite networks and analyzed their end-to-end low latency potential.In [22], the limitations of routing mechanism caused by the high dynamics of constellation topology were fully discussed, and a deployable and effective routing mechanism was proposed.In [23], it proposed a statistical method to estimate the throughput of the constellation system and ran an optimization process to reduce the total number of ground stations required to support the system throughput.In [24], it analyzed the delay performance that LEO satellite networks may provide and compared it with opportunistic networks using commercial flights as relays.As a good starting point for exploring the constellation network,the above works have won widespread attention in the field of network research.However, most of these works only focused on the evaluation of topology characteristics and do not exactly realize the network simulation, where protocol settings and traffic interactions are fully considered.

Fig.1.The software architecture of UltraStar.

Meanwhile, the research on LEO satellite network simulators has also received extensive attention from the community.In [25], a network simulation platform was proposed to support various mobility tracking in space, aerial and terrestrial networks, as well as the verification and research of various protocols.In [26], a theoretical model was proposed to estimate the number of ISL hops between ground users in the inclined orbit constellation network.The global distribution pattern of hops and the path differences caused by different access satellites were explained by using the constellation configuration parameters of Starlink.In [27], it provided an ns-3 based LEO satellite network simulator and analyzed the network characteristics of three different small-scale constellations, giving a visualization of the topology.In [28], it proposed a constellation performance simulation platform where grid systems were constructed on the Earth’s surface to geographically model network performance to measure coverage,latency and system throughput.Most of these works implemented small-scale satellite network simulations by building ns-3 based constellation simulators, but they are extremely time-consuming and resource-intensive, or even infeasible, for simulating mega-constellations containing tens of thousands of satellites.Therefore, with the continuous expansion of the network scale, new obstacles are set in the design, analysis and optimization, that is, the lack of simulation tools that include the behavior analysis of mega-constellation networks.

III.ULTRASTAR ARCHITECTURE

A.Architecture Overview

We propose a lightweight satellite network simulator, Ultra-Star, which aims to implement packet-level network simulation for ultra-large-scale LEO satellite constellations.Particularly, the high dynamics of large networks are efficiently maintained and the quantitative analysis of network performance is fully enabled.UltraStar completes the lightweight abstract design of real-world physical entities (e.g., nodes, network devices, protocols, models), which rely on the discrete event simulation mechanism to achieve time-based changes.As indicated in Fig.1, a lightweight, efficient and extensible software system architecture is fully implemented, where the optimization of computing resource allocation and the avoidance of computing redundancy are both considered.Significantly, the scalability and flexibility of the architecture design can extend the simulation to the research of 6G oriented space-air-ground integrated network [29]–[32], which has rich prospects and practical significance.

UltraStar mainly includes five core parts: control, topology,network, schedule and auxiliary.We make a basic introduction to these core modules consisting the architecture: 1) Control core is mainly the external functional interface of the software core, which realizes the interaction with users and controls the simulation process.Parameter configuration and module enabling are integrated in the simulation control module, which also provides external interfaces for data statistics and element visualization.2) For generating the ultra-largescale network topology, the physical module first instantiates network structures and nodes according to specified configuration parameters, and then maintains the node mobility model.Once one node moves, the logical module updates its link relationship with neighbors.Therefore, the high dynamics of the topology can be fully simulated.3) Network core implements the specific process of packet-level network simulation.In order to simulate the real communication capabilities, the network module is responsible for the abstraction of network devices and the maintenance of various protocols.Specifically, the network device is abstracted as the master protocol stack, interface cache and transceiver model, which correspond to the network layer, data link layer and physical layer in the Internet protocol stack respectively.Meanwhile,the effect of the accompanying path loss and attenuation on transmission is also considered by the channel model.Finally,in the traffic module designed for network data interaction,different types of packets, traffic and services can be defined.4) Auxiliary core mainly includes visualization module and data center module.The visualization module receives data in a specific format to complete the visual rendering of elements,and the data center module can flexibly detect network status,complete data statistics and output data reports.The unique identification and management of each node in the network are implemented by the factory module.5) As the engine of UltraStar, the discrete event simulation module uses discrete event simulation to simulate the flow of time in the simulation scene.Various events that symbolize physical changes are written into the discrete event list and triggered in order of time priority.The execution of an event will often cause more events to occur, that is, write more upcoming events and time points on the timeline.Finally, the network runs in a chain reaction way.

B.Mechanism of Discrete Event Simulation

Since it is difficult for the computer to simulate the continuous process, we use the DES [33]–[35] to simulate the flow of time in the simulation scene, as is shown in Fig.2.DES is composed of two closely related parts, timeline and event.The timeline represents the simulation event line, on which the upcoming events and their trigger time are recorded.Event is an abstraction used to describe the change of phenomena, and there are distinct implementation methods for different events.The event can be divided into two categories, the preset event and the triggered event.The former one is a type of event whose time resolution or other instance attributes need to be set before the simulation runs, while the latter one is triggered by other events during the simulation process.In our design,preset events include node mobility events and traffic events,and triggered events mainly include transmission events,channel propagation events and stack process events brought by complex communication protocols.

Fig.2.The design of discrete events.

In DES, discrete movements separated by preset time resolution are used to simulate the continuous process in reality.The node mobility event is to update the location and the adjacent link relationship for all nodes at every preset moment.In this way, the high dynamics of the network topology can be continuously maintained.Then, the traffic event generates data packets according to preset attributes, and arranges a stack process event for each of them in order.For the stack process event, it processes data packets and pushes them into the corresponding output cache.If there is only one packet in the queue, a transmission event will be arranged for it.Otherwise, the data packet has to queue up for arrangement.At the same time, another stack process event is arranged for the next packet in the input cache.Further, the transmission event completes the transmission of data packet, and arranges a propagation event for it.Meanwhile, another transmission event is arranged for the next packet in the output cache.Finally, the propagation event first judges whether to receive the arriving data packet, and then pushes the successfully received packet into the input cache associated with the link.If there is only one packet in the queue, a stack process event will be arranged for it.Otherwise, the data packet has to queue up for arrangement.

After the simulation runs, it jumps to the nearest time point recorded on the timeline and executes the event, and then keeps finding and executing next event until the end.The time point corresponding to the current executed event is regarded as the current simulation time.The execution of an event will often cause more events to occur, that is, write more upcoming events and time points on the timeline.In other words, the simulation runs the network in a chain reaction.Specifically, a data packet that has been processed by a stack process event will be pushed into the output cache and queue up for transmission.The simulation will write a transmission event on the timeline for the next data packet in the output cache, whose execution means that the data packet has experienced transmission delay and been pushed out of the cache.Then, the execution of the transmission event will write a propagation event on the timeline, whose execution means that the current data packet has arrived at the next node after the propagation delay and been judged by the physical layer on its receiving conditions.After the propagation event is executed, a stack process event will be written on the timeline, whose execution means that the packet has been processed by the protocol stack and pushed into the output cache for transmission.Basically in accordance with the above process, the data packet'condition is constantly changed in the network until it reaches the destination.

In this subsection, an introduction to event content and event flow is given.In the part of network module design, we will further clarify the entire process of the event definition and trigger mechanism in combination with specific attributes.

IV.MODULE DESIGN OF ULTRASTAR

In this section, we describe the design details of each module according to the order of network topology generation and maintenance, packet-level network simulation implementation, simulation control and visual rendering.

A.Generating a Ultra-Large-Scale LEO Network

UltraStar initializes the constellation topology according to configuration parameters, and then maintains satellite mobility based on orbit prediction models.In the following, the implementation details of the module design are introduced.

1) Physical Module:This module creates networking nodes and structures.Specifically, instantiated nodes are utilized to abstract real nodes in the network and different types of nodes(e.g., satellite, ground station) can be respectively simulated by different derived subclasses.In the process of node construction, the additional factory module is responsible for assigning the unique identification to each node and facilitating the node addition and deletion.Meanwhile, each node records some basic topology information, such as node ID,type, coordinate, etc.To obtain communication capability,network device instances are also added to each node, such as protocol stacks and interface caches, which will be discussed detailedly in the next sub-section.On the other hand, the network structure (e.g., satellite constellation) consisting of a large number of nodes is taken as the basic unit for constructing nodes according to configuration parameters and maintaining node coordinates through the mobility module.For the satellite network, the mobility maintaining is divided into two stages: in the topology initialization stage, the mobility module generates the initial satellite coordinates according to constellation configuration parameters; in the topology updating stage, it calculates the current satellite coordinates according to the orbit prediction model.Finally, discrete motion with an appropriate time resolution is utilized to simulate the continuous node movement.

We construct the Walker constellation by default, which belongs to the circular orbit constellation with even and symmetrical configuration [36]–[38], as shown in Fig.3.Generally, 5 parameters are used to uniquely represent the configuration characteristics of this type of constellation, usually denoted as (T,P,F,h,u), whereTrepresents the total number of satellites in the constellation,Prepresents the number of orbits in the constellation,Frepresents the phasing factor ranging from 0 to (P−1),hrepresents the orbital height, andμrepresents the orbit inclination.The phase offset between satellites in adjacent orbit planes is given by ∆f=2πF/T.In addition, according to the flying direction, all satellites can be divided into two categories, ascending satellites flying to the latitude-increasing direction, and descending satellites flying to the latitude-decreasing direction.

In the topology initialization stage, we initialize the constellation satellite coordinates according to the provided configuration parameters of Walker constellation.Firstly, we need to give the information of the first satellite as the premise of initialization, which is called the seed satellite.The orbit plane polar coordinates of one satellite can be expressed as (r,ALij),whererrepresents the satellite orbit radius,ALijrepresents the argument of latitude (AL) of thej-th satellite in thei-th orbit plane and AL refers to the angular distance between the orbiting satellite and the ascending node.According to the AL of the seed satelliteAL11and the phase offset ∆fbetween satellites in adjacent planes, the AL of the first satellite in thei-th orbit can be expressed as

Fig.3.Walker constellation and ascending and descending satellites.

Then, due to the evenly distribution of satellites in the same orbit plane, the phase difference between adjacent satellites can be given by ∆Φ=2π/(T/P), as shown in Fig.4.The AL of thej-th satellite on thei-th orbit can be expressed as

Fig.4.Constellation topology and ISLs.

whereμis the orbit inclination, Ωi=Ω1+(i−1)∆Ω is the right ascension of ascending node (RAAN) of thei-th orbit,∆Ω=2π/Pis the difference of the RAAN between adjacent orbit planes, and Ω1is the RAAN of the orbit where seed satellite is located.So far, we have accomplished the initialization of the complete constellation satellite coordinates.

In the topology updating stage, we provide an interface in the mobility module for the use of different orbit prediction models, based on which constellation satellite coordinates are updated over time.There are mainly three orbit prediction models considered in the mobility module, namely TwoBody[39], J4 perturbation [40] and simplified general perturbations(SGP4) [41].TwoBody is also called the Kepler motion model, which only considers the gravitational force of the earth on the satellite.J4 perturbation takes into account the long-term changes in satellite orbital elements caused by the non-sphericity of the Earth.As a more complex and accurate model, SGP4 considers secular and periodic variations due to non-sphericity of the earth, solar and lunar gravitational effects, gravitational resonance effects, and orbital decay using a drag model.Note that different orbit prediction models may have different effects on the communication network performance simulation.

Once the constellation is initialized, the satellite coordinates are updated with discrete time slots.The time slot of discrete motion is a preset parameter.The smaller it is, the finer the granularity of satellite coordinate updating becomes,which also brings more computing resources and time consumption.

2) Logical Module:This module maintains the link connection relationship between nodes in the network topology.

For the satellite network, the topology calculation [42], [43]includes the inter-satellite links (ISLs), inter-orbital links(IOLs) and user data links (UDLs).According to the information in the newly proposed constellation regulatory filings, we implement 4 ISLs for each satellite by default, that is, connect to adjacent satellites in the same orbit through laser links and connect to adjacent satellites in adjacent orbits through laser links.For IOLs, we calculate the coverage area based on the cone half-angle of the satellite to determine whether an interlayer connection can occur between two cross-layer satellites.In addition, each satellite communicates via radio uplink/downlink with a ground station with a sufficiently high ground elevation angle, where the range of the elevation angle is limited by the minimum ground elevation angle.A smaller minimum ground elevation angle allows the ground station to communicate with more satellites, while a larger minimum ground elevation angle setting will be more stringent on the construction of satellite-to-ground links.Once the discrete movement of the networking node occurs, the link connection relationship with adjacent nodes will be judged immediately,that is, whether to disconnect the old link or create a new link.

B.Building Package-Level Simulations

1) Network Module:This module contains basic representatives of the protocol stack.For ultra-large-scale network simulation, we simplify the network device into three parts: master protocol stack, interface cache and transceiver, as shown in Fig.5.Meanwhile, the accompanying channel model is also taken into account in the network module design.

Fig.5.The structure of UltraStar network module.

The master protocol stack module is mainly responsible for abstracting the network layer.Each master protocol stack instance maintains a unique IP address abstraction.Different network layer routing protocols can also be implemented in the stack.Once a new data packet arrives, a stack process event containing the information of processing delay and event execution method is inserted into the discrete event list.When a stack process event is executed, packet processing and routing is done.Then, the data packet is pushed into the cache associated with the output link and queues up for transmission.

The interface cache module abstracts the data link layer in the real network.The construction of each link will instantiate a cache on each linked node separately.Each cache instance maintains an MAC address abstraction.Meanwhile, a queuing model is built in the cache to simulate the queuing process of data packets.Different queuing models can be implemented according to different departure principles.Note that, newly arrived packets will be dropped when the queue is full.For the packet at the head of the queue, a transmission event is arranged for it, which will push the packet out of the queue after transmission delay.The packet transmission delay can be calculated by

wherepktrepresents the packet size andBis the bandwidth for transmission.

In order to make an appropriate choice on the trade-off between simulation time and accuracy, in the same way as some commonly used network simulators [1], [44], we make some simplified abstractions of physical layer [45], [46],where the smallest simulation data unit used is the packet.In the simulation, the packet is regarded as an indivisible unit,which is only received completely or not received at all.When the packet receiving power is greater than the receiving threshold, the packet is ready to be received, which is the packet receiving condition.The calculation of the receiving power is based on the channel loss and fading model, which can be changed according to different channel conditions.The success or rejection of packet reception depends on the probability value that depends on the SNR and the transmission mode (modulation mode, bit rate) [45].Using the packet as the basic simulation unit leads to a reduction in computational requirements, however, bit-level simulation requires more detailed model design and is computationally intensive.Once the packet is out of the cache, a propagation event is then arranged for it, which will push the packet into the interface cache at the other end of the link after the propagation delay.The propagation delay can be calculated by architecture to collect network information and flexibly interact with the data center module.The data center module collects network information through monitor instances and makes customized real-time data reports according to user requirements.At the same time, it conducts data interaction with MySQL to realize the functions of information storage and management.

D.Completing the Visual Rendering

wheredistrepresents the ISL length andcrepresents the speed of light.When perceiving the newly arrived data packet,the receiving end judges whether to receive it according to the judgment of the receiving condition and the calculation of the packet error rate.If the packet is successfully received, a stack progress event is then arranged for it.Otherwise, discard it.

2) Traffic Module:This module simulates the triggering of traffic on the specified node, that is, maintains the generation of data packets in the network.

The traffic module adds traffic events to the discrete event list for generating specific flow tasks, which is mainly defined by the flow type, flow size, preset start and end time, source and destination address.When the preset start time is reached,the traffic event starts generating data packets and pushes them into the network protocol stack of the source node for processing.Traffic events are continuously maintained until the flow task is completed or the preset end time is reached.The packet module maintains different types of packets,which are defined by corresponding fields and values.In general, different network services can be simulated in this module.

UltraStar calls the visualization module to realize the visual rendering of the target element.In order to obtain good interactivity, as shown in Fig.6, we use Qt to write the desktop application of UltraStar, and use Echarts to implement webbased display, which is a visualization library written in pure JavaScript.The visualization module receives data files in json format, and realizes the visual display of network topology, end-to-end paths, node categories and attributes, etc.Meanwhile, the interactive interface of the desktop application provides more convenient simulation control and interaction.

C.Simulation Control and Data Manager

In this part, we discuss the entire process of realizing the simulation through the external interface of UltraStar, and introduce its powerful data interaction capabilities with external databases.

1) Simulation Control Module:This module is the external functional interface of the software core, enabling the control of the simulation process.

The simulation control module realizes the simulation process control, including the basic simulation settings (e.g., simulation time, discrete time slot size), physical module configuration (e.g., constellation configuration, ground station coordinates), logical module configuration (e.g., minimum ground elevation angle), network module configuration (e.g., protocol type), traffic module configuration (e.g., traffic model),and log settings (e.g., log file address).

Fig.6.The desktop application of UltraStar.

2) Data Center Module:This module implements two functions: system status monitoring and data feedback.The content of system status monitoring includes topology information collection, service monitoring, packet tracking, etc.Then,the data feedback will generate data reports according to the user requirements.

As the cushion of the data center module, the monitor module is a more flexible module.Differentially implemented monitor modules can be placed anywhere in the software

V.NETWORK SIMULATION OF MEGA-CONSTELLATION

The space segment of SpaceX’s first-generation Starlink project consists of 11 927 LEO satellites, which aims to provide high-speed and reliable wireless ubiquitous network services to the world.As a typical Walker constellation with symmetrical and uniform configuration, it can apply UltraStar for performance evaluation.To the best of our knowledge, this is the first trial to simulate the ultra-large-scale LEO constellation consisting of more than 10 000 satellites.An authorized constellation configuration is shown in Table I.Since the information on very low earth orbit (VLEO) satellites has not been fully released, we make certain assumptions about it.Even if SpaceX will make changes to the details, this example is enough to prove UltraStar’s network simulation capabilities and demonstrate the superior characteristics of the megaconstellation.In addition, we make the following basic settings, set the time slot of discrete motion to 2 s, the link bandwidth to 1 Gbps, and the cache capacity to 100 MB.

TABLE I STARLINK CONSTELLATION CONFIGURATION PARAMETERS

A.Visualizing the Giant Network

The ultra-large-scale LEO satellite constellation is new and unfamiliar to most network researchers.Faced with different constellation configuration parameters, it is difficult to understand their differences without relying on the visualized results.Therefore, it is very important to build the visualization of some certain aspects to establish our good intuition of the expected behavior of the network, including changes in the constellation topology, satellite trajectory distribution, satellite distribution density, etc.We use UltraStar’s visualization module to visualize the network topology in different phases of Starlink, as shown in Fig.7.Different colors correspond to different orbital heights.Fig.7 shows that the first phase of the constellation achieves the coverage of mid-low latitudes,while the second phase extends the coverage to high latitude.The third phase further increases the scale of the constellation serving mid-low latitudes.

Fig.7.Starlink constellation visualization.

B.Ground Coverage Features

Due to the expansion of constellation scale and the difference of constellation configuration, the ground coverage will present different characteristics.

Fig.8 shows that the Phase 1 can basically cover the area between 58.0 degrees north and south latitude, where the global population is mainly distributed.The ground station at 49.0 degrees latitude can obtain an average of 10 satellites in line of sight (LoS) within the simulation time range of 500 s.The inclination angle of S2 is 53.8 degrees, and it still covers mid-low latitudes.Therefore, the number of satellites in LoS in mid-low latitudes has been greatly increased, but the coverage problem in high latitudes has not been resolved.The orbital inclination angles of S3–S5 are above 70.0 degrees,which greatly improves the coverage of high latitude.Ground stations in high latitudes can also have at least 14 average satellites in LoS.In Phase 3, a large number of VLEO satellites will be added, which will further increase the number of satellites in LoS in mid-low latitudes.The ground station at 50.0 degrees latitude can obtain an average of 58 satellites in LoS.

Fig.8.Ground coverage features of Starlink.

C.Examining End-to-End Performance

Firstly, for illustrating the great advantage of low latency provided by mega-constellation, we give the global RTT distribution, where New York is taken as the source.Then, the multipath characteristics are further discussed, which shows the great potential of using multiple network paths to improve transmission performance.Finally, the impact of link capacity on delay is also considered.

1) Global RTT Distribution:For simplicity, we only consider the final Phase 3 here, which contains 11 927 satellites.Fig.9 presents the global RTT distribution with New York as the source.The constellation in finally phase can provide not only seamless end-to-end connections around the world, but also extremely low latency.For example, the connection from New York to any place in the world can be realized within 200.0 ms (e.g., RTT from New York to Sydney is 147.1 ms).The transatlantic connection from New York can be realized within 80.0 ms (e.g., RTT from New York to Moscow is 74.5 ms).The connection from New York to most parts of North America can be realized within 50.0 ms (e.g., RTT from New York to Los Angeles is 41.2 ms).It is worth noting that the global distribution of RTT is in the shape of a ring, the reason for which is closely related to the uniform and symmetrical constellation configuration.

Fig.9.Global RTT distribution from New York.

2) Multipath:The bandwidth of a single path in the satellite network is insufficient to affect the status of terrestrial optical fiber in the long-distance transmission.Note that, multipath is receiving extensive attention in the research of satellite network because of its great advantage of using multiple network paths to improve the transmission delay and reliability.Therefore, it is meaningful to explore the great potential of multipath in the mega-constellation.

We use UltraStar to iteratively calculate 15 disjoint end-toend paths from New York to Beijing, e.g., P1−P15 in Fig.10(a).Here P1 is the optimal path generated by the Dijkstra algorithm, P2 is the path generated by the Dijkstra algorithm after deleting links in P1, and so on.Moreover, we further compare RTTs of these 15 paths with the ideal terrestrial optical fiber RTT and the real Internet RTT.The ideal terrestrial optical fiber RTT refers to the RTT in the shortest optical fiber path laid along the surface of the earth between two locations.In Phase 1, RTTs on the optimal and all sub-optimal paths are higher than the ideal terrestrial optical fiber RTT, and most of them are lower than the real Internet RTT.Due to the reduction of optional links for routing, RTTs on some sub-optimal paths exceed the real Internet RTT and show a larger range of fluctuations.For different phases, we give the average RTT of each priority path within 500 s, as shown in Fig.10(b).With the expansion of the constellation scale in Phase 2, more optional links are provided, which brings great improvements to the RTT performance.The deployment of a large number of VLEO satellites in the third phase brings end-to-end RTT performance comparable to the ideal terrestrial optical fiber and far lower than today’s real Internet RTT.The more stable delay fluctuations also indicate that the mega-constellation network has great potential to meet quality-of-service (QoS)requirements of future delay-sensitive applications.

Fig.10.RTT performance evaluation of multipath.

Fig.11.Influence of link capacity on network performance.

3) Link Capacity:Link capacity is an important factor concerned by network operators and users, which will directly affect the latency experience and throughput capacity.How to improve the network performance through the reasonable allocation and utilization of limited network resources has become a focus issue.As a preliminary exploration, we evaluate the end-to-end performance under different settings of link capacity, as shown in Fig.11.In Phase 1, due to frequent intersatellite hopping caused by dense satellite distribution, delay performance is greatly affected by transmission and congestion condition.So, when link capacity increases, RTT can be significantly reduced.With the expansion of constellation scale, better path options reduce the impact of multi hops on transmission.Therefore, there is only slightly improved performance in Phases 2 and 3.In conclusion, the increase of link capacity can greatly promote end-to-end performance, especially for long-distance multihop routes.

VI.A CONSTELLATION-WIDE VIEW

In this section, extensive simulation results are presented to provide insight into network dynamics, including RTT fluctuations, path structure changes and link utilization fluctuations.Typically, we consider a network-wide business model, where 200 cities distributed around the world are randomly paired to generate end-to-end connections.Among them, a source may have multiple destinations, and corresponding traffic events are set respectively.

A.Path Changes

How will the high dynamics of the network topology affect the end-to-end path structure changes? Different settings of satellite orbit height and ground minimum elevation angle bring different satellite visibility characteristics.The visible duration of a satellite to a ground station is relatively short(generally tens of seconds to hundreds of seconds), which inevitably leads to frequent switching of ground-to-satellite connections.At the same time, the relative movement between satellites of different orbital heights also causes frequent changes in inter-orbital links.The above phenomena provide a large number of frequently changed link options for end-toend routing.Thus, it is necessary to explore the impact of frequent changes in path structure on network performance.

For these source-destination pairs, Fig.12 shows the cumulative distribution function (CDF) of the number of changes in the path structure and the CDF of the path hop-count differences.When the end-to-end path structure in the current time slot is different from that in the previous time slot, it is counted as one change.Fig.12(a) shows that from the first phase to the third phase, as the scale of the constellation continues to expand, all end-to-end path structures change more and more frequently.The highly dynamic network is one of the reasons for frequent changes.At the same time, the expansion of the constellation scale also provides more optional nodes and links for end-to-end routing, thereby increasing the path variability.

For a connection, the path hop-count difference refers to the difference between the maximum hops and the minimum hops within the entire simulation time.As indicated in Fig.12(b), a large difference in path hop-count occurs in Phase 1, where pairs with a difference of more than 10 hops accounted for 46.0%.The high dynamics of the network topology is one of the reasons.On the other hand, since the minimum ground elevation angle is set to 40.0 degrees in our simulation, the satellites available to the ground station are limited in Phase 1,which leads to the problem of inconsistency between the satellite orbital direction and the source-destination direction.When the orbital direction of the optional satellite is consistent with the source-destination direction, the main direction of the path is along the orbital direction and the relay satellites mainly exist in the same orbit.When directions are inconsistent, there will be more cross-orbit transmission in the path.Note that, the closer the ground station is to the latitude boundary of constellation coverage, the denser the orbit distribution becomes, and the more frequent cross-orbit transmissions may occur.Therefore, when either the source point or the destination point have a higher latitude, the path structure may change significantly.With the continuous expansion of the constellation scale, the ground station will have more optional satellites in the orbit direction consistent with the destination direction, so the large difference in the path hopcount can be effectively avoided.

Fig.12.Path changes and hop-count differences.

B.RTT Fluctuations

Fig.13 shows the CDF of the maximum RTT and the RTT fluctuation range.In Phase 1, the maximum RTTs of all endto-end connections range from 24.3 ms to 313.8 ms.Meanwhile, due to path changes caused by high dynamics, RTT also shows large fluctuations, where connections with a fluctuation range of more than 50.0 ms account for 49.6%.As analyzed in the last section, when the orbital direction of the optional satellite is inconsistent with the destination direction,there will be more cross-orbit transmissions.Once the latitude of the ground station approaches the constellation’s latitude coverage boundary, the cross-orbit transmission will become more frequent.Thus, considering the increase in cumulative on-board processing and transmission delay, RTT may fluctuate significantly.In Phase 2, the maximum RTT of all end-toend connections ranges from 21.0 ms to 182.7 ms, where the long tail becomes shorter.Large delay connections in the first phase have been improved, and the RTT fluctuation range of all connections is only 15.4 ms.Combining with the discussion on the path structure in the last section, we can see that the increasingly superior characteristics are at the expense of the expanded constellation scale and the more frequent path structure changes.Although a large number of VLEO satellites added in the third phase only have limited improvements in delay and fluctuation, the gain they bring to the entire constellation network capacity will be huge.In conclusion, once the mega-constellation infrastructure is fully implemented, the delay and stability performance comparable to terrestrial ideal fiber will bring a new round of information revolution.

Fig.13.Maximum RTT and differences.

C.Link Utilization

Frequent path structure changes will bring considerable fluctuations in link utilization.At the same time, an end-toend connection may share the link bandwidth on its path with cross traffic from other end-to-end connections in the network.We do not simply analyze the bandwidth utilization of a single link in the network, but analyze the change in the link utilization of an end-to-end connection over time.The link utilization here refers to the bandwidth utilization of the busiest link on this end-to-end path.We count the link utilization fluctuation for the connection from New Delhi to San Francisco during the simulation time, as shown in Fig.14.

Fig.14.Link utilization from New Delhi to San Francisco.

Under the influence of cross-traffic interaction, the link utilization of the New Delhi-San Francisco connection in the first phase fluctuates greatly.For example, due to the addition of burst traffic, the link utilization rises from 58.6% to 87.9%at 128 s.From 136 s to 148 s, the link utilization drops to 0.The reason is that the ground station lacks satellites in LoS,which causes the end-to-end connection disconnected.From 156 s to 160 s, the link utilization fluctuates at 100%.The full bandwidth usage of the link means that data packet queuing may occur.After 189 s, due to changes in the end-to-end path structure and reduced cross-traffic impact, the link utilization on this end-to-end path has dropped significantly in a short period of time, which is only 31.1% at 200 s.In Phase 2, the expanded constellation scale increases the network capacity while providing more optional links for end-to-end routing.This reduces the impact of cross-traffic on this end-to-end connection, resulting in lower link utilization and reduced fluctuations.In addition, the problem of connection interruption between 136 s and 148 s has also been resolved, which shows that the global coverage is ensured.In Phase 3, new routes largely disperse the cross-traffic, but also generate new cross-traffic at the same time.Compared with the results in the second phase, the link utilization is further reduced as a whole, but it is slightly higher sometimes.Meanwhile, the reduced impact of cross-traffic also brings smaller link utilization fluctuation.

D.Influence of Orbit Prediction on Network Performance

For constellation networking, the design and selection of orbit prediction model may affect the final simulation results.What is worse, inaccurate prediction of satellite coordinates will directly affect the constellation topology and ground coverage characteristics, making the measurement of network performance unreliable.In UltraStar, once accomplishing the generation of initial constellation topology, physical module maintains the mobility through the orbit prediction model interface, where different models can be considered (e.g.,TwoBody, J4, SGP4).In order to explore their different impacts on network performance, we focus on evaluating the difference in end-to-end latency, which is strongly dependent on network topology and satellite spatial coordinates.Specifically, network-wide connections’RTTs are tracked separately,and then the average, maximum and minimum relative errors in each time slot are counted, as shown in Fig.15.Note that,the more accurate model is used as the measured object in the error calculation.It can be seen that the difference in performance within 3 hours is extremely small and the maximum relative error in both comparisons is less than 1.0%.In addition, there is a larger error fluctuation in the comparison between TwoBody and SGP4.Since SGP4 considers more spatial influence factors, the satellite coordinates prediction is more likely to deviate from the ideal TwoBody.In general,within a short simulation time (e.g., several hours), the selection of orbit prediction model has little impact on the constellation networking and performance results.Certainly, more complicated models is a prerequisite for tasks with high-precision or long-term requirements.However, especially for simulation scenarios with large constellation scale and frequent topology updating, the computational overhead of high-precision orbit prediction models cannot be ignored.In UltraStar,the orbit prediction model interface is set up to support different forms of mobility maintenance, such as the realization of various prediction modules and the import of external orbit data, so as to meet different requirements of simulation tasks.

Fig.15.Relative error of delay under different orbit prediction models.

VII.CONCLUSION

In this paper, we have developed UltraStar, a lightweight network simulator, which aims to facilitate the complicated simulation for the emerging mega-constellation.Particularly, a systematic and extensible architecture has been proposed,where the joint requirement for network simulation, quantitative evaluation, data statistics and visualization is fully considered.For characterizing the network, we have made lightweight abstractions of physical entities and models, which contain basic representatives of networking nodes, structures and protocol stacks.Then, to consider the high dynamics of Walker constellations, we have given a two-stage topology maintenance method for constellation initialization and orbit prediction.Further, based on discrete event simulation theory,a new set of discrete events has been specifically designed for basic network processes, so as to maintain network state changes over time.Finally, taking the first-generation Starlink of 11 927 LEO satellites as an example, we have provided a complete visualization of this mega-constellation for establishing a good intuition.To demonstrate the great potential of the constellation, we have fully evaluated basic network characteristics at different deployment stages.In a simulated scenario of network-wide business, extensive simulation results have been further presented to provide insight into network dynamics, including RTT fluctuations, path structure changes and link utilization fluctuations.The simulation results have demonstrated not only the mega-constellation’s superior performance, but also the effectiveness of UltraStar.For the future works, we will develop and evaluate more satellite communication protocols and carry out more simulations in the 6G oriented space-air-ground networks.