The First Verification Test of Space-Ground Collaborative Intelligence via Cloud-Native Satellites

Wang Shangguang ,Zhang Qiyang ,Xing Ruolin ,Qi Fei ,Xu Mengwei,*

1 State Key Laboratory of Networking and Switching,Beijing University of Posts and Telecommunications,Beijing 100876,China

2 Star Network and Intelligent Computing Laboratory,Beiyou Shenzhen Institute,Shenzhen 518000,China

3 Huawei Technologies Co.,Ltd,Shenzhen 518129,China

Abstract: Recent advancements in satellite technologies and the declining cost of access to space have led to the emergence of large satellite constellations in Low Earth Orbit (LEO).However,these constellations often rely on bent-pipe architecture,resulting in high communication costs.Existing onboard inference architectures suffer from limitations in terms of low accuracy and inflexibility in the deployment and management of in-orbit applications.To address these challenges,we propose a cloud-native-based satellite design specifcially tailored for Earth Observation tasks,enabling diverse computing paradigms.In this work,we present a case study of a satellite-ground collaborative inference system deployed in the Tiansuan constellation,demonstrating a remarkable 50% accuracy improvement and a substantial 90% data reduction.Our work sheds light on in-orbit energy,where in-orbit computing accounts for 17% of the total onboard energy consumption.Our approach represents a signifciant advancement of cloud-native satellite,aiming to enhance the accuracy of in-orbit computing while simultaneously reducing communication cost.

Keywords: cloud-native satellite;orbital edge computing;satellite inference;verifciation test

I.INTRODUCTION

With the evolution of next-generation communication networks and advanced technologies,the number of end-user connections and application requirements are increasing.It is estimated that two-thirds of the population in the world will have access to the Internet and billions of devices will be connected to the Internet by 2023 [1].However,there are still more than 80% of the land and 90% of the ocean on the Earth without network coverage.Moreover,real-time and high-resolution satellite imagery have vital use in getting along with nature.The terrestrial network is extremely vulnerable in the face of natural disasters,such as earthquakes and floods.

In recent years,however,the increasing investments in satellite-related technologies make the Low Earth Orbit (LEO) satellite constellation a strong complement to terrestrial networks [2].Numerous competitors have disclosed efforts to deploy LEO constellations,including SpaceX[3],Telesat[4],and Amazon[5].These LEO constellations are promising to blanket the globe with low-latency broadband Internet due to the declining costs of satellite manufacturing and launching today.Satellites are typically categorized into distinct types,each specifcially designed to fulflil various tasks such as communication,guidance,and sensing.In this work,we focus on the Earth Observation satellites.

Since bent-pipe satellites do not have in-orbit processing capabilities,raw data is completely transmitted to the ground station,incurring a communication cost [6].The architecture breaks down due to the limitations on downlink availability.Compression is useful because it reduces the resources required to store and transmit data.However,computational resources are consumed in compression and decompression.Orbital Edge Computing (OEC) is proposed to reduce the downlink transmission by placing computing resources at the LEO satellites constellation[7].A promising cloud-native satellite operates by capturing an image and processing it locally instead of transmitting it to the ground completely.Though the satellite with computing capability has been developed [8,9],satellite-ground collaborative intelligence is still required as follows: it is diffciult for satellites to perform complex tasks independently,despite their limited computing capability;the models are one-size-ftis-all for all scenarios because satellite data are inconsistently in spatial and temporal distribution.Based on these reasons,satellite-ground collaborative intelligence still plays a vital role now.

The traditional LEO satellite constellations supplement the terrestrial network and support 5G/6G communication,etc [10].The satellites are customized for specifci tasks,which will inevitably lead to the incompatibility of software and hardware.For this issue,domestic and foreign satellite manufacturers such as SmartSat [11] in the United States and TIANZHI 1 [12] in China propose software-defnied satellites.Software-defnied satellites support plug-and-play payloads,and the application software is loaded on demand to flexibly adapt to multi-tasking and multiuser scenarios [13].However,it is still challenging to meet the demand of real-time deployment and continuous update of in-orbit services.We frist introduce cloud-native technology to satellites to deal with these challenges,because cloud-native technology with container and microservice empowers satellites to build and run the scalable application in a modern,dynamic environment.These techniques enable loosely coupled systems that are resilient,manageable,and observable.Motivated by advanced techniques,cloudnative satellites address AI tasks through distributed cloud-edge collaboration paradigms (e.g.,collaborative inference and federated learning,etc) to reduce deployment cost,improve performance and protect privacy.

Cloud-native satellite technologies show great promise for in-orbit AI applications when the realworld deployment brings forth a set of unique challenges,as shown in Figure 1.One such challenge is the lack of flexibility in existing onboard inference architectures,which hinders the effciient deployment and management of in-orbit applications.Additionally,these architectures suffer from low accuracy levels that fail to meet the demanding requirements of such applications.To address these challenges,we propose a cloud-native-based collaborative intelligence architecture designed to cater to various in-orbit AI applications,including CNN-based image classifciation,object detection,and segmentation.By adopting this architecture,we aim to enhance the adaptability and accuracy of in-orbit AI applications,paving the way for their successful implementation in a real-world context.

Figure 1.The framework of in-orbit AI applications.Cloud-native satellite can support diverse tasks.

We introduce a case study of satellite-ground collaborative inference system deployed in the experimental satellites of Tiansuan constellation,Baoyun and Chuangxingleishen.Our system has yielded a remarkable 50%improvement in in-orbit inference accuracy and a substantial 90%reduction in the amount of data returned by the satellite.However,in-orbit processing must be compatible with the space environment,which poses challenges due to the higher cost and lower capabilities of space-based systems compared to corresponding COTS units on the ground.Furthermore,we report our fnidings regarding the energy consumption of in-orbit computing,which accounts for approximately 17% of overall energy consumption.These observations have important implications for future satellite designs and energy management strategies.Our work represents a signifciant step towards realizing the potential of cloud-native satellites for improving the effciiency and accuracy of in-orbit computing.

The main contributions of this paper as follows:

1) We propose the concept of cloud-native satellites to cater to in-orbit computing and facilitate promising applications.

2) We present a case study of a satellite-ground collaborative inference system deployed in Tiansuan constellation.

3) We report that the inference system not only enhances the accuracy of in-orbit computing but also effectively reduces communication costs.Additionally,we provide insights into the energy consumption associated with in-orbit computing.

The rest of this paper is organized as follows.Section II introduces the background and motivation of the paper.Section III describes the overall system architecture and main components.Then Section IV details how the satellite-cloud collaborative inference works.Finally,Section V concludes the paper and discusses the potential directions.

II.BACKGROUND&MOTIVATION

Satellite data is exploding.It’s reported that about 45% of the LEO satellites in orbit are used for earth observation.The data amount obtained by onboard cameras is too large and 60% of remote sensing images are highly similar in the same scenario [14].For example,ZY-3 is the Chinese frist civilian highprecision cartographic satellite,and it generates more than 10 of TB data every day[15].One single remote sensing image dataset can reach tens of GB[16].Such large data transmission to the ground station may take up most of the time and induce huge communication overhead under the bent-pipe architecture[17].It is a fact that not all raw observations are worth exploring further.Especially in the southwest of China,80%-90% of raw data is invalid due to cloud cover.There has been preliminary research in the feild of onboard image preprocessing for remote sensing satellites.Before satellites backhaul,redundant information such as cloud cover area can be eliminated in advance and the data returned can be greatly reduced.

Downlinks can be unreliable.Existing systems under a bent-pipe architecture stem from fundamental physical constraints simply downlink raw observations to ground stations.Surprisingly,one satellite task lost 80% of its data packets due to downlink instability [18].The time-varying relationship between the orbital position of the satellite and the geographic location of ground stations imposes limitations on link availability and can incur high downlink latency between data collection and processing.Any viable orbital system must directly address the unique physical constraints.As an alternative paradigm,OEC colocates processing hardware with high-datarate sensors in low-cost satellites.

Onboard processing is on the rise.With the development of onboard chip integration,the processing performance of satellites has been improved.Onboard systems have integrated multiple computing modules such as CPUs,GPUs,DSPs and FPGAs [19].Different from traditional CPUs,DSPs and GPUs have stronger computing capabilities.For example,based on embedded cluster computing,the next-generation onboard computer makes a system architecture that combines the master nodes (e.g.,CPUs) and slave nodes(e.g.,GPUs,FPGAs and DSPs)for parallel processing [20].These processing units can work together to provide higher onboard computing and improve reconfgiuration capabilities.In a word,the enhancement of onboard computing processing capability lays the foundation for OEC.

Enhancing OEC Landing with Cloud-Native Technology.While computing resources at the LEO satellite constellation enables OEC,meeting the demand for flexible deployment in orbit remains a signifciant challenge.Cloud-native techniques with container and microservice empowers satellites to build and run scalable applications in a modern,dynamic environment.These techniques facilitate the development of loosely coupled systems that exhibit resilience,manageability,and observability.Cloudnative satellite overcomes challenges for unprecedented success in real-world deployment.

Previous works related to this topic in the network/system community have been proposed.Denby et al.[6] propose an orbital edge computing architecture on satellite constellations to reduce system edge processing latency.Vasisht et al.[21] present a scheduler for multi-satellite,multi-ground station confgiuration to reduce data downlink latency.Jinhyun et al.[22] focus on the federated learning framework,which dynamically schedules model aggregation based on the deterministic and time-varying connectivity according to satellite orbits.[23,24] also present challenges of space-air-ground integrated network architecture.[25,26] discuss opportunities and potential application areas for intelligence in space systems.Our work is orthogonal to and compatible with those above works.To our knowledge,we present a novel approach to achieve high inference accuracy through satellite-ground collaborative intelligence using cloud-native satellites.Our work marks the effort in harnessing the full potential of cloud-native satellite,thereby paving the way for enhanced effciiency and accuracy in in-orbit computing.

III.DESIGN OF CLOUD-NATIVE SATELLITES

With the expansion of LEO constellations and the explosive growth of satellite data,it is urgent to explore how to effectively utilize local satellite data.Additionally,local processing of satellite data is promising as AI capabilities migrate from the cloud to edge nodes[27].However,each satellite has limited computing capability and energy resources,making it diffciult to perform complex tasks independently.Satelliteground collaboration is proposed to facilitate intelligent in-orbit computing.Also,motivated by cloudnative,we frist provide an overview of an individual cloud-native satellite.We then describe key techniques involved in cloud-native satellite,which organizes a system that supports in-orbit computing.

3.1 Cloud-Native Satellite

As the frist cloud-native satellite in the world,the satellite leverages advanced techniques such as containers and microservices to realize in-orbit computing.As shown in Figure 2,we integrate communication,computing,and storage capabilities into cloudnative satellites.The satellite provides open interfaces that allow third parties to deploy applications.The satellite also monitors and manages the operational status and applications.Based on the storage capacity,computing module provides onboard control capabilities for basic and third-party ubiquitous applications.Cloud-native satellites guarantee in-orbit service execution that mainly beneftis from:

Figure 2.The design of the cloud-native satellite. The communication,computing,and storage capabilities are integrated into the cloud-native satellite.

Rapid deploymentCloud-native satellite leverages agile development and DevOps project management models [28].It not only enables users to deploy applications quickly and automatically but also continuously updates onboard applications to keep up with changes in demand.

Elastic expansionThe satellite manages satelliteground clusters by automatically increasing or decreasing edge nodes.It also shrinks or expands resources according to the real-world situation to improve the utilization of onboard resources.

Highly autonomousBased on container technology,cloud-native satellite encapsulates service functions to achieve isolation and abstraction independent of location and environment.Moreover,EdgeMesh,as the data plane component of the KubeEdge cluster,provides simple service discovery and traffci proxy functions for satellite service,thereby shielding the complex network structure in edge scenarios[29].

Safe and reliableContainer orchestration is crucial for large and dynamic production environments [8].With container orchestration for microservices,onboard applications can be automatically scaled,faulttolerant which copes with the complex environment of space and keeps onboard applications available at all times.

The emergence of cloud-native satellites beneftis from the application management framework in which satellites and the ground cloud work together.KubeEdge [30] and its sub-project Sedna [31] provide unifeid resource and application management for cloud-native satellites and improve application collaboration capabilities.The system architecture is shown in Figure 3.Collaborative AI framework breaks model training and inference space constraints.In the following,we mainly introduce the two major frameworks:KubeEdge and Sedna.

Figure 3.The workflow of satellite-ground collaborative inference system architecture.

3.2 KubeEdge

KubeEdge is an open-source platform based on Kubernetes that provides infrastructure support for applications by extending containerized application orchestration to satellites[30].KubeEdge also synchronizes deployments and metadata between central cloud and satellite,which strongly supports:

Extremely lightweightThe global resources of the satellites are discrete and the local resources are limited to one satellite.The instability of the edge network leads to frequent disconnection of edge satellites and management systems.The platform extracts the core functions of Kubelet and builds a lightweight management software named EdgeCore to cover different scenarios.EdgeCore and CloudCore in the central cloud provide data centers to meet exacting requirements without compromise[32].

Reliable connectionThe network between satellites and ground station often suffers from low bandwidth and serious packet loss.The platform manages edge-cloud messages in the same way,and the data is still reliably transmitted in weak network scenarios.

Offline autonomousA lightweight management component named MetaManager stores metadata[33].When edge nodes go offline,applications are managed and restored based on storage metadata.This method not only brings less resource overhead but also ensures the security of other satellite nodes in the cluster.

Cloud-edge collaborationEdge and cloud always work together to achieve end-to-end applications.KubeEdge provides an edge-cloud data exchange mode.More importantly,EdgeMesh provides unifeid service discovery and traffci proxying between microservices.For example,the capabilities of EdgeMesh-Server are merged into the tunnel module of EdgeMesh-Agent,so that EdgeMesh-Agent with relay capability can automatically become a relay server,providing other nodes with the functions of assisting hole punching and relaying.

3.3 Sedna

Sedna’s edge-cloud synergy is implemented for collaborative inference based on the capabilities provided by KubeEdge.The component Global Manager in Figure 3 unifeis satellite-ground synergy AI task management and collaboration by well-supporting current popular frameworks including TensorFlow/ Pytorch/PaddlePaddle/ MindSpore,etc [31].Sedna helps AI applications migrate seamlessly to satellites by using cloud-edge collaboration protocols to reduce cost,improve model performance,and protect data privacy,etc.In the following section,we discuss the major components of Sedna.

Global ManagerThis component implements an edge AI controller based on Kubernetes.Users create CRD [34] to achieve model/dataset management,AI task management for edge-cloud collaboration,and edge-cloud coordination.

Local ControllerThis component performs local process control of edge-cloud collaborative AI tasks.Moreover,it also performs local general management:models,datasets,state synchronization of AI tasks,etc.

WorkerThis component performs AI tasks based on the training/ inference procedures of existing AI frameworks.Workers can be deployed on the edge or in the cloud and they work together with each other.

LibThis component provides an edge-cloud collaboration interface for applications.Users can implement training/inference and aggregation based on this Lib.

3.4 Promising Application Scenarios

To address different tasks and challenges,cloud-native satellites can support of diverse paradigms.Currently supported computing protocols are as follows:

Collaborative inferenceSatellites cannot independently perform complex inference tasks because of limited resources and weak computing capability on the satellite.The satellite performs one part of the whole inference and offloads some tasks to the cloud to improve the overall inference performance.

Incremental trainingChanges in data distribution,such as weather,make the deployed model unable to adapt to the current environment and the detection accuracy decreases.For this challenge,satellites continuously collect newly generated data and train models in the cloud.The satellite nodes regularly fnie-tune the model from the cloud to improve accuracy.

Federated learningFrom the perspective of privacy protection,federated learning can effectively alleviate data silos because raw data is reluctant to share to the cloud.The satellite trains the model and transmits the parameters(i.e.,training weights)to the cloud responsible for parameter aggregation.The system also provides a secure data channel through KubeEdge and supports encrypted transmission to ensure data security.Such a protocol may train high-precision models for privacy protection.

Lifelong learningSatellites suffer from data drift and catastrophic forgetting of onboard models.Combining incremental training and multi-task training,satellite model enables knowledge transfer across time and scenarios.Based on the knowledge library in the cloud,satellite model can be continuously updated to address unknown tasks.

IV.CASE STUDY:THE FIRST IN-ORBIT AI COLLABORATIVE INFERENCE

We’ve successfully launched two new satellites,Baoyun and Chuangxingleishen,and implemented collaborative inference on these satellites using KuebeEdge and Sedna.Earth Observation satellites incorporate cloud-native computing capabilities.Table 1 shows the real-world specifciations including vital parameters such as mass,size for both satellites.Additionally,Figure 4 provides a physical appearance of Baoyun satellite.It’s worth noting that these satellites adhere to the CubeSat standard for design.The control center operates on a Linux server,sending remote instructions to the satellites.We’ve deployed two different precision detection models,consisting of lightweight,low-precision models onboard the satellite for preprocessing to obtain preliminary results.On the ground,large,high-precision models are situated with high computing capabilities,which facilitate accurate detection.The workflow of our collaborative inference system is shown in Figure 5.The satellites perform a preliminary detection using a lightweight model to identify features of interest.If confdience threshold in the results is high,the processed results are sent back to the ground directly.However,if confdience threshold is low,the satellite transmits the images to the ground,where the high-precision detection model is used for exact detection.This system enables effciient detection of captivating images and information,ensuring that only essential data is transmitted to the ground.Our experiments are based on the fact that onboard inference can be performed anytime,when satellite computation are available.The handover between them only occurs during the contact time between the satellite and the ground.Note that the shown evaluations are conducted in Baoyun satellite.Interestingly,the experimental results on different satellites show that the inference system can achieve consistent results on satellites of different specifciations.

Table 1.Satellite platform specifications.

Figure 4.A physical appearance of the Baoyun satellite.

Figure 5.The workflow of satellite-ground collaborative inference system architecture.

Onboard image splitting signifciantly reduces the occurrence of redundant images caused by cloud cover.We conducted experiments on the widely-used DOTA [35],object detection dataset.We propose a strategy to split the images into smaller images before performing in-orbit inference.This approach is necessary due to the limited computing power of the satellite,which cannot handle high-resolution images.Figure 6 illustrates the reduction in redundant images during in-orbit operations.We observe that splitting the large images into smaller fragments,irrespective of the fragment size,leads to a remarkable 90%and 40%decrease in images for the two versions of the datasets,respectively.As a result,in-orbit processing approach offers the dual beneftis of reducing the number of images and conserving signifciant bandwidth.Collaborative inference yields a signifciant improvement in accuracy.Our satellite-ground collaborative inference system incorporates YOLOv3-tiny and YOLOv3 object detection models for onboard and ground detection,respectively.To assess the accuracy of our system,we use the mean average precision (mAP) metric,which compares ground-truth bounding boxes to detected boxes and returns a score.A higher score indicates more accurate detection[36].We analyzed the accuracy of our system through in-orbit and collabora-tive inference methods,as shown in Figure 7.The results demonstrate a remarkable accuracy enhancement of 44%and 52%respectively when employing collaborative inference compared to in-orbit inference.On average,our collaborative inference system achieves an approximate 50% improvement in accuracy.Additionally,a signifciant number of redundant images were detected.By performing preprocessing onboard and directly transmitting the inference results to the ground station,our system successfully reduced the amount of data returned by 90%.

Figure 6.The filter rate of redundant data in orbit on DOTA.

Figure 7.Accuracy (mAP in object detection task) of inorbit vs. collaborative inference.

According to the energy consumption analysis presented in Table 2,Baoyun satellite’s energy system primarily powers its payloads,accounting for approximately 53%of the total energy consumption.The remaining energy is utilized for basic satellite functions such as communication and propulsion.Further delving into the energy consumption of the satellite’s payloads,Onboard equipment measures the voltage and current of each power system and records the telemetry data,which is then transmitted to the ground.The reported power consumption is recalculated based on the collected data.Table 3 presents a breakdown of the subsystems responsible for satellite computing and their corresponding power consumption,including the camera that captures remote sensing images.Among these power subsystems,the Raspberry Pi system accounts for the largest proportion of energy consumption,representing 33% of the total energy consumed by the payloads.In summary,computing on the Baoyun satellite accounts for about 17% of energy consumption.The presented data sheds light on the energy allocation and consumption patterns of the satellite,offering value for optimizing its operational effciiency and ensuring its long-term sustainability.

Table 2.Real power distribution of energy consumption system in Baoyun satellite.

Table 3.The power of payloads subsystem of Baoyun satellite.

V.CONCLUSION

This paper introduces the emergence of a cloud-native satellite that directly leverages onboard computering for remote sensing tasks.It explores various execution protocols such as collaborative inference,federated learning,incremental learning,and lifelong learning.Additionally,we launched the frist cloud-native satellite and implemented the collaborative inference system on the satellite,with the ultimate objective of achieving high inference accuracy through the use ofcloud-native satellites.

In the future,we plan to deploy this project in batches across other satellites of Tiansuan constellation,thereby establishing a collaborative computing network in space.We also expect cloud-native satellites to unlock new capabilities in precision agriculture,infrastructure monitoring,humanitarian assistance,etc[37].

VI.ACKNOWLEDGEMENT

This work was supported by National Natural Science Foundation of China(62032003).