Project: Spacecraft Pose Networks for Vision-based Relative Navigation

The on-board estimation and tracking of the pose (i.e., position and orientation) of a target Resident Space Object (RSO) is a key enabling technology for various on-orbit servicing and active debris removal missions. In these missions, real-time information about the target’s pose with respect to the servicer spacecraft is required to plan and execute safe, autonomous and fuel-efficient rendezvous and docking trajectories. Extracting pose from a single or a sequence of images captured with a low Size-Weight-Power-Cost (SWaP-C) sensor such as a monocular camera is especially attractive in comparison to more complex sensor systems such as Light Detection and Ranging (LiDAR) or stereovision.

This research aims to develop robust and efficient Machine Learning (ML) models to extract the target’s pose from a single or sequence of 2D images captured in space. For a known non-cooperative spacecraft, which is representative of scenarios such as on-orbit servicing, a series of Convolutional Neural Networks (CNN) named Spacecraft Pose Network (SPN) are developed for pose estimation of the Tango spacecraft from the PRISMA mission. Thanks to benchmark datasets such as SPEED and SPEED+, SPN models can be trained on synthetic images and tested on on-ground images of a mockup satellite to quantify their robustness across domain gap. The SPN model is also seamlessly integrated into an Unscented Kalman Filter (UFK) to enable robust pose tracking of the target spacecraft in various rendezvous scenarios. Finally, in order to fully close the domain gap, an Online Supervised Training (OST) algorithm is proposed to further train SPN on-board the spacecraft avionics using in-coming spaceborne images of the known target with minimal computational overhead.

This research will extend in the future to address simultaneous shape characterization and pose tracking of an unknown RSO.

Related Publications

Park, T. H., D'Amico, S.;  
Online Supervised Training of Spaceborne Vision during Proximity Operations using Adaptive Kalman Filtering;
2024 IEEE International Conference on Robotics and Automation (ICRA), Yokohama, Japan, 2024.

Park, T. H., D’Amico, S.;
Adaptive Neural-Network-Based Unscented Kalman Filter for Robust Pose Tracking of Noncooperative Spacecraft;  
Journal of Guidance, Control, and Dynamics, Vol. 46, No. 9, pp. 1671-1688 (2023). DOI: 10.2514/1.G007387

Park, T. H., D’Amico, S.;
Robust Multi-Task Learning and Online Refinement for Spacecraft Pose Estimation across Domain Gap;
Advances in Space Research (2023).

Pasqualetto Cassinis, L., Park, T. H., Stacey, N., D’Amico, S., Menicucci, A., Gill, E., Ahrns, I., Sanchez-Gestido, M.;
Leveraging Neural Network Uncertainty in Adaptive Unscented Kalman Filter for Spacecraft Pose Estimation;
Advances in Space Research, Vol. 71, Issue 12, pp. 5061-5082 (2023). DOI: 10.1016/j.asr.2023.02.021

Sharma S., D'Amico S.;
Neural Network-Based Pose Estimation for Noncooperative Spacecraft Rendezvous;
IEEE Transactions on Aerospace and Electronic Systems (2020). DOI: 10.1109/TAES.2020.2999148.

Park T. H., D'Amico S.;
Generative Model for Spacecraft Image Synthesis using Limited Dataset;
2020 AAS/AIAA Astrodynamics Specialist Conference, South Lake Tahoe, California, August 9 - 13 (2020).

Park T. H., Sharma S., D'Amico S.;
Towards Robust Learning-Based Pose Estimation of Noncooperative Spacecraft;
2019 AAS/AIAA Astrodynamics Specialist Conference, Portland, Maine, August 11 - 15 (2019).

Sharma S., D'Amico S.;
Pose Estimation for Non-Cooperative Spacecraft Rendezvous Using Neural Networks;
29th AAS/AIAA Space Flight Mechanics Meeting, Ka'anapali, Maui, HI, January 13-17 (2019).

Sharma S.;
Pose Estimation of Uncooperative Spacecraft using Monocular Vision and Deep Learning;
Stanford University, PhD Thesis (2019).

Sharma S., Ventura, J., D’Amico S.;
Robust Model-Based Monocular Pose Initialization for Noncooperative Spacecraft Rendezvous;
Journal of Spacecraft and Rockets (2018).

Sharma S., Beierle C., D’Amico S.;
Pose Estimation for Non-Cooperative Spacecraft Rendezvous Using Convolutional Neural Networks;
IEEE Aerospace Conference, Yellowstone Conference Center, Big Sky, Montana, March 3-10 (2018).

Sharma S., Beierle C., D'Amico S.;
Generative Adversarial Networks for High-Fidelity Simulation of Spacecraft Proximity Operations;
Technical Note, Stanford Space Rendezvous Lab (SLAB), April 23 (2018).

Sharma S., Beierle C., D’Amico S.;
Towards Pose Determination for Non-Cooperative Spacecraft Rendezvous using Convolutional Neural Networks;
International Conference on Space Situational Awareness (ICSSA), Orlando, Florida, November 13-15 (2017).

Sharma S., D’Amico S.;
Reduced-Dynamics Pose Estimation for Non-Cooperative Spacecraft Rendezvous using Monocular Vision;
40th Annual AAS Guidance and Control Conference, Breckenridge, Colorado, February 2-8, 2017.

Sharma S., D’Amico S.;
Comparative Assessment of Techniques for Initial Pose Estimation using Monocular Vision;
Acta Astronautica, 123 pp. 435-445 (2016).
DOI: 10.1016/j.actaastro.2015.12.032