LUNA: Low-Light Robust Panoptic Lifting for Adverse Robotic 3D Scene Perception

Ahalya Ravendran, Leo Lebrat, Rodrigo Santa Cruz, Hu Zhang, Lars Petersson, Dadong Wang, Xun Li

CSIRO Data61

📄 Paper 💻 Code

Abstract

Robotic vision in low-light settings suffers from noise and blur, which hinder reliable scene understanding. LUNA is a geometry-aware extension of the panoptic lifting framework designed to address these challenges. By integrating depth cues with fast adaptive multitask optimization, LUNA delivers robust 3D panoptic segmentation and reconstruction even under degraded visual conditions. Tested on a systematically corrupted Replica dataset, LUNA consistently outperforms both baseline methods and restoration-augmented pipelines across varying levels of image quality.

Methodology

Pipeline Figure

Overview of the LUNA framework. Multi-view low-quality RGB images are processed by a volumetric renderer (TensoRF) and a Mask2Former head. Geometry-aware supervision guides training via composite depth loss, while Fast Adaptive Multitask Optimization dynamically balances photometric, geometric, semantic, and instance losses.

Experiments

Panoptic Segmentation on Original Replica Dataset

Original

Panoptic Segmentation under Strong Motion Blur

Pano Blur
Panoptic Lifting
Ours Blur
Ours

Panoptic Segmentation under Strong Gaussian Noise

Pano Noise
Panoptic Lifting
Ours Noise
Ours

Recommendations for Robotic Deployment

Guidelines for selecting imaging strategies under varying operational conditions.

Robotic Imaging Recommendations
Overview of imaging strategies across different conditions

For static robotic platform use long exposures to improve SNR. Deblurring outperforms denoising for inspection or exploration tasks. For dynamic environments opt for high ISO with short exposures to reduce blur. Prioritize depth sensors at higher frame rates to maintain geometric consistency. For highly degraded visual conditions rely on geometry-aware supervision over denoising. Depth supervision recovers global structure and object boundaries more effectively. For computational considerations depth-augmented semantic prediction offers a lightweight, efficient alternative to full geometry-aware supervision.

BibTeX

@article{ravendran2025luna,
  title={LUNA: Low-Light Robust Panoptic Lifting for Adverse Robotic 3D Scene Perception},
  author={Ravendran, Ahalya and Lebrat, Leo and Santa Cruz, Rodrigo and Zhang, Hu and Petersson, Lars and Wang, Dadong and Li, Xun},
  journal={submitted for Robotic Automation Letters (RA-L)},
  year={2026}
}