Lidar Point clound processing for Autonomous Driving

A list of references on lidar point cloud processing for autonomous driving

LiDAR Pointcloud Clustering/Semantic Segmentation/Plane extraction

Tasks : Road/Ground extraction, plane extraction, Semantic segmentation, open set instance segmentation, Clustering

• Fast Segmentation of 3D Point Clouds: A Paradigm on LiDAR Data for Autonomous Vehicle Applications ICRA 2017 [git, pdf]
• Time-series LIDAR Data Superimposition for Autonomous Driving [pdf]
• Fast segmentation of 3D point clouds for ground vehicles [ieee]
• An Improved RANSAC for 3D Point Cloud Plane Segmentation Based on Normal Distribution Transformation Cells
• Segmentation of Dynamic Objects from Laser Data [pdf]
• A Fast Ground Segmentation Method for 3D Point Cloud [pdf]
• Ground Estimation and Point Cloud Segmentation using SpatioTemporal Conditional Random Field [pdf]
• Real-Time Road Segmentation Using LiDAR Data Processing on an FPGA [pdf]
• Efficient Online Segmentation for Sparse 3D Laser Scans [pdf], [git]
• CNN for Very Fast Ground Segmentation in Velodyne LiDAR Data [pdf]
• A Comparative Study of Segmentation and Classification Methods for 3D Point Clouds 2016 Masters Thesis [pdf]
• Fast Multi-pass 3D Point Segmentation Based on a Structured Mesh Graph for Ground Vehicles pdf video
• RangeNet++: Fast and Accurate LiDAR Semantic Segmentation [[link](https://github.com/PRBonn/lidar-bonnetal], [pdf]
• Circular Convolutional Neural Networks for Panoramic Images and Laser Data pdf
• Efficient Convolutions for Real-Time Semantic Segmentation of 3D Point Clouds [pdf]
• Identifying Unknown Instances for Autonomous Driving/Open-set instance segmentation algorithm CoRL 2019 [pdf]
• RIU-Net: Embarrassingly simple semantic segmentation of3D LiDAR point cloud. [pdf, LU-net]
• SalsaNet: Fast Road and Vehicle Segmentation in LiDAR Point Clouds for Autonomous Driving [pdf]
• SqueezeSegV3: Spatially-Adaptive Convolution for Efficient Point-Cloud Segmentation [link]
• PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation [link]
• Scan-based Semantic Segmentation of LiDAR Point Clouds: An Experimental Study IV 2020 [pdf]
• Plane Segmentation Based on the Optimal-vector-field in LiDAR Point Clouds [link]
• Semantic Segmentation of 3D LiDAR Data in Dynamic Scene Using Semi-supervised Learning [link]
• Learning Hierarchical Semantic Segmentations of LIDAR Data 3DV 2015 [pdf]
• EfficientLPS: Efficient LiDAR Panoptic Segmentation 2021 pdf, video
• 4D Panoptic LiDAR Segmentation 2021 [pdf]
• urban_road_filter: a real-time LIDAR-based urban road and sidewalk detection algorithm for autonomous vehicles [git, video, pdf, code]

Pointcloud Density & Compression

• DBSCAN : A density-based algorithm for discovering clusters in large spatial databases with noise (1996) [pdf]
• Hierarchical Density Estimates for Data Clustering, Visualization, and Outlier Detection pdf
• Building Maps for Autonomous Navigation Using Sparse Visual SLAM Features [pdf]
• STD: Sparse-to-Dense 3D Object Detector for Point Cloud pdf
• Fast semantic segmentation of 3d point clounds with strongly varying density [pdf]
• The Perfect Match: 3D Point Cloud Matching with Smoothed Densities [pdf, code]
• Deep Compression for Dense Point Cloud Maps [link]
• Improved Deep Point Cloud Geometry Compression [pdf, git]
• Real-Time Spatio-Temporal LiDAR Point Cloud Compression [pdf]

Registration and Localization

• A Review of Point Cloud Registration Algorithms for Mobile Robotics 2015 [pdf]
• LOAM: Lidar Odometry and Mapping in Real-time RSS 2014 [pdf, video]
• Fast Planar Surface 3D SLAM Using LIDAR 2016 [pdf]
• Point Clouds Registration with Probabilistic Data Association IROS 2016 [git]
• Robust LIDAR Localization using Multiresolution Gaussian Mixture Maps for Autonomous Driving IJRR 2017 [pdf], [Thesis]
• Automatic Merging of Lidar Point-Clouds Using Data from Low-Cost GPS/IMU Systems SPIE 2011 [pdf]
• Fast and Robust 3D Feature Extraction from Sparse Point Clouds [pdf]
• 3DFeat-Net: Weakly Supervised Local 3D Features for Point Cloud Registration [pdf]
• Incremental Segment-Based Localization in 3D Point Clouds [pdf]
• OverlapNet: Loop Closing for LiDAR-based SLAM, RSS 2020 [[pdf](OverlapNet: Loop Closing for LiDAR-based SLAM), git, video]
• CorsNet: 3D Point Cloud Registration by Deep Neural Network, ICRA 2020 [link]
• LPD-Net: 3D Point Cloud Learning for Large-Scale Place Recognition and Environment Analysis ICCV 2019 [pdf]
• DH3D: Deep Hierarchical 3D Descriptors for Robust Large-Scale 6DoF Relocalization [pdf,project, video]
• Localisation using LiDAR and Camera Localisation in low visibility road conditions Master’s thesis 2017 [pdf]
• Monocular Camera Localization in 3D LiDAR Maps IROS 2016 [pdf]

Feature Extraction

• Fast Feature Detection and Stochastic Parameter Estimation of Road Shape using Multiple LIDAR [pdf]
• Finding Planes in LiDAR Point Clouds for Real-Time Registration [pdf]
• Online detection of planes in 2D lidar [pdf]
• A Fast RANSAC–Based Registration Algorithm for Accurate Localization in Unknown Environments using LIDAR Measurements [pdf]
• Hierarchical Plane Extraction (HPE): An Efficient Method For Extraction Of Planes From Large Pointcloud Datasets [pdf]
• A Fast and Accurate Plane Detection Algorithm for Large Noisy Point Clouds Using Filtered Normals and Voxel Growing [pdf]
• SPLATNet: Sparse Lattice Networks for Point Cloud Processing CVPR 2018 [pdf, code]
• PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding, 4D-VISION workshop at ECCV'2020 [pdf, workshop]

Object detection and Tracking

• Learning a Real-Time 3D Point Cloud Obstacle Discriminator via Bootstrapping pdf
• Terrain-Adaptive Obstacle Detection [pdf]
• 3D Object Detection from Roadside Data Using Laser Scanners [pdf]
• 3D Multiobject Tracking for Autonomous Driving : Masters thesis A S Abdul Rahman
• Motion-based Detection and Tracking in 3D LiDAR Scans [pdf]
• Lidar-histogram for fast road and obstacle detection [pdf]
• End-to-end Learning of Multi-sensor 3D Tracking by Detection pdf
• Leveraging Heteroscedastic Aleatoric Uncertainties for Robust Real-Time LiDAR 3D Object Detection pdf
• Deep tracking in the wild: End-to-end tracking using recurrent neural networks [pdf]
• Leveraging Heteroscedastic Aleatoric Uncertainties for Robust Real-Time LiDAR 3D Object Detection [pdf], video]
• VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection CVPR 2018 [pdf, code]
• PIXOR: Real-time 3D Object Detection from Point Clouds CVPR 2018 [pdf]
• Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges [pdf]
• Low resolution lidar-based multi-object tracking for driving applications [pdf]
• Patch Refinement -- Localized 3D Object Detection [pdf]
• PointPillars: Fast Encoders for Object Detection from Point Clouds CVPR 2019 [pdf]
• StarNet: Targeted Computation for Object Detection in Point Clouds NeurIPS 2019 ML4AD [pdf]
• PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection CVPR 2020 [pdf]
• LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Driving CVPR 2019 [pdf]
• Range Conditioned Dilated Convolutions for Scale Invariant 3D Object Detection 2020 [pdf]
• AFDet: Anchor Free One Stage 3D Object Detection [pdf]
• SA-SSD: Structure Aware Single-stage 3D Object Detection from Point Cloud (CVPR 2020) [pdf, git]
• Any Motion Detector: Learning Class-agnostic Scene Dynamics from a Sequence of LiDAR Point Clouds, ICRA 2020 [pdf]
• MVLidarNet: Real-Time Multi-Class Scene Understanding for Autonomous Driving Using Multiple Views [link, video]
• Learning to Optimally Segment Point Clouds, ICRA 2020 [pdf, video, git]
• What You See is What You Get: Exploiting Visibility for 3D Object Detection [pdf, video, project]

Classification/Supervised Learning

• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation [link, link2]
• SqueezeSeg: Convolutional Neural Nets with Recurrent CRF for Real-Time Road-Object Segmentation from 3D LiDAR Point Cloud pdf
• Improving LiDAR Point Cloud Classification using Intensities and Multiple Echoes [pdf]
• DepthCN: Vehicle Detection Using 3D-LIDAR and ConvNet [pdf]
• 3D Object Localisation with Convolutional Neural Networks [Thesis]
• SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud [pdf]
• PointSeg: Real-Time Semantic Segmentation Based on 3D LiDAR Point Cloud [pdf]
• Fast LIDAR-based Road Detection Using Fully Convolutional Neural Networks [pdf]
• ChipNet: Real-Time LiDAR Processing for Drivable Region Segmentation on an FPGA [pdf]

Maps / Grids / HD Maps / Occupancy grids/ Prior Maps

• Hierarchies of Octrees for Efficient 3D Mapping pdf
• Adaptive Resolution Grid Mapping using Nd-Tree [ieee], [pdf, video]
• LIDAR-Data Accumulation Strategy To Generate High Definition Maps For Autonomous Vehicles [link]
• Long-term robot mapping in dynamic environments, Aisha Naima Walcott Thesis MIT 2011 [link]
• Long-term 3D map maintenance in dynamic environments ICRA 2014 [pdf, video]
• Detection and Tracking of Moving Objects Using 2.5D Motion Grids ITSC 2015 [pdf]
• Autonomous Vehicle Navigation in Rural Environments without Detailed Prior Maps ICRA 2018 [pdf, video]
• 3D Lidar-based Static and Moving Obstacle Detection in Driving Environments: an approach based on voxels and multi-region ground planes [pdf]
• Spatio–Temporal Hilbert Maps for Continuous Occupancy Representation in Dynamic Environments [pdf]
• Dynamic Occupancy Grid Prediction for Urban Autonomous Driving: A Deep Learning Approach with Fully Automatic Labeling [pdf]
• Mobile Laser Scanned Point-Clouds for Road Object Detection and Extraction: A Review [pdf]
• Efficient Continuous-time SLAM for 3D Lidar-based Online Mapping [pdf]
• DeLS-3D: Deep Localization and Segmentation with a 3D Semantic Map [pdf],video]
• Recurrent-OctoMap: Learning State-based Map Refinement for Long-Term Semantic Mapping with 3D-Lidar Data [pdf]
• HDNET: Exploiting HD Maps for 3D Object Detection [pdf]
• Mapping with Dynamic-Object Probabilities Calculated from Single 3D Range Scans ICRA 2018 [pdf]

End-To-End Learning

• Monocular Fisheye Camera Depth Estimation Using Semi-supervised Sparse Velodyne Data [pdf]
• Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net [pdf]

Simulated pointclouds / Simulators

• Virtual Generation of Lidar Data for Autonomous Vehicles Simulation of a lidar sensor inside a virtual world Bachelors Thesis 2017 pdf
• A LiDAR Point Cloud Generator: from a Virtual World to Autonomous Driving ACM 2018 [pdf]
• Udacity based simulator [link, git]
• Tutorial on Gazebo to simulate raycasting from Velodyne lidar [link]
• Udacity Driving Dataset [link]
• Virtual KITTI [link]
• SynthCity: A large-scale synthetic point cloud 2019 [dataset, pdf]
• Precise Synthetic Image and LiDAR (PreSIL) Dataset for Autonomous Vehicle Perception [link]
• Fast Synthetic LiDAR Rendering via Spherical UV Unwrapping of Equirectangular Z-Buffer Images 2020 [pdf]

Lidar Datasets

• nuScenes : public large-scale dataset for autonomous driving [dataset]
  • nuScenes-lidarseg will be released in Q2 2020. [link]
• Ford Campus Vision and Lidar Data Set [pdf, dataset]
• Oxford RobotCar dataset dataset 1 Year, 1000km: The Oxford RobotCar Dataset pdf
• LiDAR-Video Driving Dataset: Learning Driving Policies Effectively [pdf]
• KAIST Complex Urban Data Set Dataset [dataset]
• Semantic 3D 2017 dataset
• Paris-Lille-3D: A Point Cloud Dataset for Urban Scene Segmentation and Classification [pdf dataset]
• Semantic KITTI 2019 [dataset]
• A*3D: An Autonomous Driving Dataset in Challeging Environments [dataset], video]
• HD Map Dataset & Localization Dataset NAVER Labs : [link]
• Argoverse by ARGO AI : Two public datasets supported by highly detailed maps to test, experiment, and teach self-driving vehicles how to understand the world around them. [link]
• Lyft dataset [link]
• SemanticPOSS: A Point Cloud Dataset with Large Quantity of Dynamic Instances [link, pdf]
• A2D2 Audi dataset [link]
• PandaSet : Public large-scale dataset for autonomous driving provided by Hesai & Scale. [link]
• Toronto-3D: A Large-scale Mobile LiDAR Dataset for Semantic Segmentation of Urban Roadways [pdf, dataset]

Spatio-Temporal, Movement, Flow estimation in Pointclouds

• Rigid Scene Flow for 3D LiDAR Scans IROS 2016 [pdf]
• Deep Lidar CNN to Understand the Dynamics of Moving Vehicles [pdf]
• Learning motion field of LiDAR point cloud with convolutional networks [link]
• Hallucinating Dense Optical Flow from Sparse Lidar for Autonomous Vehicles [pdf video]
• FlowNet3D: Learning Scene Flow in 3D Point Clouds CVPR 2019 [pdf, code]
• LiDAR-Flow: Dense Scene Flow Estimation from Sparse LiDAR and Stereo Images [pdf]
• 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks CVPR 2019 [pdf, code]
• MeteorNet: Deep Learning on Dynamic 3D Point Cloud Sequences, ICCV 2019 [pdf]
• DeepLiDARFlow: A Deep Learning Architecture For Scene Flow Estimation Using Monocular Camera and Sparse LiDAR 2020 [pdf]

Advanced Topics/Other applications

Tasks : Upsampling, Domain adaptation Sim2Real, NAS, SSL, shape reconstruction, outlier extraction, Compression, Change detection, Domain Transfer

• Semantic Point Cloud Filtering, Masters thesis 2017 link
• PU-GAN: a Point Cloud Upsampling Adversarial Network ICCV 2019 [pdf, code]
• Neural Architecture Search for Object Detection in Point Cloud [blog], [AutoDeepLabNAS paper]
• Weighted Point Cloud Augmentation for Neural Network Training Data Class-Imbalance [pdf]
• Quantifying Data Augmentation for LiDAR based 3D Object Detection [pdf]
• Improving 3D Object Detection through Progressive Population Based Augmentation [[pdf(https://arxiv.org/abs/2004.00831)]
• 3D Object Detection From LiDAR Data Using Distance Dependent Feature Extraction VEHITS 2020 [pdf]
• Training a Fast Object Detector for LiDAR Range Images Using Labeled Data from Sensors with Higher Resolution ITSC 2019 [pdf]
• Performance of LiDAR object detection deep learning architectures based on artificially generated point cloud data from CARLA simulator 2019 [pdf]
• Efficient Learning on Point Clouds with Basis Point Sets ICCV 2019 [pdf]
• Mastering Data Complexity for Autonomous Driving with Adaptive Point Clouds for Urban Environments 2017 [pdf]
• Visually aided changes detection in 3D lidar based reconstruction 2015 [Thesis]
• Fast Polar Attentive 3D Object Detection on LiDAR Point Clouds ML4AD 2021 pdf

Implicit representations

• Neural Implicit Embedding for Point Cloud Analysis CVPR 2020 [pdf]
• DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation [pdf]

Domain adaptation and domain transer

• Complete & Label: A Domain Adaptation Approach to Semantic Segmentation of LiDAR Point Clouds 2020 [pdf]
• PointDAN: A Multi-Scale 3D Domain Adaption Network for Point Cloud Representation [pdf]
• Domain Transfer for Semantic Segmentation of LiDAR Data using Deep Neural Networks IROS 2020 [pdf, video]
• Domain Adaptation for Vehicle Detection from Bird's Eye View LiDAR Point Cloud Data ICCVW 2019 pdf

Self-supervised learning

• Self-Supervised Deep Learning on Point Clouds by Reconstructing Space NeurIPS 2019 [pdf]
• SegContrast: 3D Point Cloud Feature Representation Learning RAL-ICRA'22 video, pdf
• Self-supervised Point Cloud Prediction Using 3D Spatio-temporal CNNs CORL 2021 pdf

Graphs and Pointclouds

• Detection of closed sharp edges in point clouds using normal estimation and graph theory CAD 2007 [link]
• Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs CVPR 2017 [pdf, vide]
• Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs CVPR2018 [pdf]
• ConvPoint: continuous convolutions for cloud processing Eurographics 3DOR, 2019 [pdf, code]
• Point Cloud Oversegmentation with Graph-Structured Deep Metric Learning [CVPR Workshop 2019], video
• Dynamic Graph CNN for Learning on Point Clouds [pdf, project] TOG 2019

Large-scale pointcloud Algorithms (vs scan based)

• Deep Parametric Continuous Convolutional Neural Networks CVPR 2018 [pdf]
• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space NeurIPS 2017 [pdf, code], semantic seg code
• Classification of Point Cloud for Road Scene Understanding with Multiscale Voxel Deep Network Slides
• Semantic Segmentation of 3D point Clouds Loic Landireu [Slides]
• KPConv: Flexible and Deformable Convolution for Point Clouds [pdf, git]
• RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds [pdf, git]
• Cylinder3D: An Effective 3D Framework for Driving-scene LiDAR Semantic Segmentation CVPR 2021 pdf

4D Panoptic LiDAR Segmentation

• Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Tracking 2021 video, pdf, nuscenes benchmark
• 4D Panoptic LiDAR Segmentation CVPR 2021 pdf, project
• Contrastive Instance Association for 4D Panoptic Segmentation using Sequences of 3D LiDAR Scans 2022 RAL pdf, video

Embedded/Resource constrained pointcloud processing

• Sparse PointPillars: Maintaining and Exploiting Input Sparsity to Improve Runtime on Embedded Systems pdf, video
• IBB-Net: Fast Iterative Bounding Box Regression for Detection on Point Clouds, Brendan Miller, 2020 CMU masters thesis
• Fast 3-D Urban Object Detection on Streaming Point Clouds [pdf]
• StrObe: Streaming Object Detection from LiDAR Packets pdf, video
• PolarStream: Streaming Lidar Object Detection and Segmentation with Polar Pillars NeurIPS 2021 pdf, video, code
• RAD: Realtime and Accurate 3D Object Detection on Embedded Systems CVPRW 2021 pdf
• BEVDetNet: Bird’s Eye View LiDAR Point Cloud based Real-time 3D Object Detection for Autonomous Driving ITSC 2021 pdf
• Streaming Object Detection for 3-D Point Clouds 2020 pdf

Tools/SW/Packages

• Python bindings for Point Cloud Library [git]
• Open3D [link]
• pyntcloud [link]
• PyVista [link]
• torch-points3d : Pytorch framework for doing deep learning on point clouds [link]
• Geometric Deep Learning Extension Library for [PyTorch link]
• kaolin : A PyTorch Library for Accelerating 3D Deep Learning Research [link]
• PyTorch3D : FAIR's library of reusable components for deep learning with 3D data [link]
• PCDet Toolbox in PyTorch for 3D Object Detection from Point Cloud [link]
• pointcloudset: Efficient analysis of large datasets of point clouds recorded over time [link]