3d-deep-learning

3D Deep Learning works

Tasks

3D Representation

Spherical CNNs

• Taco S. Cohen, Spherical CNNs, ICLR 2018 Best paper [paper]
• Learning SO(3) Equivariant Representations with Spherical CNNs [paper] [code]

3D Classification

Datasets

• ModelNet10/40

Networks

• 3D CNN
  • 3D-DenseNet
  • Voxnet: A 3d convolutional neural network for real-time object recognition, IROS 2015. [code] [paper]
  • [3D-NIN, network in network]
  • VRN Ensemble, Generative and discriminative voxel modeling with convolutional neural networks, arxiv [paper] [code]
    • Voxception-Resnet Blocks
• 2D CNN - MVCNN, Learned-Miller.Multi- view convolutional neural networks for 3d shape recognition, ICCV2015 [project] [code] [paper][data] [video]
• Point
  • PointNet [project][paper][code][video][slides]
    • global pooling
    • T-net
  • PointNet++ [paper] [code]
    • sampling & grouping to learning local feature for fine-gaint objects
    • two PointNet
• Graph/tree-based
  • Kd-Net, scape from cells: Deep kd- networks for the recognition of 3d point cloud models, arxiv2017 [paper]
    • kd-tree
  • Octnet: Learning deep 3d representations at high resolutions, CVPR2017
    • octree
  • O-cnn: Octree-based convolutional neural networks for 3d shape analysis, TOG2017
    • octree
  • SO-Net, SO-Net: Self-Organizing Network for Point Cloud Analysis, CVPR2018 [paper] [paper] [code]
    • point-to-node kNN search Self-Organizing Map (SOM)
  • KCNet, Mining Point Cloud Local Structures by Kernel Correlation and Graph Pooling, CVPR2018 [project] [code] [paper][data] [video]
    • Kernel Correlation
    • Graph Pooling

3D Segmentation

Datasets

• HVSMR
• BRATS Data
• ShapeNet

Networks

• HeartSeg, 3D-FC-Densenet [code]
• 3D-Unet [paper]
• PointNet
• PointNet++
• KCNet
• SO-Net

3D Object Detection

Datasets

Data types: RGBD, Flow, Laser

• KITTI
• KITTI Object Visualization Tool

Networks

• MV3D, Multi-View 3D Object Detection Network for Autonomous Driving [paper] [code]
• Avod, Joint 3D Proposal Generation and Object Detection from View Aggregation [paper] [code]
• F-PointNet, Frustum PointNets for 3D Object Detection from RGB-D Data [paper] [code]
• VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection [paper]

3D Reconstruction (Generation)

Datasets

Data types: RGBD, Flow, Laser

• ShapeNet

Networks

• SO-Net
• 3D-GAN

CVPR2016 Tutorial: 3D Deep Learning with Marvin

• CVPR2016 Tutorial: 3D Deep Learning with Marvin
• 3D Shape Retrieval
• C3D, website
• [Video Caffe(C3D)] [code]
• [DeepMedic, Brain Lesion Segmentation] [[code(https://github.com/Kamnitsask/deepmedic)]
• 3D Keypoint Detection and Feature Description

Codes and libs for 3D

• util3d
• spectral-lib
• 3D-Caffe

Papers

• Learning 3D Object Orientations From Synthetic Images
• Read LSTM: A Search Space Odyssey and implement LSTM.
• Use Tree LSTM in place of LSTM. Paper Code Slides
• Learning 3D Object Orientations From Synthetic Images  - 3D Shape Segmentation with Projective Convolutional Networks. CVPR2017. Project Poster Presentation

DL on Medical Image

• Antibody-supervised deep learning for quantification of tumor-infiltrating immune cells in hematoxylin and eosin stained breast cancer samples

• more usefull tools should be added in.

  • Read and process process data as needed by the network.
  • Read Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation and implement a GRU.