Skip to content
/ spc Public
forked from ucbdrive/spc

Semantic Predictive Control

Notifications You must be signed in to change notification settings

xinleipan/spc

 
 

Repository files navigation

Semantic Predictive Control for Explainable and Efficient Policy Learning

[paper] / [video demo]

Semantic predictive control (SPC) is a policy learning framework that predicts future semantic segmentation and events by aggregating multi-scale feature maps. It utilizes dense supervision from semantic segmentation for feature learning and greatly improves policy learning efficiency. The learned features are explainable as they depict future scenes with semantic segmentation and explicit events.

This repository contains a PyTorch implementation of SPC, as well as some training scripts to reproduce policy learning results reported in our paper.

Overview

Our model is composed of four sub-modules:
  1. The feature extraction module extracts multi-scale intermediate features from RGB observations;
  2. The extracted features are then concatenated with tiled actions and feed to the multi-scale prediction module that sequentially predicts future features;
  3. The information prediction module takes in the predicted latent feature representation and outputs corresponding future frame semantic segmentation and task-related signals, such as collision, off-road, and speed;
  4. The guidance network module that predicts action distribution for efficient sampling-based optimization.

Training

Our results in the paper can be reproduced with the provided scripts by running

cd scripts/
bash train_#ENVNAME.sh

About

Semantic Predictive Control

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 82.7%
  • C++ 13.1%
  • Cuda 2.0%
  • C 1.8%
  • Other 0.4%