From 1f909ddef160825091bc0743e1396cdba6be0ee1 Mon Sep 17 00:00:00 2001 From: Andrej Orsula Date: Sat, 30 Jul 2022 15:16:54 +0200 Subject: [PATCH] Update documentation Signed-off-by: Andrej Orsula --- README.md | 93 ++++++++++++++++++++++++++++++++----------------------- 1 file changed, 54 insertions(+), 39 deletions(-) diff --git a/README.md b/README.md index 866b28c..660fae6 100644 --- a/README.md +++ b/README.md @@ -3,12 +3,16 @@ This project focuses on applying deep reinforcement learning to acquire a robust policy that allows robots to grasp diverse objects from compact 3D observations in the form of octrees.

- + + + [Franka Emika Panda] Evaluation of a trained policy on novel scenes (previously unseen camera poses, objects, terrain textures, ...).

- + + + [UR5 with RG2 gripper] Sim-to-Real transfer of a policy trained solely inside a simulation (zero-shot transfer). Credit: Aalborg University

@@ -24,6 +28,8 @@ This project focuses on applying deep reinforcement learning to acquire a robust +

+

@@ -47,15 +53,21 @@ If you are interested in configuring these environments, first take a look at th - + + + Reach the end-effector goal. - + + + Grasp and lift a random object. - + + + Grasp and lift a Moon rock. @@ -119,12 +131,16 @@ By default, `Grasp` and `GraspPlanetary` tasks utilize [`GraspCurriculum`](./drl To facilitate the sim-to-real transfer of trained agents, simulation environments introduce domain randomization with the aim of improving the generalization of learned policies. This randomization is accomplished via [`ManipulationGazeboEnvRandomizer`](./drl_grasping/envs/randomizers/manipulation.py) that populates the virtual world and enables randomizing of several properties at each reset of the environment. As this randomizer is configurable with numerous parameters, please take a look at the source code to see what environments you can create.

- + + + Examples of domain randomization for the Grasp task.

- + + + Examples of domain randomization for the GraspPlanetary task.

@@ -143,7 +159,9 @@ This project initially investigated how 3D visual observations can be leveraged To enable the extraction of abstract features from 3D octree observations, an octree-based 3D CNN is employed. The network module that accomplishes such feature extraction is implemented in the form of [`OctreeCnnFeaturesExtractor`](./drl_grasping/drl_octree/features_extractor/octree_cnn.py) (PyTorch). This features extractor is part of the `OctreeCnnPolicy` policy implemented for TD3, SAC and TQC algorithms. Internally, the feature extractor utilizes [O-CNN](https://github.com/microsoft/O-CNN) implementation to benefit from hardware acceleration on NVIDIA GPUs.

- + + + Illustration of the end-to-end actor-critic network architecture with octree-based 3D CNN feature extractor.

@@ -175,15 +193,17 @@ Setup-wise, there are two options when using this repository. **Option A – Doc First, ensure your system has a setup for using Docker with NVIDIA GPUs. You can follow [`install_docker_with_nvidia.bash`](./.docker/host/install_docker_with_nvidia.bash) installation script for Debian-based distributions. Alternatively, consult the [NVIDIA Container Toolkit Installation Guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html) for other Linux distributions. ```bash +# Execute script inside a cloned repository .docker/host/install_docker_with_nvidia.bash +# (Alternative) Execute script from URL +bash -c "$(wget -qO - https://raw.githubusercontent.com/AndrejOrsula/drl_grasping/master/.docker/host/install_docker_with_nvidia.bash)" ``` ### Clone a Prebuilt Docker Image -Prebuilt Docker images of `drl_grasping` can be pulled directly from [Docker Hub](https://hub.docker.com/repository/docker/andrejorsula/drl_grasping) without needing to build them locally. You can use the following command to manually pull the latest image or one of the previous tagged [Releases](https://github.com/AndrejOrsula/drl_grasping/releases). +Prebuilt Docker images of `drl_grasping` can be pulled directly from [Docker Hub](https://hub.docker.com/repository/docker/andrejorsula/drl_grasping) without needing to build them locally. You can use the following command to manually pull the latest image or one of the previous tagged [Releases](https://github.com/AndrejOrsula/drl_grasping/releases). The average size of images is 25GB (including datasets). ```bash -# Size is ~25GB (including datasets) docker pull andrejorsula/drl_grasping:${TAG:-latest} ``` @@ -200,7 +220,10 @@ It is also possible to build the Docker image locally using the included [Docker For simplicity, please run `drl_grasping` Docker containers using the included [`run.bash`](./.docker/run.bash) script shown below (arguments are optional). It enables NVIDIA GPUs and GUI interface while automatically mounting the necessary volumes (e.g. persistent logging) and setting environment variables (e.g. synchronization of middleware communication with the host). This script will always print the corresponding low-level `docker run ...` command for your reference. ```bash +# Execute script inside a cloned repository .docker/run.bash ${TAG:-latest} ${CMD} +# (Alternative) Execute script from URL +bash -c "$(wget -qO - https://raw.githubusercontent.com/AndrejOrsula/drl_grasping/master/.docker/run.bash)" -- ${TAG:-latest} ${CMD} ``` The network communication of `drl_grasping` within this Docker container is configured based on the ROS 2 [`ROS_DOMAIN_ID`](https://docs.ros.org/en/galactic/Concepts/About-Domain-ID.html) environment variable, which can be set via `ROS_DOMAIN_ID={0...101} .docker/run.bash ${TAG:-latest} ${CMD}`. By default (`ROS_DOMAIN_ID=0`), external communication is restricted and multicast is disabled. With `ROS_DOMAIN_ID=42`, the communication remains restricted to `localhost` with multicast enabled, enabling monitoring of communication outside the container but within the same system. Using `ROS_DOMAIN_ID=69` will use the default network interface and multicast settings, which can enable monitoring of communication within the same LAN. All other `ROS_DOMAIN_ID`s share the default behaviour and can be employed to enable communication partitioning for running of multiple `drl_grasping` instances. @@ -279,23 +302,23 @@ ros2 launch drl_grasping random_agent.launch.py seed:=42 robot_model:=lunalab_su -
[WIP] Try Pre-trained Agents +
Train New Agents @@ -378,27 +401,19 @@ ros2 launch drl_grasping optimize.launch.py seed:=69 robot_model:=panda env:=Gra
- +``` ## Directory Structure