Clarify comparison to BiRRT

[v4hn]

Thanks for contributing all your work. It's really interesting and it will take some time to digest all of it... 🐱

I would like to ask you to clarify the comparison to BiRRT you show in your readme and documentation.

• The right trajectory clearly includes separate approach/put-down/retract motions without phase blending which are often preferred in physical robot experiments for predictability during execution, for clearance between motion phases, and to account for minor errors in perception. These phases were not computed through BiRRT so the label seems wrong. I don't think the left trajectory includes comparable constraints (or at least the required distances are tiny), so the comparison seems flawed. If you wanted to compare the pure transit motion + optimizing grasp pose IK(?) could you indicate that somewhere for fairness? I believe your solutions are still faster there.
• Where both trajectories computed with the same (sphere) collision geometry and how long did it take to compute both? The typical advantage of BiRRT is that it's fast to generate feasible but jerky paths and running BiRRT* with a very short time epsilon after finding the first solution will generate much shorter Cartesian paths. Obviously time to solution is a problematic comparison between GPU and CPU anyway, but I believe you want to give a good first impression here so it should be fine.

[balakumar-s]

We wanted the video to show how CuRobo can help in a full pipeline for manipulation tasks. I can more details on the text.

Regarding your questions:

1. In addition to the reasons you mentioned for the approach-grasp, grasp-lift, reach-putDown, and putDown-moveback phases, these phases also aid sampling-based planners in finding a path as the goal/start configuration is not in a highly constrained space (object between gripper fingers). We wanted to show that when planning motions with CuRobo, you can directly go to the grasp phase (with object between gripper fingers) as trajectory optimization can get you near tight tolerances. We also show that this approach works for a long horizon stacking task.
2. We do not compare the compute times in the video and only try to have a reasonable time limit on motion planning (10 seconds). We focus the comparison on motion times and quality given a reasonable time budget. We collect all motions offline and then start execution at the same time for both the robots. The BiRRT planner is from pddlstream which uses pybullet for collision checks (i.e., no sphere representation for the robot). There are definitely ways to tune a sampling-based planner to output better paths (could take more compute time) given a task.

[v4hn]

these phases also aid sampling-based planners in finding a path as the goal/start configuration is not in a highly constrained space (object between gripper fingers).

This is certainly the case for peg-in-hole style tasks with (sub)millimeter clearance where joint-space sampling without projection will never yield valid samples; which is why people would definitely use predefined Cartesian approaches or projection-based sampling there.
However, the shelf panda scenes you show here have plenty of clearance in the grasp/place poses for BiRRT in my opinion,
as the main advantage of bi-directional RRT over RRT is exactly support for queries where both start and goal state are somewhat constraint. Could you point me to the exact problem description for this pick/place example? I would like to set it up with MoveIt Task Constructor, but I have trouble finding task descriptions which include the mug.

We wanted to show that when planning motions with CuRobo, you can directly go to the grasp phase (with object between gripper fingers) as trajectory optimization can get you near tight tolerances

I see two clearly different aspects here:

There is optimality of the trajectory under a known geometric/dynamic model. It looks like you did a great job to optimize here and the trajectories look really nice. At the same time I believe BiRRT (with or without *) can give you (worse) feasible trajectories for the same motion problem (I also believe 10 seconds time budget is abundant for the selected pick/place task),
and those are the ones you should be comparing to here.

But then there is also feasibility of the trajectory under known/estimated uncertainty of both models. Obviously pose estimation is noisy and you mention you had tracking problems with the UR robot (which we have also seen a lot).
Both uncertainties contradict the aim for "tight" tolerances and are the whole reason for the Cartesian approach phases, as I use them. Incidentally, in some quick tests with shelf-style variations on the MTC tutorial I linked above, computation time to first solution with and without Cartesian approaches does not differ significantly (not a thorough evaluation though).

As it stands, I believe you are comparing apples and oranges in that visualization and it's one of the first things you show to readers.
That being said, let me repeat that I really like the framework and I think you did great work. 👍

There are definitely ways to tune a sampling-based planner to output better paths (could take more compute time) given a task.

Usually, the more time you spend to tune your planner pipelines to the task, the less computation time you need in the end. 😎 But of course that's not what we are here for, right? 😉

[balakumar-s]

I need to still process some of the text.

Regarding tracking, we were able to get UR to track the trajectories at very high accuracy once we disabled the safety mode.

MoveIt task constructor looks interesting. Do you think we could setup a pick and place task in a similar shelf with cuboids and compare methods? I am assuming you would better know the intricacies of sampling based planning and it would be nice to have benchmarks on manipulation tasks.

We could write a task and load it into isaac sim for the simulation and obtain trajectories from moveit or curobo. This would require some work on both our ends but would be interesting to see.

[v4hn]

Do you think we could setup a [MTC] pick and place task in a similar shelf with cuboids and compare methods?

Definitely. As I wrote above, I'm having trouble finding the exact task specification you used for this illustration. Could you point me to it? Feel free to drop me a mail for private exchange.

[v4hn]

friendly ping. Could you point me to the task specification used for the comparison? I would like to set up the environment myself.