Hybrid Elimination Improvements #1575

varunagrawal · 2023-07-12T16:08:19Z

Override apply with UnaryAssignment for DecisionTreeFactor.
Improve discrete elimination by eliminating the joint distribution and re-segregating back into original conditionals.
Make hybrid code use common parent DiscreteFactor where applicable.
Update the eliminate method to check the factors as mentioned in @dellaert's TODO.
Templetize methods in Switching.h to remove duplication.

varunagrawal · 2023-07-14T22:40:15Z

I changed the way we prune the discrete probabilities by pruning the joint distribution rather than the conditionals. This gives a 3x speedup. Maybe we should be pruning before discrete-only elimination.

Before

Number of timesteps: K = 16

-Total: 0 CPU (0 times, 0 wall, 0.65 children, min: 0 max: 0)
|   -SmootherEstimation: 3.65 CPU (1 times, 3.70242 wall, 0.65 children, min: 0 max: 0)
|   |   -SmootherUpdate: 3.63 CPU (15 times, 3.67252 wall, 0.65 children, min: 0 max: 0)
|   |   |   -assembleGraphTree: 0.01 CPU (30 times, 0.00976 wall, 0.01 children, min: 0 max: 0)
|   |   |   -hybrid continuous eliminate: 0 CPU (168 times, 0.006431 wall, 0 children, min: 0 max: 0)
|   |   |   -HybridBayesNet PruneDiscreteConditionals: 0.64 CPU (15 times, 0.640958 wall, 0.64 children, min: 0 max: 0)
|   |   |   -HybridBayesNet UpdateDiscreteConditionals: 2.8 CPU (15 times, 2.84436 wall, 0 children, min: 0 max: 0)
|   |   |   |   -HybridBayesNet MakeConditional: 0 CPU (120 times, 0.001385 wall, 0 children, min: 0 max: 0)
|   |   |   -HybridBayesNet PruneMixtures: 0 CPU (15 times, 0.009233 wall, 0 children, min: 0 max: 0)

After

Number of timesteps: K = 16

-Total: 0 CPU (0 times, 0 wall, 0.71 children, min: 0 max: 0)
|   -SmootherEstimation: 0.91 CPU (1 times, 0.904348 wall, 0.71 children, min: 0 max: 0)
|   |   -SmootherUpdate: 0.88 CPU (15 times, 0.882961 wall, 0.71 children, min: 0 max: 0)
|   |   |   -assembleGraphTree: 0 CPU (30 times, 0.009388 wall, 0 children, min: 0 max: 0)
|   |   |   -hybrid continuous eliminate: 0 CPU (168 times, 0.004795 wall, 0 children, min: 0 max: 0)
|   |   |   -HybridBayesNet PruneDiscreteConditionals: 0.64 CPU (15 times, 0.632673 wall, 0.64 children, min: 0 max: 0)
|   |   |   -HybridBayesNet UpdateDiscreteConditionals: 0.07 CPU (15 times, 0.071286 wall, 0.07 children, min: 0 max: 0)
|   |   |   -HybridBayesNet PruneMixtures: 0 CPU (15 times, 0.00871 wall, 0 children, min: 0 max: 0

dellaert · 2023-07-16T19:47:48Z

I changed the way we prune the discrete probabilities by pruning the joint distribution rather than the conditionals. This gives a 3x speedup. Maybe we should be pruning before discrete-only elimination.

Does this PR make both these changes? I'd prefer to review a PR that just does the tablefactor and shows the speedup...

varunagrawal · 2023-07-16T23:48:27Z

I changed the way we prune the discrete probabilities by pruning the joint distribution rather than the conditionals. This gives a 3x speedup. Maybe we should be pruning before discrete-only elimination.

Does this PR make both these changes? I'd prefer to review a PR that just does the tablefactor and shows the speedup...

This PR only updates the discrete elimination to prune the joint distribution. TableFactor is not yet used, and will come in a subsequent PR. :)

Updated PR description to reflect the changes happening here.

dellaert · 2023-07-17T06:32:23Z

I'd still like to request to split into TableFactor related changes and other, and PR straight to develop? Otherwise the base branch will become an un-reviewable kitchen-sink PR. PS CI seems to fail, so splitting might help there as well.

varunagrawal · 2023-07-17T13:02:40Z

In that case I'm going to have to do some cherry picking and force pushing.

varunagrawal · 2023-07-17T13:23:06Z

I'd still like to request to split into TableFactor related changes and other, and PR straight to develop? Otherwise the base branch will become an un-reviewable kitchen-sink PR. PS CI seems to fail, so splitting might help there as well.

It won't be a kitchen sink PR if we merge in the parent PRs first.

This reverts commit 647d3c0.

This reverts commit 8c38e45.

…ether

varunagrawal · 2023-07-17T19:58:48Z

@dellaert I split the changes to TableFactor into #1580.

dellaert

Some comments. This PR changed too many things at once to be sure of anything, though.

dellaert · 2023-07-18T13:23:19Z

gtsam/discrete/DecisionTree.h

@@ -299,6 +299,42 @@ namespace gtsam {
    /// Return the number of leaves in the tree.
    size_t nrLeaves() const;

+    /**
+     * @brief This is a convenience function which returns the total number of


Spelling. And why are we adding it ? And why is the implementation recursive.
I would just as well delete it unless it has a purpose.

I'm a bit lost on which word is misspelled. The purpose is to help with testing and ensure correctness as a convenience method.

dellaert · 2023-07-18T13:23:39Z

gtsam/hybrid/tests/testHybridBayesNet.cpp

@@ -231,7 +231,7 @@ TEST(HybridBayesNet, Pruning) {
  auto prunedTree = prunedBayesNet.evaluate(delta.continuous());

  // Regression test on pruned logProbability tree
-  std::vector<double> pruned_leaves = {0.0, 20.346113, 0.0, 19.738098};
+  std::vector<double> pruned_leaves = {0.0, 32.713418, 0.0, 31.735823};


What's happening here? Why are regressions changing ?

The decision tree is normalizing the values based on the introduced zeros from pruning. Since I changed the way we're pruning (on the joint rather than the conditionals), the normalizing factor has changed.

dellaert · 2023-07-18T13:24:17Z

gtsam/hybrid/tests/testHybridBayesNet.cpp

  double density = exp(logProbability);
-  EXPECT_DOUBLES_EQUAL(density, actualTree(discrete_values), 1e-9);
+  EXPECT_DOUBLES_EQUAL(density,


This was not a regression but suddenly there is an arbitrary mult factor here?

That is the multiplicative factor for the different normalization constant.

dellaert · 2023-07-18T13:25:16Z

gtsam/hybrid/tests/testMixtureFactor.cpp

@@ -63,8 +63,8 @@ TEST(MixtureFactor, Printing) {
      R"(Hybrid [x1 x2; 1]
 MixtureFactor
 Choice(1) 
- 0 Leaf Nonlinear factor on 2 keys
- 1 Leaf Nonlinear factor on 2 keys
+ 0 Leaf [1]Nonlinear factor on 2 keys


spacing is weird in this case.

dellaert

OK, merge this at will :-)

varunagrawal requested a review from dellaert July 12, 2023 16:08

varunagrawal self-assigned this Jul 12, 2023

varunagrawal changed the title ~~TableFactor for Hybrid Elimination~~ Hybrid Elimination Improvements Jul 16, 2023

varunagrawal force-pushed the hybrid-tablefactor branch from 7a3aa89 to 2b72f75 Compare July 17, 2023 13:15

varunagrawal force-pushed the hybrid-tablefactor-2 branch from f3c14a0 to e47531e Compare July 17, 2023 13:22

varunagrawal force-pushed the hybrid-tablefactor-2 branch from e47531e to 66f060c Compare July 17, 2023 14:58

Base automatically changed from hybrid-tablefactor to develop July 17, 2023 15:54

varunagrawal added 15 commits July 17, 2023 13:00

small improvements

f6d42d0

small cleanup

53d0086

add separate Hybrid ISAM and Smoother tests

6d69ca1

overload apply method in DecisionTreeFactor

f751a5b

fix continuousKeySet method

4e902fc

simplify HybridEliminate

4e13fb7

initial changes

f6b1872

discreteConditionals returns DiscreteConditional

2940e69

Revert "remove nrAssignments from the DecisionTree"

2f4133f

This reverts commit 647d3c0.

Revert "enumerate all assignments for computing probabilities to prune"

ddb36c2

This reverts commit 8c38e45.

Undo TableFactor return in Hybrid GFG so we can group the changes tog…

f4adfac

…ether

templetize functions in Switching fixture

6a26ecf

small improvements to comments and code structure

f707129

prune joint discrete probability which is faster

3fe2682

fix tests

2f3fcff

varunagrawal force-pushed the hybrid-tablefactor-2 branch from 0fc5ce9 to 2f3fcff Compare July 17, 2023 17:00

include fix

103641c

dellaert reviewed Jul 18, 2023

View reviewed changes

Fix spacing in DecisionTree::print

cb084b3

dellaert approved these changes Jul 19, 2023

View reviewed changes

varunagrawal merged commit ba7c077 into develop Jul 19, 2023
26 checks passed

varunagrawal deleted the hybrid-tablefactor-2 branch July 19, 2023 10:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Hybrid Elimination Improvements #1575

Hybrid Elimination Improvements #1575

varunagrawal commented Jul 12, 2023 •

edited

Loading

varunagrawal commented Jul 14, 2023

dellaert commented Jul 16, 2023

varunagrawal commented Jul 16, 2023

dellaert commented Jul 17, 2023

varunagrawal commented Jul 17, 2023

varunagrawal commented Jul 17, 2023

varunagrawal commented Jul 17, 2023

dellaert left a comment

dellaert Jul 18, 2023

varunagrawal Jul 18, 2023

dellaert Jul 18, 2023

varunagrawal Jul 18, 2023

dellaert Jul 18, 2023

varunagrawal Jul 18, 2023

dellaert Jul 18, 2023

varunagrawal Jul 18, 2023

dellaert left a comment

Hybrid Elimination Improvements #1575

Hybrid Elimination Improvements #1575

Conversation

varunagrawal commented Jul 12, 2023 • edited Loading

varunagrawal commented Jul 14, 2023

Before

After

dellaert commented Jul 16, 2023

varunagrawal commented Jul 16, 2023

dellaert commented Jul 17, 2023

varunagrawal commented Jul 17, 2023

varunagrawal commented Jul 17, 2023

varunagrawal commented Jul 17, 2023

dellaert left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dellaert left a comment

Choose a reason for hiding this comment

varunagrawal commented Jul 12, 2023 •

edited

Loading