diff --git a/notebooks/advanced_heuristics/index.md b/notebooks/advanced_heuristics/index.md new file mode 100644 index 00000000..ac93f08a --- /dev/null +++ b/notebooks/advanced_heuristics/index.md @@ -0,0 +1,233 @@ +# Advanced Heuristics +#### Kahbod Aeini, Mohammadreza Daviran and Sepehr Kianian + +## Contents +[Introduction](https://github.com/mmdrez4/notes/blob/master/notebooks/advanced_heuristics/index.md#introduction) + +* [Distance](https://github.com/mmdrez4/notes/blob/master/notebooks/advanced_heuristics/index.md#distance) + +* [Heuristic](https://github.com/mmdrez4/notes/blob/master/notebooks/advanced_heuristics/index.md#heuristic) + +* [Monotonicity implies admissibility](https://github.com/mmdrez4/notes/blob/master/notebooks/advanced_heuristics/index.md#monotonicity-implies-admissibility) + +* [Monotonic heuristic function](https://github.com/mmdrez4/notes/blob/master/notebooks/advanced_heuristics/index.md#monotonic-heuristic-function) + +* [Heuristic dominance](https://github.com/mmdrez4/notes/blob/master/notebooks/advanced_heuristics/index.md#heuristic-dominance) + +* [Relaxing](https://github.com/mmdrez4/notes/blob/master/notebooks/advanced_heuristics/index.md#relaxing) + +* [Pattern DB](https://github.com/mmdrez4/notes/blob/master/notebooks/advanced_heuristics/index.md#pattern-db) + +* [Combining Pattern DB](https://github.com/mmdrez4/notes/blob/master/notebooks/advanced_heuristics/index.md#combining-pattern-db) + +* [Disjoint Pattern DB](https://github.com/mmdrez4/notes/blob/master/notebooks/advanced_heuristics/index.md#disjoint-pattern-db) + +[Conclusion](https://github.com/mmdrez4/notes/blob/master/notebooks/advanced_heuristics/index.md#conclusion) + +[References](https://github.com/mmdrez4/notes/blob/master/notebooks/advanced_heuristics/index.md#references) + +## Introduction +Heuristics are methods for solving problems in a fast way. Actually It's about using a practical method that doesn’t necessarily need to be perfect! Heuristic methods speed up the process of reaching a satisfactory solution. +In this lecture we review some prerequisites then we present a couple of approaches and Heuristics, which seeked to be flawless, but the space and time complexity of these algorithms is not sufficient. So we try to improve the Heuristics to eliminate the recent obstacles and make our heuristics more efficient. we call these methods advanced heuristics! +We'll show how to detect complex problems then solving them by advanced heuristics and compare them with the previous methods. + +### Distance +First we review some basic definitions: + +Distance is a numerical measurement of how far apart objects or points are: +* [Euclidean Distance](https://en.wikipedia.org/wiki/Euclidean_distance) calculates the distance between two real-valued vectors. +* [Manhattan Distance](https://en.wikipedia.org/wiki/Taxicab_geometry) is sum of the absolute differences of their Cartesian coordinates. +
+ +
+ ++ +
+ +Now we define **f(n)** function as **f(n) = h(n) + g(n)** where g(n) is sum of costs from root to n node. + +***Monotonicity or Consistency is that the heuristic function holds in triangle inequality.*** Namely **f(n) is never Decreasing.** ++ +
+A heuristic is consistent if: h(n) ≤ c(n, a, n’) + h(n’) + +if h is consistent, we have: f(n’) = g(n’) + h (n’) = g(n) + c(n, a, n’) + h(n’) ≥ g(n) + h(n) [= f(n)] + +I.e., f(n) is nondecreasing along any path. + +Effect of **Monotonicity** on a Heuristic is shown in the below schema: ++ +
+ +for more information and better understanding watch this video: [watch](https://www.youtube.com/watch?v=CJmlP03ik5g) + ++ +
+ h(n1) ≤ c(n1 ,a1 , n2) + h(n2) + ≤ c(n1 ,a1 , n2) + c(n1 ,a1 , n3) + h(n3) ≤ + +… + ≤ Σ c(n1 ,a1 , ni+1) + 0 ⇒ h(n1) ≤ cost of (every) path from n1 to goal ≤ cost of optimal path from n1 to goal + +Now we want to show an **inconsistent, admissible example!** +So consider this figure: + ++ +
+ +If our heuristic is admissible, we have that **h(n) ≤ h*(n)** for every node n where **h*** is the real cost to the goal. So we deduct that **h(A) ≤ 4**, **h(C) ≤ 3** and **h(G) ≤ 0**. + +If we want our heuristic to be *Consistent* we should have **h(G) = 0** and **h(n) ≤ cost(n, c) + h(c)** so in our case we have **h(A) ≤ 1 + h(C)** and **h(C) ≤ 3 + h(G) = 3** + +Because of the *Admissibility* **h(C) should be less than 3**, but if **h(A) > 1 + h(C)** then our heuristic is *Inconsistent!*. Hence if we assume that **h(C) = 1**, **h(G) = 0** and **h(A) = 4** our heuristic is *Admissible but Inconsistent!* + ++ +
+ + + +**ℏ(n’) = max(h(n’), ℏ(n) - c(n, a, n’))** + + +Above Heuristic is defined Recursively. ++ +
+ +**ℏ(n’) ≥ ℏ(n) - c(n, a, n')** + +So obviously we have **ℏ(n’) + c(n, a, n') ≥ ℏ(n)** + ++ +
+ +Hamiltonian path is a rooted tree which every its inner has have exactly one son. Hence, the problem is converting the above graph to a 1-ary tree, which covers all nodes of the graph with existing edges and minimum sum of edges' weight. Now we ignore being 1-ary tree constraint to convert original problem to a more general one. Accordingly our problem is now a Minimum Spaning Tree problem which is soluable in n2 time complexity. + ++ +
+ ++ +
+ ++ +
+ +As you can see in this problem we have an undirected graph, which each of its nodes is a florist shop and has some flower breeds. Two brothers should march from their home towards their mother's place in a way that has the least cost and they met every existing flower breeds. In this case we choose a subset of flower breeds S that reach the goal_node(mother's place) by a specific pattern and set goal_state as satisfying path with the least cost and meeting every existing flower breeds of set S. Hitherto we used pure Pattern DB. + +Now we can have couple of flower breeds subset and combine obtained answers by them. For example if we have K flower breeds, we can solve the problem for each flower breed and calculate its heuristic function and then choose maximum of the functions. So we used Combining Pattern DB to achieve a better heuristic function. + +## Conclusion +At first we were facing with problems and previous Uninformed methods, which solved those problems with exponential time and space complexity. Therefore we introduced some new methods, namely Informed Methods, which used a heuristic function to solve those problems more efficient. Later we introduced some efficient ways to make our heuristic closer to the real answer. some of them were to make our heuristic monotonic and Dominant, or to make a dominant heuristic based on some heuristics we had. Some advanced ways were even for more basic problem, namely how to create a heuristic function for a problem. these included Relaxing problems and Patternal DBs. We then Entered into the Patternal DBs And introduced ways to improve them a lot. despite all these ways, there are so many problems that aren't solvable by even these ways, and there are so many ways ahead of us that you have to learn & use further. thanks for reading all of these concepts and we wish you have a great future. a bid farewell. + +## References +https://machinelearningmastery.com/distance-measures-for-machine-learning/ + +https://artint.info/2e/html/ArtInt2e.Ch3.S6.SS2.html + +https://en.wikipedia.org/wiki/Consistent_heuristic + +https://courses.cs.washington.edu/courses/cse473/12sp/slides/04-heuristics.pdf diff --git a/notebooks/advanced_heuristics/matadata.yml b/notebooks/advanced_heuristics/matadata.yml new file mode 100644 index 00000000..e0e98a0e --- /dev/null +++ b/notebooks/advanced_heuristics/matadata.yml @@ -0,0 +1,38 @@ +title: LN | Advanced Heuristics # shown on browser tab + +header: + title: Advanced Heuristics # title of your notebook + description: making heuristics better and more efficient # short description of your notebook + +authors: + label: + position: top + text: Authors + kind: people + content: + # list of notebook authors + - name: Mohammadreza Daviran + role: Author + contact: + - link: https://github.com/mmdrez4 + icon: fab fa-github + - link: mailto:mohammadreza.dn80@gmail.com + icon: fas fa-envelope + + + - name: Kahbod Aeini + role: Author + contact: + - link: https://github.com/kahbodaeini + icon: fab fa-github + + + - name: Sepehr Kianian + role: Author + contact: + - link: https://github.com/Sepehrkianian09 + icon: fab fa-github + +comments: + label: false + kind: comments diff --git a/notebooks/advanced_heuristics/pic/Combining-Pattern-DB.jpg b/notebooks/advanced_heuristics/pic/Combining-Pattern-DB.jpg new file mode 100644 index 00000000..728a69af Binary files /dev/null and b/notebooks/advanced_heuristics/pic/Combining-Pattern-DB.jpg differ diff --git a/notebooks/advanced_heuristics/pic/Pattern-Db.jpg b/notebooks/advanced_heuristics/pic/Pattern-Db.jpg new file mode 100644 index 00000000..cd825742 Binary files /dev/null and b/notebooks/advanced_heuristics/pic/Pattern-Db.jpg differ diff --git a/notebooks/advanced_heuristics/pic/admissible.png b/notebooks/advanced_heuristics/pic/admissible.png new file mode 100644 index 00000000..6b4dac09 Binary files /dev/null and b/notebooks/advanced_heuristics/pic/admissible.png differ diff --git a/notebooks/advanced_heuristics/pic/c.png b/notebooks/advanced_heuristics/pic/c.png new file mode 100644 index 00000000..c9302766 Binary files /dev/null and b/notebooks/advanced_heuristics/pic/c.png differ diff --git a/notebooks/advanced_heuristics/pic/consistency.png b/notebooks/advanced_heuristics/pic/consistency.png new file mode 100644 index 00000000..dbd44adb Binary files /dev/null and b/notebooks/advanced_heuristics/pic/consistency.png differ diff --git a/notebooks/advanced_heuristics/pic/distance.png b/notebooks/advanced_heuristics/pic/distance.png new file mode 100644 index 00000000..4ce0b1d4 Binary files /dev/null and b/notebooks/advanced_heuristics/pic/distance.png differ diff --git a/notebooks/advanced_heuristics/pic/example.png b/notebooks/advanced_heuristics/pic/example.png new file mode 100644 index 00000000..dd3e4e3e Binary files /dev/null and b/notebooks/advanced_heuristics/pic/example.png differ diff --git a/notebooks/advanced_heuristics/pic/hamilton path.png b/notebooks/advanced_heuristics/pic/hamilton path.png new file mode 100644 index 00000000..aa6e385d Binary files /dev/null and b/notebooks/advanced_heuristics/pic/hamilton path.png differ diff --git a/notebooks/advanced_heuristics/pic/heuristic proof.png b/notebooks/advanced_heuristics/pic/heuristic proof.png new file mode 100644 index 00000000..91454dfc Binary files /dev/null and b/notebooks/advanced_heuristics/pic/heuristic proof.png differ diff --git a/notebooks/advanced_heuristics/pic/monotonic.png b/notebooks/advanced_heuristics/pic/monotonic.png new file mode 100644 index 00000000..87e7e623 Binary files /dev/null and b/notebooks/advanced_heuristics/pic/monotonic.png differ diff --git a/notebooks/advanced_heuristics/pic/pattern.png b/notebooks/advanced_heuristics/pic/pattern.png new file mode 100644 index 00000000..1e160af9 Binary files /dev/null and b/notebooks/advanced_heuristics/pic/pattern.png differ diff --git a/notebooks/advanced_heuristics/pic/proof.png b/notebooks/advanced_heuristics/pic/proof.png new file mode 100644 index 00000000..f6d45b54 Binary files /dev/null and b/notebooks/advanced_heuristics/pic/proof.png differ