From c0023009475e16151ff8079ea98658648b6f3040 Mon Sep 17 00:00:00 2001 From: Moritz Schmidt Date: Mon, 16 Sep 2024 12:16:38 +0200 Subject: [PATCH] Remove duplicate documentation Some parts of the usage section in the procedural knowledge learning chapter of the Soar manual were not included in the cli reference, thus they were moved. Discussion see #41 --- docs/reference/cli/cmd_chunk.md | 12 + docs/reference/cli/cmd_explain.md | 43 ++ docs/reference/cli/cmd_visualize.md | 10 +- .../04_ProceduralKnowledgeLearning.md | 614 +----------------- 4 files changed, 76 insertions(+), 603 deletions(-) diff --git a/docs/reference/cli/cmd_chunk.md b/docs/reference/cli/cmd_chunk.md index 22414b4b..82636d04 100644 --- a/docs/reference/cli/cmd_chunk.md +++ b/docs/reference/cli/cmd_chunk.md @@ -118,6 +118,18 @@ This feature is not yet implemented. ## Preventing Possible Correctness Issues +It is theoretically possible to detect nearly all of the sources of correctness +issues and prevent rules from forming when those situations are detected. In +Soar 9.6.0, though, only one filter is available, `allow-local-negations`. Future +versions of Soar will include more correctness filters. + +Note that it is still possible to detect that your agent may have encountered a +known source of a correctness issue by looking at the output of the chunk +stats command. It has specific statistics for some of the sources, while others +can be gleaned indirectly. For example, if the stats show that some rules +required repair, you know that your agent testing or augmenting a previous +result in a substate. + ### chunk allow-local-negations The option `allow-local-negations` control whether or not chunks can be created diff --git a/docs/reference/cli/cmd_explain.md b/docs/reference/cli/cmd_explain.md index 4390810c..d09c5390 100644 --- a/docs/reference/cli/cmd_explain.md +++ b/docs/reference/cli/cmd_explain.md @@ -329,6 +329,49 @@ explainer will write out a file with the statistics when either Soar exits or a `soar init` is executed. This option is still considered experimental and in beta. +## Explaining Learned Procedural Knowledge + +While explanation-based chunking makes it easier for people to now incorporate +learning into their agents, the complexity of the analysis it performs makes it +far more difficult to understand how the learned rules were formed. The +explainer is a new module that has been developed to help ameliorate this +problem. The explainer allows you to interactively explore how rules were +learned. + +When requested, the explainer will make a very detailed record of everything +that happened during a learning episode. Once a user specifies a recorded chunk +to "discuss", they can browse all of the rule firings that contributed to the +learned rule, one at a time. The explainer will present each of these rules with +detailed information about the identity of the variables, whether it tested +knowledge relevant to the the superstate, and how it is connected to other rule +firings in the substate. Rule firings are assigned IDs so that user can quickly +choose a new rule to examine. + +The explainer can also present several different screens that show more verbose +analyses of how the chunk was created. Specifically, the user can ask for a +description of (1) the chunk’s initial formation, (2) the identities of +variables and how they map to identity sets, (3) the constraints that the +problem-solving placed on values that a particular identity can have, and (4) +specific statistics about that chunk, such as whether correctness issues were +detected or whether it required repair to make it fully operational. + +Finally, the explainer will also create the data necessary to visualize all of +the processing described in an image using the new ’visualize’ command. These +visualization are the easiest way to quickly understand how a rule was formed. + +Note that, despite recording so much information, a lot of effort has been put +into minimizing the cost of the explainer. When debugging, we often let it +record all chunks and justifications formed because it is efficient enough to do +so. + +Use the explain command without any arguments to display a summary of which rule +firings the explainer is watching. It also shows which chunk or justification +the user has specified is the current focus of its output, i.e. the chunk being +discussed. + +Tip: This is a good way to get a chunk id so that you don’t have to type or +paste in a chunk name. + ## Visualizing an Explanation Soar's `visualize` command allows you to create images that represent processing diff --git a/docs/reference/cli/cmd_visualize.md b/docs/reference/cli/cmd_visualize.md index 27280745..cc748d04 100644 --- a/docs/reference/cli/cmd_visualize.md +++ b/docs/reference/cli/cmd_visualize.md @@ -104,7 +104,7 @@ applies to visualizing memory systems. ## File Handling Settings `file-name` specifies the base file name that Soar will use when creating both -graphviz data files and images. You can specify a path as well, for example +Graphviz data files and images. You can specify a path as well, for example "visualization/soar_viz", but make sure the directory exists first! `use-same-file` tells the visualizer to always overwrite the same files for each @@ -114,7 +114,7 @@ command does not yet handle file creation as robustly as it could. If the file already exists, it will simply overwrite it rather than looking for a new file name. -`generate-image` specifies whether the visualizer should render the graphviz +`generate-image` specifies whether the visualizer should render the Graphviz file into an image. This setting is overridden if the viewer-launch setting is enabled. @@ -144,6 +144,12 @@ Note that your operating system chooses which program to launch based on the file type. This feature has not been tested extensively on other platforms. Certain systems may not allow Soar to launch an external program. +???+ note + For the visualizer to work, you must have Graphviz and DOT installed, which are + free third-party tools, and both must be available on your path. To date, the + visualizer has only been tested on Mac and Linux. It is possible that certain + systems may not allow Soar to launch an external program. + ## See Also - [explain](./cmd_explain.md) diff --git a/docs/soar_manual/04_ProceduralKnowledgeLearning.md b/docs/soar_manual/04_ProceduralKnowledgeLearning.md index d1829718..2ba10151 100644 --- a/docs/soar_manual/04_ProceduralKnowledgeLearning.md +++ b/docs/soar_manual/04_ProceduralKnowledgeLearning.md @@ -33,8 +33,13 @@ Note that Soar builds the chunk as soon as the result is created, rather than waiting until the impasse is resolved. While chunking is a core capability of Soar, procedural learning is disabled by -default. See section [usage](#usage) for more information about enabling and -using chunking. +default. Refer to the following CLI commands for more information about enabling +and using chunking: + +1. [chunk](../reference/cli/cmd_chunk.md) +1. [trace](../reference/cli/cmd_trace.md) +1. [explain](../reference/cli/cmd_explain.md) +1. [visualize](../reference/cli/cmd_visualize.md) ## Explanation-based Chunking @@ -382,8 +387,7 @@ created and propagated using the following rules: identities forward through the explanation trace, which allows us to represent that the variable in the condition refers to the same object as the variable in the action of the other rule. - 1.

- Any element that matches special working memory elements called singletons + 1. Any element that matches special working memory elements called singletons are assigned the same identity. Singletons are working memory elements that are guaranteed to only have a single possible value in a state. The most important singleton is the local `^superstate` singleton, which is @@ -394,10 +398,9 @@ created and propagated using the following rules: identities. While there are a variety of built-in singletons for architecturally-created WMEs, users can also specify their own domain-specific singletons to eliminate unnecessary generality when - learning. See section - [Using Singletons to Simplify a Rule's Conditions](#using-singletons-to-simplify-a-rules-conditions). - The full list of architecturally-created singletons can be found - in the [`chunk` command’s](../reference/cli/cmd_chunk.md) help entry. + learning. The full list of architecturally-created singletons can be + found in the [`chunk` command’s](../reference/cli/cmd_chunk.md) help + entry. Note that rule 1 may conflict with other rules. For example, if a variable appears in two different conditions, then two different identities may propagate @@ -565,7 +568,7 @@ would be redundant. Explanation-based chunking, though, learns based on the reasoning within the original hand-written rules. Since the reasoning behind each of the two conditions may be different even if they matched the same WME, EBC must always add both conditions. (Note that there are some exceptions. See -[using singletons to simplify rule's conditions](#using-singletons-to-simplify-a-rules-conditions) +[chunk singletons](../reference/cli/cmd_chunk.md#user-singletons) on superstate singletons and user singletons.) Negated conditions are included in a trace in the following way: when a @@ -1247,7 +1250,7 @@ conditions is true: dependent on a condition that tested whether a subgoal WME doesn’t exist. Since there is no practical way to determine why a piece of knowledge doesn’t exist, testing a local negation can result in an over-general and - incorrect chunk, cf. [prohibiting known sources for correctness issues](04_ProceduralKnowledgeLearning.md#prohibiting-known-sources-of-correctness-issues) + incorrect chunk, cf. [prohibiting known sources for correctness issues](../reference/cli/cmd_chunk.md#preventing-possible-correctness-issues) for more information. Note that correctness filters have not yet been implemented for all the identified potential sources of correctness issues. 5. The chunking option bottom-only is on and a chunk was already built in the @@ -1277,594 +1280,3 @@ conditions is true: substate. Consequently, those result WMEs will be completely dependent on the rules that fired within the substate. So, when the substate is removed, those results will also be removed. - -## Usage - -### Overview of the chunk command - -```shell -=================================================== - Chunk Commands and Settings -=================================================== -chunk ? | help Print all EBC settings -chunk stats Print statistics on learning that has occurred -------------------- Settings ---------------------- -ALWAYS | never | only | except When Soar will learn new rules -bottom-only [ on | OFF ] Learn only from bottom sub-state -naming-style [ numbered | RULE] Simple numeric chunk names or informational rule-based name -max-chunks 50 Maximum chunks that can be learned (per phase) -max-dupes 3 Maximum duplicate chunks (per rule, per phase) -------------------- Debugging --------------------- -interrupt [ on | OFF ] Stop Soar after learning from any rule -explain-interrupt [ on | OFF ] Stop Soar after learning rule watched by explainer -warning-interrupt [ on | OFF ] Stop Soar after detecting learning issue -------------------- Fine Tune --------------------- -singleton Print all WME singletons -singleton Add a WME singleton pattern -singleton -r Remove a WME singleton pattern ------------------ EBC Mechanisms ------------------ -add-ltm-links [ on | OFF ] Recreate LTM links in original results -add-osk [ ON | off ] Incorporate operator selection knowledge ----------- Correctness Guarantee Filters ---------- Allow rules to form that... -allow-local-negations [ ON | off ] ...used local negative reasoning -allow-opaque [ ON | off ] ...used knowledge from a LTM recall ---------------------------------------------------- - -To change a setting: chunk [] -For a detailed explanation of these settings: help chunk -``` - -See [chunk](../reference/cli/cmd_chunk.md) for more detailed information on the -settings. - -### Enabling Procedural Learning - -By default, explanation-based chunking is off. - -- To turn on chunking: `chunk always` -- To turn off chunking: `chunk never` - -In real world agents, there may be certain problem spaces in which you don’t -want your agent to learn rules. Chunking has a mechanism to allow agents to -dynamically specify the states in which rules are learned. - -- To turn off chunking in all states except ones manually flagged on: - - Use chunk only setting. - - Design an agent rule that executes the RHS action force-learn, which only - matches in states in which you want to learn rules. -- To turn on chunking in all states except ones manually flagged off: - - Use chunk except setting. - - Design an agent rule that executes the RHS action dont-learn, which only - matches in states in which you don’t want to learn rules. - -Depending on your agent design, you may want to consider enabling the add-osk -option. As of Soar 9.6.0, EBC does not incorporate operator selection knowledge -into learned rules by default, since there is a performance cost and not all -agents designs require its inclusion. You may want to enable this option if -your agent has rules that test knowledge in the superstate to create operator -preferences in the substate. Refer to -[learning and operator selection knowledge](#relevant-operator-selection-knowledge-tracking) -for more information. - -See [chunk command](../reference/cli/cmd_chunk.md) for more information about -using the chunk command to enable and disable procedural learning. - -### Fine-tuning What Your Agent Learns - -#### Prohibiting known sources of correctness issues - -It is theoretically possible to detect nearly all of the sources of correctness -issues and prevent rules from forming when those situations are detected. In -Soar 9.6.0, though, only one filter is available,allow-local-negations. Future -versions of Soar will include more correctness filters. - -Note that it is still possible to detect that your agent may have encountered a -known source of a correctness issue by looking at the output of the chunk -stats command. It has specific statistics for some of the sources, while others -can be gleaned indirectly. For example, if the stats show that some rules -required repair, you know that your agent testing or augmenting a previous -result in a substate. - -#### Using Singletons to Simplify a Rule’s Conditions - -Unlike previous versions of chunking, EBC adds all conditions that tested -superstate knowledge to a chunk, regardless of whether another condition -already tested that working memory element. This means that EBC can sometimes -produce learned rules with seemingly duplicate conditions. While these -conditions are logically correct, they may be redundant because the nature of -the domain may make it impossible for the two conditions to match different -working memory elements. For example, in the blocks-world domain, the fact that -there can be only one gripper in the world means that having multiple conditions -testing for a gripper would be redundant. - -Soar allows agents to specify such known domain characteristics, which EBC will -then use to create better rules that don’t have such unnecessary conditions. We -call any working memory element that is guaranteed to only have a single -possible value at any given time, a singleton. If EBC encounters two different -conditions in the backtrace that both test the same superstate WME that matches -a user singleton pattern, it will merge the two conditions. As described in -Listing 4b, there are several -architectural singleton’s that EBC already knows about. To specify patterns for -domain-specific singletons, the chunk singleton command can be used. - -See [chunk singelton](../reference/cli/cmd_chunk.md#user-singletons) for more -information about the command. - -### Examining What Was Learned - -#### Printing Rules - -- To print all chunks learned: - `print --chunks` or `print -c` - -- To print all justifications learned (and still matching): - `print --justifications` or `print -j` - -- To print a rule or justification: - `print ` - -For more information on print, see the [print command](../reference/cli/cmd_print.md). - -#### Trace Messages - -- To print when new rules are learned (just the name): - `trace --learning 1` or `trace -l 1` - -- To print when new rules are learned (the full rule): - `trace --learning 2` or `trace -l 2` - -- To print a trace of the conditions as they are collected during backtracing: - `trace --backtracing` or `trace -b` - -- To print warnings about chunking issues detected while learning: - `trace --chunk-warnings` or `trace -C` - -- To print when learned chunks match and fire: - `trace --backtracing` or `trace -b` - -For more information on trace, see the [trace command](../reference/cli/cmd_trace.md). - -Note that the most detailed information about why a particular rule was learned -can be acquired using the [explain mechanism](../reference/cli/cmd_explain.md). -That is highly recommended over printing the backtracing trace messages. - -#### Chunking Statistics - -Chunking automatically compiles various statistics about the procedural rule -learning that an agent performs. To access these stats, use the command `chunk`, -`stats` or `stats -l`. - -```shell -=========================================================================== -Explanation-Based Chunking Statistics -=========================================================================== -Substates analyzed 0 -Rules learned 0 -Justifications learned 0 - ---- - -Work Performed - -Number of rules fired 0 -Number of rule firings analyzed during backtracing 0 -Number of OSK rule firings analyzed during backtracing 0 -Number of rule firings re-visited during backtracing 0 - -Conditions merged 0 -Disjunction tests merged 0 - -- Redundant values 0 - -- Impossible values eliminated 0 - -Operational constraints 0 -Non-operational constraints detected 0 -Non-operational constraints enforced 0 - ---- - -Identity Analysis - -Identities created 0 -Distinct identities in learned rules 0 -Identity propagations 0 -Identity propagations blocked 0 -Identity propagations from local singleton 0 -Identities joined 0 - -- To unify two identities propagated into same variable 0 -- To unify two conditions that tested a superstate singleton 0 -- To connect an child result (result in rule had children WMEs) 0 - Identities literalized 0 -- Condition with variable matched a literal RHS element 0 -- Condition with variable matched a RHS function 0 -- Condition with literal value matched a RHS variable 0 -- Variable used in a RHS function 0 - ---- - -Potential Generality Issues Detected - -Rules repaired that had unconnected conditions or actions 0 -Extra conditions added during repair 0 - ---- - -Potential Correctness Issues Detected - -Chunk used negated reasoning about substate 0 -Chunk tested knowledge retrieved from long-term memory 0 -Justification used negated reasoning about substate 0 -Justification tested knowledge retrieved from long-term memory 0 - ---- - -Learning Skipped or Unsuccessful - -Ignored duplicate of existing rule 0 -Skipped because problem-solving tested ^quiescence true 0 -Skipped because no superstate knowledge tested 0 -Skipped because MAX-CHUNKS exceeded in a decision cycle 0 -Skipped because MAX-DUPES exceeded for rule this decision cycle 0 -``` - -Note that similar statistics for a specific learned rule can be acquired using -the [explain mechanism](../reference/cli/cmd_explain.md). - -#### Interrupting Execution To Examine Learning - -- To stop Soar after each successful learning episode: - -```shell -chunk interrupt on -``` - -- To stop Soar after detecting any learning issue: - -```shell -chunk warning-interrupt on -``` - -- To stop Soar after learning a rule that the explainer recorded: - -```shell -chunk explain-interrupt on -``` - -For more information about how to record when a specific rule is learned, see -the [explain mechanism](../reference/cli/cmd_explain.md). - -## Explaining Learned Procedural Knowledge - -While explanation-based chunking makes it easier for people to now incorporate -learning into their agents, the complexity of the analysis it performs makes it -far more difficult to understand how the learned rules were formed. The -explainer is a new module that has been developed to help ameliorate this -problem. The explainer allows you to interactively explore how rules were -learned. - -When requested, the explainer will make a very detailed record of everything -that happened during a learning episode. Once a user specifies a recorded chunk -to "discuss", they can browse all of the rule firings that contributed to the -learned rule, one at a time. The explainer will present each of these rules with -detailed information about the identity of the variables, whether it tested -knowledge relevant to the the superstate, and how it is connected to other rule -firings in the substate. Rule firings are assigned IDs so that user can quickly -choose a new rule to examine. - -The explainer can also present several different screens that show more verbose -analyses of how the chunk was created. Specifically, the user can ask for a -description of (1) the chunk’s initial formation, (2) the identities of -variables and how they map to identity sets, (3) the constraints that the -problem-solving placed on values that a particular identity can have, and (4) -specific statistics about that chunk, such as whether correctness issues were -detected or whether it required repair to make it fully operational. - -Finally, the explainer will also create the data necessary to visualize all of -the processing described in an image using the new ’visualize’ command. These -visualization are the easiest way to quickly understand how a rule was formed. - -Note that, despite recording so much information, a lot of effort has been put -into minimizing the cost of the explainer. When debugging, we often let it -record all chunks and justifications formed because it is efficient enough to do -so. - -Use the explain command without any arguments to display a summary of which rule -firings the explainer is watching. It also shows which chunk or justification -the user has specified is the current focus of its output, i.e. the chunk being -discussed. - -Tip: This is a good way to get a chunk id so that you don’t have to type or -paste in a chunk name. - -```shell -======================================================= - Explainer Summary -======================================================= -Watch all chunk formations Yes -Explain justifications Nof -Number of specific rules watched 0 - -Chunks available for discussion: chunkx2*apply2 (c 14) - chunk*apply*o (c 13) - chunkx2*apply2 (c 12) - chunk*apply*d (c 11) - chunkx2*apply2 (c 6) - chunk*apply* (c 15) - chunkx2*apply (c 8) - chunk*apply*c (c 5) - chunkx2*apply (c 10) - chunk*apply (c 1) - -* Note: Printed the first 10 chunks. 'explain list' to see other 6 chunks. - -Current chunk being discussed: chunk*apply*down-gripper(c 3) -``` - -### `explain chunk [ | ]` - -This command starts the explanation process by specifying which chunk’s -explanation trace you want to explore. - -Tip: Use the alias to quickly start discussing a chunk, for example: - -```shell -soar % c 3 -Now explaining chunk*apply*move-gripper-above*pass*top-state*OpNoChange*t6-1. -- Note that future explain commands are now relative - to the problem-solving that led to that chunk. - -Explanation Trace Using variable identity IDs Shortest Path to Result Instantiation - -sp {chunk*apply*move-gripper-above*pass*top-state*OpNoChange*t6-1 -1: ( ^top-state ) ([140] ^top-state [162]) - -{ -2: ( ^operator ) ([140] ^operator [141]) -3: ( ^name evaluate-operator) ([141] ^name evaluate-operator) - } -4: ( ^gripper ) ([162] ^gripper [156]) i 30 -> i 31 -5: ( ^position up) ([156] ^position up) i 30 -> i 31 -6: ( ^holding nothing) ([156] ^holding nothing) i 30 -> i 31 -7: ( ^above ) ([156] ^above [157]) i 30 -> i 31 -8: ( ^io ) ([162] ^io [163]) i 31 -9: ( ^output-link ) ([163] ^output-link [164]) i 31 -10: ( ^gripper ) ([164] ^gripper [165]) i 31 -11: ( ^clear { <> }) ([162] ^clear { <>[161] [161] }) i 30 -> i 31 -12: ( ^operator ) ([140] ^operator [149]) -13: ( ^moving-block ) ([149] ^moving-block [161]) -14: ( ^name pick-up) ([149] ^name pick-up) - --> -1: ( ^command move-gripper-above +) ([165] ^command move-gripper-above +) -2: ( ^destination +) ([165] ^destination [161] +) -} -``` - -Once you specify a rule to explain, this will be one of the first commands you -issue.explain formation provides an explanation of the initial rule that fired -which created a result. This is what is called the ‘base instantiation’ and is -what led to the chunk being learned. Other rules may also be base instantiations -if they previously created children of the base instantiation’s results. They -also will be listed in the initial formation output. soar % explain formation - -```shell -The formation of chunk ’chunk*apply*move-gripper-above*pass*top-state*OpNoChange*t6-1’ (c 1) - -Initial base instantiation (i 31) that fired when apply*move-gripper-above*pass*top-state matched at level 3 at time 6: -Explanation trace of instantiation # 31 (match of rule apply*move-gripper-above*pass*top-state at level 3) -(produced chunk result) -Identities instead of variables Operational Creator -1: ( ^operator ) ([159] ^operator [160]) No i 30 (pick-up*propose*move-gripper-above) -2: ( ^name move-gripper-above) ([160] ^name move-gripper-above) No i 30 (pick-up*propose*move-gripper-above) -3: ( ^destination ) ([160] ^destination [161]) No i 30 (pick-up*propose*move-gripper-above) -4: ( ^top-state ) ([159] ^top-state [162]) No i 27 (elaborate*state*top-state) -5: ( ^io ) ([162] ^io [163]) Yes Higher-level Problem Space -6: ( ^output-link ) ([163] ^output-link [164]) Yes Higher-level Problem Space -7: ( ^gripper ) ([164] ^gripper [165]) Yes Higher-level Problem Space ---> -1: ( ^command move-gripper-above +) ([165] ^command move-gripper-above +) -2: ( ^destination +) ([165] ^destination [161] +) - ---- - -This chunk summarizes the problem-solving involved in the following 5 -rule firings: -i 27 (elaborate*state*top-state) -i 28 (elaborate*state*operator*name) -i 29 (pick-up*elaborate*desired) -i 30 (pick-up*propose*move-gripper-above)i 31 (apply*move-gripper-above*pass*top-state) -``` - -### `explain instantiation ` - -This command prints a specific instantiation in the explanation trace. This lets -you browse the instantiation graph one rule at a time. This is probably one of -the most common things you will do while using the explainer. - -Tip: Use the alias `i ` to quickly view an instantiation, for -example: - -```shell -soar % i 30 -Explanation trace of instantiation # 30 (match of rule pick-up*propose*move-gripper-above at level 3) - -- Shortest path to a result: i 30 -> i 31 - Identities instead of variables Operational Creator - 1: ( ^name pick-up) ([152] ^name pick-up) No i 28 (elaborate*state*operator*name) - 2: ( ^desired ) ([152] ^desired [153]) No i 29 (pick-up*elaborate*desired) - 3: ( ^moving-block ) ([153] ^moving-block [154]) No i 29 (pick-up*elaborate*desired) - 4: ( ^top-state ) ([152] ^top-state [155]) No i 27 (elaborate*state*top-state) - 5: ( ^clear ) ([155] ^clear [154]) Yes Higher-level Problem Space - 6: ( ^gripper ) ([155] ^gripper [156]) Yes Higher-level Problem Space - 7: ( ^position up) ([156] ^position up) Yes Higher-level Problem Space - 8: ( ^holding nothing) ([156] ^holding nothing) Yes Higher-level Problem Space - 9: ( ^above { <> }) ([156] ^above { <>[154] [157] }) Yes Higher-level Problem Space - --> - 1: ( ^operator +) ([152] ^operator [158] +) - 2: ( ^name move-gripper-above +) ([158] ^name move-gripper-above +) - 3: ( ^destination +) ([158] ^destination [154] +) -``` - -### `explain [explanation-trace | wm-trace]` - -In most cases, users spend most of their time browsing the explanation trace. -This is where chunking learns most of the subtle relationships that you are -likely to be debugging. But users will also need to examine the working memory -trace to see the specific values matched. - -To switch between traces, you can use the `explain e` and the `explain w` commands. - -Tip: Use `et` and `wt`, which are aliases to the above two commands, to quickly -switch between traces. - -```shell -soar % explain w -Working memory trace of instantiation # 30 (match of rule pick-up*propose*move-gripper-above at level 3) -1: (S9 ^name pick-up) No i 28 (elaborate*state*operator*name) -2: (S9 ^desired D6) No i 29 (pick-up*elaborate*desired) -3: (D6 ^moving-block B3) No i 29 (pick-up*elaborate*desired) -4: (S9 ^top-state S1) No i 27 (elaborate*state\*top-state) -5: (S1 ^clear B3) Yes Higher-level Problem Space -6: (S1 ^gripper G2) Yes Higher-level Problem Space -7: (G2 ^position up) Yes Higher-level Problem Space -8: (G2 ^holding nothing) Yes Higher-level Problem Space -9: (G2 ^above { <> B3 T1 }) Yes Higher-level Problem Space ---> -1: (S9 ^operator O9) + -2: (O9 ^name move-gripper-above) + -3: (O9 ^destination B3) + -``` - -### `explain constraints` - -This feature lists all constraints found in non-operational constraints of the -explanation trace. If these constraints were not met, the problem-solving would -not have occurred. - -This feature is not yet implemented. You can use explain stats to see if any -transitive constraints were added to a particular chunk. - -### `explain identity` - -explain identity will show the mappings from variable identities to identity -sets. If available, the variable in a chunk that an identity set maps to will -also be displayed. - -By default, only identity sets that appear in the chunk will be displayed in the -identity analysis. To see the identity set mappings for other sets, change the -only-chunk-identities setting to `off`. - -```shell -# soar % explain identity - -- # Variablization Identity to Identity Set Mappings - - --== NULL Identity Set ==- - -The following variable identities map to the null identity set and will -not be generalized: 282 301 138 291 355 336 227 309 328 318 128 218 345 - --== How variable identities map to identity sets ==- - -Variablization IDs Identity CVar Mapping Type - -Instantiation 36: -125 -> 482 | IdSet 12 | | New identity set -126 -> 493 | IdSet 11 | | New identity set -Instantiation 38: -Instantiation 41: -146 -> 482 | IdSet 12 | | New identity set -147 -> 493 | IdSet 11 | | New identity set -Instantiation 42: -151 -> 180 | IdSet 1 | | New identity set -149 -> 482 | IdSet 12 | | New identity set -150 -> 493 | IdSet 11 | | New identity set -307 -> 180 | IdSet 1 | | Added to identity set -187 -> 180 | IdSet 1 | | Added to identity set -334 -> 180 | IdSet 1 | | Added to identity set -173 -> 180 | IdSet 1 | | Added to identity set -280 -> 180 | IdSet 1 | | Added to identity set -Instantiation 53: -219 -> 489 | IdSet 15 | | New identity set -Instantiation 61: -Instantiation 65: -319 -> 492 | IdSet 20 | | New identity set -``` - -### `explain stats` - -Explain’s `stat` command prints statistics about the specific chunk being -discussed. This is a good way to see whether any generality or correctness -issues were detected while learning that rule. - -```shell -=========================================================== -Statistics for ’chunk*apply*move-gripper-above*pass*top-state*OpNoChange*t6-1’ (c 1): -=========================================================== -Number of conditions 14 -Number of actions 2 -Base instantiation i 31 (apply*move-gripper-above*pass\*top-state) - -=========================================================== -Generality and Correctness -=========================================================== - -Tested negation in local substate No -LHS required repair No -RHS required repair No -Was unrepairable chunk No - -=========================================================== -Work Performed -=========================================================== -Instantiations backtraced through 5 -Instantiations skipped 6 -Constraints collected 1 -Constraints attached 0 -Duplicates chunks later created 0 -Conditions merged 2 -``` - -After-Action Reports The explainer has an option to create text files that -contain statistics about the rules learned by an agent during a particular -run. When enabled, the explainer will write out a file with the statistics when -either Soar exits or a `soar init` is executed. This option is still considered -experimental and in beta. - -![A colored visualization of an explanation trace](Images/chunking-trace-identity.png) - -## Visualizing the Explanation - -The `visualize` command can generate two graphical representations of the -analysis that chunking performed to learn a rule. While the explainer provides -more data, these images are the easiest and most effective ways to quickly -understand how a chunk was formed, especially for particularly complex chunks. -The visualizer can create two types of chunking-related images: - -1. An image that shows the entire instantiation graph at once and how it - contributed to the learned rule. Use the command `visualize ebc_analysis` to - create a very informative graph that shows all rules that fired in a substate - with arrows that indicate dependencies between actions in one rule and - conditions in others. In addition to all of the dependencies between the rules - that fired, this visualization also shows which conditions in the instantiations - tested knowledge in the superstate and hence became the basis for a condition - in the final learned rule. Finally, the individual elements in the - explanation are color-coded to show which variables share the same identity. -2. An image that shows the graph of how variable identities were combined. Use - the `visualize identity_graph` command to create a graph that shows how - identities were used to determine the variablization of a learned rule. This - shows all identities found in the chunk and how the identity analysis joined - them based on the problem-solving that occurred. This can be useful in - determining why two elements were assigned the same variable. - -Note that Soar will automatically attempt to launch a viewer to see the image -generated. If you have an editor that can open Graphviz files, you can have Soar -launch that automatically as well. (Such editors allow you to move things around -and lay out the components of the image exactly as you want them.) Your -operating system chooses which program to launch based on the file type. - -???+ note - For the visualizer to work, you must have Graphviz and DOT installed, which are - free third-party tools, and both must be available on your path. To date, the - visualizer has only been tested on Mac and Linux. It is possible that certain - systems may not allow Soar to launch an external program.