Othello_GPT_Circuits.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "\n",
    "<b style=\"color: red\">To use this notebook, go to Runtime > Change Runtime Type and select GPU as the hardware accelerator.</b>\n",
    "\n",
    "This is code to accompany my blog post building on Kenneth Li et al's paper Emergent World Representations. I found that the network actually learns a **linear** world model, but in terms of whether a cell contains a piece of **my colour** vs the **opponent's colour**. I demonstrate how to use and intervene with the linear probe I found, use the probe to start interpreting the model and studying circuits, and some starter code for neuron interpretability and activation patching\n",
    "\n",
    "If you're new to mechanistic interpretability, check out [my blog post on getting started](https://neelnanda.io/getting-started). This notebook heavily uses my TransformerLens library, check out [the main tutorial for a better introduction](https://neelnanda.io/transformer-lens-demo).\n",
    "\n",
    "Read the blog post here: https://neelnanda.io/othello\n",
    "\n",
    "Look up unfamiliar terms here: https://neelnanda.io/glossary \n",
    "\n",
    "The paper: https://arxiv.org/pdf/2210.13382.pdf\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Setup (Don't Read This)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running as a Jupyter notebook - intended for development only!\n"
     ]
    }
   ],
   "source": [
    "# Janky code to do different setup when run in a Colab notebook vs VSCode\n",
    "DEVELOPMENT_MODE = False\n",
    "try:\n",
    "    import google.colab\n",
    "    IN_COLAB = True\n",
    "    print(\"Running as a Colab notebook\")\n",
    "    %pip install transformer_lens==1.2.1\n",
    "    %pip install git+https://github.com/neelnanda-io/neel-plotly\n",
    "    \n",
    "    # PySvelte is an unmaintained visualization library, use it as a backup if circuitsvis isn't working\n",
    "    # # Install another version of node that makes PySvelte work way faster\n",
    "    # !curl -fsSL https://deb.nodesource.com/setup_16.x | sudo -E bash -; sudo apt-get install -y nodejs\n",
    "    # %pip install git+https://github.com/neelnanda-io/PySvelte.git\n",
    "except:\n",
    "    IN_COLAB = False\n",
    "    print(\"Running as a Jupyter notebook - intended for development only!\")\n",
    "    from IPython import get_ipython\n",
    "\n",
    "    ipython = get_ipython()\n",
    "    # Code to automatically update the HookedTransformer code as its edited without restarting the kernel\n",
    "    ipython.magic(\"load_ext autoreload\")\n",
    "    ipython.magic(\"autoreload 2\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using renderer: colab\n"
     ]
    }
   ],
   "source": [
    "# Plotly needs a different renderer for VSCode/Notebooks vs Colab argh\n",
    "import plotly.io as pio\n",
    "if IN_COLAB or not DEVELOPMENT_MODE:\n",
    "    pio.renderers.default = \"colab\"\n",
    "else:\n",
    "    pio.renderers.default = \"notebook_connected\"\n",
    "print(f\"Using renderer: {pio.renderers.default}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import stuff\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import torch.optim as optim\n",
    "import numpy as np\n",
    "import einops\n",
    "from fancy_einsum import einsum\n",
    "import tqdm.auto as tqdm\n",
    "import random\n",
    "from pathlib import Path\n",
    "import plotly.express as px\n",
    "from torch.utils.data import DataLoader\n",
    "\n",
    "from typing import List, Union, Optional\n",
    "from functools import partial\n",
    "import copy\n",
    "\n",
    "import itertools\n",
    "from transformers import AutoModelForCausalLM, AutoConfig, AutoTokenizer\n",
    "import dataclasses\n",
    "import datasets\n",
    "from IPython.display import HTML"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import transformer_lens\n",
    "import transformer_lens.utils as utils\n",
    "from transformer_lens.hook_points import (\n",
    "    HookedRootModule,\n",
    "    HookPoint,\n",
    ")  # Hooking utilities\n",
    "from transformer_lens import HookedTransformer, HookedTransformerConfig, FactoredMatrix, ActivationCache"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We turn automatic differentiation off, to save GPU memory, as this notebook focuses on model inference not model training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch.autograd.grad_mode.set_grad_enabled at 0x7fed526eb110>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.set_grad_enabled(False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plotting helper functions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from neel_plotly import line, scatter, imshow, histogram"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Othello GPT"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details><summary>I was in a massive rush when I made this codebase, so it's a bit of a mess, sorry! This Colab is an attempt to put a smiley face on the underlying shoggoth, but if you want more of the guts, here's an info dump</summary>\n",
    "\n",
    "This codebase is a bit of a mess! This colab is an attempt to be a pretty mask on top of the shoggoth, but if it helps, here's an info dump I wrote for someone about interpreting this codebase:\n",
    "\n",
    "Technical details:\n",
    "\n",
    "-   Games are 60 moves, but the model can only take in 59. It's trained to predict the next move, so they give it the first 59 moves (0<=...<59) and evaluate the predictions for each next move (1<=...<60). There is no Beginning of Sequence token, and the model never tries to predict the first move of the game\n",
    "\n",
    "-   This means that, while in Othello black plays first, here white plays \"first\" because first is actually second\n",
    "\n",
    "-   You can get code to load the synthetic model (ie trained to play uniformly random legal moves) into TransformerLens here: [https://colab.research.google.com/github/neelnanda-io/TransformerLens/blob/main/demos/Othello\\_GPT.ipynb](https://colab.research.google.com/github/neelnanda-io/TransformerLens/blob/main/demos/Othello_GPT.ipynb) \n",
    "-   You can load in their synthetically generated games [from their Github](https://github.com/likenneth/othello_world) (there's a google drive link)\n",
    "-   Their model has 8 layers, residual stream width 512, 8 heads per layer and 2048 neurons per layer.\n",
    "\n",
    "-   The vocab size is 61. 0 is -100, which I *think *means pass, I just filtered out the rare games that include that move and ignore it. 1 to 60 (inclusive) means the board moves in lexicographic order (A0, A1, ..., A7, B0, ...) but *skipping *D3, D4, E3 and E4. These are at the center of the board and so can never be played, because Othello starts with them filled)\n",
    "\n",
    "-   There's 3 ways to denote a board cell. I call them \"int\", \"string\" and \"label\" (which is terrible notation, sorry). \n",
    "\n",
    "-   \"label\" means the label for a board cell, \\[\"A0\", ..., \"A7\", ''', \"H7\"\\] (I index at 0 not 1, sorry!). \n",
    "-   \"int\" means \"part of the model vocabulary\", so 1 means A0, we then count up but miss the center squares, so 27 is D2, 28 is D5, 33 is E2 and 34 is E5. \n",
    "-   \"string\" means \"the input format of the OthelloBoardState class\". These are integers (sorry!) from 0 to 63, and exactly correspond to labels A0, ..., H7, without skipping any center cells. OthelloBoardState is a class in data/othello.py that can play the Othello game and tell you the board state and valid moves (created by the authors, not me)\n",
    "-   I have utility functions to\\_int, to\\_string, str\\_to\\_label and int\\_to\\_label in tl\\_othello\\_utils.py to do this\n",
    "\n",
    "-   The embedding and unembedding are untied (ie, in contrast to most language models, the map W\\_U from final residual to the logits is *not *the transpose of W\\_E, the map from tokens to the initial residual. They're unrelated matrices)\n",
    "-   tl\\_othello\\_utils.py is my utils file, with various functions to load games, etc. \\`board\\_seqs\\_string\\` and \\`board\\_seqs\\_int\\` are massive saved tensors with every move across all 4.5M synthetic games in both string and int format, these are 2.3GB so I haven't attached them lol. You can recreate them from the synthetic games they provide. It also provides a bunch of plotting functions to make nice othello board states, and some random other utilities\n",
    "-   \\`tl\\_probing.py\\` is my probe training file. But it was used to train a *second* probe, linear\\_probe\\_L4\\_blank\\_vs\\_color\\_v1.pth . This probe actually didn't work very well for analysing the model (despite getting great accuracy) and I don't know why - it was trained on layer 4, to do a binary classification on blank vs not blank, and on my color vs their color *conditional *on not being blank (ie not evaluated if blank). For some reason, the \"this cell is my color\" direction has a significant dot product with the \"is blank\" direction, and this makes it much worse for eg interpreting neurons. I don't know why!\n",
    "-   \\`tl\\_scratch.py\\` is where I did some initial exploration, including activation patching between different final moves\n",
    "-   \\`tl\\_exploration.py\\` is where I did my most recent exploration, verifying that the probe works, doing probe interventions (CTRL F for \\`newly\\_legal\\`) and using the probe to interpret neurons\n",
    "\n",
    "</details>\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading the model"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This loads a conversion of the author's synthetic model checkpoint to TransformerLens format. See [this notebook](https://colab.research.google.com/github/neelnanda-io/TransformerLens/blob/main/demos/Othello_GPT.ipynb) for how."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "import transformer_lens.utils as utils\n",
    "cfg = HookedTransformerConfig(\n",
    "    n_layers = 8,\n",
    "    d_model = 512,\n",
    "    d_head = 64,\n",
    "    n_heads = 8,\n",
    "    d_mlp = 2048,\n",
    "    d_vocab = 61,\n",
    "    n_ctx = 59,\n",
    "    act_fn=\"gelu\",\n",
    "    normalization_type=\"LNPre\"\n",
    ")\n",
    "model = HookedTransformer(cfg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<All keys matched successfully>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "sd = utils.download_file_from_hf(\"NeelNanda/Othello-GPT-Transformer-Lens\", \"synthetic_model.pth\")\n",
    "# champion_ship_sd = utils.download_file_from_hf(\"NeelNanda/Othello-GPT-Transformer-Lens\", \"championship_model.pth\")\n",
    "model.load_state_dict(sd)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Code to load and convert one of the author's checkpoints to TransformerLens:"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Testing code for the synthetic checkpoint giving the correct outputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[21, 41, 40, 34, 40, 41,  3, 11, 21, 43, 40, 21, 28, 50, 33, 50, 33,  5,\n",
       "         33,  5, 52, 46, 14, 46, 14, 47, 38, 57, 36, 50, 38, 15, 28, 26, 28, 59,\n",
       "         50, 28, 14, 28, 28, 28, 28, 45, 28, 35, 15, 14, 30, 59, 49, 59, 15, 15,\n",
       "         14, 15,  8,  7,  8]], device='cuda:0')"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# An example input\n",
    "sample_input = torch.tensor([[20, 19, 18, 10, 2, 1, 27, 3, 41, 42, 34, 12, 4, 40, 11, 29, 43, 13, 48, 56, 33, 39, 22, 44, 24, 5, 46, 6, 32, 36, 51, 58, 52, 60, 21, 53, 26, 31, 37, 9, 25, 38, 23, 50, 45, 17, 47, 28, 35, 30, 54, 16, 59, 49, 57, 14, 15, 55, 7]])\n",
    "# The argmax of the output (ie the most likely next move from each position)\n",
    "sample_output = torch.tensor([[21, 41, 40, 34, 40, 41,  3, 11, 21, 43, 40, 21, 28, 50, 33, 50, 33,  5,\n",
    "         33,  5, 52, 46, 14, 46, 14, 47, 38, 57, 36, 50, 38, 15, 28, 26, 28, 59,\n",
    "         50, 28, 14, 28, 28, 28, 28, 45, 28, 35, 15, 14, 30, 59, 49, 59, 15, 15,\n",
    "         14, 15,  8,  7,  8]])\n",
    "model(sample_input).argmax(dim=-1)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading Othello Content\n",
    "Boring setup code to load in 100K sample Othello games, the linear probe, and some utility functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "if IN_COLAB:\n",
    "    !git clone https://github.com/likenneth/othello_world\n",
    "    OTHELLO_ROOT = Path(\"/content/othello_world/\")\n",
    "    import sys\n",
    "    sys.path.append(str(OTHELLO_ROOT/\"mechanistic_interpretability\"))\n",
    "    from mech_interp_othello_utils import plot_single_board, to_string, to_int, int_to_label, string_to_label, OthelloBoardState\n",
    "else:\n",
    "    OTHELLO_ROOT = Path(\"/workspace/othello_world/\")\n",
    "    from tl_othello_utils import plot_single_board, to_string, to_int, int_to_label, string_to_label, OthelloBoardState\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We load in a big tensor of 100,000 games, each with 60 moves. This is in the format the model wants, with 1-59 representing the 60 moves, and 0 representing pass.\n",
    "\n",
    "We also load in the same set of games, in the same order, but in \"string\" format - still a tensor of ints but referring to moves with numbers from 0 to 63 rather than in the model's compressed format of 1 to 59"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of games: 100000\n",
      "Length of game: 60\n"
     ]
    }
   ],
   "source": [
    "board_seqs_int = torch.tensor(np.load(OTHELLO_ROOT/\"mechanistic_interpretability/board_seqs_int_small.npy\"), dtype=torch.long)\n",
    "board_seqs_string = torch.tensor(np.load(OTHELLO_ROOT/\"mechanistic_interpretability/board_seqs_string_small.npy\"), dtype=torch.long)\n",
    "\n",
    "num_games, length_of_game = board_seqs_int.shape\n",
    "print(\"Number of games:\", num_games,)\n",
    "print(\"Length of game:\", length_of_game)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "stoi_indices = [\n",
    "    0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 29, 30, 31, 32, 33, 34, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63,\n",
    "]\n",
    "alpha = \"ABCDEFGH\"\n",
    "\n",
    "\n",
    "def to_board_label(i):\n",
    "    return f\"{alpha[i//8]}{i%8}\"\n",
    "\n",
    "\n",
    "board_labels = list(map(to_board_label, stoi_indices))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Running the Model"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model's context length is 59, not 60, because it's trained to receive the first 59 moves and predict the final 59 moves (ie `[0:-1]` and `[1:]`. Let's run the model on the first 30 moves of game 0!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "logits: torch.Size([1, 30, 61])\n"
     ]
    }
   ],
   "source": [
    "moves_int = board_seqs_int[0, :30]\n",
    "\n",
    "# This is implicitly converted to a batch of size 1\n",
    "logits = model(moves_int)\n",
    "print(\"logits:\", logits.shape)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We take the final vector of logits. We convert it to log probs and we then remove the first element (corresponding to passing, and we've filtered out all games with passing) and get the 60 logits. This is 64-4 because the model's vocab is compressed, since the center 4 squares can't be played.\n",
    "\n",
    "We then convert it to an 8 x 8 grid and plot it, with some tensor magic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "logit_vec = logits[0, -1]\n",
    "log_probs = logit_vec.log_softmax(-1)\n",
    "# Remove passing\n",
    "log_probs = log_probs[1:]\n",
    "assert len(log_probs)==60\n",
    "\n",
    "temp_board_state = torch.zeros(64, device=logit_vec.device)\n",
    "# Set all cells to -15 by default, for a very negative log prob - this means the middle cells don't show up as mattering\n",
    "temp_board_state -= 13.\n",
    "temp_board_state[stoi_indices] = log_probs"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now plot this as a board state! We see a crisp distinction from a set of moves that the model clearly thinks are valid (at near uniform probabilities), and a bunch that aren't. Note that by training the model to predict a *uniformly* chosen next move, we incentivise it to be careful about making all valid logits be uniform!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"4b84f887-5e9d-43fa-a8c5-95a167463068\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"4b84f887-5e9d-43fa-a8c5-95a167463068\")) {                    Plotly.newPlot(                        \"4b84f887-5e9d-43fa-a8c5-95a167463068\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-12.4404745,-12.51374,-13.0445595,-2.702029,-12.476699,-2.5816052,-2.5107648,-12.25303],[-12.262719,-13.315735,-2.5008528,-12.37807,-10.954452,-11.434681,-9.553426,-12.936988],[-12.35031,-12.152403,-11.859199,-12.744285,-12.61962,-12.086893,-13.140984,-2.6213753],[-11.728302,-12.121631,-2.5382726,-13.0,-13.0,-12.107283,-11.443226,-13.725403],[-11.211905,-12.250443,-12.97645,-13.0,-13.0,-10.337581,-2.4562886,-11.803778],[-11.937233,-12.798651,-11.910555,-12.669524,-12.543441,-10.268892,-12.567521,-2.5384266],[-11.758127,-11.652883,-12.115789,-11.546722,-11.875585,-9.957391,-2.6429675,-11.781472],[-11.22251,-11.741685,-2.6555598,-2.5332296,-2.5351045,-9.175257,-2.5643203,-10.135648]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(247,251,255)\"],[0.125,\"rgb(222,235,247)\"],[0.25,\"rgb(198,219,239)\"],[0.375,\"rgb(158,202,225)\"],[0.5,\"rgb(107,174,214)\"],[0.625,\"rgb(66,146,198)\"],[0.75,\"rgb(33,113,181)\"],[0.875,\"rgb(8,81,156)\"],[1.0,\"rgb(8,48,107)\"]],\"cmin\":-13.725403,\"cmax\":0},\"title\":{\"text\":\"Example Log Probs\"}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('4b84f887-5e9d-43fa-a8c5-95a167463068');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def plot_square_as_board(state, diverging_scale=True, **kwargs):\n",
    "    \"\"\"Takes a square input (8 by 8) and plot it as a board. Can do a stack of boards via facet_col=0\"\"\"\n",
    "    if diverging_scale:\n",
    "        imshow(state, y=[i for i in alpha], x=[str(i) for i in range(8)], color_continuous_scale=\"RdBu\", color_continuous_midpoint=0., aspect=\"equal\", **kwargs)\n",
    "    else:\n",
    "        imshow(state, y=[i for i in alpha], x=[str(i) for i in range(8)], color_continuous_scale=\"Blues\", color_continuous_midpoint=None, aspect=\"equal\", **kwargs)\n",
    "plot_square_as_board(temp_board_state.reshape(8, 8), zmax=0, diverging_scale=False, title=\"Example Log Probs\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exploring Game Play"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For comparison, let's plot the board state after 30 moves, and the valid moves"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`plot_single_board` is a helper function to plot the board state after a series of moves. It takes in moves in the label format, ie A0 to H7, so we convert. We can see on inspection that our model has correctly identified the valid moves!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"3a153efc-ea1c-4417-b872-cbb13c8d9a23\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"3a153efc-ea1c-4417-b872-cbb13c8d9a23\")) {                    Plotly.newPlot(                        \"3a153efc-ea1c-4417-b872-cbb13c8d9a23\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.0,0.0,0.0,0.5,-1.0,0.5,0.5,0.0],[0.0,-0.1,0.5,-0.9,-1.0,-1.0,-0.1,1.0],[0.0,1.0,1.0,-0.9,-0.9,1.0,1.0,0.5],[0.0,-0.1,0.5,-1.0,-1.0,-1.0,-1.0,-1.0],[-0.1,-0.1,1.0,-1.0,-1.0,-1.0,0.5,-1.0],[-0.1,1.0,1.0,1.0,1.0,-1.0,-1.0,0.5],[-0.1,-0.1,-1.0,-1.0,-1.0,-1.0,0.5,1.0],[0.0,-1.0,0.5,0.5,0.5,-1.0,0.5,0.0]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"<b>%{y}%{x}</b><br>%{customdata}<extra></extra>\",\"text\":[[\"A0\",\"A1\",\"A2\",\"A3\",\"\",\"A5\",\"A6\",\"A7\"],[\"B0\",\"B1\",\"B2\",\"\",\"\",\"\",\"B6\",\"\"],[\"C0\",\"\",\"\",\"\",\"\",\"\",\"\",\"C7\"],[\"D0\",\"D1\",\"D2\",\"\",\"\",\"\",\"\",\"\"],[\"E0\",\"E1\",\"\",\"\",\"\",\"\",\"E6\",\"\"],[\"F0\",\"\",\"\",\"\",\"\",\"\",\"\",\"F7\"],[\"G0\",\"G1\",\"\",\"\",\"\",\"\",\"G6\",\"\"],[\"H0\",\"\",\"H2\",\"H3\",\"H4\",\"\",\"H6\",\"H7\"]],\"texttemplate\":\"%{text}\",\"customdata\":[[\"I:1<br>S:0\",\"I:2<br>S:1\",\"I:3<br>S:2\",\"I:4<br>S:3\",\"I:5<br>S:4\",\"I:6<br>S:5\",\"I:7<br>S:6\",\"I:8<br>S:7\"],[\"I:9<br>S:8\",\"I:10<br>S:9\",\"I:11<br>S:10\",\"I:12<br>S:11\",\"I:13<br>S:12\",\"I:14<br>S:13\",\"I:15<br>S:14\",\"I:16<br>S:15\"],[\"I:17<br>S:16\",\"I:18<br>S:17\",\"I:19<br>S:18\",\"I:20<br>S:19\",\"I:21<br>S:20\",\"I:22<br>S:21\",\"I:23<br>S:22\",\"I:24<br>S:23\"],[\"I:25<br>S:24\",\"I:26<br>S:25\",\"I:27<br>S:26\",\"I:X<br>S:27\",\"I:X<br>S:28\",\"I:28<br>S:29\",\"I:29<br>S:30\",\"I:30<br>S:31\"],[\"I:31<br>S:32\",\"I:32<br>S:33\",\"I:33<br>S:34\",\"I:X<br>S:35\",\"I:X<br>S:36\",\"I:34<br>S:37\",\"I:35<br>S:38\",\"I:36<br>S:39\"],[\"I:37<br>S:40\",\"I:38<br>S:41\",\"I:39<br>S:42\",\"I:40<br>S:43\",\"I:41<br>S:44\",\"I:42<br>S:45\",\"I:43<br>S:46\",\"I:44<br>S:47\"],[\"I:45<br>S:48\",\"I:46<br>S:49\",\"I:47<br>S:50\",\"I:48<br>S:51\",\"I:49<br>S:52\",\"I:50<br>S:53\",\"I:51<br>S:54\",\"I:52<br>S:55\"],[\"I:53<br>S:56\",\"I:54<br>S:57\",\"I:55<br>S:58\",\"I:56<br>S:59\",\"I:57<br>S:60\",\"I:58<br>S:61\",\"I:59<br>S:62\",\"I:60<br>S:63\"]]}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(0, 128, 128)\"],[0.16666666666666666,\"rgb(112, 164, 148)\"],[0.3333333333333333,\"rgb(180, 200, 168)\"],[0.5,\"rgb(246, 237, 189)\"],[0.6666666666666666,\"rgb(237, 187, 138)\"],[0.8333333333333334,\"rgb(222, 138, 90)\"],[1.0,\"rgb(202, 86, 44)\"]],\"cmid\":0.0},\"title\":{\"text\":\"Black To Play. Board State After White Plays B3 \",\"x\":0.5},\"shapes\":[{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":-0.2,\"y1\":0.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":2.75,\"x1\":3.25,\"y0\":0.75,\"y1\":1.25,\"line\":{\"color\":\"red\",\"width\":4}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":6.8,\"x1\":7.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":0.8,\"x1\":1.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":2.78,\"x1\":3.22,\"y0\":1.78,\"y1\":2.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.78,\"x1\":4.22,\"y0\":1.78,\"y1\":2.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":6.8,\"x1\":7.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":6.8,\"x1\":7.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":0.8,\"x1\":1.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":6.8,\"x1\":7.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":0.8,\"x1\":1.2,\"y0\":6.8,\"y1\":7.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":6.8,\"y1\":7.2,\"line\":{\"color\":\"green\",\"width\":0.5}}]},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('3a153efc-ea1c-4417-b872-cbb13c8d9a23');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_single_board(int_to_label(moves_int))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also compute this explicitly, using the OthelloBoardState class (thanks to Kenneth Li for writing this one and saving me a bunch of tedious effort!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"c8510782-fcd2-4419-9dab-2953cea4664b\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"c8510782-fcd2-4419-9dab-2953cea4664b\")) {                    Plotly.newPlot(                        \"c8510782-fcd2-4419-9dab-2953cea4664b\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.0,0.0,0.0,0.0,-1.0,0.0,0.0,0.0],[0.0,0.0,0.0,-1.0,-1.0,-1.0,0.0,1.0],[0.0,1.0,1.0,-1.0,-1.0,1.0,1.0,0.0],[0.0,0.0,0.0,-1.0,-1.0,-1.0,-1.0,-1.0],[0.0,0.0,1.0,-1.0,-1.0,-1.0,0.0,-1.0],[0.0,1.0,1.0,1.0,1.0,-1.0,-1.0,0.0],[0.0,0.0,-1.0,-1.0,-1.0,-1.0,0.0,1.0],[0.0,-1.0,0.0,0.0,0.0,-1.0,0.0,0.0]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"title\":{\"text\":\"Example Board State (+1 is Black, -1 is White)\"}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('c8510782-fcd2-4419-9dab-2953cea4664b');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "board = OthelloBoardState()\n",
    "board.update(to_string(moves_int))\n",
    "plot_square_as_board(board.state, title=\"Example Board State (+1 is Black, -1 is White)\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And we can get out a list of valid moves:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Valid moves: ['A3', 'A5', 'A6', 'B2', 'C7', 'D2', 'E6', 'F7', 'G6', 'H2', 'H3', 'H4', 'H6']\n"
     ]
    }
   ],
   "source": [
    "print(\"Valid moves:\", string_to_label(board.get_valid_moves()))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Making some utilities"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point, I'll stop and get some aggregate data that will be useful later - a tensor of valid moves, of board states, and a cache of all model activations across 50 games (in practice, you want as much as will comfortably fit into GPU memory). It's really convenient to have the ability to quickly run an experiment across a bunch of games! And one of the great things about small models on algorithmic tasks is that you just can do stuff like this. \n",
    "\n",
    "For want of a lack of creativity, let's call these the **focus games**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_games = 50\n",
    "focus_games_int = board_seqs_int[:num_games]\n",
    "focus_games_string = board_seqs_string[:num_games]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A big stack of each move's board state and a big stack of the valid moves in each game (one hot encoded to be a nice tensor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "focus states: (50, 60, 8, 8)\n",
      "focus_valid_moves torch.Size([50, 60, 64])\n"
     ]
    }
   ],
   "source": [
    "def one_hot(list_of_ints, num_classes=64):\n",
    "    out = torch.zeros((num_classes,), dtype=torch.float32)\n",
    "    out[list_of_ints] = 1.\n",
    "    return out\n",
    "focus_states = np.zeros((num_games, 60, 8, 8), dtype=np.float32)\n",
    "focus_valid_moves = torch.zeros((num_games, 60, 64), dtype=torch.float32)\n",
    "for i in (range(num_games)):\n",
    "    board = OthelloBoardState()\n",
    "    for j in range(60):\n",
    "        board.umpire(focus_games_string[i, j].item())\n",
    "        focus_states[i, j] = board.state\n",
    "        focus_valid_moves[i, j] = one_hot(board.get_valid_moves())\n",
    "print(\"focus states:\", focus_states.shape)\n",
    "print(\"focus_valid_moves\", focus_valid_moves.shape)\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A cache of every model activation and the logits"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "focus_logits, focus_cache = model.run_with_cache(focus_games_int[:, :-1].cuda())\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using the probe"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The training of this probe was kind of a mess, and I'd do a bunch of things differently if doing it again.\n",
    "\n",
    "<details><summary>Info dump of technical details:</summary>\n",
    "\n",
    "mode==0 was trained on black to play, ie odd moves, and the classes are \\[blank, white, black\\] ie \\[blank, their colour, my colour\\] (I *think*, they could easily be the other way round. This should be easy to sanity check)\n",
    "\n",
    "mode==1 was trained on white to play, ie even moves, and the classes are \\[blank, black, white\\] ie \\[blank, their colour, my colour\\] (I think*)*\n",
    "\n",
    "mode==2 was trained on all moves, and just doesn't work very well.\n",
    "\n",
    "\n",
    "The probes were trained on moves 5 to 54 (ie not the early or late moves, because these are weird). I literally did AdamW against cross-entropy loss for each board cell, nothing fancy. You really didn't need to train on 4M games lol, it plateaued well before the end. Which is to be expected, it's just logistic regression!\n",
    "\n",
    "</details>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But it works!"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Let's load in the probe. The shape is [modes, d_model, row, col, options]. The 3 modes are \"black to play/odd moves\", \"white to play/even moves\", and \"all moves\". The 3 options are empty, white and black in that order.\n",
    "\n",
    "We'll just focus on the black to play probe - it basically just works for the even moves too, once you realise that it's detecting my colour vs their colour!\n",
    "\n",
    "This means that the options are \"empty\", \"their's\" and \"mine\" in that order"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "full_linear_probe = torch.load(OTHELLO_ROOT/\"mechanistic_interpretability/main_linear_probe.pth\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On move 29 in game 0, we can apply the probe to the model's residual stream after layer 6. Move 29 is black to play."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "rows = 8\n",
    "cols = 8 \n",
    "options = 3\n",
    "black_to_play_index = 0\n",
    "white_to_play_index = 1\n",
    "blank_index = 0\n",
    "their_index = 1\n",
    "my_index = 2\n",
    "linear_probe = torch.zeros(cfg.d_model, rows, cols, options, device=\"cuda\")\n",
    "linear_probe[..., blank_index] = 0.5 * (full_linear_probe[black_to_play_index, ..., 0] + full_linear_probe[white_to_play_index, ..., 0])\n",
    "linear_probe[..., their_index] = 0.5 * (full_linear_probe[black_to_play_index, ..., 1] + full_linear_probe[white_to_play_index, ..., 2])\n",
    "linear_probe[..., my_index] = 0.5 * (full_linear_probe[black_to_play_index, ..., 2] + full_linear_probe[white_to_play_index, ..., 1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "residual_stream torch.Size([512])\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"ec91fb87-c801-48b6-924c-a6e0365afee8\" class=\"plotly-graph-div\" style=\"height:400px; width:600px;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"ec91fb87-c801-48b6-924c-a6e0365afee8\")) {                    Plotly.newPlot(                        \"ec91fb87-c801-48b6-924c-a6e0365afee8\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.99999976,0.968072,1.0,1.0,1.0,1.0,0.9636731,0.99928856],[0.9834541,0.9994966,1.0,1.9577124e-8,7.312644e-8,0.9999807,0.99984705,0.998917],[0.9997168,0.000083959065,1.0,0.000005185437,0.000008247951,0.0000215352,0.99999964,1.0],[0.99999046,0.99999976,7.757757e-8,2.7562294e-10,2.173676e-10,0.9999993,0.0000035743474,0.9999963],[0.99756026,0.99999547,0.0000055953874,2.4387992e-10,6.227083e-10,9.074999e-7,1.0,1.0],[0.9999678,0.99999774,4.1813355e-8,0.0000059140743,3.6093515e-9,0.00026775632,9.29474e-14,3.935381e-8],[0.99867934,0.9999471,0.00008276932,0.00019439175,3.876017e-11,0.00006263142,0.000028273336,0.0000027772874],[0.999617,0.99999356,1.0,0.99999726,1.0,1.0,1.1808284e-9,0.99997497]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"1\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[2.5712025e-8,0.0031173022,1.6883145e-16,6.733207e-14,8.8439125e-15,1.5645137e-14,0.03434719,0.00041080176],[0.00044961766,0.0003612048,1.2639593e-15,0.9997378,0.9958584,0.0000035347093,0.000006125798,0.0005669107],[0.00018196153,0.0039731096,2.780452e-15,0.9991443,0.0022878083,0.0005512753,6.148952e-8,6.342149e-11],[0.000003822013,2.4708909e-8,0.0011599979,0.9982644,0.00077748345,5.2145634e-7,0.9972325,0.0000032304877],[0.00018111797,1.4569264e-7,0.00023780167,0.9950388,0.98804456,0.9987471,3.0732555e-16,1.3278148e-12],[0.0000053312056,1.0009445e-7,0.0001360589,0.9994985,0.9982388,0.9970499,0.05853151,0.0010470838],[0.001144266,9.883679e-7,0.9844205,0.9967552,0.99885345,0.9945268,0.9873505,0.99960905],[0.000063424,0.0000014785006,2.9278386e-15,0.0000027755507,8.1484103e-10,1.891273e-9,0.00011913282,0.000004306204]],\"type\":\"heatmap\",\"xaxis\":\"x2\",\"yaxis\":\"y2\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"2\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[2.025148e-7,0.028810715,7.421175e-15,3.9040045e-16,1.2647601e-15,1.1723116e-15,0.001979726,0.00030053867],[0.01609626,0.00014209992,1.7521038e-16,0.00026213808,0.004141519,0.000015748137,0.00014681347,0.0005160725],[0.000101124446,0.9959429,3.502868e-15,0.0008504397,0.99770397,0.9994272,2.9676158e-7,2.1901647e-9],[0.000005697999,2.1064834e-7,0.9988399,0.0017356732,0.9992225,1.8157594e-7,0.0027639456,5.04101e-7],[0.0022585802,0.000004464417,0.99975663,0.0049611405,0.011955494,0.0012520029,1.0510652e-16,7.569125e-13],[0.000026781318,0.0000021464296,0.99986386,0.00049552874,0.0017611779,0.0026823042,0.94146854,0.99895287],[0.00017638777,0.000051948748,0.01549674,0.003050437,0.001146568,0.0054105935,0.012621199,0.00038815604],[0.0003196475,0.000004960617,2.594675e-14,5.269005e-11,7.390537e-11,4.6739643e-11,0.9998809,0.000020794143]],\"type\":\"heatmap\",\"xaxis\":\"x3\",\"yaxis\":\"y3\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,0.31999999999999995],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"xaxis2\":{\"anchor\":\"y2\",\"domain\":[0.33999999999999997,0.6599999999999999],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis2\":{\"anchor\":\"x2\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis3\":{\"anchor\":\"y3\",\"domain\":[0.6799999999999999,0.9999999999999999],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis3\":{\"anchor\":\"x3\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"annotations\":[{\"font\":{},\"showarrow\":false,\"text\":\"P(Empty)\",\"x\":0.15999999999999998,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"P(Their's)\",\"x\":0.49999999999999994,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"P(Mine)\",\"x\":0.8399999999999999,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"}],\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"title\":{\"text\":\"Example probe outputs after move 29 (black to play)\"},\"height\":400,\"width\":600},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('ec91fb87-c801-48b6-924c-a6e0365afee8');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"7d3dc214-f200-4c09-8188-2eeb980324f9\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"7d3dc214-f200-4c09-8188-2eeb980324f9\")) {                    Plotly.newPlot(                        \"7d3dc214-f200-4c09-8188-2eeb980324f9\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.0,0.0,-0.5,-0.5,-0.5,-0.5,0.0,0.0],[0.1,0.0,-0.5,1.0,1.0,0.1,0.1,0.0],[0.0,-1.0,-0.5,1.0,-1.0,-1.0,0.1,0.0],[0.0,0.1,-1.0,1.0,-1.0,0.1,1.0,0.0],[0.0,0.1,-1.0,1.0,1.0,1.0,-0.5,-0.5],[0.0,0.1,-1.0,1.0,0.9,1.0,-1.0,-1.0],[0.0,0.1,1.0,1.0,0.9,0.9,0.9,1.0],[0.0,0.0,-0.5,0.0,-0.5,-0.5,-1.0,-0.5]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"<b>%{y}%{x}</b><br>%{customdata}<extra></extra>\",\"text\":[[\"A0\",\"A1\",\"A2\",\"A3\",\"A4\",\"A5\",\"A6\",\"A7\"],[\"B0\",\"B1\",\"B2\",\"\",\"\",\"B5\",\"B6\",\"B7\"],[\"C0\",\"\",\"C2\",\"\",\"\",\"\",\"C6\",\"C7\"],[\"D0\",\"D1\",\"\",\"\",\"\",\"D5\",\"\",\"D7\"],[\"E0\",\"E1\",\"\",\"\",\"\",\"\",\"E6\",\"E7\"],[\"F0\",\"F1\",\"\",\"\",\"\",\"\",\"\",\"\"],[\"G0\",\"G1\",\"\",\"\",\"\",\"\",\"\",\"\"],[\"H0\",\"H1\",\"H2\",\"H3\",\"H4\",\"H5\",\"\",\"H7\"]],\"texttemplate\":\"%{text}\",\"customdata\":[[\"I:1<br>S:0\",\"I:2<br>S:1\",\"I:3<br>S:2\",\"I:4<br>S:3\",\"I:5<br>S:4\",\"I:6<br>S:5\",\"I:7<br>S:6\",\"I:8<br>S:7\"],[\"I:9<br>S:8\",\"I:10<br>S:9\",\"I:11<br>S:10\",\"I:12<br>S:11\",\"I:13<br>S:12\",\"I:14<br>S:13\",\"I:15<br>S:14\",\"I:16<br>S:15\"],[\"I:17<br>S:16\",\"I:18<br>S:17\",\"I:19<br>S:18\",\"I:20<br>S:19\",\"I:21<br>S:20\",\"I:22<br>S:21\",\"I:23<br>S:22\",\"I:24<br>S:23\"],[\"I:25<br>S:24\",\"I:26<br>S:25\",\"I:27<br>S:26\",\"I:X<br>S:27\",\"I:X<br>S:28\",\"I:28<br>S:29\",\"I:29<br>S:30\",\"I:30<br>S:31\"],[\"I:31<br>S:32\",\"I:32<br>S:33\",\"I:33<br>S:34\",\"I:X<br>S:35\",\"I:X<br>S:36\",\"I:34<br>S:37\",\"I:35<br>S:38\",\"I:36<br>S:39\"],[\"I:37<br>S:40\",\"I:38<br>S:41\",\"I:39<br>S:42\",\"I:40<br>S:43\",\"I:41<br>S:44\",\"I:42<br>S:45\",\"I:43<br>S:46\",\"I:44<br>S:47\"],[\"I:45<br>S:48\",\"I:46<br>S:49\",\"I:47<br>S:50\",\"I:48<br>S:51\",\"I:49<br>S:52\",\"I:50<br>S:53\",\"I:51<br>S:54\",\"I:52<br>S:55\"],[\"I:53<br>S:56\",\"I:54<br>S:57\",\"I:55<br>S:58\",\"I:56<br>S:59\",\"I:57<br>S:60\",\"I:58<br>S:61\",\"I:59<br>S:62\",\"I:60<br>S:63\"]]}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(0, 128, 128)\"],[0.16666666666666666,\"rgb(112, 164, 148)\"],[0.3333333333333333,\"rgb(180, 200, 168)\"],[0.5,\"rgb(246, 237, 189)\"],[0.6666666666666666,\"rgb(237, 187, 138)\"],[0.8333333333333334,\"rgb(222, 138, 90)\"],[1.0,\"rgb(202, 86, 44)\"]],\"cmid\":0.0},\"title\":{\"text\":\"White To Play. Board State After Black Plays G4 \",\"x\":0.5},\"shapes\":[{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":0.8,\"x1\":1.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.78,\"x1\":4.22,\"y0\":4.78,\"y1\":5.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":6.8,\"x1\":7.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.75,\"x1\":4.25,\"y0\":5.75,\"y1\":6.25,\"line\":{\"color\":\"red\",\"width\":4}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.78,\"x1\":5.22,\"y0\":5.78,\"y1\":6.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":5.78,\"x1\":6.22,\"y0\":5.78,\"y1\":6.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":6.8,\"x1\":7.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":6.8,\"y1\":7.2,\"line\":{\"color\":\"green\",\"width\":0.5}}]},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('7d3dc214-f200-4c09-8188-2eeb980324f9');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "layer = 6\n",
    "game_index = 1\n",
    "move = 22\n",
    "def plot_probe_outputs(layer, game_index, move, **kwargs):\n",
    "    residual_stream = focus_cache[\"resid_post\", layer][game_index, move]\n",
    "    print(\"residual_stream\", residual_stream.shape)\n",
    "    probe_out = einops.einsum(residual_stream, linear_probe, \"d_model, d_model row col options -> row col options\")\n",
    "    probabilities = probe_out.softmax(dim=-1)\n",
    "    plot_square_as_board(probabilities, facet_col=2, facet_labels=[\"P(Empty)\", \"P(Their's)\", \"P(Mine)\"], height=400, width=600, **kwargs)\n",
    "plot_probe_outputs(layer, game_index, move, title=\"Example probe outputs after move 29 (black to play)\")\n",
    "\n",
    "plot_single_board(int_to_label(focus_games_int[game_index, :move+1]))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I got the best results intervening after layer 4, but interestingly here the model has *almost* figured out the board state by then, but is missing the fact that C5 and C6 are black and is confident that they're white. My guess is that the board state calculation circuits haven't quite finished and are doing some iterative reasoning - if those cells have been taken several times, maybe it needs a layer to track the next earliest time it was taken? I don't know, and figuring this out would be a great starter project if you want to explore!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "residual_stream torch.Size([512])\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"0d5d7de5-7754-4f3e-baaa-b8e5eb753394\" class=\"plotly-graph-div\" style=\"height:400px; width:600px;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"0d5d7de5-7754-4f3e-baaa-b8e5eb753394\")) {                    Plotly.newPlot(                        \"0d5d7de5-7754-4f3e-baaa-b8e5eb753394\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.99994874,0.9999193,0.9993274,0.9999999,0.99999964,0.9999944,0.99959844,0.9994894],[0.9996791,0.99999905,0.9999975,0.016603462,0.1948627,0.99999976,1.0,0.9999963],[0.9999808,0.0019122931,0.9999999,0.001807713,0.00015448991,0.0027017605,1.0,1.0],[0.9999999,1.0,0.006699624,0.00037861895,0.00013404933,1.0,0.028186558,1.0],[0.99999285,1.0,0.0047605275,0.00056252896,0.00002791848,0.00050412683,0.99999905,0.9999987],[0.9999938,1.0,0.000077223,7.613736e-7,8.742396e-8,0.000053724747,1.2140117e-8,5.9067695e-10],[0.99530643,0.9999981,0.008593073,0.000029017589,1.3221436e-8,1.0633931e-8,2.816421e-8,0.0000013743452],[0.9995221,0.99991477,0.9999982,0.9999989,0.99881136,0.9992206,6.3482775e-9,0.910256]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"1\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.0000424507,0.000033249664,0.000066501205,1.3773877e-7,6.6012305e-8,5.273787e-7,0.00022124372,0.000006397668],[0.000044844972,1.907023e-7,7.4564366e-7,0.92986375,0.7971198,1.7268427e-8,1.0605398e-9,7.9500757e-7],[0.0000018023151,0.0007023344,9.2224495e-9,0.981448,0.013415593,0.00040986878,1.2702602e-11,1.6042853e-9],[1.5469226e-8,7.374312e-12,0.0012015981,0.9616776,0.008506409,4.2195317e-11,0.96333927,2.2530529e-8],[0.000002694717,3.2545553e-11,0.0029145472,0.87507546,0.9992207,0.9684987,4.0332958e-7,1.7459628e-7],[0.000002003134,4.363574e-11,0.0027790896,0.9996661,0.99980384,0.99972314,0.0037532737,0.0000037385787],[0.0038358795,3.8014866e-7,0.9899422,0.99529946,0.9998186,0.9996099,0.99985695,0.9996125],[0.0003411059,0.000054529173,8.942979e-7,0.0000011253768,0.0011485846,0.0006562783,0.0000022746158,0.037522838]],\"type\":\"heatmap\",\"xaxis\":\"x2\",\"yaxis\":\"y2\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"2\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.000008826877,0.00004741734,0.000606179,4.4875296e-8,2.511996e-7,0.0000050845197,0.00018026038,0.0005041525],[0.00027608578,6.998689e-7,0.0000018073383,0.053532775,0.008017536,1.8616049e-7,9.935376e-9,0.0000028941754],[0.00001735566,0.9973853,5.9674704e-8,0.016744208,0.9864299,0.99688834,1.3706766e-10,4.9661253e-8],[8.064384e-8,1.15231165e-10,0.9920988,0.03794372,0.99135953,4.9856303e-11,0.008474176,1.606172e-8],[0.000004392304,1.3198e-9,0.99232495,0.12436206,0.0007512911,0.03099721,5.4372885e-7,0.0000012016152],[0.0000041146977,5.3507526e-10,0.9971437,0.00033313048,0.00019609829,0.00022312251,0.99624676,0.9999963],[0.0008577283,0.000001501567,0.0014646793,0.004671524,0.00018141257,0.00039009817,0.00014302987,0.0003861588],[0.00013679001,0.00003071247,8.286718e-7,2.3305533e-9,0.000040102223,0.00012304402,0.99999774,0.052221082]],\"type\":\"heatmap\",\"xaxis\":\"x3\",\"yaxis\":\"y3\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,0.31999999999999995],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"xaxis2\":{\"anchor\":\"y2\",\"domain\":[0.33999999999999997,0.6599999999999999],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis2\":{\"anchor\":\"x2\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis3\":{\"anchor\":\"y3\",\"domain\":[0.6799999999999999,0.9999999999999999],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis3\":{\"anchor\":\"x3\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"annotations\":[{\"font\":{},\"showarrow\":false,\"text\":\"P(Empty)\",\"x\":0.15999999999999998,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"P(Their's)\",\"x\":0.49999999999999994,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"P(Mine)\",\"x\":0.8399999999999999,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"}],\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"title\":{\"text\":\"Example probe outputs at layer 4 after move 29 (Black to play)\"},\"height\":400,\"width\":600},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('0d5d7de5-7754-4f3e-baaa-b8e5eb753394');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_probe_outputs(layer=4, game_index=game_index, move=move, title=\"Example probe outputs at layer 4 after move 29 (Black to play)\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now look at move 30, and we see that the representations totally flip! We also see that the model gets the corner wrong - it's not a big deal, but interesting!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "residual_stream torch.Size([512])\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"a479a148-e82d-41aa-9e1a-2a9e851fa449\" class=\"plotly-graph-div\" style=\"height:400px; width:600px;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"a479a148-e82d-41aa-9e1a-2a9e851fa449\")) {                    Plotly.newPlot(                        \"a479a148-e82d-41aa-9e1a-2a9e851fa449\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.99216026,0.9523841,0.9997414,1.0,2.8299028e-9,1.0,0.00009336552,1.0],[0.99939966,1.0,2.896236e-7,0.00032306975,1.7014928e-11,4.9464573e-15,9.1936685e-15,1.0],[0.9964719,1.0825307e-7,1.5687251e-10,9.2794004e-8,0.00009037122,4.9559853e-11,1.6408563e-7,1.0],[0.82144237,1.0,1.8606582e-11,4.590823e-9,7.0589583e-9,1.0,8.167783e-7,0.9999999],[0.9998288,0.99999964,7.9893425e-11,3.6583596e-9,6.2283863e-9,0.001069652,1.0,0.9999863],[0.99410856,1.0,9.976949e-12,0.000009754302,3.829586e-11,0.000053711417,9.766205e-9,0.0000042483002],[0.99126667,0.99998534,3.669009e-12,2.1460191e-7,3.29502e-9,0.000005441924,0.0010912278,0.0000015890521],[0.9944508,1.0,1.0,1.0,7.231504e-9,0.99999964,0.000024292347,0.999915]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"1\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.0077099917,0.0001440323,0.00014339574,5.1440697e-14,0.9999523,2.6763007e-16,0.99989355,5.4109132e-14],[0.00011058863,7.9652695e-18,0.0005755925,0.98233193,0.9992186,0.9989027,0.99992144,2.8606448e-10],[0.0026222793,0.0005686712,0.99908733,0.0025974328,0.99869883,0.9993864,0.9988274,4.5687393e-8],[0.12409469,1.2802865e-11,0.9939249,0.99327517,0.003474659,2.0273864e-18,0.99533075,8.275742e-8],[0.000008867605,1.24397115e-8,0.99735475,0.9903154,0.012442523,0.99465925,1.213529e-12,0.0000076230895],[0.0012517347,7.487009e-13,0.99860865,0.9783487,0.0047416156,0.99394387,0.00043857182,0.0018419843],[0.0070935753,0.0000020531577,0.9997969,0.9990959,0.0056358553,0.0008009988,0.9350895,0.99935],[0.005195408,1.83631e-14,2.0536103e-10,5.2489427e-17,0.0000011133047,3.322262e-10,0.000041779756,0.000029623574]],\"type\":\"heatmap\",\"xaxis\":\"x2\",\"yaxis\":\"y2\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"2\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.00012979907,0.047471754,0.00011527798,1.4723999e-16,0.000047661324,6.931855e-21,0.000013040415,1.162501e-15],[0.0004897745,1.1073594e-15,0.9994242,0.017345002,0.0007813546,0.0010973588,0.00007852927,4.972117e-11],[0.000905851,0.9994312,0.000912722,0.99740237,0.0012108291,0.0006135611,0.0011724258,7.5077514e-11],[0.054462906,1.5084721e-10,0.006075051,0.006724852,0.99652535,3.0051523e-19,0.0046683946,3.9998806e-8],[0.00016232723,3.9916313e-7,0.002645281,0.009684565,0.98755753,0.004271066,1.7452598e-12,0.0000060648495],[0.0046396586,5.9803807e-12,0.0013913269,0.021641564,0.9952584,0.006002394,0.99956137,0.99815375],[0.001639864,0.000012591276,0.00020314923,0.0009038607,0.99436414,0.9991935,0.063819185,0.00064834376],[0.0003538764,4.033767e-12,3.478248e-11,2.754856e-18,0.9999989,3.5959113e-7,0.99993396,0.00005526058]],\"type\":\"heatmap\",\"xaxis\":\"x3\",\"yaxis\":\"y3\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,0.31999999999999995],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"xaxis2\":{\"anchor\":\"y2\",\"domain\":[0.33999999999999997,0.6599999999999999],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis2\":{\"anchor\":\"x2\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis3\":{\"anchor\":\"y3\",\"domain\":[0.6799999999999999,0.9999999999999999],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis3\":{\"anchor\":\"x3\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"annotations\":[{\"font\":{},\"showarrow\":false,\"text\":\"P(Empty)\",\"x\":0.15999999999999998,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"P(Their's)\",\"x\":0.49999999999999994,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"P(Mine)\",\"x\":0.8399999999999999,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"}],\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"title\":{\"text\":\"Example probe outputs at layer 4 after move 30 (White to play)\"},\"height\":400,\"width\":600},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('a479a148-e82d-41aa-9e1a-2a9e851fa449');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"80d8847c-da3a-4640-9f71-bcfa53664b3a\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"80d8847c-da3a-4640-9f71-bcfa53664b3a\")) {                    Plotly.newPlot(                        \"80d8847c-da3a-4640-9f71-bcfa53664b3a\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.0,0.1,0.1,-0.5,0.9,-0.5,1.0,-0.5],[0.1,-0.5,-1.0,0.9,0.9,0.9,1.0,-0.5],[0.1,-1.0,1.0,-1.0,1.0,1.0,1.0,-0.5],[0.0,-0.5,1.0,1.0,-1.0,-0.5,1.0,0.0],[0.0,-0.5,1.0,1.0,-1.0,1.0,-0.5,0.1],[0.0,-0.5,1.0,1.0,-1.0,1.0,-1.0,-1.0],[0.0,-0.5,1.0,1.0,-1.0,-1.0,1.0,1.0],[0.0,-0.5,-0.5,-0.5,-1.0,-0.5,-1.0,-0.5]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"<b>%{y}%{x}</b><br>%{customdata}<extra></extra>\",\"text\":[[\"A0\",\"A1\",\"A2\",\"A3\",\"\",\"A5\",\"\",\"A7\"],[\"B0\",\"B1\",\"\",\"\",\"\",\"\",\"\",\"B7\"],[\"C0\",\"\",\"\",\"\",\"\",\"\",\"\",\"C7\"],[\"D0\",\"D1\",\"\",\"\",\"\",\"D5\",\"\",\"D7\"],[\"E0\",\"E1\",\"\",\"\",\"\",\"\",\"E6\",\"E7\"],[\"F0\",\"F1\",\"\",\"\",\"\",\"\",\"\",\"\"],[\"G0\",\"G1\",\"\",\"\",\"\",\"\",\"\",\"\"],[\"H0\",\"H1\",\"H2\",\"H3\",\"\",\"H5\",\"\",\"H7\"]],\"texttemplate\":\"%{text}\",\"customdata\":[[\"I:1<br>S:0\",\"I:2<br>S:1\",\"I:3<br>S:2\",\"I:4<br>S:3\",\"I:5<br>S:4\",\"I:6<br>S:5\",\"I:7<br>S:6\",\"I:8<br>S:7\"],[\"I:9<br>S:8\",\"I:10<br>S:9\",\"I:11<br>S:10\",\"I:12<br>S:11\",\"I:13<br>S:12\",\"I:14<br>S:13\",\"I:15<br>S:14\",\"I:16<br>S:15\"],[\"I:17<br>S:16\",\"I:18<br>S:17\",\"I:19<br>S:18\",\"I:20<br>S:19\",\"I:21<br>S:20\",\"I:22<br>S:21\",\"I:23<br>S:22\",\"I:24<br>S:23\"],[\"I:25<br>S:24\",\"I:26<br>S:25\",\"I:27<br>S:26\",\"I:X<br>S:27\",\"I:X<br>S:28\",\"I:28<br>S:29\",\"I:29<br>S:30\",\"I:30<br>S:31\"],[\"I:31<br>S:32\",\"I:32<br>S:33\",\"I:33<br>S:34\",\"I:X<br>S:35\",\"I:X<br>S:36\",\"I:34<br>S:37\",\"I:35<br>S:38\",\"I:36<br>S:39\"],[\"I:37<br>S:40\",\"I:38<br>S:41\",\"I:39<br>S:42\",\"I:40<br>S:43\",\"I:41<br>S:44\",\"I:42<br>S:45\",\"I:43<br>S:46\",\"I:44<br>S:47\"],[\"I:45<br>S:48\",\"I:46<br>S:49\",\"I:47<br>S:50\",\"I:48<br>S:51\",\"I:49<br>S:52\",\"I:50<br>S:53\",\"I:51<br>S:54\",\"I:52<br>S:55\"],[\"I:53<br>S:56\",\"I:54<br>S:57\",\"I:55<br>S:58\",\"I:56<br>S:59\",\"I:57<br>S:60\",\"I:58<br>S:61\",\"I:59<br>S:62\",\"I:60<br>S:63\"]]}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(0, 128, 128)\"],[0.16666666666666666,\"rgb(112, 164, 148)\"],[0.3333333333333333,\"rgb(180, 200, 168)\"],[0.5,\"rgb(246, 237, 189)\"],[0.6666666666666666,\"rgb(237, 187, 138)\"],[0.8333333333333334,\"rgb(222, 138, 90)\"],[1.0,\"rgb(202, 86, 44)\"]],\"cmid\":0.0},\"title\":{\"text\":\"White To Play. Board State After Black Plays A4 \",\"x\":0.5},\"shapes\":[{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.75,\"x1\":4.25,\"y0\":-0.25,\"y1\":0.25,\"line\":{\"color\":\"red\",\"width\":4}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":-0.2,\"y1\":0.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.78,\"x1\":3.22,\"y0\":0.78,\"y1\":1.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.78,\"x1\":4.22,\"y0\":0.78,\"y1\":1.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.78,\"x1\":5.22,\"y0\":0.78,\"y1\":1.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":0.8,\"x1\":1.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":6.8,\"x1\":7.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":6.8,\"x1\":7.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":6.8,\"y1\":7.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":6.8,\"y1\":7.2,\"line\":{\"color\":\"green\",\"width\":0.5}}]},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('80d8847c-da3a-4640-9f71-bcfa53664b3a');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_probe_outputs(layer=6, game_index=game_index, move=30, title=\"Example probe outputs at layer 4 after move 30 (White to play)\")\n",
    "\n",
    "plot_single_board(focus_games_string[game_index, :31])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Fascinatingly, the white to play probe gets the corner right! A fact about Othello is that a piece in the corners can never be flanked and thus will never change colour once placed - perhaps the model has decided to cut corners and have a different and less symmetric circuit for these?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"1b0a2f89-0bc5-4401-8194-5b6f38a079ba\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"1b0a2f89-0bc5-4401-8194-5b6f38a079ba\")) {                    Plotly.newPlot(                        \"1b0a2f89-0bc5-4401-8194-5b6f38a079ba\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.9954332,0.99889225,0.9999522,1.0,5.9825496e-9,1.0,0.0005214925,1.0],[0.9999738,1.0,5.2521546e-7,0.000032724824,1.6561954e-12,2.2321645e-15,4.2475074e-14,1.0],[0.9945422,3.388694e-8,9.049935e-12,9.356823e-9,0.0000057255083,5.67995e-12,4.4851557e-8,1.0],[0.9242389,1.0,8.1578353e-13,9.571187e-11,2.391948e-10,1.0,3.9274445e-7,1.0],[0.9999261,0.9999989,4.119413e-12,1.1321749e-10,1.0250674e-10,0.00004010437,1.0,0.99999833],[0.99706084,1.0,2.6359234e-13,2.9131525e-7,7.302407e-12,0.000002478764,2.1725866e-9,0.000016469312],[0.99956435,0.99998033,8.365806e-13,1.6469189e-8,1.9221464e-9,0.00000176616,0.00054525933,0.0000049053683],[0.9998491,1.0,1.0,1.0,5.096414e-8,1.0,0.0010425135,0.99999523]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"1\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.000046653324,0.0010986509,0.0000143428915,3.590559e-17,0.00006808078,3.236042e-22,0.000004530105,1.9329088e-17],[0.000020991232,2.549346e-15,0.9987325,0.011571844,0.00046235166,0.00059779186,0.000016290192,1.9872541e-12],[0.00040866228,0.99926144,0.00042597612,0.99592626,0.0008881823,0.00049714313,0.0005195975,3.0612565e-12],[0.010113441,1.8382139e-9,0.0020827628,0.0035218669,0.99002975,6.178561e-18,0.002740468,5.5118106e-9],[0.000061914274,0.0000011098258,0.0016440926,0.006769091,0.9852143,0.00251154,1.2353365e-12,3.3098908e-7],[0.0020387727,2.9391472e-11,0.0008083601,0.015820052,0.98760104,0.0042090174,0.99910223,0.9983321],[0.00007000893,0.000016390808,0.00013127354,0.0006032199,0.9934402,0.99928814,0.025340894,0.00032286003],[0.000015723315,8.1803986e-13,4.3288632e-12,8.1130764e-19,0.99999726,4.8362757e-8,0.9988745,0.000002534872]],\"type\":\"heatmap\",\"xaxis\":\"x2\",\"yaxis\":\"y2\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"2\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.004520191,0.0000091903985,0.000033478264,9.797876e-15,0.99993193,2.6345158e-17,0.999474,2.6153018e-16],[0.0000051913707,2.5706517e-17,0.0012669988,0.98839545,0.9995377,0.99940217,0.99998367,2.016618e-11],[0.0050491625,0.00073846546,0.999574,0.0040736995,0.9991061,0.9995029,0.9994804,6.6745978e-9],[0.06564757,1.7779636e-10,0.99791723,0.99647814,0.009970313,4.3285796e-17,0.99725914,1.6967473e-8],[0.000012066361,4.2799446e-8,0.9983559,0.99323094,0.0147857275,0.99744844,1.5130809e-12,0.0000012962664],[0.00090040005,8.003244e-12,0.9991916,0.98417974,0.012398963,0.9957885,0.0008977712,0.0016514208],[0.00036561178,0.000003380951,0.99986875,0.9993968,0.006559807,0.0007101773,0.97411376,0.9996723],[0.0001352014,4.427094e-15,1.2667405e-10,2.619714e-17,0.0000026392056,1.0545324e-10,0.00008300505,0.000002314704]],\"type\":\"heatmap\",\"xaxis\":\"x3\",\"yaxis\":\"y3\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,0.31999999999999995],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"xaxis2\":{\"anchor\":\"y2\",\"domain\":[0.33999999999999997,0.6599999999999999],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis2\":{\"anchor\":\"x2\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis3\":{\"anchor\":\"y3\",\"domain\":[0.6799999999999999,0.9999999999999999],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis3\":{\"anchor\":\"x3\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"annotations\":[{\"font\":{},\"showarrow\":false,\"text\":\"P(Empty)\",\"x\":0.15999999999999998,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"P(White)\",\"x\":0.49999999999999994,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"P(Black)\",\"x\":0.8399999999999999,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"}],\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"title\":{\"text\":\"Probabilities after move 30 for the white to play probe - correct A7 label!\"}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('1b0a2f89-0bc5-4401-8194-5b6f38a079ba');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "residual_stream = focus_cache[\"resid_post\", layer][game_index, 30]\n",
    "white_to_play_probe = full_linear_probe[1]\n",
    "probe_out = einops.einsum(residual_stream, white_to_play_probe, \"d_model, d_model row col options -> row col options\")\n",
    "probabilities = probe_out.softmax(dim=-1)\n",
    "plot_square_as_board(probabilities, facet_col=2, facet_labels=[\"P(Empty)\", \"P(White)\", \"P(Black)\"], title=\"Probabilities after move 30 for the white to play probe - correct A7 label!\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Computing Accuracy"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hopefully I've convinced you anecdotally that a linear probe works. But to be more rigorous, let's check accuracy across our 50 games."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "def state_stack_to_one_hot(state_stack):\n",
    "    one_hot = torch.zeros(\n",
    "        state_stack.shape[0], # num games\n",
    "        state_stack.shape[1], # num moves\n",
    "        8, # rows\n",
    "        8, # cols\n",
    "        3, # the two options\n",
    "        device=state_stack.device,\n",
    "        dtype=torch.int,\n",
    "    )\n",
    "    one_hot[..., 0] = state_stack == 0 # empty\n",
    "    one_hot[..., 1] = state_stack == -1 # white\n",
    "    one_hot[..., 2] = state_stack == 1 # black\n",
    "    \n",
    "    return one_hot\n",
    "\n",
    "# We first convert the board states to be in terms of my (+1) and their (-1)\n",
    "alternating = np.array([-1 if i%2 == 0 else 1 for i in range(focus_games_int.shape[1])])\n",
    "flipped_focus_states = focus_states * alternating[None, :, None, None]\n",
    "\n",
    "# We now convert to one hot\n",
    "focus_states_flipped_one_hot = state_stack_to_one_hot(torch.tensor(flipped_focus_states))\n",
    "\n",
    "# Take the argmax\n",
    "focus_states_flipped_value = focus_states_flipped_one_hot.argmax(dim=-1)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Apply the probe to the residual stream for every move! (Taken after layer 6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "probe_out = einops.einsum(focus_cache[\"resid_post\", 6], linear_probe, \"game move d_model, d_model row col options -> game move row col options\")\n",
    "probe_out_value = probe_out.argmax(dim=-1)\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Take the average accuracy across all games and the middle moves (5:-5) we see extremely low error rate on black to play moves, and fairly low error rate when applying a zero shot transfer to all moves (by flipping the labels) - and that it's worse near corners!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"9f2564e3-63ac-463d-8b9e-d6109887c388\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"9f2564e3-63ac-463d-8b9e-d6109887c388\")) {                    Plotly.newPlot(                        \"9f2564e3-63ac-463d-8b9e-d6109887c388\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.012000024,0.008000016,0.0055999756,0.0,0.0,0.0055999756,0.024800003,0.014400005],[0.0031999946,0.00080001354,0.0031999946,0.0,0.0,0.00080001354,0.00080001354,0.00880003],[0.00080001354,0.0048000216,0.0031999946,0.006399989,0.0016000271,0.0,0.0072000027,0.0055999756],[0.00080001354,0.0,0.004000008,0.00080001354,0.0031999946,0.004000008,0.0,0.0],[0.0,0.0,0.00880003,0.0072000027,0.002399981,0.0072000027,0.0,0.0],[0.0,0.002399981,0.0031999946,0.0031999946,0.0,0.002399981,0.00080001354,0.0],[0.011200011,0.0016000271,0.0,0.0,0.0,0.002399981,0.006399989,0.0],[0.00880003,0.010399997,0.0055999756,0.0016000271,0.0,0.002399981,0.002399981,0.014400005]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"1\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.010612249,0.009795904,0.004897952,0.0,0.0,0.004897952,0.024897933,0.020816326],[0.006530583,0.0012245178,0.0032653213,0.0,0.0,0.0012245178,0.0012245178,0.008979619],[0.0008163452,0.00612247,0.0020408034,0.0069387555,0.0016326308,0.0004081726,0.0069387555,0.0044897795],[0.0004081726,0.0,0.004081607,0.0020408034,0.0032653213,0.0028571486,0.0,0.0],[0.0004081726,0.0,0.008163273,0.007346928,0.0020408034,0.0069387555,0.0004081726,0.0008163452],[0.0004081726,0.0020408034,0.002448976,0.0016326308,0.0004081726,0.0016326308,0.0008163452,0.0004081726],[0.007346928,0.0016326308,0.0,0.0,0.0,0.002448976,0.00612247,0.0004081726],[0.008163273,0.0069387555,0.0028571486,0.0008163452,0.0004081726,0.0020408034,0.004081607,0.011020422]],\"type\":\"heatmap\",\"xaxis\":\"x2\",\"yaxis\":\"y2\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,0.49],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"xaxis2\":{\"anchor\":\"y2\",\"domain\":[0.51,1.0],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis2\":{\"anchor\":\"x2\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"annotations\":[{\"font\":{},\"showarrow\":false,\"text\":\"Black to Play moves\",\"x\":0.245,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"All Moves\",\"x\":0.755,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"}],\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0,\"cmin\":-0.25,\"cmax\":0.25},\"title\":{\"text\":\"Average Error Rate of Linear Probe\"}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('9f2564e3-63ac-463d-8b9e-d6109887c388');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "correct_middle_odd_answers = (probe_out_value.cpu() == focus_states_flipped_value[:, :-1])[:, 5:-5:2]\n",
    "accuracies_odd = einops.reduce(correct_middle_odd_answers.float(), \"game move row col -> row col\", \"mean\")\n",
    "correct_middle_answers = (probe_out_value.cpu() == focus_states_flipped_value[:, :-1])[:, 5:-5]\n",
    "accuracies = einops.reduce(correct_middle_answers.float(), \"game move row col -> row col\", \"mean\")\n",
    "\n",
    "plot_square_as_board(1 - torch.stack([accuracies_odd, accuracies], dim=0), title=\"Average Error Rate of Linear Probe\", facet_col=0, facet_labels=[\"Black to Play moves\", \"All Moves\"], zmax=0.25, zmin=-0.25)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Intervening with the probe"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One of the really exciting consequences of a linear probe is that it gives us a set of interpretable directions in the residual stream! And with this, we can not only interpret the model's representations, but we can also intervene in the model's reasoning. This is a good proof of concept that if you can *really* understand a model, you can get precise and detailed control over its behaviour."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first step is to convert our probe to meaningful directions. Each square's probe has 3 vectors, but the logits go into a softmax, which is translation invariant, so this only has two degrees of freedom. A natural-ish way to convert it into two vectors is taking `blank - (mine + their's)/2` giving a \"is this cell empty or not\" direction and `mine - their's` giving a \"conditional on being blank, is this my colour vs their's\" direction.\n",
    "\n",
    "Having a single meaningful direction is important, because it allows us to interpret a feature or intervene on it. The original three directions has one degree of freedom, so each direction is arbitrary on its own."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "blank_probe = linear_probe[..., 0] - linear_probe[..., 1] * 0.5 - linear_probe[..., 2] * 0.5\n",
    "my_probe = linear_probe[..., 2] - linear_probe[..., 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"4758442e-4435-4403-a8d9-4257306f8bac\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"4758442e-4435-4403-a8d9-4257306f8bac\")) {                    Plotly.newPlot(                        \"4758442e-4435-4403-a8d9-4257306f8bac\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.0,0.0,0.0,0.0,-1.0,0.0,0.1,0.0],[0.0,0.0,0.1,-0.5,-1.0,-1.0,0.1,1.0],[0.0,0.1,-1.0,-1.0,1.0,-1.0,1.0,-0.5],[0.0,-0.5,0.1,-1.0,1.0,-1.0,0.1,-1.0],[0.0,-0.5,1.0,1.0,0.9,-1.0,0.1,0.0],[0.0,-0.5,-0.5,-1.0,0.9,0.9,1.0,-0.5],[0.0,0.0,-1.0,-1.0,-0.5,-0.5,-0.5,1.0],[0.0,-1.0,0.0,0.1,0.0,0.0,0.0,0.0]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"<b>%{y}%{x}</b><br>%{customdata}<extra></extra>\",\"text\":[[\"A0\",\"A1\",\"A2\",\"A3\",\"\",\"A5\",\"A6\",\"A7\"],[\"B0\",\"B1\",\"B2\",\"B3\",\"\",\"\",\"B6\",\"\"],[\"C0\",\"C1\",\"\",\"\",\"\",\"\",\"\",\"C7\"],[\"D0\",\"D1\",\"D2\",\"\",\"\",\"\",\"D6\",\"\"],[\"E0\",\"E1\",\"\",\"\",\"\",\"\",\"E6\",\"E7\"],[\"F0\",\"F1\",\"F2\",\"\",\"\",\"\",\"\",\"F7\"],[\"G0\",\"G1\",\"\",\"\",\"G4\",\"G5\",\"G6\",\"\"],[\"H0\",\"\",\"H2\",\"H3\",\"H4\",\"H5\",\"H6\",\"H7\"]],\"texttemplate\":\"%{text}\",\"customdata\":[[\"I:1<br>S:0\",\"I:2<br>S:1\",\"I:3<br>S:2\",\"I:4<br>S:3\",\"I:5<br>S:4\",\"I:6<br>S:5\",\"I:7<br>S:6\",\"I:8<br>S:7\"],[\"I:9<br>S:8\",\"I:10<br>S:9\",\"I:11<br>S:10\",\"I:12<br>S:11\",\"I:13<br>S:12\",\"I:14<br>S:13\",\"I:15<br>S:14\",\"I:16<br>S:15\"],[\"I:17<br>S:16\",\"I:18<br>S:17\",\"I:19<br>S:18\",\"I:20<br>S:19\",\"I:21<br>S:20\",\"I:22<br>S:21\",\"I:23<br>S:22\",\"I:24<br>S:23\"],[\"I:25<br>S:24\",\"I:26<br>S:25\",\"I:27<br>S:26\",\"I:X<br>S:27\",\"I:X<br>S:28\",\"I:28<br>S:29\",\"I:29<br>S:30\",\"I:30<br>S:31\"],[\"I:31<br>S:32\",\"I:32<br>S:33\",\"I:33<br>S:34\",\"I:X<br>S:35\",\"I:X<br>S:36\",\"I:34<br>S:37\",\"I:35<br>S:38\",\"I:36<br>S:39\"],[\"I:37<br>S:40\",\"I:38<br>S:41\",\"I:39<br>S:42\",\"I:40<br>S:43\",\"I:41<br>S:44\",\"I:42<br>S:45\",\"I:43<br>S:46\",\"I:44<br>S:47\"],[\"I:45<br>S:48\",\"I:46<br>S:49\",\"I:47<br>S:50\",\"I:48<br>S:51\",\"I:49<br>S:52\",\"I:50<br>S:53\",\"I:51<br>S:54\",\"I:52<br>S:55\"],[\"I:53<br>S:56\",\"I:54<br>S:57\",\"I:55<br>S:58\",\"I:56<br>S:59\",\"I:57<br>S:60\",\"I:58<br>S:61\",\"I:59<br>S:62\",\"I:60<br>S:63\"]]}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(0, 128, 128)\"],[0.16666666666666666,\"rgb(112, 164, 148)\"],[0.3333333333333333,\"rgb(180, 200, 168)\"],[0.5,\"rgb(246, 237, 189)\"],[0.6666666666666666,\"rgb(237, 187, 138)\"],[0.8333333333333334,\"rgb(222, 138, 90)\"],[1.0,\"rgb(202, 86, 44)\"]],\"cmid\":0.0},\"title\":{\"text\":\"White To Play. Board State After Black Plays F4 \",\"x\":0.5},\"shapes\":[{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":-0.2,\"y1\":0.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":6.8,\"x1\":7.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":6.8,\"x1\":7.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.78,\"x1\":4.22,\"y0\":3.78,\"y1\":4.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.75,\"x1\":4.25,\"y0\":4.75,\"y1\":5.25,\"line\":{\"color\":\"red\",\"width\":4}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.78,\"x1\":5.22,\"y0\":4.78,\"y1\":5.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":6.8,\"x1\":7.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":0.8,\"x1\":1.2,\"y0\":6.8,\"y1\":7.2,\"line\":{\"color\":\"green\",\"width\":0.5}}]},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('4758442e-4435-4403-a8d9-4257306f8bac');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "pos = 20\n",
    "game_index = 0\n",
    "moves = focus_games_string[game_index, :pos+1]\n",
    "plot_single_board(moves)\n",
    "state = torch.zeros((64,), dtype=torch.float32, device=\"cuda\") - 10.\n",
    "state[stoi_indices] = focus_logits[game_index, pos].log_softmax(dim=-1)[1:]\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now flip cell F4 from black to white. This makes D2 into a legal move and G4 into an illegal move."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Flipping the color of cell F4\n",
      "newly_legal ['D2']\n",
      "newly_illegal ['G4']\n"
     ]
    }
   ],
   "source": [
    "cell_r = 5\n",
    "cell_c = 4\n",
    "print(f\"Flipping the color of cell {'ABCDEFGH'[cell_r]}{cell_c}\")\n",
    "\n",
    "board = OthelloBoardState()\n",
    "board.update(moves.tolist())\n",
    "board_state = board.state.copy()\n",
    "valid_moves = board.get_valid_moves()\n",
    "flipped_board = copy.deepcopy(board)\n",
    "flipped_board.state[cell_r, cell_c] *= -1\n",
    "flipped_valid_moves = flipped_board.get_valid_moves()\n",
    "\n",
    "newly_legal = [string_to_label(move) for move in flipped_valid_moves if move not in valid_moves]\n",
    "newly_illegal = [string_to_label(move) for move in valid_moves if move not in flipped_valid_moves]\n",
    "print(\"newly_legal\", newly_legal)\n",
    "print(\"newly_illegal\", newly_illegal)\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now intervene on the model's residual stream using the \"my colour vs their colour\" direction. I get the best results intervening after layer 4. This is a **linear intervention** - we are just changing a single dimension of the residual stream and keeping the others unchanged. This is a fairly simple intervention, and it's striking that it works!\n",
    "\n",
    "I apply the fairly janky technique of taking current coordinate in the given direction, negating it, and then multiply by a hyperparameter called `scale` (scale between 1 and 8 tends to work best - small isn't enough and big tends to break things). I haven't tried hard to optimise this and I'm sure it can be improved! Eg by replacing the model's coordinate by a constant rather than scaling it. I also haven't dug into the best scale parameters, or which ones work best in which contexts - plausibly different cells have different activation scales on their world models and need different behaviour! "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"8fb7021b-0abe-4dff-a39d-d03121154f29\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"8fb7021b-0abe-4dff-a39d-d03121154f29\")) {                    Plotly.newPlot(                        \"8fb7021b-0abe-4dff-a39d-d03121154f29\",                        [{\"customdata\":[[\"A0\"],[\"A1\"],[\"A2\"],[\"A3\"],[\"A4\"],[\"A5\"],[\"A6\"],[\"A7\"],[\"B0\"],[\"B1\"],[\"B2\"],[\"B3\"],[\"B4\"],[\"B5\"],[\"B6\"],[\"B7\"],[\"C0\"],[\"C1\"],[\"C2\"],[\"C3\"],[\"C4\"],[\"C5\"],[\"C6\"],[\"C7\"],[\"D0\"],[\"D1\"],[\"D2\"],[\"D3\"],[\"D4\"],[\"D5\"],[\"D6\"],[\"D7\"],[\"E0\"],[\"E1\"],[\"E2\"],[\"E3\"],[\"E4\"],[\"E5\"],[\"E6\"],[\"E7\"],[\"F0\"],[\"F1\"],[\"F2\"],[\"F3\"],[\"F4\"],[\"F5\"],[\"F6\"],[\"F7\"],[\"G0\"],[\"G1\"],[\"G2\"],[\"G3\"],[\"G4\"],[\"G5\"],[\"G6\"],[\"G7\"],[\"H0\"],[\"H1\"],[\"H2\"],[\"H3\"],[\"H4\"],[\"H5\"],[\"H6\"],[\"H7\"]],\"hovertemplate\":\"facet=0<br>Flipped=%{x}<br>Original=%{y}<br>data=%{customdata[0]}<br>Newly Legal=%{marker.color}<extra></extra>\",\"legendgroup\":\"\",\"marker\":{\"color\":[0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,1.0,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,-1.0,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2],\"coloraxis\":\"coloraxis\",\"symbol\":\"circle\"},\"mode\":\"markers\",\"name\":\"\",\"orientation\":\"v\",\"showlegend\":false,\"x\":[-13.263582,-13.249046,-13.090075,-12.714336,-12.538256,-13.064356,-12.093853,-12.994802,-13.181288,-13.004301,-11.973778,-2.2651772,-11.218109,-12.984026,-11.537514,-13.171612,-12.382532,-12.441885,-12.683575,-12.9927025,-13.559019,-12.583767,-13.086395,-2.2547174,-13.108604,-2.321653,-8.533272,-10.0,-10.0,-12.220243,-12.334602,-13.528233,-12.601536,-2.2952104,-12.134384,-10.0,-10.0,-12.612669,-13.824457,-11.99568,-12.140308,-2.3214679,-2.3641257,-12.145279,-13.076389,-11.644718,-12.242552,-2.2567859,-13.1744995,-11.515083,-13.461906,-11.718277,-2.2316146,-2.3181682,-2.4150872,-10.572248,-12.0749855,-12.134857,-13.092152,-12.326824,-12.654989,-12.515928,-12.436363,-11.837301],\"xaxis\":\"x\",\"y\":[-13.240010261535645,-13.259102821350098,-13.178929328918457,-12.691011428833008,-12.683109283447266,-13.167738914489746,-12.102622985839844,-13.118292808532715,-12.898503303527832,-12.972034454345703,-12.101619720458984,-2.268888473510742,-11.304240226745605,-13.033951759338379,-11.63598918914795,-13.23884105682373,-12.364408493041992,-12.24697494506836,-12.757281303405762,-12.971112251281738,-13.55956745147705,-12.736376762390137,-13.172277450561523,-2.2601823806762695,-13.157661437988281,-2.305391311645508,-11.544709205627441,-10.0,-10.0,-12.24409008026123,-11.506229400634766,-13.4607572555542,-12.536737442016602,-2.2855100631713867,-12.203991889953613,-10.0,-10.0,-12.88708209991455,-13.768584251403809,-11.798258781433105,-12.231013298034668,-2.324108123779297,-2.359358787536621,-12.493218421936035,-13.294095039367676,-12.01527214050293,-12.322196006774902,-2.281527519226074,-13.316290855407715,-11.821861267089844,-13.586751937866211,-11.875118255615234,-2.2374086380004883,-2.3100643157958984,-2.4067373275756836,-10.679027557373047,-12.133708953857422,-12.213635444641113,-12.793862342834473,-12.284290313720703,-12.49877643585205,-12.545167922973633,-12.576452255249023,-11.837255477905273],\"yaxis\":\"y\",\"type\":\"scatter\"},{\"customdata\":[[\"A0\"],[\"A1\"],[\"A2\"],[\"A3\"],[\"A4\"],[\"A5\"],[\"A6\"],[\"A7\"],[\"B0\"],[\"B1\"],[\"B2\"],[\"B3\"],[\"B4\"],[\"B5\"],[\"B6\"],[\"B7\"],[\"C0\"],[\"C1\"],[\"C2\"],[\"C3\"],[\"C4\"],[\"C5\"],[\"C6\"],[\"C7\"],[\"D0\"],[\"D1\"],[\"D2\"],[\"D3\"],[\"D4\"],[\"D5\"],[\"D6\"],[\"D7\"],[\"E0\"],[\"E1\"],[\"E2\"],[\"E3\"],[\"E4\"],[\"E5\"],[\"E6\"],[\"E7\"],[\"F0\"],[\"F1\"],[\"F2\"],[\"F3\"],[\"F4\"],[\"F5\"],[\"F6\"],[\"F7\"],[\"G0\"],[\"G1\"],[\"G2\"],[\"G3\"],[\"G4\"],[\"G5\"],[\"G6\"],[\"G7\"],[\"H0\"],[\"H1\"],[\"H2\"],[\"H3\"],[\"H4\"],[\"H5\"],[\"H6\"],[\"H7\"]],\"hovertemplate\":\"facet=1<br>Flipped=%{x}<br>Original=%{y}<br>data=%{customdata[0]}<br>Newly Legal=%{marker.color}<extra></extra>\",\"legendgroup\":\"\",\"marker\":{\"color\":[0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,1.0,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,-1.0,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2],\"coloraxis\":\"coloraxis\",\"symbol\":\"circle\"},\"mode\":\"markers\",\"name\":\"\",\"orientation\":\"v\",\"showlegend\":false,\"x\":[-13.110386,-13.124985,-12.934035,-12.626893,-12.390819,-12.929208,-12.033607,-13.014115,-12.705073,-12.984462,-11.687938,-2.227252,-11.090001,-12.855896,-11.307533,-13.049039,-12.272719,-12.328091,-12.629581,-12.897787,-13.463066,-12.365051,-13.058341,-2.210495,-12.9783745,-2.295557,-2.6238155,-10.0,-10.0,-12.018996,-12.585022,-13.413386,-12.467137,-2.260521,-12.076609,-10.0,-10.0,-12.406845,-13.732444,-11.984737,-11.913776,-2.2758741,-2.330206,-11.923195,-12.674602,-11.431095,-12.009875,-2.2063875,-12.904277,-11.272495,-13.345633,-11.696317,-6.728386,-2.3054552,-2.3670359,-10.481035,-12.089015,-12.116072,-13.028611,-12.144491,-12.78615,-12.355658,-12.267385,-11.612394],\"xaxis\":\"x2\",\"y\":[-13.240010261535645,-13.259102821350098,-13.178929328918457,-12.691011428833008,-12.683109283447266,-13.167738914489746,-12.102622985839844,-13.118292808532715,-12.898503303527832,-12.972034454345703,-12.101619720458984,-2.268888473510742,-11.304240226745605,-13.033951759338379,-11.63598918914795,-13.23884105682373,-12.364408493041992,-12.24697494506836,-12.757281303405762,-12.971112251281738,-13.55956745147705,-12.736376762390137,-13.172277450561523,-2.2601823806762695,-13.157661437988281,-2.305391311645508,-11.544709205627441,-10.0,-10.0,-12.24409008026123,-11.506229400634766,-13.4607572555542,-12.536737442016602,-2.2855100631713867,-12.203991889953613,-10.0,-10.0,-12.88708209991455,-13.768584251403809,-11.798258781433105,-12.231013298034668,-2.324108123779297,-2.359358787536621,-12.493218421936035,-13.294095039367676,-12.01527214050293,-12.322196006774902,-2.281527519226074,-13.316290855407715,-11.821861267089844,-13.586751937866211,-11.875118255615234,-2.2374086380004883,-2.3100643157958984,-2.4067373275756836,-10.679027557373047,-12.133708953857422,-12.213635444641113,-12.793862342834473,-12.284290313720703,-12.49877643585205,-12.545167922973633,-12.576452255249023,-11.837255477905273],\"yaxis\":\"y2\",\"type\":\"scatter\"},{\"customdata\":[[\"A0\"],[\"A1\"],[\"A2\"],[\"A3\"],[\"A4\"],[\"A5\"],[\"A6\"],[\"A7\"],[\"B0\"],[\"B1\"],[\"B2\"],[\"B3\"],[\"B4\"],[\"B5\"],[\"B6\"],[\"B7\"],[\"C0\"],[\"C1\"],[\"C2\"],[\"C3\"],[\"C4\"],[\"C5\"],[\"C6\"],[\"C7\"],[\"D0\"],[\"D1\"],[\"D2\"],[\"D3\"],[\"D4\"],[\"D5\"],[\"D6\"],[\"D7\"],[\"E0\"],[\"E1\"],[\"E2\"],[\"E3\"],[\"E4\"],[\"E5\"],[\"E6\"],[\"E7\"],[\"F0\"],[\"F1\"],[\"F2\"],[\"F3\"],[\"F4\"],[\"F5\"],[\"F6\"],[\"F7\"],[\"G0\"],[\"G1\"],[\"G2\"],[\"G3\"],[\"G4\"],[\"G5\"],[\"G6\"],[\"G7\"],[\"H0\"],[\"H1\"],[\"H2\"],[\"H3\"],[\"H4\"],[\"H5\"],[\"H6\"],[\"H7\"]],\"hovertemplate\":\"facet=2<br>Flipped=%{x}<br>Original=%{y}<br>data=%{customdata[0]}<br>Newly Legal=%{marker.color}<extra></extra>\",\"legendgroup\":\"\",\"marker\":{\"color\":[0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,1.0,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,-1.0,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2],\"coloraxis\":\"coloraxis\",\"symbol\":\"circle\"},\"mode\":\"markers\",\"name\":\"\",\"orientation\":\"v\",\"showlegend\":false,\"x\":[-13.060325,-13.035168,-12.949223,-12.612062,-12.509002,-12.944947,-12.152205,-13.106321,-12.408462,-12.993851,-11.486622,-2.2654989,-10.988355,-12.826299,-10.984914,-13.088125,-12.09593,-12.089989,-12.667674,-12.850815,-13.433948,-12.277788,-13.286498,-2.2425401,-12.939765,-2.327728,-2.262451,-10.0,-10.0,-11.895252,-12.472575,-13.203834,-12.319026,-2.289868,-12.177435,-10.0,-10.0,-12.360599,-13.661086,-11.913662,-11.794414,-2.3064725,-2.365875,-12.024127,-12.58133,-11.501542,-11.965883,-2.2507064,-12.812397,-11.0216675,-13.426905,-11.786752,-10.125915,-2.3398216,-2.389366,-10.711594,-12.138941,-12.211348,-12.931433,-11.989243,-12.882668,-12.386763,-12.260822,-11.503483],\"xaxis\":\"x3\",\"y\":[-13.240010261535645,-13.259102821350098,-13.178929328918457,-12.691011428833008,-12.683109283447266,-13.167738914489746,-12.102622985839844,-13.118292808532715,-12.898503303527832,-12.972034454345703,-12.101619720458984,-2.268888473510742,-11.304240226745605,-13.033951759338379,-11.63598918914795,-13.23884105682373,-12.364408493041992,-12.24697494506836,-12.757281303405762,-12.971112251281738,-13.55956745147705,-12.736376762390137,-13.172277450561523,-2.2601823806762695,-13.157661437988281,-2.305391311645508,-11.544709205627441,-10.0,-10.0,-12.24409008026123,-11.506229400634766,-13.4607572555542,-12.536737442016602,-2.2855100631713867,-12.203991889953613,-10.0,-10.0,-12.88708209991455,-13.768584251403809,-11.798258781433105,-12.231013298034668,-2.324108123779297,-2.359358787536621,-12.493218421936035,-13.294095039367676,-12.01527214050293,-12.322196006774902,-2.281527519226074,-13.316290855407715,-11.821861267089844,-13.586751937866211,-11.875118255615234,-2.2374086380004883,-2.3100643157958984,-2.4067373275756836,-10.679027557373047,-12.133708953857422,-12.213635444641113,-12.793862342834473,-12.284290313720703,-12.49877643585205,-12.545167922973633,-12.576452255249023,-11.837255477905273],\"yaxis\":\"y3\",\"type\":\"scatter\"},{\"customdata\":[[\"A0\"],[\"A1\"],[\"A2\"],[\"A3\"],[\"A4\"],[\"A5\"],[\"A6\"],[\"A7\"],[\"B0\"],[\"B1\"],[\"B2\"],[\"B3\"],[\"B4\"],[\"B5\"],[\"B6\"],[\"B7\"],[\"C0\"],[\"C1\"],[\"C2\"],[\"C3\"],[\"C4\"],[\"C5\"],[\"C6\"],[\"C7\"],[\"D0\"],[\"D1\"],[\"D2\"],[\"D3\"],[\"D4\"],[\"D5\"],[\"D6\"],[\"D7\"],[\"E0\"],[\"E1\"],[\"E2\"],[\"E3\"],[\"E4\"],[\"E5\"],[\"E6\"],[\"E7\"],[\"F0\"],[\"F1\"],[\"F2\"],[\"F3\"],[\"F4\"],[\"F5\"],[\"F6\"],[\"F7\"],[\"G0\"],[\"G1\"],[\"G2\"],[\"G3\"],[\"G4\"],[\"G5\"],[\"G6\"],[\"G7\"],[\"H0\"],[\"H1\"],[\"H2\"],[\"H3\"],[\"H4\"],[\"H5\"],[\"H6\"],[\"H7\"]],\"hovertemplate\":\"facet=3<br>Flipped=%{x}<br>Original=%{y}<br>data=%{customdata[0]}<br>Newly Legal=%{marker.color}<extra></extra>\",\"legendgroup\":\"\",\"marker\":{\"color\":[0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,1.0,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,-1.0,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2],\"coloraxis\":\"coloraxis\",\"symbol\":\"circle\"},\"mode\":\"markers\",\"name\":\"\",\"orientation\":\"v\",\"showlegend\":false,\"x\":[-12.767947,-12.544588,-12.663684,-12.07008,-12.689725,-12.7041235,-12.013571,-12.778601,-12.230126,-12.5277815,-10.980312,-2.2419112,-10.371425,-12.505769,-9.987355,-12.826871,-11.468305,-11.538194,-12.433103,-12.455946,-13.125098,-11.984142,-13.497543,-2.2117798,-12.585881,-2.2869685,-2.22948,-10.0,-10.0,-11.615395,-11.21364,-12.3886,-11.573266,-2.246401,-12.070855,-10.0,-10.0,-11.995426,-13.01776,-11.243777,-11.497308,-2.276211,-2.333769,-11.902192,-13.095113,-11.352511,-11.779704,-2.221905,-12.281398,-9.890826,-13.549094,-11.998104,-11.030593,-2.7444751,-2.3373263,-10.969056,-11.652064,-12.135392,-12.741197,-11.8005705,-12.842389,-12.265981,-12.2644615,-11.163893],\"xaxis\":\"x4\",\"y\":[-13.240010261535645,-13.259102821350098,-13.178929328918457,-12.691011428833008,-12.683109283447266,-13.167738914489746,-12.102622985839844,-13.118292808532715,-12.898503303527832,-12.972034454345703,-12.101619720458984,-2.268888473510742,-11.304240226745605,-13.033951759338379,-11.63598918914795,-13.23884105682373,-12.364408493041992,-12.24697494506836,-12.757281303405762,-12.971112251281738,-13.55956745147705,-12.736376762390137,-13.172277450561523,-2.2601823806762695,-13.157661437988281,-2.305391311645508,-11.544709205627441,-10.0,-10.0,-12.24409008026123,-11.506229400634766,-13.4607572555542,-12.536737442016602,-2.2855100631713867,-12.203991889953613,-10.0,-10.0,-12.88708209991455,-13.768584251403809,-11.798258781433105,-12.231013298034668,-2.324108123779297,-2.359358787536621,-12.493218421936035,-13.294095039367676,-12.01527214050293,-12.322196006774902,-2.281527519226074,-13.316290855407715,-11.821861267089844,-13.586751937866211,-11.875118255615234,-2.2374086380004883,-2.3100643157958984,-2.4067373275756836,-10.679027557373047,-12.133708953857422,-12.213635444641113,-12.793862342834473,-12.284290313720703,-12.49877643585205,-12.545167922973633,-12.576452255249023,-11.837255477905273],\"yaxis\":\"y4\",\"type\":\"scatter\"},{\"customdata\":[[\"A0\"],[\"A1\"],[\"A2\"],[\"A3\"],[\"A4\"],[\"A5\"],[\"A6\"],[\"A7\"],[\"B0\"],[\"B1\"],[\"B2\"],[\"B3\"],[\"B4\"],[\"B5\"],[\"B6\"],[\"B7\"],[\"C0\"],[\"C1\"],[\"C2\"],[\"C3\"],[\"C4\"],[\"C5\"],[\"C6\"],[\"C7\"],[\"D0\"],[\"D1\"],[\"D2\"],[\"D3\"],[\"D4\"],[\"D5\"],[\"D6\"],[\"D7\"],[\"E0\"],[\"E1\"],[\"E2\"],[\"E3\"],[\"E4\"],[\"E5\"],[\"E6\"],[\"E7\"],[\"F0\"],[\"F1\"],[\"F2\"],[\"F3\"],[\"F4\"],[\"F5\"],[\"F6\"],[\"F7\"],[\"G0\"],[\"G1\"],[\"G2\"],[\"G3\"],[\"G4\"],[\"G5\"],[\"G6\"],[\"G7\"],[\"H0\"],[\"H1\"],[\"H2\"],[\"H3\"],[\"H4\"],[\"H5\"],[\"H6\"],[\"H7\"]],\"hovertemplate\":\"facet=4<br>Flipped=%{x}<br>Original=%{y}<br>data=%{customdata[0]}<br>Newly Legal=%{marker.color}<extra></extra>\",\"legendgroup\":\"\",\"marker\":{\"color\":[0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,1.0,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,-1.0,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2],\"coloraxis\":\"coloraxis\",\"symbol\":\"circle\"},\"mode\":\"markers\",\"name\":\"\",\"orientation\":\"v\",\"showlegend\":false,\"x\":[-11.831977,-11.118,-11.749484,-10.325095,-12.648072,-11.607033,-11.195972,-11.96593,-12.068949,-10.354589,-9.408832,-2.1687503,-9.839478,-11.417052,-7.557503,-11.681241,-9.956921,-10.5724535,-11.349158,-11.540028,-12.224445,-11.64897,-13.065686,-2.2017937,-11.327217,-2.1053352,-2.3542457,-10.0,-10.0,-10.94831,-6.3302317,-10.789858,-9.697388,-2.0885324,-11.019801,-10.0,-10.0,-11.145771,-10.765352,-9.7554035,-11.028031,-2.1982598,-2.3437,-10.844018,-15.168143,-10.068995,-10.042859,-2.121829,-10.960938,-8.453096,-13.28398,-11.706562,-13.24241,-5.1481304,-2.3184562,-10.553234,-10.810833,-11.336258,-11.832045,-11.218578,-12.102326,-11.787816,-12.209622,-10.260943],\"xaxis\":\"x5\",\"y\":[-13.240010261535645,-13.259102821350098,-13.178929328918457,-12.691011428833008,-12.683109283447266,-13.167738914489746,-12.102622985839844,-13.118292808532715,-12.898503303527832,-12.972034454345703,-12.101619720458984,-2.268888473510742,-11.304240226745605,-13.033951759338379,-11.63598918914795,-13.23884105682373,-12.364408493041992,-12.24697494506836,-12.757281303405762,-12.971112251281738,-13.55956745147705,-12.736376762390137,-13.172277450561523,-2.2601823806762695,-13.157661437988281,-2.305391311645508,-11.544709205627441,-10.0,-10.0,-12.24409008026123,-11.506229400634766,-13.4607572555542,-12.536737442016602,-2.2855100631713867,-12.203991889953613,-10.0,-10.0,-12.88708209991455,-13.768584251403809,-11.798258781433105,-12.231013298034668,-2.324108123779297,-2.359358787536621,-12.493218421936035,-13.294095039367676,-12.01527214050293,-12.322196006774902,-2.281527519226074,-13.316290855407715,-11.821861267089844,-13.586751937866211,-11.875118255615234,-2.2374086380004883,-2.3100643157958984,-2.4067373275756836,-10.679027557373047,-12.133708953857422,-12.213635444641113,-12.793862342834473,-12.284290313720703,-12.49877643585205,-12.545167922973633,-12.576452255249023,-11.837255477905273],\"yaxis\":\"y5\",\"type\":\"scatter\"},{\"customdata\":[[\"A0\"],[\"A1\"],[\"A2\"],[\"A3\"],[\"A4\"],[\"A5\"],[\"A6\"],[\"A7\"],[\"B0\"],[\"B1\"],[\"B2\"],[\"B3\"],[\"B4\"],[\"B5\"],[\"B6\"],[\"B7\"],[\"C0\"],[\"C1\"],[\"C2\"],[\"C3\"],[\"C4\"],[\"C5\"],[\"C6\"],[\"C7\"],[\"D0\"],[\"D1\"],[\"D2\"],[\"D3\"],[\"D4\"],[\"D5\"],[\"D6\"],[\"D7\"],[\"E0\"],[\"E1\"],[\"E2\"],[\"E3\"],[\"E4\"],[\"E5\"],[\"E6\"],[\"E7\"],[\"F0\"],[\"F1\"],[\"F2\"],[\"F3\"],[\"F4\"],[\"F5\"],[\"F6\"],[\"F7\"],[\"G0\"],[\"G1\"],[\"G2\"],[\"G3\"],[\"G4\"],[\"G5\"],[\"G6\"],[\"G7\"],[\"H0\"],[\"H1\"],[\"H2\"],[\"H3\"],[\"H4\"],[\"H5\"],[\"H6\"],[\"H7\"]],\"hovertemplate\":\"facet=5<br>Flipped=%{x}<br>Original=%{y}<br>data=%{customdata[0]}<br>Newly Legal=%{marker.color}<extra></extra>\",\"legendgroup\":\"\",\"marker\":{\"color\":[0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,1.0,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,-1.0,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2],\"coloraxis\":\"coloraxis\",\"symbol\":\"circle\"},\"mode\":\"markers\",\"name\":\"\",\"orientation\":\"v\",\"showlegend\":false,\"x\":[-9.056084,-8.136989,-8.87572,-6.5317554,-10.72073,-8.513678,-8.258926,-9.881763,-8.862087,-6.199177,-5.9878173,-2.0650296,-8.807329,-8.675507,-3.9982524,-9.027394,-7.194085,-7.040561,-8.519533,-8.681442,-10.172552,-9.5152645,-10.208042,-3.8589597,-8.293087,-1.7096782,-3.637601,-10.0,-10.0,-8.749128,-4.834285,-8.448879,-7.086385,-1.4923091,-8.438293,-10.0,-10.0,-8.941038,-6.529465,-7.311771,-9.327493,-2.0258365,-2.380909,-7.5716095,-14.909859,-7.957783,-7.539254,-2.4303064,-8.237187,-7.334545,-10.819197,-9.225867,-14.829643,-8.483315,-2.7513914,-8.422981,-9.158733,-9.212278,-9.811462,-9.106667,-9.735493,-9.960007,-10.145855,-8.131702],\"xaxis\":\"x6\",\"y\":[-13.240010261535645,-13.259102821350098,-13.178929328918457,-12.691011428833008,-12.683109283447266,-13.167738914489746,-12.102622985839844,-13.118292808532715,-12.898503303527832,-12.972034454345703,-12.101619720458984,-2.268888473510742,-11.304240226745605,-13.033951759338379,-11.63598918914795,-13.23884105682373,-12.364408493041992,-12.24697494506836,-12.757281303405762,-12.971112251281738,-13.55956745147705,-12.736376762390137,-13.172277450561523,-2.2601823806762695,-13.157661437988281,-2.305391311645508,-11.544709205627441,-10.0,-10.0,-12.24409008026123,-11.506229400634766,-13.4607572555542,-12.536737442016602,-2.2855100631713867,-12.203991889953613,-10.0,-10.0,-12.88708209991455,-13.768584251403809,-11.798258781433105,-12.231013298034668,-2.324108123779297,-2.359358787536621,-12.493218421936035,-13.294095039367676,-12.01527214050293,-12.322196006774902,-2.281527519226074,-13.316290855407715,-11.821861267089844,-13.586751937866211,-11.875118255615234,-2.2374086380004883,-2.3100643157958984,-2.4067373275756836,-10.679027557373047,-12.133708953857422,-12.213635444641113,-12.793862342834473,-12.284290313720703,-12.49877643585205,-12.545167922973633,-12.576452255249023,-11.837255477905273],\"yaxis\":\"y6\",\"type\":\"scatter\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,0.15],\"title\":{\"text\":\"Flipped\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"Original\"}},\"xaxis2\":{\"anchor\":\"y2\",\"domain\":[0.16999999999999998,0.31999999999999995],\"matches\":\"x\",\"title\":{\"text\":\"Flipped\"}},\"yaxis2\":{\"anchor\":\"x2\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis3\":{\"anchor\":\"y3\",\"domain\":[0.33999999999999997,0.49],\"matches\":\"x\",\"title\":{\"text\":\"Flipped\"}},\"yaxis3\":{\"anchor\":\"x3\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis4\":{\"anchor\":\"y4\",\"domain\":[0.51,0.66],\"matches\":\"x\",\"title\":{\"text\":\"Flipped\"}},\"yaxis4\":{\"anchor\":\"x4\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis5\":{\"anchor\":\"y5\",\"domain\":[0.6799999999999999,0.83],\"matches\":\"x\",\"title\":{\"text\":\"Flipped\"}},\"yaxis5\":{\"anchor\":\"x5\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis6\":{\"anchor\":\"y6\",\"domain\":[0.85,1.0],\"matches\":\"x\",\"title\":{\"text\":\"Flipped\"}},\"yaxis6\":{\"anchor\":\"x6\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"annotations\":[{\"font\":{},\"showarrow\":false,\"text\":\"Translate by 0x\",\"x\":0.075,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"Translate by 1x\",\"x\":0.24499999999999997,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"Translate by 2x\",\"x\":0.415,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"Translate by 4x\",\"x\":0.585,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"Translate by 8x\",\"x\":0.7549999999999999,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"Translate by 16x\",\"x\":0.925,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"}],\"coloraxis\":{\"colorbar\":{\"title\":{\"text\":\"Newly Legal\"}},\"colorscale\":[[0.0,\"rgb(0, 128, 128)\"],[0.16666666666666666,\"rgb(112, 164, 148)\"],[0.3333333333333333,\"rgb(180, 200, 168)\"],[0.5,\"rgb(246, 237, 189)\"],[0.6666666666666666,\"rgb(237, 187, 138)\"],[0.8333333333333334,\"rgb(222, 138, 90)\"],[1.0,\"rgb(202, 86, 44)\"]]},\"legend\":{\"tracegroupgap\":0},\"title\":{\"text\":\"Original vs Flipped F4 at Layer 4\"}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('8fb7021b-0abe-4dff-a39d-d03121154f29');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "flip_dir = my_probe[:, cell_r, cell_c]\n",
    "\n",
    "big_flipped_states_list = []\n",
    "layer = 4\n",
    "scales = [0, 1, 2, 4, 8, 16]\n",
    "for scale in scales:\n",
    "    def flip_hook(resid, hook):\n",
    "        coeff = resid[0, pos] @ flip_dir/flip_dir.norm()\n",
    "        # if coeff.item() > 0:\n",
    "        resid[0, pos] -= (scale+1) * coeff * flip_dir/flip_dir.norm()\n",
    "    flipped_logits = model.run_with_hooks(focus_games_int[game_index:game_index+1, :pos+1],\n",
    "                        fwd_hooks=[\n",
    "                        #  (\"blocks.3.hook_resid_post\", flip_hook),\n",
    "                        (f\"blocks.{layer}.hook_resid_post\", flip_hook),\n",
    "                        #  (\"blocks.5.hook_resid_post\", flip_hook),\n",
    "                        #  (\"blocks.6.hook_resid_post\", flip_hook),\n",
    "                        #  (\"blocks.7.hook_resid_post\", flip_hook),\n",
    "                        ]\n",
    "                        ).log_softmax(dim=-1)[0, pos]\n",
    "    flip_state = torch.zeros((64,), dtype=torch.float32, device=\"cuda\") - 10.\n",
    "    flip_state[stoi_indices] = flipped_logits.log_softmax(dim=-1)[1:]\n",
    "    big_flipped_states_list.append(flip_state)\n",
    "flip_state_big = torch.stack(big_flipped_states_list)\n",
    "state_big = einops.repeat(state, \"d -> b d\", b=6)\n",
    "color = torch.zeros((len(scales), 64)).cuda() + 0.2\n",
    "for s in newly_legal:\n",
    "    color[:, to_string(s)] = 1\n",
    "for s in newly_illegal:\n",
    "    color[:, to_string(s)] = -1\n",
    "scatter(y=state_big, x=flip_state_big, title=f\"Original vs Flipped {string_to_label(8*cell_r+cell_c)} at Layer {layer}\", xaxis=\"Flipped\", yaxis=\"Original\", hover=[f\"{r}{c}\" for r in \"ABCDEFGH\" for c in range(8)], facet_col=0, facet_labels=[f\"Translate by {i}x\" for i in scales], color=color, color_name=\"Newly Legal\", color_continuous_scale=\"Geyser\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Looking for Modular Circuits"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Probing Across Layers"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The probe's inputs are accumulated in the residual stream over the six layers - the residual stream is the sum of the output of each previous head and neuron. We can therefore analyse which previous model components most contribute to the overall probe computation, and use this to identify the end of the world model computing circuit."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's analyse move 20 in game 1, where we can see that the probe has perfect accuracy after layer 6"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"5434a32a-8832-41c5-abd5-fa51540f54fd\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"5434a32a-8832-41c5-abd5-fa51540f54fd\")) {                    Plotly.newPlot(                        \"5434a32a-8832-41c5-abd5-fa51540f54fd\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.0,0.0,-0.5,-0.5,-0.5,-0.5,0.0,0.0],[0.1,0.0,-0.5,0.9,1.0,0.0,0.0,0.0],[0.0,-1.0,-0.5,0.9,-1.0,-1.0,0.0,-0.5],[0.0,0.0,-1.0,0.9,-1.0,0.0,1.0,0.0],[0.0,0.1,-1.0,0.9,1.0,1.0,-0.5,-0.5],[0.0,0.1,-1.0,0.9,-1.0,1.0,-1.0,-1.0],[0.0,0.0,1.0,1.0,-0.5,1.0,1.0,1.0],[0.0,0.0,-0.5,0.0,-0.5,-0.5,-0.5,-0.5]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"<b>%{y}%{x}</b><br>%{customdata}<extra></extra>\",\"text\":[[\"A0\",\"A1\",\"A2\",\"A3\",\"A4\",\"A5\",\"A6\",\"A7\"],[\"B0\",\"B1\",\"B2\",\"\",\"\",\"B5\",\"B6\",\"B7\"],[\"C0\",\"\",\"C2\",\"\",\"\",\"\",\"C6\",\"C7\"],[\"D0\",\"D1\",\"\",\"\",\"\",\"D5\",\"\",\"D7\"],[\"E0\",\"E1\",\"\",\"\",\"\",\"\",\"E6\",\"E7\"],[\"F0\",\"F1\",\"\",\"\",\"\",\"\",\"\",\"\"],[\"G0\",\"G1\",\"\",\"\",\"G4\",\"\",\"\",\"\"],[\"H0\",\"H1\",\"H2\",\"H3\",\"H4\",\"H5\",\"H6\",\"H7\"]],\"texttemplate\":\"%{text}\",\"customdata\":[[\"I:1<br>S:0\",\"I:2<br>S:1\",\"I:3<br>S:2\",\"I:4<br>S:3\",\"I:5<br>S:4\",\"I:6<br>S:5\",\"I:7<br>S:6\",\"I:8<br>S:7\"],[\"I:9<br>S:8\",\"I:10<br>S:9\",\"I:11<br>S:10\",\"I:12<br>S:11\",\"I:13<br>S:12\",\"I:14<br>S:13\",\"I:15<br>S:14\",\"I:16<br>S:15\"],[\"I:17<br>S:16\",\"I:18<br>S:17\",\"I:19<br>S:18\",\"I:20<br>S:19\",\"I:21<br>S:20\",\"I:22<br>S:21\",\"I:23<br>S:22\",\"I:24<br>S:23\"],[\"I:25<br>S:24\",\"I:26<br>S:25\",\"I:27<br>S:26\",\"I:X<br>S:27\",\"I:X<br>S:28\",\"I:28<br>S:29\",\"I:29<br>S:30\",\"I:30<br>S:31\"],[\"I:31<br>S:32\",\"I:32<br>S:33\",\"I:33<br>S:34\",\"I:X<br>S:35\",\"I:X<br>S:36\",\"I:34<br>S:37\",\"I:35<br>S:38\",\"I:36<br>S:39\"],[\"I:37<br>S:40\",\"I:38<br>S:41\",\"I:39<br>S:42\",\"I:40<br>S:43\",\"I:41<br>S:44\",\"I:42<br>S:45\",\"I:43<br>S:46\",\"I:44<br>S:47\"],[\"I:45<br>S:48\",\"I:46<br>S:49\",\"I:47<br>S:50\",\"I:48<br>S:51\",\"I:49<br>S:52\",\"I:50<br>S:53\",\"I:51<br>S:54\",\"I:52<br>S:55\"],[\"I:53<br>S:56\",\"I:54<br>S:57\",\"I:55<br>S:58\",\"I:56<br>S:59\",\"I:57<br>S:60\",\"I:58<br>S:61\",\"I:59<br>S:62\",\"I:60<br>S:63\"]]}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(0, 128, 128)\"],[0.16666666666666666,\"rgb(112, 164, 148)\"],[0.3333333333333333,\"rgb(180, 200, 168)\"],[0.5,\"rgb(246, 237, 189)\"],[0.6666666666666666,\"rgb(237, 187, 138)\"],[0.8333333333333334,\"rgb(222, 138, 90)\"],[1.0,\"rgb(202, 86, 44)\"]],\"cmid\":0.0},\"title\":{\"text\":\"White To Play. Board State After Black Plays B3 \",\"x\":0.5},\"shapes\":[{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.75,\"x1\":3.25,\"y0\":0.75,\"y1\":1.25,\"line\":{\"color\":\"red\",\"width\":4}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":0.8,\"x1\":1.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.78,\"x1\":3.22,\"y0\":1.78,\"y1\":2.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.78,\"x1\":3.22,\"y0\":2.78,\"y1\":3.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.78,\"x1\":3.22,\"y0\":3.78,\"y1\":4.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.78,\"x1\":3.22,\"y0\":4.78,\"y1\":5.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":6.8,\"x1\":7.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":6.8,\"x1\":7.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}}]},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('5434a32a-8832-41c5-abd5-fa51540f54fd');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "residual_stream torch.Size([512])\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"9f0c9865-e618-4d72-a24d-d8fbcb878068\" class=\"plotly-graph-div\" style=\"height:400px; width:600px;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"9f0c9865-e618-4d72-a24d-d8fbcb878068\")) {                    Plotly.newPlot(                        \"9f0c9865-e618-4d72-a24d-d8fbcb878068\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.99999,0.9995745,0.99991393,1.0,1.0,0.99999964,0.9998764,0.9999994],[0.99997807,1.0,0.9999925,0.000037556565,0.012193234,1.0,1.0,0.9999994],[0.9999976,0.00085659657,1.0,0.00009182693,0.0000015295569,0.0069386656,1.0,0.99999964],[0.9999937,1.0,0.000030428684,0.00011053093,0.0000714015,1.0,0.0004169602,1.0],[0.99999917,1.0,0.0012213974,0.000033883833,0.00003632845,0.00012820729,0.99999976,0.9999995],[0.9999218,1.0,2.9932474e-7,0.0024246464,2.2992618e-8,0.08514128,0.000007291018,4.890161e-9],[0.9996542,0.99999845,0.00087070477,0.17674452,1.0,0.05896601,0.0001195336,0.000063030595],[0.9981456,0.9998679,0.99989104,1.0,0.99995947,0.9953811,0.9999751,0.69935465]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"1\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.000009257266,0.00014202684,0.00002698466,2.5410704e-10,2.656993e-9,4.2984038e-8,0.000011573966,2.6682722e-8],[0.0000074697123,1.0177328e-9,6.0744696e-7,0.9751016,0.93260497,1.3514746e-9,2.8659601e-9,4.2902712e-7],[4.2057803e-8,0.00015953045,2.7613986e-8,0.9986284,0.00035976645,0.00014326013,6.53762e-11,7.980103e-8],[5.7861666e-7,1.8636419e-10,0.0011002372,0.998063,0.00090664957,3.553408e-13,0.9818621,1.7104822e-8],[1.2637177e-7,1.3527524e-11,0.0046970383,0.9997898,0.9996382,0.978566,1.2462934e-7,1.1298699e-7],[0.000013439763,2.1497599e-11,0.0034409417,0.9975629,0.0039042262,0.91481835,0.008532766,0.000008460218],[0.0001371535,4.8246386e-7,0.99755424,0.8204308,5.3421226e-11,0.93101645,0.9965829,0.9983125],[0.0004402985,0.000060848393,0.00005246076,1.3303283e-10,0.0000138649,0.002283371,0.000018219096,0.22713894]],\"type\":\"heatmap\",\"xaxis\":\"x2\",\"yaxis\":\"y2\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"2\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[6.7635534e-7,0.00028353446,0.000059172748,6.9219026e-11,1.33516975e-8,3.3544347e-7,0.00011199493,6.036133e-7],[0.000014429326,9.936513e-9,0.0000069223233,0.024860786,0.055201843,2.545764e-8,5.52885e-8,7.310895e-8],[0.000002364042,0.9989838,3.821608e-8,0.0012797689,0.9996387,0.9929181,4.1453857e-10,2.3714564e-7],[0.000005672923,2.0714677e-9,0.99886924,0.0018264583,0.9990219,1.1038242e-13,0.017720893,3.2719305e-9],[7.138408e-7,3.7260628e-10,0.99408156,0.00017637787,0.00032547367,0.02130581,1.08995025e-7,3.227842e-7],[0.00006469447,6.1559186e-10,0.9965587,0.00001252913,0.99609584,0.00004032041,0.99145985,0.99999154],[0.00020863967,0.0000010190283,0.0015750999,0.0028247098,3.608773e-10,0.01001751,0.0032975546,0.0016245244],[0.0014141392,0.000071189745,0.000056458775,1.4958446e-11,0.00002673205,0.0023355053,0.0000067243013,0.073506385]],\"type\":\"heatmap\",\"xaxis\":\"x3\",\"yaxis\":\"y3\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,0.31999999999999995],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"xaxis2\":{\"anchor\":\"y2\",\"domain\":[0.33999999999999997,0.6599999999999999],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis2\":{\"anchor\":\"x2\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis3\":{\"anchor\":\"y3\",\"domain\":[0.6799999999999999,0.9999999999999999],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis3\":{\"anchor\":\"x3\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"annotations\":[{\"font\":{},\"showarrow\":false,\"text\":\"P(Empty)\",\"x\":0.15999999999999998,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"P(Their's)\",\"x\":0.49999999999999994,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"P(Mine)\",\"x\":0.8399999999999999,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"}],\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"margin\":{\"t\":60},\"height\":400,\"width\":600},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('9f0c9865-e618-4d72-a24d-d8fbcb878068');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "game_index = 1\n",
    "move = 20\n",
    "layer = 4\n",
    "plot_single_board(focus_games_string[game_index, :move+1])\n",
    "plot_probe_outputs(layer, game_index, move)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now plot the contributions of the attention and MLP layers to the `my_probe` direction. Strikingly, we see that the MLP layers are important for the vertical stripe that we just taken by the opponent, but that most of the rest seems to be done by the attention layers. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"565fd717-a987-45dc-a9be-5cda084c3ff4\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"565fd717-a987-45dc-a9be-5cda084c3ff4\")) {                    Plotly.newPlot(                        \"565fd717-a987-45dc-a9be-5cda084c3ff4\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.6021272,-0.067596495,1.0759158,0.13457754,-0.4454856,0.37554985,-0.03800398,-0.32565215],[-0.27753305,-0.1421929,-0.13519815,-0.37603614,-0.772841,-0.10424492,-0.16088963,-0.350285],[0.006143719,0.86717296,0.23584071,0.36725134,0.0143485665,0.20665956,0.7011918,0.46924326],[0.09957516,0.17238441,0.4339365,0.50214374,-0.14393097,-0.3651063,-0.5851087,-0.49513072],[-0.30269367,0.5204613,-0.1336683,0.010062188,-0.54305685,-0.60857475,0.36780065,-0.42480636],[-0.52095914,-0.07920483,-0.012429848,-0.7255341,-0.50165343,-1.1921688,-0.56657934,1.8516929],[-0.42362505,-0.118989795,-0.64008665,-0.84440565,-0.5217102,-1.1331071,-0.7280673,-2.086533],[0.20978928,0.45717454,-0.46944726,0.20342934,0.2503193,0.2792234,0.8453105,-0.105624095]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"1\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.021404155,1.4477122,0.552894,-0.03989297,0.5422559,0.121418536,0.81873906,1.0029979],[0.7787374,0.82154024,0.6448374,0.18857983,-1.7965057,-0.087025516,0.45588827,-0.26638013],[0.14645153,1.9808327,0.029529803,0.40456855,-0.54652494,0.24372518,0.42585075,-0.49886337],[0.41519326,-0.12759869,1.3717442,0.25217193,1.3049163,-0.18521291,-0.78132594,0.044285983],[-0.10046217,0.08415267,0.600867,0.33966693,-1.3067617,-0.97662437,-0.013060793,-0.29759628],[0.08875045,0.029780477,0.24392942,-0.7666489,-0.61995816,-1.5980382,1.7973568,2.0424414],[-0.40782803,0.54463905,-1.8010395,-2.0122755,-0.14904618,-1.3215975,-1.4324136,-2.2836168],[0.796028,-1.3310734,-1.1850104,-1.8777654,-1.1674834,0.6023171,-0.4947866,-1.5489719]],\"type\":\"heatmap\",\"xaxis\":\"x2\",\"yaxis\":\"y2\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"2\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.18589193,0.120164394,0.5828052,-0.09566349,-0.13883293,0.6824691,0.5856705,0.703127],[-0.23218572,0.43207163,0.81632864,0.20971636,-0.5638828,0.48934078,0.3736533,-1.0619009],[-0.013462022,1.2455593,0.3495683,1.5858449,2.0329125,2.5108042,-0.37699476,-0.8588917],[0.2989676,0.026319265,1.7656394,1.4088588,3.2978148,0.25966027,-0.8487934,-0.8732822],[-0.6109285,0.07811129,1.164327,0.36528328,-2.0581815,1.3832808,-0.5139249,-1.235049],[-0.22929949,0.7437481,1.9716144,-0.36945307,1.7984684,-2.243629,1.0274723,1.2966411],[-1.5245377,-0.6372289,-2.3477263,-0.83034813,0.20779054,-0.9296118,-1.8858544,-2.6780639],[-0.076616764,-0.48512256,-0.6738874,-1.8060141,-0.23169148,-0.29678938,-0.2569732,-0.35668296]],\"type\":\"heatmap\",\"xaxis\":\"x3\",\"yaxis\":\"y3\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"3\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.13922045,0.01793158,0.71670526,0.56555426,-0.022759259,0.009765089,0.77824485,2.3096857],[-0.9021735,0.07612038,0.3681674,0.58976513,-0.3928419,0.43541217,0.033181727,-0.63949764],[-0.15194112,1.1629173,0.35136604,0.37157667,2.057052,2.2487226,-0.065022856,-0.7492865],[-0.10599473,0.7858472,2.1359944,1.2375407,0.97677296,-0.30799437,-1.5139427,-0.02171126],[-0.3654276,0.40082082,1.5925689,1.5638798,-2.41076,-3.940316,-0.73513865,-0.63666487],[-0.41125375,0.8602946,3.1080933,1.933337,3.1563008,-1.3057805,0.18317187,1.7946082],[-1.5023082,0.20168665,-2.0196369,-1.4531543,-0.38185906,-1.1793475,-1.4875841,-1.1602994],[-0.5670615,0.9930573,0.24377608,-0.40760982,-1.0780962,-0.23443526,1.1003666,0.6437241]],\"type\":\"heatmap\",\"xaxis\":\"x4\",\"yaxis\":\"y4\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"4\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-1.1736096,-0.3914914,-1.2677374,0.31039667,-0.2184583,0.23510972,-0.20712668,1.2074051],[-1.2243422,-0.22969326,0.80379266,0.2617969,-0.999615,0.22180152,0.40661386,0.8576705],[-0.7137304,2.0408196,-0.194534,1.5588746,1.9874843,1.8363168,-0.98035675,0.22593397],[0.48724753,-0.33805698,1.9507406,1.315531,2.84797,-0.40487736,-1.5859684,0.73723066],[-1.305846,0.34944028,1.7449028,1.2830824,-3.232474,-1.6218383,-0.4875965,0.80814695],[0.86674273,0.27851894,1.9608676,0.2018579,1.66917,-3.8576183,1.5625105,2.8906748],[0.3756186,-0.12761545,-1.8139142,-1.6988448,0.22097293,-1.5145824,-2.7067251,0.75180477],[0.84272563,-0.048575282,-1.1023277,-1.0379796,-0.4974329,-0.7898261,-0.6450572,-1.6303034]],\"type\":\"heatmap\",\"xaxis\":\"x5\",\"yaxis\":\"y5\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,0.18400000000000002],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"xaxis2\":{\"anchor\":\"y2\",\"domain\":[0.20400000000000001,0.388],\"matches\":\"x\"},\"yaxis2\":{\"anchor\":\"x2\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis3\":{\"anchor\":\"y3\",\"domain\":[0.40800000000000003,0.5920000000000001],\"matches\":\"x\"},\"yaxis3\":{\"anchor\":\"x3\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis4\":{\"anchor\":\"y4\",\"domain\":[0.6120000000000001,0.7960000000000002],\"matches\":\"x\"},\"yaxis4\":{\"anchor\":\"x4\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis5\":{\"anchor\":\"y5\",\"domain\":[0.8160000000000001,1.0],\"matches\":\"x\"},\"yaxis5\":{\"anchor\":\"x5\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"annotations\":[{\"font\":{},\"showarrow\":false,\"text\":\"facet_col=0\",\"x\":0.09200000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"facet_col=1\",\"x\":0.29600000000000004,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"facet_col=2\",\"x\":0.5,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"facet_col=3\",\"x\":0.7040000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"facet_col=4\",\"x\":0.908,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"}],\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"title\":{\"text\":\"Attention Layer Contributions to my vs their (Game 1 Move 20)\"}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('565fd717-a987-45dc-a9be-5cda084c3ff4');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"28c777ae-521c-4991-955b-fa0fc8af1799\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"28c777ae-521c-4991-955b-fa0fc8af1799\")) {                    Plotly.newPlot(                        \"28c777ae-521c-4991-955b-fa0fc8af1799\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.053872347,-1.199197,-0.74673676,0.48142868,0.81452346,0.30020654,0.96036416,-1.0436513],[-0.120826125,-0.20532145,-0.42165822,-0.65462327,-0.41676468,0.5534128,0.02009207,-0.98426104],[1.054606,-0.74033433,-0.20263717,-1.5328152,1.01003,0.17162043,0.14758456,0.18768692],[0.6199038,0.49160546,0.7002857,-1.424134,0.10356802,-0.13876683,-0.6409716,-0.21273768],[1.9514266,-0.31927425,1.2002164,-2.1086183,0.7270626,-0.011755288,-0.4957255,0.39158642],[0.90129894,1.6190221,0.18197729,-0.74333197,0.5737083,0.44579825,0.24092573,-0.09356728],[1.3596394,1.2569759,0.45540518,-0.11337507,0.22409506,-0.28272554,0.37968507,-0.18904328],[-0.3985091,1.568708,1.8368063,0.5408649,-0.6194279,0.42820412,-0.1740272,-0.885029]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"1\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-1.5627145,1.3097008,-0.4773823,-0.2753501,-0.6386161,-0.11502403,0.7804571,0.24696094],[0.9635826,0.14322954,1.0039116,0.09275125,1.5013628,0.7266169,0.7504101,0.6495944],[0.328349,0.50750625,-0.8082588,-1.4807682,0.6421255,0.96109205,0.15626228,0.2781264],[-0.2854166,0.2506041,-0.17973863,-2.0217843,0.45576712,0.25399277,0.5523243,-0.6029668],[-0.122208595,0.3184257,-0.47983068,-3.1855648,-0.23654748,0.32707033,0.15723735,-0.82689285],[1.1994268,0.18157348,0.27600387,-2.609231,0.14216518,-0.43433094,-0.4066007,-0.75330335],[0.13893452,0.30087465,0.81816,1.0275353,0.28876364,0.86107343,-0.11770201,0.7973756],[-0.037252545,0.77150846,0.18081528,1.872401,0.9575242,0.49727574,0.8018622,0.13425064]],\"type\":\"heatmap\",\"xaxis\":\"x2\",\"yaxis\":\"y2\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"2\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.10963017,-0.91028655,-1.8599325,-1.251087,1.224262,0.5398106,-1.1496456,0.825324],[1.4275198,0.07572067,0.029055983,-0.75689566,0.36627066,0.9722427,0.8281394,-0.27785122],[0.76644146,1.0530217,-0.16901541,-2.5257459,-0.34003118,0.6428288,0.5842368,0.76857835],[1.451032,0.6073764,-0.98631376,-2.6306345,-0.9863136,0.7507521,0.8493173,-0.5039373],[1.9659886,0.53297377,0.11215773,-2.3695955,0.11559367,0.93167716,0.5877995,1.3264422],[1.2391747,-0.18849047,-0.22661695,-2.9382648,-0.06839767,0.49027777,-0.43456239,0.96017534],[0.95695555,0.74236614,1.2135789,-0.046067983,0.3612894,-0.28087544,-0.8579923,0.09146112],[2.1779628,-3.0462327,-0.05414331,1.693213,0.9825834,-1.0843935,-3.0082965,-1.4826387]],\"type\":\"heatmap\",\"xaxis\":\"x3\",\"yaxis\":\"y3\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"3\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.51628613,0.8143734,0.8059323,0.5272539,-0.1387487,-0.021147847,0.87723434,0.03544815],[0.8374063,-0.34501514,0.18129525,-0.65527344,0.43512148,-0.3521012,0.80259264,1.2653975],[1.2090974,-0.026521876,0.4779812,-1.1721936,0.8573642,-0.17699459,0.9209032,1.5364156],[-0.6291258,-0.07547447,-0.32965738,-2.3830428,-1.0878291,-0.04619944,0.9097869,1.1867683],[-1.5048301,0.95464945,-0.16215575,-1.2553012,-0.037397563,0.12659678,1.0968009,0.99618495],[-1.4989022,-0.5082413,-1.6033919,-1.3559653,-0.016322225,-0.4292724,-0.25680447,-0.5345674],[-0.17706048,-0.5531068,0.97609264,1.0013902,1.0098335,0.19934572,-0.10196364,1.2175496],[-1.058631,-0.1536215,-0.48468965,-0.33026502,0.17613149,-0.7497989,-0.71351737,0.15799856]],\"type\":\"heatmap\",\"xaxis\":\"x4\",\"yaxis\":\"y4\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"4\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.5857519,-0.28952754,1.2397947,-0.6858681,0.94222426,-0.38278317,-0.5483781,-2.6549056],[-0.37645656,1.0907781,-0.7030838,-0.8673349,0.23576498,0.012513816,-0.5536191,-1.0698334],[1.0316721,0.4490599,0.20780566,-2.9777668,0.7319013,0.156665,0.29900074,-1.0410968],[0.28364205,0.59633625,-0.044772595,-2.1777108,-0.5369415,-0.86885285,-0.5172673,-0.80035853],[1.4343789,0.20211542,-0.6346607,-2.882666,1.0191748,0.08644509,0.053179532,1.3603517],[-0.90472925,0.19048929,-0.4955158,-3.7206016,-0.6710205,-0.45685545,1.0003717,1.120566],[0.21023822,-1.1668346,-0.9413396,-0.050709188,0.88740546,0.5802164,1.952106,-1.435375],[-0.66142154,0.06596279,2.3517308,-0.8272052,0.24924266,0.7246126,0.7404418,3.259322]],\"type\":\"heatmap\",\"xaxis\":\"x5\",\"yaxis\":\"y5\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,0.18400000000000002],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"xaxis2\":{\"anchor\":\"y2\",\"domain\":[0.20400000000000001,0.388],\"matches\":\"x\"},\"yaxis2\":{\"anchor\":\"x2\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis3\":{\"anchor\":\"y3\",\"domain\":[0.40800000000000003,0.5920000000000001],\"matches\":\"x\"},\"yaxis3\":{\"anchor\":\"x3\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis4\":{\"anchor\":\"y4\",\"domain\":[0.6120000000000001,0.7960000000000002],\"matches\":\"x\"},\"yaxis4\":{\"anchor\":\"x4\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis5\":{\"anchor\":\"y5\",\"domain\":[0.8160000000000001,1.0],\"matches\":\"x\"},\"yaxis5\":{\"anchor\":\"x5\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"annotations\":[{\"font\":{},\"showarrow\":false,\"text\":\"facet_col=0\",\"x\":0.09200000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"facet_col=1\",\"x\":0.29600000000000004,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"facet_col=2\",\"x\":0.5,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"facet_col=3\",\"x\":0.7040000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"facet_col=4\",\"x\":0.908,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"}],\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"title\":{\"text\":\"MLP Layer Contributions to my vs their (Game 1 Move 20)\"}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('28c777ae-521c-4991-955b-fa0fc8af1799');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "imshow([(focus_cache[\"attn_out\", l][game_index, move][:, None, None] * my_probe).sum(0) for l in range(layer+1)], facet_col=0, y=[i for i in \"ABCDEFGH\"], facet_name=\"Layer\", title=f\"Attention Layer Contributions to my vs their (Game {game_index} Move {move})\", aspect=\"equal\")\n",
    "imshow([(focus_cache[\"mlp_out\", l][game_index, move][:, None, None] * my_probe).sum(0) for l in range(layer+1)], facet_col=0, y=[i for i in \"ABCDEFGH\"], facet_name=\"Layer\", title=f\"MLP Layer Contributions to my vs their (Game {game_index} Move {move})\", aspect=\"equal\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For reference, here's the overall probe scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"b8ee8005-cd33-4c89-a153-9d5f9bb3f492\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"b8ee8005-cd33-4c89-a153-9d5f9bb3f492\")) {                    Plotly.newPlot(                        \"b8ee8005-cd33-4c89-a153-9d5f9bb3f492\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-2.6164465,0.6913185,0.7851932,-1.3004801,1.6144471,2.0546231,2.2696948,3.1189177],[0.65840626,2.278638,2.4332428,-3.6692498,-2.8269856,2.9358187,2.9596615,-1.7695694],[4.0290833,8.742258,0.32493424,-6.659704,7.929694,8.843741,1.8470067,1.089139],[2.28282,2.4083104,6.8110976,-6.3034377,7.004776,-1.1691253,-4.0147057,-1.6539807],[1.731431,3.3157961,5.3548865,-8.642672,-8.029867,-3.8271084,-0.13404095,1.0497108],[1.5714573,3.3546433,5.668562,-11.285013,5.5417843,-10.029624,4.7552648,11.680126],[0.41950858,0.7476994,-6.450987,-5.6714244,1.9103308,-4.5319424,-5.7111506,-6.4208527],[1.1668224,0.1569624,0.07344532,-2.1853175,0.6565038,0.022576094,-0.9967427,-1.128191]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"title\":{\"text\":\"Overall Probe Score after Layer 4 for my vs their (Game 1 Move 20)\"}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('b8ee8005-cd33-4c89-a153-9d5f9bb3f492');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "imshow((focus_cache[\"resid_post\", layer][game_index, move][:, None, None] * my_probe).sum(0), y=[i for i in \"ABCDEFGH\"], title=f\"Overall Probe Score after Layer {layer} for my vs their (Game {game_index} Move {move})\", aspect=\"equal\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also look at blank, though this is less interesting and harder to interpret - the model has a good idea of whether a cell is blank or not after Attn0, since it's pretty easy to implement. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"a1576844-6c3b-42a2-87cd-61a683753ce7\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"a1576844-6c3b-42a2-87cd-61a683753ce7\")) {                    Plotly.newPlot(                        \"a1576844-6c3b-42a2-87cd-61a683753ce7\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.7615169,1.415942,2.50921,2.2752144,3.0277705,1.3433228,1.7095863,1.3356606],[1.0817883,2.4255664,3.5654993,0.61960804,-1.8427843,1.7655876,2.030184,1.6840582],[3.246375,-2.2321622,4.1066694,-5.064191,-2.9947014,-2.8553622,3.158107,2.0166821],[3.1681926,3.3393936,-2.0372927,-1.9312863,-1.7932177,3.0342321,-0.70903075,2.535937],[3.2389388,1.7724177,-0.5139547,-1.8686749,-1.7913115,-1.2736679,1.6972014,1.0345131],[0.8197987,1.5222918,-1.1231326,-2.0191097,-2.7535355,-2.4787793,-2.3646033,-3.974881],[1.2619504,2.20889,-0.18660992,-2.1252913,2.788618,-1.7439076,-2.280684,-1.9399397],[0.8133183,1.6674194,3.089951,1.9157339,0.13537198,-0.47457567,1.3632467,1.6363773]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"1\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[1.0295389,1.2176185,1.7041585,1.1121135,2.9686794,1.7786894,0.90022004,1.1739836],[0.28627783,1.1553319,1.1793807,1.0764588,1.5768863,1.1776404,1.3461045,1.3292336],[2.323728,-0.103094876,1.1536087,1.0112569,0.5867182,-1.1647061,1.7635194,1.7708725],[1.18196,1.8260219,-1.002683,-1.3431144,-1.3987372,2.069406,0.49148524,2.3720598],[2.0922928,2.2325182,-0.2839549,-1.3291204,-1.3925657,0.6695899,1.5576313,3.637662],[1.4775995,1.4595003,-1.0008308,-2.40193,-3.1455698,-0.58153117,-3.1737094,-0.605692],[0.8461877,1.1632175,0.35966468,-0.43918976,0.7218909,0.099529386,0.808444,-1.2072555],[0.31959873,1.8262731,1.7891312,1.4022092,-0.21902633,0.7473798,1.3619434,-0.010975122]],\"type\":\"heatmap\",\"xaxis\":\"x2\",\"yaxis\":\"y2\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"2\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[1.4742064,-0.41708928,0.78886414,0.96526045,2.3074698,2.8627183,0.61478496,1.6025052],[1.425082,1.9798251,1.1208633,1.1196363,1.6754535,1.2307599,3.495504,0.595258],[2.2215545,-0.15122391,1.7266037,-0.7922249,1.0070939,-1.1757575,1.9573983,1.928749],[1.5198503,1.7127832,0.52316546,-1.0655653,-1.1955229,1.1088315,0.47436494,0.47366917],[1.8706219,2.5674446,-0.54267496,-1.1792061,-1.0416007,-0.95687735,1.3807392,1.6700375],[-0.5312904,1.1682464,-0.5004141,-1.7321113,-2.1419697,2.7917216,-1.0473099,-0.4248633],[1.4926887,0.95794797,2.690269,1.4053384,3.1448622,-0.25215042,1.713294,-0.8290225],[1.4429653,0.9922434,1.2864482,3.1335173,2.2432165,2.349924,0.96775454,1.0981842]],\"type\":\"heatmap\",\"xaxis\":\"x3\",\"yaxis\":\"y3\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"3\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[3.3263307,0.23814034,2.1487074,1.3158393,0.7395973,2.4482086,2.0073445,0.642098],[1.4573927,3.1167884,2.507906,0.71355045,-0.7842249,2.2630484,2.6591568,2.8846927],[0.40806305,-0.42215222,2.3266544,-0.18278718,-1.3382144,-0.0064035654,3.0480776,2.4924078],[2.0013287,1.6564581,0.14360061,-1.7325395,-1.8068509,2.1419406,-0.26861167,0.73496556],[1.0582786,2.1691675,-0.29217255,-1.7192078,-1.6771135,0.095232844,2.43221,2.4254699],[1.1980562,3.6632152,-0.24657646,0.6248855,-1.086345,0.2491805,-3.5538507,-1.5833877],[1.626122,1.2912765,-0.30075717,-0.38501066,2.6855538,2.2541318,0.12571716,-0.8068579],[3.3401337,0.53858376,1.92548,2.1892443,2.0016239,1.5741891,1.3001785,1.3359747]],\"type\":\"heatmap\",\"xaxis\":\"x4\",\"yaxis\":\"y4\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"4\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.8799578,2.4836748,1.7531242,1.5790939,1.1526692,2.2262213,1.2541335,0.95045877],[1.2286541,1.847218,1.612428,1.1460347,0.012723744,3.631298,1.5849301,0.41872847],[3.015246,-0.8378358,0.8632052,0.8254014,0.31455472,-1.2024311,1.1140926,0.7806488],[1.5537412,3.0719292,2.6219368,-1.0016389,-1.123914,3.670708,-1.3121778,1.6790189],[2.961996,3.7430851,1.2928298,-0.8729876,-1.0955871,0.54983443,2.1979005,2.804674],[1.3235624,3.1355524,-1.7818357,-3.1213837,-2.0377517,-0.09288722,0.25668982,-2.448482],[0.8488666,1.4926177,-1.8693569,-0.88396454,2.1918895,1.3934524,-1.8080187,-0.7129495],[1.1479893,0.8798677,1.6557564,2.3959837,3.2141426,2.1685076,2.5667555,1.8785492]],\"type\":\"heatmap\",\"xaxis\":\"x5\",\"yaxis\":\"y5\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,0.18400000000000002],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"xaxis2\":{\"anchor\":\"y2\",\"domain\":[0.20400000000000001,0.388],\"matches\":\"x\"},\"yaxis2\":{\"anchor\":\"x2\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis3\":{\"anchor\":\"y3\",\"domain\":[0.40800000000000003,0.5920000000000001],\"matches\":\"x\"},\"yaxis3\":{\"anchor\":\"x3\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis4\":{\"anchor\":\"y4\",\"domain\":[0.6120000000000001,0.7960000000000002],\"matches\":\"x\"},\"yaxis4\":{\"anchor\":\"x4\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis5\":{\"anchor\":\"y5\",\"domain\":[0.8160000000000001,1.0],\"matches\":\"x\"},\"yaxis5\":{\"anchor\":\"x5\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"annotations\":[{\"font\":{},\"showarrow\":false,\"text\":\"facet_col=0\",\"x\":0.09200000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"facet_col=1\",\"x\":0.29600000000000004,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"facet_col=2\",\"x\":0.5,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"facet_col=3\",\"x\":0.7040000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"facet_col=4\",\"x\":0.908,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"}],\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"title\":{\"text\":\"Attention Layer Contributions to blank vs full (Game 1 Move 20)\"}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('a1576844-6c3b-42a2-87cd-61a683753ce7');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"54df8902-9348-4683-9356-853bae44ea6c\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"54df8902-9348-4683-9356-853bae44ea6c\")) {                    Plotly.newPlot(                        \"54df8902-9348-4683-9356-853bae44ea6c\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.034549713,1.4734818,-0.24143958,0.81149864,-0.11909473,1.0765052,0.51629984,2.0412436],[2.0411234,-0.28507054,1.2791896,-4.9501266,-0.10824418,0.8716264,2.5603447,2.0180151],[-0.22847211,-0.35416782,-0.056832552,1.4714516,0.8074653,0.5572357,1.1875155,1.3488994],[0.24467874,-2.1666975,0.7298554,-0.42819607,-0.43703893,2.1787539,-0.1327743,1.0281817],[0.19906914,0.8163053,-0.7583204,-0.42695436,-0.3983978,-0.45432124,-1.4182785,0.11271548],[1.3660963,1.344484,0.80040073,1.7834169,0.5687864,0.37428308,1.4546428,-0.48957837],[0.96478426,1.1909338,1.0672933,0.38816303,1.5826435,0.017448187,-1.7004969,-1.1486199],[1.2512639,0.5687425,1.3245535,0.5898397,2.0208673,0.25165677,-0.06021443,-1.0365236]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"1\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.98487616,1.0832635,1.305379,0.106169105,0.000838995,0.5836915,-0.016432554,-0.32320833],[0.4289623,3.5115898,0.046640635,-2.8277192,0.26844049,0.13883202,0.52664095,1.2907829],[0.2806758,0.7006325,1.4433134,0.8868072,-0.7034139,-0.26185736,1.2811952,0.41598308],[0.7113657,1.3114209,0.047378063,-0.76952344,-0.57384056,1.133925,-1.233945,-0.074664235],[1.5881727,2.286275,-0.9193423,-0.90119433,-0.9308384,-1.5100653,0.44445002,-0.61812544],[1.1097748,2.6460764,-0.93347293,-1.6382794,-0.3015442,0.30423903,-0.87089324,-0.47475004],[1.2030334,1.6798344,-0.21639371,3.0330021,0.07652807,-0.3565866,-1.6390598,0.6407837],[1.2067163,-0.024062157,1.2763417,3.74027,0.48260963,1.3498858,0.098334044,1.359226]],\"type\":\"heatmap\",\"xaxis\":\"x2\",\"yaxis\":\"y2\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"2\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.118362665,1.4286205,-1.1071115,3.1569307,3.210369,2.0658236,0.78057563,3.3139892],[-0.82646364,2.1442032,0.62706673,-3.5254753,-1.4255309,0.6941074,-0.13593423,0.7070923],[1.407212,1.2577732,0.061104894,-3.4290843,-4.0387955,2.213497,3.192335,1.1038959],[1.209604,1.3007869,-2.852509,0.66947556,0.49135524,0.32448846,-2.120986,3.0020323],[2.054852,0.025672436,-0.52107406,0.28214583,0.35348213,-0.3844564,1.1984046,3.0859761],[2.931804,5.0737343,0.6407578,2.243507,-0.4228387,1.2091105,2.7083209,-0.73733276],[1.4349775,-1.0891416,-0.4139433,1.107194,1.4248717,-0.30624032,-1.4221402,0.9716581],[2.4527116,1.9974546,-1.6612816,1.3618555,-0.52052164,-0.9504354,-1.0393792,1.3637543]],\"type\":\"heatmap\",\"xaxis\":\"x3\",\"yaxis\":\"y3\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"3\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.099175215,0.3522337,-0.2931422,2.1558235,3.0936236,1.0321769,0.18035626,2.3629928],[1.8365159,1.321733,0.13629532,0.25254488,-0.11685461,1.8487477,-1.4528542,1.3222308],[1.5863514,0.67782557,-0.05521214,-0.03212726,-1.1809678,2.0021951,1.0221854,0.5641217],[-0.43195403,5.032275,-0.7517394,1.4532964,1.547018,1.8642607,-2.2381036,1.8667793],[0.5861871,2.1401863,0.8278482,1.4935536,1.3775663,-1.770597,0.039648563,0.624623],[-0.10553038,2.1772814,-1.6729956,2.32379,1.5261531,-0.106570125,-1.2028263,-0.32475704],[-0.3687144,0.98128974,-4.155364,0.5580851,1.1819565,-2.1304572,-2.4231071,-1.452893],[-1.0818791,-0.6048825,-1.0425802,2.3735418,0.04032141,2.0328128,1.908776,-0.55224377]],\"type\":\"heatmap\",\"xaxis\":\"x4\",\"yaxis\":\"y4\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"4\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[2.9375548,-0.91748905,-0.15197706,6.355278,1.7792624,0.101231575,0.9653697,1.2857554],[0.11800218,1.0915849,1.6590481,-0.07817316,-2.531629,3.151455,4.826822,1.7839075],[-1.9367347,-2.3648865,5.371104,-0.4404565,-1.8847737,1.5326144,3.9263234,1.3820679],[0.8802779,1.9183333,-5.0667577,1.5822952,1.6465075,10.262502,-1.0258534,3.9800436],[-1.8945017,4.8284497,-1.8950467,1.9412614,1.7251539,-1.3817823,4.959153,0.15964437],[-0.19636285,1.4481628,-5.2151318,3.7917576,-4.5952473,1.1530871,-2.678544,-3.658547],[-0.88990843,1.7686505,-0.46484494,-1.3524806,4.680318,-0.2694739,1.0335253,-1.1681174],[-4.5685854,1.3589549,-2.2940278,3.1817157,-0.4320562,-3.6149755,2.1170568,-6.4291806]],\"type\":\"heatmap\",\"xaxis\":\"x5\",\"yaxis\":\"y5\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,0.18400000000000002],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"xaxis2\":{\"anchor\":\"y2\",\"domain\":[0.20400000000000001,0.388],\"matches\":\"x\"},\"yaxis2\":{\"anchor\":\"x2\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis3\":{\"anchor\":\"y3\",\"domain\":[0.40800000000000003,0.5920000000000001],\"matches\":\"x\"},\"yaxis3\":{\"anchor\":\"x3\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis4\":{\"anchor\":\"y4\",\"domain\":[0.6120000000000001,0.7960000000000002],\"matches\":\"x\"},\"yaxis4\":{\"anchor\":\"x4\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis5\":{\"anchor\":\"y5\",\"domain\":[0.8160000000000001,1.0],\"matches\":\"x\"},\"yaxis5\":{\"anchor\":\"x5\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"annotations\":[{\"font\":{},\"showarrow\":false,\"text\":\"facet_col=0\",\"x\":0.09200000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"facet_col=1\",\"x\":0.29600000000000004,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"facet_col=2\",\"x\":0.5,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"facet_col=3\",\"x\":0.7040000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"facet_col=4\",\"x\":0.908,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"}],\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"title\":{\"text\":\"MLP Layer Contributions to blank vs full (Game 1 Move 20)\"}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('54df8902-9348-4683-9356-853bae44ea6c');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "imshow([(focus_cache[\"attn_out\", l][game_index, move][:, None, None] * blank_probe).sum(0) for l in range(layer+1)], facet_col=0, y=[i for i in \"ABCDEFGH\"], facet_name=\"Layer\", title=f\"Attention Layer Contributions to blank vs full (Game {game_index} Move {move})\", aspect=\"equal\")\n",
    "imshow([(focus_cache[\"mlp_out\", l][game_index, move][:, None, None] * blank_probe).sum(0) for l in range(layer+1)], facet_col=0, y=[i for i in \"ABCDEFGH\"], facet_name=\"Layer\", title=f\"MLP Layer Contributions to blank vs full (Game {game_index} Move {move})\", aspect=\"equal\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"511002a0-2ae3-49a8-af40-0cab530db673\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"511002a0-2ae3-49a8-af40-0cab530db673\")) {                    Plotly.newPlot(                        \"511002a0-2ae3-49a8-af40-0cab530db673\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[12.898315,8.513411,10.127559,22.743507,18.938845,15.935124,10.231781,15.879787],[11.47543,19.566366,13.097372,-8.329824,-2.9236097,18.954157,18.190535,15.546527],[14.969675,-2.6903954,17.242476,-5.964381,-9.4253235,-0.541667,22.52736,15.799163],[13.221208,21.199165,-6.993444,-5.956558,-6.043824,29.250263,-5.756861,18.710896],[15.018319,23.368397,-4.02438,-5.9710255,-6.207614,-7.026642,15.964943,15.471137],[10.431484,22.885757,-12.184008,-0.3771224,-14.81329,2.6403975,-9.442656,-13.295967],[8.68431,14.170511,-3.8182652,1.3005888,22.697649,-0.49334574,-6.1729155,-6.459777],[7.1427913,9.62851,9.818616,23.833084,10.857857,6.066186,11.411385,1.6886907]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"title\":{\"text\":\"Overall Probe Score for blank vs full (Game 1 Move 20)\"}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('511002a0-2ae3-49a8-af40-0cab530db673');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "imshow((focus_cache[\"resid_post\", layer][game_index, move][:, None, None] * blank_probe).sum(0), y=[i for i in \"ABCDEFGH\"], title=f\"Overall Probe Score for blank vs full (Game {game_index} Move {move})\", aspect=\"equal\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading Off Neuron Weights"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another cool consequence of having a linear probe is having an interpretable set of directions in the residual stream. This means that we can read off the meaning of any neuron's input and output weights, in terms of the set of directions given by the probe. \n",
    "\n",
    "Let's start with neuron L5N1393, which seemed interesting from my initial investigations, and seemed to represent C0==BLANK & D1==THEIR'S and E2==MINE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Scale the probes down to be unit norm per cell\n",
    "blank_probe_normalised = blank_probe / blank_probe.norm(dim=0, keepdim=True)\n",
    "my_probe_normalised = my_probe / my_probe.norm(dim=0, keepdim=True)\n",
    "# Set the center blank probes to 0, since they're never blank so the probe is meaningless\n",
    "blank_probe_normalised[:, [3, 3, 4, 4], [3, 4, 3, 4]] = 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"670ff357-b721-4388-b2d0-8cf969ddfd56\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"670ff357-b721-4388-b2d0-8cf969ddfd56\")) {                    Plotly.newPlot(                        \"670ff357-b721-4388-b2d0-8cf969ddfd56\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.047589324,-0.010122413,0.069951706,-0.0020331703,0.01897344,-0.035740785,-0.012838831,-0.0007210914],[0.059174985,0.02575031,-0.007102605,0.021195743,-0.013032557,-0.025818132,0.03386383,0.00650098],[0.45953023,0.028274104,0.0841191,0.031225897,0.024243802,0.023261609,-0.008773248,-0.0151635725],[-0.0025134413,-0.06932345,0.03182496,0.0,0.0,0.017032888,-0.015509925,-0.0054359473],[0.053827446,0.05831263,-0.026726685,0.0,0.0,0.01141561,-0.011688196,-0.0055640284],[-0.012692273,0.069072634,0.0763057,0.13167384,0.019288644,0.026115015,0.009073098,-0.012224657],[0.0125652235,0.015017791,0.02852868,0.010163594,0.053730957,0.012848885,-0.00892733,-0.0032447781],[-0.047003336,-0.007804213,-0.009180866,-0.014486611,-0.024374643,0.0017100018,-0.00010709651,-0.0208787]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"1\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.07506359,-0.043503344,-0.10690984,-0.022490762,-0.049711533,-0.01880071,-0.013014726,-0.009587701],[0.026386812,0.05828446,0.028817074,-0.00075230654,0.0061111217,0.03529165,0.012182351,-0.025260948],[0.011165552,0.021835722,-0.045311347,-0.0031068306,0.016098646,-0.0010121469,0.016242981,-0.0013275873],[0.044188872,-0.25193158,-0.000109495595,0.027892146,-0.02113194,-0.0009752847,0.007926654,-0.03430092],[-0.08285278,-0.011646163,0.40166324,0.03057826,0.020778848,0.0005571991,0.011384781,-0.01346203],[-0.012721311,0.06536789,0.024725564,0.009786672,0.022910215,-0.009116612,0.008255877,-0.005047296],[0.003033176,-0.027575351,0.024637401,-0.0033766488,-0.019099955,0.0025962554,0.009174993,-0.0077721756],[-0.02682688,-0.03985651,-0.01870042,-0.01884472,-0.025635183,-0.042697057,-0.00872973,-0.011280015]],\"type\":\"heatmap\",\"xaxis\":\"x2\",\"yaxis\":\"y2\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,0.49],\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"title\":{\"text\":\"y\"}},\"xaxis2\":{\"anchor\":\"y2\",\"domain\":[0.51,1.0],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis2\":{\"anchor\":\"x2\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"annotations\":[{\"font\":{},\"showarrow\":false,\"text\":\"Blank In\",\"x\":0.245,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"My In\",\"x\":0.755,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"}],\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"title\":{\"text\":\"Input weights in terms of the probe for neuron L5N1393\"}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('670ff357-b721-4388-b2d0-8cf969ddfd56');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "layer = 5\n",
    "neuron = 1393\n",
    "w_in = model.blocks[layer].mlp.W_in[:, neuron].detach()\n",
    "w_in /= w_in.norm()\n",
    "w_out = model.blocks[layer].mlp.W_out[neuron, :].detach()\n",
    "w_out /= w_out.norm()\n",
    "imshow([\n",
    "    (w_in[:, None, None] * blank_probe_normalised).sum(dim=0),\n",
    "    (w_in[:, None, None] * my_probe_normalised).sum(dim=0),\n",
    "    # (w_out[:, None, None] * blank_probe_normalised).sum(dim=0),\n",
    "    # (w_out[:, None, None] * my_probe_normalised).sum(dim=0),\n",
    "    ],\n",
    "    facet_col=0,\n",
    "    y=[i for i in \"ABCDEFGH\"],\n",
    "    title=f\"Input weights in terms of the probe for neuron L{layer}N{neuron}\",\n",
    "    facet_labels=[\"Blank In\", \"My In\"])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also look at what fraction of the neuron's input and output weights are captured by the probe (because the vector was scaled to have unit norm, looking at the squared norm of its projection gives us this answer)\n",
    "\n",
    "We see that the input weights are well explained by this, while the output weights are only somewhat well explained by this. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fraction of input weights in probe basis: 0.6817981829887607\n",
      "Fraction of output weights in probe basis: 0.16325442166777648\n"
     ]
    }
   ],
   "source": [
    "U, S, Vh = torch.svd(torch.cat([my_probe.reshape(cfg.d_model, 64), blank_probe.reshape(cfg.d_model, 64)], dim=1))\n",
    "# Remove the final four dimensions of U, as the 4 center cells are never blank and so the blank probe is meaningless there\n",
    "probe_space_basis = U[:, :-4]\n",
    "\n",
    "print(\"Fraction of input weights in probe basis:\", (w_in @ probe_space_basis).norm().item()**2)\n",
    "print(\"Fraction of output weights in probe basis:\", (w_out @ probe_space_basis).norm().item()**2)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets try this on the layer 4 neurons with the top standard deviation, and look at how their output weights affect the my probe direction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"6cbca945-b297-47bb-8b27-7dbb254647ae\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"6cbca945-b297-47bb-8b27-7dbb254647ae\")) {                    Plotly.newPlot(                        \"6cbca945-b297-47bb-8b27-7dbb254647ae\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.009234954,0.01806493,0.009526523,0.018811828,0.03792089,0.0399784,0.05312369,0.021630034],[-0.002840247,0.04561815,0.007952059,0.051163327,0.060462467,0.06235422,0.022667535,0.035801344],[-0.028055154,0.06742066,0.042712066,-0.0076201623,0.0601689,0.08013856,0.09826186,-0.0017617792],[0.021186085,0.05393819,0.011061364,0.0,0.0,0.020502625,0.012596056,0.058870405],[0.037801825,0.05492661,0.070804454,0.0,0.0,0.08321741,0.077279225,0.0088561345],[0.070321426,0.0131599605,0.01841447,0.031634543,0.007120478,-0.00006439909,0.073528245,0.005029042],[-0.00077245384,0.03851662,0.046669625,0.041191883,0.06831671,0.036362063,0.027601168,0.051264822],[0.019547785,-0.03393349,0.032220546,0.031156998,0.012810335,0.033977225,0.061724164,0.039939582]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"1\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.017613996,-0.005249276,-0.0023060795,0.016199008,0.0054189153,0.011251587,-0.022851994,0.0007963171],[0.008885662,-0.0069704317,0.006582897,-0.02580607,-0.043766458,-0.014404509,-0.042973768,0.011543503],[0.02236988,-0.014656294,-0.037262373,-0.009015187,-0.05861404,-0.043118224,-0.010818642,-0.007339541],[0.008486759,0.00028117,-0.03499841,0.0,0.0,-0.0845135,-0.049339253,-0.014025889],[0.0022093104,0.014026252,-0.015132066,0.0,0.0,0.0031244755,0.013179798,0.017344408],[-0.011276489,0.014390368,-0.0052944273,-0.0011926498,-0.022549678,-0.012071815,-0.0045155445,-0.0010528136],[0.02070091,0.008995541,0.020029925,-0.01134533,0.011011006,-0.0048779696,0.015103991,0.023769336],[-0.015957516,-0.019362835,0.001209233,0.0129689695,-0.009355634,0.013266935,0.03190453,0.007206219]],\"type\":\"heatmap\",\"xaxis\":\"x2\",\"yaxis\":\"y2\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"2\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.022141911,0.014903516,0.0005204417,0.004138941,-0.0061436836,0.0038617551,-0.0015832903,-0.015410328],[0.0012490889,-0.01759985,0.039475642,0.002695905,0.0037442967,-0.0048150048,0.004029114,-0.011124685],[-0.012203608,-0.014040956,0.0025765244,0.008142918,0.0028576758,0.01483698,-0.016384885,0.006264534],[-0.000077061355,0.027257908,0.020252926,0.0,0.0,0.015918262,-0.016535789,0.021836132],[-0.004440807,0.00921141,0.005100102,0.0,0.0,-0.043960296,-0.0306584,0.0044486765],[0.009277603,0.010014462,-0.032324195,-0.027283886,-0.08796439,-0.04827445,0.0060328655,0.03039638],[-0.0064995736,0.017816102,-0.04030098,0.03009392,0.006306899,-0.00054128096,-0.016535709,0.01460285],[-0.019070894,-0.031184368,0.009362645,0.0005228119,-0.009837526,0.027197976,-0.016827509,-0.035185527]],\"type\":\"heatmap\",\"xaxis\":\"x3\",\"yaxis\":\"y3\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"3\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.005511176,-0.025425613,0.019893903,0.016006667,-0.011208709,-0.035155714,0.009652492,-0.00026803836],[0.005082175,0.043174267,-0.023302173,-0.002664661,0.01276131,-0.008987201,0.0023668911,-0.039678097],[0.021881789,0.0069754617,0.0070939492,-0.06975203,0.01139355,-0.033360362,-0.08531357,-0.039138407],[0.035575103,0.0025381558,0.028313326,0.0,0.0,-0.15908277,-0.04467953,-0.026939988],[-0.0052120667,0.01698884,0.0038927905,0.0,0.0,-0.030082319,-0.03171879,-0.02987389],[-0.0014848644,0.014691193,0.001476516,0.007751597,0.010197001,-0.04280364,-0.015599545,-0.0061322497],[-0.016181365,-0.0004379023,0.019033443,0.0036140345,0.040680017,-0.030304488,-0.01634796,-0.0065294653],[0.0011125589,-0.0040939003,-0.0003263317,0.013086313,0.004341213,0.011528565,0.00017691776,-0.0032359427]],\"type\":\"heatmap\",\"xaxis\":\"x4\",\"yaxis\":\"y4\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"4\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.04320193,-0.009473688,-0.018664991,-0.008001984,-0.017433528,-0.01730833,-0.03801103,-0.034974586],[-0.00400413,0.036057778,-0.006484708,-0.0038779126,0.042168316,0.017412942,0.0015449133,-0.018691752],[0.029135358,-0.023359118,0.001285743,-0.0028658025,0.021934446,0.014364476,0.015627652,-0.0058993157],[0.017466854,-0.013719534,0.07615234,0.0,0.0,0.034944415,-0.061886992,-0.044258803],[0.0007932782,-0.021288358,-0.008658405,0.0,0.0,0.008036185,0.029331844,-0.003260632],[0.0139192445,-0.015019394,0.015379153,0.0033412734,-0.016439728,0.006263973,0.03375993,0.004004894],[0.002045989,-0.020252297,0.018844424,-0.015947785,0.0030126814,0.021971373,-0.019376531,0.05559225],[-0.008273877,-0.010652979,0.0014316142,-0.0111579485,-0.0022773594,-0.043005295,0.033441883,0.00398273]],\"type\":\"heatmap\",\"xaxis\":\"x5\",\"yaxis\":\"y5\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"5\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.012300152,0.007879369,0.0078321155,-0.01272219,0.030038921,-0.009877553,0.0068972614,-0.0067103915],[0.0019881306,-0.031152086,-0.0040706354,0.018662252,0.016929034,-0.036495265,-0.013839837,0.016987687],[-0.020649787,0.033717137,-0.019363297,0.011159076,0.016068526,-0.013913499,0.013230514,-0.010448869],[-0.013674431,-0.010010803,0.0169598,0.0,0.0,0.02356473,0.0061438736,0.01072752],[-0.017523788,-0.04727942,-0.022478959,0.0,0.0,0.012753325,0.0040837266,0.0131936325],[-0.013614584,-0.01044945,0.05426145,0.03835072,-0.004822838,-0.06353396,0.008465946,0.0067145023],[0.021862647,0.013092134,0.036333412,0.033310197,0.0011928454,0.031231128,-0.00076387264,0.009212589],[0.01631425,0.048176423,-0.0026350003,-0.023247821,0.026847873,0.024673533,0.01945779,0.017918386]],\"type\":\"heatmap\",\"xaxis\":\"x6\",\"yaxis\":\"y6\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"6\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.020854563,0.021394515,0.0008534994,0.011142414,-0.033516444,0.0060068527,-0.014976788,0.008701587],[0.004285615,-0.024142236,0.029304495,0.00025353208,0.008482877,-0.020110104,-0.021149285,-0.013935734],[0.009772779,-0.019859988,-0.0005769823,0.0077294577,0.026072048,0.0125996135,0.03793835,-0.0016655065],[0.01703558,0.012479694,0.054106414,0.0,0.0,-0.022342065,-0.037205577,-0.0012799203],[-0.00040061213,0.0036385916,-0.027449299,0.0,0.0,-0.016490929,0.018050719,0.01180479],[-0.010976146,-0.014834564,-0.21679436,0.027323404,0.022835035,-0.052664064,0.01361211,-0.017725324],[0.006777933,-0.08557492,0.019038204,-0.0022587068,0.00569262,0.022466158,-0.021428274,0.0331508],[-0.08555965,-0.006200524,-0.010806724,0.0018102601,0.004851768,0.018114606,-0.01060697,0.0008449927]],\"type\":\"heatmap\",\"xaxis\":\"x7\",\"yaxis\":\"y7\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"7\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.002859156,-0.009925183,-0.0069112913,0.0063418597,0.0006520441,-0.0076116994,-0.00075956434,-0.0013685226],[-0.00022137165,-0.011600962,-0.0016686795,0.021790339,0.0040626638,0.0056411345,0.002470605,-0.00933052],[-0.03185431,0.0029810965,0.004536357,0.014702944,0.0005511623,0.029446067,-0.018690702,-0.0013563465],[-0.0086526545,-0.005502641,-0.013973253,0.0,0.0,0.012797598,-0.011153327,-0.008466512],[0.016166281,0.0038278913,-0.0070991106,0.0,0.0,0.0129616335,0.020266034,-0.018300224],[-0.0002006162,-0.022410473,0.010735992,-0.007397406,-0.0028592336,-0.02589346,0.009664888,-0.020291314],[-0.06261389,-0.04638648,-0.06505658,0.019792788,-0.01691261,-0.035724983,-0.05264729,-0.06395131],[0.025343243,0.1099887,0.19769114,0.05285447,0.08282721,0.18986842,0.13705143,0.033997416]],\"type\":\"heatmap\",\"xaxis\":\"x8\",\"yaxis\":\"y8\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"8\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.03317028,-0.018126868,-0.020913988,0.03068374,-0.019398773,0.009921016,-0.013159405,0.005047975],[-0.014268474,-0.011376697,0.025240283,0.02456033,0.04584854,0.022375759,-0.032308612,0.011647023],[-0.031699292,-0.029074095,0.01996508,0.009543228,0.0098504415,0.011572495,-0.018822728,-0.008013084],[-0.017373359,0.015139494,0.05049321,0.0,0.0,0.03185737,-0.008363949,0.010423806],[-0.028534021,0.021271914,0.032641396,0.0,0.0,0.06821413,0.031109784,0.05182846],[0.01852918,0.018050473,-0.045695655,-0.039810836,-0.023602039,0.04151343,-0.009790417,-0.00412431],[0.029350672,0.055933572,-0.0056930427,0.056805037,0.0009881947,0.049423564,0.04423669,0.006433852],[-0.023753934,-0.04205443,0.0011945181,-0.012282393,-0.020151686,0.012208186,-0.020669725,0.020352961]],\"type\":\"heatmap\",\"xaxis\":\"x9\",\"yaxis\":\"y9\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"9\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.034512773,0.0035050632,0.045385607,0.0020352118,-0.009187043,-0.000304956,0.000042213127,0.004487535],[-0.00100738,0.043695033,0.02541814,-0.01946786,-0.028513756,0.002013512,-0.010904118,-0.04554038],[0.007374767,-0.004985124,0.043600865,-0.016696746,0.03371471,-0.004908279,-0.013791421,0.011713687],[0.028052684,-0.01281696,-0.031054428,0.0,0.0,0.009475496,-0.02199262,-0.022972636],[0.006648257,0.014053067,0.007753983,0.0,0.0,-0.02480499,-0.006635865,-0.011134082],[0.015090494,-0.004502898,-0.032190796,-0.06455636,-0.033704516,0.0236747,-0.0035102144,0.026184835],[0.021447424,-0.01569298,-0.034865417,-0.029924799,-0.016444111,0.025727825,0.013857286,-0.025259107],[-0.0052569406,-0.01180095,0.003865106,-0.03343959,-0.0049433,0.012047993,0.02027314,-0.039550543]],\"type\":\"heatmap\",\"xaxis\":\"x10\",\"yaxis\":\"y10\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,0.08200000000000002],\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"title\":{\"text\":\"y\"}},\"xaxis2\":{\"anchor\":\"y2\",\"domain\":[0.10200000000000002,0.18400000000000005],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis2\":{\"anchor\":\"x2\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis3\":{\"anchor\":\"y3\",\"domain\":[0.20400000000000004,0.28600000000000003],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis3\":{\"anchor\":\"x3\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis4\":{\"anchor\":\"y4\",\"domain\":[0.30600000000000005,0.38800000000000007],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis4\":{\"anchor\":\"x4\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis5\":{\"anchor\":\"y5\",\"domain\":[0.4080000000000001,0.4900000000000001],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis5\":{\"anchor\":\"x5\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis6\":{\"anchor\":\"y6\",\"domain\":[0.5100000000000001,0.5920000000000001],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis6\":{\"anchor\":\"x6\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis7\":{\"anchor\":\"y7\",\"domain\":[0.6120000000000001,0.6940000000000002],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis7\":{\"anchor\":\"x7\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis8\":{\"anchor\":\"y8\",\"domain\":[0.7140000000000001,0.796],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis8\":{\"anchor\":\"x8\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis9\":{\"anchor\":\"y9\",\"domain\":[0.8160000000000002,0.8980000000000001],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis9\":{\"anchor\":\"x9\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis10\":{\"anchor\":\"y10\",\"domain\":[0.9180000000000001,1.0],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis10\":{\"anchor\":\"x10\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"annotations\":[{\"font\":{},\"showarrow\":false,\"text\":\"L3N1836\",\"x\":0.04100000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L3N1585\",\"x\":0.14300000000000004,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L3N1897\",\"x\":0.24500000000000005,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L3N524\",\"x\":0.3470000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L3N380\",\"x\":0.44900000000000007,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L3N618\",\"x\":0.5510000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L3N2045\",\"x\":0.6530000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L3N1932\",\"x\":0.7550000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L3N1556\",\"x\":0.8570000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L3N1418\",\"x\":0.9590000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"}],\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"title\":{\"text\":\"Cosine sim of Output weights and the blank color probe for top layer 3 neurons\"}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('6cbca945-b297-47bb-8b27-7dbb254647ae');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"75c66e6c-418d-48f8-b2c6-e73019eaf43a\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"75c66e6c-418d-48f8-b2c6-e73019eaf43a\")) {                    Plotly.newPlot(                        \"75c66e6c-418d-48f8-b2c6-e73019eaf43a\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.006931944,0.004510425,0.0052358797,-0.0055820243,-0.031578075,-0.007941905,-0.013246621,-0.0033472646],[0.027478235,0.009311475,-0.02118859,-0.06307887,-0.040729385,0.0015310636,0.031486098,-0.009792441],[0.015969772,-0.018193197,-0.028925553,-0.059149817,-0.055800922,0.009241379,0.0061039794,-0.046747603],[-0.008950125,-0.033683203,-0.018281147,-0.053278588,0.026462786,-0.00968291,0.017241465,0.020129364],[-0.016443353,-0.0067948503,-0.039378144,-0.027869904,-0.04437039,-0.010748725,-0.03906671,0.038802832],[-0.018287323,0.001893471,-0.010275395,-0.0026039518,-0.049118012,-0.020952782,0.032001957,-0.017626539],[-0.0032648286,0.014791765,-0.008476488,-0.05223202,0.0006272681,0.006820023,0.00026807934,0.011675958],[-0.00047457404,0.03828863,-0.007697789,-0.0485567,-0.015656073,0.005922676,0.03377539,0.007489916]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"1\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.0119408015,0.018465329,0.007569656,-0.00043702498,-0.029051919,-0.026168957,-0.017486338,-0.000024395064],[-0.0074616224,0.012573592,0.014504945,0.017244406,-0.0011536982,0.0009318255,-0.0032398477,0.0020807479],[-0.003519686,-0.01633051,-0.0041689314,0.012506325,-0.026872404,-0.049653962,-0.0102397725,-0.0066195005],[-0.0043297037,-0.000082941726,-0.029303743,-0.013728684,-0.00977575,-0.007748645,0.0016356567,0.010224186],[-0.008413751,0.010035555,-0.0063940026,-0.019767277,-0.03571332,-0.0075711273,0.000721097,0.0048146993],[-0.01086423,0.014177352,-0.027846387,-0.008905438,-0.043793023,0.0012825057,0.0026841052,0.019548286],[-0.0014259573,0.009959342,0.011460207,-0.008625869,0.0013398491,-0.015628178,-0.012353029,0.0019243956],[0.0053877654,0.020523258,0.015056665,0.0003709849,0.0115725845,0.015473806,0.0072570397,-0.042952426]],\"type\":\"heatmap\",\"xaxis\":\"x2\",\"yaxis\":\"y2\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"2\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.0070698336,0.0068538915,0.0061171725,-0.001400209,-0.006797165,-0.0070937183,-0.009890083,-0.0024087373],[-0.018035552,-0.021374103,0.0074957795,-0.01069488,-0.021532845,-0.019268682,-0.015469544,0.007594548],[0.008559031,0.007965711,0.008475665,0.0011637006,-0.0134536475,-0.00024054106,0.008821224,0.008216567],[-0.018543245,0.0012443699,0.012048539,-0.012462193,-0.036982972,-0.014765064,-0.023822831,0.0027825213],[-0.023109606,-0.009518623,0.0032453286,-0.0034671463,0.00791106,0.0006561652,0.0180726,-0.008972692],[0.016235303,0.0008352734,-0.013434101,-0.006694963,0.00463566,-0.013922246,-0.016832348,-0.011894802],[-0.029778484,0.025547704,0.0041333474,-0.0020659193,0.007802196,0.005275665,-0.010280417,0.013882598],[0.0043321527,0.026295291,0.017375756,0.015785899,-0.008372892,0.0025909273,-0.019294437,0.0186794]],\"type\":\"heatmap\",\"xaxis\":\"x3\",\"yaxis\":\"y3\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"3\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.0035476964,-0.011568068,-0.012183564,0.0004110206,-0.00959069,0.03063947,0.017788723,-0.0050584413],[-0.0043608695,-0.026552241,-0.003012003,0.017589148,0.005695261,-0.00036068074,0.011646353,0.0059280973],[-0.020718277,-0.012851264,-0.00907687,-0.015454159,-0.040375005,-0.0073338933,-0.015782727,-0.005927086],[0.003023807,-0.024189249,-0.017630085,-0.003665803,-0.046665575,-0.02519284,-0.024366258,0.019995999],[-0.0028419374,0.0034452602,-0.019712364,-0.02276684,-0.027774638,-0.037886795,-0.02395806,-0.007410057],[0.019910254,0.016858134,0.007543806,0.0013154084,-0.026949069,-0.0097501455,-0.028133297,0.002051292],[-0.004216265,-0.0149496645,-0.014803436,0.006112585,0.014814064,-0.012115605,-0.005668798,0.0061557926],[-0.01834044,0.0010542888,0.010950021,0.009601876,0.03524756,0.007630516,0.017929466,-0.025851265]],\"type\":\"heatmap\",\"xaxis\":\"x4\",\"yaxis\":\"y4\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"4\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.006220134,-0.013752297,0.014722858,-0.019270472,-0.020322274,0.032291405,0.012990198,0.04863487],[0.026637167,0.023256056,0.004794755,-0.008981843,-0.017908283,-0.0074465685,-0.02086949,-0.025106039],[0.04120712,-0.008724276,0.02618663,0.031498548,-0.0034554042,-0.012603184,0.00435475,-0.032499425],[-0.0062188413,-0.019754533,0.05668265,0.23584908,0.051791735,0.009507329,-0.005743112,-0.014492591],[-0.0067290347,-0.03059548,0.037825134,0.0565638,0.05793557,0.012934141,-0.023349533,-0.010261165],[0.007879544,-0.014477283,-0.00079423934,0.0230017,0.0070014196,0.032341912,0.007819106,-0.0063044997],[-0.0022764662,-0.0042731585,0.025972668,-0.008756468,0.028877214,-0.010182335,0.0009599719,0.0071255965],[0.014129191,-0.010700399,-0.030140683,-0.018769946,-0.02561366,-0.017898832,0.0211693,-0.015692739]],\"type\":\"heatmap\",\"xaxis\":\"x5\",\"yaxis\":\"y5\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"5\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.018173013,-0.0033182427,0.021699183,-0.0048155375,0.00054112263,-0.013598964,-0.02627626,-0.009355539],[-0.029138431,-0.033951126,-0.024564873,0.009745149,-0.022920655,0.0008435552,0.019182187,-0.020146975],[-0.0003252877,-0.018762872,0.0020347033,-0.0061158407,0.013758183,0.03952649,0.019310981,-0.049964342],[-0.056215405,0.010811995,0.005067885,0.022452498,0.032978214,0.00976946,0.0048911236,0.0068279],[0.014172059,-0.0104048,0.015328072,0.23519696,0.040308546,0.011565422,0.0053587165,0.018619558],[-0.029511102,0.0046654344,0.03536807,0.011852069,0.0049303826,0.012714636,0.009987537,0.005936828],[-0.03726892,-0.0127922,0.014153188,0.0167079,0.0066564176,0.026135342,-0.0016418174,0.008391548],[-0.064208925,0.047397185,-0.022726478,0.015798338,0.009953611,0.04852093,-0.01826566,0.0200479]],\"type\":\"heatmap\",\"xaxis\":\"x6\",\"yaxis\":\"y6\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"6\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.025408318,-0.01507669,-0.01697401,0.016642325,0.0051538013,-0.01999723,0.016335456,0.012103271],[0.005730752,-0.011897843,0.03011535,0.0012785662,0.003760893,0.028723024,-0.021051668,0.014420122],[0.018103838,0.003894501,0.008115411,0.0024097338,0.012463008,0.022757128,-0.0036910279,-0.016143117],[-0.01024945,-0.018375386,-0.0002532862,-0.012328078,0.016218545,0.02175073,0.0012322105,-0.027910851],[0.009167473,-0.020876972,-0.006192752,-0.053050935,0.015846085,0.03310764,-0.0007890314,0.020178664],[-0.0017546406,-0.008374538,0.010853318,-0.03099449,-0.008256372,0.012879235,-0.0021984796,-0.00578422],[0.02341877,-0.016404094,-0.016839737,-0.011666287,0.003667267,0.033382103,0.015194194,0.000780778],[0.01662751,0.03907254,0.0070297113,-0.015536317,0.024720758,0.0006723702,0.010554498,0.0027697943]],\"type\":\"heatmap\",\"xaxis\":\"x7\",\"yaxis\":\"y7\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"7\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.009305634,0.009522702,0.016397145,0.0022186656,-0.011226002,0.020628542,-0.00462279,0.017849486],[-0.00026887935,0.011576591,0.01582238,0.01040794,0.01686962,0.02110646,0.0018203938,0.044547077],[-0.010178748,0.01808553,0.0246531,0.011323572,0.004651238,0.0043641375,0.021064594,0.0031470992],[-0.022682717,0.027476719,0.026937712,0.016948722,0.00430749,0.014484913,-0.0042942017,-0.01056065],[-0.0009586103,0.014413048,0.008553053,0.0072847605,0.026565768,0.009456597,0.0045387885,-0.0074841324],[0.009219691,0.014176756,-0.007921705,0.016341519,0.029396363,0.017421506,0.0008288622,0.025858365],[-0.038193196,-0.093874946,-0.075068966,-0.017375514,-0.0009187609,-0.057580426,-0.046385154,-0.0412637],[0.01750635,-0.0061778724,-0.051846698,-0.040274665,-0.07095964,-0.08918299,0.0010685306,0.011525585]],\"type\":\"heatmap\",\"xaxis\":\"x8\",\"yaxis\":\"y8\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"8\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.005927138,0.0058914125,-0.0214353,-0.00054585095,0.009163456,0.032215554,-0.014301311,0.007389091],[-0.029623624,0.006499235,0.010591948,-0.009177834,0.008365402,0.01364279,0.020923235,-0.0025915615],[-0.028721534,0.01637157,0.021210331,-0.0073664626,-0.014946982,0.029696297,0.00076262094,0.017662961],[0.009987077,0.034936823,0.014407806,0.04255299,-0.020599745,0.04033242,0.015698567,-0.003229049],[-0.007886458,0.01989259,0.023284145,0.03700207,-0.09091644,0.036221016,0.03221415,0.019590814],[0.02982453,0.024143279,-0.018586777,0.018287929,0.008294159,0.040881075,-0.0071705114,0.0043407297],[-0.016912812,0.035769347,0.007302664,0.018932395,0.022224931,0.006656766,-0.0027582683,0.026791744],[0.045319438,0.0699243,0.0015627947,-0.003874153,-0.041537367,-0.0070543196,-0.03276936,0.08241919]],\"type\":\"heatmap\",\"xaxis\":\"x9\",\"yaxis\":\"y9\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"9\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.0032321243,0.019613799,-0.0043204185,0.009957543,-0.010180714,0.022354482,-0.0049795248,0.007707006],[-0.0057163816,0.0104691405,0.0033544898,-0.013294884,0.020853829,-0.008728185,-0.031180833,-0.019144215],[0.009373729,-0.004423879,-0.006739348,0.00039645005,0.019558406,-0.041394286,-0.00049121864,0.016390186],[0.050745517,0.023396563,0.015557723,-0.013978882,-0.02750121,-0.016280245,-0.0017596334,-0.01408406],[-0.010732189,0.024743885,0.0042653233,-0.119142845,-0.004254601,-0.013049976,-0.010435953,0.008091301],[-0.01781465,0.018078357,-0.018806206,-0.008154787,-0.034387477,-0.0002096789,-0.012938064,-0.027596505],[-0.015016772,-0.033108916,-0.011334442,-0.00062144175,-0.036458317,-0.046564773,-0.03852807,-0.002413135],[0.007414312,-0.011078719,-0.012391629,-0.008846991,0.06369695,-0.030338753,-0.0038762176,0.005269425]],\"type\":\"heatmap\",\"xaxis\":\"x10\",\"yaxis\":\"y10\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,0.08200000000000002],\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"title\":{\"text\":\"y\"}},\"xaxis2\":{\"anchor\":\"y2\",\"domain\":[0.10200000000000002,0.18400000000000005],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis2\":{\"anchor\":\"x2\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis3\":{\"anchor\":\"y3\",\"domain\":[0.20400000000000004,0.28600000000000003],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis3\":{\"anchor\":\"x3\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis4\":{\"anchor\":\"y4\",\"domain\":[0.30600000000000005,0.38800000000000007],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis4\":{\"anchor\":\"x4\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis5\":{\"anchor\":\"y5\",\"domain\":[0.4080000000000001,0.4900000000000001],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis5\":{\"anchor\":\"x5\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis6\":{\"anchor\":\"y6\",\"domain\":[0.5100000000000001,0.5920000000000001],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis6\":{\"anchor\":\"x6\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis7\":{\"anchor\":\"y7\",\"domain\":[0.6120000000000001,0.6940000000000002],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis7\":{\"anchor\":\"x7\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis8\":{\"anchor\":\"y8\",\"domain\":[0.7140000000000001,0.796],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis8\":{\"anchor\":\"x8\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis9\":{\"anchor\":\"y9\",\"domain\":[0.8160000000000002,0.8980000000000001],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis9\":{\"anchor\":\"x9\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis10\":{\"anchor\":\"y10\",\"domain\":[0.9180000000000001,1.0],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis10\":{\"anchor\":\"x10\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"annotations\":[{\"font\":{},\"showarrow\":false,\"text\":\"L3N1836\",\"x\":0.04100000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L3N1585\",\"x\":0.14300000000000004,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L3N1897\",\"x\":0.24500000000000005,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L3N524\",\"x\":0.3470000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L3N380\",\"x\":0.44900000000000007,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L3N618\",\"x\":0.5510000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L3N2045\",\"x\":0.6530000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L3N1932\",\"x\":0.7550000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L3N1556\",\"x\":0.8570000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L3N1418\",\"x\":0.9590000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"}],\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"title\":{\"text\":\"Cosine sim of Output weights and the my color probe for top layer 3 neurons\"}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('75c66e6c-418d-48f8-b2c6-e73019eaf43a');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "layer = 3\n",
    "top_layer_3_neurons = focus_cache[\"post\", layer][:, 3:-3].std(dim=[0, 1]).argsort(descending=True)[:10]\n",
    "heatmaps_blank = []\n",
    "heatmaps_my = []\n",
    "for neuron in top_layer_3_neurons:\n",
    "    neuron = neuron.item()\n",
    "    w_out = model.blocks[layer].mlp.W_out[neuron, :].detach()\n",
    "    w_out /= w_out.norm()\n",
    "    heatmaps_blank.append((w_out[:, None, None] * blank_probe_normalised).sum(dim=0))\n",
    "    heatmaps_my.append((w_out[:, None, None] * my_probe_normalised).sum(dim=0))\n",
    "    # state = torch.zeros(63, device=\"cuda\")\n",
    "    # state[stoi_indices] = w_out @ model.W_U[:, 1:]\n",
    "    # heatmaps.append(state.reshape(8, 8))\n",
    "imshow(heatmaps_blank,\n",
    "    facet_col=0,\n",
    "    y=[i for i in \"ABCDEFGH\"],\n",
    "    title=f\"Cosine sim of Output weights and the blank color probe for top layer 3 neurons\",\n",
    "    facet_labels=[f\"L3N{n.item()}\" for n in top_layer_3_neurons])\n",
    "imshow(heatmaps_my,\n",
    "    facet_col=0,\n",
    "    y=[i for i in \"ABCDEFGH\"],\n",
    "    title=f\"Cosine sim of Output weights and the my color probe for top layer 3 neurons\",\n",
    "    facet_labels=[f\"L3N{n.item()}\" for n in top_layer_3_neurons])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"7bb958d6-1183-4fc7-ae02-40f691362f7a\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"7bb958d6-1183-4fc7-ae02-40f691362f7a\")) {                    Plotly.newPlot(                        \"7bb958d6-1183-4fc7-ae02-40f691362f7a\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.004620673,0.003999417,-0.0151387565,-0.0036488683,-0.0035565067,-0.0005703792,0.0027337149,-0.00072012655],[-0.005134195,0.012683451,0.013976313,-0.009963853,0.007367244,0.008503251,-0.006023852,-0.00090149045],[-0.0073735025,0.019385729,0.0064500663,-0.01481563,-0.0066692494,-0.006449368,0.0011109551,-0.013609186],[-0.013106853,0.011596568,-0.021229088,0.0,0.0,0.038518857,-0.01143467,-0.010930043],[0.012364518,-0.0040434822,-0.0013714395,0.0,0.0,0.007948395,-0.003839963,-0.0066738576],[0.0077421945,0.00035388395,0.013013728,0.68313706,0.012190497,0.029800711,-0.0010450725,-0.017724743],[0.010998869,0.0034485087,0.012685085,0.0042002406,0.021784045,0.0025637504,0.0055648237,0.0013854224],[0.010265032,0.013790563,0.03210452,0.0018010964,0.010172281,0.00046942197,0.0044992603,-0.004730176]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"1\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.007363545,0.007844213,0.0119162435,-0.0027908608,0.004278874,-0.012836126,0.020091707,-0.0030194297],[-0.012124722,-0.00026003644,-0.015149095,0.007063983,0.010075994,-0.00096686743,0.014553305,-0.00005191937],[0.00785872,0.0013264418,0.008831436,0.020865362,0.008981558,-0.0011835219,-0.0054666866,-0.020125793],[0.0029345863,-0.00014255755,-0.009152781,0.0,0.0,0.0025030915,0.01233235,0.0065583233],[-0.005029694,-0.00068985857,-0.002981498,0.0,0.0,0.6662179,0.000979282,-0.011484919],[-0.0110379355,0.019891445,-0.005863741,0.005678352,-0.011206934,0.02685014,0.008588284,0.012270916],[0.003001864,-0.008096851,0.0020716395,0.007923428,0.009291496,0.01765899,0.015245891,0.016869701],[-0.004919963,-0.007604644,0.002429776,0.019435592,0.006889591,0.015666647,0.016917877,0.004788026]],\"type\":\"heatmap\",\"xaxis\":\"x2\",\"yaxis\":\"y2\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"2\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.009630322,0.012919057,0.01719737,-0.005842995,0.032516252,0.003155021,0.037038207,-0.00069046393],[0.0033542365,0.005672455,0.016084127,-0.009178756,0.010578915,0.04623775,0.02834081,0.0149701405],[0.0035184696,-0.0036749262,0.026037429,-0.00018946081,0.049068093,0.7329967,0.03152188,-0.007793309],[0.0136154,-0.008544553,-0.0027291141,0.0,0.0,0.041401383,-0.0038249148,-0.009633776],[0.017420914,-0.014935107,0.0017183181,0.0,0.0,-0.011720167,0.0022781892,0.001905296],[0.0018030051,0.015280481,-0.00023811124,0.0005189711,-0.0049662758,0.010291329,-0.01357918,0.0148684615],[-0.010484642,-0.027332854,-0.027019106,-0.01804696,-0.016180355,-0.011545714,0.0021680593,0.00911882],[-0.017435279,0.006495918,0.005203901,-0.004067477,0.01234097,0.00015055574,0.01893632,-0.0052997144]],\"type\":\"heatmap\",\"xaxis\":\"x3\",\"yaxis\":\"y3\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"3\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.015007826,-0.005072225,-0.0024787374,0.013294596,0.010241054,-0.01316338,-0.003931286,0.0030311085],[-0.006353326,0.011854107,-0.0075632203,-0.010236282,0.019743882,-0.0018661069,-0.014780767,0.011001853],[0.012335803,0.0098271845,0.021941897,-0.009136422,0.0030628042,0.000045556575,0.008377712,0.008517189],[-0.013252986,0.024714516,0.69056076,0.0,0.0,0.011714577,0.012937862,-0.0046995487],[0.016270896,0.030807696,0.0011223592,0.0,0.0,-0.013297127,-0.0023217483,0.01268098],[-0.0045334976,0.017727274,0.0007617511,-0.020264601,0.04066541,0.00093265064,0.023642525,0.0036603208],[0.005857352,0.007590562,-0.0066701546,-0.0027537812,0.011446449,-0.00051685795,0.011412837,0.013535358],[0.006653229,-0.007664403,0.00442281,0.0057833213,0.013693918,-0.0012517045,-0.003437534,0.021385705]],\"type\":\"heatmap\",\"xaxis\":\"x4\",\"yaxis\":\"y4\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"4\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.0061170696,-0.009092603,-0.017385881,0.009349702,0.009306793,0.008860614,0.011223463,0.016666502],[-0.0058601703,-0.005656993,-0.009307597,-0.014538616,0.0065603144,0.0018292051,0.010266747,0.0017414829],[-0.03964282,-0.010485142,0.0034033316,-0.008554138,0.050053593,0.006888207,0.0058883335,0.0044901306],[0.018200118,0.024352942,0.04336137,0.0,0.0,-0.018465992,0.008654814,-0.00086280424],[0.014440581,0.046880417,0.71512944,0.0,0.0,0.0033549406,0.008844236,0.0045567807],[0.028299974,0.01635484,0.028104499,-0.021974318,0.0004172325,-0.017715808,0.008207601,-0.0069206664],[0.008096041,0.021671198,0.0071596773,-0.0001802165,0.0068012923,0.0030515194,-0.011195343,-0.010692496],[0.022090454,0.009419005,0.009223379,0.0028722957,0.009870861,-0.04291875,-0.009071791,-0.0074611753]],\"type\":\"heatmap\",\"xaxis\":\"x5\",\"yaxis\":\"y5\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"5\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.0010301489,-0.0025690952,-0.003190387,-0.014047401,-0.0058727628,-0.00091482326,0.002309463,-0.010599654],[0.00058748573,0.018944968,0.009892932,0.005712359,-0.0033629797,0.0047545563,0.013796609,0.0052435463],[0.0073357522,-0.0008461843,-0.0029702708,-0.0013085734,-0.011783255,0.00552674,-0.0068361796,-0.018788485],[-0.0042861924,0.013902404,0.010192791,0.0,0.0,0.013166472,0.0006859526,0.009207499],[0.020140737,0.0025781598,0.0058435015,0.0,0.0,-0.0023060888,-0.008383889,-0.01159155],[0.0019450318,-0.0011260817,0.008376893,0.024416056,0.6610352,0.03592717,0.020473296,0.006881021],[0.0005153399,0.0028015696,0.006506989,0.014587529,0.018585531,0.00683921,0.010674047,-0.0049861744],[-0.00070090406,-0.0015793312,-0.0208399,0.0061027817,-0.0047874656,0.018901283,0.010952786,-0.0087264925]],\"type\":\"heatmap\",\"xaxis\":\"x6\",\"yaxis\":\"y6\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"6\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.00552562,0.024912287,-0.00026717968,0.008840905,0.016159132,0.012874361,0.019474618,-0.012159703],[0.0012046807,-0.005735591,0.018081065,-0.017331628,0.014324976,0.007955207,-0.0042469744,0.01474555],[-0.00014983863,-0.009731408,-0.0014024703,0.010780336,0.0019058203,0.019877952,0.003633135,0.000058766454],[0.0046735946,-0.009264251,0.04730227,0.0,0.0,-0.005808195,-0.0012206314,-0.0013149474],[0.02452724,0.01993812,0.03639678,0.0,0.0,0.013691371,-0.010971105,-0.0078851],[-0.0053956835,0.050604325,0.6971342,0.020648487,0.022867933,0.018975152,-0.0041222665,0.011174443],[0.042849146,0.05722142,0.037029058,0.020102758,-0.00927996,0.00041514542,0.013104163,0.008629073],[0.0077106915,0.041165855,-0.0053939726,-0.0043782964,0.0027347505,0.0023513827,-0.0013830448,0.00694493]],\"type\":\"heatmap\",\"xaxis\":\"x7\",\"yaxis\":\"y7\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"7\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.0005657561,0.008892132,-0.003183132,0.00037861615,-0.0012187865,0.008558748,0.016118199,-0.019821763],[0.0013752282,0.0034479462,0.002133289,-0.004123226,0.0005145632,0.004423082,0.015008908,0.0051294416],[0.00072957855,-0.0014076531,0.017070778,0.0023693275,-0.0037418203,0.02056721,0.061647564,0.012234582],[-0.019753529,-0.0027958276,0.004016429,0.0,0.0,0.6621711,0.02040331,0.0021251123],[-0.008625215,-0.004101418,-0.007975465,0.0,0.0,0.0192848,-0.0019687768,0.022849534],[0.014202723,0.006780114,0.01698923,0.04239642,-0.007804852,0.0114115765,0.014762754,-0.02027699],[0.013079058,0.0023640739,-0.019319542,-0.0020055212,-0.0072572697,0.0011892505,0.0030612806,-0.010648357],[-0.0024828091,0.0049140994,-0.0019060187,0.010385837,-0.0028359331,-0.009055622,0.011700466,-0.009568455]],\"type\":\"heatmap\",\"xaxis\":\"x8\",\"yaxis\":\"y8\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"8\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.0020697494,0.018404625,0.02317318,0.009304386,0.030414095,-0.0119395275,-0.004717093,0.0074045137],[0.00007665157,-0.0040325876,0.0026853085,0.009399738,0.014852596,-0.006293422,0.028729238,0.0022554975],[-0.014734706,-0.03763471,0.030388921,0.64629674,0.051077552,0.017655335,-0.00289411,-0.008808475],[0.0026988722,-0.019782266,-0.0189447,0.0,0.0,-0.0119683985,-0.00086789206,-0.008450951],[-0.015004309,-0.012178134,-0.005667405,0.0,0.0,0.033009313,0.004802945,0.0036103725],[0.005546147,0.009257562,0.015325187,-0.0016383938,-0.00881165,-0.0016585905,0.020429518,0.025908018],[-0.005867321,-0.012112465,0.012407525,0.0070929695,0.012530319,-0.003185805,0.002137985,-0.0095023215],[-0.005974792,0.0071835183,0.015146216,0.008027853,0.01274473,-0.02257165,0.003151752,-0.003319187]],\"type\":\"heatmap\",\"xaxis\":\"x9\",\"yaxis\":\"y9\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"9\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.027843745,0.033820465,-0.016695878,0.016693432,0.010099815,0.003623153,-0.010118571,-0.0050480664],[0.0204704,0.060186826,0.067003064,0.031213477,-0.016618159,-0.013122983,0.011994038,0.0055490285],[-0.0056208223,0.036014304,0.6529534,0.014557097,0.009513129,-0.018450912,-0.0033928677,0.002438873],[0.012013055,-0.001397402,0.016907765,0.0,0.0,-0.0025913222,-0.003208641,-0.0053801658],[0.009996109,0.005043328,-0.00078890845,0.0,0.0,-0.017203115,0.008106381,0.0010045003],[-0.0043779714,0.002283262,-0.0032992847,-0.0045409985,-0.01183182,0.01168005,-0.015507368,0.00057673454],[0.010937015,0.000565934,0.006622835,0.00954415,-0.006094481,0.0038600974,0.01601237,-0.0015472956],[0.00018640421,0.0063362718,-0.00955192,0.005447544,-0.011091974,0.006794015,0.0096686175,0.0071771145]],\"type\":\"heatmap\",\"xaxis\":\"x10\",\"yaxis\":\"y10\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,0.08200000000000002],\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"title\":{\"text\":\"y\"}},\"xaxis2\":{\"anchor\":\"y2\",\"domain\":[0.10200000000000002,0.18400000000000005],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis2\":{\"anchor\":\"x2\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis3\":{\"anchor\":\"y3\",\"domain\":[0.20400000000000004,0.28600000000000003],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis3\":{\"anchor\":\"x3\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis4\":{\"anchor\":\"y4\",\"domain\":[0.30600000000000005,0.38800000000000007],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis4\":{\"anchor\":\"x4\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis5\":{\"anchor\":\"y5\",\"domain\":[0.4080000000000001,0.4900000000000001],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis5\":{\"anchor\":\"x5\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis6\":{\"anchor\":\"y6\",\"domain\":[0.5100000000000001,0.5920000000000001],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis6\":{\"anchor\":\"x6\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis7\":{\"anchor\":\"y7\",\"domain\":[0.6120000000000001,0.6940000000000002],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis7\":{\"anchor\":\"x7\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis8\":{\"anchor\":\"y8\",\"domain\":[0.7140000000000001,0.796],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis8\":{\"anchor\":\"x8\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis9\":{\"anchor\":\"y9\",\"domain\":[0.8160000000000002,0.8980000000000001],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis9\":{\"anchor\":\"x9\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis10\":{\"anchor\":\"y10\",\"domain\":[0.9180000000000001,1.0],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis10\":{\"anchor\":\"x10\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"annotations\":[{\"font\":{},\"showarrow\":false,\"text\":\"L4N547\",\"x\":0.04100000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N1122\",\"x\":0.14300000000000004,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N691\",\"x\":0.24500000000000005,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N352\",\"x\":0.3470000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N345\",\"x\":0.44900000000000007,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N1076\",\"x\":0.5510000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N112\",\"x\":0.6530000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N115\",\"x\":0.7550000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N1285\",\"x\":0.8570000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N1882\",\"x\":0.9590000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"}],\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"title\":{\"text\":\"Cosine sim of Output weights and the blank color probe for top layer 4 neurons\"}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('7bb958d6-1183-4fc7-ae02-40f691362f7a');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"ce9894b3-644e-4c2b-8cc9-36cfba7e55c8\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"ce9894b3-644e-4c2b-8cc9-36cfba7e55c8\")) {                    Plotly.newPlot(                        \"ce9894b3-644e-4c2b-8cc9-36cfba7e55c8\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.0013549179,0.0180175,0.010061342,0.005006274,-0.00030220207,0.0144155435,0.008861615,-0.009659069],[-0.0058413967,0.0031556487,0.0074655525,0.004624754,-0.002442575,-0.004988432,-0.004785763,-0.02064031],[0.010900407,0.00427026,-0.007820511,-0.008196546,-0.016279224,0.004337296,-0.012300347,0.006804986],[0.015824696,0.02165047,0.00014667585,0.0058498587,0.005527543,-0.005793173,0.005001208,0.0054420237],[0.0027280888,0.004245691,-0.0023296122,0.0824566,0.040691935,-0.005789835,0.012917809,0.013105478],[-0.0006695483,0.008578815,-0.035375625,-0.04275009,0.019641008,0.011628353,0.013588353,0.008255681],[0.0047025615,-0.043134883,-0.04933697,-0.05143424,-0.017961565,-0.005487796,0.011820254,-0.0023790207],[-0.013270168,0.019040482,0.0024805078,0.050783876,0.0145209655,0.02079157,0.018320812,0.007719258]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"1\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.00022371113,-0.013629707,0.0081306305,-0.006180372,-0.006491458,-0.0077322517,-0.01097497,0.009449473],[0.013299737,0.009813192,0.004877854,0.005281255,0.0009972602,0.0011349507,-0.0010741837,-0.0014477186],[0.021047242,0.019141138,0.016267907,0.0058029257,0.0034649502,0.013272575,-0.012707852,0.013037634],[0.022476433,0.0061027277,0.0011371169,0.01422082,0.046846457,0.005369151,-0.0134773515,-0.012020398],[0.006689314,0.00044516474,-0.0069745136,-0.0034547895,0.054487646,-0.015280774,-0.04159026,0.016158223],[0.006544849,-0.007403532,-0.009776566,-0.01616716,0.0010155588,0.0165198,-0.02355068,-0.018348243],[0.0038406323,0.019321647,0.0072731134,0.008228077,0.004034166,0.027060913,-0.007545525,-0.015062492],[0.0019075118,0.019000713,0.018344834,0.00945694,-0.0121423835,-0.009607108,-0.004313126,-0.014025918]],\"type\":\"heatmap\",\"xaxis\":\"x2\",\"yaxis\":\"y2\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"2\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.016017586,0.011237353,-0.001580663,0.025718082,-0.010586329,0.023598284,-0.026459146,-0.025043191],[0.011334609,-0.014235459,0.00062091555,-0.01955336,-0.022960193,-0.02087619,-0.00934078,-0.0066629713],[0.015232634,0.017506577,-0.0107361525,-0.0032235868,0.024614055,-0.007631844,-0.016911542,0.012855037],[0.0073266095,-0.004477516,-0.004297983,-0.022052508,0.07065181,0.038037393,-0.01502532,0.018371172],[0.0041953837,-0.0034350175,-0.009803534,0.0035576094,-0.0008475855,0.000841734,-0.015623331,-0.0024596304],[0.021113902,-0.0057703294,-0.0027194992,0.01249694,0.013741527,-0.00044588558,0.0009962246,0.0073240567],[0.011447255,-0.0022104234,0.0017734934,0.010594387,0.022931468,-0.006419331,-0.00096477196,-0.018327277],[-0.00334711,-0.010460212,-0.0028837174,0.013303672,0.017276209,-0.013032317,0.0029404564,-0.010330359]],\"type\":\"heatmap\",\"xaxis\":\"x3\",\"yaxis\":\"y3\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"3\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.014431447,0.017708644,-0.009093478,0.008331368,0.02314128,0.0077390047,0.008023471,0.0024710838],[0.027656265,-0.007228762,0.021709666,-0.0050363825,-0.0023865644,-0.0089405775,0.0061130496,0.004865773],[-0.009183221,-0.037767604,-0.00064937025,0.025162905,-0.0047662426,0.0045510884,0.00036145002,-0.009737449],[-0.009074703,-0.007649784,-0.017782247,0.05394785,0.018481899,0.0062259138,0.007619991,-0.003481986],[-0.01099003,0.020393953,0.029938376,0.04724342,-0.008142147,-0.0004262831,0.01036937,-0.0027575642],[-0.0145367235,-0.013472842,0.0038112048,0.011290049,-0.0007243417,-0.008659339,0.018150028,-0.00088326633],[-0.014522202,-0.014867985,-0.009147165,0.00302016,0.0077672843,0.015488509,0.012018604,-0.008062551],[-0.011977453,0.017758314,0.023428222,0.001099728,0.020957353,0.017139904,0.0020953082,-0.025823787]],\"type\":\"heatmap\",\"xaxis\":\"x4\",\"yaxis\":\"y4\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"4\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.004571503,0.007407203,0.0040702596,0.0024923459,0.013650168,0.0001384262,0.0042753927,-0.00055811927],[-0.011593185,-0.004866848,0.008538447,0.012003645,-0.0034412127,-0.011948025,0.013796445,0.01255355],[0.0036053471,-0.004214388,0.0204621,-0.0072910376,0.0064177755,-0.0032788813,0.013883991,0.01236514],[0.0041567683,0.001905147,0.0038886173,0.034573134,-0.014725216,0.008386165,0.014527045,0.022075776],[0.01826359,-0.031402163,0.03845925,0.023132194,0.014239198,0.0021142624,0.024948241,0.024920028],[0.010718957,-0.034375165,-0.003151876,0.025744895,-0.011321491,0.011066668,0.017826745,0.011384645],[0.028693827,-0.026226245,0.017111987,-0.0118903555,-0.0053995736,-0.0064854007,0.017937487,0.0068077818],[-0.020422943,0.012197584,0.011932515,0.0075804023,0.009554792,-0.0066507827,-0.018828647,-0.0013057776]],\"type\":\"heatmap\",\"xaxis\":\"x5\",\"yaxis\":\"y5\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"5\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.0060120877,0.0069922702,0.010339446,-0.008962277,0.009000196,0.020674841,0.015053678,0.0010675788],[0.013548667,0.0122444555,0.012448141,0.00063499436,0.0065431395,0.0044284277,0.007074736,-0.005082489],[0.018378899,0.0058671813,-0.0014120974,-0.011005761,-0.0013780147,0.0022126809,-0.0017258544,0.007411875],[0.03321211,0.020075599,0.018373974,-0.008012515,0.015259445,-0.009838251,-0.003588669,0.0042510554],[0.0034883253,0.013203491,-0.0075223055,0.040983677,0.04794056,0.011875642,-0.0088501135,0.008549666],[0.03459225,0.021623243,0.009487126,0.008919223,-0.02376113,0.004383685,0.008829493,-0.012169609],[0.01773528,0.0041706488,-0.010592443,-0.03310947,-0.04762043,-0.041161217,-0.034474067,-0.002939947],[0.009749753,0.0013853908,0.013604954,-0.0019570813,0.00787711,-0.017580114,0.016031701,-0.00408082]],\"type\":\"heatmap\",\"xaxis\":\"x6\",\"yaxis\":\"y6\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"6\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.005460415,0.0077590793,0.0011680585,0.0131723145,0.02678528,-0.0039379112,-0.011998568,-0.010432012],[-0.010965364,-0.012775213,-0.004891962,0.0063848607,0.020161014,0.010009898,0.0031851428,0.020374443],[-0.0022777878,-0.015644796,-0.013293989,-0.010060582,0.0063211545,-0.0045579765,0.012643412,0.02624501],[-0.006899854,-0.017388238,0.0022498183,0.010197772,-0.007383693,-0.008208977,0.015479904,0.025402492],[0.0065576658,-0.009446468,0.014308014,0.076394916,-0.017267764,0.004664005,0.007024402,0.024750743],[0.022934003,-0.022052713,-0.03274358,0.06012298,0.023891788,0.0016879085,0.018413916,0.008170728],[-0.015181914,-0.055412814,-0.025575265,-0.009297621,-0.01641496,-0.00039724447,0.003854137,0.00024472363],[-0.017094204,-0.021964679,0.00047539733,0.019693505,0.01933147,0.036300853,0.016357921,0.003547728]],\"type\":\"heatmap\",\"xaxis\":\"x7\",\"yaxis\":\"y7\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"7\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.003504267,-0.019390631,-0.022208316,0.0131389145,0.02136893,0.011740208,0.019503389,0.0036475454],[-0.0070865434,0.014061288,0.011555772,0.0013723914,-0.009349085,0.019698072,-0.01646355,0.008704223],[0.005308047,0.016104419,-0.0031175874,-0.01857493,0.00564567,-0.002678433,-0.01961745,-0.010151552],[-0.002224967,-0.011069834,0.015083227,0.020867933,0.063702375,-0.017946282,-0.0033325441,-0.004402198],[0.020368138,0.006575197,0.0011048429,-0.007208513,0.026966922,0.02129576,-0.0125756515,-0.015158791],[-0.0036626477,0.0063860733,-0.009490861,-0.0038904478,-0.008735295,0.012997357,-0.015014295,0.0040030945],[-0.0064912224,0.0059330054,-0.009031196,0.00008568168,0.011596677,0.003005878,0.011823904,0.0073569957],[-0.000033734366,-0.008470107,0.0044255164,0.022333343,-0.008543195,0.0123282075,0.0073829405,0.00010135025]],\"type\":\"heatmap\",\"xaxis\":\"x8\",\"yaxis\":\"y8\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"8\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.022442084,0.004655876,-0.01637123,0.022569522,-0.008817609,0.005142293,-0.004782416,0.008137553],[0.0028909165,-0.01464965,-0.035553053,-0.027290385,-0.03337557,0.013108138,-0.0024501532,0.0010454208],[0.0016303272,-0.006211521,0.001324445,0.025250327,0.000047853217,0.014901596,-0.008446018,-0.001142784],[0.001573693,0.00675885,0.010067963,0.077842064,0.030471906,0.010409549,0.001728382,0.0021789102],[0.022846522,0.0023985328,-0.0062390147,0.010095876,-0.021882007,0.006907204,0.01393393,0.017430553],[-0.0049819443,0.01435145,0.00440884,0.015428398,0.0008345153,-0.006073946,0.009325452,-0.009951991],[0.01224498,0.015266409,0.015284278,0.0077797174,-0.0045816544,0.01778598,-0.002527915,0.0044479277],[0.0086067505,0.0005438328,0.014508378,0.011570254,-0.003777448,0.008670356,0.007149777,-0.02737698]],\"type\":\"heatmap\",\"xaxis\":\"x9\",\"yaxis\":\"y9\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"9\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.0044483803,-0.018407928,0.0028299596,0.0065803416,0.00026923418,-0.008971734,0.0025240383,0.011766028],[-0.005533647,-0.04775103,-0.011775876,-0.027385585,-0.008395963,0.005787641,0.0058689117,0.0005240422],[0.0498574,-0.02704031,0.0371,0.061901473,-0.0015480556,-0.00257124,0.019046415,0.0068839807],[0.011826297,0.014506764,0.049811035,0.09056035,-0.0010164343,0.013696288,-0.0030708667,-0.00026271306],[0.010507988,-0.024228852,0.0023541693,-0.014356785,0.002550384,0.00046362728,-0.0010157526,0.0054754205],[-0.00084050186,-0.02580044,-0.005076376,0.013939472,-0.01172539,0.000066779554,-0.003960423,0.0033299103],[-0.002531291,0.004616894,-0.0038238168,-0.0056344606,0.016157955,-0.008961712,-0.0021667145,-0.0047216676],[0.01221508,0.0065591806,0.004113759,0.023373216,0.005577652,0.010712024,-0.0074332748,0.00011142157]],\"type\":\"heatmap\",\"xaxis\":\"x10\",\"yaxis\":\"y10\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,0.08200000000000002],\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"title\":{\"text\":\"y\"}},\"xaxis2\":{\"anchor\":\"y2\",\"domain\":[0.10200000000000002,0.18400000000000005],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis2\":{\"anchor\":\"x2\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis3\":{\"anchor\":\"y3\",\"domain\":[0.20400000000000004,0.28600000000000003],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis3\":{\"anchor\":\"x3\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis4\":{\"anchor\":\"y4\",\"domain\":[0.30600000000000005,0.38800000000000007],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis4\":{\"anchor\":\"x4\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis5\":{\"anchor\":\"y5\",\"domain\":[0.4080000000000001,0.4900000000000001],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis5\":{\"anchor\":\"x5\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis6\":{\"anchor\":\"y6\",\"domain\":[0.5100000000000001,0.5920000000000001],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis6\":{\"anchor\":\"x6\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis7\":{\"anchor\":\"y7\",\"domain\":[0.6120000000000001,0.6940000000000002],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis7\":{\"anchor\":\"x7\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis8\":{\"anchor\":\"y8\",\"domain\":[0.7140000000000001,0.796],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis8\":{\"anchor\":\"x8\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis9\":{\"anchor\":\"y9\",\"domain\":[0.8160000000000002,0.8980000000000001],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis9\":{\"anchor\":\"x9\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis10\":{\"anchor\":\"y10\",\"domain\":[0.9180000000000001,1.0],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis10\":{\"anchor\":\"x10\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"annotations\":[{\"font\":{},\"showarrow\":false,\"text\":\"L4N547\",\"x\":0.04100000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N1122\",\"x\":0.14300000000000004,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N691\",\"x\":0.24500000000000005,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N352\",\"x\":0.3470000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N345\",\"x\":0.44900000000000007,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N1076\",\"x\":0.5510000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N112\",\"x\":0.6530000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N115\",\"x\":0.7550000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N1285\",\"x\":0.8570000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N1882\",\"x\":0.9590000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"}],\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"title\":{\"text\":\"Cosine sim of Output weights and the my color probe for top layer 4 neurons\"}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('ce9894b3-644e-4c2b-8cc9-36cfba7e55c8');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "layer = 4\n",
    "top_layer_4_neurons = focus_cache[\"post\", layer][:, 3:-3].std(dim=[0, 1]).argsort(descending=True)[:10]\n",
    "heatmaps_blank = []\n",
    "heatmaps_my = []\n",
    "for neuron in top_layer_4_neurons:\n",
    "    neuron = neuron.item()\n",
    "    w_out = model.blocks[layer].mlp.W_out[neuron, :].detach()\n",
    "    w_out /= w_out.norm()\n",
    "    heatmaps_blank.append((w_out[:, None, None] * blank_probe_normalised).sum(dim=0))\n",
    "    heatmaps_my.append((w_out[:, None, None] * my_probe_normalised).sum(dim=0))\n",
    "    # state = torch.zeros(64, device=\"cuda\")\n",
    "    # state[stoi_indices] = w_out @ model.W_U[:, 1:]\n",
    "    # heatmaps.append(state.reshape(8, 8))\n",
    "imshow(heatmaps_blank,\n",
    "    facet_col=0,\n",
    "    y=[i for i in \"ABCDEFGH\"],\n",
    "    title=f\"Cosine sim of Output weights and the blank color probe for top layer 4 neurons\",\n",
    "    facet_labels=[f\"L4N{n.item()}\" for n in top_layer_4_neurons])\n",
    "imshow(heatmaps_my,\n",
    "    facet_col=0,\n",
    "    y=[i for i in \"ABCDEFGH\"],\n",
    "    title=f\"Cosine sim of Output weights and the my color probe for top layer 4 neurons\",\n",
    "    facet_labels=[f\"L4N{n.item()}\" for n in top_layer_4_neurons])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Why is the blank output weights of the layer 4 neurons so striking?!\n",
    "\n",
    "It's because blank probe is actually highly aligned with the unembed weights - a cell can *only* be legal to play in if it is blank (obviously). Since calculating blankness is easy (you just check whether a move was played), the model should be able to do this in a single layer. But our probe was computed after layer 6, when there's a lot of extraneous information - probably, the probe also learned that if there's going to be a high logit for a cell then that cell is definitely blank, and so the blank directions are partially aligned with the unembed directions.\n",
    "\n",
    "Let's look at the cosine sim of those neurons and the unembed directions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"1de5ee6b-12c8-4745-a124-1c23c09f8e7d\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"1de5ee6b-12c8-4745-a124-1c23c09f8e7d\")) {                    Plotly.newPlot(                        \"1de5ee6b-12c8-4745-a124-1c23c09f8e7d\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.00009321724,-0.0075610057,0.00005429238,-0.011649113,0.0015521199,-0.0053167725,-0.010511212,-0.005087765],[-0.012519674,0.01050598,0.0043134913,-0.020243973,0.00891269,0.0041054823,-0.013784716,-0.033359703],[-0.045616813,-0.0065359455,-0.004705423,-0.025045983,-0.005028786,0.00038864696,-0.026731584,-0.012037264],[-0.006693367,-0.06463157,-0.0024981927,0.0,0.0,-0.04415019,-0.0140043525,-0.0038001197],[0.025566721,-0.008655444,0.00056338776,0.0,0.0,-0.013032788,0.0028943866,-0.003328517],[-0.024239954,-0.054709055,-0.010827966,0.8345181,0.010483582,-0.03636898,-0.023307784,-0.025625445],[-0.00024347752,0.00242462,-0.1025155,-0.06872051,-0.014940187,0.0019460022,0.013818571,-0.0076712407],[-0.0027538352,-0.110278495,0.033614676,-0.0721509,0.0287931,-0.0688242,0.0028397485,-0.006846036]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"1\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.009508073,-0.033989087,-0.015952457,-0.010303771,-0.0033479258,-0.0027841758,0.00018718839,-0.011666803],[-0.0059035867,-0.0073846374,-0.040307827,0.0034722686,0.0067780577,0.004401926,0.008337483,-0.018487321],[-0.00336699,0.0012316629,0.012703117,-0.01948525,0.015456665,-0.03378879,0.02342249,-0.06865356],[-0.004942164,0.0037813503,-0.010746721,0.0,0.0,0.0034819571,-0.022608843,0.012164265],[-0.022566162,-0.032772705,-0.022402912,0.0,0.0,0.8320414,-0.06765453,-0.06895443],[-0.021863874,-0.0026879287,0.018381927,0.0026968718,0.015792105,0.02024559,-0.06162467,0.025887059],[-0.003018288,0.0011820477,0.014959399,-0.08434452,0.019112552,-0.10245181,-0.0065436773,-0.12749961],[-0.021846905,-0.027676646,-0.0290129,-0.0021118224,-0.0010715872,-0.0046038358,-0.011804834,-0.023626067]],\"type\":\"heatmap\",\"xaxis\":\"x2\",\"yaxis\":\"y2\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"2\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.0077805603,-0.0008939281,0.007620899,-0.073814645,0.021377157,-0.07659024,0.015803153,-0.15125345],[0.0026123952,0.007097766,0.014699387,0.0053499145,-0.028539747,-0.031081747,-0.13949701,0.021187458],[-0.014620075,-0.020711686,-0.028315157,-0.035266638,0.028469827,0.799368,-0.008541212,-0.076553345],[-0.0048842467,-0.00827899,-0.030617265,0.0,0.0,0.047975402,0.004443709,0.008650878],[0.0051707253,0.0011462984,-0.020579506,0.0,0.0,-0.02400633,0.018061988,-0.06774543],[0.0058803177,0.0012737121,-0.017945081,0.0026912391,-0.017604034,-0.0029363548,-0.004019061,0.006679248],[-0.004428637,-0.025881866,0.0050340723,-0.0022610016,0.0025285902,-0.020354414,-0.0031499583,-0.014429677],[-0.028693257,0.00050216215,-0.0024277912,-0.016078033,0.0019469885,-0.021481914,-0.004129853,-0.002709793]],\"type\":\"heatmap\",\"xaxis\":\"x3\",\"yaxis\":\"y3\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"3\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.0048752166,-0.00059429463,-0.037791256,0.0149698015,0.007980866,-0.046969935,0.009876188,-0.0050713643],[-0.13848174,-0.0013509523,-0.09337318,0.018233579,-0.07530774,0.009509932,-0.0041278675,0.0046157967],[0.00979682,-0.097516075,0.007875989,0.014040649,0.005247839,0.00824414,0.0006848164,-0.010605299],[-0.0940084,-0.077683024,0.8431745,0.0,0.0,-0.0037419023,-0.010701839,-0.022562798],[0.03434514,-0.032300457,0.004809414,0.0,0.0,0.0153555665,0.0098629575,0.010201912],[-0.07114838,0.006285296,-0.059566155,-0.007926654,-0.042334076,-0.004482572,0.02629055,-0.0026706988],[-0.02154164,0.0064430377,-0.013759902,-0.019261517,0.0063056387,-0.021753222,-0.004255375,-0.00049511343],[-0.012183644,-0.0029755244,-0.0189738,-0.0099381,0.0038547087,0.007901929,-0.0384457,-0.006894417]],\"type\":\"heatmap\",\"xaxis\":\"x4\",\"yaxis\":\"y4\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"4\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.01892718,-0.012035452,-0.0074892556,0.015926734,0.0011885092,0.0144529445,-0.036573,-0.02567292],[0.0037751282,0.005989236,-0.026069537,-0.009287562,-0.0017448766,-0.029212622,0.0010247156,-0.020752078],[-0.12082976,-0.00092244614,-0.027702842,-0.006933148,-0.018924378,0.017355124,0.006795435,-0.023501264],[0.008554328,-0.036156386,0.04993678,0.0,0.0,0.016295956,0.012661134,-0.0054298025],[-0.046247207,-0.07579013,0.80671227,0.0,0.0,-0.0051117763,-0.012428079,-0.007517807],[0.011111511,-0.1174121,-0.031321727,0.011899594,-0.0035178764,0.018340059,0.022774085,-0.0041219248],[-0.10448015,-0.0029961336,-0.071755126,-0.02024388,-0.035013333,0.00087547675,0.0014520045,-0.004499488],[-0.011871964,0.010470072,-0.011676889,0.011854302,0.005186809,-0.03227969,-0.012652807,-0.010660948]],\"type\":\"heatmap\",\"xaxis\":\"x5\",\"yaxis\":\"y5\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"5\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.0072373096,-0.0030828342,0.012262203,-0.0053819213,-0.013636688,-0.0074085295,-0.004185453,-0.02463543],[-0.022243736,0.011458799,0.008457467,0.0013130447,-0.019928154,-0.0028254949,0.0015832181,-0.009601558],[-0.004810024,-0.016183112,0.0040136,-0.0047758417,-0.013919562,-0.001458656,-0.0056018066,-0.04050731],[-0.0031044558,0.0050800275,-0.039173402,0.0,0.0,0.00879043,-0.054703448,0.017016849],[-0.006560567,-0.0039063953,-0.00807818,0.0,0.0,0.013087283,-0.00039338693,-0.0031795185],[-0.009089982,-0.012925392,-0.052080773,0.011012092,0.8568386,-0.006076536,-0.073285274,-0.012861287],[-0.005904807,0.001267083,0.017482745,-0.039431967,-0.091770865,-0.1053706,0.009871405,0.0028513884],[-0.0016581342,0.0021751747,-0.07519592,0.01201967,-0.06390632,0.011341225,-0.11185053,-0.017531388]],\"type\":\"heatmap\",\"xaxis\":\"x6\",\"yaxis\":\"y6\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"6\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.005807448,0.0041784383,0.002361197,0.0036581755,0.002977576,-0.010813372,-0.004694109,-0.03043549],[-0.0065663815,0.002266677,-0.00443942,0.00861506,0.0011802483,0.003772121,-0.016978573,-0.0019876473],[-0.012904956,0.00036069006,-0.022571078,0.00084973406,-0.003605105,0.0018587941,0.009754168,-0.005520938],[-0.044007286,0.005468961,-0.0065602316,0.0,0.0,-0.010466896,0.006586544,-0.00072280224],[0.02505619,0.00399052,0.015117939,0.0,0.0,-0.02064427,-0.014979721,-0.011082727],[-0.08102531,-0.03313881,0.7956875,0.040076405,-0.023882367,-0.012472488,-0.021385413,-0.010897325],[0.013410876,-0.17502096,-0.031515226,-0.018688247,-0.007924046,-0.0124051925,0.0006804392,0.0036436096],[-0.14003202,-0.006057922,-0.1000561,0.010304868,-0.043900337,0.002680806,-0.009554791,-0.00748358]],\"type\":\"heatmap\",\"xaxis\":\"x7\",\"yaxis\":\"y7\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"7\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.008184929,-0.0071411263,-0.027213048,-0.008899916,0.0018404583,-0.004163511,0.016239606,-0.019566279],[-0.010428973,0.0015035719,-0.015855798,-0.06870228,0.004106216,-0.060434043,0.0001135692,-0.11120443],[-0.0034291362,0.0060438924,0.015591694,0.006360621,-0.0045243036,-0.017385982,-0.06279722,0.00047923392],[-0.023051085,-0.023127053,-0.014275543,0.0,0.0,0.855556,-0.08713405,-0.06840275],[-0.010609349,-0.00024539325,0.0064846054,0.0,0.0,0.0072070537,-0.033660825,0.021314746],[-0.00056535006,-0.0037950529,0.0053725243,-0.039839424,0.010920713,-0.0361443,0.012530863,-0.09926042],[0.00039813574,-0.019555494,-0.03265757,0.006312022,0.0027903542,-0.021889532,0.0015363535,-0.015596569],[-0.008846091,-0.030749027,-0.013803504,0.0067384136,-0.007460149,-0.007838082,-0.013475258,-0.0049915127]],\"type\":\"heatmap\",\"xaxis\":\"x8\",\"yaxis\":\"y8\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"8\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.014488206,-0.10013901,0.0061944705,-0.0535686,0.023511682,-0.06508359,-0.008047436,-0.009098489],[0.01527793,0.014766537,-0.093777925,-0.073621996,-0.04601443,0.0036579818,0.023968622,-0.00051391404],[-0.048909813,-0.08659883,-0.017425152,0.85348135,-0.0074326126,-0.044173222,0.00918221,-0.007857677],[-0.0034038574,0.006291572,-0.010005927,0.0,0.0,-0.0079794,0.000328172,-0.014017347],[-0.014708805,-0.054069173,-0.0048635453,0.0,0.0,-0.016675808,0.0016546724,-0.00011344999],[-0.012256775,-0.005513858,-0.0027599018,-0.0046429094,-0.009107886,-0.0143029895,-0.01305522,0.0013755895],[-0.016131327,0.005967822,-8.754432e-7,-0.007296648,0.0016682008,-0.010747386,-0.0055030454,-0.015105978],[-0.011330182,-0.00401558,0.007840027,-0.0069483686,-0.008816554,-0.014103037,-0.009864841,-0.013199145]],\"type\":\"heatmap\",\"xaxis\":\"x9\",\"yaxis\":\"y9\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"9\",\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.16455744,-0.0014210213,-0.076401345,0.00040844176,-0.0448257,-0.0040189177,0.006796824,-0.014169203],[0.0005808361,-0.19843584,-0.066967666,-0.016382748,0.0049476493,0.015409451,0.005167838,-0.005988829],[-0.09083254,-0.04659662,0.8066412,0.023849297,-0.025260538,-0.0086166095,-0.002689205,-0.014920908],[0.021208506,-0.0013505798,0.0375897,0.0,0.0,-0.0052790856,0.003249595,-0.004403595],[-0.046130326,0.008947548,-0.016688846,0.0,0.0,-0.024639253,0.0048375754,-0.0017691655],[-0.007645351,0.01628305,0.005874088,-0.0004941989,-0.0040920377,-0.017163688,0.016042607,-0.0065116864],[-0.0015605898,-0.007235947,-0.021186197,0.0065474343,0.004227399,-0.0034908867,-0.00082322955,0.009684674],[-0.008186698,-0.0007166499,-0.021281317,-0.013197369,0.0015823739,0.0005637221,0.0013554422,-0.026415791]],\"type\":\"heatmap\",\"xaxis\":\"x10\",\"yaxis\":\"y10\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,0.08200000000000002],\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"title\":{\"text\":\"y\"}},\"xaxis2\":{\"anchor\":\"y2\",\"domain\":[0.10200000000000002,0.18400000000000005],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis2\":{\"anchor\":\"x2\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis3\":{\"anchor\":\"y3\",\"domain\":[0.20400000000000004,0.28600000000000003],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis3\":{\"anchor\":\"x3\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis4\":{\"anchor\":\"y4\",\"domain\":[0.30600000000000005,0.38800000000000007],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis4\":{\"anchor\":\"x4\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis5\":{\"anchor\":\"y5\",\"domain\":[0.4080000000000001,0.4900000000000001],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis5\":{\"anchor\":\"x5\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis6\":{\"anchor\":\"y6\",\"domain\":[0.5100000000000001,0.5920000000000001],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis6\":{\"anchor\":\"x6\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis7\":{\"anchor\":\"y7\",\"domain\":[0.6120000000000001,0.6940000000000002],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis7\":{\"anchor\":\"x7\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis8\":{\"anchor\":\"y8\",\"domain\":[0.7140000000000001,0.796],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis8\":{\"anchor\":\"x8\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis9\":{\"anchor\":\"y9\",\"domain\":[0.8160000000000002,0.8980000000000001],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis9\":{\"anchor\":\"x9\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis10\":{\"anchor\":\"y10\",\"domain\":[0.9180000000000001,1.0],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis10\":{\"anchor\":\"x10\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"annotations\":[{\"font\":{},\"showarrow\":false,\"text\":\"L4N547\",\"x\":0.04100000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N1122\",\"x\":0.14300000000000004,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N691\",\"x\":0.24500000000000005,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N352\",\"x\":0.3470000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N345\",\"x\":0.44900000000000007,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N1076\",\"x\":0.5510000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N112\",\"x\":0.6530000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N115\",\"x\":0.7550000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N1285\",\"x\":0.8570000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N1882\",\"x\":0.9590000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"}],\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"title\":{\"text\":\"Cosine sim of Output weights and the unembed for top layer 4 neurons\"}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('1de5ee6b-12c8-4745-a124-1c23c09f8e7d');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "layer = 4\n",
    "top_layer_4_neurons = focus_cache[\"post\", layer][:, 3:-3].std(dim=[0, 1]).argsort(descending=True)[:10]\n",
    "W_U_norm = model.W_U / model.W_U.norm(dim=0, keepdim=True)\n",
    "W_U_norm = W_U_norm[:, 1:] # Get rid of the passing/dummy first element\n",
    "heatmaps_unembed = []\n",
    "for neuron in top_layer_4_neurons:\n",
    "    neuron = neuron.item()\n",
    "    w_out = model.blocks[layer].mlp.W_out[neuron, :].detach()\n",
    "    w_out /= w_out.norm()\n",
    "    state = torch.zeros(64, device=\"cuda\")\n",
    "    state[stoi_indices] = w_out @ W_U_norm\n",
    "    heatmaps_unembed.append(state.reshape(8, 8))\n",
    "imshow(heatmaps_unembed,\n",
    "    facet_col=0,\n",
    "    y=[i for i in \"ABCDEFGH\"],\n",
    "    title=f\"Cosine sim of Output weights and the unembed for top layer 4 neurons\",\n",
    "    facet_labels=[f\"L4N{n.item()}\" for n in top_layer_4_neurons])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Activation Patching"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A valuable technique for tracking down the action of various circuits is activation patching. Check out https://neelnanda.io/exploratory-analysis-demo for more details, but here's a quick demo of using it on Othello.\n",
    "\n",
    "The key idea is to set up a clean and corrupted input that differ in one key detail, and to **patch** a single clean activation into the corrupted run (ie replacing that activation with its value on the clean run, eg the output of an attention head), and to look at this patch's effect on a metric that depends on that detail. By carefully choosing clean and corrupted inputs that differ in one key detail, we can isolate which model components capture and depend on this detail."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Setting up clean and corrupted moves tha tresult in similar games is non-trivial, so here we take a game and just change the most recent move from E0 to C0. This turns F0 from legal to illegal, so let's make that logit our patching metric"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"8e0a9224-8d9e-4f1c-9449-15103c8712de\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"8e0a9224-8d9e-4f1c-9449-15103c8712de\")) {                    Plotly.newPlot(                        \"8e0a9224-8d9e-4f1c-9449-15103c8712de\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.0,0.1,0.1,0.1,0.1,0.1,0.1,0.1],[-1.0,0.1,-1.0,-1.0,-1.0,-1.0,-1.0,0.0],[0.1,-1.0,-1.0,-1.0,1.0,1.0,1.0,-0.5],[0.1,0.1,-1.0,-1.0,1.0,-0.5,-0.5,-0.5],[0.9,0.9,0.9,0.9,1.0,1.0,0.0,0.0],[-0.5,-0.5,1.0,-0.5,1.0,1.0,1.0,0.0],[0.0,-0.5,-0.5,0.0,-0.5,-0.5,-0.5,-0.5],[0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"<b>%{y}%{x}</b><br>%{customdata}<extra></extra>\",\"text\":[[\"A0\",\"A1\",\"A2\",\"A3\",\"A4\",\"A5\",\"A6\",\"A7\"],[\"\",\"B1\",\"\",\"\",\"\",\"\",\"\",\"B7\"],[\"C0\",\"\",\"\",\"\",\"\",\"\",\"\",\"C7\"],[\"D0\",\"D1\",\"\",\"\",\"\",\"D5\",\"D6\",\"D7\"],[\"\",\"\",\"\",\"\",\"\",\"\",\"E6\",\"E7\"],[\"F0\",\"F1\",\"\",\"F3\",\"\",\"\",\"\",\"F7\"],[\"G0\",\"G1\",\"G2\",\"G3\",\"G4\",\"G5\",\"G6\",\"G7\"],[\"H0\",\"H1\",\"H2\",\"H3\",\"H4\",\"H5\",\"H6\",\"H7\"]],\"texttemplate\":\"%{text}\",\"customdata\":[[\"I:1<br>S:0\",\"I:2<br>S:1\",\"I:3<br>S:2\",\"I:4<br>S:3\",\"I:5<br>S:4\",\"I:6<br>S:5\",\"I:7<br>S:6\",\"I:8<br>S:7\"],[\"I:9<br>S:8\",\"I:10<br>S:9\",\"I:11<br>S:10\",\"I:12<br>S:11\",\"I:13<br>S:12\",\"I:14<br>S:13\",\"I:15<br>S:14\",\"I:16<br>S:15\"],[\"I:17<br>S:16\",\"I:18<br>S:17\",\"I:19<br>S:18\",\"I:20<br>S:19\",\"I:21<br>S:20\",\"I:22<br>S:21\",\"I:23<br>S:22\",\"I:24<br>S:23\"],[\"I:25<br>S:24\",\"I:26<br>S:25\",\"I:27<br>S:26\",\"I:X<br>S:27\",\"I:X<br>S:28\",\"I:28<br>S:29\",\"I:29<br>S:30\",\"I:30<br>S:31\"],[\"I:31<br>S:32\",\"I:32<br>S:33\",\"I:33<br>S:34\",\"I:X<br>S:35\",\"I:X<br>S:36\",\"I:34<br>S:37\",\"I:35<br>S:38\",\"I:36<br>S:39\"],[\"I:37<br>S:40\",\"I:38<br>S:41\",\"I:39<br>S:42\",\"I:40<br>S:43\",\"I:41<br>S:44\",\"I:42<br>S:45\",\"I:43<br>S:46\",\"I:44<br>S:47\"],[\"I:45<br>S:48\",\"I:46<br>S:49\",\"I:47<br>S:50\",\"I:48<br>S:51\",\"I:49<br>S:52\",\"I:50<br>S:53\",\"I:51<br>S:54\",\"I:52<br>S:55\"],[\"I:53<br>S:56\",\"I:54<br>S:57\",\"I:55<br>S:58\",\"I:56<br>S:59\",\"I:57<br>S:60\",\"I:58<br>S:61\",\"I:59<br>S:62\",\"I:60<br>S:63\"]]}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(0, 128, 128)\"],[0.16666666666666666,\"rgb(112, 164, 148)\"],[0.3333333333333333,\"rgb(180, 200, 168)\"],[0.5,\"rgb(246, 237, 189)\"],[0.6666666666666666,\"rgb(237, 187, 138)\"],[0.8333333333333334,\"rgb(222, 138, 90)\"],[1.0,\"rgb(202, 86, 44)\"]],\"cmid\":0.0},\"title\":{\"text\":\"Original Game (black plays E0)\",\"x\":0.5},\"shapes\":[{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":-0.2,\"x1\":0.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":0.8,\"x1\":1.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":-0.25,\"x1\":0.25,\"y0\":3.75,\"y1\":4.25,\"line\":{\"color\":\"red\",\"width\":4}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":0.78,\"x1\":1.22,\"y0\":3.78,\"y1\":4.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":1.78,\"x1\":2.22,\"y0\":3.78,\"y1\":4.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.78,\"x1\":3.22,\"y0\":3.78,\"y1\":4.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}}]},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('8e0a9224-8d9e-4f1c-9449-15103c8712de');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"ea63387d-9834-4668-8422-f5ddc2c43fe5\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"ea63387d-9834-4668-8422-f5ddc2c43fe5\")) {                    Plotly.newPlot(                        \"ea63387d-9834-4668-8422-f5ddc2c43fe5\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.0,0.1,0.1,0.1,0.1,0.1,0.1,0.1],[-1.0,-0.5,-1.0,-1.0,-1.0,-1.0,-1.0,0.0],[0.9,0.9,0.9,0.9,1.0,1.0,1.0,0.0],[-0.5,-0.5,-1.0,-1.0,1.0,-0.5,-0.5,-0.5],[0.1,-1.0,-1.0,-1.0,1.0,1.0,-0.5,0.0],[0.0,0.1,1.0,0.0,1.0,1.0,1.0,0.0],[0.0,-0.5,-0.5,-0.5,-0.5,-0.5,-0.5,-0.5],[0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"<b>%{y}%{x}</b><br>%{customdata}<extra></extra>\",\"text\":[[\"A0\",\"A1\",\"A2\",\"A3\",\"A4\",\"A5\",\"A6\",\"A7\"],[\"\",\"B1\",\"\",\"\",\"\",\"\",\"\",\"B7\"],[\"\",\"\",\"\",\"\",\"\",\"\",\"\",\"C7\"],[\"D0\",\"D1\",\"\",\"\",\"\",\"D5\",\"D6\",\"D7\"],[\"E0\",\"\",\"\",\"\",\"\",\"\",\"E6\",\"E7\"],[\"F0\",\"F1\",\"\",\"F3\",\"\",\"\",\"\",\"F7\"],[\"G0\",\"G1\",\"G2\",\"G3\",\"G4\",\"G5\",\"G6\",\"G7\"],[\"H0\",\"H1\",\"H2\",\"H3\",\"H4\",\"H5\",\"H6\",\"H7\"]],\"texttemplate\":\"%{text}\",\"customdata\":[[\"I:1<br>S:0\",\"I:2<br>S:1\",\"I:3<br>S:2\",\"I:4<br>S:3\",\"I:5<br>S:4\",\"I:6<br>S:5\",\"I:7<br>S:6\",\"I:8<br>S:7\"],[\"I:9<br>S:8\",\"I:10<br>S:9\",\"I:11<br>S:10\",\"I:12<br>S:11\",\"I:13<br>S:12\",\"I:14<br>S:13\",\"I:15<br>S:14\",\"I:16<br>S:15\"],[\"I:17<br>S:16\",\"I:18<br>S:17\",\"I:19<br>S:18\",\"I:20<br>S:19\",\"I:21<br>S:20\",\"I:22<br>S:21\",\"I:23<br>S:22\",\"I:24<br>S:23\"],[\"I:25<br>S:24\",\"I:26<br>S:25\",\"I:27<br>S:26\",\"I:X<br>S:27\",\"I:X<br>S:28\",\"I:28<br>S:29\",\"I:29<br>S:30\",\"I:30<br>S:31\"],[\"I:31<br>S:32\",\"I:32<br>S:33\",\"I:33<br>S:34\",\"I:X<br>S:35\",\"I:X<br>S:36\",\"I:34<br>S:37\",\"I:35<br>S:38\",\"I:36<br>S:39\"],[\"I:37<br>S:40\",\"I:38<br>S:41\",\"I:39<br>S:42\",\"I:40<br>S:43\",\"I:41<br>S:44\",\"I:42<br>S:45\",\"I:43<br>S:46\",\"I:44<br>S:47\"],[\"I:45<br>S:48\",\"I:46<br>S:49\",\"I:47<br>S:50\",\"I:48<br>S:51\",\"I:49<br>S:52\",\"I:50<br>S:53\",\"I:51<br>S:54\",\"I:52<br>S:55\"],[\"I:53<br>S:56\",\"I:54<br>S:57\",\"I:55<br>S:58\",\"I:56<br>S:59\",\"I:57<br>S:60\",\"I:58<br>S:61\",\"I:59<br>S:62\",\"I:60<br>S:63\"]]}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(0, 128, 128)\"],[0.16666666666666666,\"rgb(112, 164, 148)\"],[0.3333333333333333,\"rgb(180, 200, 168)\"],[0.5,\"rgb(246, 237, 189)\"],[0.6666666666666666,\"rgb(237, 187, 138)\"],[0.8333333333333334,\"rgb(222, 138, 90)\"],[1.0,\"rgb(202, 86, 44)\"]],\"cmid\":0.0},\"title\":{\"text\":\"Corrupted Game (blank plays C0)\",\"x\":0.5},\"shapes\":[{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":-0.2,\"x1\":0.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":-0.25,\"x1\":0.25,\"y0\":1.75,\"y1\":2.25,\"line\":{\"color\":\"red\",\"width\":4}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":0.78,\"x1\":1.22,\"y0\":1.78,\"y1\":2.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":1.78,\"x1\":2.22,\"y0\":1.78,\"y1\":2.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.78,\"x1\":3.22,\"y0\":1.78,\"y1\":2.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":0.8,\"x1\":1.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}}]},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('ea63387d-9834-4668-8422-f5ddc2c43fe5');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "game_index = 4\n",
    "move = 20\n",
    "plot_single_board(focus_games_string[game_index, :move+1], title=\"Original Game (black plays E0)\")\n",
    "plot_single_board(focus_games_string[game_index, :move].tolist()+[16], title=\"Corrupted Game (blank plays C0)\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Clean ['C3', 'C4', 'C5', 'E2', 'F2', 'C2', 'D2', 'E1', 'F5', 'B6', 'C6', 'B5', 'C1', 'B0', 'B4', 'E5', 'F6', 'B2', 'F4', 'B3', 'C0']\n",
      "Corrupted ['C3', 'C4', 'C5', 'E2', 'F2', 'C2', 'D2', 'E1', 'F5', 'B6', 'C6', 'B5', 'C1', 'B0', 'B4', 'E5', 'F6', 'B2', 'F4', 'B3', 'E0']\n"
     ]
    }
   ],
   "source": [
    "clean_input = focus_games_int[game_index, :move+1].clone()\n",
    "corrupted_input = focus_games_int[game_index, :move+1].clone()\n",
    "corrupted_input[-1] = to_int(\"C0\")\n",
    "print(\"Clean\", int_to_label(corrupted_input))\n",
    "print(\"Corrupted\", int_to_label(clean_input))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "clean_logits, clean_cache = model.run_with_cache(clean_input)\n",
    "corrupted_logits, corrupted_cache = model.run_with_cache(corrupted_input)\n",
    "\n",
    "clean_log_probs = clean_logits.log_softmax(dim=-1)\n",
    "corrupted_log_probs = corrupted_logits.log_softmax(dim=-1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's patch in each attention layer in turn and analyse the effect on the logit of F0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Clean log prob tensor(-2.5273, device='cuda:0', grad_fn=<SelectBackward0>)\n",
      "Corrupted log prob tensor(-12.0080, device='cuda:0', grad_fn=<SelectBackward0>)\n",
      "Clean metric tensor(1., device='cuda:0', grad_fn=<DivBackward0>)\n",
      "Corrupted metric tensor(0., device='cuda:0', grad_fn=<DivBackward0>)\n"
     ]
    }
   ],
   "source": [
    "f0_index = to_int(\"F0\")\n",
    "clean_f0_log_prob = clean_log_probs[0, -1, f0_index]\n",
    "corrupted_f0_log_prob = corrupted_log_probs[0, -1, f0_index]\n",
    "print(\"Clean log prob\", clean_f0_log_prob)\n",
    "print(\"Corrupted log prob\", corrupted_f0_log_prob)\n",
    "\n",
    "def patching_metric(patched_logits):\n",
    "    # patched_log_probs.shape is [1, 21, 61]\n",
    "    patched_log_probs = patched_logits.log_softmax(dim=-1)\n",
    "    return (patched_log_probs[0, -1, f0_index] - corrupted_f0_log_prob)/(clean_f0_log_prob - corrupted_f0_log_prob)\n",
    "print(\"Clean metric\", patching_metric(clean_logits))\n",
    "print(\"Corrupted metric\", patching_metric(corrupted_logits))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"b7007a44-9cc5-4b73-81fd-2e2ff0a0ccba\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"b7007a44-9cc5-4b73-81fd-2e2ff0a0ccba\")) {                    Plotly.newPlot(                        \"b7007a44-9cc5-4b73-81fd-2e2ff0a0ccba\",                        [{\"hovertemplate\":\"Color=attn<br>x=%{x}<br>y=%{y}<extra></extra>\",\"legendgroup\":\"0\",\"line\":{\"color\":\"#636efa\",\"dash\":\"solid\"},\"marker\":{\"symbol\":\"circle\"},\"mode\":\"lines\",\"name\":\"attn\",\"orientation\":\"v\",\"showlegend\":true,\"x\":[0,1,2,3,4,5,6,7],\"xaxis\":\"x\",\"y\":[0.0025119504425674677,0.0016299713170155883,0.002774391556158662,0.0030779740773141384,0.0007470869459211826,0.004995232447981834,0.018300563097000122,0.26133087277412415],\"yaxis\":\"y\",\"type\":\"scatter\"},{\"hovertemplate\":\"Color=mlp<br>x=%{x}<br>y=%{y}<extra></extra>\",\"legendgroup\":\"1\",\"line\":{\"color\":\"#EF553B\",\"dash\":\"solid\"},\"marker\":{\"symbol\":\"circle\"},\"mode\":\"lines\",\"name\":\"mlp\",\"orientation\":\"v\",\"showlegend\":true,\"x\":[0,1,2,3,4,5,6,7],\"xaxis\":\"x\",\"y\":[0.7576194405555725,-0.00873619969934225,0.006583458743989468,0.007611696608364582,0.027243074029684067,0.6297826170921326,0.6275081038475037,0.015118979848921299],\"yaxis\":\"y\",\"type\":\"scatter\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"y\"}},\"legend\":{\"title\":{\"text\":\"Color\"},\"tracegroupgap\":0},\"title\":{\"text\":\"Layer Output Patching Effect on F0 Log Prob\"}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('b7007a44-9cc5-4b73-81fd-2e2ff0a0ccba');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "attn_layer_patches = []\n",
    "def patch_attn_layer_output(attn_out, hook, layer):\n",
    "    # Only patch in on the final move, prior moves are identical\n",
    "    attn_out[0, -1, :] = clean_cache[\"attn_out\", layer][0, -1, :]\n",
    "    return attn_out\n",
    "for layer in range(8):\n",
    "    patched_logits = model.run_with_hooks(corrupted_input, fwd_hooks=[(utils.get_act_name(\"attn_out\", layer), partial(patch_attn_layer_output, layer=layer))])\n",
    "    attn_layer_patches.append(patching_metric(patched_logits).item())\n",
    "\n",
    "mlp_layer_patches = []\n",
    "def patch_mlp_layer_output(mlp_out, hook, layer):\n",
    "    # Only patch in on the final move, prior moves are identical\n",
    "    mlp_out[0, -1, :] = clean_cache[\"mlp_out\", layer][0, -1, :]\n",
    "    return mlp_out\n",
    "for layer in range(8):\n",
    "    patched_logits = model.run_with_hooks(corrupted_input, fwd_hooks=[(utils.get_act_name(\"mlp_out\", layer), partial(patch_mlp_layer_output, layer=layer))])\n",
    "    mlp_layer_patches.append(patching_metric(patched_logits).item())\n",
    "line([attn_layer_patches, mlp_layer_patches], title=\"Layer Output Patching Effect on F0 Log Prob\", line_labels=[\"attn\", \"mlp\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that most layers just don't matter! But MLP0, MLP5, MLP6 and Attn7 do! My next steps would be to get more fine grained and to patch in individual neurons and see how far I can zoom in on *why* those layers matter - ideally at the level of understanding how these neurons compose with each other and the changed the embedding, using the fact that most of the model just doesn't matter here. And then to compare this data to the above techniques for understanding neurons.\n",
    "\n",
    "It's not surprising that the attention layers are fairly unimportant - attention specialises in moving information between token positions, we've only changed the information at the current position!"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Analysing Neuron L5N1393"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To practice neuron interpretability, let's take a deep dive into understanding this neuron - the techniques and code should transfer pretty well to any other neurons!\n",
    "\n",
    "The spirit of this section is to practice doing various standard things that you could go and apply to another neuron in practice - I end it still being fairly confused, and with many dangling threads!"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As above, we can use the probe weights to analyse its input weights. So we go in with the hypothesis that it detects C0==BLANK & D1==THEIR'S & E2==MINE, and that it directly boosts the C0 logit - this configuration means that C0 must be legal, because it'd flank D1 along the diagonal!"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also take the direct logit attribution of its output weights - ie w_out @ W_U, to see if the neuron significantly affects the output logits directly.\n",
    "\n",
    "Note that this is purely weights based analysis, we have not yet run the model and looked at any activations!\n",
    "\n",
    "Here we immediately see that boosting C0 is an important part of what happens, but that it also boosts D1, which is surprising! (D1 is filled in our hypothesis, so must be illegal!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"a87ac490-b29f-4d0c-be9e-99c9a6185b44\" class=\"plotly-graph-div\" style=\"height:525px; width:600px;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"a87ac490-b29f-4d0c-be9e-99c9a6185b44\")) {                    Plotly.newPlot(                        \"a87ac490-b29f-4d0c-be9e-99c9a6185b44\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.011189356,0.0012445413,-0.017113198,-0.017581677,-0.04277996,0.0013973974,-0.025884487,-0.02009257],[0.034074,-0.013575641,0.00008581113,-0.013220519,0.014656193,-0.0047652256,-0.021692311,-0.016338635],[0.73715395,-0.043972224,-0.05461748,-0.026704358,-0.053631183,-0.01118125,-0.015117623,-0.013709186],[-0.036555126,0.21900256,-0.013638377,0.0,0.0,-0.008704931,-0.029098887,-0.022870077],[-0.032188945,-0.05747528,0.03883893,0.0,0.0,-0.033800896,0.022317573,0.0067806942],[-0.014518,-0.023814328,-0.01773491,-0.03390853,-0.0148164015,-0.050225735,-0.010327473,-0.010885349],[-0.00060591474,0.008213079,-0.041887034,-0.050493263,-0.07484389,-0.008553635,0.010127753,-0.007859532],[-0.001995314,-0.03548369,-0.002260156,-0.022177152,-0.048210528,0.019773483,-0.012859665,-0.010488018]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"title\":{\"text\":\"Output weights of Neuron L5N1393 in the output logit basis\"},\"width\":600},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('a87ac490-b29f-4d0c-be9e-99c9a6185b44');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "layer = 5\n",
    "neuron = 1393\n",
    "w_out = model.blocks[layer].mlp.W_out[neuron, :]\n",
    "state = torch.zeros(8, 8, device=\"cuda\")\n",
    "state.flatten()[stoi_indices] = w_out @ model.W_U[:, 1:]\n",
    "plot_square_as_board(state, title=f\"Output weights of Neuron L{layer}N{neuron} in the output logit basis\", width=600)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One hypothesis is that the unembed for C0 and D1 are highly aligned, so it can't easily boost one but not the other. But this is false, they're almost orthogonal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cosine sim of C0 and D1 unembeds tensor(-0.0221, device='cuda:0')\n"
     ]
    }
   ],
   "source": [
    "c0_U = model.W_U[:, 17].detach()\n",
    "c0_U /= c0_U.norm()\n",
    "d1_U = model.W_U[:, 26].detach()\n",
    "d1_U /= d1_U.norm()\n",
    "print(\"Cosine sim of C0 and D1 unembeds\", c0_U @ d1_U)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To check that this is a *big* part of the neuron's output, lets look at the fraction of variance of the output captured by the W_U subspace - we see that it's only 28.6%, so something else is probably going on too! (Maybe some dropout related backup circuits? Or just assisting in further processing)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fraction of variance captured by W_U 0.2868001385215848\n"
     ]
    }
   ],
   "source": [
    "# Remove the 0th element, which never comes up\n",
    "U, S, Vh = torch.svd(model.W_U[:, 1:])\n",
    "print(\"Fraction of variance captured by W_U\", (w_out @ U).norm().item()**2)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another quick sanity check is just plotting what the neuron activations look like over the 50 games in our cache - we see that it only matters in a few games, and matters every other move (which makes sense for a feature denominated in terms of my vs their colours - this alternates every move as the colours swap!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"d557dc6a-3969-40a6-8af5-f0d333259b2a\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"d557dc6a-3969-40a6-8af5-f0d333259b2a\")) {                    Plotly.newPlot(                        \"d557dc6a-3969-40a6-8af5-f0d333259b2a\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"z\":[[-0.09947495,-0.040831033,-0.037849583,-0.023504568,-0.060453746,-0.04447717,-0.02246056,-0.042769313,-0.023498943,-0.06672587,-0.016750956,-0.0653936,-0.05243246,-0.0153404735,-0.012743145,-0.014565279,-0.000035315188,0.02176557,-0.00004870125,-0.09096778,-0.00005358667,-0.015000912,-0.00002148604,-0.06124259,-0.000056667785,-0.10174236,-0.000009992171,0.039100736,-0.000078785706,-0.050347764,-0.00001618126,-0.0188046,-0.0000028978138,-0.057895,-0.000009211622,-0.16406053,-0.000031896998,-0.07622033,-0.16765757,-0.064052664,-0.10899986,-0.10235441,-0.0017367125,-0.15556453,-0.00008456587,-0.006621796,-0.00016569506,-0.004885266,-0.010269173,-0.012632964,-0.00016051448,-0.07012375,-0.0005644457,-0.093222685,-0.00047811365,-0.018967744,-0.000052636795,-0.036836937,-0.000011947191],[-0.088697106,-0.019822182,-0.0033522772,-0.0030721256,-0.0048747044,-0.0031896373,-0.002478962,-0.0022570249,-0.009488703,-0.0024804175,-0.01383641,-0.00029102017,-0.030754764,-0.0005813641,-0.15559198,-0.00022794453,-0.15887484,-0.001608251,-0.14634195,-0.011292094,-0.01155883,-0.00070509716,-0.16946349,-0.0010542911,-0.0021517004,-0.13086386,-0.0011268628,-0.06242106,-0.014053808,-0.15610488,-0.000886677,-0.117572986,-0.00008040692,-0.0055612247,-0.000037385482,-0.001174082,-0.000003996394,-3.1806508e-7,-0.0020229234,-1.6302816e-7,-0.0039983913,-0.0000014882303,-0.0065066833,-0.0000019155323,-0.018927317,-0.0000035850646,-0.07436839,-0.00036182525,-0.106396474,-0.006137173,-0.0077131526,-0.0044204644,-0.15375866,-0.0,-0.041341413,-0.0000035851442,-0.0000092069895,-0.0523445,-0.0035900138],[-0.08975199,-0.020782711,-0.0037444478,-0.005707193,-0.003451436,-0.006198232,-0.008332663,-0.0017458046,-0.018767258,-0.011044573,-0.009210604,-0.01447473,-0.0056830207,-0.0028143928,-0.008875354,-0.0025167374,-0.16222815,-0.00550022,-0.017108873,-0.0012299973,-0.15453492,-0.054151062,-0.00022676632,-0.08915375,-0.00016777783,-0.06667322,-0.00003152865,-0.05642063,-0.00025989735,-0.0045698644,-0.0003568515,-0.009877288,-0.15038686,-0.000023988394,-0.15549262,-0.0005248493,-0.16179755,-0.009806105,-0.0015394908,-0.001748916,-0.120008536,-0.011105473,-0.14771627,-0.0046663336,-0.0025900486,-0.007086617,-0.0024707494,-0.014672591,-0.053746138,-0.010724822,-0.09634537,-0.01840554,-0.15527561,-0.000006559177,-0.1363934,-0.0000019125603,-0.0020538254,-0.0030633793,-0.0032155723],[-0.10032095,-0.0053279535,-0.0046379613,0.011868197,-0.00027167943,-0.0975912,-0.0010586068,-0.028423429,-0.00043863198,-0.07187349,-0.0010614281,-0.072887905,-0.000093867755,-0.1559153,-0.0000096004,-0.16019966,-0.000049534727,-0.000025978155,-0.15167929,-0.00008905076,-0.0008739592,-0.16959728,-0.000065032764,-0.00024924934,-0.16949104,-0.00018377756,-0.00074727944,-0.12233323,-0.000025975773,-0.16995594,-0.0002646794,-0.117633194,-0.00012447464,1.4519229,-9.1315576e-7,-0.1695985,-0.000009077656,-0.015866794,-4.6619783e-7,-0.10913472,-0.0,0.17150857,-0.0,-0.035734586,-1.6124491e-7,-0.16918676,-6.1514834e-7,-0.16651039,-0.0001111732,-0.16277768,-4.6701442e-7,-0.12446429,-0.0000027559488,-0.13116889,-0.000002478863,-0.07330663,-0.0,-0.13466315,-7.654286e-7],[-0.09947495,-0.040831033,-0.06825675,-0.0011155673,0.27651453,-0.001508019,-0.0006925826,-0.0008660091,-0.08245647,-0.00024700744,0.17673475,-0.00066612335,-0.15035933,-0.00020595003,0.07495918,-0.000024236384,0.06156495,-0.00086869515,0.08018734,-0.0056906687,-0.025794605,-0.0021563098,0.33807248,-0.00044300812,-0.009882949,-0.16688414,-0.0036782888,-0.0043170396,-0.005081648,-0.0018287104,-0.00682513,-0.01698006,-0.1461922,-0.004757091,-0.13034126,-0.0020103902,-0.15683809,-0.0000050862373,-0.16896284,-0.000037869122,-0.1680141,-0.00032960527,-0.0012132453,-0.017719567,-0.0001455753,-0.012561742,-0.00058116956,-0.000003587303,-0.013603004,-0.000058322923,-0.07708345,-0.0012043485,-0.036065448,-0.0004131979,-0.07888808,-0.00075174146,-0.036166135,-0.00016986893,-0.030027464],[-0.09947495,-0.0060757813,0.029397575,-0.0034576708,-0.012195389,-0.0051390165,-0.04663686,-0.0008003429,-0.0013193907,-0.0010547568,2.6667776,-0.00034938313,2.376906,-0.001722802,2.9171948,-0.00026701318,2.5927224,-0.00004989358,2.0756793,-0.00079164817,1.9980229,-0.16136254,-0.08670277,-0.109726,-0.00026754444,0.8327748,-0.023354767,-0.0052162088,-0.042227425,-0.055610985,-0.024672465,-0.019077055,-0.002277857,-0.000002337399,-0.019861868,-0.000055366178,-0.0141087705,-0.000063269305,-0.020435283,-0.00020876675,-0.03384367,-0.00058086484,-0.035304677,-0.00042026196,-0.0378594,-0.000057137535,-0.029447122,-0.00048711384,-0.037421938,-0.00011298232,-0.012111121,-0.00009100642,-0.009225881,-0.000007761249,-0.03644084,-0.00036534076,-0.06026551,-0.000057023102,-0.014593013],[-0.08975199,-0.03872189,-0.014372508,-0.024228742,-0.03265205,-0.04506545,-0.04214962,-0.005212026,0.3190384,-0.0018070687,0.40726358,-0.0076365154,0.4633241,-0.0043783747,0.07978141,-0.0016774083,0.013120652,-0.0041353595,-0.054434456,-0.0060171084,0.12997404,-0.015828634,0.006762783,-0.021250954,-0.16713385,-0.0023430833,-0.0505122,-0.0010523144,-0.0028798264,-0.14695948,-0.0039200727,-0.16773155,-0.0018512959,-0.0016874247,-0.16990618,-0.00026117216,-0.16737704,-0.00045947474,0.586615,-0.001699316,-0.08068826,-0.0028068093,-0.04886126,-0.0009628352,-0.028865485,-0.0013762953,-0.031045754,-0.0007582455,-0.03232438,-0.0007973501,-0.054999493,-0.0046938886,-0.13410458,-0.0,-0.13662235,-0.0,-0.16037358,-0.0,-0.06853787],[-0.088697106,-0.019822182,-0.020223469,-0.0011384329,-0.03035976,-0.0039087753,-0.0066286707,-0.0005668348,-0.018559018,-0.0031122447,-0.038635876,-0.0064580124,-0.011438556,-0.009808344,-0.02093996,-0.02484352,-0.0141234435,-0.022733968,-0.005255952,-0.020322587,-0.009152696,-0.020724976,-0.0056967945,-0.018471766,-0.026432054,-0.017447865,-0.026995072,-0.00985974,-0.008802239,-0.010277573,-0.005639374,-0.0063947756,-0.02059329,-0.0039013491,-0.01585207,-0.020959297,-0.0264031,-0.00754834,-0.04409525,-0.003417189,-0.03799042,-0.016010286,0.108561404,-0.00009019904,-0.08837361,-0.0044837254,0.048309892,-0.005502836,-0.103345074,-0.16616967,-0.010415258,0.057036925,-0.039453838,0.04699971,-0.07621567,-0.094898455,-0.0757508,-0.13682917,-0.0033694035],[-0.10032095,-0.03138299,-0.0072553204,-0.0027623638,-0.0009533447,-0.0011673953,-0.001199155,-0.002263339,-0.0019415118,-0.16921498,-0.0095827235,0.028447771,-0.009780279,-0.15577514,-0.009072947,-0.1514209,-0.008544642,-0.1631889,-0.0020228385,-0.14403722,-0.0004574444,0.055033144,-0.0009061608,0.03823102,-0.0032524893,-0.009003306,-0.00004294157,-0.07941798,-0.0000072268476,-0.055694252,-0.0,-0.09845969,-0.00065953773,-0.00030658385,-0.0051444503,-0.0008240483,-0.001557002,-0.0005704229,-0.0025198997,-0.0006316719,-0.0034404323,-0.00010083925,-0.0014530449,-0.00016359736,-0.0013792185,-0.0011333954,-0.0003261616,-0.0008597859,-0.0024778468,-0.002872601,-0.00077762216,-0.00064015575,-0.0018852524,-0.00024700683,-0.00024316192,-0.0000037257983,-0.06603402,-0.000024236075,-0.011300203],[-0.09947495,-0.0060757813,0.029397575,-0.0014909852,-0.00071036763,-0.0007842058,-0.13595998,-0.00065747945,-0.105476886,-0.00014190603,0.1580202,-0.0008444361,-0.008759434,-0.07510103,-0.0031131632,0.040864814,-0.0017983693,-0.1696055,-0.0016273853,-0.16942744,-0.0005170206,-0.16382371,-0.004546314,0.81576926,-0.1541212,-0.046340372,-0.0036909766,-0.026516989,-0.018136142,-0.0000037204702,-0.029660925,-0.00053747825,-0.12014072,-0.0022062599,-0.06914997,-0.0004555159,-0.13266733,-0.0012050848,-0.15432826,-0.0012206536,-0.1665016,-0.0022896063,-0.0879443,-0.013745359,-0.07836047,-0.0015010466,-0.13088594,-0.009844641,-0.08885371,-0.034631684,-0.0839065,-0.00008352323,-0.15563582,-0.000035679677,-0.11680143,-0.00014135118,-0.1517989,-3.138845e-7,-0.018164622],[-0.10032095,-0.035654925,-0.03239726,-0.018476076,-0.0469012,-0.0038074614,0.06624401,-0.002274313,0.03659334,-0.0050804014,0.1873576,-0.0015468018,-0.014425526,-0.002180039,0.34757775,-0.0039978274,-0.12588875,-0.0001439102,-0.07688422,-0.0001179451,-0.15291785,-0.0015603454,-0.16841693,-0.00033450584,-0.1318211,-0.00071408163,-0.018045519,-0.00077530375,-0.16995697,-0.00004953869,-0.1697614,-0.0002558579,-0.16579673,-0.00024668738,-0.13554288,-0.00057400664,-0.108812645,-0.14615703,-0.0010524129,0.71011645,-0.0000031702284,0.8718471,-0.00079483376,0.13919471,-0.0028164976,-0.052757125,-0.0039020781,0.18991241,-0.0034644806,-0.08312373,-0.031301945,-0.07487314,-0.028677465,-0.007832013,-0.00008882497,-0.005668633,-0.00023974216,-0.00026935033,-0.001182264],[-0.088697106,-0.047228705,-0.024756657,-0.012783029,-0.0046355175,-0.004257663,-0.0023125673,-0.013399509,-0.0030657935,-0.009144618,-0.017046181,-0.0029533703,-0.017415507,-0.003077375,-0.000116594805,-0.14321494,-0.00009341138,-0.16957898,-0.00038536603,-0.00007948054,-0.16980849,-0.000023612203,-0.14541014,-0.000025108471,-0.10833523,-0.00006783995,-0.14906542,-0.000011039154,-0.16996455,-0.00043373907,-0.00004810246,-0.15168677,-0.0,-0.16855691,-0.0,-0.00033116838,-0.0038904438,-0.004615253,-0.0030219234,-0.003482194,-0.006120528,-0.0032745576,-0.007008544,-0.001147009,-0.0026336622,-0.007403645,-0.0027394714,-0.0027814312,-0.0000035877413,-0.14567572,-0.0000054898974,-0.022642732,-0.0002075755,-0.0044579003,-0.000010516337,-0.029324153,-0.00012076383,-0.008390875,-0.00034199856],[-0.08975199,-0.03872189,-0.03621615,-0.06449771,-0.029003251,-0.044677623,-0.022076292,-0.0009724968,0.41232118,-0.00117455,0.1623813,-0.0029424839,0.29651976,-0.0044443365,0.0776354,-0.0032154126,0.06356525,-0.0196367,0.16212413,-0.0068001994,0.07809185,-0.020972727,0.13771033,-0.010695137,0.13482203,-0.0016667553,-0.007045492,0.06751922,-0.010878664,1.8426087,-0.022824584,2.3632088,-0.16196735,0.050662745,-0.04655927,0.006565417,-0.1659823,-0.061638232,0.052207112,-0.029741451,0.1301983,-0.07250775,-0.10959961,-0.033723544,-0.16395433,-0.0000020540963,-0.14366753,-0.007353185,-0.10779053,-0.00065511814,-0.12339092,-0.0017995199,-0.1591948,-0.0038094027,-0.0075582233,-0.15231854,-0.010306286,-0.16410725,-0.000001917394],[-0.08975199,-0.020782711,-0.00007704447,-0.0005803697,-0.16589105,-0.0001360059,-0.13911279,-0.00003944595,-0.13596162,-0.000003450311,-0.0005558651,-0.0000052203077,-0.155515,1.5293328,-0.0014577744,1.3843416,-0.00027030395,1.4302552,-0.0002820522,-0.06422248,-0.0013826023,-0.04161727,-0.0013845153,-0.012568153,-0.0023452304,-0.012237321,-0.00034615904,-0.05742052,-0.0066990787,-0.09249321,-0.013683809,-0.074898586,-0.0012062008,-0.017632447,-0.041601364,-0.027168004,-0.024663199,-0.010267376,-0.03057221,-0.014163623,-0.054650426,-0.022277718,-0.0010656522,-0.06616141,-0.0020215248,-0.02583182,-0.000114112845,-0.031149546,-0.00042250773,-0.017302819,-0.013789757,-0.022421882,-0.000249461,-0.00944596,-0.0005128968,-0.0043998174,-0.0013655887,-0.0048938766,-0.0001674521],[-0.088697106,-0.03872464,-0.034958966,-0.028399246,-0.047726195,-0.028252145,-0.07729206,-0.07073366,-0.060938086,-0.05133674,-0.065616265,-0.09248461,-0.050183833,-0.094920896,-0.061800703,-0.084565535,-0.027834954,-0.07014987,-0.11187301,-0.07634991,-0.11126285,-0.106248714,-0.057674833,-0.14961901,-0.043306917,-0.13740021,-0.0415596,-0.045286775,-0.048492935,-0.046936434,-0.047910456,-0.032943416,-0.022906123,-0.017809946,-0.00037030343,-0.1698474,-0.00055567053,-0.16282749,-0.00018759335,-0.13098392,-0.000010124813,-0.010771266,-0.000008817314,-0.0059430003,-0.0000028967706,-0.009978034,-3.142469e-7,-0.048899014,-3.160288e-7,-0.005655081,-0.005424861,-0.001529101,-7.661987e-7,-0.14621314,-0.00013276613,-0.00027993726,-0.024590207,-7.656023e-7,-0.0056788833],[-0.088697106,-0.019822182,-0.020223469,-0.0016350166,-0.008013221,-0.00090072077,-0.018324478,-0.0019089066,-0.019935073,-0.0048087337,-0.021075385,-0.00037236913,0.1703152,-0.0006835826,-0.102780744,-0.002779832,-0.0005688308,-0.07514063,-0.0051713064,0.024122592,-0.0024455662,-0.00014968445,-0.0030691226,-0.1653908,-0.004117651,-0.000105612744,-0.16881718,-0.0,-0.0050304187,-0.025463054,-0.035696812,-0.0067778635,-0.026534887,-0.022187598,-0.0068059284,-0.051251467,-0.000023236784,-0.005384999,-0.00021470618,-0.0049892524,-0.00051109144,-0.1659097,-0.0,-0.0035220848,-0.0013956558,-0.00091140415,-0.00056673994,-0.0007573695,-0.0011405752,-0.0001633772,-0.0020841558,-0.0005308616,-0.010355223,-0.00056105806,-0.00010935859,-0.03388659,-0.00006303309,-0.043502327,-0.00018737314],[-0.088697106,-0.047228705,-0.031201147,-0.007698928,-0.0047930507,-0.00055097823,-0.0008484646,-0.0016107666,-0.003486288,-0.0059026163,-0.0028041073,-0.0050699716,-0.007202998,-0.00055327435,-0.00081644487,-0.16303284,-0.00037825387,-0.000053343803,-0.1543635,-0.0004517553,-0.13761558,-0.0000042717497,0.07516948,-0.0000076249116,-0.015734004,-0.15495452,-0.0022592726,-0.13660665,-0.0014171846,-0.07369415,-0.0016519654,-0.14970239,-0.0021610938,-0.08567955,-0.005154159,-0.0002507402,-0.13840999,-0.15036863,-0.16581577,-0.16996804,-0.10005441,-0.16990617,-0.13970031,1.4992082,-0.00032323156,-0.097811066,-0.0007009962,-0.15030503,-0.11912318,-0.013331618,-0.00241882,-0.003734947,-0.0032190497,-0.00082241424,-0.002340321,-0.0075263833,-0.0028342595,-0.0013718097,-0.0002635108],[-0.088697106,-0.047228705,-0.024756657,-0.03918037,-0.024532879,-0.02578685,-0.030598236,-0.023966014,-0.020044006,-0.049975783,-0.003775119,-0.011539326,-0.002170225,-0.017456777,-0.00025883733,-0.118887275,-0.10846784,-0.16019197,-0.08833555,-0.13702016,-0.14122768,-0.16099335,-0.15705335,-0.16850184,-0.1698681,-0.15405974,-0.113993354,-0.14719656,-0.15983695,-0.16206062,-0.11677887,-0.12003161,-0.1610767,-0.067727044,-0.15882766,-0.16918977,-0.0031050956,1.5141536,-0.00035259646,0.95939565,-0.00022353968,0.79481685,-0.0011371261,0.6720088,-0.00017393017,0.6497313,-0.00019412507,1.5446357,-0.0020935803,1.5285715,-0.04661544,-0.14927754,-0.00025872944,-0.025003808,-0.00015255793,-0.010805715,-0.00008248432,-0.0033149347,-0.0000030311153],[-0.09947495,-0.035660315,-0.009023123,-0.030039871,-0.0049361656,-0.005055261,-0.00019890245,-0.0006976676,-0.1697135,-0.00010765893,-0.14714712,-0.00066621986,-0.06270465,-0.15316476,0.66153246,-0.00059149787,1.0054603,-0.000761834,1.0512135,-0.00031633346,0.8892992,-0.00006479376,1.3475142,-0.00012379776,0.9710556,-0.0006504936,0.7133106,-0.0013350865,0.840933,-0.000016181708,1.2353823,-0.00010618023,1.5426575,-0.000038231807,1.0307041,-0.0005598552,0.77718395,-0.00011546588,1.1195192,-0.0005361738,1.0397744,-0.0004478978,0.5390632,-0.000043907494,-0.092103906,-0.00017228581,-0.0000027570243,-0.16612418,-0.000007225541,-0.001093396,-0.025653511,-0.0026926028,-0.007264452,-0.003087197,-0.0021662263,-0.0012873939,-0.002540638,-0.0010381144,-0.0004278199],[-0.088697106,-0.019822182,-0.006078963,-0.005755342,-0.004368909,-0.0015335263,-0.0049232743,-0.004967793,-0.0062079015,-0.0001277257,-0.0031837532,-0.16417196,-0.0012013754,-0.16165105,-0.0036830024,-0.00045226648,0.0036113379,-0.00023470537,-0.11003575,-0.000026225269,-0.16939917,-0.00007855004,-0.16997114,-0.00024369368,-0.16917363,-0.0002807848,-0.14614862,-0.00020951848,-0.12363499,-0.00009615801,-0.15427023,-0.000049653398,-0.14620113,-0.00044393103,-0.00013779037,-0.16513646,-0.00030532695,-0.10860528,-0.16975845,-0.14099148,-0.13118899,-0.057831917,0.8432055,-0.00069933175,1.1384608,-0.0017208381,1.1246594,-0.16341652,-0.09760912,-0.124092035,-0.0016208268,-0.023458803,-0.0021701404,-0.012205089,-0.0004720493,-0.072433524,-0.000008286972,-0.022699626,-0.000013878658],[-0.10032095,-0.0053279535,-0.033779707,-0.0003248003,-0.14365277,-0.000014523156,-0.0007654186,-0.14115089,-0.000018589555,-0.16919206,-0.000027461278,-0.15140525,-0.14811935,-0.11868722,-0.122294866,-0.060291108,-0.00009421522,1.9189442,-0.000018845622,-0.075709276,-0.0025066598,-9.090595e-7,0.030185355,-1.6292718e-7,0.5095576,-0.0000014900368,0.2645094,-1.6187875e-7,0.86639565,-3.1346536e-7,0.29868832,-0.0,0.15143934,-0.00044952106,-0.08730896,-0.0010651797,-0.046233073,-0.0008336715,-0.07389878,-0.005800608,-0.07648291,-0.0025770233,-0.05368112,-0.006767068,-0.017119015,-0.016344296,-0.0071458775,-0.054453205,-0.02353258,-0.034437623,-0.0023104143,-0.0016893044,-0.00020649067,-0.00018017039,-0.00045470285,-0.00009478786,-0.00059229165,-0.000105953004,-0.0007015777],[-0.08975199,-0.03872189,-0.016192691,-0.021477921,-0.017861353,-0.03000492,-0.03238498,-0.0011804939,0.27304235,-0.0011832863,-0.0020033722,0.04289628,-0.005961309,-0.0020294993,0.13842113,-0.00087271683,-0.12521799,-0.00046464702,-0.04641091,-0.0009756244,-0.07684562,-0.0018212636,-0.13829051,-0.0010509073,-0.06472327,-0.0028956106,-0.1406853,-0.00038011017,-0.16644132,-0.04664461,-0.0020030234,0.019001372,-0.0000027547485,-0.091992304,-0.0000024774474,0.020830672,-0.0000014901941,-0.109031655,-0.0000037250518,-0.06788796,-0.000008025787,-0.06379299,-0.000013881643,-0.09667356,-0.0000034505958,-0.0067653563,-0.10954021,-0.007350461,-0.0056355447,-0.004959677,-0.031381816,-0.004473592,-0.008741664,-0.005178912,-0.04022389,-0.003501942,-0.06710155,-0.000002056878,-0.022625102],[-0.09947495,-0.040831033,-0.027510555,-0.031628385,-0.0071059116,-0.015351951,-0.017120076,-0.027091628,-0.021159938,-0.019525766,-0.034347232,-0.053395074,-0.0066828905,-0.010062223,0.03218561,-0.00692803,-0.15367295,-0.0007834343,-0.15589528,-0.16973463,2.2171173,-0.16991897,-0.0069229817,-0.0286266,-0.035720997,-0.04156066,-0.027754324,-0.003173135,-0.03716538,-0.040399637,-0.0064880145,-0.002469034,-0.0,-0.0033597653,-0.0,0.6452382,-0.0,1.1264442,-0.0,-0.000027342434,-0.017456155,-0.00002311257,-0.01585977,-0.00012727793,-0.046051763,-1.6470914e-7,-0.04103871,-4.6425353e-7,-0.05375521,-0.000003173811,-0.0987455,-0.0000021994363,-0.015463017,-0.000002478769,-0.010852369,-0.000002337987,-0.00006819071,-0.07677457,-0.0000010547368],[-0.08975199,-0.047722783,-0.027511943,-0.039432205,-0.018557785,-0.006008689,-0.0021023124,-0.0064777387,-0.0023755266,-0.013035459,-0.0020555742,-0.027840652,-0.000438126,-0.1495752,-0.015853979,-0.11184766,-0.0072643845,-0.11221079,-0.020054692,-0.14456569,-0.037733924,-0.10743734,-0.008385622,-0.15326306,-0.01644655,-0.09249805,-0.000003996278,-0.00052354415,-0.00021470257,-0.00024049224,-0.10617016,-0.00014101624,-0.093241714,-0.00021005816,-0.10248257,-0.0000070926903,-0.16964269,-4.6619073e-7,-0.0043410757,-0.00037360945,-0.0019396577,-0.0,-0.15771975,-0.0,-0.0017240497,-0.005909156,-0.00003956489,-0.04652696,-0.00001298079,-0.014051193,-0.0000751865,-0.00802924,-0.00006725763,-0.0053600785,-0.00018497385,-0.004075624,-0.000100610894,-0.0022601392,-0.00016536309],[-0.088697106,-0.047228705,-0.024756657,-0.03918037,-0.018425493,-0.016452735,-0.0034774386,-0.02358879,-0.0034918662,-0.16642728,-0.006624933,-0.082107574,-0.011209948,-0.057783775,-0.008254283,-0.16915604,-0.14594088,-0.16056226,0.236718,-0.14214565,-0.16550338,-0.1697663,1.5676725,-0.005974986,1.092071,-0.007069019,1.0245391,-0.011357706,0.73719513,-0.0014374933,0.24330355,-0.0000033101032,-0.012821161,-0.060335036,-0.0017136948,-0.069689706,-0.0034680865,-0.0001397965,-0.15926498,-0.00018814097,-0.013102255,-0.04029244,-0.058312144,-0.0018865727,-0.03800816,-0.0000024796057,-0.1423806,-0.000030543742,-0.16506939,-0.0000118153075,-0.062210534,-0.0002604294,-0.08666226,-0.000059145143,-0.02930004,-0.00002597769,-0.0156556,-0.0000078881985,-0.01804484],[-0.09947495,-0.035660315,-0.009023123,-0.030039871,-0.009586022,-0.028894648,-0.0021686594,-0.0051355152,-0.0042472086,-0.0048818993,-0.003238261,-0.008930945,-0.011363087,-0.020715455,-0.009799524,-0.03989594,-0.0062549794,-0.0010461095,-0.14150383,-0.00079048943,-0.09643738,-0.001635733,-0.14340681,-0.0011622723,-0.1678272,1.6428837,-0.0033441202,1.2309476,-0.0022789855,-0.06407961,0.09333209,-0.12988582,0.020264402,-0.094363414,-0.06217285,-0.013502351,-0.049332615,-0.0072995694,-0.041501403,-0.01761416,-0.019641554,-0.020204315,-0.011400048,-0.03166102,-0.023698179,-0.019030882,-0.01172718,-0.07310175,-0.0046994733,-0.06563937,-0.0007173918,-0.084553815,-0.00038300297,-0.025353236,-0.000641335,-0.029855717,-0.00023385115,-0.005456829,-0.0014442246],[-0.10032095,-0.03138299,-0.013045667,-0.03588519,-0.004629767,-0.055264987,-0.001916931,-0.057928193,-0.0012600212,-0.027401559,-0.0008278068,-0.07468129,-0.000021986754,-0.020235267,-0.0000021988378,-0.0051082433,-0.015038268,-0.0017067294,-0.13095132,-0.0000021961816,-0.02776164,-0.00093473116,-0.07447068,-0.000414941,-0.076542065,-0.0007780148,-0.0797592,-0.0009988899,-0.06717244,-0.00013823631,-0.0001137741,-0.16598205,-0.000029558412,-0.16931105,-0.000081099446,-0.16896556,-1.6084076e-7,-0.12138887,-0.0000028940374,-0.13693982,-0.00042771836,-0.0023660606,-0.13406366,-0.0003621341,-0.13131043,-0.0006759343,-0.018680176,-0.0037429826,-0.027731095,-0.018812971,-0.035141923,-0.00005168239,-0.16328527,-0.00001897176,-0.09109979,-3.177206e-7,-0.0684853,-0.000024484474,-0.03150119],[-0.10032095,-0.0053279535,-0.16828103,-0.00054917793,-0.000043306154,0.008820117,-0.0005150085,-0.106862925,-0.0031934513,-0.16633585,-0.005843148,-0.14558049,-0.003584538,-0.15711471,-0.006369892,-0.00014812722,-0.042930234,-0.000049534454,-0.16156013,-0.000028204147,-0.16437604,-0.00010254285,-0.15132421,-0.00013231796,-0.00040459348,-0.007655065,-0.0002611771,-0.019925399,-0.000048701564,-0.0070254314,-0.00044819847,-0.014026834,-0.00020973677,-0.002903078,-0.0001034547,-1.6173405e-7,-0.12477756,-4.694496e-7,-0.010672474,-4.694165e-7,-0.007179464,-0.0,-0.0000016289589,-0.0007174919,-0.0000045411552,-0.00007774075,-0.14711212,-1.6001357e-7,0.7322729,-0.0,0.35210934,-0.0,0.24327037,-0.0,-0.08453605,-0.00008074966,-0.04362064,-0.00009604427,-0.06877221],[-0.088697106,-0.047228705,-0.047036625,-0.018331934,-0.017014537,-0.006395682,-0.0021106785,-0.004382027,-0.00445956,-0.011092274,-0.008621636,-0.0031410027,-0.007365601,-0.0027784759,-0.0029102103,-0.16988155,-0.008361451,-0.16981944,-0.005367487,-0.14894405,-0.11308571,-0.0010786001,-0.14352813,-0.00036389593,1.6355057,-0.0004100277,0.7971788,-0.0070980047,0.4138862,-0.01913608,0.37870228,-0.0781719,-0.10947377,-0.12925765,-0.00008225687,-0.11097057,-0.0014965188,-0.10567359,-0.00031643364,-0.16842842,-0.008639603,-0.064051904,-0.017154744,-0.10212038,-0.030383084,-0.0416667,-0.032897636,-0.04537055,-0.054795414,-0.030592361,-0.0031630914,-0.0011238774,-0.00012761408,-0.007322966,-0.0008816173,-0.020871742,-0.00201302,-0.048208997,-0.00056743214],[-0.10032095,-0.035654925,-0.053947534,-0.0031738011,-0.002140828,-0.002505126,0.2362383,-0.0010230453,2.344092,-0.016750762,-9.110263e-7,-0.0038916592,0.19462335,-0.0089922305,-0.01789477,-0.009786512,-0.027099991,-0.030988758,-0.024838656,-0.022147626,-0.01144774,-0.0027837914,-0.011547073,-0.0032384226,-0.036248785,-0.001714676,-0.0060555562,-0.012635901,-0.15072946,-0.000033238437,-0.10939301,-1.6002393e-7,-0.1640398,-0.0000016312058,-0.14207664,-0.0000035850271,-0.16781454,-0.00006303427,-0.12245014,-0.00017951461,-0.1151872,-0.000007627555,-0.15582438,-0.000032629727,-0.13503519,-9.087263e-7,-0.0011450486,-0.00674181,-0.00072003016,-0.0000023390821,-0.16095175,-0.0000014855559,-0.15999383,-0.0000014903607,-0.16788901,-1.6456022e-7,-0.15163185,-0.0,-0.12889884],[-0.10032095,-0.035654925,-0.015528718,-0.0077247424,-0.0058629606,-0.0014521334,-0.005674984,-0.00008479032,-0.001608247,-0.13183844,-0.001542923,-0.1397141,-0.00002597418,-0.14302394,-0.12275744,-0.16042852,-0.13706514,-0.11299308,0.041697428,-0.11743933,-0.16948347,-0.13536054,-0.1635489,-0.13038154,-0.12482326,-0.14491028,-0.13223758,-0.14551923,0.22107437,-0.16987814,0.23860975,-0.118634745,-0.020697005,-0.021682981,0.8178504,-0.0000672607,0.11346844,-0.00002386303,-0.1598913,-0.0000033092083,-0.16370438,-1.629524e-7,-0.1108816,-0.0000039999104,-0.11401168,-0.00003799127,-0.0115776155,-0.022127243,-0.024477528,-0.00048630036,-0.12963226,-0.000004268805,-0.14315082,-0.00028933655,-0.08141039,-0.0017881502,-0.05906871,-0.00090253487,-0.009846106],[-0.08975199,-0.047722783,-0.037521973,-0.023224138,-0.0108384965,-0.09260862,-0.13960353,-0.08672474,-0.09395611,-0.024432419,-0.13930807,-0.031182764,-0.16979225,-0.01822901,-0.12903039,-0.07164104,-0.16539818,-0.0709613,-0.16088632,-0.00015046043,-0.05277312,-0.0032032332,-0.1407397,-0.026216963,-0.11427283,-0.031386305,-0.1037446,-0.020991351,-0.10570472,-0.021312451,-0.00008698149,0.06207213,-0.00014557302,-0.10519491,-0.0000118129365,-0.0000033112642,-0.10449119,-0.0033435433,-0.018703606,-0.0014326687,-0.010590984,-0.0008275148,-0.022789814,-0.020216428,-0.027609363,-0.037988458,-0.024331555,-0.03418493,-0.040649578,-0.01353644,-0.07779379,-0.023000535,-0.028579876,-0.00047518415,-0.025663307,-0.0028845307,-0.010215446,-0.00008767281,-0.052103277],[-0.09947495,-0.040831033,-0.027510555,-0.031628385,-0.0071059116,-0.004120934,-0.0006849519,-0.0032833132,-0.001711014,-0.010108446,-0.000019855628,-0.1209858,-0.000031769498,-0.039464187,-0.00037216872,-0.00015201044,-0.11398827,-0.00044179222,-0.0005292601,-0.1356052,-0.0009677653,-0.15556702,-0.00020746591,-0.13792308,-0.00047558453,-0.16890402,-0.00037712418,-0.16996503,-0.00034647124,-0.10768763,-0.14053564,-0.1621691,-0.15772247,-0.14979228,-0.16912557,1.3281723,-0.01298976,0.5732202,-0.004742876,0.9798261,-0.03753025,-0.13113208,0.9278973,-0.006961263,-0.08455655,-0.021936337,0.10189168,-0.0395126,-0.11244735,-0.048449,-0.15767574,-0.07705262,-0.0973889,-0.021404253,-0.083181545,-0.002259185,-0.035282567,-0.0073184757,-0.0012121306],[-0.08975199,-0.03872189,-0.030243305,-0.048121214,-0.030345775,-0.00041944132,-0.019794887,-0.08692707,-0.004159775,0.12091817,-0.011305385,-0.07454214,-0.0042842985,0.1679932,-0.0045448127,-0.12147378,-0.00015654122,0.003209772,-0.000065855405,-0.112482466,-0.0011157526,-0.13868003,-0.002907529,-0.0057209376,-0.0040974435,-0.008716197,-0.106038004,0.49131978,-0.16784978,0.5501175,-0.1684344,0.6347293,-0.16809502,2.4164221,-0.011044231,1.9185832,-0.027956067,-0.1582297,0.20602064,-0.100392655,0.04177361,-0.0647462,0.00524342,-0.11374937,-0.05586827,-0.09092298,-0.02432748,-0.07520202,-0.119922064,-0.069134556,-0.006918152,-0.11076195,-0.00023342123,-0.077661,-0.0006198262,-0.028005406,-0.0002559603,-0.018925726,-0.000022117545],[-0.09947495,-0.0060757813,0.029397575,-0.0034576708,-0.0023954676,0.3688178,-0.0025881766,0.018727813,-0.0019679912,0.25188482,-0.0062950016,0.16210705,-0.0047423984,0.11220026,-0.0011504531,0.10875049,-0.0016859042,-0.07941582,-0.008539408,-0.002715537,-0.006509374,-0.12907746,-0.00017469742,1.0776261,-0.0007078331,1.0615078,-0.00082742155,0.10752909,-0.0076455735,0.21376044,-0.0023031668,0.15370917,-0.08972559,-0.13672534,-0.055901468,-0.0016419226,-0.012502134,-0.000002338207,-0.10706997,-0.000018340705,-0.09621881,-0.0000030311357,-0.16864702,-0.00022762877,-0.1699306,-0.00018060845,-0.06658872,-0.0000035872479,-0.10574832,-0.0000031702734,-0.1284392,-0.000006564781,-0.013433877,-1.6048045e-7,0.19594781,-0.0,-0.10350913,-1.5964375e-7,-0.04199476],[-0.09947495,-0.040831033,-0.024414968,-0.090483315,-0.06449297,-0.003852138,0.6087367,-0.0035116002,0.6970751,-0.0013601008,0.9051193,-0.0009171191,-0.02026736,-0.05213727,-0.008976808,0.120200455,-0.009758716,0.061560724,-0.000039325005,-0.16383433,-0.0000013426923,-0.13317455,-0.0005676337,-0.16929142,-0.00038258723,-0.16696501,-0.000022113712,-0.12495786,-0.0003359641,-0.15676335,-0.0001813676,-0.09418896,-0.00007518458,-0.06293908,-0.000038351165,-0.000011165651,-0.054299176,-0.000004133006,-0.046988472,-0.0000068320155,-0.037609257,-0.00019521524,-0.0000011998357,-0.07091811,-6.137641e-7,-0.03986149,-1.5996827e-7,-0.0024030241,-0.0,-0.00023813378,-0.09742853,-0.00051832775,-0.0001879162,-0.0005238424,-0.0005743063,-0.0026888729,-0.0,-0.13575292,-1.6330556e-7],[-0.08975199,-0.020782711,-0.0037444478,-0.0066828136,-0.002528779,-0.0025319357,-0.00020464837,-0.0044475114,-0.0008388593,-0.012227722,-0.0037967665,-0.0017803358,-0.0013936547,-0.0023639977,-0.0021616183,-0.0017995245,-0.00845813,-0.002890489,-0.022367455,-0.00710658,-0.020896355,-0.007816152,-0.027320584,-0.11535656,-0.015963133,0.10678762,-0.13976021,-0.032487288,-0.09346603,-0.010376344,-0.1645622,-0.023386944,-0.16995221,-0.036918018,0.11930397,-0.008539981,-0.12647144,-0.020134607,-0.12508497,-0.013653659,-0.16953552,-0.04138614,-0.06300966,-0.072064236,-0.066270545,-0.15974514,-0.08815288,-0.0037431451,-0.015129563,-0.0012305544,-0.000024613173,-0.0123060895,-0.0006574792,-0.0684111,-0.0009745807,-0.0425473,-0.0016395891,-0.010298588,-0.0010754102],[-0.09947495,-0.035660315,-0.0087335175,-0.009055071,-0.009621071,-0.04210529,-0.030089589,-0.030983873,-0.017834939,-0.047472835,-0.0063820044,-0.042987518,-0.0052058985,-0.039227396,-0.04385896,-0.053889565,-0.033218425,-0.023813386,-0.003243224,-0.16970669,-0.09400691,-0.16656351,-0.16990569,-0.16477273,-0.1690625,0.042861175,-0.1421794,-0.062425144,-0.16508244,-0.033382166,-0.00086907885,-0.03133384,-0.021489132,-0.031095251,-0.026812823,-0.061068352,-0.056662466,-0.083086245,-0.040402193,-0.095231846,-0.034197003,-0.07128658,-0.017342398,-0.047789462,-0.0038750998,-0.036433905,-0.006324224,-0.026309626,-0.0024834108,-0.007881601,-0.009249926,-0.009906204,-0.00927798,-0.015951497,-0.014446689,-0.00037175123,-0.011792392,-0.00007053223,-0.013616345],[-0.088697106,-0.03872464,-0.034958966,-0.019703202,-0.017811377,-0.02174707,-0.034350287,-0.0007236305,-0.0049041226,0.071739934,-0.00001745164,-0.16643111,-0.0001158039,-0.16473714,-0.00013108634,-0.000042581247,-0.16242726,-0.000018342762,-0.104741365,-6.146355e-7,-0.16974463,-0.0000012003077,-0.14520787,-9.0894605e-7,-0.11554825,-0.0,-0.001955676,-0.0011135065,-0.0005201357,-0.00004318248,-0.00026850286,-0.000074605734,-0.0023820573,-0.00022644101,-0.008277211,-0.000012719583,-0.0000030338567,-0.000021111791,-0.000011166899,-0.000003309185,-0.000003862158,-0.000004271775,-0.00007588757,-0.0000031725688,-0.000048703663,-1.6439378e-7,-0.0005458785,-0.0,-0.00045054243,-1.612486e-7,-0.00032762447,-0.0,-0.00023470393,-0.000001056093,-0.000059855614,-0.0,-0.0008720447,-0.000009860799,-0.0007386223],[-0.09947495,-0.0060757813,0.04006292,-0.0017947241,0.11962152,-0.0021756995,0.35965887,-0.0016840283,-0.13213997,-0.000054057833,-0.00028796395,-0.00024796804,-0.045795064,-0.00043618726,-0.0000014883188,0.37174886,-0.00024038,-0.0055800215,-0.0004528798,-0.1681119,-0.0007071503,-0.0001376755,-0.08505915,-0.000052396816,-0.11812603,-0.0005377735,-0.15578781,-0.0031015184,-0.138383,-0.016902575,-0.0003414804,0.046070814,-0.0004931657,-0.049536657,-0.00017929531,-0.10210687,-0.00021286942,-0.03883079,-0.0002544727,0.029787356,-0.000055127,0.09026279,-0.0006354241,0.0037438772,-0.0007349291,-0.014979265,-0.0044142744,-0.0032738992,-0.12683609,-0.010043209,-0.0051895455,-0.010533067,-0.006806231,-0.0029448234,-0.00028859297,-0.008916289,-0.000003312437,-0.01750655,-0.00003677794],[-0.10032095,-0.03138299,-0.0031111676,-0.00850635,-0.02761548,-0.006221783,-0.020426644,-0.008482893,-0.0047085993,-0.16961819,-0.011351167,-0.0004072547,-0.06536717,-0.000002194423,-0.10414174,-0.00004342673,-0.091074966,-0.000014522378,-0.06451273,-0.000063035004,-0.0029250516,-0.00003397504,-0.007781943,-0.000022990738,-0.013001574,-0.00003152864,-0.0087258825,-0.0034690774,-0.094785415,-0.0000030319516,-0.0014883578,-0.104202665,-0.0058602947,-0.01845527,-0.003828106,-0.012649705,-0.003917009,-0.1064982,-0.0000014884728,-0.012944831,-1.6048956e-7,-0.082639314,-0.0,-0.145223,-0.000010513341,-0.1330741,-0.0000014887391,-0.00001349654,-0.036813926,-0.0000978697,-0.0022482832,-0.009729574,-0.0009349213,-0.00044524806,-0.013949314,-0.020937154,-0.000027213646,-0.0021733525,-0.0000714649],[-0.10032095,-0.0053279535,-0.16828103,-0.000011686799,-0.15170456,-0.00014813077,-0.13797265,-0.000055247376,-0.1551662,-0.000050849034,-0.16821353,-0.0008123988,-0.09796411,-0.0017943701,-0.000025355186,-0.13632455,-0.00002733668,-0.1466991,-0.00016667975,-0.1644894,-0.0006477365,-0.12489389,-0.000088820314,-0.16969311,-0.0014144491,2.017746,-0.0013072663,0.14334531,-0.0000013441564,0.073474005,-0.008545794,-0.06755094,-0.01039324,-0.020584304,-0.005744799,-0.0685856,-0.0062180175,-0.04396428,-0.0066408482,-0.09738858,-0.0009056825,-0.16985928,-0.00079812255,-0.09611385,-0.000031647844,-0.13923354,-0.000031651653,-0.12937993,-0.006417241,-0.012715507,-0.0037265779,-0.032810334,-0.00010038393,-0.0066055697,-0.0001548863,-0.009935395,-0.00027464965,-0.000086979206,-0.0042108754],[-0.088697106,-0.019822182,-0.006078963,-0.0044625783,-0.0080473395,-0.001419825,-0.0053511267,-0.0008227987,-0.002507344,-0.0032238523,-0.002383868,-0.009138598,-0.006517316,-0.016308239,-0.0015894502,-0.00020919288,-0.019395286,-0.00038382536,-0.0074689393,-0.0000028967047,-0.02812203,-9.117124e-7,-0.13957058,-0.16884129,-0.050540358,-0.16570002,-0.00022461715,-0.00068093685,-0.16269839,-0.0008137448,-0.16986713,-0.0032533982,-0.000028943594,-0.04753922,-0.0,-0.1687878,-0.0000014879548,-0.044925407,-0.000100608966,-0.014317103,-0.0000014886066,-0.0733406,-1.6292284e-7,-0.055248577,-0.0000010540926,-0.047524307,-0.000005759853,-0.15310031,-0.0000089439545,-0.13503377,-0.000021235348,-0.092128955,-9.1251286e-7,-0.065618776,-0.000017200302,-0.055374295,-0.000013753436,-0.029883832,-0.00001516246],[-0.088697106,-0.03872464,-0.013733427,-0.031654138,-0.032428548,-0.044863064,-0.01127225,-0.08716536,-0.018116437,-0.043118417,-0.0016069935,-0.018766519,-0.023512226,-0.020607328,-0.012874286,-0.0444559,-0.01755561,-0.007623952,-0.042149376,-0.02288592,-0.008933928,-0.010589191,-0.03503511,-0.024712255,-0.016083611,-0.014858212,-0.0012071291,-0.14041933,-0.16411217,-0.0082146125,1.3577714,-0.004934606,1.9929144,-0.013851473,1.4083836,-0.00034387544,2.091915,-0.00023577883,1.3851224,-0.00031465568,1.3452857,-0.0024857991,0.635653,-0.0053416244,0.6501464,-0.000831846,-0.15835823,-0.000012847869,-0.11114959,-0.00009695527,-0.16953726,-0.030499857,-0.022892578,-0.00034688928,-0.10094735,-0.0017727724,-0.024535347,-0.00007344476,-0.006834425],[-0.10032095,-0.035654925,-0.0230308,-0.017829426,-0.007544393,0.034857616,-0.002761434,-0.031050026,-0.009757873,-0.029953819,-0.026346816,-0.097664654,-0.01735694,-0.14322029,-0.000024360217,-0.06079066,-0.0000021952271,-0.08900904,-6.14105e-7,-0.015157851,-0.000017196326,-0.010977103,-0.0000024783412,-0.039921734,-1.6178885e-7,-0.071374945,-0.0000017736077,0.002248009,-0.0000026151972,-0.116305664,-0.0000017722429,-0.111467466,-0.0,-0.00009558641,-0.0547509,-0.00013566564,-0.06987138,-0.0,-0.09149604,-7.654205e-7,-0.14945503,-0.0,-0.15458478,-1.622685e-7,-0.007304499,-0.0070240223,-0.0046839695,-0.0031779434,-0.0037741442,-0.0,0.14280942,-0.0,-0.12772657,-1.6053329e-7,-0.07006941,-4.6773675e-7,-0.12691312,-4.6441946e-7,-0.14195557],[-0.09947495,-0.040831033,-0.036548346,-0.06650041,-0.012038965,-0.041774154,-0.0429642,-0.022636874,-0.0021395166,0.21775429,-0.00048185902,0.56554335,-0.0012197285,0.025540654,-0.000068428926,2.4962347,-0.0019871367,2.0685396,-0.16907485,-0.06005873,-0.08205157,-0.077057034,-0.11131377,0.17610754,-0.029732712,0.040370353,-0.041247662,0.3557846,-0.04906311,-0.036638867,-0.034610525,-0.021560771,-0.06892921,-0.09214747,-0.017694162,-0.15434253,-0.008527153,-0.08457939,-0.005494766,-0.06535227,-0.0,0.46384507,-3.15641e-7,0.34723723,-0.00023727622,-0.0001089061,-0.1508616,-0.00014967936,-0.11269187,-0.00018410345,-0.000001053782,-0.04178766,-0.0000027550502,-0.03672696,-0.0000014901643,-0.0015347943,-0.0007735635,-0.0004938705,-0.0001570923],[-0.09947495,-0.040831033,-0.06825675,-0.022927664,-0.031999394,-0.03967678,-0.029619884,-0.059828043,-0.044226144,-0.06242866,-0.021074971,-0.020202331,-0.04656523,-0.051488873,-0.00882969,-0.0343651,-0.01438017,-0.000527856,-0.008439955,-0.02146018,-0.0023244699,-0.021908179,-0.0043811547,-0.020991826,-0.0044563953,-0.020505177,-0.0020563642,-0.0012384123,-0.0037897928,-0.002709022,-0.16639337,-0.001096953,-0.14756298,-0.004937336,-0.16978295,-0.16992936,-1.6055839e-7,-0.09990555,-0.000005084007,-0.012410615,-0.00007948019,-0.013424749,-0.00051370746,-0.015742231,-0.00010980877,-0.0044674124,-0.000034586585,-0.0022003613,-0.0000045422044,-0.0054586777,-0.0000026192486,-0.0021112068,-0.002254424,-0.0003269971,-0.0025659495,-0.00015278222,-0.0000031747652,-0.045261223,-0.000004678922],[-0.09947495,-0.040831033,-0.024414968,-0.030776586,-0.028433269,-0.0121419225,-0.0051097977,-0.031637028,-0.054444443,-0.0010252172,-0.006439357,-0.0001903185,0.1680399,-0.00048468378,0.18382473,-0.0011423491,0.124419,-0.00018573916,-0.15509582,-0.00014546563,-0.000028698629,-0.15485519,-0.00001630769,-0.1683223,-0.000050252012,-0.00011478556,-0.14099298,-0.0005028363,-0.092106014,-0.00006737539,-0.09637786,-0.000057378948,-0.11629686,-0.00021340832,-0.020709528,-0.00014379826,-0.02002421,-0.00072227325,-0.0049783285,-0.000085833344,-0.006920535,-0.000012076505,-0.027345305,-9.1181914e-7,-0.000056073986,-0.060203675,-0.00093624654,-0.042149715,-0.0036328386,-0.015958874,-0.0090119755,-0.05114442,-0.0,-0.07002739,-0.000001056607,-0.07585174,-0.0000023376392,-0.057561014,-0.0000070964206],[-0.09947495,-0.0060757813,-0.009377803,-0.001544185,-0.114144646,-0.00009513047,-0.14819604,-0.00018312098,-0.16714343,-0.000019854117,-0.16956209,-0.00041995407,-0.16987824,-0.00011772336,-0.001624963,-0.16835989,-0.015182073,-0.16984625,-0.0034726027,-0.065917,-0.021942358,-0.16420656,-0.010320985,-0.05423676,-0.0038085915,-0.1378819,-0.0033216167,-0.04926813,-0.0031709769,-0.1625897,-0.0001752444,-0.06528835,-0.0030851183,-0.12044976,-0.005076355,-0.16934562,-0.059404857,-0.022748826,0.9180871,-0.0002141628,-0.13610369,-0.010160891,-0.15133919,-0.021639168,-0.16975386,-0.017179012,-0.14104824,-0.006229866,-0.11566081,-0.015164535,-0.10805075,-0.00013879356,-0.15511453,-0.0,-0.07289736,-0.0000010535053,-0.10134715,-0.0,-0.080195226],[-0.08975199,-0.020782711,-0.0037444478,-0.0058081537,-0.00005619626,-0.15644875,-0.0003833095,-0.00022429423,-0.1579946,-0.001502043,-0.12908888,-0.0012053681,-0.077105604,-0.000272427,-0.00017074698,0.10388051,-0.00003519338,-0.14946394,-0.000004681214,-0.04205721,-0.00011388708,0.05263868,-0.00010788393,-0.11411446,-0.000018843068,-0.16955006,-0.0008374213,1.398841,-0.0002827888,0.74872345,-0.01186315,1.3185259,-0.014163486,-0.16692217,-0.06131677,-0.14796707,-0.04689428,-0.1199694,-0.017892286,-0.0014439774,-0.04384865,-0.13158299,-0.16479816,-0.16230825,-0.12138491,-0.1692129,-0.022297472,-0.10801448,-0.04977086,-0.00008974022,-0.013476475,-0.00672257,-0.015948508,-0.00082655344,-0.0031548783,-0.00215788,-0.0079060765,-0.00060042035,-0.014538697]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"Move\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"title\":{\"text\":\"Game\"}},\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"title\":{\"text\":\"L5N1393 Activations over 50 games\"}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('d557dc6a-3969-40a6-8af5-f0d333259b2a');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "neuron_acts = focus_cache[\"post\", layer, \"mlp\"][:, :, neuron]\n",
    "imshow(neuron_acts, title=f\"L{layer}N{neuron} Activations over 50 games\", yaxis=\"Game\", xaxis=\"Move\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Max Activating Dataset Examples"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another tactic is studying the max activating dataset examples for the neuron. Doing this properly across many games is effort, but here we have cached outputs on 50 games (3000 moves), so we can look at the top 30 games there. This is hard to look at individually, so we can aggregate according to how often each cell is mine vs their's (setting their's==1, mine==-1 and blank==0, and averaging)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "# focus_states_flipped_value does this but has their's==2 and mine==1, not 1 and -1. So let's convert, this is boring and you don't need to understand it.\n",
    "focus_states_flipped_pm1 = torch.zeros_like(focus_states_flipped_value, device=\"cuda\")\n",
    "focus_states_flipped_pm1[focus_states_flipped_value==2] = 1.\n",
    "focus_states_flipped_pm1[focus_states_flipped_value==1] = -1.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"c0baccb0-6f3b-45c1-a7d3-8155e7511c60\" class=\"plotly-graph-div\" style=\"height:525px; width:600px;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"c0baccb0-6f3b-45c1-a7d3-8155e7511c60\")) {                    Plotly.newPlot(                        \"c0baccb0-6f3b-45c1-a7d3-8155e7511c60\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.033333335,0.0,-0.033333335,-0.10000001,-0.26666668,-0.23333335,-0.06666667,0.0],[-0.033333335,0.033333335,-0.10000001,-0.36666667,-0.10000001,-0.13333334,-0.06666667,0.10000001],[0.0,0.0,-0.16666667,-0.4666667,-0.20000002,0.06666667,-0.13333334,0.36666667],[-0.26666668,-1.0,-0.6,-0.4666667,-0.6666667,-0.33333334,-0.13333334,-0.13333334],[0.10000001,-0.40000004,1.0,-0.06666667,0.26666668,-0.033333335,-0.20000002,-0.10000001],[0.06666667,-0.033333335,0.3,-0.13333334,0.06666667,-0.33333334,-0.13333334,0.0],[0.16666667,-0.06666667,-0.13333334,-0.06666667,0.06666667,-0.16666667,0.06666667,-0.033333335],[0.033333335,-0.033333335,0.06666667,-0.33333334,-0.13333334,-0.10000001,-0.13333334,0.16666667]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"title\":{\"text\":\"Aggregated top 30 moves for neuron L5N1393\"},\"width\":600},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('c0baccb0-6f3b-45c1-a7d3-8155e7511c60');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# This involves a lot of fucky tensor manipulation, no need to follow closely\n",
    "top_moves = neuron_acts.flatten() > neuron_acts.flatten().quantile(0.99)\n",
    "plot_square_as_board(focus_states_flipped_pm1[:, :-1].reshape(-1, 64)[top_moves].float().mean(0).reshape(8, 8), title=f\"Aggregated top 30 moves for neuron L{layer}N{neuron}\", width=600)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can try the same thing for the top 10 neurons in layer 5 (by std)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"472c6e66-bef0-440d-b5c3-3877132ca7c2\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"472c6e66-bef0-440d-b5c3-3877132ca7c2\")) {                    Plotly.newPlot(                        \"472c6e66-bef0-440d-b5c3-3877132ca7c2\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.13333334,0.0,-0.13333334,0.0,0.06666667,0.0,0.0,0.0],[0.16666667,0.0,-0.10000001,-0.26666668,-0.10000001,-0.16666667,0.06666667,0.0],[0.10000001,0.16666667,-0.10000001,-0.3,-0.5666667,-0.43333337,-0.23333335,0.0],[0.0,0.033333335,0.033333335,-0.20000002,-0.9333334,0.0,-0.13333334,0.0],[-0.13333334,-0.06666667,0.23333335,0.0,-0.20000002,-0.53333336,-0.10000001,-0.033333335],[0.0,-0.13333334,0.3,0.23333335,-0.23333335,-0.36666667,-0.13333334,-0.033333335],[-0.033333335,0.0,0.16666667,0.16666667,0.16666667,0.10000001,0.16666667,0.033333335],[0.0,0.0,-0.033333335,0.0,0.10000001,0.0,0.0,-0.033333335]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"1\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.16666667,-0.033333335,-0.10000001,0.0,0.06666667,-0.033333335,0.033333335,0.033333335],[0.16666667,0.16666667,0.033333335,0.10000001,-0.10000001,0.10000001,0.06666667,-0.06666667],[0.23333335,0.23333335,-0.033333335,-0.36666667,0.16666667,0.033333335,-0.16666667,-0.06666667],[0.16666667,0.20000002,0.10000001,0.0,-0.26666668,-0.20000002,-0.06666667,0.0],[0.13333334,0.0,0.10000001,-0.53333336,-0.6666667,-0.70000005,-0.23333335,-0.10000001],[0.16666667,0.16666667,-0.033333335,-0.6666667,0.0,-0.53333336,-0.033333335,0.0],[0.20000002,-0.033333335,-0.26666668,-0.13333334,0.06666667,0.0,-0.06666667,-0.033333335],[-0.033333335,-0.13333334,-0.033333335,0.10000001,0.033333335,0.13333334,0.033333335,0.0]],\"type\":\"heatmap\",\"xaxis\":\"x2\",\"yaxis\":\"y2\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"2\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.0,-0.13333334,-0.3,-0.10000001,-0.10000001,-0.13333334,-0.40000004,-0.033333335],[0.16666667,-0.033333335,0.20000002,-0.16666667,-0.3,-0.5666667,-0.6,0.23333335],[0.0,-0.06666667,0.23333335,0.33333334,-0.70000005,0.0,-0.5,0.06666667],[-0.16666667,-0.26666668,-0.23333335,-0.13333334,-0.4666667,-0.6666667,-0.36666667,-0.20000002],[-0.10000001,0.0,-0.16666667,0.06666667,-0.06666667,0.13333334,-0.4666667,-0.13333334],[0.10000001,-0.13333334,0.3,0.20000002,0.06666667,0.53333336,0.0,0.0],[-0.33333334,0.20000002,0.10000001,0.10000001,-0.20000002,0.10000001,0.3,-0.13333334],[0.13333334,-0.033333335,0.10000001,0.0,-0.16666667,0.13333334,-0.16666667,-0.23333335]],\"type\":\"heatmap\",\"xaxis\":\"x3\",\"yaxis\":\"y3\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"3\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.033333335,-0.13333334,0.0,0.0,-0.16666667,0.10000001,-0.06666667,-0.033333335],[0.033333335,0.06666667,0.033333335,0.3,0.16666667,0.10000001,0.06666667,-0.033333335],[0.10000001,-0.06666667,0.33333334,0.16666667,-0.3,0.23333335,0.23333335,0.13333334],[0.10000001,0.23333335,-0.26666668,-0.13333334,-0.26666668,0.10000001,0.20000002,-0.13333334],[0.033333335,0.0,-0.6333334,-0.9333334,-0.73333335,0.0,-0.40000004,0.0],[-0.10000001,-0.16666667,-0.20000002,0.0,-0.26666668,-0.13333334,0.033333335,0.10000001],[-0.033333335,-0.13333334,0.0,-0.033333335,0.20000002,0.033333335,0.0,0.033333335],[-0.033333335,0.0,-0.10000001,0.3,0.0,0.16666667,0.0,0.16666667]],\"type\":\"heatmap\",\"xaxis\":\"x4\",\"yaxis\":\"y4\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"4\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.033333335,0.0,-0.13333334,0.0,0.0,0.033333335,-0.10000001,-0.033333335],[-0.10000001,0.06666667,0.0,0.033333335,0.13333334,0.23333335,-0.10000001,0.0],[-0.06666667,0.0,-0.23333335,0.0,0.10000001,-0.033333335,-0.23333335,-0.033333335],[-0.10000001,-0.13333334,-0.10000001,0.13333334,-0.6,-0.16666667,-0.16666667,0.06666667],[0.06666667,0.033333335,0.06666667,-0.26666668,-0.53333336,0.0,-0.40000004,0.0],[0.033333335,0.06666667,0.3,-0.10000001,-0.6333334,-0.5666667,-0.43333337,-0.06666667],[0.06666667,0.10000001,-0.10000001,-0.06666667,-0.33333334,-0.16666667,-0.20000002,0.0],[0.06666667,0.0,-0.033333335,0.13333334,-0.06666667,0.033333335,0.13333334,-0.033333335]],\"type\":\"heatmap\",\"xaxis\":\"x5\",\"yaxis\":\"y5\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"5\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.10000001,0.0,0.13333334,-0.20000002,-0.16666667,-0.20000002,-0.13333334,0.0],[-0.10000001,0.06666667,-0.033333335,-0.53333336,-0.26666668,-0.033333335,0.0,0.033333335],[0.06666667,-0.33333334,-0.3,0.0,-0.16666667,-0.26666668,-0.20000002,0.10000001],[-0.20000002,-0.20000002,-0.5666667,-0.73333335,-0.33333334,-0.3,-0.36666667,0.06666667],[-0.06666667,-0.10000001,0.16666667,-0.33333334,-0.13333334,0.16666667,0.06666667,0.06666667],[-0.06666667,0.33333334,-0.53333336,0.16666667,0.06666667,0.40000004,0.06666667,0.0],[0.16666667,-0.06666667,0.033333335,0.10000001,0.06666667,0.26666668,0.20000002,-0.033333335],[-0.10000001,-0.06666667,-0.06666667,-0.10000001,-0.10000001,0.033333335,0.16666667,0.13333334]],\"type\":\"heatmap\",\"xaxis\":\"x6\",\"yaxis\":\"y6\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"6\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.033333335,-0.06666667,-0.10000001,-0.06666667,0.13333334,0.0,0.0,0.0],[0.10000001,0.13333334,-0.13333334,-0.10000001,0.3,0.0,-0.06666667,0.0],[0.0,-0.10000001,-0.43333337,-0.33333334,-0.033333335,-0.033333335,0.10000001,0.10000001],[0.0,0.06666667,0.0,-0.73333335,-0.33333334,-0.10000001,-0.10000001,-0.13333334],[0.033333335,-0.23333335,-0.6,-0.86666673,-0.20000002,-0.43333337,0.0,-0.10000001],[0.16666667,0.033333335,0.20000002,0.13333334,0.33333334,0.16666667,-0.06666667,0.033333335],[0.06666667,-0.13333334,0.033333335,0.033333335,0.0,-0.13333334,0.16666667,-0.06666667],[0.0,-0.13333334,0.13333334,0.26666668,0.06666667,0.033333335,0.0,0.13333334]],\"type\":\"heatmap\",\"xaxis\":\"x7\",\"yaxis\":\"y7\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"7\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.033333335,0.06666667,0.0,0.0,0.0,0.0,0.0,0.0],[0.06666667,0.0,0.10000001,-0.06666667,0.0,-0.06666667,-0.06666667,-0.033333335],[0.0,0.033333335,-0.13333334,0.033333335,-0.16666667,-0.20000002,-0.23333335,-0.033333335],[0.0,-0.20000002,-0.33333334,-0.26666668,-0.13333334,-0.33333334,-0.13333334,-0.033333335],[-0.033333335,-0.10000001,-0.13333334,-0.13333334,1.0,0.20000002,-0.06666667,-0.06666667],[0.0,-0.033333335,-0.20000002,-0.16666667,0.0,-0.10000001,-0.16666667,-0.20000002],[0.033333335,0.033333335,-0.16666667,-0.10000001,-0.10000001,-0.13333334,-0.13333334,-0.06666667],[0.0,0.0,-0.033333335,-0.10000001,-0.033333335,-0.10000001,0.033333335,0.0]],\"type\":\"heatmap\",\"xaxis\":\"x8\",\"yaxis\":\"y8\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"8\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.0,-0.10000001,0.033333335,0.10000001,0.06666667,-0.033333335,0.06666667,0.033333335],[-0.033333335,0.16666667,-0.033333335,0.20000002,0.3,0.10000001,-0.06666667,0.06666667],[0.033333335,0.033333335,0.0,-0.73333335,-0.33333334,0.3,0.13333334,-0.20000002],[-0.06666667,-0.033333335,-0.4666667,-0.86666673,-0.6,0.26666668,-0.033333335,-0.06666667],[0.033333335,0.3,-0.33333334,0.26666668,0.0,-0.20000002,0.13333334,0.06666667],[0.16666667,0.16666667,0.10000001,-0.033333335,0.06666667,0.26666668,0.10000001,0.033333335],[0.0,0.23333335,-0.36666667,-0.13333334,-0.26666668,0.16666667,0.033333335,0.06666667],[-0.033333335,-0.033333335,-0.16666667,-0.13333334,-0.16666667,-0.06666667,-0.033333335,0.033333335]],\"type\":\"heatmap\",\"xaxis\":\"x9\",\"yaxis\":\"y9\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"9\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[-0.033333335,-0.16666667,0.0,0.0,0.10000001,0.10000001,0.36666667,0.16666667],[-0.06666667,-0.10000001,-0.10000001,0.23333335,-0.033333335,0.033333335,0.033333335,0.0],[-0.10000001,-0.20000002,0.10000001,0.23333335,0.033333335,-0.43333337,0.0,-0.06666667],[-0.033333335,0.13333334,-0.20000002,0.13333334,-0.40000004,-0.86666673,-0.13333334,0.06666667],[0.0,0.13333334,0.0,0.0,0.13333334,-0.26666668,0.10000001,0.0],[0.10000001,0.33333334,0.0,-0.16666667,0.0,0.06666667,0.20000002,-0.06666667],[0.10000001,0.3,0.10000001,0.0,-0.20000002,-0.13333334,0.10000001,-0.16666667],[-0.10000001,0.20000002,0.16666667,-0.06666667,-0.06666667,-0.06666667,-0.16666667,-0.13333334]],\"type\":\"heatmap\",\"xaxis\":\"x10\",\"yaxis\":\"y10\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,0.08200000000000002],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"xaxis2\":{\"anchor\":\"y2\",\"domain\":[0.10200000000000002,0.18400000000000005],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis2\":{\"anchor\":\"x2\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis3\":{\"anchor\":\"y3\",\"domain\":[0.20400000000000004,0.28600000000000003],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis3\":{\"anchor\":\"x3\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis4\":{\"anchor\":\"y4\",\"domain\":[0.30600000000000005,0.38800000000000007],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis4\":{\"anchor\":\"x4\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis5\":{\"anchor\":\"y5\",\"domain\":[0.4080000000000001,0.4900000000000001],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis5\":{\"anchor\":\"x5\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis6\":{\"anchor\":\"y6\",\"domain\":[0.5100000000000001,0.5920000000000001],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis6\":{\"anchor\":\"x6\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis7\":{\"anchor\":\"y7\",\"domain\":[0.6120000000000001,0.6940000000000002],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis7\":{\"anchor\":\"x7\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis8\":{\"anchor\":\"y8\",\"domain\":[0.7140000000000001,0.796],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis8\":{\"anchor\":\"x8\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis9\":{\"anchor\":\"y9\",\"domain\":[0.8160000000000002,0.8980000000000001],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis9\":{\"anchor\":\"x9\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis10\":{\"anchor\":\"y10\",\"domain\":[0.9180000000000001,1.0],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis10\":{\"anchor\":\"x10\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"annotations\":[{\"font\":{},\"showarrow\":false,\"text\":\"L5N1406\",\"x\":0.04100000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L5N1985\",\"x\":0.14300000000000004,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L5N54\",\"x\":0.24500000000000005,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L5N35\",\"x\":0.3470000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L5N584\",\"x\":0.44900000000000007,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L5N1325\",\"x\":0.5510000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L5N1949\",\"x\":0.6530000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L5N566\",\"x\":0.7550000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L5N1514\",\"x\":0.8570000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L5N295\",\"x\":0.9590000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"}],\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"title\":{\"text\":\"Aggregated top 30 moves for each top 10 neuron in layer 5\"}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('472c6e66-bef0-440d-b5c3-3877132ca7c2');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "top_neurons = focus_cache[\"post\", 5, \"mlp\"].std(dim=[0, 1]).argsort(descending=True)[:10]\n",
    "heatmaps = []\n",
    "for neuron in top_neurons:\n",
    "    neuron = neuron.item()\n",
    "    neuron_acts = focus_cache[\"post\", 5, \"mlp\"][:, :, neuron]\n",
    "    top_moves = neuron_acts.flatten() > neuron_acts.flatten().quantile(0.99)\n",
    "    heatmaps.append(focus_states_flipped_pm1[:, :-1].reshape(-1, 64)[top_moves].float().mean(0).reshape(8, 8))\n",
    "plot_square_as_board(heatmaps, title=f\"Aggregated top 30 moves for each top 10 neuron in layer 5\",facet_col=0, facet_labels=[f\"L5N{n.item()}\" for n in top_neurons])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also try layer 4, but this time let's look at the fraction of the time a cell is blank vs not blank (regardless of the colour) - we some neurons that strikingly only activate when certain cells are blank (and the neighbours are often not blank, showing it's not just eg the neuron activates in the early game when most cells are blank).\n",
    "\n",
    "Layer 4 is late enough that it's probably using the \"is blank\" feature rather than computing it, and I'm curious what these do!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"f3737ce2-f5ed-45ff-ade2-bce0f15deef1\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"f3737ce2-f5ed-45ff-ade2-bce0f15deef1\")) {                    Plotly.newPlot(                        \"f3737ce2-f5ed-45ff-ade2-bce0f15deef1\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.0,0.0,0.033333335,0.0,0.06666667,0.06666667,0.033333335,0.0],[0.033333335,0.06666667,0.16666667,0.3,0.20000002,0.36666667,0.10000001,0.0],[0.10000001,0.13333334,0.70000005,0.5666667,0.70000005,0.40000004,0.23333335,0.06666667],[0.033333335,0.10000001,0.43333337,1.0,1.0,0.36666667,0.16666667,0.033333335],[0.13333334,0.20000002,0.90000004,1.0,1.0,0.6666667,0.10000001,0.0],[0.16666667,0.3,0.6333334,0.0,0.73333335,0.6,0.33333334,0.0],[0.0,0.0,0.16666667,0.3,0.10000001,0.26666668,0.13333334,0.06666667],[0.0,0.0,0.0,0.0,0.033333335,0.033333335,0.033333335,0.033333335]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"1\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.26666668,0.16666667,0.033333335,0.06666667,0.33333334,0.033333335,0.033333335,0.0],[0.10000001,0.4666667,0.40000004,0.43333337,0.53333336,0.43333337,0.033333335,0.06666667],[0.033333335,0.70000005,0.9666667,1.0,0.9333334,0.26666668,0.20000002,0.033333335],[0.16666667,0.13333334,0.0,1.0,1.0,0.70000005,0.3,0.06666667],[0.10000001,0.33333334,0.73333335,1.0,1.0,0.86666673,0.3,0.06666667],[0.06666667,0.16666667,0.4666667,0.26666668,0.6666667,0.6666667,0.20000002,0.06666667],[0.0,0.033333335,0.26666668,0.26666668,0.3,0.20000002,0.16666667,0.10000001],[0.0,0.20000002,0.20000002,0.26666668,0.23333335,0.13333334,0.033333335,0.06666667]],\"type\":\"heatmap\",\"xaxis\":\"x2\",\"yaxis\":\"y2\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"2\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.13333334,0.13333334,0.06666667,0.0,0.10000001,0.10000001,0.06666667,0.033333335],[0.10000001,0.36666667,0.06666667,0.20000002,0.3,0.40000004,0.23333335,0.06666667],[0.10000001,0.5666667,0.73333335,0.6,0.73333335,0.23333335,0.6,0.033333335],[0.033333335,0.3,0.73333335,1.0,1.0,0.7666667,0.43333337,0.13333334],[0.033333335,0.23333335,0.53333336,1.0,1.0,0.0,0.26666668,0.033333335],[0.033333335,0.23333335,0.5,0.8333334,0.90000004,0.9333334,0.40000004,0.23333335],[0.10000001,0.10000001,0.13333334,0.4666667,0.26666668,0.33333334,0.6,0.033333335],[0.06666667,0.06666667,0.06666667,0.10000001,0.10000001,0.06666667,0.13333334,0.36666667]],\"type\":\"heatmap\",\"xaxis\":\"x3\",\"yaxis\":\"y3\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"3\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.033333335,0.0,0.033333335,0.0,0.033333335,0.033333335,0.0,0.0],[0.0,0.16666667,0.23333335,0.16666667,0.10000001,0.13333334,0.033333335,0.033333335],[0.0,0.23333335,0.4666667,0.40000004,0.6333334,0.33333334,0.26666668,0.033333335],[0.0,0.23333335,0.6666667,1.0,1.0,0.73333335,0.13333334,0.0],[0.0,0.16666667,0.0,1.0,1.0,0.6333334,0.10000001,0.0],[0.033333335,0.10000001,0.8000001,0.90000004,0.70000005,0.53333336,0.13333334,0.0],[0.033333335,0.033333335,0.43333337,0.26666668,0.20000002,0.20000002,0.10000001,0.0],[0.0,0.0,0.0,0.033333335,0.0,0.0,0.10000001,0.033333335]],\"type\":\"heatmap\",\"xaxis\":\"x4\",\"yaxis\":\"y4\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"4\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.06666667,0.10000001,0.16666667,0.13333334,0.20000002,0.10000001,0.033333335,0.033333335],[0.20000002,0.23333335,0.26666668,0.4666667,0.3,0.36666667,0.033333335,0.16666667],[0.16666667,0.4666667,0.33333334,0.7666667,0.5666667,0.4666667,0.4666667,0.13333334],[0.23333335,0.20000002,0.86666673,1.0,1.0,0.90000004,0.13333334,0.033333335],[0.26666668,0.53333336,0.6333334,1.0,1.0,0.9333334,0.3,0.13333334],[0.23333335,0.23333335,0.6,0.9666667,0.0,0.5,0.36666667,0.10000001],[0.06666667,0.33333334,0.6666667,0.40000004,0.4666667,0.36666667,0.13333334,0.06666667],[0.033333335,0.40000004,0.06666667,0.13333334,0.0,0.20000002,0.10000001,0.10000001]],\"type\":\"heatmap\",\"xaxis\":\"x5\",\"yaxis\":\"y5\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"5\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.16666667,0.20000002,0.10000001,0.16666667,0.5666667,0.23333335,0.20000002,0.033333335],[0.13333334,0.20000002,0.3,0.16666667,0.8333334,0.53333336,0.40000004,0.26666668],[0.23333335,0.3,0.8000001,0.8000001,0.9666667,0.0,0.4666667,0.06666667],[0.3,0.3,0.5,1.0,1.0,0.9666667,0.6333334,0.26666668],[0.13333334,0.33333334,0.53333336,1.0,1.0,0.8000001,0.3,0.23333335],[0.13333334,0.40000004,0.33333334,0.6666667,0.7666667,0.6666667,0.70000005,0.43333337],[0.33333334,0.20000002,0.20000002,0.6666667,0.36666667,0.5666667,0.5666667,0.26666668],[0.16666667,0.16666667,0.23333335,0.3,0.26666668,0.3,0.20000002,0.20000002]],\"type\":\"heatmap\",\"xaxis\":\"x6\",\"yaxis\":\"y6\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"6\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0],[0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0],[0.0,0.0,0.1724138,0.44827586,0.06896552,0.0,0.0,0.0],[0.0,0.0,0.37931034,1.0,1.0,0.10344827,0.0,0.0],[0.0,0.0,0.51724136,1.0,1.0,0.31034482,0.0,0.0],[0.0,0.0,0.10344827,0.06896552,0.31034482,0.1724138,0.0,0.0],[0.0,0.0,0.0,0.0,0.0,0.03448276,0.0,0.0],[0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0]],\"type\":\"heatmap\",\"xaxis\":\"x7\",\"yaxis\":\"y7\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"7\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.53333336,0.3,0.33333334,0.20000002,0.13333334,0.033333335,0.23333335,0.16666667],[0.10000001,0.5666667,0.23333335,0.36666667,0.40000004,0.53333336,0.3,0.23333335],[0.06666667,0.5666667,0.73333335,0.90000004,0.8000001,0.6666667,0.6,0.26666668],[0.20000002,0.3,0.6666667,1.0,1.0,0.70000005,0.5666667,0.33333334],[0.13333334,0.7666667,0.9333334,1.0,1.0,0.90000004,0.43333337,0.3],[0.10000001,0.7666667,0.0,0.9666667,0.6333334,0.86666673,0.6,0.26666668],[0.5,0.26666668,0.70000005,0.8333334,0.73333335,0.6,0.5666667,0.3],[0.06666667,0.40000004,0.43333337,0.40000004,0.20000002,0.3,0.36666667,0.36666667]],\"type\":\"heatmap\",\"xaxis\":\"x8\",\"yaxis\":\"y8\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"8\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.033333335,0.033333335,0.033333335,0.033333335,0.10000001,0.16666667,0.06666667,0.033333335],[0.033333335,0.20000002,0.23333335,0.40000004,0.40000004,0.5666667,0.23333335,0.033333335],[0.033333335,0.33333334,0.8000001,0.0,0.7666667,0.43333337,0.5,0.13333334],[0.033333335,0.16666667,0.86666673,1.0,1.0,0.7666667,0.4666667,0.26666668],[0.033333335,0.06666667,0.6,1.0,1.0,0.53333336,0.3,0.033333335],[0.033333335,0.3,0.4666667,0.5,0.70000005,0.7666667,0.13333334,0.06666667],[0.10000001,0.06666667,0.13333334,0.40000004,0.36666667,0.33333334,0.16666667,0.033333335],[0.033333335,0.06666667,0.06666667,0.10000001,0.033333335,0.06666667,0.033333335,0.033333335]],\"type\":\"heatmap\",\"xaxis\":\"x9\",\"yaxis\":\"y9\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"9\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.13333334,0.0,0.0,0.0,0.0,0.0,0.0,0.0],[0.033333335,0.43333337,0.0,0.10000001,0.16666667,0.4666667,0.13333334,0.0],[0.033333335,0.26666668,0.73333335,0.8000001,0.9666667,0.6333334,0.33333334,0.0],[0.033333335,0.16666667,0.7666667,1.0,1.0,0.0,0.3,0.06666667],[0.0,0.23333335,0.86666673,1.0,1.0,0.73333335,0.13333334,0.0],[0.0,0.23333335,0.40000004,0.4666667,0.6,0.70000005,0.33333334,0.06666667],[0.06666667,0.0,0.13333334,0.3,0.36666667,0.36666667,0.26666668,0.10000001],[0.0,0.0,0.06666667,0.0,0.0,0.033333335,0.06666667,0.13333334]],\"type\":\"heatmap\",\"xaxis\":\"x10\",\"yaxis\":\"y10\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,0.08200000000000002],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"}},\"xaxis2\":{\"anchor\":\"y2\",\"domain\":[0.10200000000000002,0.18400000000000005],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis2\":{\"anchor\":\"x2\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis3\":{\"anchor\":\"y3\",\"domain\":[0.20400000000000004,0.28600000000000003],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis3\":{\"anchor\":\"x3\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis4\":{\"anchor\":\"y4\",\"domain\":[0.30600000000000005,0.38800000000000007],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis4\":{\"anchor\":\"x4\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis5\":{\"anchor\":\"y5\",\"domain\":[0.4080000000000001,0.4900000000000001],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis5\":{\"anchor\":\"x5\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis6\":{\"anchor\":\"y6\",\"domain\":[0.5100000000000001,0.5920000000000001],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis6\":{\"anchor\":\"x6\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis7\":{\"anchor\":\"y7\",\"domain\":[0.6120000000000001,0.6940000000000002],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis7\":{\"anchor\":\"x7\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis8\":{\"anchor\":\"y8\",\"domain\":[0.7140000000000001,0.796],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis8\":{\"anchor\":\"x8\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis9\":{\"anchor\":\"y9\",\"domain\":[0.8160000000000002,0.8980000000000001],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis9\":{\"anchor\":\"x9\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis10\":{\"anchor\":\"y10\",\"domain\":[0.9180000000000001,1.0],\"matches\":\"x\",\"title\":{\"text\":\"x\"}},\"yaxis10\":{\"anchor\":\"x10\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"annotations\":[{\"font\":{},\"showarrow\":false,\"text\":\"L4N547\",\"x\":0.04100000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N352\",\"x\":0.14300000000000004,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N1122\",\"x\":0.24500000000000005,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N345\",\"x\":0.3470000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N1076\",\"x\":0.44900000000000007,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N691\",\"x\":0.5510000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N540\",\"x\":0.6530000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N112\",\"x\":0.7550000000000001,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N1285\",\"x\":0.8570000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"L4N115\",\"x\":0.9590000000000002,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"}],\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0},\"title\":{\"text\":\"Aggregated top 40 moves for each top 10 neuron in layer 4\"}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('f3737ce2-f5ed-45ff-ade2-bce0f15deef1');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "top_neurons = focus_cache[\"post\", 4, \"mlp\"].std(dim=[0, 1]).argsort(descending=True)[:10]\n",
    "heatmaps = []\n",
    "for neuron in top_neurons:\n",
    "    neuron = neuron.item()\n",
    "    neuron_acts = focus_cache[\"post\", 4, \"mlp\"][:, :, neuron]\n",
    "    top_moves = neuron_acts.flatten() > neuron_acts.flatten().quantile(0.99)\n",
    "    heatmaps.append(focus_states_flipped_pm1[:, :-1].abs().reshape(-1, 64)[top_moves].float().mean(0).reshape(8, 8))\n",
    "plot_square_as_board(heatmaps, title=f\"Aggregated top 40 moves for each top 10 neuron in layer 4\",facet_col=0, facet_labels=[f\"L4N{n.item()}\" for n in top_neurons])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Spectrum Plots\n",
    "\n",
    "One of the best ways to validate a hypothesis about neurons is to use a **spectrum plot**, where we plot a histogram of the neuron activations across the full data distribution (or at least some random sample) and categorise each activation by whether it has the property we think the neuron is detecting. We can do a janky version of that with the neuron's activations on our 50 games (plotting each histogram group as percent of the group size, to normalise for most games not having the configuration hypothesised)\n",
    "\n",
    "Fascinatingly, we see that our hypothesis did *not* fully capture the neuron - almost every high activation had the hypothesised configuration, but so did some low ones! I'm not sure what's going on here, but it reveals a weakness of max activating dataset examples - they don't let you notice when your hypothesis allows false positives!\n",
    "\n",
    "**Exercise**: Look at the moves with this configuration and low activations - what's going on there? Can you see any patterns in the board states? In the moves? What does the neuron activation look like over the course of the game?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"f6c57c79-e970-40b9-a893-78163fc62f34\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"f6c57c79-e970-40b9-a893-78163fc62f34\")) {                    Plotly.newPlot(                        \"f6c57c79-e970-40b9-a893-78163fc62f34\",                        [{\"alignmentgroup\":\"True\",\"bingroup\":\"x\",\"histnorm\":\"percent\",\"hovertemplate\":\"label=False<br>acts=%{x}<br>percent=%{y}<extra></extra>\",\"legendgroup\":\"False\",\"marker\":{\"color\":\"#636efa\",\"pattern\":{\"shape\":\"\"}},\"name\":\"False\",\"nbinsx\":100,\"offsetgroup\":\"False\",\"orientation\":\"v\",\"showlegend\":true,\"x\":[-0.0994749516248703,-0.040831033140420914,-0.037849582731723785,-0.02350456826388836,-0.06045374646782875,-0.04447716847062111,-0.022460559383034706,-0.04276931285858154,-0.02349894307553768,-0.06672587245702744,-0.01675095595419407,-0.0653935968875885,-0.052432458847761154,-0.015340473502874374,-0.012743145227432251,-0.01456527877599001,-0.000035315188142703846,0.02176556922495365,-0.00004870125121669844,-0.09096778184175491,-0.00005358667112886906,-0.015000912360846996,-0.000021486039258888923,-0.061242591589689255,-0.00005666778451995924,-0.10174235701560974,-9.99217081698589e-6,0.03910073637962341,-0.00007878570613684133,-0.050347764045000076,-0.000016181260434677824,-0.018804600462317467,-2.8978138288948685e-6,-0.0578950010240078,-9.211621545546222e-6,-0.1640605330467224,-0.00003189699782524258,-0.07622033357620239,-0.16765756905078888,-0.06405266374349594,-0.10899986326694489,-0.10235440731048584,-0.0017367125255987048,-0.15556453168392181,-0.00008456587238470092,-0.0066217961721122265,-0.00016569506260566413,-0.0048852660693228245,-0.010269173420965672,-0.012632964178919792,-0.00016051447892095894,-0.07012374699115753,-0.0005644456832669675,-0.09322268515825272,-0.00047811365220695734,-0.01896774396300316,-0.00005263679486233741,-0.03683693706989288,-0.000011947190614591818,-0.08869710564613342,-0.01982218213379383,-0.0033522772137075663,-0.003072125604376197,-0.004874704405665398,-0.0031896373257040977,-0.002478962065652013,-0.002257024869322777,-0.00948870275169611,-0.002480417490005493,-0.013836409896612167,-0.0002910201728809625,-0.03075476363301277,-0.0005813640891574323,-0.15559197962284088,-0.00022794453252572566,-0.15887483954429626,-0.001608251011930406,-0.1463419497013092,-0.011292094364762306,-0.011558829806745052,-0.0007050971616990864,-0.16946348547935486,-0.0010542911477386951,-0.002151700435206294,-0.13086386024951935,-0.0011268628295511007,-0.06242106109857559,-0.014053807593882084,-0.15610487759113312,-0.0008866770076565444,-0.11757298558950424,-0.00008040692046051845,-0.005561224650591612,-0.000037385481846285984,-0.0011740820482373238,-3.996393843408441e-6,-3.180650764988968e-7,-0.0020229234360158443,-1.630281616371576e-7,-0.003998391330242157,-1.4882302821206395e-6,-0.006506683304905891,-1.915532266139053e-6,-0.01892731711268425,-3.5850646327162394e-6,-0.07436838746070862,-0.00036182525218464434,-0.10639647394418716,-0.006137173157185316,-0.007713152561336756,-0.004420464392751455,-0.15375865995883942,-0.0,-0.04134141281247139,-3.5851442135026446e-6,-9.206989489030093e-6,-0.05234450101852417,-0.0035900138318538666,-0.08975198864936829,-0.02078271098434925,-0.003744447836652398,-0.005707193166017532,-0.003451436059549451,-0.006198232062160969,-0.008332663215696812,-0.0017458045622333884,-0.018767258152365685,-0.011044573038816452,-0.00921060424298048,-0.014474730007350445,-0.005683020688593388,-0.0028143927920609713,-0.008875354193150997,-0.0025167374406009912,-0.16222815215587616,-0.005500220227986574,-0.017108872532844543,-0.0012299972586333752,-0.15453492105007172,-0.05415106192231178,-0.0002267663221573457,-0.08915375173091888,-0.0001677778345765546,-0.06667321920394897,-0.000031528648833045736,-0.05642063170671463,-0.00025989735149778426,-0.0045698643662035465,-0.0003568514948710799,-0.009877287782728672,-0.15038685500621796,-0.0000239883938775165,-0.1554926186800003,-0.0005248492816463113,-0.16179755330085754,-0.009806104935705662,-0.0015394907677546144,-0.001748915994539857,-0.12000853568315506,-0.011105473153293133,-0.14771626889705658,-0.004666333552449942,-0.0025900485925376415,-0.007086616940796375,-0.00247074943035841,-0.014672591350972652,-0.05374613776803017,-0.01072482205927372,-0.09634537249803543,-0.01840553991496563,-0.1552756130695343,-6.559177109011216e-6,-0.13639339804649353,-1.9125602648273343e-6,-0.002053825417533517,-0.0030633793212473392,-0.0032155723311007023,-0.10032095015048981,-0.005327953491359949,-0.004637961275875568,0.011868196539580822,-0.0002716794260777533,-0.09759119898080826,-0.0010586067801341414,-0.028423428535461426,-0.00043863197788596153,-0.0718734934926033,-0.0010614281054586172,-0.07288790494203568,-0.0000938677549129352,-0.155915305018425,-9.600399607734289e-6,-0.16019965708255768,-0.00004953472671331838,-0.000025978155463235453,-0.15167929232120514,-0.00008905075810616836,-0.000873959215823561,-0.16959728300571442,-0.00006503276381408796,-0.0002492493367753923,-0.16949103772640228,-0.00018377756350673735,-0.0007472794386558235,-0.12233322858810425,-0.000025975772587116808,-0.16995593905448914,-0.00026467940188013017,-0.11763319373130798,-0.00012447463814169168,-9.131557590080774e-7,-0.1695985049009323,-9.077655704459175e-6,-0.01586679369211197,-4.661978323383664e-7,-0.1091347187757492,-0.0,0.1715085655450821,-0.0,-0.03573458641767502,-1.6124491253322049e-7,-0.1691867560148239,-6.151483376015676e-7,-0.16651038825511932,-0.00011117319809272885,-0.16277767717838287,-4.6701441647201136e-7,-0.12446428835391998,-2.755948798949248e-6,-0.1311688870191574,-2.4788630526018096e-6,-0.0733066275715828,-0.0,-0.1346631497144699,-7.654285809621797e-7,-0.0994749516248703,-0.040831033140420914,-0.06825675070285797,-0.001115567283704877,0.27651453018188477,-0.0015080190496519208,-0.0006925825728103518,-0.0008660090970806777,-0.08245646953582764,-0.00024700743961147964,0.17673474550247192,-0.0006661233492195606,-0.15035933256149292,-0.00020595002570189536,0.0749591812491417,-0.00002423638397885952,0.06156494840979576,-0.0008686951478011906,0.0801873430609703,-0.00569066870957613,-0.025794604793190956,-0.0021563097834587097,0.3380724787712097,-0.0004430081171449274,-0.00988294929265976,-0.1668841391801834,-0.0036782887764275074,-0.0043170396238565445,-0.005081648007035255,-0.001828710432164371,-0.006825129967182875,-0.016980059444904327,-0.14619219303131104,-0.00475709093734622,-0.13034126162528992,-0.0020103901624679565,-0.1568380892276764,-5.086237251816783e-6,-0.16896283626556396,-0.00003786912202485837,-0.16801409423351288,-0.00032960527460090816,-0.001213245326653123,-0.017719566822052002,-0.00014557529357261956,-0.012561742216348648,-0.0005811695591546595,-3.5873028991773026e-6,-0.013603003695607185,-0.000058322922996012494,-0.07708345353603363,-0.001204348518513143,-0.03606544807553291,-0.0004131979076191783,-0.07888808101415634,-0.0007517414633184671,-0.03616613522171974,-0.00016986893024295568,-0.030027464032173157,-0.0994749516248703,-0.006075781304389238,0.029397575184702873,-0.003457670798525214,-0.012195388786494732,-0.005139016546308994,-0.04663686081767082,-0.0008003428811207414,-0.0013193907216191292,-0.0010547568090260029,-0.00034938313183374703,-0.0017228019423782825,-0.00026701317983679473,-0.00004989358058082871,-0.0007916481699794531,-0.16136254370212555,-0.08670277148485184,-0.10972599685192108,-0.00026754444115795195,0.832774817943573,-0.02335476689040661,-0.005216208752244711,-0.042227424681186676,-0.055610984563827515,-0.02467246539890766,-0.019077055156230927,-0.0022778569255024195,-2.337399109819671e-6,-0.019861867651343346,-0.00005536617754842155,-0.014108770526945591,-0.00006326930451905355,-0.020435282960534096,-0.00020876675262115896,-0.033843670040369034,-0.0005808648420497775,-0.03530467674136162,-0.00042026196024380624,-0.03785939887166023,-0.00005713753489544615,-0.029447121545672417,-0.0004871138371527195,-0.03742193803191185,-0.00011298232129774988,-0.012111120857298374,-0.00009100641909753904,-0.009225880727171898,-7.761249435134232e-6,-0.03644084185361862,-0.0003653407620731741,-0.06026551127433777,-0.00005702310227206908,-0.014593012630939484,-0.08975198864936829,-0.03872188925743103,-0.01437250804156065,-0.024228742346167564,-0.032652050256729126,-0.04506545141339302,-0.042149618268013,-0.005212026182562113,0.31903839111328125,-0.001807068707421422,0.4072635769844055,-0.00763651542365551,0.46332409977912903,-0.004378374665975571,0.0797814130783081,-0.0016774083487689495,0.013120652176439762,-0.0041353595443069935,-0.054434455931186676,-0.0060171084478497505,0.12997403740882874,-0.015828633680939674,0.006762783043086529,-0.021250953897833824,-0.1671338528394699,-0.0023430832661688328,-0.050512198358774185,-0.0010523144155740738,-0.0028798263520002365,-0.14695948362350464,-0.003920072689652443,-0.16773155331611633,-0.001851295935921371,-0.001687424723058939,-0.16990618407726288,-0.0002611721574794501,-0.1673770397901535,-0.0004594747442752123,-0.0016993159661069512,-0.08068826049566269,-0.002806809265166521,-0.04886126145720482,-0.0009628352127037942,-0.02886548452079296,-0.0013762953458353877,-0.03104575350880623,-0.0007582454709336162,-0.03232438117265701,-0.0007973500760272145,-0.054999493062496185,-0.00469388859346509,-0.13410457968711853,-0.0,-0.136622354388237,-0.0,-0.16037358343601227,-0.0,-0.0685378685593605,-0.08869710564613342,-0.01982218213379383,-0.020223468542099,-0.0011384328827261925,-0.03035975992679596,-0.00390877528116107,-0.006628670729696751,-0.0005668348167091608,-0.018559018149971962,-0.003112244652584195,-0.03863587602972984,-0.0064580123871564865,-0.011438556015491486,-0.009808343835175037,-0.020939959213137627,-0.024843519553542137,-0.014123443514108658,-0.022733967751264572,-0.005255952011793852,-0.020322587341070175,-0.009152695536613464,-0.020724976435303688,-0.005696794483810663,-0.018471766263246536,-0.026432054117321968,-0.01744786463677883,-0.02699507214128971,-0.009859739802777767,-0.00880223885178566,-0.010277573019266129,-0.005639373790472746,-0.0063947755843400955,-0.02059328928589821,-0.0039013491477817297,-0.0158520694822073,-0.020959297195076942,-0.026403099298477173,-0.007548340130597353,-0.044095251709222794,-0.003417189000174403,-0.037990421056747437,-0.016010286286473274,0.10856140404939651,-0.00009019904246088117,-0.08837360888719559,-0.004483725409954786,0.048309892416000366,-0.005502835847437382,-0.10334507375955582,-0.166169673204422,-0.010415257886052132,-0.039453838020563126,-0.07621566951274872,-0.07575079798698425,-0.003369403537362814,-0.10032095015048981,-0.03138298913836479,-0.0072553204372525215,-0.0027623637579381466,-0.000953344686422497,-0.0011673952685669065,-0.0011991549981757998,-0.002263339003548026,-0.0019415117567405105,-0.16921497881412506,-0.00958272349089384,0.028447771444916725,-0.009780279360711575,-0.15577514469623566,-0.009072947315871716,-0.15142090618610382,-0.00854464154690504,-0.1631889045238495,-0.0020228384528309107,-0.14403721690177917,-0.00045744440285488963,0.05503314360976219,-0.0009061607997864485,0.0382310189306736,-0.003252489259466529,-0.009003305807709694,-0.000042941570427501574,-0.07941798120737076,-7.226847628771793e-6,-0.055694252252578735,-0.0,-0.09845969080924988,-0.0006595377344638109,-0.00030658385367132723,-0.005144450347870588,-0.0008240483002737164,-0.0015570019604638219,-0.0005704229115508497,-0.002519899746403098,-0.0006316719227470458,-0.0034404322504997253,-0.00010083925008075312,-0.0014530449407175183,-0.00016359736036974937,-0.001379218534566462,-0.0011333953589200974,-0.0003261615929659456,-0.0008597858832217753,-0.002477846806868911,-0.002872600918635726,-0.0007776221609674394,-0.0006401557475328445,-0.0018852524226531386,-0.00024700682843104005,-0.00024316192138940096,-3.7257982512528542e-6,-0.06603401899337769,-0.00002423607475066092,-0.011300203390419483,-0.0994749516248703,-0.006075781304389238,0.029397575184702873,-0.0014909851597622037,-0.0007103676325641572,-0.0007842057966627181,-0.13595998287200928,-0.0006574794533662498,-0.10547688603401184,-0.00014190602814778686,0.15802019834518433,-0.0008444361155852675,-0.008759434334933758,-0.07510103285312653,-0.0031131631694734097,0.040864814072847366,-0.0017983693396672606,-0.16960549354553223,-0.0016273852670565248,-0.16942743957042694,-0.0005170205840840936,-0.1638237088918686,-0.004546314012259245,-0.15412120521068573,-0.046340372413396835,-0.0036909766495227814,-0.02651698887348175,-0.018136141821742058,-3.720470203916193e-6,-0.029660925269126892,-0.0005374782485887408,-0.12014071643352509,-0.002206259872764349,-0.06914997100830078,-0.0004555158957373351,-0.13266733288764954,-0.0012050848454236984,-0.15432825684547424,-0.0012206536484882236,-0.16650159657001495,-0.0022896062582731247,-0.08794429898262024,-0.013745359145104885,-0.07836046814918518,-0.0015010465867817402,-0.13088594377040863,-0.009844641201198101,-0.08885370939970016,-0.03463168442249298,-0.08390650153160095,-0.00008352322765858844,-0.15563581883907318,-0.000035679677239386365,-0.1168014332652092,-0.00014135117817204446,-0.15179890394210815,-3.1388449883706926e-7,-0.0181646216660738,-0.10032095015048981,-0.03565492480993271,-0.03239725902676582,-0.018476076424121857,-0.04690119996666908,-0.003807461354881525,0.06624401360750198,-0.002274313010275364,0.03659334033727646,-0.005080401431769133,0.1873576045036316,-0.0015468017663806677,-0.01442552637308836,-0.0021800389513373375,0.3475777506828308,-0.003997827414423227,-0.12588874995708466,-0.0001439102052245289,-0.07688421756029129,-0.0001179450991912745,-0.15291784703731537,-0.0015603454085066915,-0.16841693222522736,-0.0003345058357808739,-0.13182109594345093,-0.000714081630576402,-0.018045518547296524,-0.000775303749833256,-0.1699569672346115,-0.0000495386884722393,-0.16976140439510345,-0.00025585791445337236,-0.16579672694206238,-0.00024668738478794694,-0.1355428844690323,-0.00057400664081797,-0.10881264507770538,-0.14615702629089355,-0.0010524129029363394,-3.1702284104540013e-6,-0.0007948337588459253,-0.0028164975810796022,-0.00390207814052701,-0.0034644806291908026,-0.03130194544792175,-0.02867746539413929,-0.007832013070583344,-0.00008882497058948502,-0.00566863315179944,-0.00023974216310307384,-0.0002693503338377923,-0.0011822639498859644,-0.08869710564613342,-0.04722870513796806,-0.0247566569596529,-0.012783029116690159,-0.004635517485439777,-0.004257663153111935,-0.0023125673178583384,-0.013399508781731129,-0.0030657935421913862,-0.009144618175923824,-0.017046181485056877,-0.0029533703345805407,-0.01741550676524639,-0.003077375004068017,-0.00011659480514936149,-0.14321494102478027,-0.00009341137774754316,-0.16957898437976837,-0.0003853660309687257,-0.000079480538261123,-0.169808492064476,-0.00002361220322200097,-0.14541013538837433,-0.00002510847116354853,-0.1083352267742157,-0.00006783995195291936,-0.14906542003154755,-0.000011039153832825832,-0.16996455192565918,-0.0004337390710134059,-0.000048102461732923985,-0.15168677270412445,-0.0,-0.16855691373348236,-0.0,-0.0003311683831270784,-0.0038904438260942698,-0.004615252837538719,-0.0030219233594834805,-0.0034821939188987017,-0.006120528094470501,-0.0032745576463639736,-0.00700854416936636,-0.001147008966654539,-0.0026336621958762407,-0.007403645198792219,-0.002739471383392811,-0.0027814311906695366,-3.587741275623557e-6,-0.14567571878433228,-5.48989737580996e-6,-0.02264273166656494,-0.0002075755037367344,-0.004457900300621986,-0.000010516337169974577,-0.029324153438210487,-0.000120763826998882,-0.008390874601900578,-0.0003419985587242991,-0.08975198864936829,-0.03872188925743103,-0.03621615096926689,-0.06449770927429199,-0.02900325134396553,-0.044677622616291046,-0.02207629196345806,-0.0009724968113005161,0.41232118010520935,-0.001174550037831068,0.16238130629062653,-0.0029424838721752167,0.29651975631713867,-0.0044443365186452866,0.07763539999723434,-0.0032154126092791557,0.06356524676084518,-0.019636699929833412,0.1621241271495819,-0.006800199393182993,0.07809185236692429,-0.020972726866602898,0.1377103328704834,-0.010695137083530426,0.1348220258951187,-0.0016667552990838885,-0.007045492064207792,0.06751921772956848,-0.010878664441406727,-0.02282458357512951,-0.16196735203266144,0.05066274479031563,-0.04655927047133446,0.006565417163074017,-0.16598230600357056,-0.061638232320547104,0.052207112312316895,-0.029741451144218445,0.13019829988479614,-0.07250774651765823,-0.10959961265325546,-0.03372354432940483,-0.16395433247089386,-2.0540962850645883e-6,-0.143667533993721,-0.007353185210376978,-0.10779052972793579,-0.0006551181431859732,-0.12339092046022415,-0.0017995198722928762,-0.15919479727745056,-0.003809402696788311,-0.0075582233257591724,-0.010306285694241524,-1.917394001793582e-6,-0.08975198864936829,-0.02078271098434925,-0.00007704446761636063,-0.0005803697276860476,-0.16589105129241943,-0.0001360058959107846,-0.13911278545856476,-0.00003944594936911017,-0.1359616219997406,-3.4503109418437816e-6,-0.0005558651173487306,-5.220307684794534e-6,-0.15551500022411346,-0.0014577744295820594,-0.0002703039499465376,-0.00028205220587551594,-0.06422247737646103,-0.0013826022623106837,-0.041617270559072495,-0.0013845153152942657,-0.012568153440952301,-0.0023452304303646088,-0.01223732065409422,-0.0003461590385995805,-0.0574205182492733,-0.006699078716337681,-0.0924932137131691,-0.013683808967471123,-0.07489858567714691,-0.0012062008026987314,-0.01763244718313217,-0.04160136356949806,-0.02716800384223461,-0.024663198739290237,-0.010267375968396664,-0.030572209507226944,-0.014163622632622719,-0.05465042591094971,-0.022277718409895897,-0.0010656522354111075,-0.06616140902042389,-0.0020215248223394156,-0.02583182044327259,-0.00011411284503992647,-0.031149545684456825,-0.00042250772821716964,-0.017302818596363068,-0.013789757154881954,-0.022421881556510925,-0.0002494610089343041,-0.009445959702134132,-0.0005128968041390181,-0.0043998174369335175,-0.0013655887451022863,-0.004893876612186432,-0.00016745210450608283,-0.08869710564613342,-0.038724638521671295,-0.034958966076374054,-0.02839924581348896,-0.04772619530558586,-0.028252145275473595,-0.0772920623421669,-0.07073365896940231,-0.06093808636069298,-0.05133673921227455,-0.06561626493930817,-0.09248460829257965,-0.05018383264541626,-0.09492089599370956,-0.06180070340633392,-0.08456553518772125,-0.027834953740239143,-0.07014986872673035,-0.11187300831079483,-0.0763499066233635,-0.11126285046339035,-0.10624871402978897,-0.0576748326420784,-0.14961901307106018,-0.04330691695213318,-0.13740020990371704,-0.041559599339962006,-0.04528677463531494,-0.04849293455481529,-0.046936433762311935,-0.047910455614328384,-0.03294341638684273,-0.022906122729182243,-0.017809946089982986,-0.0003703034308273345,-0.16984739899635315,-0.0005556705291382968,-0.1628274917602539,-0.00018759335216600448,-0.13098391890525818,-0.000010124813343281858,-0.010771266184747219,-8.817313755571377e-6,-0.005943000316619873,-2.896770638471935e-6,-0.009978033602237701,-3.1424690405401634e-7,-0.04889901354908943,-3.160288031267555e-7,-0.005655081011354923,-0.005424860864877701,-0.001529101049527526,-7.661986956009059e-7,-0.14621314406394958,-0.00013276613026391715,-0.00027993725962005556,-0.024590207263827324,-7.656022944502183e-7,-0.005678883288055658,-0.08869710564613342,-0.01982218213379383,-0.020223468542099,-0.0016350166406482458,-0.008013221435248852,-0.0009007207700051367,-0.0183244775980711,-0.0019089066190645099,-0.01993507333099842,-0.004808733705431223,-0.021075384691357613,-0.0003723691334016621,0.1703152060508728,-0.0006835826206952333,-0.10278074443340302,-0.002779832109808922,-0.0005688308156095445,-0.07514063268899918,-0.0051713064312934875,0.02412259206175804,-0.0024455662351101637,-0.00014968444884289056,-0.0030691225547343493,-0.16539080440998077,-0.0041176509112119675,-0.00010561274393694475,-0.1688171774148941,-0.0,-0.005030418746173382,-0.025463053956627846,-0.035696811974048615,-0.00677786348387599,-0.02653488703072071,-0.022187598049640656,-0.00680592842400074,-0.05125146731734276,-0.000023236783817992546,-0.005384998861700296,-0.0002147061750292778,-0.004989252425730228,-0.000511091435328126,-0.16590969264507294,-0.0,-0.003522084793075919,-0.001395655795931816,-0.0009114041458815336,-0.0005667399382218719,-0.0007573695038445294,-0.0011405751574784517,-0.0001633772044442594,-0.002084155799821019,-0.0005308616091497242,-0.010355222970247269,-0.0005610580556094646,-0.00010935858881566674,-0.033886589109897614,-0.00006303309055510908,-0.043502327054739,-0.0001873731380328536,-0.08869710564613342,-0.04722870513796806,-0.03120114654302597,-0.007698928005993366,-0.0047930506989359856,-0.0005509782349690795,-0.0008484646095894277,-0.0016107666306197643,-0.0034862880129367113,-0.005902616307139397,-0.002804107265546918,-0.005069971550256014,-0.007202997803688049,-0.000553274352569133,-0.0008164448663592339,-0.16303284466266632,-0.00037825386971235275,-0.00005334380330168642,-0.1543634980916977,-0.00045175530249252915,-0.13761557638645172,-4.271749730833108e-6,0.07516948133707047,-7.624911631864961e-6,-0.015734003856778145,-0.15495452284812927,-0.0022592726163566113,-0.13660664856433868,-0.001417184597812593,-0.07369414716959,-0.0016519654309377074,-0.14970238506793976,-0.0021610937546938658,-0.0856795534491539,-0.005154158920049667,-0.00025074020959436893,-0.13840998709201813,-0.15036863088607788,-0.16581577062606812,-0.16996803879737854,-0.10005441308021545,-0.16990616917610168,-0.13970030844211578,-0.00032323156483471394,-0.000700996199157089,-0.1191231831908226,-0.013331618160009384,-0.002418820047751069,-0.0037349469494074583,-0.003219049656763673,-0.0008224142366088927,-0.0023403209634125233,-0.0075263832695782185,-0.002834259532392025,-0.001371809747070074,-0.00026351079577580094,-0.08869710564613342,-0.04722870513796806,-0.0247566569596529,-0.039180368185043335,-0.024532878771424294,-0.025786850601434708,-0.030598236247897148,-0.02396601438522339,-0.020044006407260895,-0.04997578263282776,-0.0037751190830022097,-0.011539326049387455,-0.0021702249068766832,-0.0174567773938179,-0.0002588373317848891,-0.1188872754573822,-0.10846783965826035,-0.16019196808338165,-0.0883355513215065,-0.13702015578746796,-0.14122767746448517,-0.16099335253238678,-0.15705335140228271,-0.1685018390417099,-0.1698680967092514,-0.1540597379207611,-0.11399335414171219,-0.14719656109809875,-0.15983694791793823,-0.16206061840057373,-0.11677887290716171,-0.12003161013126373,-0.16107669472694397,-0.06772704422473907,-0.15882766246795654,-0.16918976604938507,-0.003105095587670803,-0.00035259645665064454,-0.00022353968233801425,-0.0011371261207386851,-0.00017393016605637968,-0.0001941250666277483,-0.0020935803186148405,-0.046615440398454666,-0.14927753806114197,-0.000258729443885386,-0.025003807619214058,-0.0001525579282315448,-0.010805714875459671,-0.00008248432277468964,-0.0033149346709251404,-3.0311152841022704e-6,-0.0994749516248703,-0.03566031530499458,-0.009023123420774937,-0.030039871111512184,-0.004936165641993284,-0.005055260844528675,-0.00019890244584530592,-0.0006976675940677524,-0.16971349716186523,-0.0001076589323929511,-0.14714711904525757,-0.0006662198575213552,-0.0627046525478363,-0.15316475927829742,-0.0005914978682994843,-0.0007618339732289314,-0.0003163334622513503,-0.00006479376315837726,-0.0001237977558048442,-0.0006504936027340591,-0.0013350865337997675,-0.000016181707906071097,-0.00010618023225106299,-0.00003823180668405257,-0.0005598551942966878,-0.00011546588211786002,-0.00053617381490767,-0.0004478977934923023,-0.0000439074938185513,-0.09210390597581863,-0.00017228581418748945,-2.7570242764340946e-6,-0.16612417995929718,-7.225541139632696e-6,-0.0010933959856629372,-0.02565351128578186,-0.0026926028076559305,-0.007264452055096626,-0.0030871969647705555,-0.0021662262734025717,-0.0012873938539996743,-0.0025406379718333483,-0.0010381144238635898,-0.00042781990487128496,-0.08869710564613342,-0.01982218213379383,-0.006078963167965412,-0.005755342077463865,-0.004368909168988466,-0.0015335263451561332,-0.004923274274915457,-0.004967792890965939,-0.006207901518791914,-0.00012772569607477635,-0.003183753229677677,-0.16417196393013,-0.001201375387609005,-0.16165104508399963,-0.00368300243280828,-0.0004522664821706712,0.00361133785918355,-0.00023470536689274013,-0.11003574728965759,-0.000026225268811685964,-0.1693991720676422,-0.00007855003786971793,-0.16997113823890686,-0.00024369367747567594,-0.16917362809181213,-0.0002807847922667861,-0.14614862203598022,-0.00020951847545802593,-0.12363498657941818,-0.00009615800809115171,-0.1542702317237854,-0.00004965339758200571,-0.14620113372802734,-0.00044393102871254086,-0.00013779036817140877,-0.165136456489563,-0.0003053269465453923,-0.1086052805185318,-0.16975845396518707,-0.1409914791584015,-0.1311889886856079,-0.05783191695809364,-0.0006993317510932684,-0.001720838132314384,-0.1634165197610855,-0.09760911762714386,-0.12409203499555588,-0.0016208267770707607,-0.023458803072571754,-0.002170140389353037,-0.012205089442431927,-0.00047204928705468774,-0.07243352383375168,-8.286971933557652e-6,-0.0226996261626482,-0.000013878658137400635,-0.10032095015048981,-0.005327953491359949,-0.03377970680594444,-0.00032480029040016234,-0.1436527669429779,-0.00001452315609640209,-0.0007654185756109655,-0.14115089178085327,-0.000018589555111248046,-0.16919206082820892,-0.000027461277568363585,-0.1514052450656891,-0.14811934530735016,-0.1186872199177742,-0.12229486554861069,-0.0602911077439785,-0.00009421521826880053,-0.00001884562152554281,-0.0757092759013176,-0.002506659831851721,-9.090595085581299e-7,0.030185354873538017,-1.6292717930355138e-7,0.5095576047897339,-1.490036765972036e-6,0.2645094096660614,-1.6187874507522793e-7,0.8663956522941589,-3.134653638880991e-7,0.2986883223056793,-0.0,0.1514393389225006,-0.0004495210596360266,-0.08730895817279816,-0.001065179705619812,-0.04623307287693024,-0.0008336714818142354,-0.07389877736568451,-0.005800608079880476,-0.07648290693759918,-0.002577023347839713,-0.05368112027645111,-0.006767068058252335,-0.017119014635682106,-0.01634429581463337,-0.0071458774618804455,-0.054453205317258835,-0.023532580584287643,-0.034437622874975204,-0.002310414332896471,-0.0016893043648451567,-0.00020649067300837487,-0.0001801703911041841,-0.0004547028511296958,-0.0000947878579609096,-0.0005922916461713612,-0.00010595300409477204,-0.0007015776936896145,-0.08975198864936829,-0.03872188925743103,-0.016192691400647163,-0.021477920934557915,-0.01786135323345661,-0.030004920437932014,-0.03238498046994209,-0.0011804938549175858,0.27304235100746155,-0.0011832863092422485,-0.0020033721812069416,0.04289627820253372,-0.005961309187114239,-0.0020294992718845606,0.13842113316059113,-0.0008727168315090239,-0.1252179890871048,-0.0004646470188163221,-0.04641091078519821,-0.0009756244253367186,-0.07684562355279922,-0.0018212635768577456,-0.13829050958156586,-0.0010509073035791516,-0.06472326815128326,-0.0028956106398254633,-0.1406853049993515,-0.00038011017022654414,-0.16644132137298584,-0.04664460942149162,-0.002003023400902748,0.019001372158527374,-2.7547484933165833e-6,-0.09199230372905731,-2.4774474240985e-6,0.0208306722342968,-1.4901941085554427e-6,-0.10903165489435196,-3.725051783476374e-6,-0.06788796186447144,-8.02578688308131e-6,-0.06379298865795135,-0.000013881643099011853,-0.09667356312274933,-3.450595841059112e-6,-0.006765356287360191,-0.10954020917415619,-0.007350461091846228,-0.005635544657707214,-0.004959676880389452,-0.031381815671920776,-0.004473592154681683,-0.00874166376888752,-0.005178912077099085,-0.04022388905286789,-0.003501941915601492,-0.06710155308246613,-2.056877974609961e-6,-0.02262510173022747,-0.0994749516248703,-0.040831033140420914,-0.02751055546104908,-0.031628385186195374,-0.0071059116162359715,-0.015351951122283936,-0.017120076343417168,-0.027091627940535545,-0.021159937605261803,-0.019525766372680664,-0.034347232431173325,-0.05339507386088371,-0.006682890467345715,-0.010062223300337791,0.03218561038374901,-0.00692802993580699,-0.1536729484796524,-0.000783434312324971,-0.15589527785778046,-0.1697346270084381,-0.1699189692735672,-0.0069229817017912865,-0.02862660028040409,-0.03572099655866623,-0.04156066104769707,-0.027754323557019234,-0.0031731349881738424,-0.037165381014347076,-0.04039963707327843,-0.006488014478236437,-0.002469033934175968,-0.0,-0.0,-0.0,-0.0,-0.000027342433895682916,-0.01745615527033806,-0.00002311257048859261,-0.01585976965725422,-0.0001272779336431995,-0.046051762998104095,-1.6470913521970942e-7,-0.04103871062397957,-4.642535316179419e-7,-0.05375520884990692,-3.173810910084285e-6,-0.09874550253152847,-2.1994362668920076e-6,-0.015463016927242279,-2.478768919900176e-6,-0.010852368548512459,-2.3379870981443673e-6,-0.00006819071131758392,-0.07677456736564636,-1.0547367992330692e-6,-0.08975198864936829,-0.04772278293967247,-0.027511943131685257,-0.03943220525979996,-0.01855778507888317,-0.006008688826113939,-0.0021023123990744352,-0.006477738730609417,-0.002375526586547494,-0.013035459443926811,-0.002055574208498001,-0.02784065157175064,-0.0004381260077934712,-0.14957520365715027,-0.01585397869348526,-0.1118476614356041,-0.007264384534209967,-0.11221078783273697,-0.020054692402482033,-0.14456568658351898,-0.03773392364382744,-0.10743734240531921,-0.008385621942579746,-0.15326306223869324,-0.0164465494453907,-0.0924980491399765,-3.996277882833965e-6,-0.0005235441494733095,-0.00021470256615430117,-0.00024049224157352,-0.1061701625585556,-0.00014101623673923314,-0.09324171394109726,-0.00021005816233810037,-0.10248257219791412,-7.092690339050023e-6,-0.16964268684387207,-4.661907269110088e-7,-0.004341075662523508,-0.00037360945134423673,-0.0019396577263250947,-0.0,-0.15771974623203278,-0.0,-0.0017240496817976236,-0.0059091560542583466,-0.000039564889448229223,-0.0465269610285759,-0.000012980790415895171,-0.014051193371415138,-0.0000751865009078756,-0.008029240183532238,-0.00006725762796122581,-0.005360078532248735,-0.00018497384735383093,-0.004075624048709869,-0.00010061089415103197,-0.002260139212012291,-0.0001653630897635594,-0.08869710564613342,-0.04722870513796806,-0.0247566569596529,-0.039180368185043335,-0.01842549256980419,-0.016452735289931297,-0.003477438585832715,-0.023588789626955986,-0.0034918661694973707,-0.16642728447914124,-0.006624932866543531,-0.08210757374763489,-0.011209947988390923,-0.05778377503156662,-0.008254283107817173,-0.1691560447216034,-0.1459408849477768,-0.16056226193904877,0.23671799898147583,-0.14214564859867096,-0.1655033826828003,-0.16976630687713623,-0.005974986124783754,-0.0070690191350877285,-0.011357706040143967,-0.0014374932507053018,0.2433035522699356,-3.310103238618467e-6,-0.012821161188185215,-0.06033503636717796,-0.0017136947717517614,-0.06968970596790314,-0.00346808647736907,-0.00013979649520479143,-0.15926498174667358,-0.00018814096983987838,-0.013102254830300808,-0.040292441844940186,-0.05831214413046837,-0.001886572688817978,-0.03800816088914871,-2.479605655025807e-6,-0.14238059520721436,-0.000030543742468580604,-0.16506938636302948,-0.000011815307516371831,-0.062210533767938614,-0.0002604293986223638,-0.08666226267814636,-0.00005914514258620329,-0.029300039634108543,-0.000025977689801948145,-0.015655599534511566,-7.888198524597101e-6,-0.018044840544462204,-0.0994749516248703,-0.03566031530499458,-0.009023123420774937,-0.030039871111512184,-0.009586022235453129,-0.02889464795589447,-0.0021686593536287546,-0.005135515239089727,-0.004247208591550589,-0.0048818993382155895,-0.003238260978832841,-0.008930944837629795,-0.011363087221980095,-0.02071545459330082,-0.009799524210393429,-0.03989594057202339,-0.006254979409277439,-0.0010461094789206982,-0.14150382578372955,-0.0007904894300736487,-0.09643737971782684,-0.001635732944123447,-0.14340680837631226,-0.0011622722959145904,-0.16782720386981964,-0.0033441202249377966,-0.0022789854556322098,-0.06407961249351501,0.09333208948373795,-0.12988582253456116,0.0202644020318985,-0.09436341375112534,-0.06217284873127937,-0.013502351008355618,-0.04933261498808861,-0.007299569435417652,-0.041501402854919434,-0.017614159733057022,-0.019641553983092308,-0.020204314962029457,-0.011400047689676285,-0.0316610187292099,-0.02369817905128002,-0.01903088204562664,-0.01172718033194542,-0.07310175150632858,-0.0046994732692837715,-0.06563936918973923,-0.000717391783837229,-0.08455381542444229,-0.00038300297455862164,-0.025353236123919487,-0.0006413349765352905,-0.029855716973543167,-0.00023385115491691977,-0.005456828977912664,-0.0014442246174439788,-0.10032095015048981,-0.03138298913836479,-0.013045666739344597,-0.03588518872857094,-0.004629767034202814,-0.05526498705148697,-0.0019169310107827187,-0.05792819336056709,-0.001260021235793829,-0.027401559054851532,-0.00082780682714656,-0.07468128949403763,-0.000021986754290992394,-0.020235266536474228,-2.198837819378241e-6,-0.005108243320137262,-0.015038267709314823,-0.0017067294102162123,-0.13095131516456604,-2.196181640101713e-6,-0.027761640027165413,-0.0009347311570309103,-0.07447067648172379,-0.000414940994232893,-0.07654206454753876,-0.0007780147716403008,-0.07975920289754868,-0.0009988899109885097,-0.06717243790626526,-0.00013823631161358207,-0.00011377409828128293,-0.16598205268383026,-0.000029558412279584445,-0.1693110466003418,-0.00008109944610623643,-0.16896556317806244,-1.6084075582512014e-7,-0.12138886749744415,-2.894037379519432e-6,-0.13693982362747192,-0.0004277183616068214,-0.002366060623899102,-0.13406366109848022,-0.0003621341020334512,-0.13131043314933777,-0.0006759343086741865,-0.01868017576634884,-0.0037429826334118843,-0.027731094509363174,-0.01881297118961811,-0.035141922533512115,-0.000051682389312190935,-0.1632852703332901,-0.000018971759345731698,-0.09109979122877121,-3.177206053806003e-7,-0.06848529726266861,-0.000024484474124619737,-0.03150118887424469,-0.10032095015048981,-0.005327953491359949,-0.16828103363513947,-0.0005491779302246869,-0.00004330615411163308,0.008820117451250553,-0.0005150085198692977,-0.10686292499303818,-0.003193451324477792,-0.1663358509540558,-0.005843148101121187,-0.1455804854631424,-0.0035845378879457712,-0.15711471438407898,-0.006369892042130232,-0.00014812721929047257,-0.0429302342236042,-0.000049534453864907846,-0.16156013309955597,-0.000028204147383803502,-0.16437603533267975,-0.00010254285007249564,-0.15132421255111694,-0.0001323179603787139,-0.0004045934765599668,-0.007655065041035414,-0.00026117710513062775,-0.019925398752093315,-0.00004870156408287585,-0.007025431375950575,-0.0004481984651647508,-0.014026833698153496,-0.00020973676873836666,-0.00290307798422873,-0.00010345470218453556,-1.6173405015251774e-7,-0.12477756291627884,-4.694495885360084e-7,-0.010672474279999733,-4.694165056662314e-7,-0.007179464213550091,-0.0,-1.6289588984363945e-6,-0.0007174919010140002,-4.541155249171425e-6,-0.00007774074765620753,-0.14711211621761322,-1.6001357039385766e-7,0.7322729229927063,-0.0,0.3521093428134918,-0.0,-0.0,-0.08453605324029922,-0.00008074966171989217,-0.04362063854932785,-0.0000960442703217268,-0.06877221167087555,-0.08869710564613342,-0.04722870513796806,-0.04703662544488907,-0.01833193376660347,-0.017014537006616592,-0.006395682226866484,-0.002110678469762206,-0.004382026847451925,-0.004459559917449951,-0.011092274449765682,-0.008621635846793652,-0.003141002729535103,-0.007365601137280464,-0.002778475871309638,-0.002910210285335779,-0.16988155245780945,-0.00836145132780075,-0.1698194444179535,-0.0053674872033298016,-0.14894405007362366,-0.11308570951223373,-0.001078600063920021,-0.14352813363075256,-0.00036389593151398003,-0.0004100276855751872,-0.007098004687577486,-0.019136080518364906,-0.07817190140485764,-0.10947377234697342,-0.12925764918327332,-0.00008225686906371266,-0.11097057163715363,-0.0014965188456699252,-0.10567358881235123,-0.00031643363763578236,-0.1684284210205078,-0.008639602921903133,-0.06405190378427505,-0.017154743894934654,-0.10212037712335587,-0.03038308396935463,-0.041666701436042786,-0.03289763629436493,-0.04537054896354675,-0.054795414209365845,-0.030592361465096474,-0.0031630913726985455,-0.001123877358622849,-0.00012761408288497478,-0.007322966121137142,-0.0008816173067316413,-0.020871741697192192,-0.002013019984588027,-0.048208996653556824,-0.0005674321437254548,-0.10032095015048981,-0.03565492480993271,-0.053947534412145615,-0.003173801116645336,-0.0021408279426395893,-0.002505125943571329,0.2362383008003235,-0.0010230452753603458,-0.01675076223909855,-9.110262908507138e-7,-0.003891659202054143,0.19462335109710693,-0.008992230519652367,-0.017894769087433815,-0.009786511771380901,-0.027099991217255592,-0.03098875842988491,-0.024838656187057495,-0.02214762568473816,-0.011447739787399769,-0.002783791394904256,-0.011547072790563107,-0.0032384225632995367,-0.03624878451228142,-0.0017146760364994407,-0.006055556237697601,-0.012635900638997555,-0.1507294625043869,-0.00003323843702673912,-0.10939300805330276,-1.6002393010694504e-7,-0.16403980553150177,-1.6312058050971245e-6,-0.14207664132118225,-3.5850271160597913e-6,-0.16781453788280487,-0.00006303426926024258,-0.12245014309883118,-0.00017951460904441774,-0.11518719792366028,-7.627555078215664e-6,-0.15582437813282013,-0.00003262972677475773,-0.1350351870059967,-9.087262924367678e-7,-0.0011450486490502954,-0.0067418101243674755,-0.0007200301624834538,-2.339082129765302e-6,-0.16095174849033356,-1.4855559129500762e-6,-0.15999382734298706,-1.4903606597727048e-6,-0.16788901388645172,-1.6456021967314882e-7,-0.15163184702396393,-0.0,-0.12889884412288666,-0.10032095015048981,-0.03565492480993271,-0.015528718009591103,-0.007724742405116558,-0.00586296059191227,-0.0014521334087476134,-0.005674983840435743,-0.00008479032112518325,-0.001608247053809464,-0.13183844089508057,-0.0015429230406880379,-0.1397141069173813,-0.0000259741791523993,-0.14302393794059753,-0.1227574422955513,-0.16042852401733398,-0.13706514239311218,-0.11299307644367218,0.0416974276304245,-0.11743932962417603,-0.1694834679365158,-0.13536053895950317,-0.16354890167713165,-0.1303815394639969,-0.12482325732707977,-0.1449102759361267,-0.13223758339881897,-0.1455192267894745,0.22107437252998352,-0.16987814009189606,0.23860974609851837,-0.11863474547863007,-0.020697005093097687,-0.0216829814016819,-0.00006726069841533899,-0.00002386302912782412,-0.1598913073539734,-3.3092082958319224e-6,-0.16370438039302826,-1.6295240357067087e-7,-0.11088159680366516,-3.999910404672846e-6,-0.11401168256998062,-0.00003799127080128528,-0.011577615514397621,-0.022127242758870125,-0.024477528408169746,-0.00048630035598762333,-0.12963226437568665,-4.268804786988767e-6,-0.14315082132816315,-0.0002893365453928709,-0.08141039311885834,-0.0017881501698866487,-0.0590687096118927,-0.0009025348699651659,-0.009846106171607971,-0.08975198864936829,-0.04772278293967247,-0.03752197325229645,-0.023224137723445892,-0.010838496498763561,-0.09260862320661545,-0.13960352540016174,-0.08672474324703217,-0.0939561128616333,-0.02443241886794567,-0.13930806517601013,-0.03118276409804821,-0.16979224979877472,-0.018229009583592415,-0.12903039157390594,-0.07164104282855988,-0.16539818048477173,-0.07096129655838013,-0.16088631749153137,-0.00015046042972244322,-0.0527731217443943,-0.003203233238309622,-0.14073969423770905,-0.026216963306069374,-0.1142728328704834,-0.03138630464673042,-0.10374460369348526,-0.020991351455450058,-0.10570471733808517,-0.021312450990080833,-0.00008698149031260982,0.06207213178277016,-0.00014557302347384393,-0.10519491136074066,-0.00001181293646368431,-3.31126420860528e-6,-0.10449118912220001,-0.0033435432706028223,-0.018703605979681015,-0.001432668650522828,-0.010590983554720879,-0.0008275147993117571,-0.022789813578128815,-0.020216427743434906,-0.027609363198280334,-0.03798845782876015,-0.024331554770469666,-0.034184928983449936,-0.04064957797527313,-0.013536440208554268,-0.07779379189014435,-0.023000534623861313,-0.028579875826835632,-0.0004751841479446739,-0.025663306936621666,-0.002884530695155263,-0.01021544635295868,-0.00008767280814936385,-0.052103277295827866,-0.0994749516248703,-0.040831033140420914,-0.02751055546104908,-0.031628385186195374,-0.0071059116162359715,-0.004120933823287487,-0.0006849518977105618,-0.0032833132427185774,-0.0017110139597207308,-0.010108445771038532,-0.000019855628124787472,-0.12098579853773117,-0.000031769497581990436,-0.03946418687701225,-0.00037216872442513704,-0.00015201044152490795,-0.11398827284574509,-0.00044179221731610596,-0.0005292600835673511,-0.13560520112514496,-0.0009677652851678431,-0.1555670201778412,-0.00020746591326314956,-0.13792307674884796,-0.0004755845293402672,-0.16890402138233185,-0.0003771241754293442,-0.16996502876281738,-0.0003464712353888899,-0.10768762975931168,-0.1405356377363205,-0.16216909885406494,-0.15772247314453125,-0.14979228377342224,-0.16912557184696198,-0.012989760376513004,-0.004742876160889864,-0.03753025084733963,-0.1311320811510086,-0.006961262784898281,-0.021936336532235146,-0.039512600749731064,-0.04844899848103523,-0.07705262303352356,-0.02140425331890583,-0.0022591850720345974,-0.03528256714344025,-0.007318475749343634,-0.00121213064994663,-0.08975198864936829,-0.03872188925743103,-0.03024330548942089,-0.04812121391296387,-0.030345775187015533,-0.0004194413195364177,-0.019794886931777,-0.08692707121372223,-0.004159775096923113,0.1209181696176529,-0.011305385269224644,-0.07454214245080948,-0.00428429851308465,0.1679932028055191,-0.004544812720268965,-0.1214737817645073,-0.00015654122398700565,0.003209772054105997,-0.00006585540540982038,-0.1124824658036232,-0.0011157526168972254,-0.13868002593517303,-0.0029075290076434612,-0.005720937624573708,-0.004097443539649248,-0.00871619675308466,-0.10603800415992737,0.49131977558135986,-0.16784977912902832,0.5501174926757812,-0.16843439638614655,0.6347293257713318,-0.16809502243995667,-0.011044231243431568,-0.02795606665313244,-0.15822969377040863,0.20602063834667206,-0.1003926545381546,0.041773609817028046,-0.0647462010383606,0.005243420135229826,-0.11374936997890472,-0.05586827173829079,-0.09092298150062561,-0.024327479302883148,-0.07520201802253723,-0.11992206424474716,-0.006918151862919331,-0.0002334212331334129,-0.07766100019216537,-0.0006198261980898678,-0.028005406260490417,-0.00025596030172891915,-0.018925726413726807,-0.000022117545086075552,-0.0994749516248703,-0.006075781304389238,0.029397575184702873,-0.003457670798525214,-0.0023954675998538733,0.3688178062438965,-0.0025881766341626644,0.01872781291604042,-0.0019679912365972996,0.2518848180770874,-0.006295001599937677,0.16210705041885376,-0.004742398392409086,0.11220026016235352,-0.001150453113950789,0.10875049233436584,-0.0016859042225405574,-0.07941582053899765,-0.008539408445358276,-0.002715537091717124,-0.0065093738958239555,-0.1290774643421173,-0.00017469741578679532,-0.0007078330963850021,-0.0008274215506389737,-0.00764557346701622,-0.0023031667806208134,-0.08972559124231339,-0.13672533631324768,-0.05590146780014038,-0.0016419226303696632,-0.012502133846282959,-2.338206968488521e-6,-0.1070699691772461,-0.000018340704627917148,-0.09621880948543549,-3.0311357477330603e-6,-0.16864702105522156,-0.00022762877051718533,-0.1699306070804596,-0.00018060844740830362,-0.06658872216939926,-3.5872478747478453e-6,-0.10574831813573837,-3.170273430441739e-6,-0.12843920290470123,-6.56478096061619e-6,-0.013433877378702164,-1.604804538146709e-7,0.1959478110074997,-0.0,-0.10350912809371948,-1.5964374711074925e-7,-0.04199476167559624,-0.0994749516248703,-0.040831033140420914,-0.024414967745542526,-0.09048331528902054,-0.06449297070503235,-0.003852138062939048,0.6087366938591003,-0.0035116001963615417,0.6970751285552979,-0.0013601008104160428,0.9051192998886108,-0.0009171190904453397,-0.020267359912395477,-0.05213727056980133,-0.008976807817816734,0.12020045518875122,-0.009758716449141502,0.061560723930597305,-0.0000393250047636684,-0.1638343334197998,-1.3426922578219092e-6,-0.13317455351352692,-0.0005676337168551981,-0.1692914217710495,-0.0003825872263405472,-0.16696500778198242,-0.00002211371247540228,-0.1249578595161438,-0.000335964112309739,-0.15676334500312805,-0.0001813676062738523,-0.09418895840644836,-0.00007518458005506545,-0.06293907761573792,-0.000038351165130734444,-0.00001116565090342192,-0.05429917573928833,-4.1330058593302965e-6,-0.04698847234249115,-6.832015515101375e-6,-0.03760925680398941,-0.00019521523790899664,-1.1998356512776809e-6,-0.07091811299324036,-6.137640866654692e-7,-0.039861489087343216,-1.599682661890256e-7,-0.0024030241183936596,-0.0,-0.00023813378356862813,-0.09742853045463562,-0.0005183277535252273,-0.00018791620095726103,-0.0005238424055278301,-0.0005743062938563526,-0.0026888728607445955,-0.0,-0.13575291633605957,-1.6330555752119835e-7,-0.08975198864936829,-0.02078271098434925,-0.003744447836652398,-0.006682813633233309,-0.002528778975829482,-0.0025319356936961412,-0.00020464837143663317,-0.004447511397302151,-0.0008388592978008091,-0.012227722443640232,-0.003796766512095928,-0.0017803357914090157,-0.001393654732964933,-0.002363997744396329,-0.002161618322134018,-0.0017995245289057493,-0.008458130061626434,-0.002890489064157009,-0.022367455065250397,-0.007106579840183258,-0.02089635469019413,-0.007816151715815067,-0.027320584282279015,-0.11535655707120895,-0.015963133424520493,0.10678762197494507,-0.13976021111011505,-0.03248728811740875,-0.09346602857112885,-0.010376344434916973,-0.1645621955394745,-0.023386944085359573,-0.16995221376419067,-0.03691801801323891,0.11930397152900696,-0.008539981208741665,-0.12647144496440887,-0.020134607329964638,-0.12508496642112732,-0.013653659261763096,-0.16953551769256592,-0.041386138647794724,-0.06300965696573257,-0.07206423580646515,-0.06627054512500763,-0.15974514186382294,-0.08815287798643112,-0.0037431451492011547,-0.01512956339865923,-0.0012305544223636389,-0.000024613173081888817,-0.012306089513003826,-0.0006574792205356061,-0.06841109693050385,-0.0009745807037688792,-0.04254730045795441,-0.0016395890852436423,-0.010298588313162327,-0.0010754101676866412,-0.0994749516248703,-0.03566031530499458,-0.008733517490327358,-0.009055070579051971,-0.009621070697903633,-0.0421052910387516,-0.030089588835835457,-0.030983872711658478,-0.017834939062595367,-0.04747283458709717,-0.0063820043578743935,-0.04298751801252365,-0.00520589854568243,-0.03922739624977112,-0.04385896027088165,-0.05388956516981125,-0.03321842476725578,-0.023813385516405106,-0.003243223996832967,-0.16970668733119965,-0.09400691092014313,-0.1665635108947754,-0.16990569233894348,-0.16477273404598236,-0.16906249523162842,0.042861174792051315,-0.14217939972877502,-0.062425144016742706,-0.1650824397802353,-0.03338216617703438,-0.0008690788527019322,-0.031333841383457184,-0.021489132195711136,-0.031095251441001892,-0.026812823489308357,-0.061068352311849594,-0.05666246637701988,-0.08308624476194382,-0.04040219262242317,-0.09523184597492218,-0.03419700264930725,-0.07128658145666122,-0.017342397943139076,-0.04778946191072464,-0.0038750998210161924,-0.036433905363082886,-0.0063242241740226746,-0.026309626176953316,-0.0024834107607603073,-0.007881601341068745,-0.00924992561340332,-0.009906204417347908,-0.009277979843318462,-0.015951497480273247,-0.014446688815951347,-0.0003717512299772352,-0.011792391538619995,-0.00007053223089314997,-0.013616344891488552,-0.08869710564613342,-0.038724638521671295,-0.034958966076374054,-0.019703201949596405,-0.017811376601457596,-0.02174706943333149,-0.03435028716921806,-0.0007236304809339345,-0.004904122557491064,0.07173993438482285,-0.000017451640815124847,-0.16643111407756805,-0.00011580390128074214,-0.16473713517189026,-0.00013108634448144585,-0.000042581246816553175,-0.16242726147174835,-0.00001834276190493256,-0.10474136471748352,-6.146354962766054e-7,-0.16974462568759918,-1.200307679027901e-6,-0.14520786702632904,-9.089460490940837e-7,-0.11554825305938721,-0.0,-0.001955675892531872,-0.0011135064996778965,-0.0005201356834731996,-0.000043182481022085994,-0.00026850285939872265,-0.00007460573397111148,-0.0023820572532713413,-0.00022644101409241557,-0.008277211338281631,-0.000012719582628051285,-3.0338567285070894e-6,-0.00002111179128405638,-0.000011166898730152752,-3.3091851037170272e-6,-3.862157882394968e-6,-4.271775196684757e-6,-0.00007588756852783263,-3.1725687676953385e-6,-0.00004870366319664754,-1.6439378214272438e-7,-0.0005458784871734679,-0.0,-0.00045054242946207523,-1.6124859314459172e-7,-0.00032762446790002286,-0.0,-0.00023470392625313252,-1.0560929695202503e-6,-0.000059855614381376654,-0.0,-0.0008720447076484561,-9.8607988547883e-6,-0.0007386222714558244,-0.0994749516248703,-0.006075781304389238,0.04006291925907135,-0.0017947241431102157,0.11962152272462845,-0.0021756994538009167,0.3596588671207428,-0.0016840283060446382,-0.13213996589183807,-0.00005405783304013312,-0.00028796395054087043,-0.0002479680406395346,-0.04579506441950798,-0.0004361872561275959,-1.4883188441672246e-6,0.3717488646507263,-0.0002403800026513636,-0.005580021534115076,-0.000452879787189886,-0.1681119054555893,-0.0007071503205224872,-0.00013767549535259604,-0.08505915105342865,-0.000052396815590327606,-0.1181260272860527,-0.0005377734778448939,-0.15578781068325043,-0.003101518377661705,-0.13838300108909607,-0.01690257526934147,-0.0003414803941268474,0.04607081413269043,-0.0004931656876578927,-0.049536656588315964,-0.00017929531168192625,-0.10210686922073364,-0.0002128694177372381,-0.03883079066872597,-0.00025447268853895366,0.029787356033921242,-0.0000551269986317493,0.09026279300451279,-0.0006354241049848497,0.0037438771687448025,-0.0007349291117861867,-0.014979264698922634,-0.0044142743572592735,-0.0032738992013037205,-0.12683609127998352,-0.010043209418654442,-0.005189545452594757,-0.010533067397773266,-0.00680623110383749,-0.0029448233544826508,-0.0002885929716285318,-0.008916288614273071,-3.3124370020232163e-6,-0.017506549134850502,-0.00003677793938550167,-0.10032095015048981,-0.03138298913836479,-0.0031111675780266523,-0.008506350219249725,-0.02761548012495041,-0.006221782881766558,-0.020426644012331963,-0.008482892997562885,-0.004708599299192429,-0.16961818933486938,-0.011351167224347591,-0.0004072547017131001,-0.06536716967821121,-2.1944229047221597e-6,-0.10414174199104309,-0.000043426731281215325,-0.09107496589422226,-0.000014522377568937372,-0.06451272964477539,-0.00006303500413196161,-0.002925051609054208,-0.00003397504042368382,-0.007781942840665579,-0.000022990738216321915,-0.013001574203372002,-0.00003152864155708812,-0.00872588250786066,-0.003469077404588461,-0.0947854146361351,-3.0319515644805506e-6,-0.0014883577823638916,-0.10420266538858414,-0.005860294681042433,-0.018455270677804947,-0.0038281059823930264,-0.012649704702198505,-0.003917009104043245,-0.10649819672107697,-1.4884727761454997e-6,-0.01294483058154583,-1.6048956297254335e-7,-0.08263931423425674,-0.0,-0.145223006606102,-0.000010513341294426937,-0.1330741047859192,-1.4887391444062814e-6,-0.000013496540304913651,-0.036813925951719284,-0.00009786969894776121,-0.0022482832428067923,-0.00972957443445921,-0.0009349213214591146,-0.00044524806435219944,-0.013949314132332802,-0.020937154069542885,-0.000027213645807933062,-0.0021733525209128857,-0.00007146489951992407,-0.10032095015048981,-0.005327953491359949,-0.16828103363513947,-0.000011686798643495422,-0.1517045646905899,-0.0001481307699577883,-0.1379726529121399,-0.00005524737571249716,-0.15516619384288788,-0.00005084903386887163,-0.16821353137493134,-0.0008123987936414778,-0.09796410799026489,-0.00179437012411654,-0.000025355186153319664,-0.13632455468177795,-0.0000273366804321995,-0.14669910073280334,-0.00016667974705342203,-0.16448940336704254,-0.0006477364804595709,-0.12489388883113861,-0.00008882031397661194,-0.16969311237335205,-0.0014144490705803037,-0.0013072662986814976,0.14334531128406525,-1.3441564306049258e-6,0.07347400486469269,-0.00854579359292984,-0.06755094230175018,-0.010393239557743073,-0.02058430388569832,-0.005744799040257931,-0.0685855969786644,-0.006218017544597387,-0.04396428167819977,-0.006640848238021135,-0.09738858044147491,-0.0009056825074367225,-0.16985927522182465,-0.0007981225498951972,-0.09611385315656662,-0.00003164784357068129,-0.1392335444688797,-0.000031651652534492314,-0.12937992811203003,-0.00641724094748497,-0.012715507298707962,-0.00372657785192132,-0.032810334116220474,-0.00010038392792921513,-0.006605569738894701,-0.0001548862928757444,-0.009935394860804081,-0.0002746496465988457,-0.00008697920566191897,-0.004210875369608402,-0.08869710564613342,-0.01982218213379383,-0.006078963167965412,-0.00446257833391428,-0.008047339506447315,-0.0014198250137269497,-0.0053511266596615314,-0.0008227986982092261,-0.002507343888282776,-0.003223852254450321,-0.0023838679771870375,-0.00913859810680151,-0.006517316214740276,-0.016308238729834557,-0.0015894501702859998,-0.00020919287635479122,-0.019395286217331886,-0.00038382536149583757,-0.007468939293175936,-2.8967047001060564e-6,-0.028122030198574066,-9.117123909163638e-7,-0.13957057893276215,-0.16884128749370575,-0.05054035782814026,-0.1657000184059143,-0.0002246171497972682,-0.0006809368496760726,-0.0008137447875924408,-0.0032533982302993536,-0.000028943593861185946,-0.04753921926021576,-0.0,-0.16878780722618103,-1.48795481891284e-6,-0.04492540657520294,-0.00010060896602226421,-0.0143171027302742,-1.488606585553498e-6,-0.07334060221910477,-1.6292284499286325e-7,-0.055248577147722244,-1.0540926496105385e-6,-0.047524306923151016,-5.75985313844285e-6,-0.15310031175613403,-8.943954526330344e-6,-0.1350337713956833,-0.000021235347958281636,-0.09212895482778549,-9.125128599407617e-7,-0.06561877578496933,-0.000017200301954289898,-0.05537429451942444,-0.000013753436178376433,-0.02988383173942566,-0.00001516245993116172,-0.08869710564613342,-0.038724638521671295,-0.013733427040278912,-0.03165413811802864,-0.032428547739982605,-0.04486306384205818,-0.0112722497433424,-0.08716536313295364,-0.018116436898708344,-0.043118417263031006,-0.0016069934936240315,-0.01876651868224144,-0.023512225598096848,-0.020607328042387962,-0.012874285690486431,-0.04445590078830719,-0.017555609345436096,-0.007623951882123947,-0.0421493761241436,-0.022885920479893684,-0.008933927863836288,-0.010589190758764744,-0.035035111010074615,-0.024712255224585533,-0.0160836111754179,-0.014858212321996689,-0.0012071290984749794,-0.14041933417320251,-0.1641121655702591,-0.008214612491428852,-0.004934606142342091,-0.013851473107933998,-0.0003438754356466234,-0.0002357788325753063,-0.0003146556846331805,-0.0024857991375029087,-0.00534162437543273,-0.0008318459731526673,-0.15835823118686676,-0.00001284786867472576,-0.11114958673715591,-0.00009695527114672586,-0.1695372611284256,-0.030499856919050217,-0.022892577573657036,-0.00034688928280957043,-0.10094735026359558,-0.0017727724043652415,-0.024535346776247025,-0.00007344476034631953,-0.0068344250321388245,-0.10032095015048981,-0.03565492480993271,-0.023030800744891167,-0.017829425632953644,-0.007544393185526133,0.0348576158285141,-0.0027614340651780367,-0.031050026416778564,-0.0097578726708889,-0.029953818768262863,-0.02634681575000286,-0.09766465425491333,-0.017356939613819122,-0.14322029054164886,-0.000024360217139474116,-0.060790661722421646,-2.1952271254122024e-6,-0.08900903910398483,-6.141049766483775e-7,-0.0151578513905406,-0.000017196325643453747,-0.010977103374898434,-2.4783412300166674e-6,-0.03992173448204994,-1.6178884720829956e-7,-0.07137494534254074,-1.7736076642904663e-6,0.002248008968308568,-2.6151972178922733e-6,-0.11630566418170929,-1.7722428538036183e-6,-0.11146746575832367,-0.0,-0.00009558640886098146,-0.05475090071558952,-0.00013566564302891493,-0.06987138092517853,-0.0,-0.091496042907238,-7.654205091967015e-7,-0.1494550257921219,-0.0,-0.15458478033542633,-1.6226850618750177e-7,-0.007304498925805092,-0.007024022284895182,-0.004683969542384148,-0.003177943406626582,-0.003774144221097231,-0.0,0.1428094208240509,-0.0,-0.12772656977176666,-1.6053328977250203e-7,-0.07006940990686417,-4.677367542171851e-7,-0.12691311538219452,-4.644194575575966e-7,-0.14195556938648224,-0.0994749516248703,-0.040831033140420914,-0.036548346281051636,-0.06650041043758392,-0.01203896477818489,-0.04177415370941162,-0.04296420142054558,-0.02263687364757061,-0.0021395166404545307,0.2177542895078659,-0.0004818590241484344,0.5655433535575867,-0.001219728495925665,0.025540653616189957,-0.00006842892616987228,-0.0019871366675943136,-0.16907484829425812,-0.06005873158574104,-0.08205156773328781,-0.07705703377723694,-0.11131376773118973,0.17610754072666168,-0.029732711613178253,0.04037035256624222,-0.0412476621568203,0.3557845950126648,-0.04906310886144638,-0.03663886711001396,-0.03461052477359772,-0.021560771390795708,-0.06892921030521393,-0.09214746952056885,-0.01769416220486164,-0.15434253215789795,-0.008527153171598911,-0.08457939326763153,-0.005494765937328339,-0.06535226851701736,-0.0,0.46384507417678833,-3.1564098890157766e-7,0.347237229347229,-0.00023727622465230525,-0.00010890609701164067,-0.15086160600185394,-0.00014967935567256063,-0.11269187182188034,-0.00018410345364827663,-1.0537819434830453e-6,-0.041787661612033844,-2.7550502181838965e-6,-0.03672695904970169,-1.4901643226039596e-6,-0.0015347943408414721,-0.0007735635153949261,-0.0004938705242238939,-0.0001570923050167039,-0.0994749516248703,-0.040831033140420914,-0.06825675070285797,-0.0229276642203331,-0.03199939429759979,-0.03967678174376488,-0.029619883745908737,-0.05982804298400879,-0.04422614350914955,-0.062428660690784454,-0.021074971184134483,-0.020202331244945526,-0.046565230935811996,-0.05148887261748314,-0.008829689584672451,-0.03436509892344475,-0.014380170032382011,-0.0005278559983707964,-0.008439955301582813,-0.02146017923951149,-0.002324469853192568,-0.02190817892551422,-0.004381154663860798,-0.020991826429963112,-0.004456395283341408,-0.020505176857113838,-0.0020563642028719187,-0.0012384123401716352,-0.0037897927686572075,-0.0027090220246464014,-0.16639336943626404,-0.0010969530558213592,-0.14756298065185547,-0.004937335848808289,-0.16978295147418976,-0.1699293553829193,-1.6055838614192908e-7,-0.09990555047988892,-5.084007170808036e-6,-0.012410614639520645,-0.00007948018901515752,-0.013424748554825783,-0.0005137074622325599,-0.015742231160402298,-0.00010980876686517149,-0.004467412363737822,-0.00003458658466115594,-0.002200361341238022,-4.54220435130992e-6,-0.005458677653223276,-2.619248562041321e-6,-0.0021112067624926567,-0.0022544239182025194,-0.0003269971057306975,-0.0025659494567662477,-0.0001527822169009596,-3.17476519740012e-6,-0.04526122286915779,-4.678921868617181e-6,-0.0994749516248703,-0.040831033140420914,-0.024414967745542526,-0.03077658638358116,-0.028433268889784813,-0.012141922488808632,-0.005109797697514296,-0.031637027859687805,-0.05444444343447685,-0.001025217236019671,-0.006439357064664364,-0.00019031850388273597,0.16803990304470062,-0.00048468378372490406,0.18382473289966583,-0.0011423490941524506,0.12441900372505188,-0.0001857391616795212,-0.15509581565856934,-0.00014546563033945858,-0.00002869862873922102,-0.15485519170761108,-0.00001630768929317128,-0.16832229495048523,-0.00005025201244279742,-0.00011478555825306103,-0.14099298417568207,-0.0005028363084420562,-0.09210601449012756,-0.00006737538933521137,-0.09637785702943802,-0.00005737894753110595,-0.11629685759544373,-0.00021340831881389022,-0.020709527656435966,-0.00014379825734067708,-0.020024210214614868,-0.0007222732529044151,-0.004978328477591276,-0.00008583334420109168,-0.006920535117387772,-0.000012076505299773999,-0.027345305308699608,-9.118191428569844e-7,-0.00005607398634310812,-0.0602036751806736,-0.0009362465352751315,-0.04214971512556076,-0.0036328386049717665,-0.0159588735550642,-0.00901197548955679,-0.051144421100616455,-0.0,-0.07002738863229752,-1.0566069477135898e-6,-0.07585173845291138,-2.337639216420939e-6,-0.05756101384758949,-7.0964206315693446e-6,-0.0994749516248703,-0.006075781304389238,-0.009377802722156048,-0.0015441849827766418,-0.11414464563131332,-0.00009513046825304627,-0.1481960415840149,-0.00018312098109163344,-0.16714343428611755,-0.000019854116544593126,-0.16956208646297455,-0.00041995407082140446,-0.16987824440002441,-0.00011772335710702464,-0.0016249630134552717,-0.1683598905801773,-0.0151820732280612,-0.16984625160694122,-0.003472602693364024,-0.0659170001745224,-0.02194235846400261,-0.16420656442642212,-0.01032098475843668,-0.0542367585003376,-0.003808591514825821,-0.13788190484046936,-0.003321616677567363,-0.04926813021302223,-0.0031709768809378147,-0.16258969902992249,-0.00017524439317639917,-0.06528834998607635,-0.0030851182527840137,-0.12044975906610489,-0.005076354835182428,-0.16934561729431152,-0.05940485745668411,-0.022748826071619987,-0.00021416280651465058,-0.13610368967056274,-0.010160891339182854,-0.15133918821811676,-0.02163916826248169,-0.16975386440753937,-0.017179012298583984,-0.14104823768138885,-0.006229865830391645,-0.11566080898046494,-0.01516453456133604,-0.10805074870586395,-0.00013879356265533715,-0.1551145315170288,-0.0,-0.07289735972881317,-1.0535053434068686e-6,-0.10134714841842651,-0.0,-0.08019522577524185,-0.08975198864936829,-0.02078271098434925,-0.003744447836652398,-0.005808153655380011,-0.00005619625881081447,-0.15644875168800354,-0.00038330949610099196,-0.00022429422824643552,-0.1579945981502533,-0.001502042985521257,-0.1290888786315918,-0.0012053680839017034,-0.07710560411214828,-0.00027242698706686497,-0.0001707469782559201,0.10388050973415375,-0.0000351933813362848,-0.1494639366865158,-4.6812137952656485e-6,-0.042057208716869354,-0.00011388707935111597,0.05263867974281311,-0.0001078839268302545,-0.11411446332931519,-0.000018843067664420232,-0.1695500612258911,-0.0008374212775379419,-0.0002827887947205454,-0.01186314970254898,-0.01416348572820425,-0.06131676957011223,-0.04689427837729454,-0.01789228618144989,-0.04384864866733551,-0.13158299028873444,-0.16479815542697906,-0.1623082458972931,-0.12138491123914719,-0.16921289265155792,-0.02229747176170349,-0.10801447927951813,-0.04977085813879967,-0.00008974022057373077,-0.013476475141942501,-0.006722569931298494,-0.01594850793480873,-0.0008265534415841103,-0.003154878271743655,-0.0021578799933195114,-0.00790607649832964,-0.0006004203460179269,-0.014538696967065334],\"xaxis\":\"x\",\"yaxis\":\"y\",\"type\":\"histogram\"},{\"alignmentgroup\":\"True\",\"bingroup\":\"x\",\"histnorm\":\"percent\",\"hovertemplate\":\"label=True<br>acts=%{x}<br>percent=%{y}<extra></extra>\",\"legendgroup\":\"True\",\"marker\":{\"color\":\"#EF553B\",\"pattern\":{\"shape\":\"\"}},\"name\":\"True\",\"nbinsx\":100,\"offsetgroup\":\"True\",\"orientation\":\"v\",\"showlegend\":true,\"x\":[1.45192289352417,2.6667776107788086,2.376905918121338,2.9171948432922363,2.5927224159240723,2.075679302215576,1.9980229139328003,0.5866150259971619,0.05703692510724068,0.04699971154332161,-0.09489845484495163,-0.1368291676044464,0.8157692551612854,0.710116446018219,0.8718470931053162,0.13919471204280853,-0.05275712534785271,0.18991240859031677,-0.08312372863292694,-0.07487314194440842,1.8426086902618408,2.363208770751953,-0.15231853723526,-0.16410724818706512,1.5293327569961548,1.3843415975570679,1.4302551746368408,1.4992082118988037,-0.09781106561422348,-0.15030503273010254,1.5141535997390747,0.9593956470489502,0.7948168516159058,0.6720088124275208,0.6497312784194946,1.5446356534957886,1.5285714864730835,0.661532461643219,1.005460262298584,1.0512135028839111,0.889299213886261,1.3475141525268555,0.9710556268692017,0.7133105993270874,0.8409330248832703,1.235382318496704,1.542657494544983,1.030704140663147,0.7771839499473572,1.1195192337036133,1.0397744178771973,0.5390632152557373,0.8432055115699768,1.1384607553482056,1.1246594190597534,1.918944239616394,2.2171173095703125,-0.0033597652800381184,0.6452382206916809,1.1264442205429077,1.5676724910736084,1.0920710563659668,1.0245391130447388,0.7371951341629028,1.6428836584091187,1.2309476137161255,0.2432703673839569,1.6355056762695312,0.7971788048744202,0.4138861894607544,0.3787022829055786,2.3440918922424316,0.8178504109382629,0.11346843838691711,1.3281723260879517,0.5732201933860779,0.9798260927200317,0.9278972744941711,-0.08455654978752136,0.10189168155193329,-0.1124473512172699,-0.15767574310302734,-0.09738890081644058,-0.08318154513835907,2.416422128677368,1.9185831546783447,-0.06913455575704575,-0.11076194792985916,1.07762610912323,1.0615078210830688,0.10752908885478973,0.21376043558120728,0.15370917320251465,2.0177459716796875,-0.1626983880996704,-0.1698671281337738,1.357771396636963,1.9929144382476807,1.4083836078643799,2.0919148921966553,1.3851224184036255,1.3452856540679932,0.635653018951416,0.6501464247703552,2.496234655380249,2.068539619445801,0.9180871248245239,1.3988410234451294,0.7487234473228455,1.3185259103775024,-0.1669221669435501,-0.14796707034111023,-0.11996939778327942,-0.0014439773513004184],\"xaxis\":\"x\",\"yaxis\":\"y\",\"type\":\"histogram\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"acts\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"percent\"}},\"legend\":{\"title\":{\"text\":\"label\"},\"tracegroupgap\":0},\"title\":{\"text\":\"Spectrum plot for neuron L5N1393 testing C0==BLANK & D1==THEIR'S & E2==MINE\"},\"barmode\":\"relative\"},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('f6c57c79-e970-40b9-a893-78163fc62f34');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "c0 = focus_states_flipped_pm1[:, :, 2, 0]\n",
    "d1 = focus_states_flipped_pm1[:, :, 3, 1]\n",
    "e2 = focus_states_flipped_pm1[:, :, 4, 2]\n",
    "\n",
    "label = (c0==0) & (d1==-1) & (e2==1)\n",
    "\n",
    "import pandas as pd\n",
    "neuron_acts = focus_cache[\"post\", 5][:, :, 1393]\n",
    "df = pd.DataFrame({\"acts\":neuron_acts.flatten().tolist(), \"label\":label[:, :-1].flatten().tolist()})\n",
    "px.histogram(df, x=\"acts\", color=\"label\", histnorm=\"percent\", nbins=100, title=\"Spectrum plot for neuron L5N1393 testing C0==BLANK & D1==THEIR'S & E2==MINE\")\n",
    "# %%\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Making Figure 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "residual_stream torch.Size([512])\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"410c2620-13c0-42bb-8537-563aac33609f\" class=\"plotly-graph-div\" style=\"height:300px; width:600px;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"410c2620-13c0-42bb-8537-563aac33609f\")) {                    Plotly.newPlot(                        \"410c2620-13c0-42bb-8537-563aac33609f\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"z\":[[0.99999964,0.91042936,1.0,1.0,1.0,1.0,0.99368215,0.99995804],[0.99813086,0.99994516,1.0,3.8604124e-8,1.5641194e-9,0.99998987,0.9998692,0.99995184],[0.9998603,0.00008565692,1.0,0.0000063882735,4.58324e-7,0.00003055291,0.99999154,1.0],[0.999331,0.9999993,2.6851613e-10,3.918924e-10,3.1879088e-10,0.9999987,6.3820955e-11,0.99997735],[0.999295,0.9999927,0.0000014871049,1.6030755e-10,6.2675287e-10,2.695002e-8,1.0,1.0],[0.9996513,0.99999964,1.1029049e-8,0.1048025,4.6698184e-10,0.03946298,3.1607515e-11,0.000002291175],[0.99913895,0.9999386,0.000028339782,0.00021306203,1.0,0.00010205577,0.000008894276,0.0000373183],[0.9986532,0.9999304,1.0,0.99999976,1.0,1.0,1.0,1.0]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"1\",\"z\":[[1.3753593e-7,0.0021002232,2.34588e-16,7.6763056e-17,1.8036042e-14,1.8367198e-14,0.0037780753,0.00004046137],[0.0009736979,0.00002408737,1.3623619e-14,0.9996276,0.9976539,0.0000030132721,0.000004808557,0.00004568517],[0.000115437026,0.002262018,6.5796634e-15,0.99863976,0.0009257535,0.00036728987,0.0000028027457,4.5729005e-18],[0.00028837065,1.0679145e-7,0.0038866503,0.99722904,0.00085997826,9.1629914e-7,0.99848086,0.000022545357],[0.000056045636,2.7808701e-7,0.00026378187,0.99928063,0.99554586,0.9993893,3.2924462e-16,1.3760792e-12],[0.000034290042,1.5392052e-8,0.00014721192,0.8950627,0.017611148,0.96043015,0.029936861,0.0071147736],[0.00045106304,0.0000040965815,0.96608335,0.99814475,9.719864e-19,0.999382,0.9941725,0.9994518],[0.0003045321,0.000018250592,2.2975645e-12,2.344526e-7,8.482711e-13,3.0441517e-11,1.1717244e-10,3.3030942e-9]],\"type\":\"heatmap\",\"xaxis\":\"x2\",\"yaxis\":\"y2\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"2\",\"z\":[[2.4672605e-7,0.087470464,2.1173958e-15,2.1725542e-19,1.3332463e-15,8.7743153e-16,0.0025398566,0.0000015696404],[0.00089545484,0.000030728257,1.1652499e-14,0.00037240508,0.0023460859,0.0000071495324,0.00012594576,0.0000024817655],[0.000024335166,0.99765235,3.1675966e-15,0.0013538029,0.99907374,0.99960226,0.0000056146746,1.1372005e-17],[0.00038063453,6.122668e-7,0.9961133,0.0027709245,0.99914,3.8803688e-7,0.001519159,7.493601e-8],[0.0006490006,0.000007076643,0.9997348,0.00071939145,0.0044540744,0.000610615,2.3300585e-16,2.388769e-13],[0.0003144128,4.0108557e-7,0.9998528,0.00013485995,0.9823889,0.00010693099,0.9700631,0.99288297],[0.00040995778,0.000057329493,0.033888254,0.0016422407,3.683523e-18,0.00051589275,0.005818633,0.0005108813],[0.0010423089,0.000051339535,6.430202e-11,2.820164e-10,4.291941e-12,1.5906216e-11,6.7674073e-13,9.944344e-9]],\"type\":\"heatmap\",\"xaxis\":\"x3\",\"yaxis\":\"y3\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,0.31999999999999995],\"title\":{\"text\":\"x\"},\"visible\":false},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"title\":{\"text\":\"y\"},\"visible\":false},\"xaxis2\":{\"anchor\":\"y2\",\"domain\":[0.33999999999999997,0.6599999999999999],\"matches\":\"x\",\"title\":{\"text\":\"x\"},\"visible\":false},\"yaxis2\":{\"anchor\":\"x2\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis3\":{\"anchor\":\"y3\",\"domain\":[0.6799999999999999,0.9999999999999999],\"matches\":\"x\",\"title\":{\"text\":\"x\"},\"visible\":false},\"yaxis3\":{\"anchor\":\"x3\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"annotations\":[{\"font\":{},\"showarrow\":false,\"text\":\"P(Empty)\",\"x\":0.15999999999999998,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"P(Their Colour)\",\"x\":0.49999999999999994,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"P(My Colour)\",\"x\":0.8399999999999999,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"}],\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0,\"showscale\":false},\"margin\":{\"t\":60},\"height\":300,\"width\":600},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('410c2620-13c0-42bb-8537-563aac33609f');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"58bad204-16ed-4d49-90fe-8863716b5f76\" class=\"plotly-graph-div\" style=\"height:400px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"58bad204-16ed-4d49-90fe-8863716b5f76\")) {                    Plotly.newPlot(                        \"58bad204-16ed-4d49-90fe-8863716b5f76\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.0,0.0,-0.5,-0.5,-0.5,-0.5,0.0,0.0],[0.1,0.0,-0.5,0.9,1.0,0.0,0.0,0.0],[0.0,-1.0,-0.5,0.9,-1.0,-1.0,0.0,-0.5],[0.0,0.0,-1.0,0.9,-1.0,0.0,1.0,0.0],[0.0,0.1,-1.0,0.9,1.0,1.0,-0.5,-0.5],[0.0,0.1,-1.0,0.9,-1.0,1.0,-1.0,-1.0],[0.0,0.0,1.0,1.0,-0.5,1.0,1.0,1.0],[0.0,0.0,-0.5,0.0,-0.5,-0.5,-0.5,-0.5]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"<b>%{y}%{x}</b><br>%{customdata}<extra></extra>\",\"text\":[[\"A0\",\"A1\",\"A2\",\"A3\",\"A4\",\"A5\",\"A6\",\"A7\"],[\"B0\",\"B1\",\"B2\",\"\",\"\",\"B5\",\"B6\",\"B7\"],[\"C0\",\"\",\"C2\",\"\",\"\",\"\",\"C6\",\"C7\"],[\"D0\",\"D1\",\"\",\"\",\"\",\"D5\",\"\",\"D7\"],[\"E0\",\"E1\",\"\",\"\",\"\",\"\",\"E6\",\"E7\"],[\"F0\",\"F1\",\"\",\"\",\"\",\"\",\"\",\"\"],[\"G0\",\"G1\",\"\",\"\",\"G4\",\"\",\"\",\"\"],[\"H0\",\"H1\",\"H2\",\"H3\",\"H4\",\"H5\",\"H6\",\"H7\"]],\"texttemplate\":\"%{text}\",\"customdata\":[[\"I:1<br>S:0\",\"I:2<br>S:1\",\"I:3<br>S:2\",\"I:4<br>S:3\",\"I:5<br>S:4\",\"I:6<br>S:5\",\"I:7<br>S:6\",\"I:8<br>S:7\"],[\"I:9<br>S:8\",\"I:10<br>S:9\",\"I:11<br>S:10\",\"I:12<br>S:11\",\"I:13<br>S:12\",\"I:14<br>S:13\",\"I:15<br>S:14\",\"I:16<br>S:15\"],[\"I:17<br>S:16\",\"I:18<br>S:17\",\"I:19<br>S:18\",\"I:20<br>S:19\",\"I:21<br>S:20\",\"I:22<br>S:21\",\"I:23<br>S:22\",\"I:24<br>S:23\"],[\"I:25<br>S:24\",\"I:26<br>S:25\",\"I:27<br>S:26\",\"I:X<br>S:27\",\"I:X<br>S:28\",\"I:28<br>S:29\",\"I:29<br>S:30\",\"I:30<br>S:31\"],[\"I:31<br>S:32\",\"I:32<br>S:33\",\"I:33<br>S:34\",\"I:X<br>S:35\",\"I:X<br>S:36\",\"I:34<br>S:37\",\"I:35<br>S:38\",\"I:36<br>S:39\"],[\"I:37<br>S:40\",\"I:38<br>S:41\",\"I:39<br>S:42\",\"I:40<br>S:43\",\"I:41<br>S:44\",\"I:42<br>S:45\",\"I:43<br>S:46\",\"I:44<br>S:47\"],[\"I:45<br>S:48\",\"I:46<br>S:49\",\"I:47<br>S:50\",\"I:48<br>S:51\",\"I:49<br>S:52\",\"I:50<br>S:53\",\"I:51<br>S:54\",\"I:52<br>S:55\"],[\"I:53<br>S:56\",\"I:54<br>S:57\",\"I:55<br>S:58\",\"I:56<br>S:59\",\"I:57<br>S:60\",\"I:58<br>S:61\",\"I:59<br>S:62\",\"I:60<br>S:63\"]]}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"},\"visible\":false},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"},\"visible\":false},\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(0, 128, 128)\"],[0.16666666666666666,\"rgb(112, 164, 148)\"],[0.3333333333333333,\"rgb(180, 200, 168)\"],[0.5,\"rgb(246, 237, 189)\"],[0.6666666666666666,\"rgb(237, 187, 138)\"],[0.8333333333333334,\"rgb(222, 138, 90)\"],[1.0,\"rgb(202, 86, 44)\"]],\"cmid\":0.0,\"showscale\":false},\"title\":{\"text\":\"White To Play. Board State After Black Plays B3 \",\"x\":0.5},\"shapes\":[{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.75,\"x1\":3.25,\"y0\":0.75,\"y1\":1.25,\"line\":{\"color\":\"red\",\"width\":4}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":0.8,\"x1\":1.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.78,\"x1\":3.22,\"y0\":1.78,\"y1\":2.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.78,\"x1\":3.22,\"y0\":2.78,\"y1\":3.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.78,\"x1\":3.22,\"y0\":3.78,\"y1\":4.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.78,\"x1\":3.22,\"y0\":4.78,\"y1\":5.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":6.8,\"x1\":7.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":6.8,\"x1\":7.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}}],\"height\":400},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('58bad204-16ed-4d49-90fe-8863716b5f76');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "residual_stream torch.Size([512])\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"04c07e69-3b6d-43f1-8169-71a9b35103df\" class=\"plotly-graph-div\" style=\"height:300px; width:600px;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"04c07e69-3b6d-43f1-8169-71a9b35103df\")) {                    Plotly.newPlot(                        \"04c07e69-3b6d-43f1-8169-71a9b35103df\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"z\":[[0.999985,0.9570909,0.9988524,0.9977679,0.9999591,0.99929595,0.99919254,0.9652594],[1.0,0.999995,0.9999964,0.00011596162,1.7511247e-9,1.0,1.0,0.9999974],[0.999948,6.380411e-7,1.0,0.000066997636,2.47827e-7,1.3327319e-12,1.0,0.9996438],[0.99999106,1.0,3.0622468e-7,4.2550424e-10,2.3945146e-10,1.0,4.218091e-7,0.9999839],[0.999913,0.99999964,5.4767796e-10,1.1291112e-10,6.035238e-10,4.110643e-9,0.99952817,1.0],[0.99999225,1.0,0.0000041976277,0.00023428454,0.000002232375,1.3763218e-9,0.0011802493,0.0002195173],[0.99407613,1.0,0.00046286493,1.2286266e-9,1.0,1.8409818e-9,1.8173951e-10,1.6257247e-12],[0.99579763,0.9986534,0.9920345,0.9996246,0.9976047,1.0,6.88509e-8,1.0]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"1\",\"z\":[[0.0000015726989,0.01684063,0.000027156993,0.0015727772,0.000039024002,0.00047807573,0.0004801524,0.03403372],[2.8943133e-15,0.0000020474445,4.124615e-7,0.0003151587,0.0010808016,1.0538143e-18,1.5786064e-16,2.1769534e-8],[0.00003397664,0.99776316,5.212865e-17,0.0035788668,0.99581367,0.99909043,2.557457e-17,0.00005389028],[0.000008696847,3.9633498e-13,0.9990625,0.0020109469,0.99677175,2.3508991e-14,0.00038296013,0.0000037233037],[0.000059714755,1.4055124e-8,0.9969062,0.0011602069,0.0019459622,0.00037363943,0.00012434382,2.75211e-10],[7.8815594e-7,1.7297107e-12,0.9985838,0.0045888866,0.99794024,0.0040132753,0.995421,0.9963374],[0.003409451,5.0873355e-16,0.001018977,0.0002485278,3.6027377e-12,0.9996774,0.99959105,0.000015536665],[0.0032470606,0.00028756264,0.0007708607,0.000028210954,0.0023675775,6.938653e-14,0.99923253,2.2490605e-15]],\"type\":\"heatmap\",\"xaxis\":\"x2\",\"yaxis\":\"y2\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"2\",\"z\":[[0.000013423831,0.026068421,0.001120444,0.00065935944,0.000001933591,0.00022599555,0.00032730497,0.0007068109],[4.729979e-15,0.0000029782639,0.0000032714497,0.9995689,0.99891925,1.3598979e-18,8.399382e-16,0.0000026636014],[0.000018043342,0.0022362403,2.3318902e-16,0.9963541,0.0041860156,0.00090956804,1.9793128e-17,0.00030228976],[2.3543328e-7,8.1925623e-13,0.00093717146,0.99798906,0.0032282271,1.1138293e-14,0.9996165,0.000012423316],[0.000027270375,4.0053095e-7,0.0030937777,0.9988398,0.998054,0.9996264,0.00034741004,5.912957e-10],[0.000006860327,3.3029445e-12,0.0014120796,0.9951768,0.002057565,0.99598676,0.0033987726,0.0034431084],[0.0025144129,1.0992079e-14,0.99851817,0.9997515,1.630785e-11,0.00032260193,0.0004089528,0.9999845],[0.00095531717,0.0010589028,0.007194657,0.00034714252,0.000027682301,3.911837e-13,0.0007673911,9.1790806e-14]],\"type\":\"heatmap\",\"xaxis\":\"x3\",\"yaxis\":\"y3\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,0.31999999999999995],\"title\":{\"text\":\"x\"},\"visible\":false},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"title\":{\"text\":\"y\"},\"visible\":false},\"xaxis2\":{\"anchor\":\"y2\",\"domain\":[0.33999999999999997,0.6599999999999999],\"matches\":\"x\",\"title\":{\"text\":\"x\"},\"visible\":false},\"yaxis2\":{\"anchor\":\"x2\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis3\":{\"anchor\":\"y3\",\"domain\":[0.6799999999999999,0.9999999999999999],\"matches\":\"x\",\"title\":{\"text\":\"x\"},\"visible\":false},\"yaxis3\":{\"anchor\":\"x3\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"annotations\":[{\"font\":{},\"showarrow\":false,\"text\":\"P(Empty)\",\"x\":0.15999999999999998,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"P(Their Colour)\",\"x\":0.49999999999999994,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"P(My Colour)\",\"x\":0.8399999999999999,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"}],\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0,\"showscale\":false},\"margin\":{\"t\":60},\"height\":300,\"width\":600},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('04c07e69-3b6d-43f1-8169-71a9b35103df');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"ad04ccce-487a-49d4-ba66-0dc1cc939a4c\" class=\"plotly-graph-div\" style=\"height:400px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"ad04ccce-487a-49d4-ba66-0dc1cc939a4c\")) {                    Plotly.newPlot(                        \"ad04ccce-487a-49d4-ba66-0dc1cc939a4c\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.0,0.0,-0.1,-0.1,-0.1,-0.1,0.0,0.0],[0.5,0.0,-0.1,1.0,1.0,0.5,0.5,0.0],[0.0,-1.0,0.5,1.0,-1.0,-1.0,0.5,-0.1],[0.0,0.5,-1.0,1.0,-1.0,0.5,1.0,0.0],[0.0,0.5,-1.0,1.0,1.0,1.0,-0.1,0.5],[0.0,0.5,-1.0,1.0,-1.0,1.0,-1.0,-1.0],[0.0,0.5,1.0,1.0,0.5,-0.9,-0.9,1.0],[0.0,0.0,-0.1,0.0,-0.1,0.5,-0.9,0.5]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"<b>%{y}%{x}</b><br>%{customdata}<extra></extra>\",\"text\":[[\"A0\",\"A1\",\"A2\",\"A3\",\"A4\",\"A5\",\"A6\",\"A7\"],[\"B0\",\"B1\",\"B2\",\"\",\"\",\"B5\",\"B6\",\"B7\"],[\"C0\",\"\",\"C2\",\"\",\"\",\"\",\"C6\",\"C7\"],[\"D0\",\"D1\",\"\",\"\",\"\",\"D5\",\"\",\"D7\"],[\"E0\",\"E1\",\"\",\"\",\"\",\"\",\"E6\",\"E7\"],[\"F0\",\"F1\",\"\",\"\",\"\",\"\",\"\",\"\"],[\"G0\",\"G1\",\"\",\"\",\"G4\",\"\",\"\",\"\"],[\"H0\",\"H1\",\"H2\",\"H3\",\"H4\",\"H5\",\"\",\"H7\"]],\"texttemplate\":\"%{text}\",\"customdata\":[[\"I:1<br>S:0\",\"I:2<br>S:1\",\"I:3<br>S:2\",\"I:4<br>S:3\",\"I:5<br>S:4\",\"I:6<br>S:5\",\"I:7<br>S:6\",\"I:8<br>S:7\"],[\"I:9<br>S:8\",\"I:10<br>S:9\",\"I:11<br>S:10\",\"I:12<br>S:11\",\"I:13<br>S:12\",\"I:14<br>S:13\",\"I:15<br>S:14\",\"I:16<br>S:15\"],[\"I:17<br>S:16\",\"I:18<br>S:17\",\"I:19<br>S:18\",\"I:20<br>S:19\",\"I:21<br>S:20\",\"I:22<br>S:21\",\"I:23<br>S:22\",\"I:24<br>S:23\"],[\"I:25<br>S:24\",\"I:26<br>S:25\",\"I:27<br>S:26\",\"I:X<br>S:27\",\"I:X<br>S:28\",\"I:28<br>S:29\",\"I:29<br>S:30\",\"I:30<br>S:31\"],[\"I:31<br>S:32\",\"I:32<br>S:33\",\"I:33<br>S:34\",\"I:X<br>S:35\",\"I:X<br>S:36\",\"I:34<br>S:37\",\"I:35<br>S:38\",\"I:36<br>S:39\"],[\"I:37<br>S:40\",\"I:38<br>S:41\",\"I:39<br>S:42\",\"I:40<br>S:43\",\"I:41<br>S:44\",\"I:42<br>S:45\",\"I:43<br>S:46\",\"I:44<br>S:47\"],[\"I:45<br>S:48\",\"I:46<br>S:49\",\"I:47<br>S:50\",\"I:48<br>S:51\",\"I:49<br>S:52\",\"I:50<br>S:53\",\"I:51<br>S:54\",\"I:52<br>S:55\"],[\"I:53<br>S:56\",\"I:54<br>S:57\",\"I:55<br>S:58\",\"I:56<br>S:59\",\"I:57<br>S:60\",\"I:58<br>S:61\",\"I:59<br>S:62\",\"I:60<br>S:63\"]]}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"},\"visible\":false},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"},\"visible\":false},\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(0, 128, 128)\"],[0.16666666666666666,\"rgb(112, 164, 148)\"],[0.3333333333333333,\"rgb(180, 200, 168)\"],[0.5,\"rgb(246, 237, 189)\"],[0.6666666666666666,\"rgb(237, 187, 138)\"],[0.8333333333333334,\"rgb(222, 138, 90)\"],[1.0,\"rgb(202, 86, 44)\"]],\"cmid\":0.0,\"showscale\":false},\"title\":{\"text\":\"Black To Play. Board State After White Plays H6 \",\"x\":0.5},\"shapes\":[{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":0.8,\"x1\":1.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":6.8,\"x1\":7.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":4.78,\"x1\":5.22,\"y0\":5.78,\"y1\":6.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":5.78,\"x1\":6.22,\"y0\":5.78,\"y1\":6.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":6.8,\"x1\":7.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":5.75,\"x1\":6.25,\"y0\":6.75,\"y1\":7.25,\"line\":{\"color\":\"red\",\"width\":4}}],\"height\":400},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('ad04ccce-487a-49d4-ba66-0dc1cc939a4c');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "residual_stream torch.Size([512])\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"5ed9279d-acce-4010-8465-06c21cdce292\" class=\"plotly-graph-div\" style=\"height:300px; width:600px;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"5ed9279d-acce-4010-8465-06c21cdce292\")) {                    Plotly.newPlot(                        \"5ed9279d-acce-4010-8465-06c21cdce292\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"z\":[[0.99999976,0.968072,1.0,1.0,1.0,1.0,0.9636731,0.99928856],[0.9834541,0.9994966,1.0,1.9577124e-8,7.312644e-8,0.9999807,0.99984705,0.998917],[0.9997168,0.000083959065,1.0,0.000005185437,0.000008247951,0.0000215352,0.99999964,1.0],[0.99999046,0.99999976,7.757757e-8,2.7562294e-10,2.173676e-10,0.9999993,0.0000035743474,0.9999963],[0.99756026,0.99999547,0.0000055953874,2.4387992e-10,6.227083e-10,9.074999e-7,1.0,1.0],[0.9999678,0.99999774,4.1813355e-8,0.0000059140743,3.6093515e-9,0.00026775632,9.29474e-14,3.935381e-8],[0.99867934,0.9999471,0.00008276932,0.00019439175,3.876017e-11,0.00006263142,0.000028273336,0.0000027772874],[0.999617,0.99999356,1.0,0.99999726,1.0,1.0,1.1808284e-9,0.99997497]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"1\",\"z\":[[2.5712025e-8,0.0031173022,1.6883145e-16,6.733207e-14,8.8439125e-15,1.5645137e-14,0.03434719,0.00041080176],[0.00044961766,0.0003612048,1.2639593e-15,0.9997378,0.9958584,0.0000035347093,0.000006125798,0.0005669107],[0.00018196153,0.0039731096,2.780452e-15,0.9991443,0.0022878083,0.0005512753,6.148952e-8,6.342149e-11],[0.000003822013,2.4708909e-8,0.0011599979,0.9982644,0.00077748345,5.2145634e-7,0.9972325,0.0000032304877],[0.00018111797,1.4569264e-7,0.00023780167,0.9950388,0.98804456,0.9987471,3.0732555e-16,1.3278148e-12],[0.0000053312056,1.0009445e-7,0.0001360589,0.9994985,0.9982388,0.9970499,0.05853151,0.0010470838],[0.001144266,9.883679e-7,0.9844205,0.9967552,0.99885345,0.9945268,0.9873505,0.99960905],[0.000063424,0.0000014785006,2.9278386e-15,0.0000027755507,8.1484103e-10,1.891273e-9,0.00011913282,0.000004306204]],\"type\":\"heatmap\",\"xaxis\":\"x2\",\"yaxis\":\"y2\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"},{\"coloraxis\":\"coloraxis\",\"name\":\"2\",\"z\":[[2.025148e-7,0.028810715,7.421175e-15,3.9040045e-16,1.2647601e-15,1.1723116e-15,0.001979726,0.00030053867],[0.01609626,0.00014209992,1.7521038e-16,0.00026213808,0.004141519,0.000015748137,0.00014681347,0.0005160725],[0.000101124446,0.9959429,3.502868e-15,0.0008504397,0.99770397,0.9994272,2.9676158e-7,2.1901647e-9],[0.000005697999,2.1064834e-7,0.9988399,0.0017356732,0.9992225,1.8157594e-7,0.0027639456,5.04101e-7],[0.0022585802,0.000004464417,0.99975663,0.0049611405,0.011955494,0.0012520029,1.0510652e-16,7.569125e-13],[0.000026781318,0.0000021464296,0.99986386,0.00049552874,0.0017611779,0.0026823042,0.94146854,0.99895287],[0.00017638777,0.000051948748,0.01549674,0.003050437,0.001146568,0.0054105935,0.012621199,0.00038815604],[0.0003196475,0.000004960617,2.594675e-14,5.269005e-11,7.390537e-11,4.6739643e-11,0.9998809,0.000020794143]],\"type\":\"heatmap\",\"xaxis\":\"x3\",\"yaxis\":\"y3\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,0.31999999999999995],\"title\":{\"text\":\"x\"},\"visible\":false},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"title\":{\"text\":\"y\"},\"visible\":false},\"xaxis2\":{\"anchor\":\"y2\",\"domain\":[0.33999999999999997,0.6599999999999999],\"matches\":\"x\",\"title\":{\"text\":\"x\"},\"visible\":false},\"yaxis2\":{\"anchor\":\"x2\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"xaxis3\":{\"anchor\":\"y3\",\"domain\":[0.6799999999999999,0.9999999999999999],\"matches\":\"x\",\"title\":{\"text\":\"x\"},\"visible\":false},\"yaxis3\":{\"anchor\":\"x3\",\"domain\":[0.0,1.0],\"matches\":\"y\",\"showticklabels\":false},\"annotations\":[{\"font\":{},\"showarrow\":false,\"text\":\"P(Empty)\",\"x\":0.15999999999999998,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"P(Their Colour)\",\"x\":0.49999999999999994,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{},\"showarrow\":false,\"text\":\"P(My Colour)\",\"x\":0.8399999999999999,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"}],\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"cmid\":0.0,\"showscale\":false},\"margin\":{\"t\":60},\"height\":300,\"width\":600},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('5ed9279d-acce-4010-8465-06c21cdce292');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.16.1.min.js\"></script>                <div id=\"ab0b33ef-8178-494f-b0c9-ef526f0f7528\" class=\"plotly-graph-div\" style=\"height:400px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"ab0b33ef-8178-494f-b0c9-ef526f0f7528\")) {                    Plotly.newPlot(                        \"ab0b33ef-8178-494f-b0c9-ef526f0f7528\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\"],\"y\":[\"A\",\"B\",\"C\",\"D\",\"E\",\"F\",\"G\",\"H\"],\"z\":[[0.0,0.0,-0.5,-0.5,-0.5,-0.5,0.0,0.0],[0.1,0.0,-0.5,1.0,1.0,0.1,0.1,0.0],[0.0,-1.0,-0.5,1.0,-1.0,-1.0,0.1,0.0],[0.0,0.1,-1.0,1.0,-1.0,0.1,1.0,0.0],[0.0,0.1,-1.0,1.0,1.0,1.0,-0.5,-0.5],[0.0,0.1,-1.0,1.0,0.9,1.0,-1.0,-1.0],[0.0,0.1,1.0,1.0,0.9,0.9,0.9,1.0],[0.0,0.0,-0.5,0.0,-0.5,-0.5,-1.0,-0.5]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"<b>%{y}%{x}</b><br>%{customdata}<extra></extra>\",\"text\":[[\"A0\",\"A1\",\"A2\",\"A3\",\"A4\",\"A5\",\"A6\",\"A7\"],[\"B0\",\"B1\",\"B2\",\"\",\"\",\"B5\",\"B6\",\"B7\"],[\"C0\",\"\",\"C2\",\"\",\"\",\"\",\"C6\",\"C7\"],[\"D0\",\"D1\",\"\",\"\",\"\",\"D5\",\"\",\"D7\"],[\"E0\",\"E1\",\"\",\"\",\"\",\"\",\"E6\",\"E7\"],[\"F0\",\"F1\",\"\",\"\",\"\",\"\",\"\",\"\"],[\"G0\",\"G1\",\"\",\"\",\"\",\"\",\"\",\"\"],[\"H0\",\"H1\",\"H2\",\"H3\",\"H4\",\"H5\",\"\",\"H7\"]],\"texttemplate\":\"%{text}\",\"customdata\":[[\"I:1<br>S:0\",\"I:2<br>S:1\",\"I:3<br>S:2\",\"I:4<br>S:3\",\"I:5<br>S:4\",\"I:6<br>S:5\",\"I:7<br>S:6\",\"I:8<br>S:7\"],[\"I:9<br>S:8\",\"I:10<br>S:9\",\"I:11<br>S:10\",\"I:12<br>S:11\",\"I:13<br>S:12\",\"I:14<br>S:13\",\"I:15<br>S:14\",\"I:16<br>S:15\"],[\"I:17<br>S:16\",\"I:18<br>S:17\",\"I:19<br>S:18\",\"I:20<br>S:19\",\"I:21<br>S:20\",\"I:22<br>S:21\",\"I:23<br>S:22\",\"I:24<br>S:23\"],[\"I:25<br>S:24\",\"I:26<br>S:25\",\"I:27<br>S:26\",\"I:X<br>S:27\",\"I:X<br>S:28\",\"I:28<br>S:29\",\"I:29<br>S:30\",\"I:30<br>S:31\"],[\"I:31<br>S:32\",\"I:32<br>S:33\",\"I:33<br>S:34\",\"I:X<br>S:35\",\"I:X<br>S:36\",\"I:34<br>S:37\",\"I:35<br>S:38\",\"I:36<br>S:39\"],[\"I:37<br>S:40\",\"I:38<br>S:41\",\"I:39<br>S:42\",\"I:40<br>S:43\",\"I:41<br>S:44\",\"I:42<br>S:45\",\"I:43<br>S:46\",\"I:44<br>S:47\"],[\"I:45<br>S:48\",\"I:46<br>S:49\",\"I:47<br>S:50\",\"I:48<br>S:51\",\"I:49<br>S:52\",\"I:50<br>S:53\",\"I:51<br>S:54\",\"I:52<br>S:55\"],[\"I:53<br>S:56\",\"I:54<br>S:57\",\"I:55<br>S:58\",\"I:56<br>S:59\",\"I:57<br>S:60\",\"I:58<br>S:61\",\"I:59<br>S:62\",\"I:60<br>S:63\"]]}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"x\"},\"visible\":false},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"y\"},\"visible\":false},\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(0, 128, 128)\"],[0.16666666666666666,\"rgb(112, 164, 148)\"],[0.3333333333333333,\"rgb(180, 200, 168)\"],[0.5,\"rgb(246, 237, 189)\"],[0.6666666666666666,\"rgb(237, 187, 138)\"],[0.8333333333333334,\"rgb(222, 138, 90)\"],[1.0,\"rgb(202, 86, 44)\"]],\"cmid\":0.0,\"showscale\":false},\"title\":{\"text\":\"White To Play. Board State After Black Plays G4 \",\"x\":0.5},\"shapes\":[{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":0.8,\"y1\":1.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":0.8,\"x1\":1.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":1.8,\"y1\":2.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":2.8,\"y1\":3.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.8,\"x1\":4.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":3.8,\"y1\":4.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.78,\"x1\":4.22,\"y0\":4.78,\"y1\":5.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.8,\"x1\":5.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":6.8,\"x1\":7.2,\"y0\":4.8,\"y1\":5.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":1.8,\"x1\":2.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":2.8,\"x1\":3.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":3.75,\"x1\":4.25,\"y0\":5.75,\"y1\":6.25,\"line\":{\"color\":\"red\",\"width\":4}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":4.78,\"x1\":5.22,\"y0\":5.78,\"y1\":6.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":5.78,\"x1\":6.22,\"y0\":5.78,\"y1\":6.22,\"line\":{\"color\":\"purple\",\"width\":3}},{\"fillcolor\":\"black\",\"type\":\"circle\",\"x0\":6.8,\"x1\":7.2,\"y0\":5.8,\"y1\":6.2,\"line\":{\"color\":\"green\",\"width\":0.5}},{\"fillcolor\":\"white\",\"type\":\"circle\",\"x0\":5.8,\"x1\":6.2,\"y0\":6.8,\"y1\":7.2,\"line\":{\"color\":\"green\",\"width\":0.5}}],\"height\":400},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('ab0b33ef-8178-494f-b0c9-ef526f0f7528');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "layer = 6\n",
    "game_index = 1\n",
    "for move in [20, 21, 22]:\n",
    "    def plot_probe_outputs_figure(layer, game_index, move, **kwargs):\n",
    "        residual_stream = focus_cache[\"resid_post\", layer][game_index, move]\n",
    "        print(\"residual_stream\", residual_stream.shape)\n",
    "        probe_out = einops.einsum(residual_stream, linear_probe, \"d_model, d_model row col options -> row col options\")\n",
    "        probabilities = probe_out.softmax(dim=-1)\n",
    "        fig = imshow(probabilities, facet_col=2, facet_labels=[\"P(Empty)\", \"P(Their Colour)\", \"P(My Colour)\"], height=300, width=600, return_fig=True)\n",
    "        fig.update_layout(xaxis_visible=False, yaxis_visible=False, xaxis2_visible=False, xaxis3_visible=False, coloraxis_showscale=False)\n",
    "        fig.show()\n",
    "    plot_probe_outputs_figure(layer, game_index, move)\n",
    "\n",
    "    fig = plot_single_board(int_to_label(focus_games_int[game_index, :move+1]), return_fig=True)\n",
    "    fig.update_layout(xaxis_visible=False, yaxis_visible=False, coloraxis_showscale=False, height=400)\n",
    "    fig.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'F4 F3 D2 F5 G2 F2 G3 C4 E5 F6 D6 E2 B4 C5 G7 C1 G6 F7 G5 C3 B3 H6 G4'"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\" \".join(int_to_label(focus_games_int[game_index, :move+1]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.13"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "d4d1e4263499bec80672ea0156c357c1ee493ec2b1c70f0acce89fc37c4a6abe"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}