Skip to content

Commit

Permalink
Merge pull request #5 from GalyaZalesskaya/gz/refactor_names_verbosity
Browse files Browse the repository at this point in the history 
Refactor names/folders/objects for better verbosity
  • Loading branch information
@negvet
negvet authored Jun 6, 2024
2 parents c16465a + 418c01d commit 8438854
Show file tree
Hide file tree
Showing 50 changed files with 524 additions and 545 deletions.
1 change: 1 addition & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ otx_models/

*.jpg
*.jpeg
*.JPEG
*.png

html_build/
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@

### What's Changed

* Refactor names/folders/objects for better verbosity by @GalyaZalesskaya in https://github.com/openvinotoolkit/openvino_xai/pull/5
* Support classification task by @negvet in https://github.com/intel-sandbox/openvino_xai/commit/dd5fd9b73fe8c12e2d741792043372bcd900a850
* Support detection task by @negvet in https://github.com/intel-sandbox/openvino_xai/commit/84f285f2f40a8b1fc50a8cd49798aae37afd58dc
* Support Model API as inference engine by @negvet in https://github.com/intel-sandbox/openvino_xai/commit/5f575f122dedc0461975bd58f81e730a901a69a6
Expand Down
50 changes: 25 additions & 25 deletions GETTING_STARTED.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -272,7 +272,7 @@
"# This code returns gray-scale unprocessed saliency map\n",
"explanation = auto_explainer.explain(image)\n",
"logger.info(f\"Auto example: generated {len(explanation.map)} classification \"\n",
" f\"saliency maps of layout {explanation.layout} with shape {explanation.sal_map_shape}.\")\n",
" f\"saliency maps of layout {explanation.layout} with shape {explanation.shape}.\")\n",
"\n",
"# Save saliency maps stored in `explanation` object\n",
"output_dir = \"saliency_map/auto_explain/wo_postrocessing\"\n",
Expand Down Expand Up @@ -403,7 +403,7 @@
"explanation = explainer.explain(image)\n",
"logger.info(\n",
" f\"White-Box example w/o explain_parameters: generated {len(explanation.map)} classification \"\n",
" f\"saliency maps of layout {explanation.layout} with shape {explanation.sal_map_shape}.\"\n",
" f\"saliency maps of layout {explanation.layout} with shape {explanation.shape}.\"\n",
")\n",
"\n",
"# Save saliency maps stored in `explanation` object\n",
Expand All @@ -430,9 +430,9 @@
"`White Box explainer` can be configured with the following parameters:\n",
"- `target_layer` - specifies the layer after which the XAI nodes should be inserted (the last convolutional layer is a good default option). Example: `/backbone/conv/conv.2/Div`. This parameter can be useful if `WhiteBoxExplainer` fails to find a place where to insert XAI branch.\n",
"- `embed_normalization` - **default True** (for speed purposes), but you can disable embedding of normalization into the model.\n",
"- `explain_method_type` - **default reciprocam**:\n",
"- `explain_method` - **default reciprocam**:\n",
"\n",
" For Classification models `White Box` algorithm supports 2 `XAIMethodType`:\n",
" For Classification models `White Box` algorithm supports 2 `Method`:\n",
" - activationmap - returns a single saliency map regardless of the classes\n",
" - reciprocam - returns saliency maps for each class the model can detect\n",
"\n",
Expand Down Expand Up @@ -471,13 +471,13 @@
],
"source": [
"from openvino_xai.model import XAIClassificationModel\n",
"from openvino_xai.parameters import ClassificationExplainParametersWB, XAIMethodType\n",
"from openvino_xai.parameters import ClassificationExplainParametersWB, Method\n",
"\n",
"# Parametrize White Box Explainer\n",
"explain_parameters = ClassificationExplainParametersWB(\n",
" # target_layer=\"/backbone/conv/conv.2/Div\", # OTX mobilenet_v3\n",
" # target_layer=\"/backbone/features/final_block/activate/Mul\", # OTX efficientnet\n",
" explain_method_type=XAIMethodType.RECIPROCAM,\n",
" explain_method=Method.RECIPROCAM,\n",
")\n",
"\n",
"# Create an OpenVINO™ ModelAPI model wrapper for Classification model\n",
Expand All @@ -491,7 +491,7 @@
"explanation = explainer.explain(image)\n",
"logger.info(\n",
" f\"White-Box example w/ explain_parameters: generated {len(explanation.map)} classification \"\n",
" f\"saliency maps of layout {explanation.layout} with shape {explanation.sal_map_shape}.\"\n",
" f\"saliency maps of layout {explanation.layout} with shape {explanation.shape}.\"\n",
")\n",
"\n",
"# Save saliency maps stored in `explanation` object\n",
Expand Down Expand Up @@ -564,11 +564,11 @@
"source": [
"from openvino_xai.explain import WhiteBoxExplainer\n",
"from openvino_xai.model import XAIClassificationModel\n",
"from openvino_xai.parameters import PostProcessParameters\n",
"from openvino_xai.parameters import VisualizationParameters\n",
"from openvino_xai.saliency_map import TargetExplainGroup\n",
"\n",
"# Pass postprocessing parameters\n",
"post_processing_parameters = PostProcessParameters(overlay=True)\n",
"visualization_parameters = VisualizationParameters(overlay=True)\n",
"\n",
"# Create an OpenVINO™ ModelAPI model wrapper with XAI head inserted into the model graph\n",
"model = XAIClassificationModel.create_model(model_path, model_type=\"Classification\")\n",
Expand All @@ -578,11 +578,11 @@
"explainer = WhiteBoxExplainer(model)\n",
"explanation = explainer.explain(image, \n",
" TargetExplainGroup.PREDICTED_CLASSES, # default option, can be ommited\n",
" post_processing_parameters=post_processing_parameters\n",
" visualization_parameters=visualization_parameters\n",
" )\n",
"logger.info(\n",
" f\"White-Box example w/ postprocessing: generated {len(explanation.map)} classification \"\n",
" f\"saliency maps of layout {explanation.layout} with shape {explanation.sal_map_shape}.\"\n",
" f\"saliency maps of layout {explanation.layout} with shape {explanation.shape}.\"\n",
")\n",
"\n",
"# Save saliency maps stored in `explanation` object\n",
Expand Down Expand Up @@ -646,7 +646,7 @@
"from openvino_xai.saliency_map import TargetExplainGroup\n",
"\n",
"# Pass postprocessing parameters\n",
"post_processing_parameters = PostProcessParameters(overlay=True)\n",
"visualization_parameters = VisualizationParameters(overlay=True)\n",
"\n",
"# Create an OpenVINO™ ModelAPI model wrapper with XAI head inserted into the model graph\n",
"model = XAIClassificationModel.create_model(model_path, model_type=\"Classification\")\n",
Expand All @@ -666,7 +666,7 @@
"# TargetExplainGroup.PREDICTED_CLASSES)\n",
"logger.info(\n",
" f\"White-Box example w/ all_classes: generated {len(explanation.map)} classification \"\n",
" f\"saliency maps of layout {explanation.layout} with shape {explanation.sal_map_shape}.\"\n",
" f\"saliency maps of layout {explanation.layout} with shape {explanation.shape}.\"\n",
")\n",
"\n",
"# Save saliency maps stored in `explanation` object\n",
Expand Down Expand Up @@ -752,7 +752,7 @@
],
"source": [
"from openvino.model_api.models import ClassificationModel\n",
"from openvino_xai.parameters import PostProcessParameters\n",
"from openvino_xai.parameters import VisualizationParameters\n",
"from openvino_xai.explain import RISEExplainer\n",
"\n",
"# Create an OpenVINO™ ModelAPI model wrapper for Classification model\n",
Expand All @@ -763,15 +763,15 @@
")\n",
"\n",
"# Pass postprocessing parameters\n",
"post_processing_parameters = PostProcessParameters(overlay=True)\n",
"visualization_parameters = VisualizationParameters(overlay=True)\n",
"\n",
"# Explainer initialization and explanation\n",
"explainer = RISEExplainer(model)\n",
"# This code returns colored saliency map after processing\n",
"explanation = explainer.explain(image, post_processing_parameters=post_processing_parameters)\n",
"explanation = explainer.explain(image, visualization_parameters=visualization_parameters)\n",
"logger.info(\n",
" f\"Black-Box example w/ post_processing_parameters: generated {len(explanation.map)} \"\n",
" f\"classification saliency maps of layout {explanation.layout} with shape {explanation.sal_map_shape}.\"\n",
" f\"Black-Box example w/ visualization_parameters: generated {len(explanation.map)} \"\n",
" f\"classification saliency maps of layout {explanation.layout} with shape {explanation.shape}.\"\n",
")\n",
"\n",
"# Save saliency maps stored in `explanation` object\n",
Expand Down Expand Up @@ -828,19 +828,19 @@
],
"source": [
"explainer = RISEExplainer(model, num_cells=4, num_masks=1000)\n",
"explanation = explainer.explain(image, post_processing_parameters=post_processing_parameters)\n",
"explanation = explainer.explain(image, visualization_parameters=visualization_parameters)\n",
"explanation.save(\"saliency_map/black_box/4_cells_1000_masks\", image_name)\n",
"\n",
"explainer = RISEExplainer(model, num_cells=8, num_masks=5000)\n",
"explanation = explainer.explain(image, post_processing_parameters=post_processing_parameters)\n",
"explanation = explainer.explain(image, visualization_parameters=visualization_parameters)\n",
"explanation.save(\"saliency_map/black_box/8_cells_5000_masks\", image_name)\n",
"\n",
"explainer = RISEExplainer(model, num_cells=16, num_masks=10000)\n",
"explanation = explainer.explain(image, post_processing_parameters=post_processing_parameters)\n",
"explanation = explainer.explain(image, visualization_parameters=visualization_parameters)\n",
"explanation.save(\"saliency_map/black_box/16_cells_10000_masks\", image_name)\n",
"\n",
"explainer = RISEExplainer(model, num_cells=24, num_masks=15000)\n",
"explanation = explainer.explain(image, post_processing_parameters=post_processing_parameters)\n",
"explanation = explainer.explain(image, visualization_parameters=visualization_parameters)\n",
"explanation.save(\"saliency_map/black_box/24_cells_15000_masks\", image_name)"
]
},
Expand Down Expand Up @@ -1083,12 +1083,12 @@
"\n",
"# Explainer initialization and explanation\n",
"auto_explainer = ClassificationAutoExplainer(model)\n",
"post_processing_parameters = PostProcessParameters(overlay=True)\n",
"visualization_parameters = VisualizationParameters(overlay=True)\n",
"\n",
"# This code returns gray-scale unprocessed saliency map\n",
"explanation = auto_explainer.explain(image, post_processing_parameters=post_processing_parameters)\n",
"explanation = auto_explainer.explain(image, visualization_parameters=visualization_parameters)\n",
"logger.info(f\"Auto example: generated {len(explanation.map)} classification \"\n",
" f\"saliency maps of layout {explanation.layout} with shape {explanation.sal_map_shape}.\")\n",
" f\"saliency maps of layout {explanation.layout} with shape {explanation.shape}.\")\n",
"\n",
"# Save saliency maps stored in `explanation` object\n",
"output_dir = \"saliency_map/timm_model\"\n",
Expand Down
15 changes: 7 additions & 8 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,9 @@ To explain [OpenVINO™](https://github.com/openvinotoolkit/openvino) Intermedia
preprocessing function (and sometimes postprocessing).

```python
explainer = Explainer(
explainer = xai.Explainer(
model,
task_type=TaskType.CLASSIFICATION,
task=xai.Task.CLASSIFICATION,
preprocess_fn=preprocess_fn,
)
explanation = explainer(data, explanation_parameters)
Expand All @@ -66,9 +66,8 @@ import cv2
import numpy as np
import openvino.runtime as ov

from openvino_xai.common.parameters import TaskType
from openvino_xai.explanation.explainer import Explainer
from openvino_xai.explanation.explanation_parameters import ExplanationParameters
import openvino_xai as xai
from openvino_xai.explainer.explanation_parameters import ExplanationParameters


def preprocess_fn(x: np.ndarray) -> np.ndarray:
Expand All @@ -82,9 +81,9 @@ def preprocess_fn(x: np.ndarray) -> np.ndarray:
model = ov.Core().read_model("path/to/model.xml") # type: ov.Model

# Explainer object will prepare and load the model once in the beginning
explainer = Explainer(
explainer = xai.Explainer(
model,
task_type=TaskType.CLASSIFICATION,
task=xai.Task.CLASSIFICATION,
preprocess_fn=preprocess_fn,
)

Expand All @@ -95,7 +94,7 @@ explanation_parameters = ExplanationParameters(
)
explanation = explainer(image, explanation_parameters)

explanation: ExplanationResult
explanation: Explanation
explanation.saliency_map: Dict[int: np.ndarray] # key - class id, value - processed saliency map e.g. 354x500x3

# Saving saliency maps
Expand Down
52 changes: 25 additions & 27 deletions docs/Usage.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,11 @@ Content:
## Explainer - interface to XAI algorithms

```python
explainer = Explainer(
import openvino_xai as xai

explainer = xai.Explainer(
model,
task_type=TaskType.CLASSIFICATION,
task=xai.Task.CLASSIFICATION,
preprocess_fn=preprocess_fn,
)
explanation = explainer(data, explanation_parameters)
Expand All @@ -43,9 +45,8 @@ import cv2
import numpy as np
import openvino.runtime as ov

from openvino_xai.common.parameters import TaskType
from openvino_xai.explanation.explainer import Explainer
from openvino_xai.explanation.explanation_parameters import ExplanationParameters
import openvino_xai as xai
from openvino_xai.explainer.explanation_parameters import ExplanationParameters


def preprocess_fn(x: np.ndarray) -> np.ndarray:
Expand All @@ -59,9 +60,9 @@ def preprocess_fn(x: np.ndarray) -> np.ndarray:
model = ov.Core().read_model("path/to/model.xml") # type: ov.Model

# Explainer object will prepare and load the model once in the beginning
explainer = Explainer(
explainer = xai.Explainer(
model,
task_type=TaskType.CLASSIFICATION,
task=xai.Task.CLASSIFICATION,
preprocess_fn=preprocess_fn,
)

Expand Down Expand Up @@ -90,11 +91,9 @@ import cv2
import numpy as np
import openvino.runtime as ov

from openvino_xai.common.parameters import TaskType, XAIMethodType
from openvino_xai.explanation.explainer import Explainer
from openvino_xai.explanation.explanation_parameters import ExplainMode, ExplanationParameters, TargetExplainGroup,
PostProcessParameters
from openvino_xai.insertion.insertion_parameters import ClassificationInsertionParameters
import openvino_xai as xai
from openvino_xai.explainer.parameters import ExplainMode, ExplanationParameters, TargetExplainGroup, VisualizationParameters
from openvino_xai.inserter.parameters import ClassificationInsertionParameters


def preprocess_fn(x: np.ndarray) -> np.ndarray:
Expand All @@ -111,13 +110,13 @@ model = ov.Core().read_model("path/to/model.xml") # type: ov.Model
insertion_parameters = ClassificationInsertionParameters(
# target_layer="last_conv_node_name", # target_layer - node after which XAI branch will be inserted
embed_normalization=True, # True by default. If set to True, saliency map normalization is embedded in the model
explain_method_type=XAIMethodType.RECIPROCAM, # ReciproCAM is the default XAI method for CNNs
explain_method=xai.Method.RECIPROCAM, # ReciproCAM is the default XAI method for CNNs
)

# Explainer object will prepare and load the model once in the beginning
explainer = Explainer(
explainer = xai.Explainer(
model,
task_type=TaskType.CLASSIFICATION,
task=xai.Task.CLASSIFICATION,
preprocess_fn=preprocess_fn,
explain_mode=ExplainMode.WHITEBOX,
insertion_parameters=insertion_parameters,
Expand All @@ -131,7 +130,7 @@ explanation_parameters = ExplanationParameters(
target_explain_group=TargetExplainGroup.CUSTOM,
target_explain_labels=[11, 14], # target classes to explain, also ['dog', 'person'] is a valid input
label_names=voc_labels,
post_processing_parameters=PostProcessParameters(overlay=True), # by default, saliency map overlay over image
visualization_parameters=VisualizationParameters(overlay=True), # by default, saliency map overlay over image
)
explanation = explainer(image, explanation_parameters)

Expand All @@ -154,9 +153,8 @@ import cv2
import numpy as np
import openvino.runtime as ov

from openvino_xai.common.parameters import TaskType
from openvino_xai.explanation.explainer import Explainer
from openvino_xai.explanation.explanation_parameters import ExplainMode, ExplanationParameters
import openvino_xai as xai
from openvino_xai.explainer.explanation_parameters import ExplainMode, ExplanationParameters


def preprocess_fn(x: np.ndarray) -> np.ndarray:
Expand All @@ -176,9 +174,9 @@ def postprocess_fn(x: ov.utils.data_helpers.wrappers.OVDict):
model = ov.Core().read_model("path/to/model.xml") # type: ov.Model

# Explainer object will prepare and load the model once in the beginning
explainer = Explainer(
explainer = xai.Explainer(
model,
task_type=TaskType.CLASSIFICATION,
task=xai.Task.CLASSIFICATION,
preprocess_fn=preprocess_fn,
postprocess_fn=postprocess_fn,
explain_mode=ExplainMode.BLACKBOX,
Expand Down Expand Up @@ -212,9 +210,9 @@ Note: original model outputs are not affected and the model should be inferable

```python
import openvino.runtime as ov
import openvino_xai as ovxai
from openvino_xai.common.parameters import TaskType, XAIMethodType
from openvino_xai.insertion.insertion_parameters import ClassificationInsertionParameters

import openvino_xai as xai
from openvino_xai.inserter.parameters import ClassificationInsertionParameters


# Creating model
Expand All @@ -224,13 +222,13 @@ model = ov.Core().read_model("path/to/model.xml") # type: ov.Model
insertion_parameters = ClassificationInsertionParameters(
# target_layer="last_conv_node_name", # target_layer - node after which XAI branch will be inserted
embed_normalization=True, # True by default. If set to True, saliency map normalization is embedded in the model
explain_method_type=XAIMethodType.RECIPROCAM, # ReciproCAM is the default XAI method for CNNs
explain_method=xai.Method.RECIPROCAM, # ReciproCAM is the default XAI method for CNNs
)

# Inserting XAI branch into the model graph
model_xai = ovxai.insert_xai(
model_xai = xai.insert_xai(
model=model,
task_type=TaskType.CLASSIFICATION,
task=xai.Task.CLASSIFICATION,
insertion_parameters=insertion_parameters,
) # type: ov.Model

Expand Down
4 changes: 2 additions & 2 deletions docs/api/source/api.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,9 @@ To explain the model (getting saliency maps), use openvino_xai.explanation
Algorithms
----------

To access/modify implemented XAI methods, use openvino_xai.algorithms
To access/modify implemented XAI methods, use openvino_xai.methods

.. automodule:: openvino_xai.algorithms
.. automodule:: openvino_xai.methods
:members:

Insertion
Expand Down
Loading

0 comments on commit 8438854

Please sign in to comment.