notebooks/220-yolov5-accuracy-check-and-quantization/220-yolov5-accuracy-check-and-quantization.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "af86d01b",
   "metadata": {
    "id": "JwEAhQVzkAwA"
   },
   "source": [
    "# Quantize the Ultralytics YOLOv5 model and check accuracy using the OpenVINO POT API\n",
    "\n",
    "This tutorial demonstrates step-by-step how to perform model quantization using the OpenVINO [Post-Training Optimization Tool (POT)](https://docs.openvino.ai/latest/pot_introduction.html), compare model accuracy between the FP32 precision and quantized INT8 precision models and run a demo of model inference based on sample code from [Ultralytics Yolov5](https://github.com/ultralytics/yolov5) with the OpenVINO backend.\n",
    "\n",
    "First, we will export the YOLOv5m model to OpenVINO IR by following the [export instructions](https://github.com/ultralytics/yolov5/issues/251) in the [Ultralytics YOLOv5 repo](https://github.com/ultralytics/yolov5). Then we use the OpenVINO [Post-Training Optimization Tool (POT)](https://docs.openvino.ai/latest/pot_introduction.html) API to quantize the model based on the Non-Max Suppression (NMS) processing provided by Ultralytics.\n",
    "\n",
    "OpenVINO POT provides two usages:\n",
    "1. Use the API to override the model DataLoader class with custom image/annotation loading and preprocessing and identify your own class which is inherited from Metric for inference result postprocessing and accuracy calculation.\n",
    "2. Use POT command line tool with the adapters provided by [Accuracy Checker](https://github.com/openvinotoolkit/open_model_zoo/blob/master/tools/accuracy_checker/README.md), pre/postprocessing and metric by configuration file. This is recommend when using [Open Model Zoo](https://github.com/openvinotoolkit/open_model_zoo) models with the omz_quantizer tool.\n",
    "\n",
    "The data pre/post-processing functions provided by Ultralytics are different from the Accuracy Checker provided configuration for YOLOv3, so we will need to use the POT API (approach #1) with a custom DataLoader and Metric. This allows us to include pre- and post-processing from Ultralytics in our quantization pipeline. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4cdf8008",
   "metadata": {
    "id": "QB4Yo-rGGLmV"
   },
   "source": [
    "## Preparation\n",
    "\n",
    "### Download the YOLOv5 model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1c8cc881",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import torch\n",
    "from pathlib import Path\n",
    "from addict import Dict \n",
    "from IPython.display import Markdown, display\n",
    "\n",
    "if not Path(\"./yolov5/\").exists():\n",
    "    command_download = f'{\"git clone https://github.com/ultralytics/yolov5.git -b v6.1\"}'\n",
    "    command_download = \" \".join(command_download.split())\n",
    "    print(\"Download Ultralytics Yolov5 project source:\")\n",
    "    display(Markdown(f\"`{command_download}`\"))\n",
    "    download_res = %sx $command_download\n",
    "else:\n",
    "    print(\"Ultralytics Yolov5 repo already exists.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "527d85d8-2cc9-4073-a3d8-1c2f6872ee2f",
   "metadata": {},
   "source": [
    "### Conversion of the YOLOv5 model to OpenVINO\n",
    "1. Convert Pytorch model to ONNX\n",
    "    \n",
    "2. Convert ONNX to OpenVINO\n",
    "    \n",
    "    Call the OpenVINO Model Optimizer tool to convert the ONNX model to OpenVINO IR, with FP32 precision."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ae772049-9f8d-42c9-b2c2-7566d71e6988",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Convert PyTorch model to ONNX Model:\")\n",
    "command_export = f'{\"cd yolov5 && python export.py --weights yolov5m/yolov5m.pt --imgsz 640 --batch-size 1 --include onnx \"}'\n",
    "display(Markdown(f\"`{command_export}`\"))\n",
    "! $command_export\n",
    "\n",
    "print(\"Convert ONNX model to OpenVINO IR:\")\n",
    "onnx_path = \"./yolov5/yolov5m/yolov5m.onnx\"\n",
    "IMAGE_HEIGHT = 640\n",
    "IMAGE_WIDTH = 640\n",
    "model_output_path = \"./yolov5/yolov5m/yolov5m_openvino_model/\"\n",
    "\n",
    "# Construct the command for Model Optimizer.\n",
    "command_mo = f\"\"\"mo\n",
    "                 --input_model \"{onnx_path}\"\n",
    "                 --output_dir \"{model_output_path}\"\n",
    "                 \"\"\"\n",
    "command_mo = \" \".join(command_mo.split())\n",
    "display(Markdown(f\"`{command_mo}`\"))\n",
    "! $command_mo"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37b80fe4",
   "metadata": {},
   "source": [
    "### Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3a9303a8",
   "metadata": {},
   "outputs": [],
   "source": [
    "sys.path.append(\"./yolov5\")\n",
    "\n",
    "from yolov5.utils.datasets import create_dataloader\n",
    "from yolov5.utils.general import check_dataset, non_max_suppression, scale_coords, xywh2xyxy, check_yaml,increment_path\n",
    "from yolov5.utils.metrics import ap_per_class\n",
    "from yolov5.val import process_batch\n",
    "\n",
    "from openvino.tools.pot.api import Metric, DataLoader\n",
    "from openvino.tools.pot.engines.ie_engine import IEEngine\n",
    "from openvino.tools.pot.graph import load_model, save_model\n",
    "from openvino.tools.pot.graph.model_utils import compress_model_weights\n",
    "from openvino.tools.pot.pipeline.initializer import create_pipeline\n",
    "from openvino.tools.pot.utils.logger import init_logger, get_logger"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b33033e",
   "metadata": {
    "id": "u5xKw0hR0jq6"
   },
   "source": [
    "## Model Quantization with POT\n",
    "### Create YOLOv5 DataLoader class\n",
    "\n",
    "Create a class for the loading YOLOv5 dataset and annotation which inherits from POT API class DataLoader. The Ultralytics YOLOv5 training process requires image data normalization from [0,225] 8-bit integer range to [0.0,1.0] 32-bit floating point range."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c9600481",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "xGKkMRfvi0op",
    "outputId": "4eb1f9af-a4c5-424c-f808-dd9cc2600975"
   },
   "outputs": [],
   "source": [
    "class YOLOv5DataLoader(DataLoader):\n",
    "    \"\"\" Inherit from DataLoader function and implement for YOLOv5.\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, config):\n",
    "        if not isinstance(config, Dict):\n",
    "            config = Dict(config)\n",
    "        super().__init__(config)\n",
    "\n",
    "        self._data_source = config.data_source\n",
    "        self._imgsz = config.imgsz\n",
    "        self._batch_size = 1\n",
    "        self._stride = 32\n",
    "        self._single_cls = config.single_cls\n",
    "        self._pad = 0.5\n",
    "        self._rect = False\n",
    "        self._workers = 1\n",
    "        self._data_loader = self._init_dataloader()\n",
    "        self._data_iter = iter(self._data_loader)\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self._data_loader.dataset)\n",
    "\n",
    "    def _init_dataloader(self):\n",
    "        dataloader = create_dataloader(self._data_source['val'], imgsz=self._imgsz, batch_size=self._batch_size, stride=self._stride,\n",
    "                                       single_cls=self._single_cls, pad=self._pad, rect=self._rect, workers=self._workers)[0]\n",
    "        return dataloader\n",
    "\n",
    "    def __getitem__(self, item):\n",
    "        try:\n",
    "            batch_data = next(self._data_iter)\n",
    "        except StopIteration:\n",
    "            self._data_iter = iter(self._data_loader)\n",
    "            batch_data = next(self._data_iter)\n",
    "\n",
    "        im, target, path, shape = batch_data\n",
    "\n",
    "        im = im.float()  \n",
    "        im /= 255  \n",
    "        nb, _, height, width = im.shape  \n",
    "        img = im.cpu().detach().numpy()\n",
    "        target = target.cpu().detach().numpy()\n",
    "\n",
    "        annotation = dict()\n",
    "        annotation['image_path'] = path\n",
    "        annotation['target'] = target\n",
    "        annotation['batch_size'] = nb\n",
    "        annotation['shape'] = shape\n",
    "        annotation['width'] = width\n",
    "        annotation['height'] = height\n",
    "        annotation['img'] = img\n",
    "\n",
    "        return (item, annotation), img"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bad92bb9",
   "metadata": {
    "id": "Rhc_7EObUypw"
   },
   "source": [
    "### Create YOLOv5 Metric Class\n",
    "\n",
    "Create a class to measure the model performance by Mean Average Precision (mAP) with the COCO dataset predicted result and annotation value, after applying Ultralytics NMS routine (`yolov5.utils.general.non_max_suppression`). Here we use both AP\\@0.5 and AP\\@0.5:0.95 as the measurement standard. This class should be inherited from the POT API Metric class.\n",
    "\n",
    "The COCOMetric.update() function contains post-processing with Non-Max Suppression to sort boxes by score and select the box with the highest score."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "659aeac7",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ipQWpbgQUxoo",
    "outputId": "bbc1734a-c2a2-4261-ed45-264b9e3edd00"
   },
   "outputs": [],
   "source": [
    "class COCOMetric(Metric):\n",
    "    \"\"\" Inherit from DataLoader function and implement for YOLOv5.\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, config):\n",
    "        super().__init__()\n",
    "        self._metric_dict = {\"AP@0.5\": [], \"AP@0.5:0.95\": []}\n",
    "        self._names = (*self._metric_dict,)\n",
    "        self._stats = []\n",
    "        self._last_stats = []\n",
    "        self._conf_thres = config.conf_thres\n",
    "        self._iou_thres = config.iou_thres\n",
    "        self._single_cls = config.single_cls\n",
    "        self._nc = config.nc\n",
    "        self._class_names = {idx:name for idx,name in enumerate(config.names)}\n",
    "        self._device = config.device\n",
    "\n",
    "    @property\n",
    "    def value(self):\n",
    "        \"\"\" Returns metric value for the last model output.\n",
    "        Both use AP@0.5 and AP@0.5:0.95\n",
    "        \"\"\"\n",
    "        mp, mr, map50, map = self._process_stats(self._last_stats)\n",
    "\n",
    "        return {self._names[0]: [map50], self._names[1]: [map]}\n",
    "\n",
    "    @property\n",
    "    def avg_value(self):\n",
    "        \"\"\" Returns metric value for all model outputs.\n",
    "        Both use AP@0.5 and AP@0.5:0.95\n",
    "        \"\"\"\n",
    "        mp, mr, map50, map = self._process_stats(self._stats)\n",
    "\n",
    "        return {self._names[0]: map50, self._names[1]: map}\n",
    "\n",
    "    def _process_stats(self, stats):\n",
    "        mp, mr, map50, map = 0.0, 0.0, 0.0, 0.0\n",
    "        stats = [np.concatenate(x, 0) for x in zip(*stats)]  \n",
    "        if len(stats) and stats[0].any():\n",
    "            tp, fp, p, r, f1, ap, ap_class = ap_per_class(*stats, plot=False, save_dir=None, names=self._class_names)\n",
    "            ap50, ap = ap[:, 0], ap.mean(1) \n",
    "            mp, mr, map50, map = p.mean(), r.mean(), ap50.mean(), ap.mean()\n",
    "            np.bincount(stats[3].astype(np.int64), minlength=self._nc)  \n",
    "        else:\n",
    "            torch.zeros(1)\n",
    "\n",
    "        return mp, mr, map50, map\n",
    "\n",
    "    def update(self, output, target):\n",
    "        \"\"\" Calculates and updates metric value\n",
    "        Contains postprocessing part from Ultralytics YOLOv5 project\n",
    "        :param output: model output\n",
    "        :param target: annotations\n",
    "        \"\"\"\n",
    "\n",
    "        annotation = target[0][\"target\"]\n",
    "        width = target[0][\"width\"]\n",
    "        height = target[0][\"height\"]\n",
    "        shapes = target[0][\"shape\"]\n",
    "        paths = target[0][\"image_path\"]\n",
    "        im = target[0][\"img\"]\n",
    "\n",
    "        iouv = torch.linspace(0.5, 0.95, 10).to(self._device)  # iou vector for mAP@0.5:0.95\n",
    "        niou = iouv.numel()\n",
    "        seen = 0\n",
    "        stats = []\n",
    "        # NMS\n",
    "        annotation = torch.Tensor(annotation)\n",
    "        annotation[:, 2:] *= torch.Tensor([width, height, width, height]).to(self._device)  # to pixels\n",
    "        lb = []\n",
    "        out = output[0]\n",
    "        out = torch.Tensor(out).to(self._device)\n",
    "        out = non_max_suppression(out, self._conf_thres, self._iou_thres, labels=lb,\n",
    "                                  multi_label=True, agnostic=self._single_cls)\n",
    "        # Metrics\n",
    "        for si, pred in enumerate(out):\n",
    "            labels = annotation[annotation[:, 0] == si, 1:]\n",
    "            nl = len(labels)\n",
    "            tcls = labels[:, 0].tolist() if nl else []  # target class\n",
    "            _, shape = Path(paths[si]), shapes[si][0]\n",
    "            seen += 1\n",
    "\n",
    "            if len(pred) == 0:\n",
    "                if nl:\n",
    "                    stats.append((torch.zeros(0, niou, dtype=torch.bool), torch.Tensor(), torch.Tensor(), tcls))\n",
    "                continue\n",
    "\n",
    "            # Predictions\n",
    "            if self._single_cls:\n",
    "                pred[:, 5] = 0\n",
    "            predn = pred.clone()\n",
    "            scale_coords(im[si].shape[1:], predn[:, :4], shape, shapes[si][1])  # native-space pred\n",
    "\n",
    "            # Evaluate\n",
    "            if nl:\n",
    "                tbox = xywh2xyxy(labels[:, 1:5])  # target boxes\n",
    "                scale_coords(im[si].shape[1:], tbox, shape, shapes[si][1])  # native-space labels\n",
    "                labelsn = torch.cat((labels[:, 0:1], tbox), 1)  # native-space labels\n",
    "                correct = process_batch(predn, labelsn, iouv)\n",
    "            else:\n",
    "                correct = torch.zeros(pred.shape[0], niou, dtype=torch.bool)\n",
    "            stats.append((correct.cpu(), pred[:, 4].cpu(), pred[:, 5].cpu(), tcls))\n",
    "            self._stats.append((correct.cpu(), pred[:, 4].cpu(), pred[:, 5].cpu(), tcls))\n",
    "        self._last_stats = stats\n",
    "\n",
    "    def reset(self):\n",
    "        \"\"\" Resets metric \"\"\"\n",
    "        self._metric_dict = {\"AP@0.5\": [], \"AP@0.5:0.95\": []}\n",
    "        self._last_stats = []\n",
    "        self._stats = []\n",
    "\n",
    "    def get_attributes(self):\n",
    "        \"\"\"\n",
    "        Returns a dictionary of metric attributes {metric_name: {attribute_name: value}}.\n",
    "        Required attributes: 'direction': 'higher-better' or 'higher-worse'\n",
    "                                                 'type': metric type\n",
    "        \"\"\"\n",
    "        return {self._names[0]: {'direction': 'higher-better',\n",
    "                                 'type': 'AP@0.5'},\n",
    "                self._names[1]: {'direction': 'higher-better',\n",
    "                                 'type': 'AP@0.5:0.95'}}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b490241b",
   "metadata": {
    "id": "6JSoEIk60uxV"
   },
   "source": [
    "### Set POT Configuration\n",
    "\n",
    "Create a function to set the configuration of the model, engine, dataset, metric and algorithms, which are used by the POT quantization pipeline API.\n",
    "\n",
    "Here we use \"DefaultQuantization\" method for best performance of the quantization step. Alternatively, the \"AccuracyAwareQuantization\" method can be used to keep accuracy loss below a predefined threshold, at the cost of performance during the quantization process (it takes longer to quantize the model). This is useful when DefaultQuantization results in more accuracy loss than desired."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5c1e8029",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def get_config():\n",
    "    \"\"\" Set the configuration of the model, engine, \n",
    "    dataset, metric and quantization algorithm.\n",
    "    \"\"\"\n",
    "    config = dict()\n",
    "    data_yaml = check_yaml(\"./yolov5/data/coco128.yaml\")\n",
    "    data = check_dataset(data_yaml)\n",
    "\n",
    "    model_config = Dict({\n",
    "        \"model_name\": \"yolov5m\",\n",
    "        \"model\": \"./yolov5/yolov5m/yolov5m_openvino_model/yolov5m.xml\",\n",
    "        \"weights\": \"./yolov5/yolov5m/yolov5m_openvino_model/yolov5m.bin\"\n",
    "    })\n",
    "\n",
    "    engine_config = Dict({\n",
    "        \"device\": \"CPU\",\n",
    "        \"stat_requests_number\": 8,\n",
    "        \"eval_requests_number\": 8\n",
    "    })\n",
    "\n",
    "    dataset_config = Dict({\n",
    "        \"data_source\": data,\n",
    "        \"imgsz\": 640,\n",
    "        \"single_cls\": True,\n",
    "    })\n",
    "\n",
    "    metric_config = Dict({\n",
    "        \"conf_thres\": 0.001,\n",
    "        \"iou_thres\": 0.65,\n",
    "        \"single_cls\": True,\n",
    "        \"nc\": 1 ,  # if opt.single_cls else int(data['nc']),\n",
    "        \"names\": data[\"names\"],\n",
    "        \"device\": \"cpu\"\n",
    "    })\n",
    "\n",
    "    algorithms = [\n",
    "        {\n",
    "            \"name\": \"DefaultQuantization\",  # or AccuracyAwareQuantization\n",
    "            \"params\": {\n",
    "                    \"target_device\": \"CPU\",\n",
    "                    \"preset\": \"mixed\",\n",
    "                    \"stat_subset_size\": 300\n",
    "            }\n",
    "        }\n",
    "    ]\n",
    "\n",
    "    config[\"model\"] = model_config\n",
    "    config[\"engine\"] = engine_config\n",
    "    config[\"dataset\"] = dataset_config\n",
    "    config[\"metric\"] = metric_config\n",
    "    config[\"algorithms\"] = algorithms\n",
    "    \n",
    "    return config"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "608279bb",
   "metadata": {
    "id": "FAGmlKQ83ecE"
   },
   "source": [
    "### Run Quantization Pipeline and Accuracy Verification\n",
    "\n",
    "The following 9 steps show how to quantize the model using the POT API. The optimized model and collected min-max values will be saved. \n",
    "\n",
    "It will take few minutes for generating FP32-INT8 model, please check the quantized model is successfully saved and you get the metric results of INT8 model before running on next cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "738cb5e7",
   "metadata": {
    "id": "QTOoQnSetzQM",
    "tags": []
   },
   "outputs": [],
   "source": [
    "\"\"\" Download dataset and set config\n",
    "\"\"\"\n",
    "print(\"Run the POT. This will take few minutes...\")\n",
    "config = get_config()  \n",
    "init_logger(level='INFO')\n",
    "logger = get_logger(__name__)\n",
    "save_dir = increment_path(Path(\"./yolov5/yolov5m/yolov5m_openvino_model/\"), exist_ok=True)  # increment run\n",
    "save_dir.mkdir(parents=True, exist_ok=True)  # make dir\n",
    "\n",
    "# Step 1: Load the model.\n",
    "model = load_model(config[\"model\"])\n",
    "\n",
    "# Step 2: Initialize the data loader.\n",
    "data_loader = YOLOv5DataLoader(config[\"dataset\"])\n",
    "\n",
    "# Step 3 (Optional. Required for AccuracyAwareQuantization): Initialize the metric.\n",
    "metric = COCOMetric(config[\"metric\"])\n",
    "\n",
    "# Step 4: Initialize the engine for metric calculation and statistics collection.\n",
    "engine = IEEngine(config=config[\"engine\"], data_loader=data_loader, metric=metric)\n",
    "\n",
    "# Step 5: Create a pipeline of compression algorithms.\n",
    "pipeline = create_pipeline(config[\"algorithms\"], engine)\n",
    "\n",
    "metric_results = None\n",
    "\n",
    "# Check the FP32 model accuracy.\n",
    "metric_results_fp32 = pipeline.evaluate(model)\n",
    "\n",
    "logger.info(\"FP32 model metric_results: {}\".format(metric_results_fp32))\n",
    "\n",
    "# Step 6: Execute the pipeline to calculate Min-Max value\n",
    "compressed_model = pipeline.run(model)\n",
    "\n",
    "# Step 7 (Optional):  Compress model weights to quantized precision\n",
    "#                     in order to reduce the size of final .bin file.\n",
    "compress_model_weights(compressed_model)\n",
    "\n",
    "# Step 8: Save the compressed model to the desired path.\n",
    "optimized_save_dir = Path(save_dir).joinpath(\"optimized\")\n",
    "save_model(compressed_model, Path(Path.cwd()).joinpath(optimized_save_dir), config[\"model\"][\"model_name\"])\n",
    "\n",
    "# Step 9 (Optional): Evaluate the compressed model. Print the results.\n",
    "metric_results_i8 = pipeline.evaluate(compressed_model)\n",
    "\n",
    "logger.info(\"Save quantized model in {}\".format(optimized_save_dir))\n",
    "logger.info(\"Quantized INT8 model metric_results: {}\".format(metric_results_i8))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "998f20b3",
   "metadata": {},
   "source": [
    "Compare Average Precision of quantized INT8 model with FP32 original model. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "97feb8b6",
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "plt.style.use('seaborn-deep')\n",
    "fp32_acc = np.array(list(metric_results_fp32.values()))\n",
    "int8_acc = np.array(list(metric_results_i8.values()))\n",
    "x_data = (\"AP@0.5\",\"AP@0.5:0.95\")\n",
    "x_axis = np.arange(len(x_data))\n",
    "fig = plt.figure()\n",
    "fig.patch.set_facecolor('#FFFFFF')\n",
    "fig.patch.set_alpha(0.7)\n",
    "ax = fig.add_subplot(111)\n",
    "plt.bar(x_axis - 0.2, fp32_acc, 0.3, label='FP32')\n",
    "for i in range(0, len(x_axis)):\n",
    "    plt.text(i - 0.3, round(fp32_acc[i],3) + 0.01, str(round(fp32_acc[i],3)),fontweight=\"bold\")\n",
    "plt.bar(x_axis + 0.2, int8_acc, 0.3, label='INT8')\n",
    "for i in range(0, len(x_axis)):\n",
    "    plt.text(i + 0.1, round(int8_acc[i],3) + 0.01, str(round(int8_acc[i],3)),fontweight=\"bold\")\n",
    "plt.xticks(x_axis, x_data)\n",
    "plt.xlabel(\"Average Precision\")\n",
    "plt.title(\"Compare Yolov5 FP32 and INT8 model average precision\")\n",
    "\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b8de873",
   "metadata": {
    "id": "w3UUduQEGsQm"
   },
   "source": [
    "## Inference Demo Performance Comparison\n",
    "\n",
    "This part shows how to use the Ultralytics model detection code [\"detect.py\"](https://github.com/ultralytics/yolov5/blob/master/detect.py) to run synchronous inference using the OpenVINO Python API on 2 images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "663ed4f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "command_detect = f'{\"cd yolov5 && python detect.py --weights ./yolov5m/yolov5m_openvino_model/optimized/yolov5m.xml \"}'\n",
    "display(Markdown(f\"`{command_detect}`\"))\n",
    "%sx $command_detect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c757c451",
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.image as mpimg\n",
    "fig2 = plt.figure(figsize=(12, 9))\n",
    "fig2.patch.set_facecolor('#FFFFFF')\n",
    "fig2.patch.set_alpha(0.7)\n",
    "axarr1 = fig2.add_subplot(121)\n",
    "axarr2 = fig2.add_subplot(122)\n",
    "ori = mpimg.imread('./yolov5/data/images/bus.jpg')\n",
    "result = mpimg.imread('./yolov5/runs/detect/exp/bus.jpg')\n",
    "_ = axarr1.imshow(ori)\n",
    "_ = axarr2.imshow(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8ed2610",
   "metadata": {},
   "source": [
    "## References\n",
    "\n",
    "* [Ultralytics YOLOv5](https://github.com/ultralytics/yolov5)\n",
    "* [OpenVINO Post-training Optimization Tool](https://docs.openvino.ai/latest/pot_introduction.html)\n",
    "* [Open Model Zoo](https://github.com/openvinotoolkit/open_model_zoo)\n",
    "* [Accuracy Checker](https://github.com/openvinotoolkit/open_model_zoo/blob/master/tools/accuracy_checker/README.md)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  },
  "vscode": {
   "interpreter": {
    "hash": "847ab657bef8b70b4b42969ea9d57d9d306292cb6d75a7a5e12e767d03f52544"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}