notebooks/0_prepare_dataset.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# Prepare Dataset\n",
    "\n",
    "*[Florian Roscheck](https://www.linkedin.com/in/florianroscheck/), 2024-03-30*\n",
    "\n",
    "In this notebook, we prepare the TACO waste dataset for using it with machine learning on Azure ML. We will download the data and make it available as data asset on Azure ML. To use the labels with Azure Machine Learning, we have to transform them into a different format. Since, in this project, we do not want to leverage the full taxonomy of the TACO-supplied labels, but a more general taxonomy, we will process the labels. Finally, we will create an MLTable with the data that Azure ML can ingest."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Download Dataset\n",
    "\n",
    "In this section, we download the dataset. To do this, we will clone the [TACO dataset GitHub repository](https://github.com/pedropro/TACO) and then use the provided download script to receive images and annotations. Since the downloaded data directory structure does not match our best practices, we will move the files to a fitting location in the `data/raw` directory."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Create directory for cloning repo into\n",
    "\n",
    "!mkdir -p ../src/data/TACO"
   ],
   "outputs": [],
   "execution_count": 1,
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "gather": {
     "logged": 1711957599025
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Clone repo\n",
    "\n",
    "!git clone https://github.com/pedropro/TACO ../src/data/TACO"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "Cloning into '../src/data/TACO'...\nremote: Enumerating objects: 740, done.\u001B[K\nremote: Counting objects: 100% (160/160), done.\u001B[K\nremote: Compressing objects: 100% (72/72), done.\u001B[K\nremote: Total 740 (delta 117), reused 128 (delta 88), pack-reused 580\u001B[K\nReceiving objects: 100% (740/740), 98.70 MiB | 22.80 MiB/s, done.\nResolving deltas: 100% (494/494), done.\nUpdating files: 100% (25/25), done.\n"
    }
   ],
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "gather": {
     "logged": 1711957614197
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Run data download script\n",
    "\n",
    "!python ../src/data/TACO/download.py --dataset_path ../src/data/TACO/data/annotations.json"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "Note. If for any reason the connection is broken. Just call me again and I will start where I left.\nDownloading: [..............................] - 1500/1500\n"
    }
   ],
   "execution_count": 1,
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "gather": {
     "logged": 1711957621620
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Create directory to move downloaded data into\n",
    "\n",
    "!mkdir -p ../data/raw/TACO/images"
   ],
   "outputs": [],
   "execution_count": 1,
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "gather": {
     "logged": 1711957625288
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Move downloaded data into data/raw directory\n",
    "\n",
    "from pathlib import Path\n",
    "\n",
    "for dir in Path('../src/data/TACO/data/').glob('*'):\n",
    "    for file in dir.glob('*'):\n",
    "        if file.name.startswith('.aml'):\n",
    "            # When manually exploring data through the Azure ML\n",
    "            # frontend, Azure ML creates temporary files that \n",
    "            # we don't want in our image dataset. Here, we remove \n",
    "            # these files before moving the directory.\n",
    "            file.unlink()\n",
    "    if dir.is_dir():\n",
    "        dir.rename(Path('../data/raw/TACO/images/').joinpath(dir.name))\n",
    "\n",
    "Path('../src/data/TACO/data/annotations.json').rename('../data/raw/TACO/annotations.json')"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "execution_count": 1,
     "data": {
      "text/plain": "PosixPath('../data/raw/TACO/annotations.json')"
     },
     "metadata": {}
    }
   ],
   "execution_count": 1,
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "gather": {
     "logged": 1711957628389
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Upload and Register the Dataset\n",
    "\n",
    "In this section, we upload and register the dataset so we can use it in machine learning through Azure ML. We will upload the dataset to the blob storage attached to the Azure ML workspace. We register the dataset so that it becomes available as data asset and we can use it in machine learning workflows."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "# Make a connection to the Azure ML workspace and\n",
    "# its default blob storage so we can interact with it.\n",
    "\n",
    "from azureml.core import Workspace\n",
    "\n",
    "ws = Workspace.from_config()\n",
    "datastore = ws.get_default_datastore()\n",
    "\n",
    "print('Data will be uploaded here:')\n",
    "print(datastore)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "Data will be uploaded here:\n{\n  \"name\": \"***\",\n  \"container_name\": \"***\",\n  \"account_name\": \"***\",\n  \"protocol\": \"https\",\n  \"endpoint\": \"core.windows.net\"\n}\n"
    }
   ],
   "execution_count": 1,
   "metadata": {
    "gather": {
     "logged": 1711975526656
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Now that we have a connection to the blob storage, we can upload the data to it. "
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "# Upload the data to the blob storage\n",
    "\n",
    "from azureml.core import Dataset\n",
    "\n",
    "# Here is the subdirectory which the data will be uploaded to \n",
    "# on the blob storage\n",
    "blob_storage_path = 'data/raw/TACO'\n",
    "\n",
    "uploaded_data = Dataset.File.upload_directory(\n",
    "    src_dir='../data/raw/TACO/',\n",
    "    target=(datastore, blob_storage_path),\n",
    "    overwrite=True\n",
    "    )"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "Validating arguments.\nArguments validated.\n'overwrite' is set to True. Any file already present in the target will be overwritten.\nUploading files from '/mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO' to 'data/raw/TACO'\nCopying 66 files with concurrency set to 2\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/.amlignore, file 1 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/.amlignore\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000001.jpg, file 2 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000001.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/annotations.json, file 3 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/annotations.json\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/.amlignore.amltmp, file 4 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/.amlignore.amltmp\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000005.jpg, file 5 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000005.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000007.jpg, file 6 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000007.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000003.jpg, file 7 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000003.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000006.jpg, file 8 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000006.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000010.jpg, file 9 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000010.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000012.jpg, file 10 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000012.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000008.jpg, file 11 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000008.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000019.jpg, file 12 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000019.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000011.jpg, file 13 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000011.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000014.jpg, file 14 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000014.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000022.jpg, file 15 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000022.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000024.jpg, file 16 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000024.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000021.jpg, file 17 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000021.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000027.jpg, file 18 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000027.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000023.jpg, file 19 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000023.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000030.jpg, file 20 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000030.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000026.jpg, file 21 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000026.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000028.jpg, file 22 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000028.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000040.jpg, file 23 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000040.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000047.jpg, file 24 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000047.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000032.jpg, file 25 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000032.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000049.jpg, file 26 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000049.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000043.jpg, file 27 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000043.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000048.jpg, file 28 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000048.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000054.jpg, file 29 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000054.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000056.jpg, file 30 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000056.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000060.jpg, file 31 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000060.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000053.jpg, file 32 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000053.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000055.jpg, file 33 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000055.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000058.jpg, file 34 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000058.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000001.jpg, file 35 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000001.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/batch_1/000061.jpg, file 36 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/batch_1/000061.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000005.jpg, file 37 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000005.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000003.jpg, file 38 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000003.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000007.jpg, file 39 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000007.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000006.jpg, file 40 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000006.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000010.jpg, file 41 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000010.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000008.jpg, file 42 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000008.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000011.jpg, file 43 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000011.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000012.jpg, file 44 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000012.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000014.jpg, file 45 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000014.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000019.jpg, file 46 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000019.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000022.jpg, file 47 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000022.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000024.jpg, file 48 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000024.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000021.jpg, file 49 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000021.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000027.jpg, file 50 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000027.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000032.jpg, file 51 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000032.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000023.jpg, file 52 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000023.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000043.jpg, file 53 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000043.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000048.jpg, file 54 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000048.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000026.jpg, file 55 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000026.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000053.jpg, file 56 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000053.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000028.jpg, file 57 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000028.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000040.jpg, file 58 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000040.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000047.jpg, file 59 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000047.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000049.jpg, file 60 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000049.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000054.jpg, file 61 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000054.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000056.jpg, file 62 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000056.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000060.jpg, file 63 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000060.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000055.jpg, file 64 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000055.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000058.jpg, file 65 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000058.jpg\nCopied /mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks/../data/raw/TACO/images/batch_1/000061.jpg, file 66 out of 66. Destination path: https://***.blob.core.windows.net/***/data/raw/TACO/images/batch_1/000061.jpg\nFiles copied=66, skipped=0, failed=0\nCreating new dataset\n"
    }
   ],
   "execution_count": 1,
   "metadata": {
    "gather": {
     "logged": 1711957674126
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Through the `uploaded_data` object we now have a `azureml.data.file_dataset.FileDataset` to work with. We can use its convenience method `register` to register it as a data asset on Azure ML."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "# Register data asset in workspace\n",
    "# Note that the description field is a great way to inform your team\n",
    "# members of important information about the dataset, enabling them\n",
    "# to use it.\n",
    "\n",
    "uploaded_data.register(\n",
    "    workspace=ws,\n",
    "    name='TACO',\n",
    "    description=(\n",
    "        \"Image dataset of trash pictures, from here: http://tacodataset.org/ \",\n",
    "        \"(open-source licenses, see https://github.com/pedropro/TACO/blob/master/data/annotations.json \",\n",
    "        \" and https://openlittermap.com/about)\")\n",
    ")"
   ],
   "outputs": [],
   "execution_count": 1,
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "gather": {
     "logged": 1711957680073
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "That's it! We have now uploaded and registered the dataset on Azure ML. This is the first step in using the data in our experiments."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Transform Annotations\n",
    "\n",
    "In order to use the annotations of the images in machine learning with AzureML, we have to convert them into the JSONL format. This is what we are going to do in this section.\n",
    "\n",
    "The labels in the TACO dataset are provided in [COCO Format](http://cocodataset.org/#format-data). Unfortunately, Azure ML expects data to be in [JSONL](https://learn.microsoft.com/en-us/azure/machine-learning/reference-automl-images-schema?view=azureml-api-2#instance-segmentation) format for use with Azure automated machine learning. To not lose time here, we are going to leverage code written by [Microsoft](https://github.com/Azure/azureml-examples/tree/36296068c9292a37323a83bc6d1ae23a2c2bc87f/sdk/python/jobs/automl-standalone-jobs/jsonl-conversion) which implements this conversion from COCO to JSONL. Let's download this code and use it for transforming the labels!"
   ],
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Create directory for JSONL conversion code\n",
    "\n",
    "!mkdir -p ../src/features/jsonl_conversion"
   ],
   "outputs": [],
   "execution_count": null,
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Download conversion code\n",
    "\n",
    "!wget https://raw.githubusercontent.com/Azure/azureml-examples/36296068c9292a37323a83bc6d1ae23a2c2bc87f/sdk/python/jobs/automl-standalone-jobs/jsonl-conversion/base_jsonl_converter.py -P ../src/features/jsonl_conversion/\n",
    "!wget https://raw.githubusercontent.com/Azure/azureml-examples/36296068c9292a37323a83bc6d1ae23a2c2bc87f/sdk/python/jobs/automl-standalone-jobs/jsonl-conversion/coco_jsonl_converter.py -P ../src/features/jsonl_conversion/\n",
    "!wget https://raw.githubusercontent.com/Azure/azureml-examples/36296068c9292a37323a83bc6d1ae23a2c2bc87f/sdk/python/jobs/automl-standalone-jobs/jsonl-conversion/masktools.py -P ../src/features/jsonl_conversion/"
   ],
   "outputs": [],
   "execution_count": null,
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "To execute the newly downloaded code, we need to install some dependencies. Let's do that now!"
   ],
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "%pip install pycocotools simplification"
   ],
   "outputs": [],
   "execution_count": null,
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Now we are ready to convert the TACO COCO file to JSONL. Let's import the newly downloaded code."
   ],
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Import downloaded code\n",
    "\n",
    "import sys\n",
    "\n",
    "sys.path.append('../src/features/jsonl_conversion')\n",
    "\n",
    "from base_jsonl_converter import write_json_lines\n",
    "from coco_jsonl_converter import COCOJSONLConverter"
   ],
   "outputs": [],
   "execution_count": 1,
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "gather": {
     "logged": 1711975516169
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Although we try to reuse as much as possible, there is one conflict between the TACO data and the `COCOJSONLConverter`: In its method `_populate_image_url`, it removes all directories from the image path:\n",
    "\n",
    "```python\n",
    "\n",
    "def _populate_image_url(self, index, coco_image):\n",
    "    \"\"\"\n",
    "    populates image url for jsonl entry\n",
    "\n",
    "    Parameters:\n",
    "        index (int): image entry index\n",
    "        coco_image (dict): image entry from coco data file\n",
    "    \"\"\"\n",
    "    image_url = coco_image[\"file_name\"]\n",
    "    self.jsonl_data[index][\"image_url\"] = (\n",
    "        self.base_url + image_url[image_url.rfind(\"/\") + 1 :] # <-- removal of directories\n",
    "    )\n",
    "    self.image_id_to_data_index[coco_image[\"id\"]] = index\n",
    "\n",
    "```\n",
    "\n",
    "Since the images are stored in subdirectories in the TACO dataset, and the image paths in `annotations.json` file point to these subdirectories, we need to modify the method to keep the subdirectories. Let's do this quickly:"
   ],
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Create a new class with a patched _populate_image_url method\n",
    "# that keeps the subdirectories.\n",
    "\n",
    "class ModifiedCOCOJSONLConverter(COCOJSONLConverter):\n",
    "    def _populate_image_url(self, index, coco_image):\n",
    "        \"\"\"\n",
    "        populates image url for jsonl entry\n",
    "\n",
    "        Parameters:\n",
    "            index (int): image entry index\n",
    "            coco_image (dict): image entry from coco data file\n",
    "        \"\"\"\n",
    "        image_url = coco_image[\"file_name\"]\n",
    "        self.jsonl_data[index][\"image_url\"] = (\n",
    "            self.base_url + image_url\n",
    "        )\n",
    "        self.image_id_to_data_index[coco_image[\"id\"]] = index"
   ],
   "outputs": [],
   "execution_count": 1,
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "gather": {
     "logged": 1711977640296
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Now, to have any existing directory reference in the TACO COCO annotations point to the right location, we will have to inform the converter of the location of the TACO images on our blob storage. To get the correct directory here, we will leverage the Azure ML Python library and get a reference to the data asset we created earlier."
   ],
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Connect Azure ML client to Azure ML workspace\n",
    "\n",
    "from azure.ai.ml import MLClient\n",
    "from azure.identity import DefaultAzureCredential\n",
    "\n",
    "ml_client = MLClient(\n",
    "    DefaultAzureCredential(), \n",
    "    ws.subscription_id, \n",
    "    ws.resource_group, \n",
    "    ws.name\n",
    ")\n",
    "data_asset = ml_client.data.get(\"TACO\", version=\"1\")\n",
    "\n",
    "print(data_asset.path)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "azureml://subscriptions/***/resourcegroups/***/workspaces/***/datastores/***/paths/data/raw/TACO/\n"
    }
   ],
   "execution_count": 1,
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "gather": {
     "logged": 1711978034956
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Now that we have the libraries imported, patched, and the new path to the images set, we can use the `COCOJSONLConverter` to convert the TACO-supplied `annotations.json` file to a JSONL file we can use with Azure ML."
   ],
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Convert TACO-supplied annotations.json file to JSONL\n",
    "\n",
    "from pathlib import Path\n",
    "\n",
    "converter = ModifiedCOCOJSONLConverter(\n",
    "    base_url=data_asset.path, \n",
    "    coco_file='../data/raw/TACO/annotations.json'\n",
    ")\n",
    "\n",
    "target_file = Path('../data/processed/TACO/annotations.jsonl')\n",
    "target_file.parent.mkdir(parents=True, exist_ok=True)\n",
    "\n",
    "write_json_lines(converter, target_file)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "Conversion completed. Converted 1500 lines.\n"
    }
   ],
   "execution_count": 1,
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "gather": {
     "logged": 1711978047271
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Process Labels\n",
    "\n",
    "The TACO dataset offers a [rich taxonomy](http://tacodataset.org/taxonomy). But for sorting trash in a modern society, usually just a couple of categories are needed. It is mostly important into which trash can a specific item goes, not what the item specifically is. For example, a cardbox and a paper wrapper both go into the paper recycling trash can.\n",
    "\n",
    "> Please note that the simplification presented here is a simplified and opinionated categorization based on experience living in Germany and being subject to the German recycling system – it is not an expert-informed assessment. Some important categories like battery or non-glass bottle recycling are not considered. If you want to implement trash sorting for the \"real world\", it is advisable you understand the trash processing and recycling system in the environment you are aiming to use the system in. The purpose of the categorization presented here is to show how labels of existing datasets can be manipulated before training with Azure ML, not to build a highly reliable trash categorization system.\n",
    "\n",
    "Here are the categories (trash cans) we want to sort trash into:\n",
    "- Blue: Paper recycling\n",
    "- Yellow: Plastics, metal, composite material recycling\n",
    "- Glass: Glass recycling\n",
    "- Other: Anything else\n",
    "\n",
    "Let's first have a glance at the existing labels and then develop a mapping mechanism for simplifying them so they end up as the categories listed above."
   ],
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Read in JSONL file as Pandas DataFrame\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "df = pd.read_json(target_file, lines=True)\n",
    "df.head()"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "execution_count": 1,
     "data": {
      "text/plain": "                                           image_url  \\\n0  azureml://subscriptions/***...   \n1  azureml://subscriptions/***...   \n2  azureml://subscriptions/***...   \n3  azureml://subscriptions/***...   \n4  azureml://subscriptions/***...   \n\n                                      image_details  \\\n0  {'format': 'jpg', 'width': 1537, 'height': 2049}   \n1  {'format': 'jpg', 'width': 1537, 'height': 2049}   \n2  {'format': 'jpg', 'width': 1537, 'height': 2049}   \n3  {'format': 'jpg', 'width': 2049, 'height': 1537}   \n4  {'format': 'jpg', 'width': 1537, 'height': 2049}   \n\n                                               label  \n0  [{'label': 'Glass bottle', 'polygon': [[0.3649...  \n1  [{'label': 'Meal carton', 'polygon': [[0.60377...  \n2  [{'label': 'Clear plastic bottle', 'polygon': ...  \n3  [{'label': 'Clear plastic bottle', 'polygon': ...  \n4  [{'label': 'Drink can', 'polygon': [[0.6714378...  ",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>image_url</th>\n      <th>image_details</th>\n      <th>label</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>azureml://subscriptions/***...</td>\n      <td>{'format': 'jpg', 'width': 1537, 'height': 2049}</td>\n      <td>[{'label': 'Glass bottle', 'polygon': [[0.3649...</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>azureml://subscriptions/***...</td>\n      <td>{'format': 'jpg', 'width': 1537, 'height': 2049}</td>\n      <td>[{'label': 'Meal carton', 'polygon': [[0.60377...</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>azureml://subscriptions/***...</td>\n      <td>{'format': 'jpg', 'width': 1537, 'height': 2049}</td>\n      <td>[{'label': 'Clear plastic bottle', 'polygon': ...</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>azureml://subscriptions/***...</td>\n      <td>{'format': 'jpg', 'width': 2049, 'height': 1537}</td>\n      <td>[{'label': 'Clear plastic bottle', 'polygon': ...</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>azureml://subscriptions/***...</td>\n      <td>{'format': 'jpg', 'width': 1537, 'height': 2049}</td>\n      <td>[{'label': 'Drink can', 'polygon': [[0.6714378...</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "metadata": {}
    }
   ],
   "execution_count": 1,
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "gather": {
     "logged": 1711978051301
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Extract labels and sort by occurence\n",
    "\n",
    "labels_series = df['label'].explode().apply(lambda x: x['label'])\n",
    "labels_series.value_counts(normalize=True)"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "execution_count": 1,
     "data": {
      "text/plain": "label\nCigarette                    0.139423\nUnlabeled litter             0.108069\nPlastic film                 0.094273\nClear plastic bottle         0.059574\nOther plastic                0.057065\nOther plastic wrapper        0.054348\nDrink can                    0.047868\nPlastic bottle cap           0.043687\nPlastic straw                0.032818\nBroken glass                 0.028846\nStyrofoam piece              0.023411\nDisposable plastic cup       0.021739\nGlass bottle                 0.021739\nPop tab                      0.020694\nOther carton                 0.019440\nNormal paper                 0.017140\nMetal bottle cap             0.016722\nPlastic lid                  0.016095\nPaper cup                    0.014005\nCorrugated carton            0.013378\nAluminium foil               0.012960\nSingle-use carrier bag       0.012751\nOther plastic bottle         0.010452\nDrink carton                 0.009406\nTissues                      0.008779\nCrisp packet                 0.008152\nDisposable food container    0.007943\nPlastic utensils             0.007734\nFood Can                     0.007107\nGarbage bag                  0.006480\nMeal carton                  0.006271\nRope & strings               0.006062\nPaper bag                    0.005644\nScrap metal                  0.004181\nFoam food container          0.003135\nFoam cup                     0.002717\nMagazine paper               0.002508\nWrapping paper               0.002508\nEgg carton                   0.002299\nAerosol                      0.002090\nMetal lid                    0.002090\nSpread tub                   0.001881\nFood waste                   0.001672\nShoe                         0.001463\nSqueezable tube              0.001463\nAluminium blister pack       0.001254\nGlass cup                    0.001254\nGlass jar                    0.001254\nOther plastic container      0.001254\nToilet tube                  0.001045\nSix pack rings               0.001045\nPaper straw                  0.000836\nPlastic glooves              0.000836\nTupperware                   0.000836\nPolypropylene bag            0.000627\nPizza box                    0.000627\nBattery                      0.000418\nOther plastic cup            0.000418\nCarded blister pack          0.000209\nName: proportion, dtype: float64"
     },
     "metadata": {}
    }
   ],
   "execution_count": 1,
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "gather": {
     "logged": 1711978052313
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Based on the unique labels, we can now develop a mapping function. To facilitate, we first transform all labels to lowercase. Then, we go through the list above and manually extract as many key terms as necessary to build a mapping function that encompasses all existing labels."
   ],
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Build mapping function\n",
    "\n",
    "# Transform all labels to lowercase\n",
    "labels = set([label.lower() for label in labels_series.values.tolist()])\n",
    "\n",
    "# Collect key terms\n",
    "# (To belong to a specific category, the term needs to appear in the existing\n",
    "# label in full.)\n",
    "key_terms = {'yellow': ('plastic', 'six pack rings', 'carrier bag', 'can', 'aluminum', 'aluminium', \n",
    "                        'metal', 'foam', 'polypropylene', 'tupperware', 'aerosol', 'garbage bag',\n",
    "                        'pop tab'), \n",
    "            'blue': ('paper', 'carton', 'toilet tube'), \n",
    "            'glass': ('glass',)}\n",
    "\n",
    "# Produce mapping\n",
    "bins = {}\n",
    "for keyword, items in key_terms.items():\n",
    "    if keyword not in bins:\n",
    "        bins[keyword] = []\n",
    "    for item in items:\n",
    "        bins[keyword].extend([label for label in labels if item in label])\n",
    "    labels = labels - set(bins[keyword])\n",
    "bins['other'] = list(labels)"
   ],
   "outputs": [],
   "execution_count": 1,
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "gather": {
     "logged": 1711978053493
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Let's sanity check the mapping. We want to see a list of the original item category and, next to it, the newly assigned category."
   ],
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "bins_inverse = {}\n",
    "for keyword, items in bins.items():\n",
    "    for item in items:\n",
    "        bins_inverse[item] = keyword\n",
    "\n",
    "bins_inverse"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "execution_count": 1,
     "data": {
      "text/plain": "{'plastic straw': 'yellow',\n 'disposable plastic cup': 'yellow',\n 'other plastic cup': 'yellow',\n 'plastic film': 'yellow',\n 'plastic utensils': 'yellow',\n 'plastic lid': 'yellow',\n 'other plastic wrapper': 'yellow',\n 'plastic glooves': 'yellow',\n 'other plastic bottle': 'yellow',\n 'plastic bottle cap': 'yellow',\n 'clear plastic bottle': 'yellow',\n 'other plastic': 'yellow',\n 'other plastic container': 'yellow',\n 'six pack rings': 'yellow',\n 'single-use carrier bag': 'yellow',\n 'drink can': 'yellow',\n 'food can': 'yellow',\n 'aluminium foil': 'yellow',\n 'aluminium blister pack': 'yellow',\n 'metal bottle cap': 'yellow',\n 'metal lid': 'yellow',\n 'scrap metal': 'yellow',\n 'styrofoam piece': 'yellow',\n 'foam cup': 'yellow',\n 'foam food container': 'yellow',\n 'polypropylene bag': 'yellow',\n 'tupperware': 'yellow',\n 'aerosol': 'yellow',\n 'garbage bag': 'yellow',\n 'pop tab': 'yellow',\n 'paper straw': 'blue',\n 'paper cup': 'blue',\n 'magazine paper': 'blue',\n 'normal paper': 'blue',\n 'paper bag': 'blue',\n 'wrapping paper': 'blue',\n 'drink carton': 'blue',\n 'egg carton': 'blue',\n 'meal carton': 'blue',\n 'other carton': 'blue',\n 'corrugated carton': 'blue',\n 'toilet tube': 'blue',\n 'broken glass': 'glass',\n 'glass jar': 'glass',\n 'glass bottle': 'glass',\n 'glass cup': 'glass',\n 'crisp packet': 'other',\n 'battery': 'other',\n 'rope & strings': 'other',\n 'cigarette': 'other',\n 'squeezable tube': 'other',\n 'unlabeled litter': 'other',\n 'pizza box': 'other',\n 'food waste': 'other',\n 'carded blister pack': 'other',\n 'shoe': 'other',\n 'disposable food container': 'other',\n 'tissues': 'other',\n 'spread tub': 'other'}"
     },
     "metadata": {}
    }
   ],
   "execution_count": 1,
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "gather": {
     "logged": 1711978054524
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "This mapping generally looks good, but for some items we might need a more differentiated category. For example, for a pizza box, we might risk discarding it into the wrong trash can, other, if it is made out of paper and overall clean – in which case it should go into the blue can. But for the sake of this exercise, the quality of the mapping it sufficient.\n",
    "\n",
    "Let's apply it to the annotations!"
   ],
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Replace labels in JSONL DataFrame\n",
    "\n",
    "def replace_label(row):\n",
    "    for entry in row:\n",
    "        entry['label'] = bins_inverse[entry['label'].lower()]\n",
    "    return row\n",
    "\n",
    "df['label'].apply(replace_label)"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "execution_count": 1,
     "data": {
      "text/plain": "0       [{'label': 'glass', 'polygon': [[0.36499674690...\n1       [{'label': 'blue', 'polygon': [[0.603773584905...\n2       [{'label': 'yellow', 'polygon': [[0.4359141184...\n3       [{'label': 'yellow', 'polygon': [[0.1727672035...\n4       [{'label': 'yellow', 'polygon': [[0.6714378659...\n                              ...                        \n1495    [{'label': 'blue', 'polygon': [[0.545504385964...\n1496    [{'label': 'yellow', 'polygon': [[0.5526315789...\n1497    [{'label': 'glass', 'polygon': [[0.10350000000...\n1498    [{'label': 'blue', 'polygon': [[0.211622807017...\n1499    [{'label': 'yellow', 'polygon': [[0.75, 0.4437...\nName: label, Length: 1500, dtype: object"
     },
     "metadata": {}
    }
   ],
   "execution_count": 1,
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "gather": {
     "logged": 1711978056451
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Great! Now, we should write the modified data into JSONL format so we can use it with Azure ML. We are going to leverage Pandas' `DataFrame.to_json()` functionality. There is just one gotcha: The `to_json()` command will escape `/` characters like `\\/` – but this affects the file paths stored in the JSONL file. So, this is why we write the JSONL file to a string buffer and manipulate the buffer to undo the escaping before we finally write it to the labels file."
   ],
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Write new mapping to JSONL file\n",
    "\n",
    "from io import StringIO\n",
    "\n",
    "buffer = StringIO()\n",
    "df.to_json(buffer, lines=True, orient='records')\n",
    "\n",
    "unescaped_buffer = buffer.getvalue().replace('\\/','/')\n",
    "\n",
    "modified_annotations_file = Path('../data/processed/TACO/annotations_modified.jsonl')\n",
    "\n",
    "with open(modified_annotations_file, 'w') as file:\n",
    "    file.write(unescaped_buffer)"
   ],
   "outputs": [],
   "execution_count": 1,
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "gather": {
     "logged": 1711978060230
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Perfect! We have modified the annotations and are now ready for the final step of data preparation."
   ],
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Create MLTable\n",
    "\n",
    "Azure ML uses so-called \"Azure Machine Learning tables\" (find out more on the [Azure ML website](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-mltable?view=azureml-api-2&tabs=cli) to facilitate working with data. These tables are like blueprints for loading and modifying data. I personally find them especially useful for computer vision tasks, as their native support on Azure ML helps leverage Azure ML's features to the fullest. Let's create an Azure ML table for the newly created annotations file."
   ],
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Create directory for MLTable and move annotations file\n",
    "# into this directory\n",
    "\n",
    "mltable_path = Path('../data/processed/TACO_labels')\n",
    "mltable_path.mkdir(parents=True, exist_ok=True)\n",
    "\n",
    "Path(modified_annotations_file).rename(mltable_path.joinpath(f'./{modified_annotations_file.name}'))\n"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "execution_count": 1,
     "data": {
      "text/plain": "PosixPath('../data/processed/TACO_labels/annotations_modified.jsonl')"
     },
     "metadata": {}
    }
   ],
   "execution_count": 1,
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "gather": {
     "logged": 1711978119822
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Create MLTable file\n",
    "\n",
    "content = f\"\"\"paths:\n",
    "  - file: ./{modified_annotations_file.name}\n",
    "transformations:\n",
    "  - read_json_lines:\n",
    "        encoding: utf8\n",
    "        invalid_lines: error\n",
    "        include_path_column: false\n",
    "  - convert_column_types:\n",
    "      - columns: image_url\n",
    "        column_type: stream_info\n",
    "\"\"\"\n",
    "\n",
    "with open(str(mltable_path) + '/MLTable', 'w') as file:\n",
    "    file.write(content)"
   ],
   "outputs": [],
   "execution_count": 1,
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "gather": {
     "logged": 1711978129772
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Now we can register the file as a data asset on Azure ML so we can use it to train a model with the added benefit that any training job on Azure ML refers to this data asset. This enables model inputs to be reproducible."
   ],
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "from azure.ai.ml.entities import Data\n",
    "from azure.ai.ml.constants import AssetTypes\n",
    "\n",
    "# Define the Data asset object\n",
    "dataset = Data(\n",
    "    path=mltable_path,\n",
    "    type=AssetTypes.MLTABLE,\n",
    "    description=(\"Annotations for TACO dataset, with yellow, blue, \",\n",
    "                 \"glass, and other classes of trash\"),\n",
    "    name=\"TACO-annotations\",\n",
    "    version=\"1\",\n",
    ")\n",
    "\n",
    "# Create the data asset in the workspace\n",
    "ml_client.data.create_or_update(dataset)"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "execution_count": 1,
     "data": {
      "text/plain": "Data({'skip_validation': False, 'mltable_schema_url': None, 'referenced_uris': ['./annotations_modified.jsonl'], 'type': 'mltable', 'is_anonymous': False, 'auto_increment_version': False, 'auto_delete_setting': None, 'name': 'TACO-annotations', 'description': \"('Annotations for TACO dataset, with yellow, blue, ', 'glass, and other classes of trash')\", 'tags': {}, 'properties': {}, 'print_as_yaml': True, 'id': '/subscriptions/***/resourceGroups/***/providers/Microsoft.MachineLearningServices/workspaces/***/data/TACO-annotations/versions/1', 'Resource__source_path': None, 'base_path': '/mnt/batch/tasks/shared/LS_root/mounts/clusters/florian-inference-test/code/Users/florian.roscheck/trash_recognizer/notebooks', 'creation_context': <azure.ai.ml.entities._system_data.SystemData object at 0x7fb2f7514d00>, 'serialize': <msrest.serialization.Serializer object at 0x7fb2f75178b0>, 'version': '1', 'latest_version': None, 'path': 'azureml://subscriptions/***/resourcegroups/***/workspaces/***/datastores/workspaceblobstore/paths/LocalUpload/7e2f2d8876365ded8ad08302ce861358/TACO_labels/', 'datastore': None})"
     },
     "metadata": {}
    }
   ],
   "execution_count": 1,
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "gather": {
     "logged": 1711978425574
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "If you now explore the dataset in the UI of Azure ML, you will notice that the preview window shows images with labels included!"
   ],
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   }
  }
 ],
 "metadata": {
  "kernel_info": {
   "name": "python310-sdkv2"
  },
  "kernelspec": {
   "name": "python310-sdkv2",
   "language": "python",
   "display_name": "Python 3.10 - SDK v2"
  },
  "language_info": {
   "name": "python",
   "version": "3.10.11",
   "mimetype": "text/x-python",
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "pygments_lexer": "ipython3",
   "nbconvert_exporter": "python",
   "file_extension": ".py"
  },
  "microsoft": {
   "host": {
    "AzureML": {
     "notebookHasBeenCompleted": true
    }
   },
   "ms_spell_check": {
    "ms_spell_check_language": "en"
   }
  },
  "nteract": {
   "version": "nteract-front-end@1.0.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}