text-classification-with-transformers-and-comet.ipynb

{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"![](https://storage.googleapis.com/kaggle-datasets-images/955509/1617920/e3577a34996d1c3eec200d2ee8b74bec/dataset-cover.jpg?t=2020-11-04-17-03-44)\n\nThis notebook walks you through how to perform text classification with Transformers, how to monitor hyperparameters with Comet ML, and how to deploy with Gradio. ","metadata":{}},{"cell_type":"markdown","source":"# <b><div style='padding:15px;background-color:#850E35;color:white;border-radius:2px;font-size:110%;text-align: center'>1. Install Required Libraries</div></b>","metadata":{}},{"cell_type":"code","source":"!pip install -q comet_ml gradio","metadata":{"execution":{"iopub.status.busy":"2023-09-03T08:39:30.367937Z","iopub.execute_input":"2023-09-03T08:39:30.368261Z","iopub.status.idle":"2023-09-03T08:39:54.347200Z","shell.execute_reply.started":"2023-09-03T08:39:30.368233Z","shell.execute_reply":"2023-09-03T08:39:54.345833Z"},"trusted":true},"execution_count":1,"outputs":[{"name":"stdout","text":"\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\njupyterlab-lsp 4.2.0 requires jupyter-lsp>=2.0.0, but you have jupyter-lsp 1.5.1 which is incompatible.\u001b[0m\u001b[31m\n\u001b[0m","output_type":"stream"}]},{"cell_type":"markdown","source":"# <b><div style='padding:15px;background-color:#850E35;color:white;border-radius:2px;font-size:110%;text-align: center'>2. Initialize Comet and HuggingFace</div></b>","metadata":{}},{"cell_type":"code","source":"import comet_ml\n\ncomet_ml.init(project_name=\"text-classification-with-transfomers\")","metadata":{"execution":{"iopub.status.busy":"2023-09-03T08:39:54.350007Z","iopub.execute_input":"2023-09-03T08:39:54.350408Z","iopub.status.idle":"2023-09-03T08:40:26.024925Z","shell.execute_reply.started":"2023-09-03T08:39:54.350371Z","shell.execute_reply":"2023-09-03T08:40:26.023961Z"},"trusted":true},"execution_count":2,"outputs":[{"name":"stdout","text":"Please paste your Comet API key from https://www.comet.com/api/my/settings/\n(api key may not show as you type)\n","output_type":"stream"},{"output_type":"stream","name":"stdin","text":"Comet API key:  ·························\n"},{"name":"stderr","text":"\u001b[1;38;5;39mCOMET INFO:\u001b[0m Valid Comet API Key saved in /root/.comet.config (set COMET_CONFIG to change where it is saved).\n","output_type":"stream"}]},{"cell_type":"code","source":"from huggingface_hub import notebook_login\n\nnotebook_login()","metadata":{"execution":{"iopub.status.busy":"2023-09-03T08:40:30.018013Z","iopub.execute_input":"2023-09-03T08:40:30.019076Z","iopub.status.idle":"2023-09-03T08:40:30.268749Z","shell.execute_reply.started":"2023-09-03T08:40:30.019030Z","shell.execute_reply":"2023-09-03T08:40:30.267840Z"},"trusted":true},"execution_count":3,"outputs":[{"output_type":"display_data","data":{"text/plain":"VBox(children=(HTML(value='<center> <img\\nsrc=https://huggingface.co/front/assets/huggingface_logo-noborder.sv…","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"021b17bca6b54009bffa67dc8deaac19"}},"metadata":{}}]},{"cell_type":"markdown","source":"# <b><div style='padding:15px;background-color:#850E35;color:white;border-radius:2px;font-size:110%;text-align: center'>3. Load Data</div></b>","metadata":{}},{"cell_type":"code","source":"from datasets import load_dataset\n\nraw_datasets = load_dataset(\"rotten_tomatoes\")","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-09-03T08:40:45.929162Z","iopub.execute_input":"2023-09-03T08:40:45.929642Z","iopub.status.idle":"2023-09-03T08:40:50.177714Z","shell.execute_reply.started":"2023-09-03T08:40:45.929605Z","shell.execute_reply":"2023-09-03T08:40:50.176617Z"},"trusted":true},"execution_count":4,"outputs":[{"output_type":"display_data","data":{"text/plain":"Downloading builder script:   0%|          | 0.00/1.89k [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"ea3188601f2b475ebccd95e0184eb507"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Downloading metadata:   0%|          | 0.00/921 [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"1c40a41e885442a29b90c4e95b77d900"}},"metadata":{}},{"name":"stdout","text":"Downloading and preparing dataset rotten_tomatoes_movie_review/default (download: 476.34 KiB, generated: 1.28 MiB, post-processed: Unknown size, total: 1.75 MiB) to /root/.cache/huggingface/datasets/rotten_tomatoes_movie_review/default/1.0.0/40d411e45a6ce3484deed7cc15b82a53dad9a72aafd9f86f8f227134bec5ca46...\n","output_type":"stream"},{"output_type":"display_data","data":{"text/plain":"Downloading data:   0%|          | 0.00/488k [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"2581965a2ca04a65b44ac14843647eec"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Generating train split:   0%|          | 0/8530 [00:00<?, ? examples/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":""}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Generating validation split:   0%|          | 0/1066 [00:00<?, ? examples/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":""}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Generating test split:   0%|          | 0/1066 [00:00<?, ? examples/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":""}},"metadata":{}},{"name":"stdout","text":"Dataset rotten_tomatoes_movie_review downloaded and prepared to /root/.cache/huggingface/datasets/rotten_tomatoes_movie_review/default/1.0.0/40d411e45a6ce3484deed7cc15b82a53dad9a72aafd9f86f8f227134bec5ca46. Subsequent calls will reuse this data.\n","output_type":"stream"},{"output_type":"display_data","data":{"text/plain":"  0%|          | 0/3 [00:00<?, ?it/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"a2abb421a0584dcb8d9d2f67386147dc"}},"metadata":{}}]},{"cell_type":"code","source":"raw_datasets","metadata":{"execution":{"iopub.status.busy":"2023-09-03T08:41:09.019683Z","iopub.execute_input":"2023-09-03T08:41:09.020566Z","iopub.status.idle":"2023-09-03T08:41:09.030569Z","shell.execute_reply.started":"2023-09-03T08:41:09.020521Z","shell.execute_reply":"2023-09-03T08:41:09.029499Z"},"trusted":true},"execution_count":5,"outputs":[{"execution_count":5,"output_type":"execute_result","data":{"text/plain":"DatasetDict({\n    train: Dataset({\n        features: ['text', 'label'],\n        num_rows: 8530\n    })\n    validation: Dataset({\n        features: ['text', 'label'],\n        num_rows: 1066\n    })\n    test: Dataset({\n        features: ['text', 'label'],\n        num_rows: 1066\n    })\n})"},"metadata":{}}]},{"cell_type":"code","source":"raw_datasets[\"train\"][0]","metadata":{"execution":{"iopub.status.busy":"2023-09-03T08:41:11.365904Z","iopub.execute_input":"2023-09-03T08:41:11.366311Z","iopub.status.idle":"2023-09-03T08:41:11.374485Z","shell.execute_reply.started":"2023-09-03T08:41:11.366280Z","shell.execute_reply":"2023-09-03T08:41:11.373358Z"},"trusted":true},"execution_count":6,"outputs":[{"execution_count":6,"output_type":"execute_result","data":{"text/plain":"{'text': 'the rock is destined to be the 21st century\\'s new \" conan \" and that he\\'s going to make a splash even greater than arnold schwarzenegger , jean-claud van damme or steven segal .',\n 'label': 1}"},"metadata":{}}]},{"cell_type":"markdown","source":"# <b><div style='padding:15px;background-color:#850E35;color:white;border-radius:2px;font-size:110%;text-align: center'>4. Understand Data</div></b>","metadata":{}},{"cell_type":"markdown","source":"## From Datasets to Pandas DataFrames","metadata":{}},{"cell_type":"code","source":"import pandas as pd\n\nraw_datasets.set_format(type=\"pandas\")\ndf = raw_datasets[\"train\"][:]\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2023-09-03T08:41:14.948516Z","iopub.execute_input":"2023-09-03T08:41:14.948874Z","iopub.status.idle":"2023-09-03T08:41:14.993093Z","shell.execute_reply.started":"2023-09-03T08:41:14.948845Z","shell.execute_reply":"2023-09-03T08:41:14.992216Z"},"trusted":true},"execution_count":7,"outputs":[{"execution_count":7,"output_type":"execute_result","data":{"text/plain":"                                                text  label\n0  the rock is destined to be the 21st century's ...      1\n1  the gorgeously elaborate continuation of \" the...      1\n2                     effective but too-tepid biopic      1\n3  if you sometimes like to go to the movies to h...      1\n4  emerges as something rare , an issue movie tha...      1","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>text</th>\n      <th>label</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>the rock is destined to be the 21st century's ...</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>the gorgeously elaborate continuation of \" the...</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>effective but too-tepid biopic</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>if you sometimes like to go to the movies to h...</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>emerges as something rare , an issue movie tha...</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>"},"metadata":{}}]},{"cell_type":"code","source":"def label_int2str(row):\n    return raw_datasets[\"train\"].features[\"label\"].int2str(row)\n\ndf[\"label_name\"] = df[\"label\"].apply(label_int2str)\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2023-09-03T08:41:18.055962Z","iopub.execute_input":"2023-09-03T08:41:18.056351Z","iopub.status.idle":"2023-09-03T08:41:18.101385Z","shell.execute_reply.started":"2023-09-03T08:41:18.056318Z","shell.execute_reply":"2023-09-03T08:41:18.100383Z"},"trusted":true},"execution_count":8,"outputs":[{"execution_count":8,"output_type":"execute_result","data":{"text/plain":"                                                text  label label_name\n0  the rock is destined to be the 21st century's ...      1        pos\n1  the gorgeously elaborate continuation of \" the...      1        pos\n2                     effective but too-tepid biopic      1        pos\n3  if you sometimes like to go to the movies to h...      1        pos\n4  emerges as something rare , an issue movie tha...      1        pos","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>text</th>\n      <th>label</th>\n      <th>label_name</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>the rock is destined to be the 21st century's ...</td>\n      <td>1</td>\n      <td>pos</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>the gorgeously elaborate continuation of \" the...</td>\n      <td>1</td>\n      <td>pos</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>effective but too-tepid biopic</td>\n      <td>1</td>\n      <td>pos</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>if you sometimes like to go to the movies to h...</td>\n      <td>1</td>\n      <td>pos</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>emerges as something rare , an issue movie tha...</td>\n      <td>1</td>\n      <td>pos</td>\n    </tr>\n  </tbody>\n</table>\n</div>"},"metadata":{}}]},{"cell_type":"markdown","source":"## Looking at the Label Distribution","metadata":{}},{"cell_type":"code","source":"import matplotlib.pyplot as plt\n\ndf[\"label_name\"].value_counts(ascending=True).plot.barh()\nplt.title(\"Frequency of Classes\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-09-03T08:41:20.806802Z","iopub.execute_input":"2023-09-03T08:41:20.807187Z","iopub.status.idle":"2023-09-03T08:41:21.149071Z","shell.execute_reply.started":"2023-09-03T08:41:20.807151Z","shell.execute_reply":"2023-09-03T08:41:21.148124Z"},"trusted":true},"execution_count":9,"outputs":[{"output_type":"display_data","data":{"text/plain":"<Figure size 640x480 with 1 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAicAAAGxCAYAAAC5hxYeAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAkkUlEQVR4nO3deVTVdf7H8ddV4CoIiCYgimZmarlVZmKlpqYmmtmmk5lNx1ZRPNmULb8DNs5R26ystCkzW0ZzskXTsdywGnFJUzEbo3FJx1BzAdJEhc/vjznc8coiEsr70vNxzj1Hvvf7/d7Ph89Rnud7v1c8zjknAAAAI6pV9gAAAABORpwAAABTiBMAAGAKcQIAAEwhTgAAgCnECQAAMIU4AQAAphAnAADAFOIEAACYQpwA5fTWW2/J4/EU+3j44Ycre3i/W0uWLFH79u0VFhYmj8ejjz/+uNT99+zZozFjxqh169aqVauWatSooWbNmik5OVmZmZm+/VJTU+XxeM7y6AFIUlBlDwAIdNOnT1eLFi38tsXFxVXSaH7fnHO67bbbdNFFF2nu3LkKCwtT8+bNS9x/9erV6tu3r5xzSkpKUkJCgkJCQrRlyxa9++676tChgw4ePHgOZwBAIk6A36xVq1Zq3759mfY9fvy4PB6PgoL4q3c27N69WwcOHNCAAQPUvXv3UvfNyclR//79VaNGDa1YsUINGzb0Pde1a1fdd999+uCDD872kAEUg7d1gLMkLS1NHo9H77zzjkaPHq0GDRrI6/Xqhx9+kCQtXrxY3bt3V0REhEJDQ3XVVVdpyZIlRc4zf/58tWvXTl6vV02aNNGzzz5b5C2G7du3y+Px6K233ipyvMfjUWpqqt+2zMxM3X777YqOjpbX61XLli31yiuvFDv+mTNn6oknnlBcXJwiIiLUo0cPbdmypcjrLFy4UN27d1dkZKRCQ0PVsmVLjR8/XpL0zjvvyOPxKD09vchxTz31lIKDg7V79+5Sv59fffWVunfvrvDwcIWGhqpTp06aP3++7/nU1FRfYDz66KPyeDw6//zzSzzf66+/rqysLD399NN+YXKyW265pdQxvf/+++rZs6fq16+vmjVrqmXLlhozZowOHz7st9/WrVs1aNAgxcXFyev1KiYmRt27d9f69et9+yxdulRdu3ZV3bp1VbNmTTVq1Eg333yzjhw54tvn2LFjGjdunFq0aCGv16t69erpj3/8o/bt2+f3emU5F2AZcQL8Rvn5+Tpx4oTf42SPPfaYfvzxR02dOlXz5s1TdHS03n33XfXs2VMRERGaMWOGZs+erTp16qhXr15+gbJkyRL1799f4eHhmjVrlp555hnNnj1b06dPL/d4N2/erCuuuEKbNm3Sc889p08//VSJiYkaOXKkxo4dW2T/xx9/XDt27NAbb7yhv/71r8rMzFS/fv2Un5/v22fatGnq06ePCgoKfPMcOXKkdu3aJUkaOHCgYmNjiwTQiRMn9Nprr2nAgAGlvhW2fPlydevWTdnZ2Zo2bZpmzpyp8PBw9evXT++//74kadiwYfrwww8lSSNGjFB6ero++uijEs/5+eefq3r16urXr1/Zv3mnyMzMVJ8+fTRt2jQtXLhQo0aN0uzZs4ucs0+fPlq7dq2efvppLVq0SFOmTNGll16qQ4cOSfpvXCYmJiokJERvvvmmFi5cqAkTJigsLEzHjh2TJBUUFKh///6aMGGCbr/9ds2fP18TJkzQokWL1LVrV/36669lPhdgngNQLtOnT3eSin0cP37cLVu2zElynTt39jvu8OHDrk6dOq5fv35+2/Pz813btm1dhw4dfNuuvPJKFxcX53799VfftpycHFenTh138l/fbdu2OUlu+vTpRcYpyaWkpPi+7tWrl2vYsKHLzs722y8pKcnVqFHDHThwwDnnfOPv06eP336zZ892klx6erpzzrnc3FwXERHhrr76aldQUFDi9yslJcWFhIS4PXv2+La9//77TpJbvnx5icc551zHjh1ddHS0y83N9W07ceKEa9WqlWvYsKHvdQu/D88880yp53POuRYtWrjY2NjT7nfy+Ev7J7OgoMAdP37cLV++3ElyGzZscM459/PPPztJ7oUXXijx2A8++MBJcuvXry9xn5kzZzpJbs6cOX7b16xZ4yS5V199tcznAqzjygnwG7399ttas2aN3+Pke0puvvlmv/1XrFihAwcOaOjQoX5XWwoKCtS7d2+tWbNGhw8f1uHDh7VmzRrddNNNqlGjhu/4wisG5XH06FEtWbJEAwYMUGhoqN/r9+nTR0ePHtXKlSv9jrnhhhv8vm7Tpo0kaceOHb755OTk6MEHHyz10ywPPPCApP++nVLo5ZdfVuvWrdW5c+cSjzt8+LBWrVqlW265RbVq1fJtr169uoYMGaJdu3YV+zbTubB161bdfvvtio2NVfXq1RUcHKwuXbpIkr777jtJUp06ddS0aVM988wzev755/XNN9+ooKDA7zzt2rVTSEiI7r33Xs2YMUNbt24t8lqffvqpateurX79+vmtW7t27RQbG6u0tLQynwuwjjgBfqOWLVuqffv2fo+T1a9f3+/rPXv2SPrv/QzBwcF+j4kTJ8o5pwMHDujgwYMqKChQbGxskdcsbltZ7N+/XydOnNDkyZOLvHafPn0kST///LPfMXXr1vX72uv1SpLvbYTC+x1Kum+jUExMjAYOHKjXXntN+fn52rhxo7788kslJSWVetzBgwflnCvyfZT+96mo/fv3l3qO4jRq1Ej79u0rcn9IWf3yyy+65pprtGrVKo0bN05paWlas2aN762lwu+Px+PRkiVL1KtXLz399NO67LLLVK9ePY0cOVK5ubmSpKZNm2rx4sWKjo7W8OHD1bRpUzVt2lQvvvii7/X27NmjQ4cOKSQkpMjaZWVl+datLOcCrOMjA8BZdurVhPPOO0+SNHnyZHXs2LHYY2JiYnyf7MnKyiry/KnbCq+s5OXl+W0/9Yd2VFSU74rD8OHDi33tJk2alDKbourVqydJvvtLSpOcnKx33nlHn3zyiRYuXKjatWtr8ODBpR4TFRWlatWq6aeffiryXOFNtIXf0zPRq1cvff7555o3b54GDRp0xscvXbpUu3fvVlpamu9qiSTffSQna9y4saZNmyZJ+v777zV79mylpqbq2LFjmjp1qiTpmmuu0TXXXKP8/Hx9/fXXmjx5skaNGqWYmBgNGjRI5513nurWrauFCxcWO57w8HDfn093LsC8yn5fCQhUhfecrFmzptjnC+/Z+Pvf/+63PTc319WuXds98MADp32Nst5zUlBQ4GrUqOEefPBBv+OnTZtW5J6THj16uLZt27q8vLxSX7uk8Z96f0tubq6LjIx0nTt3LvWek0KdOnVyHTp0cKGhoW7UqFGn3d855xISElxsbKw7cuSIb1t+fr5r3bp1ue85OXTokIuNjXXx8fFu165dxe5z8v0dp95zMnfuXL97bwrdcsstJd7/c7J27dq5K664otTxSXJ/+tOfnHPOvfvuu06SW7ly5emmdtpzAdZx5QQ4x2rVqqXJkydr6NChOnDggG655RZFR0dr37592rBhg/bt26cpU6ZIkv785z+rd+/euu666zR69Gjl5+dr4sSJCgsL04EDB3zn9Hg8uuOOO/Tmm2+qadOmatu2rVavXq2//e1vRV7/xRdf1NVXX61rrrlGDzzwgM4//3zl5ubqhx9+0Lx587R06dIzns9zzz2nYcOGqUePHrrnnnsUExOjH374QRs2bNDLL7/st39ycrIGDhwoj8ejBx98sEyvMX78eF133XW69tpr9fDDDyskJESvvvqqNm3apJkzZ5brf26NjIzUJ598or59++rSSy/1+0/YMjMz9e6772rDhg266aabij2+U6dOioqK0v3336+UlBQFBwfrvffe04YNG/z227hxo5KSknTrrbeqWbNmCgkJ0dKlS7Vx40aNGTNGkjR16lQtXbpUiYmJatSokY4ePao333xTktSjRw9J0qBBg/Tee++pT58+Sk5OVocOHRQcHKxdu3Zp2bJl6t+/vwYMGFCmcwHmVXYdAYGqvFdOCi1fvtwlJia6OnXquODgYNegQQOXmJhYZP+5c+e6Nm3auJCQENeoUSM3YcKEYj85kp2d7YYNG+ZiYmJcWFiY69evn9u+fXuRKyfO/fcKw9133+0aNGjggoODXb169VynTp3cuHHjTjv+kj4ZtGDBAtelSxcXFhbmQkND3cUXX+wmTpxYZN55eXnO6/W63r17F/t9KcmXX37punXr5sLCwlzNmjVdx44d3bx584odW1munBTKyspyjz76qLvkkktcaGio83q97sILL3T33Xefy8jI8O1X3Pd8xYoVLiEhwYWGhrp69eq5YcOGuXXr1vl9f/bs2ePuuusu16JFCxcWFuZq1arl2rRp4yZNmuROnDjhnHMuPT3dDRgwwDVu3Nh5vV5Xt25d16VLFzd37ly/1zt+/Lh79tlnXdu2bV2NGjVcrVq1XIsWLdx9993nMjMzz+hcgGUe55yrtDICUC6pqakaO3asAvGv77x583TDDTdo/vz5vptwAeBkvK0D4JzYvHmzduzYodGjR6tdu3a6/vrrK3tIAIzio8QAzokHH3xQN9xwg6Kiosp9nwiA3wfe1gEAAKZw5QQAAJhCnAAAAFOIEwAAYEpAflqnoKBAu3fvVnh4ODfVAQAQIJxzys3NVVxcnKpVK/n6SEDGye7duxUfH1/ZwwAAAOWwc+fOUn9ZaEDGSeEvuNq5c6ciIiIqeTQAAKAscnJyFB8f7/eLKosTkHFS+FZOREQEcQIAQIA53S0Z3BALAABMIU4AAIApxAkAADCFOAEAAKYQJwAAwBTiBAAAmEKcAAAAU4gTAABgCnECAABMIU4AAIApxAkAADCFOAEAAKYQJwAAwBTiBAAAmEKcAAAAU4gTAABgCnECAABMIU4AAIApxAkAADCFOAEAAKYQJwAAwBTiBAAAmEKcAAAAU4gTAABgCnECAABMIU4AAIApxAkAADCFOAEAAKYQJwAAwBTiBAAAmEKcAAAAU4gTAABgCnECAABMIU4AAIApxAkAADCFOAEAAKYQJwAAwBTiBAAAmEKcAAAAU4gTAABgCnECAABMIU4AAIApxAkAADAlqLIH8Fu0SvlM1byhlT0MAACqjO0TEit7CFw5AQAAthAnAADAFOIEAACYQpwAAABTiBMAAGAKcQIAAEwhTgAAgCnECQAAMIU4AQAAphAnAADAFOIEAACYQpwAAABTiBMAAGAKcQIAAEwhTgAAgCnECQAAMIU4AQAAphAnAADAFOIEAACYQpwAAABTiBMAAGAKcQIAAEwhTgAAgCnECQAAMIU4AQAAphAnAADAFOIEAACYQpwAAABTiBMAAGAKcQIAAEwhTgAAgCnECQAAMIU4AQAAphAnAADAFOIEAACYQpwAAABTiBMAAGAKcQIAAEwhTgAAgCnECQAAMIU4AQAAphAnAADAFOIEAACYckZx0rVrV40cOVKPPPKI6tSpo9jYWKWmpvqez87O1r333qvo6GhFRESoW7du2rBhg985xo0bp+joaIWHh2vYsGEaM2aM2rVrVxFzAQAAVcAZXzmZMWOGwsLCtGrVKj399NN66qmntGjRIjnnlJiYqKysLC1YsEBr167VZZddpu7du+vAgQOSpPfee09/+ctfNHHiRK1du1aNGjXSlClTTvuaeXl5ysnJ8XsAAICqyeOcc2XduWvXrsrPz9eXX37p29ahQwd169ZNPXv21IABA7R37155vV7f8xdeeKEeeeQR3XvvverYsaPat2+vl19+2ff81VdfrV9++UXr168v8XVTU1M1duzYItvjR81WNW9oWYcPAABOY/uExLN27pycHEVGRio7O1sREREl7nfGV07atGnj93X9+vW1d+9erV27Vr/88ovq1q2rWrVq+R7btm3Tv//9b0nSli1b1KFDB7/jT/26OI899piys7N9j507d57psAEAQIAIOtMDgoOD/b72eDwqKChQQUGB6tevr7S0tCLH1K5d22//k5Xlwo3X6/W7GgMAAKquM46Tklx22WXKyspSUFCQzj///GL3ad68uVavXq0hQ4b4tn399dcVNQQAAFAFVNhHiXv06KGEhATdeOON+uyzz7R9+3atWLFCTz75pC9ARowYoWnTpmnGjBnKzMzUuHHjtHHjxiJXUwAAwO9XhV058Xg8WrBggZ544gndfffd2rdvn2JjY9W5c2fFxMRIkgYPHqytW7fq4Ycf1tGjR3Xbbbfprrvu0urVqytqGAAAIMCd0ad1zobrrrtOsbGxeuedd8p8TOHdvnxaBwCAimXh0zoVduWkLI4cOaKpU6eqV69eql69umbOnKnFixdr0aJF53IYAADAsHMaJ4Vv/YwbN055eXlq3ry55syZox49epzLYQAAAMPOaZzUrFlTixcvPpcvCQAAAgy/+A8AAJhCnAAAAFOIEwAAYApxAgAATCFOAACAKcQJAAAwhTgBAACmECcAAMAU4gQAAJhCnAAAAFOIEwAAYApxAgAATCFOAACAKcQJAAAwhTgBAACmECcAAMAU4gQAAJhCnAAAAFOIEwAAYApxAgAATCFOAACAKcQJAAAwhTgBAACmECcAAMAU4gQAAJhCnAAAAFOIEwAAYApxAgAATCFOAACAKcQJAAAwhTgBAACmECcAAMAU4gQAAJhCnAAAAFOIEwAAYApxAgAATAmq7AH8FpvG9lJERERlDwMAAFQgrpwAAABTiBMAAGAKcQIAAEwhTgAAgCnECQAAMIU4AQAAphAnAADAFOIEAACYQpwAAABTiBMAAGAKcQIAAEwhTgAAgCnECQAAMIU4AQAAphAnAADAFOIEAACYQpwAAABTiBMAAGAKcQIAAEwhTgAAgCnECQAAMIU4AQAAphAnAADAFOIEAACYQpwAAABTiBMAAGAKcQIAAEwhTgAAgCnECQAAMIU4AQAAphAnAADAFOIEAACYQpwAAABTiBMAAGAKcQIAAEwhTgAAgCnECQAAMIU4AQAAphAnAADAFOIEAACYQpwAAABTiBMAAGAKcQIAAEwhTgAAgCnECQAAMIU4AQAAphAnAADAFOIEAACYQpwAAABTiBMAAGAKcQIAAEwhTgAAgCnECQAAMIU4AQAAphAnAADAFOIEAACYQpwAAABTiBMAAGAKcQIAAEwhTgAAgCnECQAAMIU4AQAAphAnAADAFOIEAACYQpwAAABTiBMAAGAKcQIAAEwhTgAAgCnECQAAMIU4AQAAphAnAADAFOIEAACYQpwAAABTiBMAAGAKcQIAAEwhTgAAgCnECQAAMIU4AQAAphAnAADAFOIEAACYQpwAAABTiBMAAGAKcQIAAEwhTgAAgCnECQAAMIU4AQAAphAnAADAFOIEAACYQpwAAABTiBMAAGAKcQIAAEwhTgAAgClBlT2A36JVymeq5g2t7GEAAFBlbJ+QWNlD4MoJAACwhTgBAACmECcAAMAU4gQAAJhCnAAAAFOIEwAAYApxAgAATCFOAACAKcQJAAAwhTgBAACmECcAAMAU4gQAAJhCnAAAAFOIEwAAYApxAgAATCFOAACAKcQJAAAwhTgBAACmECcAAMAU4gQAAJhCnAAAAFOIEwAAYApxAgAATCFOAACAKcQJAAAwhTgBAACmECcAAMAU4gQAAJhCnAAAAFOIEwAAYApxAgAATCFOAACAKcQJAAAwhTgBAACmECcAAMAU4gQAAJhCnAAAAFOIEwAAYApxAgAATCFOAACAKcQJAAAwhTgBAACmECcAAMCUcsVJ165dlZSUpKSkJNWuXVt169bVk08+KeecJOngwYO68847FRUVpdDQUF1//fXKzMz0Hb9jxw7169dPUVFRCgsL0yWXXKIFCxZUzIwAAEBAK/eVkxkzZigoKEirVq3SSy+9pEmTJumNN96QJN111136+uuvNXfuXKWnp8s5pz59+uj48eOSpOHDhysvL09ffPGFMjIyNHHiRNWqVavE18rLy1NOTo7fAwAAVE1B5T0wPj5ekyZNksfjUfPmzZWRkaFJkyapa9eumjt3rv75z3+qU6dOkqT33ntP8fHx+vjjj3Xrrbfqxx9/1M0336zWrVtLki644IJSX2v8+PEaO3ZseYcKAAACSLmvnHTs2FEej8f3dUJCgjIzM7V582YFBQXpyiuv9D1Xt25dNW/eXN99950kaeTIkRo3bpyuuuoqpaSkaOPGjaW+1mOPPabs7GzfY+fOneUdNgAAMO6c3RDrnPPFzLBhw7R161YNGTJEGRkZat++vSZPnlzisV6vVxEREX4PAABQNZU7TlauXFnk62bNmuniiy/WiRMntGrVKt9z+/fv1/fff6+WLVv6tsXHx+v+++/Xhx9+qNGjR+v1118v71AAAEAVUu442blzpx566CFt2bJFM2fO1OTJk5WcnKxmzZqpf//+uueee/TVV19pw4YNuuOOO9SgQQP1799fkjRq1Ch99tln2rZtm9atW6elS5f6hQsAAPj9KvcNsXfeead+/fVXdejQQdWrV9eIESN07733SpKmT5+u5ORk9e3bV8eOHVPnzp21YMECBQcHS5Ly8/M1fPhw7dq1SxEREerdu7cmTZpUMTMCAAABrdxxEhwcrBdeeEFTpkwp8lxUVJTefvvtEo8t7f4SAADw+8b/EAsAAEwhTgAAgCnlelsnLS2tgocBAADwX1w5AQAAphAnAADAFOIEAACYQpwAAABTiBMAAGAKcQIAAEwhTgAAgCnECQAAMIU4AQAAphAnAADAFOIEAACYQpwAAABTiBMAAGAKcQIAAEwhTgAAgCnECQAAMIU4AQAAphAnAADAFOIEAACYQpwAAABTiBMAAGAKcQIAAEwhTgAAgCnECQAAMIU4AQAAphAnAADAFOIEAACYQpwAAABTiBMAAGAKcQIAAEwhTgAAgCnECQAAMIU4AQAAphAnAADAFOIEAACYQpwAAABTgip7AL/FprG9FBERUdnDAAAAFYgrJwAAwBTiBAAAmEKcAAAAU4gTAABgCnECAABMIU4AAIApxAkAADCFOAEAAKYQJwAAwBTiBAAAmEKcAAAAU4gTAABgCnECAABMIU4AAIApxAkAADCFOAEAAKYQJwAAwBTiBAAAmEKcAAAAU4gTAABgCnECAABMIU4AAIApxAkAADCFOAEAAKYQJwAAwBTiBAAAmEKcAAAAU4gTAABgCnECAABMIU4AAIApxAkAADCFOAEAAKYQJwAAwBTiBAAAmEKcAAAAU4gTAABgCnECAABMIU4AAIApxAkAADCFOAEAAKYQJwAAwBTiBAAAmEKcAAAAU4gTAABgSlBlD6A8nHOSpJycnEoeCQAAKKvCn9uFP8dLEpBxsn//fklSfHx8JY8EAACcqdzcXEVGRpb4fEDGSZ06dSRJP/74Y6mTC0Q5OTmKj4/Xzp07FRERUdnDqXBVeX7MLXBV5fkxt8BVFefnnFNubq7i4uJK3S8g46Ratf/eKhMZGVllFuxUERERVXZuUtWeH3MLXFV5fswtcFW1+ZXlogI3xAIAAFOIEwAAYEpAxonX61VKSoq8Xm9lD6XCVeW5SVV7fswtcFXl+TG3wFXV51cajzvd53kAAADOoYC8cgIAAKou4gQAAJhCnAAAAFOIEwAAYApxAgAATAm4OHn11VfVpEkT1ahRQ5dffrm+/PLLyh7SaaWmpsrj8fg9YmNjfc8755Samqq4uDjVrFlTXbt21bfffut3jry8PI0YMULnnXeewsLCdMMNN2jXrl3neir64osv1K9fP8XFxcnj8ejjjz/2e76i5nLw4EENGTJEkZGRioyM1JAhQ3To0KGzPLvTz++uu+4qspYdO3b028fq/MaPH68rrrhC4eHhio6O1o033qgtW7b47ROo61eWuQXq2k2ZMkVt2rTx/S+hCQkJ+sc//uF7PlDXrKzzC9R1K8748ePl8Xg0atQo37ZAX7+zxgWQWbNmueDgYPf666+7zZs3u+TkZBcWFuZ27NhR2UMrVUpKirvkkkvcTz/95Hvs3bvX9/yECRNceHi4mzNnjsvIyHADBw509evXdzk5Ob597r//ftegQQO3aNEit27dOnfttde6tm3buhMnTpzTuSxYsMA98cQTbs6cOU6S++ijj/yer6i59O7d27Vq1cqtWLHCrVixwrVq1cr17du30uc3dOhQ17t3b7+13L9/v98+VufXq1cvN336dLdp0ya3fv16l5iY6Bo1auR++eUX3z6Bun5lmVugrt3cuXPd/Pnz3ZYtW9yWLVvc448/7oKDg92mTZucc4G7ZmWdX6Cu26lWr17tzj//fNemTRuXnJzs2x7o63e2BFScdOjQwd1///1+21q0aOHGjBlTSSMqm5SUFNe2bdtinysoKHCxsbFuwoQJvm1Hjx51kZGRburUqc455w4dOuSCg4PdrFmzfPv85z//cdWqVXMLFy48q2Mvzak/vCtqLps3b3aS3MqVK337pKenO0nuX//611me1f+UFCf9+/cv8ZhAmt/evXudJLd8+XLnXNVav1Pn5lzVWruoqCj3xhtvVKk1O1nh/JyrGuuWm5vrmjVr5hYtWuS6dOnii5Oqun4VIWDe1jl27JjWrl2rnj17+m3v2bOnVqxYUUmjKrvMzEzFxcWpSZMmGjRokLZu3SpJ2rZtm7Kysvzm5fV61aVLF9+81q5dq+PHj/vtExcXp1atWpmae0XNJT09XZGRkbryyit9+3Ts2FGRkZEm5puWlqbo6GhddNFFuueee7R3717fc4E0v+zsbEn/+y3fVWn9Tp1boUBfu/z8fM2aNUuHDx9WQkJClVozqej8CgX6ug0fPlyJiYnq0aOH3/aqtn4VKWB+K/HPP/+s/Px8xcTE+G2PiYlRVlZWJY2qbK688kq9/fbbuuiii7Rnzx6NGzdOnTp10rfffusbe3Hz2rFjhyQpKytLISEhioqKKrKPpblX1FyysrIUHR1d5PzR0dGVPt/rr79et956qxo3bqxt27bp//7v/9StWzetXbtWXq83YObnnNNDDz2kq6++Wq1atfKNq3CsJwu09StublJgr11GRoYSEhJ09OhR1apVSx999JEuvvhi3w+eQF+zkuYnBfa6SdKsWbO0bt06rVmzpshzVeXv3NkQMHFSyOPx+H3tnCuyzZrrr7/e9+fWrVsrISFBTZs21YwZM3w3dpVnXlbnXhFzKW5/C/MdOHCg78+tWrVS+/bt1bhxY82fP1833XRTicdZm19SUpI2btyor776qshzgb5+Jc0tkNeuefPmWr9+vQ4dOqQ5c+Zo6NChWr58eYljCrQ1K2l+F198cUCv286dO5WcnKzPP/9cNWrUKHG/QF+/syFg3tY577zzVL169SIVuHfv3iLVaV1YWJhat26tzMxM36d2SptXbGysjh07poMHD5a4jwUVNZfY2Fjt2bOnyPn37dtnar6SVL9+fTVu3FiZmZmSAmN+I0aM0Ny5c7Vs2TI1bNjQt70qrF9JcytOIK1dSEiILrzwQrVv317jx49X27Zt9eKLL1aJNZNKnl9xAmnd1q5dq7179+ryyy9XUFCQgoKCtHz5cr300ksKCgryvXagr9/ZEDBxEhISossvv1yLFi3y275o0SJ16tSpkkZVPnl5efruu+9Uv359NWnSRLGxsX7zOnbsmJYvX+6b1+WXX67g4GC/fX766Sdt2rTJ1Nwrai4JCQnKzs7W6tWrffusWrVK2dnZpuYrSfv379fOnTtVv359Sbbn55xTUlKSPvzwQy1dulRNmjTxez6Q1+90cytOIK3dqZxzysvLC+g1K03h/IoTSOvWvXt3ZWRkaP369b5H+/btNXjwYK1fv14XXHBBlVy/CnGObrytEIUfJZ42bZrbvHmzGzVqlAsLC3Pbt2+v7KGVavTo0S4tLc1t3brVrVy50vXt29eFh4f7xj1hwgQXGRnpPvzwQ5eRkeH+8Ic/FPtRsoYNG7rFixe7devWuW7dulXKR4lzc3PdN99847755hsnyT3//PPum2++8X2cu6Lm0rt3b9emTRuXnp7u0tPTXevWrc/Jx+JKm19ubq4bPXq0W7Fihdu2bZtbtmyZS0hIcA0aNAiI+T3wwAMuMjLSpaWl+X0s88iRI759AnX9Tje3QF67xx57zH3xxRdu27ZtbuPGje7xxx931apVc59//rlzLnDXrCzzC+R1K8nJn9ZxLvDX72wJqDhxzrlXXnnFNW7c2IWEhLjLLrvM76OCVhV+bj04ONjFxcW5m266yX377be+5wsKClxKSoqLjY11Xq/Xde7c2WVkZPid49dff3VJSUmuTp06rmbNmq5v377uxx9/PNdTccuWLXOSijyGDh1aoXPZv3+/Gzx4sAsPD3fh4eFu8ODB7uDBg5U6vyNHjriePXu6evXqueDgYNeoUSM3dOjQImO3Or/i5iXJTZ8+3bdPoK7f6eYWyGt39913+/7Nq1evnuvevbsvTJwL3DUry/wCed1KcmqcBPr6nS0e55w7d9dpAAAAShcw95wAAIDfB+IEAACYQpwAAABTiBMAAGAKcQIAAEwhTgAAgCnECQAAMIU4AQAAphAnAADAFOIEAACYQpwAAABT/h/x8VcIly/w4gAAAABJRU5ErkJggg=="},"metadata":{}}]},{"cell_type":"markdown","source":"## How Long Are Our Texts?","metadata":{}},{"cell_type":"code","source":"df[\"Words Per Text\"] = df[\"text\"].str.split().apply(len)\ndf.boxplot(\"Words Per Text\", by=\"label_name\", grid=False, showfliers=False, color=\"black\")\nplt.suptitle(\"\")\nplt.xlabel(\"\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-09-03T08:41:26.485741Z","iopub.execute_input":"2023-09-03T08:41:26.486096Z","iopub.status.idle":"2023-09-03T08:41:26.767374Z","shell.execute_reply.started":"2023-09-03T08:41:26.486064Z","shell.execute_reply":"2023-09-03T08:41:26.766366Z"},"trusted":true},"execution_count":10,"outputs":[{"output_type":"display_data","data":{"text/plain":"<Figure size 640x480 with 1 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAi8AAAGoCAYAAACOiQW5AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAjsElEQVR4nO3de1DU9f7H8dcCui4KZF5YUQpKVNJQ0TTvaKF5aTLPqTN5SafMNNTI09G8/JIag6MzqZVl0505ZpcpU095VCrROmYHUdNRNCtIndooNUBEvH1+fzTstKEmsrh85PmY2Tlnv9/vfr/vbWb16Xe/u+swxhgBAABYIijQAwAAAFQF8QIAAKxCvAAAAKsQLwAAwCrECwAAsArxAgAArEK8AAAAqxAvAADAKsQLAACwCvEC1GLvvfeeHA6H3nnnnUrrOnbsKIfDoXXr1lVad/311ysxMbFGZ8vOzpbD4VB2dna19zVu3Dg5HA7vzel0qm3btpo7d65OnDhR/WHP44033vA57vluMTExfjvmmjVrlJaW5rf9AXUR8QLUYklJSXI4HNqwYYPP8iNHjmjXrl1q2LBhpXWHDh3Sd999p/79+1/OUavN5XLpiy++0BdffKGVK1eqe/fuevLJJzV27NgaO+bQoUO9x6y4SdJf//pXn2UffPCB3465Zs0aPfHEE37bH1AXhQR6AADn17RpU3Xo0KHS2Y2NGzcqJCRE999/f6V4qbjvj3gpKyuTy+Wq9n4uRlBQkG6++Wbv/cGDB6ugoEDvvvuuFi5cqJYtW17yvo0xOnHiRKXn0qxZMzVr1qzS9pGRkT6zAKhdOPMC1HL9+/fXvn379OOPP3qXZWdn66abbtKQIUOUm5urkpISn3XBwcHq06ePJOnEiROaOXOmYmNjVb9+fbVs2VIpKSn69ddffY4TExOjYcOGacWKFercubMaNGjgPUOwd+9e3XbbbQoNDVXTpk01ceJEn2NW2L59u4YNG6bmzZvL6XQqKipKQ4cO1aFDhy7puVcExPfffy9JKi4u1qOPPurzXFJTU1VaWurzOIfDocmTJ+vFF19UfHy8nE6nMjMzL2kGSdq/f79GjhzpfV7x8fF6/vnnvetPnDihzp07q3Xr1ioqKvIu93g8crvdSkpK0pkzZzRu3Djv437/tlRBQcElzwbURZx5AWq5/v3769lnn1V2drbuueceSb+dXRk2bJh69eolh8Ohzz77TEOGDPGuS0xMVEREhIwxGj58uD755BPNnDlTffr00c6dOzV37lzvWyJOp9N7rG3btikvL09z5sxRbGysGjZsqJ9++kn9+vVTvXr19MILLygyMlJvvvmmJk+e7DNnaWmpkpOTFRsbq+eff16RkZHyeDzasGHDOUPnYnzzzTeSfjtDcvz4cfXr10+HDh3SrFmzlJCQoN27d+vxxx/Xrl279PHHH8vhcHgfu3LlSn322Wd6/PHH5Xa71bx580uaYc+ePerZs6euueYaPf3003K73Vq3bp2mTp2qX375RXPnzlWDBg307rvvqkuXLrrvvvv0/vvv6+zZsxo1apSMMXrrrbcUHBys//u//1Npaanee+8971tUktSiRYtLmg2oswyAWu3IkSMmKCjITJgwwRhjzC+//GIcDodZu3atMcaYbt26mUcffdQYY8yBAweMJDN9+nRjjDFr1641ksyCBQt89vnOO+8YSeall17yLrv22mtNcHCw2bdvn8+2M2bMMA6Hw+zYscNneXJyspFkNmzYYIwxZuvWrUaSWblyZZWf49ixY03Dhg3NqVOnzKlTp8zPP/9snnnmGeNwOMxNN91kjDEmIyPDBAUFmZycHJ/Hvvfee0aSWbNmjXeZJBMREWGOHDlS5VkkmZSUFO/9QYMGmVatWpmioiKf7SZPnmwaNGjgc4yK/66LFy82jz/+uAkKCjLr16/3eVxKSorhj16genjbCKjlGjdurI4dO3qve9m4caOCg4PVq1cvSVK/fv2817n88XqXTz/9VNJvn+b5vbvuuksNGzbUJ5984rM8ISFBbdq08Vm2YcMGtW/fXh07dvRZPnLkSJ/7rVu3VuPGjTVjxgy9+OKL2rNnT5WeZ2lpqerVq6d69eqpWbNmSk1N1eDBg70Xy3744Yfq0KGDOnXqpNOnT3tvgwYNOuenngYMGKDGjRtXaYY/OnHihD755BPdeeedCg0N9TnukCFDdOLECW3ZssW7/d13361JkybpH//4h+bNm6dZs2YpOTm5WjMAqIx4ASzQv39/ff311/rhhx+0YcMGdenSRY0aNZL0W7xs375dRUVF2rBhg0JCQtS7d29J0uHDhxUSElLpolSHwyG3263Dhw/7LD/X2xeHDx+W2+2utPyPyyIiIrRx40Z16tRJs2bNUvv27RUVFaW5c+fq1KlTf/ocXS6XcnJylJOTo507d+rXX3/VRx995L1Q96efftLOnTu9gVNxCwsLkzFGv/zyy58+l6o6fPiwTp8+reeee67ScSvepvvjce+77z6dOnVKISEhmjp1arVnAFAZ17wAFujfv78WLlyo7OxsZWdne//ilOQNlU2bNnkv5K0ImyZNmuj06dP6+eeffQLGGCOPx6ObbrrJ5zi/v2akQpMmTeTxeCotP9eyG2+8UW+//baMMdq5c6feeOMNPfnkk3K5XHrssccu+ByDgoLUtWvX865v2rSpXC6XXnvttfOu/7PnUlWNGzdWcHCwxowZo5SUlHNuExsb6/3/paWlGjNmjNq0aaOffvpJ48eP16pVq6o9BwBfnHkBLNC3b18FBwfrvffe0+7du5WUlORdFxERoU6dOikzM1MFBQU+H5G+5ZZbJEnLli3z2d/777+v0tJS7/oL6d+/v3bv3q2vvvrKZ/ny5cvP+xiHw6GOHTtq0aJFuuqqq7Rt27aLeZoXNGzYMH377bdq0qSJunbtWunmzy+SqxAaGqr+/ftr+/btSkhIOOdxmzRp4t1+4sSJOnDggFasWKFXX31Vq1ev1qJFi3z2WXGBdFlZmd/nBeoKzrwAFggPD1diYqJWrlypoKAg7/UuFfr166fFixdL8v1+l+TkZA0aNEgzZsxQcXGxevXq5f20UefOnTVmzJg/PXZqaqpee+01DR06VPPmzfN+2mjv3r0+23344Yd64YUXNHz4cF133XUyxmjFihX69ddf/XLdR2pqqt5//3317dtXjzzyiBISEnT27FkdOHBA69ev19///nd179692sf5o2eeeUa9e/dWnz59NGnSJMXExKikpETffPON/v3vf3uvK3rllVe0bNkyvf7662rfvr3at2+vyZMna8aMGerVq5e6desm6bezU5I0f/58DR48WMHBwUpISFD9+vX9PjtwxQrs9cIALtb06dONJNO1a9dK61auXGkkmfr165vS0lKfdWVlZWbGjBnm2muvNfXq1TMtWrQwkyZNMkePHvXZ7tprrzVDhw4957H37NljkpOTTYMGDczVV19t7r//frNq1SqfTxvt3bvX3HPPPeb66683LpfLREREmG7dupk33njjT59bxaeN/syxY8fMnDlzTNu2bU39+vVNRESEufHGG80jjzxiPB6Pdzv94RNDVXGux+bn55v77rvPtGzZ0tSrV880a9bM9OzZ08ybN88YY8zOnTuNy+UyY8eO9XnciRMnTJcuXUxMTIz3v3d5ebkZP368adasmXE4HEaSyc/Pv6RZgbrKYYwxgYwnAACAquCaFwAAYBXiBQAAWIV4AQAAViFeAACAVYgXAABgFeIFAABYpdZ9Sd3Zs2f1ww8/KCwszC9f7w0AAGo/Y4xKSkoUFRWloKALn1updfHyww8/KDo6OtBjAACAADh48KBatWp1wW1qXbyEhYVJ+m348PDwAE8DAAAuh+LiYkVHR3s74EJqXbxUvFUUHh5OvAAAUMdczCUjXLALAACsQrwAAACrEC8AAMAqxAsAALAK8QIAAKxCvAAAAKsQLwAAwCrECwAAsArxAgAArEK8AAAAqxAvAADAKsQLAACwCvECAACsUut+VRpXruPHj2vv3r0X3KasrEwFBQWKiYmRy+X60322a9dOoaGh/hoRQA3h9Q9/Il5w2ezdu1ddunTx6z5zc3OVmJjo130C8D9e//An4gWXTbt27ZSbm3vBbfLy8jR69GgtW7ZM8fHxF7VPALUfr3/4E/GCyyY0NPSi/5UUHx/Pv6iAKwivf/gTF+wCAACrEC8AAMAqxAsAALAK8QIAAKxCvAAAAKsQLwAAwCrECwAAsArxAgAArEK8AAAAqxAvAADAKsQLAACwCvECAACsQrwAAACrEC8AAMAqxAsAALAK8QIAAKxCvAAAAKsQLwAAwCrECwAAsArxAgAArEK8AAAAqxAvAADAKsQLAACwCvECAACsQrwAAACrEC8AAMAqxAsAALAK8QIAAKxCvAAAAKsQLwAAwCrECwAAsArxAgAArEK8AAAAqxAvAADAKsQLAACwCvECAACsUq14ycjIkMPhUGpqqneZMUZpaWmKioqSy+VSUlKSdu/eXd05AQAAJFUjXnJycvTSSy8pISHBZ/mCBQu0cOFCLVmyRDk5OXK73UpOTlZJSUm1hwUAALikeDl27JhGjRqll19+WY0bN/YuN8Zo8eLFmj17tkaMGKEOHTooMzNTx48f1/Lly/02NAAAqLsuKV5SUlI0dOhQ3XrrrT7L8/Pz5fF4NHDgQO8yp9Opfv36afPmzefcV3l5uYqLi31uAAAA5xNS1Qe8/fbb2rZtm3Jyciqt83g8kqTIyEif5ZGRkfr+++/Pub+MjAw98cQTVR0DAADUUVU683Lw4EE9/PDDWrZsmRo0aHDe7RwOh899Y0ylZRVmzpypoqIi7+3gwYNVGQkAANQxVTrzkpubq8LCQnXp0sW77MyZM9q0aZOWLFmiffv2SfrtDEyLFi282xQWFlY6G1PB6XTK6XReyuwAAKAOqtKZl1tuuUW7du3Sjh07vLeuXbtq1KhR2rFjh6677jq53W5lZWV5H3Py5Elt3LhRPXv29PvwAACg7qnSmZewsDB16NDBZ1nDhg3VpEkT7/LU1FSlp6crLi5OcXFxSk9PV2hoqEaOHOm/qQEAQJ1V5Qt2/8z06dNVVlamhx56SEePHlX37t21fv16hYWF+ftQAACgDqp2vGRnZ/vcdzgcSktLU1paWnV3DQAAUAm/bQQAAKxCvAAAAKsQLwAAwCrECwAAsArxAgAArEK8AAAAqxAvAADAKsQLAACwCvECAACsQrwAAACrEC8AAMAqxAsAALAK8QIAAKxCvAAAAKsQLwAAwCrECwAAsArxAgAArEK8AAAAqxAvAADAKsQLAACwCvECAACsQrwAAACrEC8AAMAqxAsAALAK8QIAAKxCvAAAAKsQLwAAwCrECwAAsArxAgAArEK8AAAAqxAvAADAKsQLAACwCvECAACsQrwAAACrEC8AAMAqxAsAALAK8QIAAKxCvAAAAKsQLwAAwCrECwAAsArxAgAArEK8AAAAqxAvAADAKsQLAACwCvECAACsQrwAAACrEC8AAMAqxAsAALAK8QIAAKxCvAAAAKsQLwAAwCrECwAAsArxAgAArEK8AAAAqxAvAADAKsQLAACwCvECAACsQrwAAACrEC8AAMAqxAsAALAK8QIAAKxCvAAAAKtUKV6WLl2qhIQEhYeHKzw8XD169NB//vMf73pjjNLS0hQVFSWXy6WkpCTt3r3b70MDAIC6q0rx0qpVK/3zn//U1q1btXXrVg0YMEB33HGHN1AWLFighQsXasmSJcrJyZHb7VZycrJKSkpqZHgAAFD3VClebr/9dg0ZMkRt2rRRmzZt9NRTT6lRo0basmWLjDFavHixZs+erREjRqhDhw7KzMzU8ePHtXz58pqaHwAA1DGXfM3LmTNn9Pbbb6u0tFQ9evRQfn6+PB6PBg4c6N3G6XSqX79+2rx583n3U15eruLiYp8bAADA+VQ5Xnbt2qVGjRrJ6XRq4sSJ+uCDD3TDDTfI4/FIkiIjI322j4yM9K47l4yMDEVERHhv0dHRVR0JAADUIVWOl7Zt22rHjh3asmWLJk2apLFjx2rPnj3e9Q6Hw2d7Y0ylZb83c+ZMFRUVeW8HDx6s6kgAAKAOCanqA+rXr6/WrVtLkrp27aqcnBw988wzmjFjhiTJ4/GoRYsW3u0LCwsrnY35PafTKafTWdUxAABAHVXt73kxxqi8vFyxsbFyu93Kysryrjt58qQ2btyonj17VvcwAAAAkqp45mXWrFkaPHiwoqOjVVJSorffflvZ2dlau3atHA6HUlNTlZ6erri4OMXFxSk9PV2hoaEaOXJkTc0PAADqmCrFy08//aQxY8boxx9/VEREhBISErR27VolJydLkqZPn66ysjI99NBDOnr0qLp3767169crLCysRoYHAAB1T5Xi5dVXX73geofDobS0NKWlpVVnJgAAgPPit40AAIBViBcAAGAV4gUAAFiFeAEAAFYhXgAAgFWIFwAAYBXiBQAAWIV4AQAAViFeAACAVYgXAABgFeIFAABYhXgBAABWIV4AAIBViBcAAGCVkEAPgCvD/v37VVJSUu395OXl+fyvP4SFhSkuLs5v+wPgi9c/LjeHMcYEeojfKy4uVkREhIqKihQeHh7ocXAR9u/frzZt2gR6jAv6+uuv+QMMqAG8/uEvVfn7nzMvqLaKf3EtW7ZM8fHx1dpXWVmZCgoKFBMTI5fLVe3Z8vLyNHr0aL/8qxBAZbz+EQjEC/wmPj5eiYmJ1d5Pr169/DANgMuJ1z8uJy7YBQAAViFeAACAVYgXAABgFeIFAABYhXgBAABWIV4AAIBViBcAAGAV4gUAAFiFeAEAAFYhXgAAgFWIFwAAYBXiBQAAWIV4AQAAViFeAACAVYgXAABgFeIFAABYhXgBAABWIV4AAIBViBcAAGAV4gUAAFiFeAEAAFYhXgAAgFWIFwAAYBXiBQAAWIV4AQAAViFeAACAVYgXAABgFeIFAABYhXgBAABWIV4AAIBViBcAAGAV4gUAAFiFeAEAAFYhXgAAgFWIFwAAYBXiBQAAWIV4AQAAViFeAACAVYgXAABgFeIFAABYhXgBAABWIV4AAIBViBcAAGCVKsVLRkaGbrrpJoWFhal58+YaPny49u3b57ONMUZpaWmKioqSy+VSUlKSdu/e7dehAQBA3VWleNm4caNSUlK0ZcsWZWVl6fTp0xo4cKBKS0u92yxYsEALFy7UkiVLlJOTI7fbreTkZJWUlPh9eAAAUPeEVGXjtWvX+tx//fXX1bx5c+Xm5qpv374yxmjx4sWaPXu2RowYIUnKzMxUZGSkli9frgcffNB/kwMAgDqpWte8FBUVSZKuvvpqSVJ+fr48Ho8GDhzo3cbpdKpfv37avHlzdQ4FAAAgqYpnXn7PGKNp06apd+/e6tChgyTJ4/FIkiIjI322jYyM1Pfff3/O/ZSXl6u8vNx7v7i4+FJHQoCUlZVJkvLy8gI8SWUVM1XMCMC/eP0jEC45XiZPnqydO3fq888/r7TO4XD43DfGVFpWISMjQ0888cSljoFaoKCgQJI0evTowA5yAQUFBerVq1egxwCuOLz+EQiXFC9TpkzR6tWrtWnTJrVq1cq73O12S/rtDEyLFi28ywsLCyudjakwc+ZMTZs2zXu/uLhY0dHRlzIWAiQmJkaStGzZMsXHxwd2mD/Iy8vT6NGjvTMC8C9e/wiEKsWLMUZTpkzRBx98oOzsbMXGxvqsj42NldvtVlZWljp37ixJOnnypDZu3Kj58+efc59Op1NOp/MSx0dt4HK5JEnx8fFKTEwM8DTnVjEjAP/i9Y9AqFK8pKSkaPny5Vq1apXCwsK817hERETI5XLJ4XAoNTVV6enpiouLU1xcnNLT0xUaGqqRI0fWyBMAAAB1S5XiZenSpZKkpKQkn+Wvv/66xo0bJ0maPn26ysrK9NBDD+no0aPq3r271q9fr7CwML8MDAAA6rYqv230ZxwOh9LS0pSWlnapMwEAAJwXv20EAACsQrwAAACrEC8AAMAqxAsAALAK8QIAAKxCvAAAAKsQLwAAwCrECwAAsArxAgAArEK8AAAAqxAvAADAKsQLAACwCvECAACsQrwAAACrEC8AAMAqxAsAALAK8QIAAKxCvAAAAKsQLwAAwCrECwAAsArxAgAArEK8AAAAqxAvAADAKsQLAACwCvECAACsQrwAAACrEC8AAMAqxAsAALAK8QIAAKxCvAAAAKsQLwAAwCrECwAAsArxAgAArEK8AAAAqxAvAADAKsQLAACwSkigB4D9jh8/Lknatm1btfdVVlamgoICxcTEyOVyVXt/eXl51d4HgPPj9Y9AIF5QbXv37pUkPfDAAwGe5PzCwsICPQJwReL1j0AgXlBtw4cPlyS1a9dOoaGh1dpXXl6eRo8erWXLlik+Pt4P0/32B1dcXJxf9gXAF69/BALxgmpr2rSpxo8f79d9xsfHKzEx0a/7BOB/vP4RCFywCwAArEK8AAAAqxAvAADAKsQLAACwCvECAACsQrwAAACrEC8AAMAqxAsAALAK8QIAAKxCvAAAAKsQLwAAwCrECwAAsArxAgAArEK8AAAAqxAvAADAKsQLAACwCvECAACsQrwAAACrEC8AAMAqxAsAALAK8QIAAKxCvAAAAKtUOV42bdqk22+/XVFRUXI4HFq5cqXPemOM0tLSFBUVJZfLpaSkJO3evdtf8wIAgDquyvFSWlqqjh07asmSJedcv2DBAi1cuFBLlixRTk6O3G63kpOTVVJSUu1hAQAAQqr6gMGDB2vw4MHnXGeM0eLFizV79myNGDFCkpSZmanIyEgtX75cDz74YPWmBQAAdZ5fr3nJz8+Xx+PRwIEDvcucTqf69eunzZs3n/Mx5eXlKi4u9rkBAACcj1/jxePxSJIiIyN9lkdGRnrX/VFGRoYiIiK8t+joaH+OBAAArjA18mkjh8Phc98YU2lZhZkzZ6qoqMh7O3jwYE2MBAAArhBVvublQtxut6TfzsC0aNHCu7ywsLDS2ZgKTqdTTqfTn2MAAIArmF/PvMTGxsrtdisrK8u77OTJk9q4caN69uzpz0MBAIA6qspnXo4dO6ZvvvnGez8/P187duzQ1VdfrWuuuUapqalKT09XXFyc4uLilJ6ertDQUI0cOdKvgwMAgLqpyvGydetW9e/f33t/2rRpkqSxY8fqjTfe0PTp01VWVqaHHnpIR48eVffu3bV+/XqFhYX5b2oAAFBnVTlekpKSZIw573qHw6G0tDSlpaVVZy4AAIBz4reNAACAVYgXAABgFeIFAABYhXgBAABWIV4AAIBViBcAAGAV4gUAAFiFeAEAAFYhXgAAgFWIFwAAYBXiBQAAWIV4AQAAViFeAACAVYgXAABgFeIFAABYhXgBAABWIV4AAIBViBcAAGAV4gUAAFiFeAEAAFYhXgAAgFWIFwAAYBXiBQAAWIV4AQAAViFeAACAVYgXAABgFeIFAABYhXgBAABWIV4AAIBViBcAAGAV4gUAAFiFeAEAAFYhXgAAgFWIFwAAYBXiBQAAWIV4AQAAViFeAACAVYgXAABgFeIFAABYhXgBAABWIV4AAIBViBcAAGAV4gUAAFiFeAEAAFYhXgAAgFWIFwAAYBXiBQAAWIV4AQAAViFeAACAVYgXAABgFeIFAABYJSTQAwAArnzHjx/X3r17L7hNXl6ez//+mXbt2ik0NLTas8E+xAsAoMbt3btXXbp0uahtR48efVHb5ebmKjExsTpjwVLECwCgxrVr1065ubkX3KasrEwFBQWKiYmRy+W6qH2ibiJeAAA1LjQ09KLOkvTq1esyTAPbccEuAACwCvECAACsQrwAAACrcM0LLhs+KgkA8AfiBZcNH5UEAPgD8YLLho9KAgD8wWGMMYEe4veKi4sVERGhoqIihYeHB3ocAABwGVTl7/8au2D3hRdeUGxsrBo0aKAuXbros88+q6lDAQCAOqRG4uWdd95RamqqZs+ere3bt6tPnz4aPHiwDhw4UBOHAwAAdUiNvG3UvXt3JSYmaunSpd5l8fHxGj58uDIyMi74WN42AgCg7gno20YnT55Ubm6uBg4c6LN84MCB2rx5c6Xty8vLVVxc7HMDAAA4H7/Hyy+//KIzZ84oMjLSZ3lkZKQ8Hk+l7TMyMhQREeG9RUdH+3skAABwBamxC3YdDofPfWNMpWWSNHPmTBUVFXlvBw8erKmRAADAFcDv3/PStGlTBQcHVzrLUlhYWOlsjCQ5nU45nU5/jwEAAK5Qfj/zUr9+fXXp0kVZWVk+y7OystSzZ09/Hw4AANQxNfINu9OmTdOYMWPUtWtX9ejRQy+99JIOHDigiRMn1sThAABAHVIj8fK3v/1Nhw8f1pNPPqkff/xRHTp00Jo1a3TttdfWxOEAAEAdws8DAACAgKsVPw8AAABQE2rdr0pXnAjiy+oAAKg7Kv7ev5g3hGpdvJSUlEgSX1YHAEAdVFJSooiIiAtuU+uueTl79qx++OEHhYWFnfNL7XBlKy4uVnR0tA4ePMg1T0Adw+u/bjPGqKSkRFFRUQoKuvBVLbXuzEtQUJBatWoV6DEQYOHh4fzhBdRRvP7rrj8741KBC3YBAIBViBcAAGAV4gW1itPp1Ny5c/m9K6AO4vWPi1XrLtgFAAC4EM68AAAAqxAvAADAKsQLAACwCvECAACsQrwAAACrEC8AAMAqxAsui6SkJE2dOlXTp0/X1VdfLbfbrbS0NO/6oqIiTZgwQc2bN1d4eLgGDBigr776ymcf8+bNU/PmzRUWFqbx48frscceU6dOnS7vEwFQZUlJSZo8ebImT56sq666Sk2aNNGcOXO8vx589OhR3XvvvWrcuLFCQ0M1ePBg7d+/3/v477//XrfffrsaN26shg0bqn379lqzZk2gng5qAeIFl01mZqYaNmyoL7/8UgsWLNCTTz6prKwsGWM0dOhQeTwerVmzRrm5uUpMTNQtt9yiI0eOSJLefPNNPfXUU5o/f75yc3N1zTXXaOnSpQF+RgAuVmZmpkJCQvTll1/q2Wef1aJFi/TKK69IksaNG6etW7dq9erV+uKLL2SM0ZAhQ3Tq1ClJUkpKisrLy7Vp0ybt2rVL8+fPV6NGjQL5dBBgfEkdLoukpCSdOXNGn332mXdZt27dNGDAAA0cOFB33nmnCgsLfb5Zs3Xr1po+fbomTJigm2++WV27dtWSJUu863v37q1jx45px44dl/OpAKiipKQkFRYWavfu3XI4HJKkxx57TKtXr9aqVavUpk0b/fe//1XPnj0lSYcPH1Z0dLQyMzN11113KSEhQX/5y180d+7cQD4N1CKcecFlk5CQ4HO/RYsWKiwsVG5uro4dO6YmTZqoUaNG3lt+fr6+/fZbSdK+ffvUrVs3n8f/8T6A2uvmm2/2hosk9ejRQ/v379eePXsUEhKi7t27e9c1adJEbdu2VV5eniRp6tSpmjdvnnr16qW5c+dq586dl31+1C4hgR4AdUe9evV87jscDp09e1Znz55VixYtlJ2dXekxV111lc/2v8dJQ+DKZYzxvubHjx+vQYMG6aOPPtL69euVkZGhp59+WlOmTAnwlAgUzrwg4BITE+XxeBQSEqLWrVv73Jo2bSpJatu2rf73v//5PG7r1q2BGBfAJdiyZUul+3Fxcbrhhht0+vRpffnll951hw8f1tdff634+HjvsujoaE2cOFErVqzQ3//+d7388suXbXbUPsQLAu7WW29Vjx49NHz4cK1bt04FBQXavHmz5syZ4w2UKVOm6NVXX1VmZqb279+vefPmaefOnZXOxgConQ4ePKhp06Zp3759euutt/Tcc8/p4YcfVlxcnO644w498MAD+vzzz/XVV19p9OjRatmype644w5JUmpqqtatW6f8/Hxt27ZNn376qU/YoO7hbSMEnMPh0Jo1azR79mzdd999+vnnn+V2u9W3b19FRkZKkkaNGqXvvvtOjz76qE6cOKG7775b48aNq3Q2BkDtdO+996qsrEzdunVTcHCwpkyZogkTJkiSXn/9dT388MMaNmyYTp48qb59+2rNmjXet5rPnDmjlJQUHTp0SOHh4brtttu0aNGiQD4dBBifNoK1kpOT5Xa79a9//SvQowC4gKSkJHXq1EmLFy8O9Ci4QnDmBVY4fvy4XnzxRQ0aNEjBwcF666239PHHHysrKyvQowEALjPiBVaoeGtp3rx5Ki8vV9u2bfX+++/r1ltvDfRoAIDLjLeNAACAVfi0EQAAsArxAgAArEK8AAAAqxAvAADAKsQLAACwCvECAACsQrwAAACrEC8AAMAqxAsAALDK/wNrfLjPyNafoQAAAABJRU5ErkJggg=="},"metadata":{}}]},{"cell_type":"code","source":"raw_datasets.reset_format()","metadata":{"execution":{"iopub.status.busy":"2023-09-03T08:41:31.031829Z","iopub.execute_input":"2023-09-03T08:41:31.032215Z","iopub.status.idle":"2023-09-03T08:41:31.038460Z","shell.execute_reply.started":"2023-09-03T08:41:31.032163Z","shell.execute_reply":"2023-09-03T08:41:31.037081Z"},"trusted":true},"execution_count":11,"outputs":[]},{"cell_type":"markdown","source":"# <b><div style='padding:15px;background-color:#850E35;color:white;border-radius:2px;font-size:110%;text-align: center'>5. Preprocess Data</div></b>","metadata":{}},{"cell_type":"code","source":"from transformers import AutoTokenizer\n\ncheckpoint = \"distilbert-base-uncased\"\ntokenizer = AutoTokenizer.from_pretrained(checkpoint)","metadata":{"execution":{"iopub.status.busy":"2023-09-03T08:41:34.395276Z","iopub.execute_input":"2023-09-03T08:41:34.395749Z","iopub.status.idle":"2023-09-03T08:41:38.375226Z","shell.execute_reply.started":"2023-09-03T08:41:34.395715Z","shell.execute_reply":"2023-09-03T08:41:38.374237Z"},"trusted":true},"execution_count":12,"outputs":[{"output_type":"display_data","data":{"text/plain":"Downloading (…)okenizer_config.json:   0%|          | 0.00/28.0 [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"590c21920cab460db43c36a84a4ac440"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Downloading (…)lve/main/config.json:   0%|          | 0.00/483 [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"5a8fd0a29b8c42c3b3176182eef934ee"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Downloading (…)solve/main/vocab.txt:   0%|          | 0.00/232k [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"61935b4dd2c24ee6a9742318976b2c20"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Downloading (…)/main/tokenizer.json:   0%|          | 0.00/466k [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"0c4df997db3441448da019eef7eae34c"}},"metadata":{}}]},{"cell_type":"code","source":"def tokenize_function(examples):\n    return tokenizer(examples[\"text\"], truncation=True)\n\ntokenized_datasets = raw_datasets.map(tokenize_function, batched=True)","metadata":{"execution":{"iopub.status.busy":"2023-09-03T08:41:49.408932Z","iopub.execute_input":"2023-09-03T08:41:49.409590Z","iopub.status.idle":"2023-09-03T08:41:50.671360Z","shell.execute_reply.started":"2023-09-03T08:41:49.409554Z","shell.execute_reply":"2023-09-03T08:41:50.670366Z"},"trusted":true},"execution_count":13,"outputs":[{"output_type":"display_data","data":{"text/plain":"  0%|          | 0/9 [00:00<?, ?ba/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"2335bb2edf8249588d47714e09c70a4a"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"  0%|          | 0/2 [00:00<?, ?ba/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"e9cd155ca9414296896d6b6d794d5155"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"  0%|          | 0/2 [00:00<?, ?ba/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"6ce29159712246d7bf6461b9aacc4cce"}},"metadata":{}}]},{"cell_type":"code","source":"tokenized_datasets","metadata":{"execution":{"iopub.status.busy":"2023-09-03T08:41:53.175473Z","iopub.execute_input":"2023-09-03T08:41:53.175837Z","iopub.status.idle":"2023-09-03T08:41:53.182295Z","shell.execute_reply.started":"2023-09-03T08:41:53.175809Z","shell.execute_reply":"2023-09-03T08:41:53.181282Z"},"trusted":true},"execution_count":14,"outputs":[{"execution_count":14,"output_type":"execute_result","data":{"text/plain":"DatasetDict({\n    train: Dataset({\n        features: ['text', 'label', 'input_ids', 'attention_mask'],\n        num_rows: 8530\n    })\n    validation: Dataset({\n        features: ['text', 'label', 'input_ids', 'attention_mask'],\n        num_rows: 1066\n    })\n    test: Dataset({\n        features: ['text', 'label', 'input_ids', 'attention_mask'],\n        num_rows: 1066\n    })\n})"},"metadata":{}}]},{"cell_type":"code","source":"from transformers import DataCollatorWithPadding\n\ndata_collator = DataCollatorWithPadding(tokenizer=tokenizer)","metadata":{"execution":{"iopub.status.busy":"2023-09-03T08:41:55.899779Z","iopub.execute_input":"2023-09-03T08:41:55.900141Z","iopub.status.idle":"2023-09-03T08:42:14.657183Z","shell.execute_reply.started":"2023-09-03T08:41:55.900111Z","shell.execute_reply":"2023-09-03T08:42:14.656055Z"},"trusted":true},"execution_count":15,"outputs":[{"name":"stderr","text":"/opt/conda/lib/python3.10/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.23.5\n  warnings.warn(f\"A NumPy version >={np_minversion} and <{np_maxversion}\"\n/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/__init__.py:98: UserWarning: unable to load libtensorflow_io_plugins.so: unable to open file: libtensorflow_io_plugins.so, from paths: ['/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so']\ncaused by: ['/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so: undefined symbol: _ZN3tsl6StatusC1EN10tensorflow5error4CodeESt17basic_string_viewIcSt11char_traitsIcEENS_14SourceLocationE']\n  warnings.warn(f\"unable to load libtensorflow_io_plugins.so: {e}\")\n/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/__init__.py:104: UserWarning: file system plugins are not loaded: unable to open file: libtensorflow_io.so, from paths: ['/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so']\ncaused by: ['/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so: undefined symbol: _ZTVN10tensorflow13GcsFileSystemE']\n  warnings.warn(f\"file system plugins are not loaded: {e}\")\n","output_type":"stream"}]},{"cell_type":"markdown","source":"# <b><div style='padding:15px;background-color:#850E35;color:white;border-radius:2px;font-size:110%;text-align: center'>6. Setup Evaluation Function</div></b>","metadata":{}},{"cell_type":"code","source":"from sklearn.metrics import accuracy_score, precision_recall_fscore_support\n\ndef get_example(index):\n    return tokenized_datasets[\"test\"][index][\"text\"]\n\n\ndef compute_metrics(pred):\n    experiment = comet_ml.get_global_experiment()\n\n    labels = pred.label_ids\n    preds = pred.predictions.argmax(-1)\n    precision, recall, f1, _ = precision_recall_fscore_support(\n        labels, preds, average=\"macro\"\n    )\n    acc = accuracy_score(labels, preds)\n\n    if experiment:\n        epoch = int(experiment.curr_epoch) if experiment.curr_epoch is not None else 0\n        experiment.set_epoch(epoch)\n        experiment.log_confusion_matrix(\n            y_true=labels,\n            y_predicted=preds,\n            file_name=f\"confusion-matrix-epoch-{epoch}.json\",\n            labels=[\"negative\", \"postive\"],\n            index_to_example_function=get_example,\n        )\n\n    return {\"accuracy\": acc, \"f1\": f1, \"precision\": precision, \"recall\": recall}","metadata":{"execution":{"iopub.status.busy":"2023-09-03T08:42:14.659529Z","iopub.execute_input":"2023-09-03T08:42:14.660520Z","iopub.status.idle":"2023-09-03T08:42:14.671252Z","shell.execute_reply.started":"2023-09-03T08:42:14.660478Z","shell.execute_reply":"2023-09-03T08:42:14.669384Z"},"trusted":true},"execution_count":16,"outputs":[]},{"cell_type":"markdown","source":"# <b><div style='padding:15px;background-color:#850E35;color:white;border-radius:2px;font-size:110%;text-align: center'>7. Setup Transformer Model</div></b>","metadata":{}},{"cell_type":"code","source":"from transformers import AutoModelForSequenceClassification\n\nid2label = {0: \"NEGATIVE\", 1: \"POSITIVE\"}\nlabel2id = {\"NEGATIVE\": 0, \"POSITIVE\": 1}\n\nmodel = AutoModelForSequenceClassification.from_pretrained(checkpoint, num_labels=2, id2label=id2label, label2id=label2id)","metadata":{"execution":{"iopub.status.busy":"2023-09-03T08:42:21.338888Z","iopub.execute_input":"2023-09-03T08:42:21.339300Z","iopub.status.idle":"2023-09-03T08:42:25.272518Z","shell.execute_reply.started":"2023-09-03T08:42:21.339268Z","shell.execute_reply":"2023-09-03T08:42:25.271614Z"},"trusted":true},"execution_count":17,"outputs":[{"output_type":"display_data","data":{"text/plain":"Downloading model.safetensors:   0%|          | 0.00/268M [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"407c2e6a1af84280b15da799a02ec38c"}},"metadata":{}},{"name":"stderr","text":"Some weights of the model checkpoint at distilbert-base-uncased were not used when initializing DistilBertForSequenceClassification: ['vocab_projector.bias', 'vocab_layer_norm.bias', 'vocab_layer_norm.weight', 'vocab_transform.bias', 'vocab_transform.weight']\n- This IS expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n- This IS NOT expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\nSome weights of DistilBertForSequenceClassification were not initialized from the model checkpoint at distilbert-base-uncased and are newly initialized: ['pre_classifier.bias', 'classifier.bias', 'pre_classifier.weight', 'classifier.weight']\nYou should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n","output_type":"stream"}]},{"cell_type":"markdown","source":"# <b><div style='padding:15px;background-color:#850E35;color:white;border-radius:2px;font-size:110%;text-align: center'>8. Run Training</div></b>","metadata":{}},{"cell_type":"code","source":"from transformers import TrainingArguments, Trainer\n\n%env COMET_MODE=ONLINE\n%env COMET_LOG_ASSETS=TRUE\n\ntraining_args = TrainingArguments(\n    output_dir=\"my_distilbert_model\",\n    learning_rate=2e-5,\n    per_device_train_batch_size=16,\n    per_device_eval_batch_size=16,\n    num_train_epochs=3,\n    weight_decay=0.01,\n    evaluation_strategy=\"epoch\",\n    save_strategy=\"epoch\",\n    load_best_model_at_end=True,\n    push_to_hub=True,\n    report_to=[\"comet_ml\"],\n)\n\ntrainer = Trainer(\n    model=model,\n    args=training_args,\n    train_dataset=tokenized_datasets[\"train\"],\n    eval_dataset=tokenized_datasets[\"test\"],\n    compute_metrics=compute_metrics,\n    data_collator=data_collator,\n)\n\ntrainer.train()","metadata":{"execution":{"iopub.status.busy":"2023-09-03T08:42:30.425613Z","iopub.execute_input":"2023-09-03T08:42:30.426011Z","iopub.status.idle":"2023-09-03T08:49:17.020923Z","shell.execute_reply.started":"2023-09-03T08:42:30.425975Z","shell.execute_reply":"2023-09-03T08:49:17.019759Z"},"trusted":true},"execution_count":18,"outputs":[{"name":"stdout","text":"env: COMET_MODE=ONLINE\nenv: COMET_LOG_ASSETS=TRUE\n","output_type":"stream"},{"name":"stderr","text":"Cloning https://huggingface.co/Tirendaz/my_distilbert_model into local empty directory.\n","output_type":"stream"},{"output_type":"display_data","data":{"text/plain":"Download file pytorch_model.bin:   0%|          | 16.0k/255M [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"e220a52efbf641f691c136cdf95d86e5"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Download file training_args.bin: 100%|##########| 3.87k/3.87k [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"237ea5d62f1149bcb61f0df7bd60ff6a"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Clean file training_args.bin:  26%|##5       | 1.00k/3.87k [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"839cf89f86264d3aa298c94ff7ae5163"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Clean file pytorch_model.bin:   0%|          | 1.00k/255M [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"7d50e54b10a248da8181c9503061e320"}},"metadata":{}},{"name":"stderr","text":"/opt/conda/lib/python3.10/site-packages/transformers/optimization.py:411: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n  warnings.warn(\n\u001b[1;38;5;214mCOMET WARNING:\u001b[0m As you are running in a Jupyter environment, you will need to call `experiment.end()` when finished to ensure all metrics and code are logged before exiting.\n\u001b[1;38;5;214mCOMET WARNING:\u001b[0m You are trying to log string value as a metric. This is not recommended.\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m Couldn't find a Git repository in '/kaggle/working' nor in any parent directory. Set `COMET_GIT_DIRECTORY` if your Git Repository is elsewhere.\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m Experiment is live on comet.com https://www.comet.com/tirendaz-academy/text-classification-with-transfomers/0ecea65ea5384a24a9b9d5d9e98d6799\n\nYou're using a DistilBertTokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.\n/opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.\n  warnings.warn('Was asked to gather along dimension 0, but all '\n","output_type":"stream"},{"output_type":"display_data","data":{"text/plain":"<IPython.core.display.HTML object>","text/html":"\n    <div>\n      \n      <progress value='801' max='801' style='width:300px; height:20px; vertical-align: middle;'></progress>\n      [801/801 04:35, Epoch 3/3]\n    </div>\n    <table border=\"1\" class=\"dataframe\">\n  <thead>\n <tr style=\"text-align: left;\">\n      <th>Epoch</th>\n      <th>Training Loss</th>\n      <th>Validation Loss</th>\n      <th>Accuracy</th>\n      <th>F1</th>\n      <th>Precision</th>\n      <th>Recall</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <td>1</td>\n      <td>No log</td>\n      <td>0.399037</td>\n      <td>0.824578</td>\n      <td>0.824265</td>\n      <td>0.826908</td>\n      <td>0.824578</td>\n    </tr>\n    <tr>\n      <td>2</td>\n      <td>0.353400</td>\n      <td>0.395053</td>\n      <td>0.845216</td>\n      <td>0.845216</td>\n      <td>0.845217</td>\n      <td>0.845216</td>\n    </tr>\n    <tr>\n      <td>3</td>\n      <td>0.353400</td>\n      <td>0.441819</td>\n      <td>0.843340</td>\n      <td>0.843336</td>\n      <td>0.843370</td>\n      <td>0.843340</td>\n    </tr>\n  </tbody>\n</table><p>"},"metadata":{}},{"name":"stderr","text":"/opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.\n  warnings.warn('Was asked to gather along dimension 0, but all '\n/opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.\n  warnings.warn('Was asked to gather along dimension 0, but all '\n/opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.\n  warnings.warn('Was asked to gather along dimension 0, but all '\n/opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.\n  warnings.warn('Was asked to gather along dimension 0, but all '\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m ---------------------------------------------------------------------------------------\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m Comet.ml Experiment Summary\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m ---------------------------------------------------------------------------------------\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m   Data:\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     display_summary_level : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     url                   : https://www.comet.com/tirendaz-academy/text-classification-with-transfomers/0ecea65ea5384a24a9b9d5d9e98d6799\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m   Metrics [count] (min, max):\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     epoch [5]                   : (1.0, 3.0)\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     eval_accuracy [3]           : (0.8245778611632271, 0.8452157598499062)\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     eval_f1 [3]                 : (0.8242646987519517, 0.8452156236387396)\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     eval_loss [3]               : (0.3950529992580414, 0.4418192505836487)\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     eval_precision [3]          : (0.8269080775994101, 0.8452169750218241)\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     eval_recall [3]             : (0.8245778611632271, 0.8452157598499062)\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     eval_runtime [3]            : (1.7073, 2.1657)\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     eval_samples_per_second [3] : (492.221, 624.386)\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     eval_steps_per_second [3]   : (15.699, 19.915)\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     learning_rate               : 7.515605493133583e-06\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     loss [82]                   : (0.044084347784519196, 0.6792322397232056)\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     total_flos                  : 347538569389536.0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     train_loss                  : 0.2934642891758837\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     train_runtime               : 206.3449\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     train_samples_per_second    : 124.016\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     train_steps_per_second      : 3.882\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m   Parameters:\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/_n_gpu                             : 2\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/_no_sync_in_gradient_accumulation  : True\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/_setup_devices                     : cuda:0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/adafactor                          : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/adam_beta1                         : 0.9\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/adam_beta2                         : 0.999\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/adam_epsilon                       : 1e-08\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/auto_find_batch_size               : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/bf16                               : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/bf16_full_eval                     : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/data_seed                          : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/dataloader_drop_last               : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/dataloader_num_workers             : 0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/dataloader_pin_memory              : True\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/ddp_backend                        : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/ddp_bucket_cap_mb                  : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/ddp_find_unused_parameters         : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/ddp_timeout                        : 1800\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/ddp_timeout_delta                  : 0:30:00\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/debug                              : []\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/deepspeed                          : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/deepspeed_plugin                   : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/default_optim                      : adamw_hf\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/device                             : cuda:0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/disable_tqdm                       : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/distributed_state                  : Distributed environment: NO\nNum processes: 1\nProcess index: 0\nLocal process index: 0\nDevice: cuda\n\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/do_eval                            : True\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/do_predict                         : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/do_train                           : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/eval_accumulation_steps            : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/eval_batch_size                    : 32\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/eval_delay                         : 0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/eval_steps                         : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/evaluation_strategy                : IntervalStrategy.EPOCH\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/fp16                               : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/fp16_backend                       : auto\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/fp16_full_eval                     : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/fp16_opt_level                     : O1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/framework                          : pt\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/fsdp                               : []\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/fsdp_config                        : {'fsdp_min_num_params': 0, 'xla': False, 'xla_fsdp_grad_ckpt': False}\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/fsdp_min_num_params                : 0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/fsdp_transformer_layer_cls_to_wrap : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/full_determinism                   : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/gradient_accumulation_steps        : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/gradient_checkpointing             : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/greater_is_better                  : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/group_by_length                    : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/half_precision_backend             : auto\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/hub_model_id                       : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/hub_private_repo                   : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/hub_strategy                       : HubStrategy.EVERY_SAVE\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/hub_token                          : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/ignore_data_skip                   : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/include_inputs_for_metrics         : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/jit_mode_eval                      : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/label_names                        : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/label_smoothing_factor             : 0.0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/learning_rate                      : 2e-05\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/length_column_name                 : length\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/load_best_model_at_end             : True\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/local_process_index                : 0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/local_rank                         : 0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/log_level                          : passive\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/log_level_replica                  : warning\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/log_on_each_node                   : True\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/logging_dir                        : my_distilbert_model/runs/Sep03_08-42-30_fb74967525ab\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/logging_first_step                 : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/logging_nan_inf_filter             : True\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/logging_steps                      : 500\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/logging_strategy                   : IntervalStrategy.STEPS\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/lr_scheduler_type                  : SchedulerType.LINEAR\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/max_grad_norm                      : 1.0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/max_steps                          : -1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/metric_for_best_model              : loss\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/mp_parameters                      : \n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/n_gpu                              : 2\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/no_cuda                            : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/num_train_epochs                   : 3\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/optim                              : OptimizerNames.ADAMW_HF\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/optim_args                         : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/output_dir                         : my_distilbert_model\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/overwrite_output_dir               : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/parallel_mode                      : ParallelMode.NOT_DISTRIBUTED\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/past_index                         : -1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/per_device_eval_batch_size         : 16\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/per_device_train_batch_size        : 16\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/per_gpu_eval_batch_size            : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/per_gpu_train_batch_size           : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/place_model_on_device              : True\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/prediction_loss_only               : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/process_index                      : 0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/push_to_hub                        : True\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/push_to_hub_model_id               : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/push_to_hub_organization           : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/push_to_hub_token                  : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/ray_scope                          : last\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/remove_unused_columns              : True\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/report_to                          : ['comet_ml']\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/resume_from_checkpoint             : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/run_name                           : my_distilbert_model\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/save_on_each_node                  : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/save_safetensors                   : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/save_steps                         : 500\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/save_strategy                      : IntervalStrategy.EPOCH\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/save_total_limit                   : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/seed                               : 42\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/sharded_ddp                        : []\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/should_log                         : True\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/should_save                        : True\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/skip_memory_metrics                : True\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/tf32                               : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/torch_compile                      : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/torch_compile_backend              : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/torch_compile_mode                 : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/torchdynamo                        : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/tpu_metrics_debug                  : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/tpu_num_cores                      : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/train_batch_size                   : 32\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/use_ipex                           : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/use_legacy_prediction_loop         : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/use_mps_device                     : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/warmup_ratio                       : 0.0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/warmup_steps                       : 0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/weight_decay                       : 0.01\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/world_size                         : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     args/xpu_backend                        : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/_auto_class                      : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/_commit_hash                     : 6cdc0aad91f5ae2e6712e91bc7b65d1cf5c05411\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/_name_or_path                    : distilbert-base-uncased\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/activation                       : gelu\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/add_cross_attention              : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/architectures                    : ['DistilBertForMaskedLM']\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/attention_dropout                : 0.1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/attribute_map                    : {'hidden_size': 'dim', 'num_attention_heads': 'n_heads', 'num_hidden_layers': 'n_layers'}\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/bad_words_ids                    : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/begin_suppress_tokens            : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/bos_token_id                     : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/chunk_size_feed_forward          : 0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/cross_attention_hidden_size      : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/decoder_start_token_id           : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/dim                              : 768\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/diversity_penalty                : 0.0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/do_sample                        : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/dropout                          : 0.1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/early_stopping                   : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/encoder_no_repeat_ngram_size     : 0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/eos_token_id                     : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/exponential_decay_length_penalty : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/finetuning_task                  : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/forced_bos_token_id              : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/forced_eos_token_id              : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/hidden_dim                       : 3072\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/id2label                         : {0: 'NEGATIVE', 1: 'POSITIVE'}\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/initializer_range                : 0.02\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/is_composition                   : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/is_decoder                       : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/is_encoder_decoder               : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/label2id                         : {'NEGATIVE': 0, 'POSITIVE': 1}\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/length_penalty                   : 1.0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/max_length                       : 20\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/max_position_embeddings          : 512\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/min_length                       : 0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/model_type                       : distilbert\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/n_heads                          : 12\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/n_layers                         : 6\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/name_or_path                     : distilbert-base-uncased\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/no_repeat_ngram_size             : 0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/num_beam_groups                  : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/num_beams                        : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/num_labels                       : 2\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/num_return_sequences             : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/output_attentions                : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/output_hidden_states             : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/output_scores                    : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/pad_token_id                     : 0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/prefix                           : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/problem_type                     : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/pruned_heads                     : {}\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/qa_dropout                       : 0.1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/remove_invalid_values            : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/repetition_penalty               : 1.0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/return_dict                      : True\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/return_dict_in_generate          : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/sep_token_id                     : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/seq_classif_dropout              : 0.2\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/sinusoidal_pos_embds             : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/suppress_tokens                  : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/task_specific_params             : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/temperature                      : 1.0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/tf_legacy_loss                   : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/tie_encoder_decoder              : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/tie_weights_                     : True\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/tie_word_embeddings              : True\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/tokenizer_class                  : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/top_k                            : 50\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/top_p                            : 1.0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/torch_dtype                      : None\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/torchscript                      : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/transformers_version             : 4.10.0.dev0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/typical_p                        : 1.0\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/use_bfloat16                     : False\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/use_return_dict                  : True\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     config/vocab_size                       : 30522\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m   Uploads:\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     asset                        : 134 (2.99 GB)\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     conda-environment-definition : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     conda-info                   : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     conda-specification          : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     confusion-matrix             : 3\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     environment details          : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     filename                     : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     installed packages           : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     model graph                  : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     notebook                     : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     os packages                  : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     source_code                  : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m \n\u001b[1;38;5;39mCOMET INFO:\u001b[0m Please wait for metadata to finish uploading (timeout is 3600 seconds)\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m Uploading 1 metrics, params and output messages\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m Please wait for assets to finish uploading (timeout is 10800 seconds)\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m Still uploading 6 file(s), remaining 1.36 GB/2.99 GB\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m Still uploading 5 asset(s), remaining 767.59 MB/2.99 GB, Throughput 41.53 MB/s, ETA ~19s\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m Still uploading 2 asset(s), remaining 132.76 MB/2.99 GB, Throughput 42.28 MB/s, ETA ~4s\n","output_type":"stream"},{"execution_count":18,"output_type":"execute_result","data":{"text/plain":"TrainOutput(global_step=801, training_loss=0.2934642891758837, metrics={'train_runtime': 206.3449, 'train_samples_per_second': 124.016, 'train_steps_per_second': 3.882, 'total_flos': 347538569389536.0, 'train_loss': 0.2934642891758837, 'epoch': 3.0})"},"metadata":{}}]},{"cell_type":"code","source":"trainer.push_to_hub()","metadata":{"execution":{"iopub.status.busy":"2023-09-03T08:49:48.917982Z","iopub.execute_input":"2023-09-03T08:49:48.918437Z","iopub.status.idle":"2023-09-03T08:50:14.674689Z","shell.execute_reply.started":"2023-09-03T08:49:48.918405Z","shell.execute_reply":"2023-09-03T08:50:14.673610Z"},"trusted":true},"execution_count":19,"outputs":[{"name":"stderr","text":"To https://huggingface.co/Tirendaz/my_distilbert_model\n   4d3e39f..3dd2150  main -> main\n\nTo https://huggingface.co/Tirendaz/my_distilbert_model\n   3dd2150..1b96200  main -> main\n\n","output_type":"stream"},{"execution_count":19,"output_type":"execute_result","data":{"text/plain":"'https://huggingface.co/Tirendaz/my_distilbert_model/commit/3dd21507d9e0265c66941b22888ecc04efeed221'"},"metadata":{}}]},{"cell_type":"markdown","source":"# <b><div style='padding:15px;background-color:#850E35;color:white;border-radius:2px;font-size:110%;text-align: center'>9. Inference</div></b>","metadata":{}},{"cell_type":"code","source":"text = \"This is a great movie. It may be my favourite.\"","metadata":{"execution":{"iopub.status.busy":"2023-09-03T08:50:41.168601Z","iopub.execute_input":"2023-09-03T08:50:41.168970Z","iopub.status.idle":"2023-09-03T08:50:41.174440Z","shell.execute_reply.started":"2023-09-03T08:50:41.168940Z","shell.execute_reply":"2023-09-03T08:50:41.173224Z"},"trusted":true},"execution_count":20,"outputs":[]},{"cell_type":"code","source":"from transformers import pipeline\n\nclassifier = pipeline(\"sentiment-analysis\", model=\"Tirendaz/my_distilbert_model\")\nclassifier(text)","metadata":{"execution":{"iopub.status.busy":"2023-09-03T08:50:44.128679Z","iopub.execute_input":"2023-09-03T08:50:44.129084Z","iopub.status.idle":"2023-09-03T08:50:55.554901Z","shell.execute_reply.started":"2023-09-03T08:50:44.129052Z","shell.execute_reply":"2023-09-03T08:50:55.553242Z"},"trusted":true},"execution_count":21,"outputs":[{"output_type":"display_data","data":{"text/plain":"Downloading (…)lve/main/config.json:   0%|          | 0.00/735 [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"a360108351a74195a2e982383194b27a"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Downloading pytorch_model.bin:   0%|          | 0.00/268M [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"5184973e1a16481797570b3e1546cddd"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Downloading (…)okenizer_config.json:   0%|          | 0.00/360 [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"fdad744bb31f4be2a5adb5063e621811"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Downloading (…)/main/tokenizer.json:   0%|          | 0.00/711k [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"a41093e4b84e4aae9f92bf69e8947a02"}},"metadata":{}},{"name":"stderr","text":"Xformers is not installed correctly. If you want to use memory_efficient_attention to accelerate training use the following command to install Xformers\npip install xformers.\n","output_type":"stream"},{"execution_count":21,"output_type":"execute_result","data":{"text/plain":"[{'label': 'POSITIVE', 'score': 0.976166307926178}]"},"metadata":{}}]},{"cell_type":"markdown","source":"# <b><div style='padding:15px;background-color:#850E35;color:white;border-radius:2px;font-size:110%;text-align: center'>10. Deploy</div></b>","metadata":{}},{"cell_type":"code","source":"import gradio as gr\nfrom transformers import pipeline\n\nclassifier = pipeline(\"sentiment-analysis\", model=\"Tirendaz/my_distilbert_model\")\n\ndef text_classification(text):\n    result= classifier(text)\n    sentiment_label = result[0]['label']\n    sentiment_score = result[0]['score']\n    formatted_output = f\"This sentiment is {sentiment_label} with the probability {sentiment_score*100:.2f}%\"\n    return formatted_output\n\nexamples=[\"This is wonderful movie!\", \"The movie was really bad; I didn't like it.\"]\n\nio = gr.Interface(fn=text_classification, \n                         inputs= gr.Textbox(lines=2, label=\"Text\", placeholder=\"Enter title here...\"), \n                         outputs=gr.Textbox(lines=2, label=\"Text Classification Result\"),\n                         title=\"Text Classification\",\n                         description=\"Enter a text and see the text classification result!\",\n                         examples=examples)\n\nio.launch(inline=False, share=True)\n\n\nexperiment = comet_ml.Experiment()\nexperiment.add_tag(\"text-classifier\")\n\nio.integrate(comet_ml=experiment)","metadata":{"execution":{"iopub.status.busy":"2023-09-03T08:50:59.375741Z","iopub.execute_input":"2023-09-03T08:50:59.376125Z","iopub.status.idle":"2023-09-03T08:52:29.963210Z","shell.execute_reply.started":"2023-09-03T08:50:59.376094Z","shell.execute_reply":"2023-09-03T08:52:29.961696Z"},"trusted":true},"execution_count":22,"outputs":[{"name":"stdout","text":"Running on local URL:  http://127.0.0.1:7860\n","output_type":"stream"},{"name":"stderr","text":"\u001b[1;38;5;214mCOMET WARNING:\u001b[0m As you are running in a Jupyter environment, you will need to call `experiment.end()` when finished to ensure all metrics and code are logged before exiting.\n","output_type":"stream"},{"name":"stdout","text":"Running on public URL: https://cccb260e2784f81d50.gradio.live\n\nThis share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)\n","output_type":"stream"},{"name":"stderr","text":"\u001b[1;38;5;39mCOMET INFO:\u001b[0m Couldn't find a Git repository in '/kaggle/working' nor in any parent directory. Set `COMET_GIT_DIRECTORY` if your Git Repository is elsewhere.\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m Experiment is live on comet.com https://www.comet.com/tirendaz-academy/text-classification-with-transfomers/bb24396e75134b72816e64ef545e3de8\n\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m ---------------------------------------------------------------------------------------\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m Comet.ml Experiment Summary\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m ---------------------------------------------------------------------------------------\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m   Data:\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     display_summary_level : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     url                   : https://www.comet.com/tirendaz-academy/text-classification-with-transfomers/bb24396e75134b72816e64ef545e3de8\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m   Others:\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     Created from : Gradio\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m   Uploads:\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     conda-environment-definition : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     conda-info                   : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     conda-specification          : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     environment details          : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     filename                     : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     installed packages           : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     notebook                     : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     os packages                  : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     source_code                  : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m     text-sample                  : 1\n\u001b[1;38;5;39mCOMET INFO:\u001b[0m \n\u001b[1;38;5;39mCOMET INFO:\u001b[0m Uploading 1 metrics, params and output messages\n","output_type":"stream"}]},{"cell_type":"markdown","source":"## Resources:\n\n- [Text classification with HuggingFace](https://huggingface.co/docs/transformers/tasks/sequence_classification)\n- [Integrate with Hugging Face Transformers](https://www.comet.com/docs/v2/integrations/ml-frameworks/huggingface/)\n- [Natural Language Processing with Transformers](https://www.oreilly.com/library/view/natural-language-processing/9781098136789/)","metadata":{}},{"cell_type":"markdown","source":"Thanks for reading. Let's connect [YouTube](http://youtube.com/tirendazacademy) | [Medium](http://tirendazacademy.medium.com) | [Twitter](http://twitter.com/tirendazacademy) | [Linkedin](https://www.linkedin.com/in/tirendaz-academy)","metadata":{}}]}