tutorials/basics/modelling/logging.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logging\n",
    "\n",
    "To keep track of what is happening in your code, you can use the provided logger instance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-01-02T13:52:50.616170706Z",
     "start_time": "2024-01-02T13:52:49.868229894Z"
    },
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "qsprpred - WARNING - This is a warning.\n"
     ]
    }
   ],
   "source": [
    "from qsprpred.logs import logger\n",
    "\n",
    "logger.warning(\"This is a warning.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "You can see the logger displays its name, the level of the message, the current file with a line number and the message itself. This logger is used throughout the codebase to keep track of what is happening, but feel free to use it for your own code as well just like in the example above. You can set the message level to `debug`, `info`, `warning`, `error` or `critical`. The default level is `warning`, so `debug` and `info` messages will not be displayed. This is useful to know when you want to debug a piece of QSPRpred code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-01-02T13:52:52.796847160Z",
     "start_time": "2024-01-02T13:52:50.615005022Z"
    },
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "qsprpred - WARNING - Existing data set found, but also found a data frame in store. Refusing to overwrite data. If you want to overwrite data in store, set overwrite=True.\n",
      "qsprpred - INFO - Loading data from ../../tutorial_data/A2A_LIGANDS.tsv...\n",
      "qsprpred - WARNING - Existing data set found, but also found a data frame in store. Refusing to overwrite data. If you want to overwrite data in store, set overwrite=True.\n",
      "qsprpred - DEBUG - Adding target property 'pchembl_value_Mean' to dataset.\n",
      "qsprpred - DEBUG - Restoring training data...\n",
      "qsprpred - DEBUG - Training data restored.\n",
      "qsprpred - DEBUG - Training features shape: (3265, 0)\n",
      "qsprpred - DEBUG - Test set features shape: (817, 0)\n",
      "qsprpred - DEBUG - Training labels shape: (3265, 1)\n",
      "qsprpred - DEBUG - Test set labels shape: (817, 1)\n",
      "qsprpred - DEBUG - Training features indices: Index(['LoggingTutorialDataset_0000', 'LoggingTutorialDataset_0001',\n",
      "       'LoggingTutorialDataset_0002', 'LoggingTutorialDataset_0003',\n",
      "       'LoggingTutorialDataset_0004', 'LoggingTutorialDataset_0005',\n",
      "       'LoggingTutorialDataset_0007', 'LoggingTutorialDataset_0009',\n",
      "       'LoggingTutorialDataset_0010', 'LoggingTutorialDataset_0011',\n",
      "       ...\n",
      "       'LoggingTutorialDataset_4066', 'LoggingTutorialDataset_4067',\n",
      "       'LoggingTutorialDataset_4068', 'LoggingTutorialDataset_4070',\n",
      "       'LoggingTutorialDataset_4071', 'LoggingTutorialDataset_4072',\n",
      "       'LoggingTutorialDataset_4074', 'LoggingTutorialDataset_4075',\n",
      "       'LoggingTutorialDataset_4078', 'LoggingTutorialDataset_4080'],\n",
      "      dtype='object', name='QSPRID', length=3265)\n",
      "qsprpred - DEBUG - Test set features indices: Index(['LoggingTutorialDataset_0006', 'LoggingTutorialDataset_0008',\n",
      "       'LoggingTutorialDataset_0012', 'LoggingTutorialDataset_0014',\n",
      "       'LoggingTutorialDataset_0017', 'LoggingTutorialDataset_0023',\n",
      "       'LoggingTutorialDataset_0025', 'LoggingTutorialDataset_0026',\n",
      "       'LoggingTutorialDataset_0029', 'LoggingTutorialDataset_0030',\n",
      "       ...\n",
      "       'LoggingTutorialDataset_4035', 'LoggingTutorialDataset_4054',\n",
      "       'LoggingTutorialDataset_4056', 'LoggingTutorialDataset_4057',\n",
      "       'LoggingTutorialDataset_4069', 'LoggingTutorialDataset_4073',\n",
      "       'LoggingTutorialDataset_4076', 'LoggingTutorialDataset_4077',\n",
      "       'LoggingTutorialDataset_4079', 'LoggingTutorialDataset_4081'],\n",
      "      dtype='object', name='QSPRID', length=817)\n",
      "qsprpred - DEBUG - Training labels indices: Index(['LoggingTutorialDataset_0000', 'LoggingTutorialDataset_0001',\n",
      "       'LoggingTutorialDataset_0002', 'LoggingTutorialDataset_0003',\n",
      "       'LoggingTutorialDataset_0004', 'LoggingTutorialDataset_0005',\n",
      "       'LoggingTutorialDataset_0007', 'LoggingTutorialDataset_0009',\n",
      "       'LoggingTutorialDataset_0010', 'LoggingTutorialDataset_0011',\n",
      "       ...\n",
      "       'LoggingTutorialDataset_4066', 'LoggingTutorialDataset_4067',\n",
      "       'LoggingTutorialDataset_4068', 'LoggingTutorialDataset_4070',\n",
      "       'LoggingTutorialDataset_4071', 'LoggingTutorialDataset_4072',\n",
      "       'LoggingTutorialDataset_4074', 'LoggingTutorialDataset_4075',\n",
      "       'LoggingTutorialDataset_4078', 'LoggingTutorialDataset_4080'],\n",
      "      dtype='object', name='QSPRID', length=3265)\n",
      "qsprpred - DEBUG - Test set labels indices: Index(['LoggingTutorialDataset_0006', 'LoggingTutorialDataset_0008',\n",
      "       'LoggingTutorialDataset_0012', 'LoggingTutorialDataset_0014',\n",
      "       'LoggingTutorialDataset_0017', 'LoggingTutorialDataset_0023',\n",
      "       'LoggingTutorialDataset_0025', 'LoggingTutorialDataset_0026',\n",
      "       'LoggingTutorialDataset_0029', 'LoggingTutorialDataset_0030',\n",
      "       ...\n",
      "       'LoggingTutorialDataset_4035', 'LoggingTutorialDataset_4054',\n",
      "       'LoggingTutorialDataset_4056', 'LoggingTutorialDataset_4057',\n",
      "       'LoggingTutorialDataset_4069', 'LoggingTutorialDataset_4073',\n",
      "       'LoggingTutorialDataset_4076', 'LoggingTutorialDataset_4077',\n",
      "       'LoggingTutorialDataset_4079', 'LoggingTutorialDataset_4081'],\n",
      "      dtype='object', name='QSPRID', length=817)\n",
      "qsprpred - DEBUG - Restoring training data...\n",
      "qsprpred - DEBUG - Training data restored.\n",
      "qsprpred - DEBUG - Training features shape: (3265, 0)\n",
      "qsprpred - DEBUG - Test set features shape: (817, 0)\n",
      "qsprpred - DEBUG - Training labels shape: (3265, 1)\n",
      "qsprpred - DEBUG - Test set labels shape: (817, 1)\n",
      "qsprpred - DEBUG - Training features indices: Index(['LoggingTutorialDataset_0000', 'LoggingTutorialDataset_0001',\n",
      "       'LoggingTutorialDataset_0002', 'LoggingTutorialDataset_0003',\n",
      "       'LoggingTutorialDataset_0004', 'LoggingTutorialDataset_0005',\n",
      "       'LoggingTutorialDataset_0007', 'LoggingTutorialDataset_0009',\n",
      "       'LoggingTutorialDataset_0010', 'LoggingTutorialDataset_0011',\n",
      "       ...\n",
      "       'LoggingTutorialDataset_4066', 'LoggingTutorialDataset_4067',\n",
      "       'LoggingTutorialDataset_4068', 'LoggingTutorialDataset_4070',\n",
      "       'LoggingTutorialDataset_4071', 'LoggingTutorialDataset_4072',\n",
      "       'LoggingTutorialDataset_4074', 'LoggingTutorialDataset_4075',\n",
      "       'LoggingTutorialDataset_4078', 'LoggingTutorialDataset_4080'],\n",
      "      dtype='object', name='QSPRID', length=3265)\n",
      "qsprpred - DEBUG - Test set features indices: Index(['LoggingTutorialDataset_0006', 'LoggingTutorialDataset_0008',\n",
      "       'LoggingTutorialDataset_0012', 'LoggingTutorialDataset_0014',\n",
      "       'LoggingTutorialDataset_0017', 'LoggingTutorialDataset_0023',\n",
      "       'LoggingTutorialDataset_0025', 'LoggingTutorialDataset_0026',\n",
      "       'LoggingTutorialDataset_0029', 'LoggingTutorialDataset_0030',\n",
      "       ...\n",
      "       'LoggingTutorialDataset_4035', 'LoggingTutorialDataset_4054',\n",
      "       'LoggingTutorialDataset_4056', 'LoggingTutorialDataset_4057',\n",
      "       'LoggingTutorialDataset_4069', 'LoggingTutorialDataset_4073',\n",
      "       'LoggingTutorialDataset_4076', 'LoggingTutorialDataset_4077',\n",
      "       'LoggingTutorialDataset_4079', 'LoggingTutorialDataset_4081'],\n",
      "      dtype='object', name='QSPRID', length=817)\n",
      "qsprpred - DEBUG - Training labels indices: Index(['LoggingTutorialDataset_0000', 'LoggingTutorialDataset_0001',\n",
      "       'LoggingTutorialDataset_0002', 'LoggingTutorialDataset_0003',\n",
      "       'LoggingTutorialDataset_0004', 'LoggingTutorialDataset_0005',\n",
      "       'LoggingTutorialDataset_0007', 'LoggingTutorialDataset_0009',\n",
      "       'LoggingTutorialDataset_0010', 'LoggingTutorialDataset_0011',\n",
      "       ...\n",
      "       'LoggingTutorialDataset_4066', 'LoggingTutorialDataset_4067',\n",
      "       'LoggingTutorialDataset_4068', 'LoggingTutorialDataset_4070',\n",
      "       'LoggingTutorialDataset_4071', 'LoggingTutorialDataset_4072',\n",
      "       'LoggingTutorialDataset_4074', 'LoggingTutorialDataset_4075',\n",
      "       'LoggingTutorialDataset_4078', 'LoggingTutorialDataset_4080'],\n",
      "      dtype='object', name='QSPRID', length=3265)\n",
      "qsprpred - DEBUG - Test set labels indices: Index(['LoggingTutorialDataset_0006', 'LoggingTutorialDataset_0008',\n",
      "       'LoggingTutorialDataset_0012', 'LoggingTutorialDataset_0014',\n",
      "       'LoggingTutorialDataset_0017', 'LoggingTutorialDataset_0023',\n",
      "       'LoggingTutorialDataset_0025', 'LoggingTutorialDataset_0026',\n",
      "       'LoggingTutorialDataset_0029', 'LoggingTutorialDataset_0030',\n",
      "       ...\n",
      "       'LoggingTutorialDataset_4035', 'LoggingTutorialDataset_4054',\n",
      "       'LoggingTutorialDataset_4056', 'LoggingTutorialDataset_4057',\n",
      "       'LoggingTutorialDataset_4069', 'LoggingTutorialDataset_4073',\n",
      "       'LoggingTutorialDataset_4076', 'LoggingTutorialDataset_4077',\n",
      "       'LoggingTutorialDataset_4079', 'LoggingTutorialDataset_4081'],\n",
      "      dtype='object', name='QSPRID', length=817)\n",
      "qsprpred - DEBUG - Applying processor '<qsprpred.data.chem.standardization.CheckSmilesValid object at 0x7fe833d16ad0>' to 'LoggingTutorialDataset' in serial.\n",
      "qsprpred - DEBUG - Restoring training data...\n",
      "qsprpred - DEBUG - Training data restored.\n",
      "qsprpred - DEBUG - Training features shape: (3265, 0)\n",
      "qsprpred - DEBUG - Test set features shape: (817, 0)\n",
      "qsprpred - DEBUG - Training labels shape: (3265, 1)\n",
      "qsprpred - DEBUG - Test set labels shape: (817, 1)\n",
      "qsprpred - DEBUG - Training features indices: Index(['LoggingTutorialDataset_0000', 'LoggingTutorialDataset_0001',\n",
      "       'LoggingTutorialDataset_0002', 'LoggingTutorialDataset_0003',\n",
      "       'LoggingTutorialDataset_0004', 'LoggingTutorialDataset_0005',\n",
      "       'LoggingTutorialDataset_0007', 'LoggingTutorialDataset_0009',\n",
      "       'LoggingTutorialDataset_0010', 'LoggingTutorialDataset_0011',\n",
      "       ...\n",
      "       'LoggingTutorialDataset_4066', 'LoggingTutorialDataset_4067',\n",
      "       'LoggingTutorialDataset_4068', 'LoggingTutorialDataset_4070',\n",
      "       'LoggingTutorialDataset_4071', 'LoggingTutorialDataset_4072',\n",
      "       'LoggingTutorialDataset_4074', 'LoggingTutorialDataset_4075',\n",
      "       'LoggingTutorialDataset_4078', 'LoggingTutorialDataset_4080'],\n",
      "      dtype='object', name='QSPRID', length=3265)\n",
      "qsprpred - DEBUG - Test set features indices: Index(['LoggingTutorialDataset_0006', 'LoggingTutorialDataset_0008',\n",
      "       'LoggingTutorialDataset_0012', 'LoggingTutorialDataset_0014',\n",
      "       'LoggingTutorialDataset_0017', 'LoggingTutorialDataset_0023',\n",
      "       'LoggingTutorialDataset_0025', 'LoggingTutorialDataset_0026',\n",
      "       'LoggingTutorialDataset_0029', 'LoggingTutorialDataset_0030',\n",
      "       ...\n",
      "       'LoggingTutorialDataset_4035', 'LoggingTutorialDataset_4054',\n",
      "       'LoggingTutorialDataset_4056', 'LoggingTutorialDataset_4057',\n",
      "       'LoggingTutorialDataset_4069', 'LoggingTutorialDataset_4073',\n",
      "       'LoggingTutorialDataset_4076', 'LoggingTutorialDataset_4077',\n",
      "       'LoggingTutorialDataset_4079', 'LoggingTutorialDataset_4081'],\n",
      "      dtype='object', name='QSPRID', length=817)\n",
      "qsprpred - DEBUG - Training labels indices: Index(['LoggingTutorialDataset_0000', 'LoggingTutorialDataset_0001',\n",
      "       'LoggingTutorialDataset_0002', 'LoggingTutorialDataset_0003',\n",
      "       'LoggingTutorialDataset_0004', 'LoggingTutorialDataset_0005',\n",
      "       'LoggingTutorialDataset_0007', 'LoggingTutorialDataset_0009',\n",
      "       'LoggingTutorialDataset_0010', 'LoggingTutorialDataset_0011',\n",
      "       ...\n",
      "       'LoggingTutorialDataset_4066', 'LoggingTutorialDataset_4067',\n",
      "       'LoggingTutorialDataset_4068', 'LoggingTutorialDataset_4070',\n",
      "       'LoggingTutorialDataset_4071', 'LoggingTutorialDataset_4072',\n",
      "       'LoggingTutorialDataset_4074', 'LoggingTutorialDataset_4075',\n",
      "       'LoggingTutorialDataset_4078', 'LoggingTutorialDataset_4080'],\n",
      "      dtype='object', name='QSPRID', length=3265)\n",
      "qsprpred - DEBUG - Test set labels indices: Index(['LoggingTutorialDataset_0006', 'LoggingTutorialDataset_0008',\n",
      "       'LoggingTutorialDataset_0012', 'LoggingTutorialDataset_0014',\n",
      "       'LoggingTutorialDataset_0017', 'LoggingTutorialDataset_0023',\n",
      "       'LoggingTutorialDataset_0025', 'LoggingTutorialDataset_0026',\n",
      "       'LoggingTutorialDataset_0029', 'LoggingTutorialDataset_0030',\n",
      "       ...\n",
      "       'LoggingTutorialDataset_4035', 'LoggingTutorialDataset_4054',\n",
      "       'LoggingTutorialDataset_4056', 'LoggingTutorialDataset_4057',\n",
      "       'LoggingTutorialDataset_4069', 'LoggingTutorialDataset_4073',\n",
      "       'LoggingTutorialDataset_4076', 'LoggingTutorialDataset_4077',\n",
      "       'LoggingTutorialDataset_4079', 'LoggingTutorialDataset_4081'],\n",
      "      dtype='object', name='QSPRID', length=817)\n",
      "qsprpred - INFO - Dataset 'LoggingTutorialDataset' created for target Properties: '[TargetProperty(name=pchembl_value_Mean, task=REGRESSION)]'. Number of samples: 4082. Chunk size: 4082. Number of CPUs: 1.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<qsprpred.data.tables.qspr.QSPRDataset at 0x7fe834d9dd80>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "from qsprpred.data import QSPRDataset\n",
    "\n",
    "os.makedirs(\"../../tutorial_output/data\", exist_ok=True)\n",
    "QSPRDataset.fromTableFile(  # no message is displayed\n",
    "    filename=\"../../tutorial_data/A2A_LIGANDS.tsv\",\n",
    "    store_dir=\"../../tutorial_output/data\",\n",
    "    name=\"LoggingTutorialDataset\",\n",
    "    target_props=[{\"name\": \"pchembl_value_Mean\", \"task\": \"REGRESSION\"}],\n",
    "    random_state=42\n",
    ")\n",
    "\n",
    "# change logger for one with a lower level\n",
    "from qsprpred.logs import setLogger\n",
    "import logging\n",
    "\n",
    "logger.setLevel(logging.DEBUG)\n",
    "setLogger(logger)\n",
    "\n",
    "QSPRDataset.fromTableFile(  # more information is shown\n",
    "    filename=\"../../tutorial_data/A2A_LIGANDS.tsv\",\n",
    "    store_dir=\"../../tutorial_output/data\",\n",
    "    name=\"LoggingTutorialDataset\",\n",
    "    target_props=[{\"name\": \"pchembl_value_Mean\", \"task\": \"REGRESSION\"}],\n",
    "    random_state=42\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-01-02T13:52:52.801006856Z",
     "start_time": "2024-01-02T13:52:52.798085430Z"
    },
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# set the level back to warnings only\n",
    "\n",
    "logger.setLevel(logging.WARNING)\n",
    "setLogger(logger)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "The `enable_file_logger` utility function will\n",
    "create a file logger that will write to the file `TutorialLogging.log` in the `tutorial_output` directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-01-02T13:52:52.916439691Z",
     "start_time": "2024-01-02T13:52:52.800322763Z"
    }
   },
   "outputs": [],
   "source": [
    "# logging settings\n",
    "from qsprpred.logs import logger\n",
    "import logging\n",
    "from qsprpred.logs import setLogger\n",
    "from qsprpred.logs.utils import enable_file_logger\n",
    "\n",
    "logSettings = enable_file_logger(\n",
    "    log_folder=\"../../tutorial_output/\",\n",
    "    filename=\"TutorialLogging.log\",\n",
    "    debug=False,\n",
    "    disable_existing_loggers=False\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A utility function is provided to save your conda environment to a file. This is useful if you want to share your code with others, or if you want to recreate your environment at a later time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-01-02T13:52:57.200644968Z",
     "start_time": "2024-01-02T13:52:52.918730795Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Environment exported to ../../tutorial_output/TutorialLoggingEnvironment.yml successfully!\n"
     ]
    }
   ],
   "source": [
    "from qsprpred.logs.utils import export_conda_environment\n",
    "\n",
    "# save the environment to a yaml file\n",
    "export_conda_environment(\"../../tutorial_output/TutorialLoggingEnvironment.yml\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below we run a standard data preparation and modelling workflow to show logging output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-01-02T13:53:07.593653873Z",
     "start_time": "2024-01-02T13:52:57.203105287Z"
    }
   },
   "outputs": [],
   "source": [
    "from qsprpred.data.descriptors.fingerprints import MorganFP\n",
    "from qsprpred.data import RandomSplit\n",
    "from qsprpred.data import QSPRDataset\n",
    "\n",
    "import os\n",
    "\n",
    "os.makedirs(\"../../tutorial_output/data\", exist_ok=True)\n",
    "\n",
    "dataset = QSPRDataset.fromTableFile(\n",
    "    filename=\"../../tutorial_data/A2A_LIGANDS.tsv\",\n",
    "    store_dir=\"../../tutorial_output/data\",\n",
    "    name=\"LoggingTutorialDataset\",\n",
    "    target_props=[{\"name\": \"pchembl_value_Mean\", \"task\": \"REGRESSION\"}],\n",
    "    random_state=42\n",
    ")\n",
    "\n",
    "dataset.prepareDataset(\n",
    "    split=RandomSplit(test_fraction=0.2, dataset=dataset),\n",
    "    feature_calculators=[MorganFP(radius=3, nBits=2048)],\n",
    ")\n",
    "\n",
    "_ = dataset.save()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-01-02T13:53:20.636361248Z",
     "start_time": "2024-01-02T13:53:07.597306132Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "qsprpred - WARNING - Random state supplied, but alg <class 'sklearn.neighbors._regression.KNeighborsRegressor'> does not support it. Ignoring this setting.\n",
      "Random state supplied, but alg <class 'sklearn.neighbors._regression.KNeighborsRegressor'> does not support it. Ignoring this setting.\n"
     ]
    }
   ],
   "source": [
    "# Standard model training\n",
    "from qsprpred.models import SklearnModel\n",
    "from sklearn.neighbors import KNeighborsRegressor\n",
    "from qsprpred.models import CrossValAssessor, TestSetAssessor\n",
    "\n",
    "os.makedirs(\"../../tutorial_output/models\", exist_ok=True)\n",
    "\n",
    "model = SklearnModel(\n",
    "    base_dir=\"../../tutorial_output/models\",\n",
    "    alg=KNeighborsRegressor,\n",
    "    name=\"TutorialLoggingModel\"\n",
    ")\n",
    "\n",
    "CrossValAssessor(\"r2\")(model, dataset)\n",
    "TestSetAssessor(\"r2\")(model, dataset)\n",
    "\n",
    "model.fitDataset(dataset)\n",
    "\n",
    "_ = model.save()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "qsprpred",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}