stable-diffusion-notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Paperspace Stable Diffusion Notebook\n",
    "\n",
    "This Notebook contains instructions and easy steps for generating images with Stable Diffusion on any of Paperspace's GPUs with little set up required. Follow the instructions in the markdown cells in this Notebook to quickly launch the Stable Diffusion Web UI or begin generating images using the Diffusers Stable Diffusion Pipeline.\n",
    "\n",
    "Contents:\n",
    "\n",
    "- Use the Stable Diffusion Web UI\n",
    "    - Loading in the Stable Diffusion models for the Web UI with support for v1-5 and v2 models\n",
    "- The Diffusers Stable Diffusion Pipeline\n",
    "    - Loading in the Stable Diffusion models for the Diffusers StableDiffusionPipeline with support for v1-4, v1-5 and v2 models\n",
    "    - Access additional models: log in to HuggingFace for online access model files\n",
    "    - Inference with the Diffusers pipeline\n",
    "- CLIP reranking\n",
    "- GFPGAN face restoration and image upscaling\n",
    "- CodeFormer face restoration and image upscaling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "First, run the cell below to get everything we need for this Notebook to run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fed23e3a",
   "metadata": {
    "collapsed": true,
    "execution": {
     "iopub.execute_input": "2022-11-29T01:17:34.402699Z",
     "iopub.status.busy": "2022-11-29T01:17:34.401879Z",
     "iopub.status.idle": "2022-11-29T01:17:58.874403Z",
     "shell.execute_reply": "2022-11-29T01:17:58.873489Z",
     "shell.execute_reply.started": "2022-11-29T01:17:34.402612Z"
    },
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": [
    "!pip install --upgrade git+https://github.com/huggingface/diffusers.git \n",
    "!jupyter nbextension enable --py widgetsnbextension\n",
    "!git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui\n",
    "!%cd /notebooks/stable-diffusion-webui\n",
    "!pip install -r /notebooks/stable-diffusion-webui/requirements.txt\n",
    "!mkdir outputs\n",
    "%cd /notebooks/stable-diffusion-webui/extensions\n",
    "!rm -rf ./stable-diffusion-webui-nsfw-censor\n",
    "!git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui-nsfw-censor\n",
    "!git clone https://github.com/Gourieff/sd-webui-reactor\n",
    "!pip install -r /notebooks/stable-diffusion-webui/extensions/sd-webui-reactor/requirements.txt\n",
    "%cd /notebooks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e3ac3a2f",
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors -O /notebooks/stable-diffusion-webui/models/Stable-diffusion/sd_xl_base_1.0.safetensors\n",
    "!wget https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0/resolve/main/sd_xl_refiner_1.0.safetensors -O /notebooks/stable-diffusion-webui/models/Stable-diffusion/sd_xl_refiner_1.0.safetensors\n",
    "!wget https://huggingface.co/stabilityai/sdxl-vae/resolve/main/sdxl_vae.safetensors -O /notebooks/stable-diffusion-webui/models/VAE/sdxl_vae.safetensors\n",
    "!wget https://huggingface.co/AICademyHK/easynegative/resolve/main/hyperRealism_30.ckpt -O /notebooks/stable-diffusion-webui/models/Stable-diffusion/hyperRealism_30.ckpt\n",
    "!wget https://huggingface.co/AICademyHK/easynegative/resolve/main/styles.csv -O /notebooks/stable-diffusion-webui/styles.csv\n",
    "!mkdir /notebooks/stable-diffusion-webui/models/Lora\n",
    "!mkdir /notebooks/stable-diffusion-webui/models/ESRGAN\n",
    "!mkdir /notebooks/stable-diffusion-webui/embeddings\n",
    "!wget https://huggingface.co/AICademyHK/easynegative/resolve/main/easynegative.safetensors -O /notebooks/stable-diffusion-webui/embeddings/easynegative.safetensors\n",
    "!cp /storage/4x_foolhardy_Remacri.pth /notebooks/stable-diffusion-webui/models/ESRGAN/\n",
    "!cp /storage/Retro_Illustration.safetensors /notebooks/stable-diffusion-webui/models/Lora/\n",
    "!cp /storage/psyAnimatedXL_v11.safetensors /notebooks/stable-diffusion-webui/models/Stable-diffusion/\n",
    "!unzip /storage/lora_pack.zip -d /notebooks/stable-diffusion-webui/models/Lora\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eaa37f86",
   "metadata": {},
   "source": [
    "## Use the Stable Diffusion Web UI\n",
    "\n",
    "Run the cell below to setup, and launch the Automatic1111 Stable Diffusion Web UI from this notebook. Click the Gradio link after the setup completes to access the Web UI in your local browser. \n",
    "\n",
    "Skip this step to use the Diffusers library methodology.\n",
    "\n",
    "### Loading in the Stable Diffusion models for the Web UI \n",
    "\n",
    "To make accessing the Stable Diffusion models easy and not take up any storage, we have added the Stable Diffusion v1-5 and v2 models as mountable public datasets!\n",
    "\n",
    "> These files will not count towards the storage limit for free notebooks.\n",
    "\n",
    "To access the model files:\n",
    "\n",
    "First, navigate to the \"Data Sources\" tab using the navigator on the far left of the page.\n",
    "\n",
    "Next, click \"Public\" to switch into the Gradient Public Datasets, and scroll down until you find the Stable Diffusion datasets near the bottom of the list. You can mount `stable-diffusion-classic` for access to the v1-5 model checkpoints for the web UI, `stable-diffusion-diffusers` for the v1-5 models as Diffusers StableDiffusionPipeline binaries, `stable-diffusion-classic-v2` for access to the v2 model checkpoints and config file for use with the web UI, and `stable-diffusion-diffusers-v2` for access to the v2 models as Diffusers StableDiffusionPipeline binaries.\n",
    "\n",
    "Finally, click \"mount\" to make any of these Public Datasets accessible from the \"datasets\" directory. This directory is in the root folder, so access it with the path `~/../datasets/stable-diffusion-classic/SDv1.5.ckpt`. For example, after mounting, the v1-5 checkpoint can be found at `~/../datasets/stable-diffusion-classic/SDv1.5.ckpt`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "05dd0049",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Launch the Stable Diffusion Web UI. \n",
    "# Be sure to hash out whichever you are not using.\n",
    "%cd /notebooks/stable-diffusion-webui\n",
    "\n",
    "## Launch Web UI for Stable Diffusion v1-x models.\n",
    "!python launch.py --xformers --share\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4a7b504c",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "#%cd /notebooks/stable-diffusion-webui/extensions/sd-webui-controlnet/models\n",
    "#!wget https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11e_sd15_ip2p.pth\n",
    "#!wget https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11e_sd15_shuffle.pth\n",
    "#!wget https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11p_sd15_inpaint.pth\n",
    "#!wget https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11p_sd15_lineart.pth\n",
    "#!wget https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11p_sd15_mlsd.pth\n",
    "#!wget https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11p_sd15_normalbae.pth\n",
    "#!wget https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11p_sd15_scribble.pth\n",
    "#!wget https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11p_sd15_seg.pth\n",
    "#!wget https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11p_sd15_softedge.pth\n",
    "#!wget https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11p_sd15s2_lineart_anime.pth\n",
    "#%cd /notebooks\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}