1_workshop_tools.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to LLM Agents with LangChain\n",
    "\n",
    "## Notebook 1: Tools\n",
    "---\n",
    "\n",
    "Welcome to the workshop on building LLM agents with LangChain!\n",
    "\n",
    "**The goal:** \n",
    "\n",
    "With this notebook you will familiarize yourself with the key concepts of Tools as building blocks of an LLM Agent. At the end, you will have all the code you need to use custom LangChain tools as well as build your own custom tools.\n",
    "\n",
    "🌟 So ... let us begin!  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**The use case:** \n",
    "\n",
    "Summer holidays are coming up and you still don't know where to go. Oh no! \n",
    "\n",
    "You decide to build a tool that helps you get information on holiday locations. For example, you would like to to find out how big a specific city is, what sights are there to see, how the weather there is, and you would like to get a drawing of that place, to get a first impression. Because who does not like art? \n",
    "\n",
    "You will implement this through an LLM agent, who has access to\n",
    "* the wikipedia API,\n",
    "* a weather API,\n",
    "* can generate images by using a HuggingFace API. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Content:**\n",
    "\n",
    "1. [Default LangChain tools](#1)\n",
    "        <ol type=\"a\">\n",
    "        <li>[Exercise 1 (a): Explore tool parameters](#1a)</li>\n",
    "        <li>[Exercise 1 (b): Run tool and explore output](#1b)</li>\n",
    "        </ol>\n",
    "2. [Custom tools](#2)\n",
    "        <ol type=\"a\">\n",
    "        <li>[Exercise 2 (a): Build your own Weather tool](#2a)</li>\n",
    "        <li>[Exercise 2 (b): Build your own Image tool](#2b)</li>\n",
    "        </ol>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Reminder:** Make sure to update `helper_functions/keys.py` based on keys in [privatebin](https://privatebin.molops.io/?a6459e88fa282c28#DsFZvkZSiuPcNQzNXvmtvmTozihhaf1hQdqBCd7r3q5s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ONLY IF YOU USE GOOGLE COLAB: copy code from helper_functions/colab_utils.txt here and compile"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "from helper_functions.keys import WEATHER_KEY, HUGGING_FACE_KEY\n",
    "from langchain_community.tools import WikipediaQueryRun\n",
    "from langchain_community.utilities import WikipediaAPIWrapper\n",
    "from langchain.pydantic_v1 import BaseModel, Field\n",
    "from langchain.tools import StructuredTool\n",
    "from PIL import Image\n",
    "import io"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A Langchain Default tool\n",
    "### Default [Wikipedia tool](https://python.langchain.com/v0.1/docs/integrations/tools/wikipedia/) <a id='1'></a>\n",
    "\n",
    "The cell below loads the full wikipedia tool. It makes an API call to Wikipedia using the ``WikipediaAPIWrapper`` and returns a summary of the queried article. ``WikipediaQueryRun`` then wraps this into a ready made tool. \n",
    "\n",
    "Each tool is a ``BaseTool`` class object, you can find its definition [here](https://api.python.langchain.com/en/latest/tools/langchain_core.tools.BaseTool.html#langchain_core.tools.BaseTool)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "api_wrapper = WikipediaAPIWrapper(top_k_results=1)\n",
    "wiki_tool = WikipediaQueryRun(api_wrapper=api_wrapper)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise 1 (a): Explore tool parameters <a id='1a'></a>\n",
    "\n",
    "**TASK:**  \n",
    "Use the methods ``name``, ``description``, ``args``, ``return_direct``, ``metadata`` to familiarize yourself with the parameters of the tool. What is the meaning of the different parameters?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Name: \", wiki_tool.name)\n",
    "\n",
    "# TODO: insert your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise 1 (b): Run tool and explore output <a id='1b'></a>\n",
    "\n",
    "**TASK:**\n",
    "* Use the ``.run(tool_input)`` method to execute the tool. The ``tool_input`` is the search term that you'd like to query wikipedia with.\n",
    "* [Optional] Check out the arguments of the WikipediaAPIWrapper [here](https://api.python.langchain.com/en/latest/utilities/langchain_community.utilities.wikipedia.WikipediaAPIWrapper.html) and modify its parameters above. How does the output change? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tool_input = \"\"\"\n",
    "TODO: insert your code here\n",
    "\"\"\"\n",
    "\n",
    "# Run tool\n",
    "# TODO: insert your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Custom tools <a id='2'></a>\n",
    "### Custom Wikipedia tool\n",
    "\n",
    "You can build your own tools and don't have to rely on default tools. Tools can be built from any function with the LangChain class method ``StructuredTool.from_function()``(see [here](https://python.langchain.com/v0.1/docs/modules/tools/custom_tools/#structuredtool-dataclass)). The basic elements are\n",
    "* The **function** you would like to be executed when the tool is called\n",
    "* The definition of the **input parameters**\n",
    "* The tool **description**\n",
    "\n",
    "The tool description is especially important, since this is what the agent will use to make the decision if this tool should be used.\n",
    "\n",
    "Below you see the wikipedia tool, built from the basic elements described above:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define the function\n",
    "def wikipedia_caller(query:str) ->str:\n",
    "    \"\"\"This function queries wikipedia through a search query.\"\"\"\n",
    "    return api_wrapper.run(query)\n",
    "\n",
    "# Input parameter definition\n",
    "class QueryInput(BaseModel):\n",
    "    query: str = Field(description=\"Input search query\")\n",
    "\n",
    "# the tool description\n",
    "description: str = (\n",
    "        \"A wrapper around Wikipedia. \"\n",
    "        \"Useful for when you need to answer general questions about \"\n",
    "        \"people, places, companies, facts, historical events, or other subjects. \"\n",
    "        \"Input should be a search query.\"\n",
    "    )\n",
    "\n",
    "\n",
    "# fuse the function, input parameters and description into a tool. \n",
    "my_own_wiki_tool = StructuredTool.from_function(\n",
    "    func=wikipedia_caller,\n",
    "    name=\"wikipedia\",\n",
    "    description=description,\n",
    "    args_schema=QueryInput,\n",
    "    return_direct=False,\n",
    ")\n",
    "\n",
    "# test the output of the tool\n",
    "print(my_own_wiki_tool.run('pyladies'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise 2 (a): Build your own Weather tool <a id='2a'></a>\n",
    "The goal is to build a tool that extracts weather information from the weather site visualcrossing.com. You typically need an API key to extract information from a website. In this example we provide you with the API key. \n",
    "\n",
    "**TASK:** \n",
    "- Build the tool by defining the input parameters and the descriptions. The tool function is already provided to you. \n",
    "- Turn function, description and input parameters into a tool through ``StructuredTool.from_function()``.\n",
    "- Test if the tool gives an output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define the function\n",
    "def extract_city_weather(city:str)->str:\n",
    "\n",
    "    # Build the API URL\n",
    "    url = f\"https://weather.visualcrossing.com/VisualCrossingWebServices/rest/services/timeline/{city}?key={WEATHER_KEY}&unitGroup=metric\"\n",
    "\n",
    "    response = requests.get(url)\n",
    "\n",
    "    # extract response\n",
    "    if response.status_code == 200:\n",
    "        data = response.json()\n",
    "        current_temp = data['days'][0]['temp']\n",
    "        output = f\"Current temperature in {city}: {current_temp}°C\"\n",
    "    else:\n",
    "        output = f\"Error: {response.status_code}\"\n",
    "\n",
    "    return output\n",
    "\n",
    "# Input parameter definition\n",
    "class WeatherInput(BaseModel):\n",
    "    # insert your code here\n",
    "\n",
    "# the tool description\n",
    "description: str = (\n",
    "        # TODO: insert your code here\n",
    "    )\n",
    "\n",
    "# fuse the function, input parameters and description into a tool. \n",
    "my_weather_tool = StructuredTool.from_function(\n",
    "    # TODO: insert your code here\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Test the output of your Tool\n",
    "print(my_weather_tool.run('Amsterdam'))\n",
    "\n",
    "# TODO: Try generating more ouputs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's do something even more fun. As we previously saw, we can utilize APIs to build tools. Thinking about APIs, one of the biggest collection of models are available via APIs on HuggingFace. So, how about we try to utilize this. \n",
    "\n",
    "#### Exercise 2 (b): Build your own Image tool <a id='2b'></a>\n",
    "The goal is to build a tool that generates an image based on a given prompt. **That means that later when you can build the Agent you can have an LLM that not only outputs text, but also images!**  \n",
    "\n",
    "To develop this, you can make use of `mobius`, text-to-image model available on HuggingFace. We provided a HuggingFace token (that you loaded in the start). \n",
    "\n",
    "**TASK:** \n",
    "- Build the tool by defining the input parameters and the descriptions. The tool function is already provided to you. \n",
    "- Turn function, description and input parameters into a tool through ``StructuredTool.from_function()``.\n",
    "- Test if the tool gives an output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def text_to_image(payload:str):\n",
    "\n",
    "    # Call the text-to-image API with the provided palaod\n",
    "    API_URL = \"https://api-inference.huggingface.co/models/Corcelio/mobius\"\n",
    "    headers = {\"Authorization\": f\"Bearer {HUGGING_FACE_KEY}\"}\n",
    "\n",
    "    def query(payload):\n",
    "        response = requests.post(API_URL, headers=headers, json=payload)\n",
    "        return response.content\n",
    "    \n",
    "    image_bytes = query({\n",
    "        \"inputs\": payload,\n",
    "    })\n",
    "\n",
    "    image = Image.open(io.BytesIO(image_bytes))\n",
    "    \n",
    "    # Resize the image\n",
    "    new_size = (400, 400)  # Example new size (width, height)\n",
    "    resized_image = image.resize(new_size)\n",
    "\n",
    "\n",
    "    # Save the resized image to a file\n",
    "    image_path = f'images/image_{payload.replace(\" \", \"_\")}.jpg'\n",
    "    resized_image.save(image_path)\n",
    "    \n",
    "    # Return the path to the saved image\n",
    "    return f'{image_path} '\n",
    "\n",
    "\n",
    "# Input parameter definition\n",
    "class ImageInput(BaseModel):\n",
    "    payload: str = Field(description=\n",
    "        # TODO: insert your code here\n",
    "    )\n",
    "\n",
    "\n",
    "# the tool description\n",
    "images_description: str = (\n",
    "       # TODO: insert your code here\n",
    "    )\n",
    "\n",
    "# fuse the function, input parameters and description into a tool. \n",
    "my_image_tool = StructuredTool.from_function(\n",
    "        # TODO: insert your code here\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Test the output of your Tool\n",
    "print(my_image_tool.run('Amsterdam'))\n",
    "\n",
    "# TODO: Try generating more ouputs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**If you developed an additional tool, make sure to copy your code in `helper_functions/tools.py` in order to later be able to use your tool in an Agent.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Combine the individual tools into a list. Your collection of tools is now ready to be used by an agent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tools = [my_own_wiki_tool, my_weather_tool, my_image_tool]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "🌟 Done - you are now ready to proceed to the next notebook."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}