Merge branch 'master' into cc/run_how_tos

.github/workflows/api_doc_build.yml

@@ -90,6 +90,10 @@ jobs:
         with:
           repository: langchain-ai/langchain-mongodb
           path: langchain-mongodb
+      - uses: actions/checkout@v4
+        with:
+          repository: langchain-ai/langchain-redis
+          path: langchain-redis
 
 
 
@@ -119,7 +123,8 @@ jobs:
             langchain/libs/partners/databricks \
             langchain/libs/partners/ibm \
             langchain/libs/partners/azure-dynamic-sessions \
-            langchain/libs/partners/mongodb
+            langchain/libs/partners/mongodb \
+            langchain/libs/partners/redis
           mv langchain-google/libs/genai langchain/libs/partners/google-genai
           mv langchain-google/libs/vertexai langchain/libs/partners/google-vertexai
           mv langchain-google/libs/community langchain/libs/partners/google-community
@@ -139,6 +144,7 @@ jobs:
           mv langchain-ibm/libs/ibm langchain/libs/partners/ibm
           mv langchain-azure/libs/azure-dynamic-sessions langchain/libs/partners/azure-dynamic-sessions
           mv langchain-mongodb/libs/mongodb langchain/libs/partners/mongodb
+          mv langchain-redis/libs/redis langchain/libs/partners/redis
 
       - name: Rm old html
         run:

docs/docs/concepts/retrieval.mdx

@@ -65,7 +65,7 @@ Various techniques have been developed to leverage models for query re-writing,
 
 | Name                                                                                                      | When to use                                                                                     | Description                                                                                                                                                                                                                                                                            |
 |-----------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| [Multi-query](/docs/how_to/MultiQueryRetriever/)                                                          | When you want to ensure high recall in retrieval by providing multiple pharsings of a question. | Rewrite the user question with multiple pharsings, retrieve documents for each rewritten question, return the unique documents for all queries.                                                                                                                                        |
+| [Multi-query](/docs/how_to/MultiQueryRetriever/)                                                          | When you want to ensure high recall in retrieval by providing multiple phrasings of a question. | Rewrite the user question with multiple phrasings, retrieve documents for each rewritten question, return the unique documents for all queries.                                                                                                                                        |
 | [Decomposition](https://github.com/langchain-ai/rag-from-scratch/blob/main/rag_from_scratch_5_to_9.ipynb) | When a question can be broken down into smaller subproblems.                                    | Decompose a question into a set of subproblems / questions, which can either be solved sequentially (use the answer from first + retrieval to answer the second) or in parallel (consolidate each answer into final answer).                                                           |
 | [Step-back](https://github.com/langchain-ai/rag-from-scratch/blob/main/rag_from_scratch_5_to_9.ipynb)     | When a higher-level conceptual understanding is required.                                       | First prompt the LLM to ask a generic step-back question about higher-level concepts or principles, and retrieve relevant facts about them. Use this grounding to help answer the user question. [Paper](https://arxiv.org/pdf/2310.06117).                                            |
 | [HyDE](https://github.com/langchain-ai/rag-from-scratch/blob/main/rag_from_scratch_5_to_9.ipynb)          | If you have challenges retrieving relevant documents using the raw user inputs.                 | Use an LLM to convert questions into hypothetical documents that answer the question. Use the embedded hypothetical documents to retrieve real documents with the premise that doc-doc similarity search can produce more relevant matches. [Paper](https://arxiv.org/abs/2212.10496). |

docs/docs/integrations/chat/ibm_watsonx.ipynb

@@ -34,9 +34,9 @@
     "## Overview\n",
     "\n",
     "### Integration details\n",
-    "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/openai) | Package downloads | Package latest |\n",
+    "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/ibm/) | Package downloads | Package latest |\n",
     "| :--- | :--- | :---: | :---: |  :---: | :---: | :---: |\n",
-    "| ChatWatsonx | ❌ | ❌ | ❌ | ❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-ibm?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-ibm?style=flat-square&label=%20) |\n",
+    "| [ChatWatsonx](https://python.langchain.com/api_reference/ibm/chat_models/langchain_ibm.chat_models.ChatWatsonx.html#langchain_ibm.chat_models.ChatWatsonx) | [langchain-ibm](https://python.langchain.com/api_reference/ibm/index.html) | ❌ | ❌ | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-ibm?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-ibm?style=flat-square&label=%20) |\n",
     "\n",
     "### Model features\n",
     "| [Tool calling](/docs/how_to/tool_calling/) | [Structured output](/docs/how_to/structured_output/) | JSON mode | Image input | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n",
@@ -549,6 +549,16 @@
    "source": [
     "ai_msg.tool_calls"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "95fcbf93",
+   "metadata": {},
+   "source": [
+    "## API reference\n",
+    "\n",
+    "For detailed documentation of all `ChatWatsonx` features and configurations head to the [API reference](https://python.langchain.com/api_reference/ibm/chat_models/langchain_ibm.chat_models.ChatWatsonx.html)."
+   ]
   }
  ],
  "metadata": {

docs/docs/integrations/llms/ibm_watsonx.ipynb

@@ -14,33 +14,32 @@
   },
   {
    "cell_type": "markdown",
-   "id": "ea35b2b7",
+   "id": "5b8d9390",
    "metadata": {},
    "source": [
-    "## Setting up\n",
+    "## Overview\n",
     "\n",
-    "Install the package `langchain-ibm`."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "id": "2f1fff4e",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "!pip install -qU langchain-ibm"
+    "### Integration details\n",
+    "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/llms/ibm/) | Package downloads | Package latest |\n",
+    "| :--- | :--- | :---: | :---: |  :---: | :---: | :---: |\n",
+    "| [WatsonxLLM](https://python.langchain.com/api_reference/ibm/llms/langchain_ibm.llms.WatsonxLLM.html) | [langchain-ibm](https://python.langchain.com/api_reference/ibm/index.html) | ❌ | ❌ | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-ibm?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-ibm?style=flat-square&label=%20) |"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "f406e092",
+   "id": "ea35b2b7",
    "metadata": {},
    "source": [
-    "This cell defines the WML credentials required to work with watsonx Foundation Model inferencing.\n",
+    "## Setup\n",
+    "\n",
+    "To access IBM watsonx.ai models you'll need to create an IBM watsonx.ai account, get an API key, and install the `langchain-ibm` integration package.\n",
+    "\n",
+    "### Credentials\n",
+    "\n",
+    "The cell below defines the credentials required to work with watsonx Foundation Model inferencing.\n",
     "\n",
     "**Action:** Provide the IBM Cloud user API key. For details, see\n",
-    "[documentation](https://cloud.ibm.com/docs/account?topic=account-userapikey&interface=ui)."
+    "[Managing user API keys](https://cloud.ibm.com/docs/account?topic=account-userapikey&interface=ui)."
    ]
   },
   {
@@ -81,19 +80,39 @@
     "os.environ[\"WATSONX_INSTANCE_ID\"] = \"your instance_id for accessing the CPD cluster\""
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "f918d229",
+   "metadata": {},
+   "source": [
+    "### Installation\n",
+    "\n",
+    "The LangChain IBM integration lives in the `langchain-ibm` package:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f925c9aa",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install -qU langchain-ibm"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "e36acbef",
    "metadata": {},
    "source": [
-    "## Load the model\n",
+    "## Instantiation\n",
     "\n",
     "You might need to adjust model `parameters` for different models or tasks. For details, refer to [documentation](https://ibm.github.io/watsonx-ai-python-sdk/fm_model.html#metanames.GenTextParamsMetaNames)."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 1,
    "id": "407cd500",
    "metadata": {},
    "outputs": [],
@@ -124,7 +143,7 @@
     "In this example, we’ll use the `project_id` and Dallas url.\n",
     "\n",
     "\n",
-    "You need to specify `model_id` that will be used for inferencing. All available models you can find in [documentation](https://ibm.github.io/watsonx-ai-python-sdk/fm_model.html#ibm_watsonx_ai.foundation_models.utils.enums.ModelTypes)."
+    "You need to specify `model_id` that will be used for inferencing. All available models you can find in [documentation](https://ibm.github.io/watsonx-ai-python-sdk/fm_model.html#TextModels)."
    ]
   },
   {
@@ -241,84 +260,28 @@
     "watsonx_llm = WatsonxLLM(watsonx_model=model)"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "id": "c25ecbd1",
-   "metadata": {},
-   "source": [
-    "## Create Chain\n",
-    "Create `PromptTemplate` objects which will be responsible for creating a random question."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "id": "c7d80c05",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain_core.prompts import PromptTemplate\n",
-    "\n",
-    "template = \"Generate a random question about {topic}: Question: \"\n",
-    "\n",
-    "prompt = PromptTemplate.from_template(template)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "79056d8e",
-   "metadata": {},
-   "source": [
-    "Provide a topic and run the chain."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 9,
-   "id": "dc076c56",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "'What is the difference between a dog and a wolf?'"
-      ]
-     },
-     "execution_count": 9,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "llm_chain = prompt | watsonx_llm\n",
-    "\n",
-    "topic = \"dog\"\n",
-    "\n",
-    "llm_chain.invoke(topic)"
-   ]
-  },
   {
    "cell_type": "markdown",
    "id": "f571001d",
    "metadata": {},
    "source": [
-    "## Calling the Model Directly\n",
+    "## Invocation\n",
     "To obtain completions, you can call the model directly using a string prompt."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 3,
    "id": "beea2b5b",
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "\"Man's best friend is his dog. \""
+       "\"Man's best friend is his dog. Dogs are man's best friend because they are always there for you, they never judge you, and they love you unconditionally. Dogs are also great companions and can help reduce stress levels. \""
       ]
      },
-     "execution_count": 7,
+     "execution_count": 3,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -331,17 +294,17 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 11,
+   "execution_count": 4,
    "id": "8ab1a25a",
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "LLMResult(generations=[[Generation(text='The fastest dog in the world is the greyhound, which can run up to 45 miles per hour. This is about the same speed as a human running down a track. Greyhounds are very fast because they have long legs, a streamlined body, and a strong tail. They can run this fast for short distances, but they can also run for long distances, like a marathon. ', generation_info={'finish_reason': 'eos_token'})], [Generation(text='The Beagle is a scent hound, meaning it is bred to hunt by following a trail of scents.', generation_info={'finish_reason': 'eos_token'})]], llm_output={'token_usage': {'generated_token_count': 106, 'input_token_count': 13}, 'model_id': 'ibm/granite-13b-instruct-v2', 'deployment_id': ''}, run=[RunInfo(run_id=UUID('52cb421d-b63f-4c5f-9b04-d4770c664725')), RunInfo(run_id=UUID('df2ea606-1622-4ed7-8d5d-8f6e068b71c4'))])"
+       "LLMResult(generations=[[Generation(text='The fastest dog in the world is the greyhound. Greyhounds can run up to 45 mph, which is about the same speed as a Usain Bolt.', generation_info={'finish_reason': 'eos_token'})], [Generation(text='The Labrador Retriever is a breed of retriever that was bred for hunting. They are a very smart breed and are very easy to train. They are also very loyal and will make great companions. ', generation_info={'finish_reason': 'eos_token'})]], llm_output={'token_usage': {'generated_token_count': 82, 'input_token_count': 13}, 'model_id': 'ibm/granite-13b-instruct-v2', 'deployment_id': None}, run=[RunInfo(run_id=UUID('750b8a0f-8846-456d-93d0-e039e95b1276')), RunInfo(run_id=UUID('aa4c2a1c-5b08-4fcf-87aa-50228de46db5'))], type='LLMResult')"
       ]
      },
-     "execution_count": 11,
+     "execution_count": 4,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -369,15 +332,15 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 12,
+   "execution_count": 5,
    "id": "3f63166a",
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "My favorite breed of dog is a Labrador Retriever. Labradors are my favorite because they are extremely smart, very friendly, and love to be with people. They are also very playful and love to run around and have a lot of energy. "
+      "My favorite breed of dog is a Labrador Retriever. They are my favorite breed because they are my favorite color, yellow. They are also very smart and easy to train. "
      ]
     }
    ],
@@ -387,6 +350,72 @@
     "):\n",
     "    print(chunk, end=\"\")"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9fc88fdd",
+   "metadata": {},
+   "source": [
+    "## Chaining\n",
+    "Create `PromptTemplate` objects which will be responsible for creating a random question."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "ad63fa27",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain_core.prompts import PromptTemplate\n",
+    "\n",
+    "template = \"Generate a random question about {topic}: Question: \"\n",
+    "\n",
+    "prompt = PromptTemplate.from_template(template)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "677699db",
+   "metadata": {},
+   "source": [
+    "Provide a topic and run the chain."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "868af75c",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'What is the origin of the name \"Pomeranian\"?'"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "llm_chain = prompt | watsonx_llm\n",
+    "\n",
+    "topic = \"dog\"\n",
+    "\n",
+    "llm_chain.invoke(topic)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "59480270",
+   "metadata": {},
+   "source": [
+    "## API reference\n",
+    "\n",
+    "For detailed documentation of all `WatsonxLLM` features and configurations head to the [API reference](https://python.langchain.com/api_reference/ibm/llms/langchain_ibm.llms.WatsonxLLM.html)."
+   ]
   }
  ],
  "metadata": {
@@ -405,7 +434,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.13"
+   "version": "3.10.14"
   }
  },
  "nbformat": 4,

docs/docs/integrations/providers/databricks.md

@@ -20,6 +20,18 @@ First-party Databricks integrations are available in the langchain-databricks pa
 pip install langchain-databricks
 ```
 
+🚧 Upcoming Package Consolidation Notice
+
+This package (`langchain-databricks`) will soon be consolidated into a new package: `databricks-langchain`. The new package will serve as the primary hub for all Databricks Langchain integrations.
+
+What’s Changing?
+In the coming months, `databricks-langchain` will include all features currently in `langchain-databricks`, as well as additional integrations to provide a unified experience for Databricks users.
+
+What You Need to Know
+For now, continue to use `langchain-databricks` as usual. When `databricks-langchain` is ready, we’ll provide clear migration instructions to make the transition seamless. During the transition period, `langchain-databricks` will remain operational, and updates will be shared here with timelines and guidance.
+
+Thank you for your support as we work toward an improved, streamlined experience!
+
 Chat Model
 ----------