Skip to content

Commit

Permalink
community[minor]: Add Initial Support for TiDB Vector Store (#15796)
Browse files Browse the repository at this point in the history
This pull request introduces initial support for the TiDB vector store.
The current version is basic, laying the foundation for the vector store
integration. While this implementation provides the essential features,
we plan to expand and improve the TiDB vector store support with
additional enhancements in future updates.

Upcoming Enhancements:
* Support for Vector Index Creation: To enhance the efficiency and
performance of the vector store.
* Support for max marginal relevance search. 
* Customized Table Structure Support: Recognizing the need for
flexibility, we plan for more tailored and efficient data store
solutions.

Simple use case exmaple

```python
from typing import List, Tuple
from langchain.docstore.document import Document
from langchain_community.vectorstores import TiDBVectorStore
from langchain_openai import OpenAIEmbeddings

db = TiDBVectorStore.from_texts(
    embedding=embeddings,
    texts=['Andrew like eating oranges', 'Alexandra is from England', 'Ketanji Brown Jackson is a judge'],
    table_name="tidb_vector_langchain",
    connection_string=tidb_connection_url,
    distance_strategy="cosine",
)

query = "Can you tell me about Alexandra?"
docs_with_score: List[Tuple[Document, float]] = db.similarity_search_with_score(query)
for doc, score in docs_with_score:
    print("-" * 80)
    print("Score: ", score)
    print(doc.page_content)
    print("-" * 80)
```
  • Loading branch information
IANTHEREAL authored Mar 8, 2024
1 parent 3b1eb1f commit 390ef6a
Show file tree
Hide file tree
Showing 10 changed files with 1,425 additions and 5 deletions.
2 changes: 1 addition & 1 deletion docs/docs/integrations/document_loaders/tidb.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"source": [
"# TiDB\n",
"\n",
"> [TiDB](https://github.com/pingcap/tidb) is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics.\n",
"> [TiDB Cloud](https://tidbcloud.com/), is a comprehensive Database-as-a-Service (DBaaS) solution, that provides dedicated and serverless options. TiDB Serverless is now integrating a built-in vector search into the MySQL landscape. With this enhancement, you can seamlessly develop AI applications using TiDB Serverless without the need for a new database or additional technical stacks. Be among the first to experience it by joining the waitlist for the private beta at https://tidb.cloud/ai.\n",
"\n",
"This notebook introduces how to use `TiDBLoader` to load data from TiDB in langchain."
]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"source": [
"# TiDB\n",
"\n",
"> [TiDB](https://github.com/pingcap/tidb) is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics.\n",
"> [TiDB Cloud](https://tidbcloud.com/), is a comprehensive Database-as-a-Service (DBaaS) solution, that provides dedicated and serverless options. TiDB Serverless is now integrating a built-in vector search into the MySQL landscape. With this enhancement, you can seamlessly develop AI applications using TiDB Serverless without the need for a new database or additional technical stacks. Be among the first to experience it by joining the waitlist for the private beta at https://tidb.cloud/ai.\n",
"\n",
"This notebook introduces how to use TiDB to store chat message history. "
]
Expand Down
682 changes: 682 additions & 0 deletions docs/docs/integrations/vectorstores/tidb_vector.ipynb

Large diffs are not rendered by default.

9 changes: 9 additions & 0 deletions libs/community/langchain_community/vectorstores/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -436,6 +436,12 @@ def _import_tencentvectordb() -> Any:
return TencentVectorDB


def _import_tidb_vectorstore() -> Any:
from langchain_community.vectorstores.tidb_vector import TiDBVectorStore

return TiDBVectorStore


def _import_tiledb() -> Any:
from langchain_community.vectorstores.tiledb import TileDB

Expand Down Expand Up @@ -651,6 +657,8 @@ def __getattr__(name: str) -> Any:
return _import_tair()
elif name == "TencentVectorDB":
return _import_tencentvectordb()
elif name == "TiDBVectorStore":
return _import_tidb_vectorstore()
elif name == "TileDB":
return _import_tiledb()
elif name == "Tigris":
Expand Down Expand Up @@ -746,6 +754,7 @@ def __getattr__(name: str) -> Any:
"SupabaseVectorStore",
"SurrealDBStore",
"Tair",
"TiDBVectorStore",
"TileDB",
"Tigris",
"TimescaleVector",
Expand Down
Loading

0 comments on commit 390ef6a

Please sign in to comment.