diff --git a/src/vanna/ZhipuAI/ZhipuAI_Chat.py.md b/src/vanna/ZhipuAI/ZhipuAI_Chat.py.md new file mode 100644 index 00000000..06137351 --- /dev/null +++ b/src/vanna/ZhipuAI/ZhipuAI_Chat.py.md @@ -0,0 +1,153 @@ +## ๐Ÿค– ZhipuAI_Chat ๐Ÿค– + +### Table of Contents + +- [Introduction](#introduction) +- [Dependencies](#dependencies) +- [Class Definition](#class-definition) + - [Attributes](#attributes) + - [Methods](#methods) +- [Usage](#usage) +- [Example](#example) + +### Introduction + +The `ZhipuAI_Chat` class is an easy-to-use wrapper for the ZhipuAI Chat API, which allows you to generate text and code from natural language prompts. It provides a simplified interface for sending prompts to the API and receiving the results. + +### Dependencies + +This library depends on the following: + +- `pandas` +- `zhipuai` + +### Class Definition + +#### Attributes + +- `api_key`: Your ZhipuAI API key. +- `model`: The ZhipuAI model to use. The default is "glm-4". + +#### Methods + +- `get_sql_prompt`: Generates a prompt for the ZhipuAI Chat API to generate SQL code from a natural language prompt. +- `get_followup_questions_prompt`: Generates a prompt for the ZhipuAI Chat API to generate followup questions from a previous question and SQL code. +- `generate_question`: Generates a natural language question from a SQL query. +- `generate_plotly_code`: Generates Python Plotly code to chart the results of a pandas DataFrame. +- `submit_prompt`: Sends a prompt to the ZhipuAI Chat API and returns the response. + +### Usage + +To use the `ZhipuAI_Chat` class, simply initialize it with your API key. You can then use the `get_sql_prompt`, `get_followup_questions_prompt`, `generate_question`, `generate_plotly_code`, and `submit_prompt` methods to interact with the ZhipuAI Chat API. + +### Example + +The following code shows how to use the `ZhipuAI_Chat` class to generate SQL code from a natural language prompt: + +``` +from zhipuai_chat import ZhipuAI_Chat + +# Initialize the ZhipuAI_Chat class with your API key +chat = ZhipuAI_Chat(api_key="YOUR_API_KEY") + +# Generate a prompt for the ZhipuAI Chat API to generate SQL code +prompt = chat.get_sql_prompt( + "What is the total sales for each product category?", + question_sql_list=[], + ddl_list=[], + doc_list=[], +) + +# Send the prompt to the ZhipuAI Chat API and receive the response +response = chat.submit_prompt(prompt) + +# The response will be a SQL query that answers the natural language prompt +print(response) +``` + +### Example Usage - Generate SQL + +```python +from zhipuai_chat import ZhipuAI_Chat + +# Initialize the ZhipuAI_Chat class with your API key +chat = ZhipuAI_Chat(api_key="YOUR_API_KEY") + +# Generate a prompt for the ZhipuAI Chat API to generate SQL code +prompt = chat.get_sql_prompt( + "What is the total sales for each product category?", + question_sql_list=[], + ddl_list=[], + doc_list=[], +) + +# Send the prompt to the ZhipuAI Chat API and receive the response +response = chat.submit_prompt(prompt) + +# The response will be a SQL query that answers the natural language prompt +sql_query = response +``` + +### Example Usage - Generate Follow-up Question + +```python +from zhipuai_chat import ZhipuAI_Chat + +# Initialize the ZhipuAI_Chat class with your API key +chat = ZhipuAI_Chat(api_key="YOUR_API_KEY") + +# Generate a prompt for the ZhipuAI Chat API to generate SQL code +prompt = chat.get_followup_questions_prompt( + "What is the total sales for each product category?", + df=pd.DataFrame(), + question_sql_list=[], + ddl_list=[], + doc_list=[], +) + +# Send the prompt to the ZhipuAI Chat API and receive the response +response = chat.submit_prompt(prompt) + +# The response will be a list of follow-up questions +followup_questions = response +``` + +### Example Usage - Generate Question from SQL + +```python +from zhipuai_chat import ZhipuAI_Chat + +# Initialize the ZhipuAI_Chat class with your API key +chat = ZhipuAI_Chat(api_key="YOUR_API_KEY") + +# Generate a prompt for the ZhipuAI Chat API to generate SQL code +prompt = chat.generate_question( + "SELECT SUM(sales) FROM products GROUP BY product_category" +) + +# Send the prompt to the ZhipuAI Chat API and receive the response +response = chat.submit_prompt(prompt) + +# The response will be a question in natural language +question = response +``` + +### Example Usage - Generate Plotly Code + +```python +from zhipuai_chat import ZhipuAI_Chat + +# Initialize the ZhipuAI_Chat class with your API key +chat = ZhipuAI_Chat(api_key="YOUR_API_KEY") + +# Generate a prompt for the ZhipuAI Chat API to generate SQL code +prompt = chat.generate_plotly_code( + df=pd.DataFrame(), +) + +# Send the prompt to the ZhipuAI Chat API and receive the response +response = chat.submit_prompt(prompt) + +# The response will be Python Plotly code +plotly_code = response +``` \ No newline at end of file diff --git a/src/vanna/ZhipuAI/ZhipuAI_embeddings.py.md b/src/vanna/ZhipuAI/ZhipuAI_embeddings.py.md new file mode 100644 index 00000000..c4701d21 --- /dev/null +++ b/src/vanna/ZhipuAI/ZhipuAI_embeddings.py.md @@ -0,0 +1,139 @@ +## ๐Ÿ”— Table of Contents + +1. Overview +2. Usage +3. Example + +--- + +## ๐Ÿ“š Overview + +### Description +The `ZhipuAI_Embeddings` class is a `VannaBase` class that allows users to generate embeddings from ZhipuAI. It is designed to work with `chromadb` and can be used to create embedding functions for `chromadb` vector stores. + +### ๐Ÿ› ๏ธ Requirements +- Python 3.6 or higher +- `zhipuai` package +- `chromadb` package + +--- + +## ๐Ÿง™โ€โ™‚๏ธ Usage + +### Initialization +``` +from typing import List +from zhipuai import ZhipuAI +from chromadb import Documents, EmbeddingFunction, Embeddings +from ..base import VannaBase + +class ZhipuAI_Embeddings(VannaBase): + """ + [future functionality] This function is used to generate embeddings from ZhipuAI. + + Args: + VannaBase (_type_): _description_ + """ + def __init__(self, config=None): + VannaBase.__init__(self, config=config) + if "api_key" not in config: + raise Exception("Missing api_key in config") + self.api_key = config["api_key"] + self.client = ZhipuAI(api_key=self.api_key) + + def generate_embedding(self, data: str, **kwargs) -> List[float]: + + embedding = self.client.embeddings.create( + model="embedding-2", + input=data, + ) + + return embedding.data[0].embedding + + + +class ZhipuAIEmbeddingFunction(EmbeddingFunction[Documents]): + """ + A embeddingFunction that uses ZhipuAI to generate embeddings which can use in chromadb. + usage: + class MyVanna(ChromaDB_VectorStore, ZhipuAI_Chat): + def __init__(self, config=None): + ChromaDB_VectorStore.__init__(self, config=config) + ZhipuAI_Chat.__init__(self, config=config) + + config={'api_key': 'xxx'} + zhipu_embedding_function = ZhipuAIEmbeddingFunction(config=config) + config = {"api_key": "xxx", "model": "glm-4","path":"xy","embedding_function":zhipu_embedding_function} + + vn = MyVanna(config) + + """ + def __init__(self, config=None): + if config is None or "api_key" not in config: + raise ValueError("Missing 'api_key' in config") + + self.api_key = config["api_key"] + self.model_name = config.get("model_name", "embedding-2") + + try: + self.client = ZhipuAI(api_key=self.api_key) + except Exception as e: + raise ValueError(f"Error initializing ZhipuAI client: {e}") + + def __call__(self, input: Documents) -> Embeddings: + # Replace newlines, which can negatively affect performance. + input = [t.replace("\n", " ") for t in input] + all_embeddings = [] + print(f"Generating embeddings for {len(input)} documents") + + # Iterating over each document for individual API calls + for document in input: + try: + response = self.client.embeddings.create( + model=self.model_name, + input=document + ) + # print(response) + embedding = response.data[0].embedding + all_embeddings.append(embedding) + # print(f"Cost required: {response.usage.total_tokens}") + except Exception as e: + raise ValueError(f"Error generating embedding for document: {e}") + + return all_embeddings +``` + +**Initialization** + - `__init__(self, config=None)`: Initializes the ZhipuAIEmbeddingFunction object. Requires an `api_key` in the `config` dictionary. + - `model_name`: The name of the ZhipuAI model to use for generating embeddings. Defaults to "embedding-2". + +**Embedding Generation** + - `__call__(self, input: Documents) -> Embeddings`: Generates embeddings for a list of documents. The input documents should be a list of strings. The function returns a list of embeddings, where each embedding is a list of floats. + +--- + +## ๐Ÿƒ Example + +```python +# Import the necessary libraries +from zhipuai_embeddings import ZhipuAIEmbeddingFunction + +# Initialize the ZhipuAIEmbeddingFunction object +embedding_function = ZhipuAIEmbeddingFunction(config={ + "api_key": "YOUR_API_KEY", + "model_name": "YOUR_MODEL_NAME" +}) + +# Generate embeddings for a list of documents +documents = ["document 1", "document 2", "document 3"] +embeddings = embedding_function(documents) + +# The `embeddings` variable now contains a list of embeddings for the input documents. +``` + +--- + +## ๐Ÿ’ก Additional Notes + +- The cost of using the ZhipuAIEmbeddingFunction depends on the number of tokens in the input documents and the model used for generating embeddings. +- The ZhipuAIEmbeddingFunction can be used with any `chromadb` vector store. \ No newline at end of file diff --git a/src/vanna/ZhipuAI/__init__.py.md b/src/vanna/ZhipuAI/__init__.py.md new file mode 100644 index 00000000..32bc79e5 --- /dev/null +++ b/src/vanna/ZhipuAI/__init__.py.md @@ -0,0 +1,59 @@ +**ZhipuAI Language Models** + + **Table of Contents** + + - [ZhipuAI_Chat](#zhipuai_chat) + - [ZhipuAI_Embeddings](#zhipuai_embeddings) + - [ZhipuAIEmbeddingFunction](#zhipuaiembeddingfunction) + + +**ZhipuAI_Chat** + +The `ZhipuAI_Chat` class provides an interface to interact with the ZhipuAI chat assistant API. This class makes it easy to send messages to the API and receive responses. + +```python +from .ZhipuAI_Chat import ZhipuAI_Chat + +# Create a new ZhipuAI_Chat object +chat = ZhipuAI_Chat() + +# Send a message to the API +response = chat.send_message("Hello, ZhipuAI!") + +# Print the response from the API +print(response) +``` + +**ZhipuAI_Embeddings** + +The `ZhipuAI_Embeddings` class provides an interface to interact with the ZhipuAI embeddings API. This class makes it easy to generate embeddings for text data. + +```python +from .ZhipuAI_Embeddings import ZhipuAI_Embeddings, ZhipuAIEmbeddingFunction + +# Create a new ZhipuAI_Embeddings object +embeddings = ZhipuAI_Embeddings() + +# Generate embeddings for a piece of text +embeddings = embeddings.generate_embeddings("Hello, world!") + +# Print the embeddings +print(embeddings) +``` + +**ZhipuAIEmbeddingFunction** + +The `ZhipuAIEmbeddingFunction` class is a helper class that can be used to generate embeddings for text data using a specific embedding function. + +```python +from .ZhipuAI_Embeddings import ZhipuAIEmbeddingFunction + +# Create a new ZhipuAIEmbeddingFunction object +embedding_function = ZhipuAIEmbeddingFunction("average") + +# Generate embeddings for a piece of text +embeddings = embedding_function.generate_embeddings("Hello, world!") + +# Print the embeddings +print(embeddings) +``` \ No newline at end of file diff --git a/src/vanna/__init__.py.md b/src/vanna/__init__.py.md new file mode 100644 index 00000000..74434068 --- /dev/null +++ b/src/vanna/__init__.py.md @@ -0,0 +1,133 @@ +## ๐Ÿ“˜ Vanna.AI Python SDK Documentation + +This document provides a comprehensive guide to using the Vanna.AI Python SDK. + +### Table of Contents + +- [Introduction](#introduction) +- [Installation](#installation) +- [API Reference](#api-reference) + - [Authentication](#authentication) + - [Model Management](#model-management) + - [Training Data Management](#training-data-management) + - [SQL Generation](#sql-generation) + - [Meta Generation](#meta-generation) + - [Followup Question Generation](#followup-question-generation) + - [Question Generation](#question-generation) + - [Training](#training) +- [Example Usage](#example-usage) +- [Support](#support) + +### Introduction + +The Vanna.AI Python SDK is a library that enables you to interact with the Vanna.AI API from within your Python code. With this SDK, you can: + +- Authenticate to the Vanna.AI API +- Manage models +- Manage training data +- Generate SQL queries +- Generate meta information about data +- Generate followup questions +- Generate questions +- Train models + +### Installation + +To install the Vanna.AI Python SDK, run the following command: + +``` +pip install vanna +``` + +### API Reference + +**Authentication** + +```python +from vanna.api import get_api_key + +api_key = get_api_key(email="my-email@example.com") +``` + +**Model Management** + +```python +from vanna.api import create_model, set_model + +create_model(model="my-model", db_type="postgres") +set_model(model="my-model") +``` + +**Training Data Management** + +```python +from vanna.api import add_sql, add_ddl, add_documentation, remove_sql + +add_sql(question="What is the average salary of employees?", sql="SELECT AVG(salary) FROM employees") +add_ddl(ddl="CREATE TABLE employees (id INT, name VARCHAR(255), salary INT)") +add_documentation(documentation="This is a table of employees.") +remove_sql(question="What is the average salary of employees?") +``` + +**SQL Generation** + +```python +from vanna.api import generate_sql + +sql = generate_sql(question="What is the average salary of employees?") +``` + +**Meta Generation** + +```python +from vanna.api import generate_meta + +meta = generate_meta(question="What is the average salary of employees?") +``` + +**Followup Question Generation** + +```python +from vanna.api import generate_followup_questions + +followup_questions = generate_followup_questions(question="What is the average salary of employees?", df=pd.DataFrame()) +``` + +**Question Generation** + +```python +from vanna.api import generate_questions + +questions = generate_questions() +``` + +**Training** + +```python +from vanna.api import train + +train(question="What is the average salary of employees?", sql="SELECT AVG(salary) FROM employees") +``` + +### Example Usage + +The following code snippet shows you how to use the Vanna.AI Python SDK to generate a SQL query from a natural language question: + +```python +from vanna.api import generate_sql + +question = "What is the average salary of employees?" +sql = generate_sql(question) + +print(sql) +``` + +### Support + +If you have any questions or need help using the Vanna.AI Python SDK, please reach out to our support team at support@vanna.ai. + +### Changelog + +**0.1.0** (2023-08-08) + +- Initial release \ No newline at end of file diff --git a/src/vanna/anthropic/__init__.py.md b/src/vanna/anthropic/__init__.py.md new file mode 100644 index 00000000..c6f4663e --- /dev/null +++ b/src/vanna/anthropic/__init__.py.md @@ -0,0 +1,58 @@ +## ๐Ÿค– Anthropic Chat ๐Ÿ’ฌ + +### Table of Contents ๐Ÿ“– + +- [Description](#description) +- [Requirements](#requirements) +- [Installation](#installation) +- [Usage](#usage) +- [Methods](#methods) +- [Examples](#examples) + +### Description ๐Ÿ“™ + +The `Anthropic_Chat` class provides a simple interface to interact with Anthropic's artificial intelligence chatbot service. + +### Requirements ๐Ÿ› ๏ธ + +- Python 3.6 or later +- Internet connection + +### Installation ๐Ÿ“ฅ + +```bash +pip install anthropic-chat +``` + +### Usage ๐Ÿ“š + +To use the `Anthropic_Chat` class, first create an instance of the class and provide your API key. + +```python +from anthropic_chat import Anthropic_Chat + +# Create an instance of the class +chat_client = Anthropic_Chat(api_key="YOUR_API_KEY") +``` + +### Methods ๐Ÿ“‹ + +The `Anthropic_Chat` class has the following methods: + +- `get_chat_response`: Sends a message to the chatbot and returns the response. + +### Examples ๐Ÿ’ก + +```python +# Send a message to the chatbot +response = chat_client.get_chat_response(message="Hello, world!") + +# Print the chatbot's response +print(response) +``` + +### Output ๐Ÿ’ฌ + +``` +Hello, world! I am an artificial intelligence chatbot. What can I help you with today? +``` \ No newline at end of file diff --git a/src/vanna/anthropic/anthropic_chat.py.md b/src/vanna/anthropic/anthropic_chat.py.md new file mode 100644 index 00000000..fd87559d --- /dev/null +++ b/src/vanna/anthropic/anthropic_chat.py.md @@ -0,0 +1,200 @@ +## Table of Contents + +- [Overview](#overview) +- [Initialization](#initialization) + - [Using a Custom Client](#using-a-custom-client) + - [Using the Default Client with Custom Parameters](#using-the-default-client-with-custom-parameters) +- [Message Formatting](#message-formatting) + - [System Message](#system-message) + - [User Message](#user-message) + - [Assistant Message](#assistant-message) +- [Prompt Submission](#prompt-submission) +- [Example Usage](#example-usage) + +## Overview + +The `Anthropic_Chat` class is a Vanna AI wrapper for the [Anthropic API](https://anthropic.com/docs/api). It allows you to easily send and receive messages from the Anthropic language model. + +## Initialization + +### Using a Custom Client + +You can pass an existing `anthropic.Anthropic` client to the `Anthropic_Chat` constructor. This is useful if you want to use a custom API key or model. + +```python +import os + +import anthropic + +from ..base import VannaBase + + +class Anthropic_Chat(VannaBase): + def __init__(self, client=None, config=None): + VannaBase.__init__(self, config=config) + + if client is not None: + self.client = client + return + + if config is None and client is None: + self.client = anthropic.Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY")) + return + + # default parameters - can be overrided using config + self.temperature = 0.7 + self.max_tokens = 500 + + if "temperature" in config: + self.temperature = config["temperature"] + + if "max_tokens" in config: + self.max_tokens = config["max_tokens"] + + if "api_key" in config: + self.client = anthropic.Anthropic(api_key=config["api_key"]) +``` + +### Using the Default Client with Custom Parameters + +You can also use the default client with custom parameters. This is useful if you want to use a different temperature or max tokens value. + +```python +import os + +from ..base import VannaBase + + +class Anthropic_Chat(VannaBase): + def __init__(self, config=None): + VannaBase.__init__(self, config=config) + + if config is None: + self.client = anthropic.Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY")) + return + + # default parameters - can be overrided using config + self.temperature = 0.7 + self.max_tokens = 500 + + if "temperature" in config: + self.temperature = config["temperature"] + + if "max_tokens" in config: + self.max_tokens = config["max_tokens"] + + if "api_key" in config: + self.client = anthropic.Anthropic(api_key=config["api_key"]) +``` + +## Message Formatting + +### System Message + +```python +def system_message(self, message: str) -> any: + return {"role": "system", "content": message} +``` + +### User Message + +```python +def user_message(self, message: str) -> any: + return {"role": "user", "content": message} +``` + +### Assistant Message + +```python +def assistant_message(self, message: str) -> any: + return {"role": "assistant", "content": message} +``` + +## Prompt Submission + +```python +def submit_prompt(self, prompt, **kwargs) -> str: + if prompt is None: + raise Exception("Prompt is None") + + if len(prompt) == 0: + raise Exception("Prompt is empty") + + # Count the number of tokens in the message log + # Use 4 as an approximation for the number of characters per token + num_tokens = 0 + for message in prompt: + num_tokens += len(message["content"]) / 4 + + if self.config is not None and "model" in self.config: + print( + f"Using model {self.config['model']} for {num_tokens} tokens (approx)" + ) + # claude required system message is a single filed + # https://docs.anthropic.com/claude/reference/messages_post + system_message = '' + no_system_prompt = [] + for prompt_message in prompt: + role = prompt_message['role'] + if role == 'system': + system_message = prompt_message['content'] + else: + no_system_prompt.append({"role": role, "content": prompt_message['content']}) + + response = self.client.messages.create( + model=self.config["model"], + messages=no_system_prompt, + system=system_message, + max_tokens=self.max_tokens, + temperature=self.temperature, + ) + + return response.content[0].text +``` + +## Example Usage + +```python +import os + +from ..base import VannaBase + + +class Anthropic_Chat(VannaBase): + def __init__(self, config=None): + VannaBase.__init__(self, config=config) + + if config is None: + self.client = anthropic.Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY")) + return + + # default parameters - can be overrided using config + self.temperature = 0.7 + self.max_tokens = 500 + + if "temperature" in config: + self.temperature = config["temperature"] + + if "max_tokens" in config: + self.max_tokens = config["max_tokens"] + + if "api_key" in config: + self.client = anthropic.Anthropic(api_key=config["api_key"]) + +# Create an `Anthropic_Chat` instance +chat = Anthropic_Chat( + config={ + "model": "text-vaughan-001", + "temperature": 0.3, + "max_tokens": 500, + } +) + +# Send a message to the assistant +response = chat.submit_prompt([ + chat.user_message("Hello, can you help me with a task?"), +]) + +# Print the assistant's response +print(response) +``` \ No newline at end of file diff --git a/src/vanna/base/__init__.py.md b/src/vanna/base/__init__.py.md new file mode 100644 index 00000000..ea1ea0fb --- /dev/null +++ b/src/vanna/base/__init__.py.md @@ -0,0 +1,43 @@ +## ๐Ÿ“ Table of Contents + +- [Introduction](#introduction) +- [Class: VannaBase](#class-vannabase) + +## ๐Ÿ“š Introduction + +This module provides a base class for vanna models. + +## ๐Ÿ Class: VannaBase + +```python +from .base import VannaBase +``` + +The `VannaBase` class is a base class for vanna models. It provides the following functionality: + +- A constructor that takes a `model` and a `name` as arguments. +- A `fit` method that fits the model to the data. +- A `predict` method that predicts the model's output for a given input. +- A `save` method that saves the model to a file. +- A `load` method that loads the model from a file. + +### Example Usage + +```python +import vanna + +# Create a vanna model. +model = vanna.VannaBase(model, name="my_model") + +# Fit the model to the data. +model.fit(data) + +# Predict the model's output for a given input. +predictions = model.predict(input) + +# Save the model to a file. +model.save("my_model.pkl") + +# Load the model from a file. +model = vanna.VannaBase.load("my_model.pkl") +``` \ No newline at end of file diff --git a/src/vanna/base/base.py.md b/src/vanna/base/base.py.md new file mode 100644 index 00000000..05ad7bd9 --- /dev/null +++ b/src/vanna/base/base.py.md @@ -0,0 +1,94 @@ +# Vanna AI Base Class for Local or Server-Side Deployment ๐Ÿค– + +## Table of Contents ๐Ÿ”— +- [Introduction](#introduction) +- [Class Methods](#methods) + - [Initialization](#initialization) + - [Getting Information](#getting-information) + - [Training](#training) + - [Generating Content](#generating-content) + - [Generate SQL](#generate-sql) + - [Generate Followup Questions](#generate-followup-questions) + - [Visualization](#visualization) + - [Database Connectivity](#database-connectivity) + - [Plotly Code](#plotly-code) + - [Extract Python Code](#extract-python-code) + - [Sanitize Plotly Code](#sanitize-plotly-code) + - [Connecting to Databases](#connect-to-database) + - [Training Plans](#training-plans) + - [Creating Plotly Figures](#plotly-figures) + +## Introduction ๐Ÿ“š + +The Vanna AI Base Class is the foundation for both local and server-side deployments of Vanna AI. It provides a consistent interface for interacting with the Vanna AI API, regardless of the deployment environment. + +## Class Methods ๐Ÿ“ + +### Initialization ๐Ÿ”ง +- `__init__(self, config=None)`: Initializes the Vanna AI Base Class with an optional configuration dictionary. + +### Getting Information โ„น๏ธ +- `get_similar_question_sql(self, question: str, **kwargs) -> list`: Retrieves similar questions and their corresponding SQL statements. +- `get_related_ddl(self, question: str, **kwargs) -> list`: Retrieves related DDL statements to a question. +- `get_related_documentation(self, question: str, **kwargs) -> list`: Retrieves related documentation to a question. + +### Training ๐Ÿ‹๏ธ +- `add_question_sql(self, question: str, sql: str, **kwargs) -> str`: Adds a question and its corresponding SQL query to the training data. +- `add_ddl(self, ddl: str, **kwargs) -> str`: Adds a DDL statement to the training data. +- `add_documentation(self, documentation: str, **kwargs) -> str`: Adds documentation to the training data. +- `get_training_data(self, **kwargs) -> pd.DataFrame`: Retrieves all the training data from the retrieval layer. +- `remove_training_data(id: str, **kwargs) -> bool`: Removes training data from the retrieval layer. +- `get_training_plan_generic(self, df) -> TrainingPlan`: Generates a training plan from an information schema dataframe. +- `get_training_plan_snowflake(self, filter_databases: Union[List[str], None] = None, filter_schemas: Union[List[str], None] = None, include_information_schema: bool = False, use_historical_queries: bool = True) -> TrainingPlan`: Generates a training plan by querying Snowflake's information schema and historical query data. + +### Generating Content ๐Ÿ“ +#### Generate SQL ๐Ÿ’ป +- `generate_sql(self, question: str, **kwargs) -> str`: Generates an SQL query that answers a question using an LLM. +- `generate_followup_questions(self, question: str, sql: str, df: pd.DataFrame, n_questions: int = 5, **kwargs) -> list`: Generates a list of followup questions that a user might ask about the results of an SQL query. +- `generate_questions(self, **kwargs) -> List[str]`: Generates a list of questions that a user might ask Vanna AI. +- `generate_summary(self, question: str, df: pd.DataFrame, **kwargs) -> str`: Generates a summary of the results of an SQL query. + +#### Visualization ๐Ÿ“Š +- `generate_plotly_code(self, question: str = None, sql: str = None, df_metadata: str = None, **kwargs) -> str`: Generates Plotly code to visualize the results of an SQL query. +- `get_plotly_figure(self, plotly_code: str, df: pd.DataFrame, dark_mode: bool = True) -> plotly.graph_objs.Figure`: Retrieves a Plotly figure from a dataframe and Plotly code. + +### Database Connectivity ๐Ÿ”Œ +- `connect_to_snowflake(self, account: str, username: str, password: str, database: str, role: Union[str, None] = None, warehouse: Union[str, None] = None)`: Connects to a Snowflake database. +- `connect_to_sqlite(self, url: str)`: Connects to a SQLite database. +- `connect_to_postgres(self, host: str = None, dbname: str = None, user: str = None, password: str = None, port: int = None)`: Connects to a Postgres database. +- `connect_to_mysql(self, host: str = None, dbname: str = None, user: str = None, password: str = None, port: int = None)`: Connects to a MySQL database. +- `connect_to_oracle(self, user: str = None, password: str = None, dsn: str = None)`: Connects to an Oracle database. +- `connect_to_bigquery(self, cred_file_path: str = None, project_id: str = None)`: Connects to a BigQuery database. +- `connect_to_duckdb(self, url: str, init_sql: str = None)`: Connects to a DuckDB database. +- `connect_to_mssql(self, odbc_conn_str: str)`: Connects to a Microsoft SQL Server database. + +### Plotly Code ๐Ÿ“Š +#### Extract Python Code ๐Ÿ +- `_extract_python_code(self, markdown_string: str) -> str`: Extracts Python code from a markdown string. + +#### Sanitize Plotly Code ๐Ÿงน +- `_sanitize_plotly_code(self, raw_plotly_code: str) -> str`: Removes the fig.show() statement from the plotly code. + +### Creating Plotly Figures ๐Ÿ–ผ๏ธ +- `get_plotly_figure(self, plotly_code: str, df: pd.DataFrame, dark_mode: bool = True) -> plotly.graph_objs.Figure`: Retrieves a Plotly figure from a dataframe and Plotly code. + +### Example Usage ๐Ÿ’ป + +```python +import vanna + +# Initialize the Vanna AI Base Class +vn = vanna.VannaBase() + +# Connect to a Snowflake database +vn.connect_to_snowflake( + account="myaccount", + username="myusername", + password="mypassword", + database="mydatabase", +) + +# Ask Vanna AI a question +question = "What are the top 10 customers by sales?" +sql, df, fig = vn.ask(question, print_results=True) +``` \ No newline at end of file diff --git a/src/vanna/chromadb/__init__.py.md b/src/vanna/chromadb/__init__.py.md new file mode 100644 index 00000000..839d1092 --- /dev/null +++ b/src/vanna/chromadb/__init__.py.md @@ -0,0 +1,65 @@ +## ChromaDB_VectorStore โœจ + +A vector store for ChromaDB. + +### Table of Contents + +- [Overview](#overview) +- [Usage](#usage) +- [Implementation Details](#implementation-details) + +### Overview + +This store allows you to store and retrieve vectors in ChromaDB. Vectors can be used for a variety of purposes, such as: + +- Search +- Recommendation +- Classification + +### Usage + +To use this store, you will need to create a `ChromaDB_VectorStore` object. You can do this by passing a `ChromaDB` object to the constructor. + +```python +from chromadb_vector import ChromaDB_VectorStore + +vector_store = ChromaDB_VectorStore(chromadb) +``` + +Once you have created a `ChromaDB_VectorStore` object, you can use it to store and retrieve vectors. To store a vector, you will need to use the `put` method. The `put` method takes two arguments: a key and a vector. + +```python +vector_store.put("my-key", [1, 2, 3]) +``` + +To retrieve a vector, you will need to use the `get` method. The `get` method takes one argument: a key. + +```python +vector = vector_store.get("my-key") +``` + +### Implementation Details + +The `ChromaDB_VectorStore` class uses a B-tree to store vectors. The B-tree is a balanced search tree that is optimized for storing and retrieving data. + +The `ChromaDB_VectorStore` class also uses a cache to improve performance. The cache is a temporary storage area that stores frequently accessed vectors. When a vector is requested, the cache is checked first to see if the vector is already stored in the cache. If the vector is not in the cache, it is loaded from the B-tree. + +### Example Usage + +The following code shows how to use the `ChromaDB_VectorStore` class to store and retrieve vectors. + +```python +from chromadb_vector import ChromaDB_VectorStore + +# Create a ChromaDB object +chromadb = ChromaDB() + +# Create a ChromaDB_VectorStore object +vector_store = ChromaDB_VectorStore(chromadb) + +# Store a vector +vector_store.put("my-key", [1, 2, 3]) + +# Retrieve a vector +vector = vector_store.get("my-key") +``` \ No newline at end of file diff --git a/src/vanna/chromadb/chromadb_vector.py.md b/src/vanna/chromadb/chromadb_vector.py.md new file mode 100644 index 00000000..e1965ede --- /dev/null +++ b/src/vanna/chromadb/chromadb_vector.py.md @@ -0,0 +1,144 @@ +## ๐Ÿ“– ChromaDB VectorStore Documentation + +### Table of Contents + +- [Overview](#overview) +- [Initialization](#initialization) +- [Embedding Generation](#embedding-generation) +- [Adding Training Data](#adding-training-data) + - [SQL Queries](#sql-queries) + - [DDL Statements](#ddl-statements) + - [Documentation](#documentation) +- [Retrieving Training Data](#retrieving-training-data) +- [Removing Training Data](#removing-training-data) +- [Removing Collections](#removing-collections) +- [Searching for Similar Training Data](#searching-for-similar-training-data) + - [Similar SQL Queries](#similar-sql-queries) + - [Related DDL Statements](#related-ddl-statements) + - [Related Documentation](#related-documentation) +- [Example Usage](#example-usage) + +### Overview + +The `ChromaDB_VectorStore` class provides a vectorized interface to a ChromaDB datastore, allowing for efficient search and retrieval of similar training data. It supports adding and retrieving SQL queries, DDL statements, and documentation. + +### Initialization + +```python +from chromadb_vectorstore import ChromaDB_VectorStore + +# Create a vector store +vector_store = ChromaDB_VectorStore() +``` + +### Embedding Generation + +The vector store uses an embedding function to generate vector representations of training data. The default embedding function uses a neural network to learn vector representations that capture the semantic meaning of the data. You can customize the embedding function by passing a different function to the constructor. + +### Adding Training Data + +#### SQL Queries + +```python +# Add a SQL query to the vector store +vector_store.add_question_sql( + question="What is the average salary of employees in the sales department?", + sql="SELECT AVG(salary) FROM employees WHERE department = 'Sales';", +) +``` + +#### DDL Statements + +```python +# Add a DDL statement to the vector store +vector_store.add_ddl( + ddl="CREATE TABLE employees (id INT, name TEXT, salary INT);", +) +``` + +#### Documentation + +```python +# Add documentation to the vector store +vector_store.add_documentation( + documentation=""" + The `employees` table stores information about employees, including their ID, name, and salary. + """, +) +``` + +### Retrieving Training Data + +```python +# Get all training data +training_data = vector_store.get_training_data() +``` + +### Removing Training Data + +```python +# Remove a training data item by its ID +vector_store.remove_training_data(id="my-training-data-id") +``` + +### Removing Collections + +```python +# Remove the "sql" collection +vector_store.remove_collection(collection_name="sql") +``` + +### Searching for Similar Training Data + +#### Similar SQL Queries + +```python +# Find similar SQL queries to "What is the average salary of employees in the sales department?" +similar_sql = vector_store.get_similar_question_sql( + question="What is the average salary of employees in the sales department?" +) +``` + +#### Related DDL Statements + +```python +# Find DDL statements related to "CREATE TABLE employees" +related_ddl = vector_store.get_related_ddl(question="CREATE TABLE employees") +``` + +#### Related Documentation + +```python +# Find documentation related to "employees" table +related_documentation = vector_store.get_related_documentation(question="employees") +``` + +### Example Usage + +```python +# Initialize the vector store +vector_store = ChromaDB_VectorStore() + +# Add training data +vector_store.add_question_sql( + question="What is the average salary of employees in the sales department?", + sql="SELECT AVG(salary) FROM employees WHERE department = 'Sales';", +) +vector_store.add_ddl( + ddl="CREATE TABLE employees (id INT, name TEXT, salary INT);", +) +vector_store.add_documentation( + documentation=""" + The `employees` table stores information about employees, including their ID, name, and salary. + """, +) + +# Get similar training data +similar_sql = vector_store.get_similar_question_sql( + question="What is the average salary of employees in the sales department?" +) +related_ddl = vector_store.get_related_ddl(question="CREATE TABLE employees") +related_documentation = vector_store.get_related_documentation(question="employees") + +# Use the training data to improve your search or recommendation models +``` \ No newline at end of file diff --git a/src/vanna/exceptions/__init__.py.md b/src/vanna/exceptions/__init__.py.md new file mode 100644 index 00000000..88b4bd34 --- /dev/null +++ b/src/vanna/exceptions/__init__.py.md @@ -0,0 +1,113 @@ +## Table of Contents ๐Ÿ”— + +- [ImproperlyConfigured](#ImproperlyConfigured) +- [DependencyError](#DependencyError) +- [ConnectionError](#ConnectionError) +- [OTPCodeError](#OTPCodeError) +- [SQLRemoveError](#SQLRemoveError) +- [ExecutionError](#ExecutionError) +- [ValidationError](#ValidationError) +- [APIError](#APIError) + + +## ImproperlyConfigured ๐Ÿ›‘ + +`ImproperlyConfigured` is raised when the configuration is incorrect. + +**Example usage:** + +```python +if not settings.REQUIRED_SETTING: + raise ImproperlyConfigured("REQUIRED_SETTING is not set") +``` + +## DependencyError ๐Ÿงฉ + +`DependencyError` is raised when a required dependency is missing. + +**Example usage:** + +```python +try: + import missing_dependency +except ImportError: + raise DependencyError("missing_dependency is not installed") +``` + +## ConnectionError ๐Ÿ”Œ + +`ConnectionError` is raised when unable to establish a connection. + +**Example usage:** + +```python +try: + client.connect() +except ConnectionError: + print("Unable to connect to the database.") +``` + +## OTPCodeError ๐Ÿ”‘ + +`OTPCodeError` is raised when the OTP code is invalid or cannot be sent. + +**Example usage:** + +```python +try: + send_otp(user.phone_number) +except OTPCodeError: + print("Invalid or expired OTP code.") +``` + +## SQLRemoveError ๐Ÿ—‘๏ธ + +`SQLRemoveError` is raised when unable to remove SQL. + +**Example usage:** + +```python +try: + delete_user(user.id) +except SQLRemoveError: + print("Unable to remove user from the database.") +``` + +## ExecutionError ๐Ÿšซ + +`ExecutionError` is raised when unable to execute code. + +**Example usage:** + +```python +try: + execute_query("SELECT * FROM users") +except ExecutionError: + print("Unable to execute query.") +``` + +## ValidationError โš ๏ธ + +`ValidationError` is raised for validation errors. + +**Example usage:** + +```python +try: + validate_form(request.POST) +except ValidationError as e: + print(e.messages) +``` + +## APIError ๐ŸŒ + +`APIError` is raised for API errors. + +**Example usage:** + +```python +try: + response = api_client.get("/users") +except APIError as e: + print(e.status_code, e.message) +``` \ No newline at end of file diff --git a/src/vanna/flask/__init__.py.md b/src/vanna/flask/__init__.py.md new file mode 100644 index 00000000..f28a6bd3 --- /dev/null +++ b/src/vanna/flask/__init__.py.md @@ -0,0 +1,157 @@ +## Vanna Flask App + +**Table of Contents:** + +- [Introduction](#introduction) +- [Installation](#installation) +- [Usage](#usage) + - [Initialization](#initialization) + - [Available Routes](#available-routes) +- [Configuration](#configuration) +- [Auth](#auth) +- [Cache](#cache) +- [Security](#security) +- [Example](#example) + +## Introduction + +Vanna Flask App is a Flask-based application that allows you to interact with a Vanna instance. Vanna is an AI-powered copilot for SQL queries. With Vanna Flask App, you can generate SQL queries, run them, and visualize the results. You can also train Vanna on your own data. + +## Installation + +To install Vanna Flask App, run the following command: + +``` +pip install vanna-flask +``` + +## Usage + +### Initialization + +To initialize a Vanna Flask app, create an instance of the `VannaFlaskApp` class: + +```python +from vanna_flask import VannaFlaskApp + +# Create a Vanna instance +vn = Vanna() + +# Create a Vanna Flask app +app = VannaFlaskApp(vn) +``` + +### Available Routes + +The following routes are available in Vanna Flask App: + +| Route | Method | Description | +|---|---|---| +| /auth/login | POST | Login | +| /auth/callback | GET | Callback after login | +| /auth/logout | GET | Logout | +| /api/v0/get_config | GET | Get the configuration of the app | +| /api/v0/generate_questions | GET | Generate questions | +| /api/v0/generate_sql | GET | Generate SQL | +| /api/v0/run_sql | GET | Run SQL | +| /api/v0/fix_sql | POST | Fix SQL | +| /api/v0/download_csv | GET | Download the results of a SQL query as a CSV file | +| /api/v0/generate_plotly_figure | GET | Generate a Plotly figure | +| /api/v0/get_training_data | GET | Get the training data | +| /api/v0/remove_training_data | POST | Remove training data | +| /api/v0/train | POST | Train Vanna | +| /api/v0/generate_followup_questions | GET | Generate followup questions | +| /api/v0/generate_summary | GET | Generate a summary of the results of a SQL query | +| /api/v0/load_question | GET | Load a question from the cache | +| /api/v0/get_question_history | GET | Get the question history | + +## Configuration + +The following configuration options are available in Vanna Flask App: + +| Option | Default Value | Description | +|---|---|---| +| allow_llm_to_see_data | False | Whether to allow the LLM to see data | +| logo | https://img.vanna.ai/vanna-flask.svg | The logo to display in the UI | +| title | Welcome to Vanna.AI | The title to display in the UI | +| subtitle | Your AI-powered copilot for SQL queries. | The subtitle to display in the UI | +| show_training_data | True | Whether to show the training data in the UI | +| suggested_questions | True | Whether to show suggested questions in the UI | +| sql | True | Whether to show the SQL input in the UI | +| table | True | Whether to show the table output in the UI | +| csv_download | True | Whether to allow downloading the table output as a CSV file | +| chart | True | Whether to show the chart output in the UI | +| redraw_chart | True | Whether to allow redrawing the chart | +| auto_fix_sql | True | Whether to allow auto-fixing SQL errors | +| ask_results_correct | True | Whether to ask the user if the results are correct | +| followup_questions | True | Whether to show followup questions | +| summarization | True | Whether to show summarization | + +## Auth + +Vanna Flask App supports authentication using the `AuthInterface` interface. You can implement your own authentication method by creating a class that implements the `AuthInterface` interface. + +To use your own authentication method, pass it to the `VannaFlaskApp` constructor: + +```python +from vanna_flask import VannaFlaskApp +from my_auth import MyAuth + +# Create a Vanna instance +vn = Vanna() + +# Create a MyAuth instance +auth = MyAuth() + +# Create a Vanna Flask app +app = VannaFlaskApp(vn, auth=auth) +``` + +## Cache + +Vanna Flask App uses a cache to store data. The cache is implemented using the `Cache` interface. You can implement your own cache by creating a class that implements the `Cache` interface. + +To use your own cache, pass it to the `VannaFlaskApp` constructor: + +```python +from vanna_flask import VannaFlaskApp +from my_cache import MyCache + +# Create a Vanna instance +vn = Vanna() + +# Create a MyCache instance +cache = MyCache() + +# Create a Vanna Flask app +app = VannaFlaskApp(vn, cache=cache) +``` + +## Security + +Vanna Flask App uses the following security measures: + +- **HTTPS:** All traffic is encrypted using HTTPS. +- **CSRF protection:** All forms are protected against CSRF attacks. +- **XSS protection:** All output is escaped to prevent XSS attacks. + +## Example + +The following code shows how to use Vanna Flask App: + +```python +from vanna_flask import VannaFlaskApp + +# Create a Vanna instance +vn = Vanna() + +# Create a Vanna Flask app +app = VannaFlaskApp(vn) + +# Run the app +app.run() +``` + +## Conclusion + +Vanna Flask App is a powerful tool for interacting with Vanna. With Vanna Flask App, you can easily generate SQL queries, run them, and visualize the results. You can also train Vanna on your own data. \ No newline at end of file