[Feat] Add LiteLLMEmbeddings - Support SemanticChunking through LiteLLM #154
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…run timings (chonkie-ai#156) * [DOCS] Benchmarking update (chonkie-ai#145) * Add wiki 500k benchmark results * Update benchmarks * bahut tej hai chonkie bhai * blah blah --------- Co-authored-by: Bhavnick Minhas <[email protected]> * Update benchmarks with corrected memory usage and size metrics --------- Co-authored-by: Shreyash Nigam <[email protected]>
bhavnicksm
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
added litellm support to identify and embed with any Huggingface embedding model that has feature extraction
added the same to the registry to support AutoEmbeddings.
added tests for the same.
key things to notice:
timeout is dependent on model size, since the model needs to be loaded onto the local hardware first.
context length, dimensions, and such measures are dependent on the model
token_counter is a callable from litellm, which would also need time to load.
currently only supports huggingface models. Litellm can support more models such as voyage, mistral, etc, but the API keys should be given in parameters.