Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Redis Storage and add JSON support #78

Merged
merged 37 commits into from
Nov 21, 2023
Merged

Conversation

tylerhutcherson
Copy link
Collaborator

PR introduces a new BaseStorage class, subclassed by HashStorage and JsonStorage, to handle the underlying data structures (as well as I/O) for Redis. Includes:

  • New storage classes and type enums
  • New key_separator param for better use-customizability when constructing redis keys
  • Updated documentation and new user guide
  • Updated docstrings and examples

@tylerhutcherson tylerhutcherson added the enhancement New feature or request label Nov 17, 2023
@codecov-commenter
Copy link

codecov-commenter commented Nov 17, 2023

Codecov Report

Attention: 56 lines in your changes are missing coverage. Please review.

Comparison is base (2f23e10) 80.55% compared to head (5206960) 81.07%.

Files Patch % Lines
redisvl/storage.py 74.49% 38 Missing ⚠️
redisvl/index.py 90.80% 16 Missing ⚠️
redisvl/utils/utils.py 77.77% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #78      +/-   ##
==========================================
+ Coverage   80.55%   81.07%   +0.51%     
==========================================
  Files          17       18       +1     
  Lines         967     1189     +222     
==========================================
+ Hits          779      964     +185     
- Misses        188      225      +37     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@tylerhutcherson
Copy link
Collaborator Author

@Spartee good to go from my end.

Copy link
Contributor

@Spartee Spartee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nearly there. Looks good to me. Running tests now.

redisvl/index.py Outdated Show resolved Hide resolved
redisvl/schema.py Outdated Show resolved Hide resolved
docs/api/searchindex.rst Outdated Show resolved Hide resolved
@@ -371,61 +486,94 @@ def delete(self, drop: bool = True):
self._redis_conn.ft(self._name).dropindex(delete_documents=drop) # type: ignore

@check_connected("_redis_conn")
@check_modules_present("_redis_conn")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use lru_cache to cache the calls to this within an invocated client?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm i've not done that. are you saying cache the module list client side? I was going to recommend some kind of private attribute on the class like _modules that gets populated first time you run it. Then next time the check is called, it looks at this attribute first, and then executes the redis command. Something like that. Lmk what you're thinking? Feels like a later optimization to me

@Spartee Spartee self-requested a review November 21, 2023 23:50
Copy link
Contributor

@Spartee Spartee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Spartee Spartee merged commit 6a2809b into main Nov 21, 2023
18 checks passed
@Spartee Spartee deleted the json-support-sam-edit branch November 21, 2023 23:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants