Skip to content

Commit

Permalink
iter
Browse files Browse the repository at this point in the history
  • Loading branch information
@glemaitre
glemaitre committed Apr 17, 2024
1 parent 402b127 commit a7223e8
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion scripts/configuration.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@
GALLERY_LEXICAL_RETRIEVER_PATH = "../models/gallery_lexical_retrieval.joblib"

# Parameters for the scraper
CHUNK_SIZE = 1_500
# gte-large will truncate at 512 tokens so let's limit to 700 characters
CHUNK_SIZE = 700
CHUNK_OVERLAP = 10

# Sentence transformer model
Expand Down

0 comments on commit a7223e8

Please sign in to comment.