Make search queries more coherent

[alexgarel]

We are trying to abstract out complexity of Elasticsearch through a configuration and Lucene query language.

Right now this abstraction is quite leaky. For example if you declare labels as a text_lang field, you should query, eg. labels.en:"my query" to reach it. While you also specify which language you want to query with. There are a number of other problems of this type.

Most of those problems can be solved by using Lucene to modify the request on the fly, based upon field deficinition

To be fixed

Preview Give feedback

1. replace text_lang field with right expression on the fly (eg labels: --> labels.en:)
2. treat phrase match as word match (use all text search fields)
3. explore using the synonyms feature of elasticsearch instead of injecting synonyms / or replace in terms query
4. Add a fallback language for queries on certain fields (eg. product_name) corresponding to main product language
5. Raise or at least warn if using an unknown field in a query (should catch labels_tags:en:fair-trade)
6. labels_tags:"en:fair-trade" OR labels_tags:"en:eu-organic" should work (currently failing)

[alexgarel]

@raphael0202 this should be discussed to see what's best (next month)

[alexgarel]

I started working on this, to better handle text and avoid having a big index because of synonyms.

Here is my plan:

• I will use synonyms files (I prefer to synonyms sets, to keep compat with OpenSearch) generated from taxonomies and we will use a token filter on query analyzer to use synonyms on search
• I will try to rewrite the query for text matches at lucene language level transforming the simple word and phrase to more complex expressions to match against the various fields with eventual boosting. An alternative being to rewrite the final ES query (if for example, it's simpler to use a multimatch)

I'm currently working on generating synonyms files and integrating them in query analyzer.