Skip to content

Commit

Permalink
feat(query): added support for numeric queries
Browse files Browse the repository at this point in the history
Numeric queries can be specified by including two dots ``..`` in the beginning, middle, or end of the value. Dots in the beginning let you specifiy a minimum e.g. ``1..``, dots and the end let you specify a maximum e.g. ``..10``, and dots in the middle let you specify a range e.g. ``1..10``.

Fixes #199
  • Loading branch information
Moe bot authored and jtpavlock committed Dec 20, 2022
1 parent 7a9f6a2 commit fac29a1
Show file tree
Hide file tree
Showing 3 changed files with 83 additions and 12 deletions.
55 changes: 44 additions & 11 deletions docs/query.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ The query must be in the format ``field:value`` where ``field`` is, by default,

By default, tracks will be returned by the query, but you can choose to return albums by using the ``-a, --album`` option, or you can return extras using the ``-e, --extra`` option.

Normal Queries
==============
If you would like to specify a value with whitespace or multiple words, enclose the
term in quotes.

Expand All @@ -28,7 +30,41 @@ term in quotes.
For powershell users, it's necessary to use ``"`` as the outer quotes.

.. note::
When querying for a field that supports multiple values, query for one term per value. For example, to query for tracks with the genres 'hip hop' and 'pop', use:

.. code-block:: bash
"'genre:hip hop' genre:pop"
Numeric Range Queries
=====================
Queries on numeric fields can specify an acceptable range. To do this, specify the range by using two dots ``..`` in the beginning, middle, or end of the value. Dots in the beginning let you specifiy a minimum e.g. ``1..``, dots and the end let you specify a maximum e.g. ``..10``, and dots in the middle let you specify a range e.g. ``1..10``.

For instance, the following will query for any albums released between 2010 and 2020:

.. code-block::
"a:year:2010..2020"
Whereas this query will find any albums released prior to 2020:

.. code-block::
"a:year:..2020"
And finally, this query will find any albums released after 2010:

.. code-block::
"a:year:2010.."
.. note::

Query ranges are *inclusive* i.e. items matching the minimum or maximum value will also be included.

SQL Like Queries
================
`SQL LIKE <https://www.w3schools.com/sql/sql_like.asp>`_ query syntax is used for normal queries, which means
the ``_`` and ``%`` characters have special meaning:

Expand All @@ -41,6 +77,8 @@ To match these special characters as normal, use ``/`` as an escape character.
'title:100/%'
Regular Express Queries
=======================
The value can also be a regular expression. To enforce this, use two colons
e.g. ``field::value.*``

Expand All @@ -54,7 +92,12 @@ As a shortcut to matching all entries, use ``*`` as the term.
'*'
Finally, you can also specify any number of terms.
.. tip::
Normal queries may be faster when compared to regular expression queries. If you are experiencing performance issues with regex queries, see if you can make an equivalent normal query using the ``%`` and ``_`` wildcard characters.

Multiple Query Terms
====================
You can also specify any number of terms.
For example, to match all Wu-Tang Clan tracks that start with the letter 'A', use:

.. code-block:: bash
Expand All @@ -64,15 +107,5 @@ For example, to match all Wu-Tang Clan tracks that start with the letter 'A', us
.. note::
When using multiple terms, they are joined together using AND logic, meaning all terms must be true to return a match.

.. note::
When querying for a field that supports multiple values, query for one term per value. For example, to query for tracks with the genres 'hip hop' and 'pop', use:

.. code-block:: bash
"'genre:hip hop' genre:pop"
.. tip::
Fields of different types can be mixed and matched in a query string. For example, the query ``--extras 'album:The College Dropout' e:path:%jpg$`` will return any extras with the 'jpg' file extension belonging to the album titled 'The College Dropout'.

.. tip::
Normal queries may be faster when compared to regular expression queries. If you are experiencing performance issues with regex queries, see if you can make an equivalent normal query using the ``%`` and ``_`` wildcard characters.
9 changes: 8 additions & 1 deletion moe/query.py
Original file line number Diff line number Diff line change
Expand Up @@ -157,12 +157,19 @@ def _create_filter_expression(field_type: str, field: str, separator: str, value
attr = _get_field_attr(field, field_type)

if separator == ":":
if match := re.fullmatch(r"(?P<low>\d*)..(?P<high>\d*)", value):
if match["low"] and match["high"]:
return sa.and_(attr >= match["low"], attr <= match["high"])
if match["low"]:
return attr >= match["low"]
if match["high"]:
return attr <= match["high"]

if str(attr).endswith(".path"):
return attr == Path(value)

# normal string match query - should be case insensitive
return attr.ilike(value, escape="/")

elif separator == "::":
try:
re.compile(value)
Expand Down
31 changes: 31 additions & 0 deletions tests/test_query.py
Original file line number Diff line number Diff line change
Expand Up @@ -271,3 +271,34 @@ def test_custom_list_field(self, tmp_session):
assert query(tmp_session, "a:blah:album t:blah:track e:blah:extra", "album")
assert query(tmp_session, "a:blah:1 e:blah:2 t:blah:3", "album")
assert query(tmp_session, "t:blah:3 t:blah:track", "album")

def test_numeric_query_range(self, tmp_session):
"""We can query for a range."""
tmp_session.add(track_factory(track_num=2))
tmp_session.flush()

assert query(tmp_session, "t:track_num:1..3", "track")

def test_numeric_lt(self, tmp_session):
"""Numeric queries without a lower bound test less than the upper bound."""
tmp_session.add(track_factory(track_num=1))
tmp_session.flush()

assert query(tmp_session, "t:track_num:..3", "track")

def test_numeric_gt(self, tmp_session):
"""Numeric queries without an upper bound test greater than the lower bound."""
tmp_session.add(track_factory(track_num=2))
tmp_session.flush()

assert query(tmp_session, "t:track_num:1..", "track")

def test_numeric_query_inclusive(self, tmp_session):
"""Numeric queries should be inclusive of their upper or lower bounds."""
tmp_session.add(track_factory(track_num=1))
tmp_session.add(track_factory(track_num=3))
tmp_session.flush()

assert len(query(tmp_session, "t:track_num:1..3", "track")) == 2
assert len(query(tmp_session, "t:track_num:1..", "track")) == 2
assert len(query(tmp_session, "t:track_num:..3", "track")) == 2

0 comments on commit fac29a1

Please sign in to comment.