Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Experimental: Add JobList constraint parser and support constraint query strings in flux jobs -f #5711

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Prev Previous commit
Next Next commit
python: add convert_values member to ConstraintParser class
Problem: Values are always returned as strings by the ConstraintParser
parser, but there are times when another type may be more
suitable. Additionally, there is currently no way to reduce a set
of values to a single element if this is required or beneficial in
the result.

Add a convert_values mapping to the ConstraintParser class. If an
operator is in this dictionary, then the values of a term are passed
to the provided callable, which should return a new list of values
as a result.
grondo committed Aug 12, 2024
commit 95386e5d905a8f62c10d764452a74fc686e07abb
22 changes: 19 additions & 3 deletions src/bindings/python/flux/constraint/parser.py
Original file line number Diff line number Diff line change
@@ -231,6 +231,16 @@ class default operator may optionally be substituted, e.g. "operand"
to split values of that operator. For instance ``{"op": ","}``
would autosplit operator ``op`` values on comma.

convert_values (dict): A mapping of operator name to callable which
should take a single list argument containing the values from
the obtained from the current term for the operator after the
operator_map and split_values operations have been applied. The
callable should return a new list of values. This can be used
to convert values to a new type or to combine multiple values
into a single element, e.g. ::

convert_values = {"ints": lambda args: [int(x) for x in args]}

combined_terms (set): A set of operator terms whose values can be
combined when joined with the AND logical operator. E.g. if
"test" is in ``combined_terms``, then
@@ -284,6 +294,10 @@ class MyConstraintParser(ConstraintParser):
# Combined terms
combined_terms = set()

# Mapping of operator name to value conversion function.
# E.g. { "integer": lambda args: [ int(x) for x in args ] }
convert_values = {}

def __init__(
self, lexer=None, optimize=True, debug=False, write_tables=False, **kw_args
):
@@ -408,10 +422,12 @@ def p_expression_token(self, p):
f"invalid character '{invalid}' in operator '{op}:'"
)

values = [value]
if op in self.split_values:
p[0] = {op: value.split(self.split_values[op])}
else:
p[0] = {op: [value]}
values = value.split(self.split_values[op])
if op in self.convert_values:
values = self.convert_values[op](values)
p[0] = {op: values}

def p_quoted_token(self, p):
"""