Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Filter/square brackets #849

Merged
merged 5 commits into from
Apr 19, 2024
Merged

Filter/square brackets #849

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
b28da0e
Add a test for pattern-matching with square brackets
danielhuppmann Apr 18, 2024
a4de52b
Add square brackets to `escape_regexp()`
danielhuppmann Apr 18, 2024
b86f5e1
Simplify tests, add test for curly brackets
danielhuppmann Apr 19, 2024
c85d23d
Simplicy regexp-escaping
danielhuppmann Apr 19, 2024
6b6574a
Add `^` and `?` to escaped regexp characters
danielhuppmann Apr 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 9 additions & 11 deletions pyam/str.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
import pandas as pd
from pandas.api.types import is_list_like

REGEXP_CHARACTERS = r".+()[]{}|$"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not that I'd ever seen it but we can add the caret symbol as well ^, especially since you have the dollar sign:

Suggested change
REGEXP_CHARACTERS = r".+()[]{}|$"
REGEXP_CHARACTERS = r".+()[]{}|$^"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this comment - went through the list of regex-characters at https://docs.python.org/3/library/re.html and also added ?...



def concat_with_pipe(x, *args, cols=None):
"""Concatenate a list or pandas.Series using ``|``, drop None or numpy.nan"""
Expand Down Expand Up @@ -123,21 +125,17 @@ def reduce_hierarchy(x, depth):
"""
_x = x.split("|")
depth = len(_x) + depth - 1 if depth < 0 else depth
return "|".join(_x[0 : (depth + 1)])
return "|".join(_x[0: (depth + 1)])


def escape_regexp(s):
"""Escape characters with specific regexp use"""
return (
str(s)
.replace("|", "\\|")
.replace(".", r"\.") # `.` has to be replaced before `*`
.replace("*", ".*")
.replace("+", r"\+")
.replace("(", r"\(")
.replace(")", r"\)")
.replace("$", "\\$")
)
s = str(s)
for c in REGEXP_CHARACTERS:
s = s.replace(c, "\\" + c)
# pyam uses `*` as wildcard, replace with `.*` for regex
s = s.replace("*", ".*")
return s


def is_str(x):
Expand Down
10 changes: 7 additions & 3 deletions tests/test_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,9 +78,13 @@ def test_pattern_match_dot():
assert (obs == [False, True]).all()


def test_pattern_match_brackets():
data = pd.Series(["foo (bar)", "foo bar"])
values = ["foo (bar)"]
@pytest.mark.parametrize(
"bracket", ("(bar)", "[bar]", "{2}")
)
def test_pattern_match_brackets(bracket):
s = f"foo {bracket}"
data = pd.Series([s, "foo bar"])
values = [s]

obs = pattern_match(data, values)
assert (obs == [True, False]).all()
Expand Down