Matching non-backslash characters

[billminckler]

Hi Everyone,
I'm trying to match file paths under windows using the following expression:
(?i)(^\w+:\\){1}Users\\[^\\]+?\\AppData\\Local\\.*

but RegexBuilder::new fails with:

thread 'main' panicked at 'Failed to compile regular expression: Syntax(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
regex parse error:
    (?i)(^\w+:\\){1}Users\\[^\\]+?\\AppData\\Local\\.*
                           ^^^^^
error: pattern can match invalid UTF-8
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I don't understand how UTF8 is involved since the methods for Regex take utf8 strings.
As I see it the pattern [^\\]+? should match any non-backslash character. if that matches invalid utf8, than wouldn't . as well?

I've attached a sample program.

Thanks,
Bill

main.rs.txt

[billminckler]

Turns out I had unicode disabled (from a different project I was working on)

[BurntSushi]

than wouldn't . as well?

Yes. If you try compiling (?i)(^\w+:\\){1}Users\\[\\]+?\\AppData\\Local\\.* without any other changes, then you'll get this error:

regex parse error:
    (?i)(^\w+:\\){1}Users\\[\\]+?\\AppData\\Local\\.*
                                                   ^
error: pattern can match invalid UTF-8

Let's take a look at how you're building the regex:

    let r1 = RegexBuilder::new(REGEX_TEST)
        .unicode(false)
        .build()
        .expect("Failed to compile regular expression");

When unicode is disabled (which you could also do with (?-u) or (?i-u) in your case), then everything in the regex that was Unicode aware becomes a simple operation on bytes. So instead of [^a] matching any Unicode scalar value except for a, it now matches any byte except for a. Similarly, . stops meaning "match any Unicode scalar value except for \n," and now means "match any byte except for \n."

If disabling Unicode wasn't intentional, then, well, don't disable it and your regex will work. IIRC, Windows paths are Unicode aware. And if they're in a &str, then they are guaranteed to be valid UTF-8. So a Unicode-aware regex is probably what you want.

If disabling Unicode was intentional, then these errors are correct and are preventing you from making mistakes. In particular, since you are searching a &str, a regex that can match invalid UTF-8 will never match anything inside a &str because &str is guaranteed to be valid UTF-8. Or perhaps even worse, your regex might match in the middle of an encoded codepoint and return an offset into it, which would subsequently panic if you tried to slice your string with those offsets.

If you do want to be able to match arbitrary bytes like this, then the correct answer is to use the bytes submodule, which is mentioned in the documentation for opting out of Unicode support. The syntax section of the bytes module goes into a bit more detail. But in your code, all that's needed is to have use regex::bytes::RegexBuilder; and call is_match with file_name.as_bytes() instead of &file_name.

[billminckler]

Nice explanation. Thanks. I did not intend to disable unicode. I re-used the project from a previous issue I was working on and forgot to remove the unicode(false).