why are there two different modules for &str and &[u8] searching instead of one generic API?

[BurntSushi]

Originally asked by @johnmcilwain:

Hi, (rust beginner here) - hope this isn't a dumb question :-)
I have always wondered why there is a separate areas for regex and bytes (strings vs bytes?)?
Since rust supports generics, why 2 whole ?modules?
thanks!

[BurntSushi]

This is a good question. The short answer is that I personally find that generics makes APIs more complex, both in terms of understanding and for use. For use is easy to answer: it can negatively impact inference. It's much nicer, for example, to accept an &str instead of an AsRef<[u8]>. For understanding, it's a bit more subtle and reasonable people can disagree. But in my experience, the existence of type parameters requires reasoning through an abstraction instead of reasoning concretely with specific types. For something like the regex crate, I tried hard to optimize for ease of understanding due to how many folks of varying experience levels are likely to use the crate. regex might be one of the first libraries that a Rust programmer uses, so it should be as easy and straight-forward as possible.

There is also a longer answer which is that at a technical level, it's actually a little trickier than you might imagine to craft a single generic API that is convenient to use. I elaborated a little bit on this in the regex 1.0 RFC a number of years ago. In particular, one of the "convenient" aspects of the regex API is that the Match and Captures type both borrow the haystack that is being searched, which in turn makes it possible to define convenience methods on those types for accessing both offsets and the actual sub-string of the haystack that was matched. The lower level regex-automata crate, for example, uses Match and Captures types that don't borrow the haystack and thus can only report offsets. This makes the types simpler and more flexible, but more cumbersome to use in common cases.

I think the bottom line here is that, in my experience, the regex::bytes sub-module is a niche (but extremely important) API. Most Rust programmers probably aren't using that API. Most probably just want the &str APIs and I did not want to make the common case more complicated in order to support a niche use case.

I want to be very clear that reasonable people can disagree here, and the API design I chose reflects a fair bit of subjective judgment on my part that tries to balance a lot of things:

• My understanding of what other people find difficult to understand.
• My understanding of what makes an API simple to use.
• The extent to which the API should offer conveniences.

And probably other unstated things as well.

[johnmcilwain]

Thanks for the info. In my case, I ended up using regex::bytes, but I really don't know why I ended up with it.
I was searching the results of a HTTP request, could be text or an image, and looking for a matching string or set of bytes.
I was never sure if I should implement both. My initial thought was that I needed to use both bytes and &str so had duplicated a lot of code to do that. It was a long time ago, but for some reason I just ended up using bytes. Seemed to cover both cases.
Was, and still am a total beginner at rust, but did look to your create as one of the first things I knew I needed!!! Thanks for creating it.

[BurntSushi]

Yeah, you can use regex::bytes::Regex even with &str. You just have to use haystack.as_bytes() as an argument to the search routines. Of course, the types you get back will give you &[u8] instead. But if you only need the offsets, then it's a totally valid approach.

(Internally, the regex crate just converts &str to &[u8] almost immediately. The &str API is really just a facade over a meta::Regex.)