Implement AMBIG_MULTIPLE for llvm, Rust, go, and the vmops dump #486
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Now the VM IR ops point to retlist entries, rather than carrying their own endid sets. This means we can use the same indexing to de-dup them for every VM-based format.
The generated code looks like this:
```go
package fsm_fsm
var ret0 []uint = []uint{1}
var ret1 []uint = []uint{2}
var ret2 []uint = []uint{1, 2}
var ret3 []uint = []uint{0, 1, 2}
func fsm_Match(data string) (bool, []uint) {
var idx = ^uint(0)
if idx++; idx >= uint(len(data)) {
return true, ret0
}
if data[idx] == 'a' {
goto l3
}
if data[idx] == 'b' {
goto l2
}
if data[idx] != 'c' {
return false, nil
}
l0: // e.g. "c"
if idx++; idx >= uint(len(data)) {
return true, ret2
}
if data[idx] <= '`' {
return false, nil
}
if data[idx] <= 'b' {
goto l1
}
if data[idx] == 'c' {
goto l0
}
{
return false, nil
}
l1: // e.g. "aa"
if idx++; idx >= uint(len(data)) {
return true, ret1
}
if data[idx] <= '`' {
return false, nil
}
if data[idx] <= 'c' {
goto l1
}
{
return false, nil
}
l2: // e.g. "b"
if idx++; idx >= uint(len(data)) {
return true, ret3
}
if data[idx] == 'a' {
goto l1
}
if data[idx] == 'b' {
goto l2
}
if data[idx] == 'c' {
goto l1
}
{
return false, nil
}
l3: // e.g. "a"
if idx++; idx >= uint(len(data)) {
return true, ret1
}
if data[idx] == 'a' {
goto l1
}
if data[idx] == 'b' {
goto l2
}
if data[idx] == 'c' {
goto l1
}
{
return false, nil
}
}
```
I've handled AMBIG_NONE here, but I haven't distinguished the other ambig modes. Other than AMBIG_NONE, the other modes are all presented as an array of ids, even if it's just a single element. That's because I don't see any reason to give these specialised APIs for the current use-cases for this generated code, which is supposed to be a direct representation of our VM opcodes.
It's a bit rough, this isn't what I want to end up with, but I wanted to commit this as a waypoint.
This means the phi instruction now only carries an array index. Many thanks to @mcy for advice and patient help here.
…e `poison`. Thanks to @mcy for this.
This allows callers to default the codegen for accepting states (in particular outputting the values for endids) independently of commenting caller-specific meanings for the IDs.
Originally I'd intended this as a demonstration of how various applications can handle ambiguities differently. But now we have library support for AMBIG_MULTIPLE, I think this is just confusing.
The idea here is just to trim down %rt to:
```
%rt = type { ptr, i64 }
```
where we clearly don't need a uint64_t's count of unique ids.
I've purposefully not done the same for single-id interfaces. I don't want to mix success/failure with an id *value*, because the values are opaque (i.e. the meaning of an id value is the responsibility of the caller). Whereas here for AMBIG_MULTIPLE I'm mixing success/failure with the count, not with the id values.
Suggested by @mcy, thank you.
The generated code looks like this:
```rust
pub fn fsm_main(input: &str) -> Option<&'static [u32]> {
use Label::*;
static RET0: [u32; 1] = [1];
static RET1: [u32; 1] = [2];
static RET2: [u32; 3] = [0, 1, 2];
let mut bytes = input.bytes();
pub enum Label {
Ls, L0,
}
let mut l = Ls;
loop {
match l {
Ls => { // e.g. ""
let c = match bytes.next() {
None => return Some(&RET0) /* "x?" */,
Some(c) => c,
};
if c != b'x' { return None }
let c = match bytes.next() {
None => return Some(&RET2) /* "x", "x?", "x+" */,
Some(c) => c,
};
if c != b'x' { return None }
l = L0; continue;
}
L0 => { // e.g. "xx"
let c = match bytes.next() {
None => return Some(&RET1) /* "x+" */,
Some(c) => c,
};
if c != b'x' { return None }
l = L0; continue;
}
}
}
}
```
This gives better control over whitespace and punctuation between the hooks. For example we can output "<accept>, <comment>\n" with a comma between, and that sits more nicely for single-line comments. Previously these had to be "<accept> <comment>,\n"
silentbicycle
approved these changes
Aug 15, 2024
There was a problem hiding this comment.
This makes sense to me overall.
I'm not familiar with Go or LLVM IR syntax, but I think I get the gist.
I will be working on integrating early endid matching into codegen soon, so I will spend a lot of time with the C interfaces, but any changes necessary for that can go in later PRs.
include/fsm/print.h
Outdated
| * but simply not yet implemented, where fsm_print() will print a message | ||
| * to stderr and exit. | ||
| * | ||
| * The code generation for the typical case of matching input require the FSM |
There was a problem hiding this comment.
Minor typo: "requires" (plural)
tfreiberg-fastly
approved these changes
Aug 16, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR:
AMBIG_MULTIPLEfor various languages (go, rust, llvm especially)(struct fsm_hooks).comment()(struct fsm_options).commentsHere's how the generated
AMBIG_MULTIPLEcode looks for the following example:for go:
Rust:
and (my current favourite) for llvm, with many thanks to @mcy for the guidance:
And the vmops structures also now carry the endids for all ambig modes except for
AMBIG_NONE: