Skip to content
/ lemon-rs Public

LALR(1) parser generator for Rust based on Lemon + SQL parser

License

Notifications You must be signed in to change notification settings

gwenn/lemon-rs

Repository files navigation

Build Status Latest Version Docs dependency status

LEMON parser generator modified to generate Rust code.

Lemon source and SQLite3 grammar were last synced as of July 2024.

Unsupported

Unsupported Grammar syntax

  • %token_destructor: Code to execute to destroy token data
  • %default_destructor: Code for the default non-terminal destructor
  • %destructor: Code which executes whenever this symbol is popped from the stack during error processing

https://www.codeproject.com/Articles/1056460/Generating-a-High-Speed-Parser-Part-Lemon https://www.sqlite.org/lemon.html

SQLite

SQLite lexer and SQLite parser have been ported from C to Rust. The parser generates an AST.

Lexer/Parser:

  • Keep track of position (line, column).
  • Streamable (stop at the end of statement).
  • Resumable (restart after the end of statement).

Lexer and parser have been tested with the following scripts:

TODO:

Unsupported by Rust

  • #line directive

API change

  • No ParseAlloc/ParseFree anymore

Features not tested

  • NDEBUG
  • YYNOERRORRECOVERY
  • YYERRORSYMBOL

To be fixed

  • RHS are moved. Maybe it is not a problem if they are always used once. Just add a check in lemon...
  • %extra_argument is not supported.
  • Terminal symbols generated by lemon should be dumped in a specified file.

Raison d'être

  • lemon_rust does the same thing but with an old version of lemon. And it seems not possible to use yystack as a stack because items may be access randomly and the top+1 item can be used.

  • lalrpop would be the perfect alternative but it does not support fallback/streaming (see this issue) and compilation/generation is slow.

Minimum supported Rust version (MSRV)

Latest stable Rust version at the time of release. It might compile with older versions.