Remove antlr-rust

[35VLG84]

Remove (or update and maintain) Antlr-Rust

The antlr-rust was great help for PoC of Tackler-NG, but it's unmaintained. Also it's not an official Antlr target. Hence remove it and replace the parser with something else.

The Gitoxide uses winnow.

[RagibHasin]

Great project! Would you accept a PR replacing antlr-rust with winnow, or are you working on it yourself?

[35VLG84]

Hi, I just noticed this now, after answering the #36, Welcome again, and really nice to see interest in tackler, thanks!

This is a huge offer for help, and I highly appreciate that. It would be awesome if you could provide PR for winnow parser, as I have some initial Proof-of-Concepts but nothing serious yet.

You could take look of some of the parsing related code and see if it makes sense to you:

• date parsing is in hot path, so it would be nice if that could be as efficient as possible. This is non-optimized code, but the date handling logic is here:

  tackler-ng/tackler-core/src/parser/ctx_handler.rs

  Line 51 in dc82cdb

  fn handle_date(

• Due date handling logic and account, commodity and tags checks, there is need to carry Settings around as statefull winnow input
• The public functions (and usage) for parser are here:

  • tackler-ng/tackler-core/src/parser/tackler_txns.rs

    Line 37 in dc82cdb

    pub fn string_to_txns(input: &str, settings: &mut Settings) -> Result<TxnData, Box<dyn Error>> {

  • tackler-ng/tackler-core/src/parser/tackler_txns.rs

    Line 46 in dc82cdb

    pub fn paths_to_txns(

  • tackler-ng/tackler-core/src/parser/tackler_txns.rs

    Line 132 in dc82cdb

    tackler_parser::txns_text(str::from_utf8(&obj.data)?, settings);

• Current implementation naively copies all input into memory, this is problematic especially with files. It would be great to have streaming implementation for parsing, which would avoid copying all input into memory.
• Parser unit tests are here: https://github.com/e257-fi/tackler-ng/tree/main/tackler-core/src/parser/tests, these should be kept as they are, but especially in error cases there could be need to change test matchers.
  • 'pok_´ is for "parser-ok" test cases
  • 'perr' is for "parser-err" test cases
• The parser implementation should be Sync + Send (Settings isn't at the moment, but it needs to be refactored), so that parsing can be parallelized.

The current ANTLR based parser is fairly well containerized behind few function calls, and there is really good test coverage for it.

But as said, this would be really big effort, and if you are not already familiar with winnow and parsing in general, this won't be easy thing to do. If that's the case, it might be better if I keep working with the parser refactoring. On the other hand, if you know winnow already, help would great and truly highly appreciated!

[35VLG84]

Thinking this overnight, it would be better if I replace the ANTLR as this is fairly involved part of the code base (with all tests etc). In any case, thank you for offering to port this!

[RagibHasin]

Thanks for the pointers! I would start looking into it.

[35VLG84]

Please do not, as I'm working on this.

[RagibHasin]

Alright.

[35VLG84]

This is still missing some logic and special case wiring, but here is a sneak-peak of winnow port:

ANTLR

cargo bench

parser                  time:   [94.569 µs 94.721 µs 94.897 µs]"

cargo run --release -p tackler-core

On average 45903 txn/s
Reference implementation:  40000 txn/s ( +5903 txn/s)

winnow

cargo bench

parser                  time:   [2.8559 µs 2.8606 µs 2.8654 µs]

cargo run --release -p tackler-core

On average 137565 txn/s
Reference implementation:  40000 txn/s (+97565 txn/s)

It's now gix and io-bound, as the new parser is 30-35 times faster!

[RagibHasin]

That is impressive!