Feature description

The sql_table source/resource is hard coded to use the operator ge (>=) when comparing data with incremental state (or le (>=) depending on the incremental function/sort order), we would like the option to configure it to use gt (>) / lt in stead.

Are you a dlt user?

Yes, I run dlt in production.

Use case

We have tables which are loaded seldom, but with large volumes, all with the same incremental key value. We would like to run dlt on top of these often, but not use resource re-loading the same data every time no new data has been added.

Proposed solution

An option to the sql_table source/resource whether to use classic (e.g. operator.ge) or aggressive (operator.gt) incremental filtering on the extracted sql-table.

Related issues

No response