You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I had searched in the feature and found no similar feature requirement.
Description
Hi team,
Love SeaTunnel and have been seriously considering it for our future data integration needs at work.
The one problem I stumble into with Oracle, one of our bigger databases is the NUMBER type.
Looking through the issues & pull requests etc, I see that many other people are facing this issue.
We as well cannot use the parallel feature to optimally move thousands of records unfortunately.
I ran the OracleCatalog.java SELECT_COLUMNS_SQL_TEMPLATE SQL script
and this was returned:
clearly with the defaults throwing me into Decimal territory.
Given that this is a hard problem to solve with Oracle and the code is already complex to try to turn it into a number.
I started thinking, what if there is an additional config option for JDBC, called something like
partition_type
partition_datatype
partition_overwrite_type
and setting this to integer or true (depending on your future thoughts regarding parallel support)
would force SeaTunnel to act as if the column is indeed an Integer (or future supported type) and not a Decimal.
Thank you for considering and keep up the good work!
Usage Scenario
For Oracle connector parallel execution, speeding up data transfers
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.
Search before asking
Description
Hi team,
Love SeaTunnel and have been seriously considering it for our future data integration needs at work.
The one problem I stumble into with Oracle, one of our bigger databases is the NUMBER type.
Looking through the issues & pull requests etc, I see that many other people are facing this issue.
We as well cannot use the parallel feature to optimally move thousands of records unfortunately.
I ran the OracleCatalog.java SELECT_COLUMNS_SQL_TEMPLATE SQL script
and this was returned:
clearly with the defaults throwing me into Decimal territory.
Given that this is a hard problem to solve with Oracle and the code is already complex to try to turn it into a number.
I started thinking, what if there is an additional config option for JDBC, called something like
partition_type
partition_datatype
partition_overwrite_type
and setting this to integer or true (depending on your future thoughts regarding parallel support)
would force SeaTunnel to act as if the column is indeed an Integer (or future supported type) and not a Decimal.
Thank you for considering and keep up the good work!
Usage Scenario
For Oracle connector parallel execution, speeding up data transfers
Related issues
#4439
#5209
#5204
Are you willing to submit a PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: