Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][Connector][JDBC] Add config option, Partition datatype #5656

Closed
2 of 3 tasks
naxels opened this issue Oct 18, 2023 · 3 comments
Closed
2 of 3 tasks

[Feature][Connector][JDBC] Add config option, Partition datatype #5656

naxels opened this issue Oct 18, 2023 · 3 comments
Labels

Comments

@naxels
Copy link

naxels commented Oct 18, 2023

Search before asking

  • I had searched in the feature and found no similar feature requirement.

Description

Hi team,

Love SeaTunnel and have been seriously considering it for our future data integration needs at work.

The one problem I stumble into with Oracle, one of our bigger databases is the NUMBER type.
Looking through the issues & pull requests etc, I see that many other people are facing this issue.

We as well cannot use the parallel feature to optimally move thousands of records unfortunately.

I ran the OracleCatalog.java SELECT_COLUMNS_SQL_TEMPLATE SQL script
and this was returned:

image

clearly with the defaults throwing me into Decimal territory.

Given that this is a hard problem to solve with Oracle and the code is already complex to try to turn it into a number.

I started thinking, what if there is an additional config option for JDBC, called something like
partition_type
partition_datatype
partition_overwrite_type

and setting this to integer or true (depending on your future thoughts regarding parallel support)
would force SeaTunnel to act as if the column is indeed an Integer (or future supported type) and not a Decimal.

Thank you for considering and keep up the good work!

Usage Scenario

For Oracle connector parallel execution, speeding up data transfers

Related issues

#4439

#5209

#5204

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Copy link

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

@github-actions github-actions bot added the stale label Nov 18, 2023
Copy link

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.

@yjkim0083
Copy link

@naxels
I found a solution.
add java_opt "-Doracle.jdbc.J2EE13Compliant=true"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants