Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improve][Connector-V2] Change read excel util from POI to EasyExcel #8064

Open
wants to merge 3 commits into
base: dev
Choose a base branch
from

Conversation

dwave
Copy link

@dwave dwave commented Nov 15, 2024

#8040

Purpose of this pull request

Does this PR introduce any user-facing change?

How was this patch tested?

Check list

@Hisoka-X Hisoka-X changed the title [Hotfix][Connector-V2] ExcelReader read more than 65000 rows XSSFWorkbook will cause oom . so change POI to EasyExcel #8040 [Improve][Connector-V2] Change read excel util from POI to EasyExcel Nov 15, 2024
@@ -54,7 +55,7 @@ public class ExcelReadStrategyTest {

@Test
public void testExcelRead() throws IOException, URISyntaxException {
testExcelRead("/excel/test_read_excel.xlsx");
// testExcelRead("/excel/test_read_excel.xlsx");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why disable this?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the test excel used in the commented out code, and the date string that needs to be converted is 2024/1/31, and the format is
{mso-generic-font-family:auto;
mso-font-charset:134;
mso-number-format:"yyyy/m/d"; }

In POI, we can get the correct data type according to the format of the cell, but in EasyExcel, we can only get the string, and the conversion of the string to the Date type does not conform to the defined YYYYY/MM/dd format, which causes the test case to fail, so I commented out this one test case

image

image

image

@@ -15,8 +15,9 @@
* limitations under the License.
*/

package org.apache.seatunnel.connectors.seatunnel.file.writer;
package org.apache.seatunnel.connectors.seatunnel.file.Reader;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
package org.apache.seatunnel.connectors.seatunnel.file.Reader;
package org.apache.seatunnel.connectors.seatunnel.file.reader;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants