[CARBONDATA-4291] Carbon hive table supports float datatype #4231
Conversation
|
Build Failed with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/429/ |
|
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6039/ |
|
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4294/ |
|
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6041/ |
|
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4297/ |
|
Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/431/ |
|
retest this please |
|
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6048/ |
|
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4303/ |
|
Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/437/ |
...org/apache/carbondata/integration/spark/testsuite/primitiveTypes/FloatDataTypeTestCase.scala
Outdated
Show resolved
Hide resolved
|
please rebase |
|
@akashrn5 please check once |
|
please rebase @jack86596 |
jack86596
force-pushed
the
float_type
branch
from
adb93ea
to
e8cf6b3
Compare
|
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6115/ |
|
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4372/ |
|
Build Failed with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/505/ |
|
retest this please |
|
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4373/ |
|
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6116/ |
|
Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/506/ |
|
@vikramahuja1001 @kunal642 rebased, please review, thanks. |
jack86596
force-pushed
the
float_type
branch
from
e8cf6b3
to
f24095f
Compare
|
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6121/ |
|
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4378/ |
|
Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/511/ |
Why is this PR needed?
Currently when create carbon hive table, if data type is float, will convert to double type. This means all float data will be stored as double.
In CTAS secnario, if source table column is float type, the data in newly created carbon table will be incorrect.
Reproduce steps:
CREATE TABLE p1(f float) stored as parquet;
insert into table p1 select 12.36;
create table carbon1 stored as carbondata as select * from p1;
select * from carbon1;
Result:
5.410467587E-315
What changes were proposed in this PR?
Carbon should support store float datatype directly. Handle float type in read and write logic.
Does this PR introduce any user interface change?
Is any new testcase added?