Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[refactor](datatype) Introduce a toolkits for clarify datelike types usage #26810

Merged
merged 1 commit into from
Nov 22, 2023
Merged

[refactor](datatype) Introduce a toolkits for clarify datelike types usage #26810

merged 1 commit into from
Nov 22, 2023

Conversation

zclllyybb
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

github-actions[bot]
github-actions bot reviewed Nov 11, 2023
View reviewed changes
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

be/src/vec/functions/function_convert_tz.h Show resolved Hide resolved
@zclllyybb
Copy link
Contributor Author

run buildall

@zclllyybb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.77% (8403/22855)
Line Coverage: 29.28% (68095/232566)
Region Coverage: 27.90% (35191/126145)
Branch Coverage: 24.72% (18003/72830)
Coverage Report: http://coverage.selectdb-in.cc/coverage/19a43f46b9e01fb3b49ebd224ff46dfedb9d2e19_19a43f46b9e01fb3b49ebd224ff46dfedb9d2e19/report/index.html

@zclllyybb
Copy link
Contributor Author

run buildall

@zclllyybb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.74% (8419/22916)
Line Coverage: 29.27% (68368/233562)
Region Coverage: 27.87% (35347/126816)
Branch Coverage: 24.66% (18068/73280)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f9a60d2e427cae32375a7365584409396904a2b5_f9a60d2e427cae32375a7365584409396904a2b5/report/index.html

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' 
Tpch sf100 test result on commit f9a60d2e427cae32375a7365584409396904a2b5, data reload: false

run tpch-sf100 query with default conf and session variables
q1	5308	5035	5131	5035
q2	363	180	160	160
q3	2054	2044	2022	2022
q4	1414	1380	1379	1379
q5	3995	4007	4022	4007
q6	256	132	131	131
q7	1473	884	893	884
q8	2804	2796	2778	2778
q9	9856	9511	9455	9455
q10	3449	3522	3557	3522
q11	392	267	253	253
q12	440	290	291	290
q13	4588	4158	4145	4145
q14	317	278	308	278
q15	642	558	551	551
q16	687	607	585	585
q17	1135	1080	1104	1080
q18	8093	7704	7478	7478
q19	1675	1721	1685	1685
q20	585	294	295	294
q21	4743	4418	4420	4418
q22	508	419	417	417
Total cold run time: 54777 ms
Total hot run time: 50847 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	5006	5053	5071	5053
q2	369	218	241	218
q3	4076	4047	4028	4028
q4	2823	2787	2779	2779
q5	9601	9728	9682	9682
q6	248	122	126	122
q7	3063	2473	2532	2473
q8	4843	4870	4837	4837
q9	13075	12987	12964	12964
q10	4058	4157	4179	4157
q11	761	691	659	659
q12	1016	848	854	848
q13	4305	3914	3920	3914
q14	376	344	347	344
q15	585	546	526	526
q16	779	685	730	685
q17	3849	3874	3897	3874
q18	9692	9495	9555	9495
q19	1902	1803	1764	1764
q20	2399	2082	2054	2054
q21	9093	8853	8714	8714
q22	928	852	845	845
Total cold run time: 82847 ms
Total hot run time: 80035 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.54 seconds
stream load tsv: 556 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.5 seconds inserted 10000000 Rows, about 350K ops/s
storage size: 17096376519 Bytes

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' 
Tpch sf100 test result on commit f9a60d2e427cae32375a7365584409396904a2b5, data reload: false

run tpch-sf100 query with default conf and session variables
q1	5400	5200	5179	5179
q2	360	162	139	139
q3	2031	1999	1987	1987
q4	1405	1376	1358	1358
q5	3972	4040	3987	3987
q6	255	136	137	136
q7	1501	880	902	880
q8	2776	2791	2758	2758
q9	9819	9928	9564	9564
q10	3465	3530	3545	3530
q11	381	254	250	250
q12	433	280	284	280
q13	4541	4137	4148	4137
q14	336	285	284	284
q15	595	549	538	538
q16	676	592	576	576
q17	1137	1096	1080	1080
q18	8148	7587	7640	7587
q19	1689	1687	1681	1681
q20	573	309	272	272
q21	4695	4373	4379	4373
q22	507	407	419	407
Total cold run time: 54695 ms
Total hot run time: 50983 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	5101	5111	5085	5085
q2	327	223	240	223
q3	4062	4011	4055	4011
q4	2797	2770	2765	2765
q5	9639	9601	9673	9601
q6	246	126	128	126
q7	3096	2481	2484	2481
q8	4848	4821	4825	4821
q9	13293	13128	13197	13128
q10	4100	4185	4179	4179
q11	748	639	652	639
q12	999	808	824	808
q13	4296	3899	3878	3878
q14	393	346	361	346
q15	592	526	543	526
q16	757	692	693	692
q17	3907	3905	3868	3868
q18	9717	9388	9433	9388
q19	1894	1780	1794	1780
q20	2443	2059	2058	2058
q21	8825	8677	8874	8677
q22	944	823	878	823
Total cold run time: 83024 ms
Total hot run time: 79903 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.68 seconds
stream load tsv: 555 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 34 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17095997370 Bytes

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@zclllyybb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.52% (8449/23135)
Line Coverage: 28.88% (68644/237720)
Region Coverage: 27.85% (35511/127503)
Branch Coverage: 24.61% (18114/73612)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f9a60d2e427cae32375a7365584409396904a2b5_f9a60d2e427cae32375a7365584409396904a2b5/report/index.html

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' 
Tpch sf100 test result on commit f9a60d2e427cae32375a7365584409396904a2b5, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4930	4689	4659	4659
q2	366	159	157	157
q3	2037	1978	1999	1978
q4	1384	1271	1232	1232
q5	4005	3966	4004	3966
q6	244	130	130	130
q7	1412	887	900	887
q8	2763	2803	2787	2787
q9	9785	9747	9614	9614
q10	3472	3534	3543	3534
q11	376	252	247	247
q12	440	296	292	292
q13	4590	3828	3785	3785
q14	322	290	290	290
q15	592	531	517	517
q16	664	586	587	586
q17	1145	984	929	929
q18	7768	7305	7349	7305
q19	1671	1689	1678	1678
q20	524	314	313	313
q21	4351	4024	3964	3964
q22	474	366	376	366
Total cold run time: 53315 ms
Total hot run time: 49216 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4569	4606	4565	4565
q2	350	225	249	225
q3	3995	3984	3984	3984
q4	2683	2691	2690	2690
q5	9782	9854	9840	9840
q6	243	123	119	119
q7	2993	2510	2483	2483
q8	4495	4514	4489	4489
q9	13224	13219	13169	13169
q10	4118	4216	4213	4213
q11	775	655	642	642
q12	972	808	811	808
q13	4287	3593	3585	3585
q14	394	355	354	354
q15	577	519	527	519
q16	744	667	666	666
q17	3862	3927	3887	3887
q18	9583	9112	9040	9040
q19	1799	1813	1801	1801
q20	2382	2074	2085	2074
q21	8842	8578	8728	8578
q22	866	847	795	795
Total cold run time: 81535 ms
Total hot run time: 78526 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.88 seconds
stream load tsv: 567 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17100808597 Bytes

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 22, 2023
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@Gabriel39 Gabriel39 merged commit a68b62b into apache:master Nov 22, 2023
19 of 20 checks passed
@zclllyybb zclllyybb mentioned this pull request Nov 22, 2023
Open
3 tasks
seawinde pushed a commit to seawinde/doris that referenced this pull request Nov 28, 2023
@zclllyybb @seawinde
[refactor](datatype) Introduce a toolkits for clarify datelike types …
ab89bd3 
…usage (apache#26810)
@zclllyybb zclllyybb deleted the re_functions branch December 5, 2023 06:09
XuJianxu pushed a commit to XuJianxu/doris that referenced this pull request Dec 14, 2023
@zclllyybb
[refactor](datatype) Introduce a toolkits for clarify datelike types …
fc4de07 
…usage (apache#26810)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@github-actions github-actions[bot] github-actions[bot] left review comments

@Mryange Mryange Mryange approved these changes

@Gabriel39 Gabriel39 Gabriel39 approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@zclllyybb @doris-robot @Gabriel39 @Mryange