Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[regression test](routine test) add case for exec_mem_limit #27308

Merged
merged 4 commits into from
Nov 23, 2023
Merged

[regression test](routine test) add case for exec_mem_limit #27308

merged 4 commits into from
Nov 23, 2023

Conversation

liugddx
Copy link
Member

@liugddx liugddx commented Nov 20, 2023

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@liugddx liugddx closed this Nov 20, 2023
@liugddx liugddx reopened this Nov 21, 2023
@liugddx
Copy link
Member Author

liugddx commented Nov 21, 2023

run buildall

liugddx and others added 2 commits November 21, 2023 10:46
@liugddx
Copy link
Member Author

liugddx commented Nov 21, 2023

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.2 seconds
stream load tsv: 567 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size: 17100779459 Bytes

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' 
Tpch sf100 test result on commit 3caf89ebf10fefafce48f0f398a19d3fa0ae72dc, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4874	4703	4644	4644
q2	359	156	158	156
q3	2033	1992	1912	1912
q4	1391	1302	1230	1230
q5	3992	3950	4019	3950
q6	247	134	129	129
q7	1428	879	893	879
q8	2762	2799	2772	2772
q9	34733	13725	9567	9567
q10	13200	3529	3536	3529
q11	393	253	240	240
q12	827	292	289	289
q13	21873	3804	3803	3803
q14	325	301	287	287
q15	582	523	528	523
q16	677	588	585	585
q17	1158	976	979	976
q18	7720	7294	7274	7274
q19	1692	1677	1698	1677
q20	569	287	287	287
q21	4814	3971	3984	3971
q22	473	382	375	375
Total cold run time: 106122 ms
Total hot run time: 49055 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4654	4588	4592	4588
q2	345	230	264	230
q3	4008	3982	3969	3969
q4	2700	2699	2680	2680
q5	9746	9598	9680	9598
q6	241	126	124	124
q7	3004	2506	2502	2502
q8	4429	4470	4444	4444
q9	13167	13163	13215	13163
q10	4120	4224	4192	4192
q11	745	637	637	637
q12	980	836	833	833
q13	4287	3525	3629	3525
q14	398	358	364	358
q15	576	528	522	522
q16	725	677	672	672
q17	3855	3854	3964	3854
q18	9528	9001	9101	9001
q19	1805	1771	1789	1771
q20	2370	2061	2051	2051
q21	8710	8422	8570	8422
q22	894	778	824	778
Total cold run time: 81287 ms
Total hot run time: 77914 ms

@liugddx
Copy link
Member Author

liugddx commented Nov 21, 2023

run p0

@liugddx
Copy link
Member Author

liugddx commented Nov 21, 2023

run pipelinex_p0

@liugddx
Copy link
Member Author

liugddx commented Nov 21, 2023

run buildall

dataroaring
dataroaring approved these changes Nov 21, 2023
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 21, 2023
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

dataroaring
dataroaring approved these changes Nov 21, 2023
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@liugddx
Copy link
Member Author

liugddx commented Nov 21, 2023

run p0

@liugddx
Copy link
Member Author

liugddx commented Nov 21, 2023

run pipelinex_p0

@liugddx
Copy link
Member Author

liugddx commented Nov 21, 2023

run P0

@liugddx
Copy link
Member Author

liugddx commented Nov 21, 2023

run pipelinex_p0

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' 
Tpch sf100 test result on commit 80a413e590d5bed74aedf184c061b593bff72be6, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4887	4645	4621	4621
q2	357	168	136	136
q3	2038	1866	1886	1866
q4	1373	1223	1227	1223
q5	3940	3934	3989	3934
q6	243	131	134	131
q7	1424	880	897	880
q8	2732	2783	2740	2740
q9	9709	9707	9457	9457
q10	3462	3566	3524	3524
q11	386	243	247	243
q12	441	285	304	285
q13	4577	3818	3822	3818
q14	329	290	293	290
q15	579	543	529	529
q16	660	589	585	585
q17	1139	955	926	926
q18	7880	7409	7456	7409
q19	1676	1664	1670	1664
q20	552	303	294	294
q21	4344	3939	3918	3918
q22	475	374	374	374
Total cold run time: 53203 ms
Total hot run time: 48847 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4596	4561	4568	4561
q2	340	233	275	233
q3	4021	3988	3987	3987
q4	2697	2676	2689	2676
q5	9737	9682	9672	9672
q6	237	124	126	124
q7	3024	2481	2479	2479
q8	4483	4445	4473	4445
q9	13199	13040	13041	13040
q10	4092	4221	4191	4191
q11	781	645	632	632
q12	978	811	806	806
q13	4257	3598	3574	3574
q14	382	362	341	341
q15	578	517	533	517
q16	727	662	684	662
q17	3872	3826	3839	3826
q18	9562	9180	9145	9145
q19	1801	1778	1756	1756
q20	2425	2065	2062	2062
q21	8825	8681	8805	8681
q22	876	802	790	790
Total cold run time: 81490 ms
Total hot run time: 78200 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.37 seconds
stream load tsv: 580 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17099833554 Bytes

sollhui
sollhui approved these changes Nov 23, 2023
View reviewed changes
Copy link
Contributor

@sollhui sollhui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 93cfdff into apache:master Nov 23, 2023
18 of 20 checks passed
seawinde pushed a commit to seawinde/doris that referenced this pull request Nov 28, 2023
@liugddx @seawinde
[regression test](routine test) add case for exec_mem_limit (apache#2…
c39d45c 
…7308)
XuJianxu pushed a commit to XuJianxu/doris that referenced this pull request Dec 14, 2023
@liugddx
[regression test](routine test) add case for exec_mem_limit (apache#2…
6d26d8f 
…7308)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dataroaring dataroaring dataroaring approved these changes

@sollhui sollhui sollhui approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.4 reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@liugddx @doris-robot @sollhui @dataroaring