Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](backup) Fix compressed backup job #43197

Merged
merged 1 commit into from
Nov 4, 2024
Merged

[fix](backup) Fix compressed backup job #43197

merged 1 commit into from
Nov 4, 2024

Conversation

w41ter
Copy link
Contributor

@w41ter w41ter commented Nov 4, 2024

This was introduced in #42462

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@w41ter
Copy link
Contributor Author

w41ter commented Nov 4, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 50239 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1ba3a8a1caf87b6ca69f0495a033c29a3e8278fc, data reload: false

------ Round 1 ----------------------------------
q1	17543	4436	4394	4394
q2	2068	157	149	149
q3	10266	1947	1981	1947
q4	10106	1283	1342	1283
q5	8834	3935	3937	3935
q6	238	125	125	125
q7	2037	1610	1626	1610
q8	9307	2797	2764	2764
q9	10332	10556	10485	10485
q10	8692	3606	3550	3550
q11	424	246	254	246
q12	472	296	301	296
q13	18346	3967	4031	3967
q14	355	335	324	324
q15	524	466	460	460
q16	543	451	455	451
q17	1161	996	969	969
q18	7297	6880	6885	6880
q19	1717	1586	1564	1564
q20	542	304	304	304
q21	4479	4158	4143	4143
q22	501	399	393	393
Total cold run time: 115784 ms
Total hot run time: 50239 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4359	4380	4334	4334
q2	321	223	217	217
q3	4182	4191	4131	4131
q4	2758	2780	2761	2761
q5	7239	7143	7119	7119
q6	244	120	119	119
q7	3258	2838	2844	2838
q8	4376	4509	4494	4494
q9	13772	13653	13727	13653
q10	4276	4303	4274	4274
q11	736	676	699	676
q12	1021	855	858	855
q13	6793	3793	3794	3793
q14	448	427	427	427
q15	507	473	461	461
q16	644	587	593	587
q17	3837	3825	3811	3811
q18	8714	8779	8749	8749
q19	1734	1706	1667	1667
q20	2358	2129	2148	2129
q21	8551	8487	8479	8479
q22	1004	922	902	902
Total cold run time: 81132 ms
Total hot run time: 76476 ms

dataroaring
dataroaring approved these changes Nov 4, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 4, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 4, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 4, 2024

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-DS: Total hot run time: 213048 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1ba3a8a1caf87b6ca69f0495a033c29a3e8278fc, data reload: false

query1	932	424	388	388
query2	6538	2283	2167	2167
query3	6921	197	200	197
query4	23226	21267	21571	21267
query5	19746	6560	6504	6504
query6	274	226	231	226
query7	4325	311	310	310
query8	261	271	243	243
query9	3026	2591	2540	2540
query10	469	335	289	289
query11	15675	15201	15495	15201
query12	124	74	72	72
query13	1029	443	430	430
query14	17167	13506	13512	13506
query15	383	220	224	220
query16	6484	281	259	259
query17	1740	929	891	891
query18	904	313	323	313
query19	207	154	151	151
query20	99	97	105	97
query21	189	95	89	89
query22	5269	5150	5073	5073
query23	34743	33534	33755	33534
query24	7892	6289	6470	6289
query25	529	447	435	435
query26	1266	162	159	159
query27	2464	305	292	292
query28	6099	2220	2217	2217
query29	2897	2682	2720	2682
query30	240	166	170	166
query31	977	738	723	723
query32	72	63	61	61
query33	453	256	254	254
query34	870	467	496	467
query35	1139	940	907	907
query36	1161	1271	1291	1271
query37	173	62	63	62
query38	3072	2935	2913	2913
query39	1369	1323	1312	1312
query40	314	101	95	95
query41	41	37	37	37
query42	87	80	85	80
query43	608	565	639	565
query44	1161	732	725	725
query45	244	231	230	230
query46	1235	977	958	958
query47	1756	1875	1799	1799
query48	504	411	405	405
query49	651	364	386	364
query50	869	604	627	604
query51	4818	4679	4737	4679
query52	103	84	79	79
query53	224	186	197	186
query54	2655	2519	2534	2519
query55	80	90	80	80
query56	215	220	214	214
query57	1344	1166	1258	1166
query58	213	217	201	201
query59	3558	3357	3432	3357
query60	229	210	219	210
query61	97	94	117	94
query62	800	473	470	470
query63	197	188	179	179
query64	5824	1635	1402	1402
query65	3677	3518	3557	3518
query66	624	412	390	390
query67	18185	15789	15642	15642
query68	9326	656	656	656
query69	482	283	257	257
query70	1645	1379	1313	1313
query71	397	311	311	311
query72	6848	4826	4876	4826
query73	754	310	316	310
query74	6231	5802	5796	5796
query75	4971	3609	3603	3603
query76	5094	1159	1187	1159
query77	771	255	261	255
query78	12765	12113	11629	11629
query79	10809	659	654	654
query80	1851	376	391	376
query81	495	238	241	238
query82	1303	100	99	99
query83	172	131	129	129
query84	257	69	73	69
query85	1018	326	320	320
query86	351	292	291	291
query87	3306	2993	3008	2993
query88	5048	2270	2276	2270
query89	416	296	279	279
query90	1936	214	214	214
query91	159	139	129	129
query92	62	51	51	51
query93	6028	587	595	587
query94	823	211	213	211
query95	2106	1902	2002	1902
query96	643	318	329	318
query97	6542	6316	6347	6316
query98	225	203	211	203
query99	3075	859	873	859
Total cold run time: 326626 ms
Total hot run time: 213048 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.23 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1ba3a8a1caf87b6ca69f0495a033c29a3e8278fc, data reload: false

query1	0.03	0.03	0.02
query2	0.07	0.03	0.03
query3	0.25	0.05	0.04
query4	1.78	0.06	0.07
query5	0.53	0.53	0.52
query6	1.23	0.61	0.62
query7	0.01	0.01	0.01
query8	0.04	0.02	0.02
query9	0.52	0.49	0.48
query10	0.55	0.53	0.53
query11	0.12	0.09	0.09
query12	0.12	0.08	0.09
query13	0.62	0.63	0.60
query14	0.78	0.78	0.78
query15	0.79	0.76	0.76
query16	0.38	0.36	0.35
query17	1.01	1.01	1.02
query18	0.24	0.25	0.25
query19	1.95	1.83	1.77
query20	0.02	0.01	0.01
query21	15.48	0.54	0.57
query22	1.95	2.21	2.35
query23	17.53	1.14	1.00
query24	7.21	0.93	0.96
query25	0.36	0.11	0.05
query26	0.76	0.16	0.14
query27	0.05	0.03	0.04
query28	6.06	0.76	0.76
query29	12.62	2.29	2.25
query30	0.58	0.50	0.51
query31	2.78	0.40	0.38
query32	3.39	0.49	0.51
query33	3.07	3.07	3.06
query34	15.27	4.81	4.76
query35	4.82	4.86	4.84
query36	1.06	1.03	1.01
query37	0.06	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.02	0.02
query40	0.16	0.14	0.14
query41	0.06	0.02	0.01
query42	0.02	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 104.4 s
Total hot run time: 31.23 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 1ba3a8a1caf87b6ca69f0495a033c29a3e8278fc with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.6 seconds inserted 10000000 Rows, about 462K ops/s

@w41ter w41ter merged commit d1867fe into apache:branch-2.0 Nov 4, 2024
23 of 26 checks passed
@w41ter w41ter deleted the fix_write_backup_job_2.0 branch November 4, 2024 08:20
@w41ter w41ter removed the dev/2.0.x label Nov 6, 2024
w41ter added a commit to w41ter/incubator-doris that referenced this pull request Nov 8, 2024
@w41ter
[fix](backup) Fix compressed backup job (apache#43197)
a392e25 
This was introduced in apache#42462
@w41ter w41ter mentioned this pull request Nov 8, 2024
Merged
w41ter added a commit that referenced this pull request Nov 8, 2024
@w41ter
[fix](backup) Fix compressed backup job #43197 (#43484)
57c794b 
cherry pick from #43197
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.8-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@w41ter @doris-robot @dataroaring