Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature](binlog) Allow BarrierLog to wrap another binlog #42993 #43027

Merged
merged 1 commit into from
Oct 31, 2024

Conversation

w41ter
Copy link
Contributor

@w41ter w41ter commented Oct 31, 2024

cherry pick from #42993

to make it work in the old Doris version without breaking compatibility
@w41ter
Copy link
Contributor Author

w41ter commented Oct 31, 2024

run buildall

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@doris-robot
Copy link

TPC-H: Total hot run time: 49010 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ed865f50332efd9ea81533c6c18add55f1c73e23, data reload: false

------ Round 1 ----------------------------------
q1	17723	4372	4310	4310
q2	2083	154	143	143
q3	10297	1882	1893	1882
q4	10332	1279	1340	1279
q5	8385	3910	3885	3885
q6	226	126	123	123
q7	2064	1613	1601	1601
q8	9287	2724	2710	2710
q9	9974	9843	9809	9809
q10	8669	3518	3548	3518
q11	422	245	251	245
q12	467	299	304	299
q13	18330	3949	4016	3949
q14	352	341	332	332
q15	519	456	465	456
q16	528	445	486	445
q17	1135	956	930	930
q18	7263	6839	6774	6774
q19	1666	1560	1552	1552
q20	547	322	290	290
q21	4500	4167	4093	4093
q22	490	394	385	385
Total cold run time: 115259 ms
Total hot run time: 49010 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4295	4386	4286	4286
q2	324	223	219	219
q3	4150	4183	4119	4119
q4	2757	2726	2738	2726
q5	7156	7187	7093	7093
q6	237	118	121	118
q7	3224	2822	2847	2822
q8	4370	4474	4493	4474
q9	13705	13501	13646	13501
q10	4235	4259	4290	4259
q11	761	691	693	691
q12	1026	849	864	849
q13	7080	3811	3733	3733
q14	464	440	430	430
q15	506	455	453	453
q16	622	577	591	577
q17	3766	3777	3797	3777
q18	8765	8819	8902	8819
q19	1759	1705	1677	1677
q20	2372	2146	2154	2146
q21	8642	8486	8508	8486
q22	1075	921	964	921
Total cold run time: 81291 ms
Total hot run time: 76176 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 213543 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ed865f50332efd9ea81533c6c18add55f1c73e23, data reload: false

query1	931	396	413	396
query2	6532	2145	2226	2145
query3	7439	207	215	207
query4	25352	22209	22542	22209
query5	20062	6556	6670	6556
query6	444	217	244	217
query7	5373	304	310	304
query8	264	243	259	243
query9	3026	2610	2566	2566
query10	484	328	294	294
query11	16454	15178	15037	15037
query12	132	74	75	74
query13	1023	438	418	418
query14	19025	13587	13323	13323
query15	378	212	237	212
query16	6464	286	268	268
query17	1559	978	924	924
query18	883	314	309	309
query19	216	154	151	151
query20	101	93	99	93
query21	196	98	92	92
query22	5211	4983	4922	4922
query23	34222	33451	33503	33451
query24	6530	6402	6294	6294
query25	540	442	444	442
query26	808	164	159	159
query27	2241	296	291	291
query28	6102	2218	2202	2202
query29	2847	2854	2760	2760
query30	241	168	168	168
query31	975	771	757	757
query32	70	62	60	60
query33	435	265	258	258
query34	856	483	471	471
query35	1118	941	937	937
query36	1385	1302	1063	1063
query37	94	62	63	62
query38	3150	2914	2877	2877
query39	1363	1338	1328	1328
query40	215	97	97	97
query41	41	36	37	36
query42	97	89	85	85
query43	615	650	558	558
query44	1177	723	731	723
query45	242	227	230	227
query46	1242	961	970	961
query47	1815	1840	1771	1771
query48	510	397	395	395
query49	626	366	385	366
query50	874	668	595	595
query51	4793	4699	4663	4663
query52	95	88	87	87
query53	241	189	189	189
query54	2678	2498	2489	2489
query55	78	83	78	78
query56	225	207	195	195
query57	1200	1148	1172	1148
query58	224	216	208	208
query59	3363	3642	3366	3366
query60	215	239	209	209
query61	96	96	103	96
query62	853	480	459	459
query63	203	172	175	172
query64	3306	1577	1449	1449
query65	3628	3580	3561	3561
query66	760	415	411	411
query67	16252	15787	15845	15787
query68	9350	638	641	638
query69	513	275	275	275
query70	1488	1313	1478	1313
query71	411	314	315	314
query72	7010	4809	4778	4778
query73	760	316	322	316
query74	6216	5930	5860	5860
query75	5184	3660	3713	3660
query76	5051	1134	1157	1134
query77	742	254	260	254
query78	12738	11452	12393	11452
query79	7809	641	646	641
query80	1698	390	388	388
query81	485	239	241	239
query82	1043	98	101	98
query83	170	134	131	131
query84	257	72	69	69
query85	991	320	315	315
query86	337	283	303	283
query87	3248	2989	3054	2989
query88	4479	2288	2295	2288
query89	432	278	274	274
query90	1994	217	222	217
query91	161	125	127	125
query92	60	54	53	53
query93	5095	549	538	538
query94	763	206	212	206
query95	2045	2007	2109	2007
query96	653	330	322	322
query97	6469	6404	6402	6402
query98	228	206	231	206
query99	3032	837	894	837
Total cold run time: 321102 ms
Total hot run time: 213543 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.28 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ed865f50332efd9ea81533c6c18add55f1c73e23, data reload: false

query1	0.02	0.02	0.02
query2	0.06	0.02	0.02
query3	0.25	0.05	0.04
query4	1.79	0.06	0.06
query5	0.54	0.52	0.51
query6	1.24	0.61	0.60
query7	0.01	0.01	0.01
query8	0.04	0.02	0.02
query9	0.53	0.50	0.47
query10	0.54	0.54	0.53
query11	0.12	0.08	0.09
query12	0.12	0.09	0.09
query13	0.62	0.61	0.61
query14	0.79	0.80	0.80
query15	0.77	0.77	0.77
query16	0.36	0.35	0.39
query17	1.01	1.00	1.03
query18	0.22	0.28	0.23
query19	1.93	1.85	1.82
query20	0.01	0.01	0.02
query21	15.45	0.54	0.54
query22	2.30	1.88	1.46
query23	17.03	0.81	0.93
query24	3.92	0.89	2.21
query25	0.39	0.08	0.06
query26	0.54	0.15	0.16
query27	0.04	0.05	0.04
query28	8.50	0.72	0.72
query29	12.72	2.31	2.35
query30	0.56	0.51	0.52
query31	2.81	0.39	0.37
query32	3.38	0.50	0.49
query33	3.09	3.04	3.11
query34	15.26	4.78	4.77
query35	4.85	4.82	4.84
query36	1.05	1.01	1.02
query37	0.06	0.05	0.05
query38	0.03	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.07	0.01	0.01
query42	0.03	0.01	0.02
query43	0.03	0.01	0.02
Total cold run time: 103.26 s
Total hot run time: 30.28 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit ed865f50332efd9ea81533c6c18add55f1c73e23 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.8 seconds inserted 10000000 Rows, about 458K ops/s

@w41ter w41ter merged commit 693c806 into apache:branch-2.0 Oct 31, 2024
22 of 24 checks passed
@w41ter w41ter deleted the pick_42993_to_origin_branch-2.0 branch October 31, 2024 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants