[fix](delete) fix insert into cols should be corresponding to the query output for delete from command #47406

mymeiyi · 2025-01-24T07:30:41Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

for mow table with function_column.sequence_col map, if use delete from command and run into partial update, may get:

 errCode = 2, detailMessage = insert into cols should be corresponding to the query output

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-01-24T07:30:47Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

zhannngchen

LGTM

github-actions · 2025-01-24T07:35:25Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-01-24T07:35:27Z

PR approved by anyone and no changes requested.

mymeiyi · 2025-01-24T07:38:39Z

run buildall

doris-robot · 2025-01-24T07:59:08Z

TPC-H: Total hot run time: 32244 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 818674726c8cdda366a9f5d65409571daf31d6c5, data reload: false

------ Round 1 ----------------------------------
q1	17624	5492	5373	5373
q2	2054	308	179	179
q3	10398	1199	736	736
q4	10232	971	540	540
q5	8013	2389	2173	2173
q6	201	167	132	132
q7	892	762	598	598
q8	9239	1331	1232	1232
q9	5193	4815	4876	4815
q10	6792	2316	1893	1893
q11	488	278	251	251
q12	344	352	217	217
q13	17758	3732	3086	3086
q14	223	234	209	209
q15	511	468	462	462
q16	620	620	593	593
q17	571	851	319	319
q18	7030	6402	6428	6402
q19	2763	954	540	540
q20	305	315	185	185
q21	2826	2158	1997	1997
q22	353	333	312	312
Total cold run time: 104430 ms
Total hot run time: 32244 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5568	5464	5493	5464
q2	239	336	228	228
q3	2256	2632	2346	2346
q4	1413	1850	1370	1370
q5	4305	4740	4674	4674
q6	176	173	131	131
q7	2084	1924	1796	1796
q8	2644	2842	2706	2706
q9	7333	7157	7248	7157
q10	3060	3268	2808	2808
q11	580	515	481	481
q12	626	733	560	560
q13	3434	3942	4028	3942
q14	281	291	292	291
q15	532	494	480	480
q16	648	676	640	640
q17	1235	1737	1276	1276
q18	7692	7493	7359	7359
q19	823	1182	1074	1074
q20	1997	2034	1917	1917
q21	5794	5049	4918	4918
q22	608	620	575	575
Total cold run time: 53328 ms
Total hot run time: 52193 ms

doris-robot · 2025-01-24T08:11:02Z

TPC-DS: Total hot run time: 191148 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 818674726c8cdda366a9f5d65409571daf31d6c5, data reload: false

query1	1326	977	919	919
query2	6258	1974	1961	1961
query3	11329	4694	4717	4694
query4	32392	23765	23032	23032
query5	4847	597	449	449
query6	279	218	172	172
query7	3985	488	311	311
query8	296	248	225	225
query9	9298	2574	2567	2567
query10	493	312	253	253
query11	17912	15264	14867	14867
query12	152	110	105	105
query13	1570	519	390	390
query14	9818	6871	6367	6367
query15	251	197	186	186
query16	8037	648	487	487
query17	1552	743	595	595
query18	2131	430	319	319
query19	219	195	166	166
query20	128	125	114	114
query21	213	123	107	107
query22	4518	4531	4401	4401
query23	34411	33534	33417	33417
query24	6579	2273	2390	2273
query25	490	484	399	399
query26	796	275	163	163
query27	2108	465	335	335
query28	5470	2443	2396	2396
query29	571	552	417	417
query30	212	184	170	170
query31	1020	854	829	829
query32	83	60	59	59
query33	486	357	289	289
query34	760	841	517	517
query35	827	825	726	726
query36	1031	1039	972	972
query37	124	102	78	78
query38	4266	4304	4282	4282
query39	1601	1455	1472	1455
query40	236	121	114	114
query41	64	59	62	59
query42	121	103	104	103
query43	509	529	496	496
query44	1424	831	825	825
query45	183	169	161	161
query46	857	1055	651	651
query47	1887	1929	1836	1836
query48	396	414	322	322
query49	748	498	399	399
query50	655	688	397	397
query51	4359	4320	4250	4250
query52	113	100	96	96
query53	233	261	192	192
query54	505	505	413	413
query55	85	76	79	76
query56	267	258	253	253
query57	1211	1174	1154	1154
query58	257	233	232	232
query59	3125	3285	3027	3027
query60	306	268	254	254
query61	124	118	118	118
query62	746	764	750	750
query63	248	190	190	190
query64	3662	1026	695	695
query65	3345	3321	3334	3321
query66	832	403	339	339
query67	16314	15699	15420	15420
query68	4303	825	529	529
query69	466	290	256	256
query70	1198	1150	1099	1099
query71	383	280	273	273
query72	5982	3816	3850	3816
query73	643	762	365	365
query74	9841	9121	9052	9052
query75	3215	3173	2626	2626
query76	3047	1168	769	769
query77	519	371	272	272
query78	10116	10003	9403	9403
query79	2303	807	587	587
query80	700	504	444	444
query81	494	271	240	240
query82	227	146	121	121
query83	172	169	148	148
query84	238	101	77	77
query85	759	354	311	311
query86	385	317	302	302
query87	4483	4499	4453	4453
query88	3552	2207	2243	2207
query89	442	324	290	290
query90	1909	189	187	187
query91	129	134	106	106
query92	65	56	51	51
query93	1749	875	527	527
query94	648	400	294	294
query95	344	266	255	255
query96	491	612	278	278
query97	2767	2878	2739	2739
query98	217	208	195	195
query99	1259	1356	1269	1269
Total cold run time: 283697 ms
Total hot run time: 191148 ms

doris-robot · 2025-01-24T08:16:23Z

ClickBench: Total hot run time: 31.02 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 818674726c8cdda366a9f5d65409571daf31d6c5, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.61	0.10	0.11
query5	0.42	0.41	0.43
query6	1.15	0.68	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.57	0.50	0.50
query10	0.56	0.57	0.55
query11	0.14	0.09	0.10
query12	0.14	0.11	0.12
query13	0.62	0.60	0.59
query14	2.83	2.73	2.75
query15	0.90	0.82	0.85
query16	0.37	0.38	0.39
query17	1.02	1.05	1.06
query18	0.22	0.20	0.21
query19	1.94	1.90	2.01
query20	0.02	0.00	0.02
query21	15.37	1.00	0.58
query22	0.74	0.84	0.71
query23	15.19	1.48	0.59
query24	3.27	1.38	0.92
query25	0.21	0.20	0.18
query26	0.24	0.15	0.15
query27	0.09	0.05	0.05
query28	14.03	1.04	0.43
query29	12.60	4.03	3.36
query30	0.25	0.09	0.06
query31	2.83	0.61	0.39
query32	3.23	0.55	0.46
query33	3.01	3.00	3.04
query34	16.77	5.18	4.58
query35	4.54	4.52	4.50
query36	0.65	0.52	0.48
query37	0.08	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.18	0.13	0.13
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.42 s
Total hot run time: 31.02 s

dataroaring

LGTM

mymeiyi · 2025-01-24T10:45:21Z

run buildall

doris-robot · 2025-01-24T11:05:13Z

TPC-H: Total hot run time: 32248 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3a18587dce44b03d4d063ddd82c0e4c2e6557467, data reload: false

------ Round 1 ----------------------------------
q1	17594	5572	5423	5423
q2	2043	302	164	164
q3	10428	1295	723	723
q4	10211	962	529	529
q5	7521	2435	2141	2141
q6	187	163	135	135
q7	915	776	596	596
q8	9225	1349	1148	1148
q9	5352	4920	4865	4865
q10	6829	2340	1894	1894
q11	453	269	266	266
q12	349	350	224	224
q13	17761	3699	3079	3079
q14	242	221	215	215
q15	523	466	456	456
q16	614	628	590	590
q17	556	867	322	322
q18	7182	6447	6606	6447
q19	1211	979	527	527
q20	309	323	197	197
q21	2808	2122	1982	1982
q22	367	333	325	325
Total cold run time: 102680 ms
Total hot run time: 32248 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5564	5488	5543	5488
q2	235	335	230	230
q3	2297	2666	2324	2324
q4	1439	1850	1432	1432
q5	4344	4767	4686	4686
q6	166	157	124	124
q7	2072	2007	1802	1802
q8	2640	2814	2691	2691
q9	7215	7208	7266	7208
q10	2980	3269	2821	2821
q11	580	523	493	493
q12	688	759	586	586
q13	3566	3942	3314	3314
q14	287	307	272	272
q15	518	483	479	479
q16	651	696	650	650
q17	1215	1735	1251	1251
q18	7752	7482	7271	7271
q19	792	1112	1070	1070
q20	2000	2064	1901	1901
q21	5741	5320	5158	5158
q22	615	640	563	563
Total cold run time: 53357 ms
Total hot run time: 51814 ms

doris-robot · 2025-01-24T11:16:57Z

TPC-DS: Total hot run time: 185183 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3a18587dce44b03d4d063ddd82c0e4c2e6557467, data reload: false

query1	958	370	387	370
query2	6532	2087	2089	2087
query3	6798	214	226	214
query4	33642	23234	23044	23044
query5	4323	615	453	453
query6	298	203	182	182
query7	4608	479	310	310
query8	301	242	232	232
query9	9403	2576	2577	2576
query10	476	311	271	271
query11	18352	15202	15000	15000
query12	157	104	100	100
query13	1659	514	390	390
query14	10388	6792	6802	6792
query15	217	196	179	179
query16	7796	608	460	460
query17	1598	719	546	546
query18	1977	392	288	288
query19	220	173	156	156
query20	118	107	111	107
query21	212	121	100	100
query22	4502	4563	4095	4095
query23	34004	33346	33104	33104
query24	6611	2335	2247	2247
query25	501	478	417	417
query26	893	279	160	160
query27	2001	466	331	331
query28	5369	2435	2419	2419
query29	657	557	441	441
query30	239	186	167	167
query31	955	858	794	794
query32	76	64	63	63
query33	519	363	323	323
query34	743	848	512	512
query35	808	857	741	741
query36	1012	1054	953	953
query37	118	102	87	87
query38	4303	4233	4060	4060
query39	1465	1404	1404	1404
query40	209	119	107	107
query41	116	51	53	51
query42	115	105	102	102
query43	531	536	499	499
query44	1304	810	792	792
query45	175	173	165	165
query46	848	1024	655	655
query47	1821	1838	1803	1803
query48	379	384	317	317
query49	741	495	391	391
query50	616	668	400	400
query51	4180	4172	4129	4129
query52	106	102	89	89
query53	229	259	183	183
query54	490	480	391	391
query55	81	82	76	76
query56	267	263	245	245
query57	1164	1169	1107	1107
query58	250	237	236	236
query59	3216	3171	3122	3122
query60	281	264	253	253
query61	116	116	114	114
query62	788	714	665	665
query63	224	192	186	186
query64	4281	1024	664	664
query65	3289	3183	3176	3176
query66	1047	407	343	343
query67	15959	15626	15729	15626
query68	5145	845	526	526
query69	484	298	258	258
query70	1233	1176	1150	1150
query71	371	286	257	257
query72	6209	3812	3876	3812
query73	652	756	363	363
query74	10356	9256	8661	8661
query75	3162	3176	2665	2665
query76	3182	1201	760	760
query77	462	357	281	281
query78	10129	9976	9357	9357
query79	3095	801	598	598
query80	664	530	442	442
query81	522	286	231	231
query82	579	152	123	123
query83	172	171	152	152
query84	315	102	78	78
query85	851	352	300	300
query86	397	311	307	307
query87	4459	4414	4506	4414
query88	4992	2167	2137	2137
query89	393	329	290	290
query90	1869	190	187	187
query91	130	138	116	116
query92	64	54	52	52
query93	2667	887	532	532
query94	689	406	278	278
query95	333	262	256	256
query96	484	614	276	276
query97	2834	2914	2789	2789
query98	237	201	200	200
query99	1286	1358	1273	1273
Total cold run time: 286049 ms
Total hot run time: 185183 ms

doris-robot · 2025-01-24T11:22:17Z

ClickBench: Total hot run time: 31.29 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3a18587dce44b03d4d063ddd82c0e4c2e6557467, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.03	0.04
query3	0.24	0.07	0.06
query4	1.61	0.11	0.10
query5	0.43	0.42	0.41
query6	1.14	0.67	0.65
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.58	0.50	0.49
query10	0.56	0.57	0.57
query11	0.16	0.11	0.10
query12	0.14	0.10	0.10
query13	0.63	0.60	0.59
query14	2.80	2.71	2.73
query15	0.89	0.83	0.83
query16	0.38	0.40	0.39
query17	1.07	1.06	0.99
query18	0.23	0.20	0.21
query19	1.91	1.80	1.92
query20	0.02	0.02	0.01
query21	15.36	0.90	0.58
query22	0.75	0.88	0.70
query23	15.19	1.50	0.57
query24	2.97	1.84	1.41
query25	0.18	0.16	0.12
query26	0.32	0.15	0.14
query27	0.05	0.07	0.06
query28	14.09	0.98	0.43
query29	12.57	4.04	3.41
query30	0.25	0.09	0.07
query31	2.82	0.59	0.40
query32	3.23	0.58	0.46
query33	3.13	3.01	3.00
query34	16.19	5.23	4.54
query35	4.57	4.53	4.60
query36	0.65	0.49	0.47
query37	0.09	0.06	0.06
query38	0.04	0.04	0.03
query39	0.04	0.02	0.03
query40	0.17	0.14	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 105.79 s
Total hot run time: 31.29 s

dataroaring

LGTM

github-actions · 2025-01-24T14:46:53Z

PR approved by at least one committer and no changes requested.

bobhan1 · 2025-01-25T11:28:13Z

fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/DeleteFromCommand.java

@@ -475,7 +475,8 @@ public LogicalPlan completeQueryPlan(ConnectContext ctx, LogicalPlan logicalQuer
                expr = new UnboundAlias(new TinyIntLiteral(((byte) 1)), Column.DELETE_SIGN);
            } else if (column.getName().equalsIgnoreCase(Column.SEQUENCE_COL)


is it better to change like this?

} else if (column.getName().equalsIgnoreCase(Column.SEQUENCE_COL)) { expr = new UnboundSlot(tableName, Column.SEQUENCE_COL); }

dataroaring

LGTM

dataroaring

LGTM

…ry output for delete from command (#47406) for mow table with `function_column.sequence_col` map, if use `delete from` command and run into partial update, may get: ``` errCode = 2, detailMessage = insert into cols should be corresponding to the query output ```

… to the query output for delete from command #47406 (#47454) Cherry-picked from #47406 Co-authored-by: meiyi <[email protected]>

zhannngchen previously approved these changes Jan 24, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 24, 2025

github-actions bot added the reviewed label Jan 24, 2025

zhannngchen added dev/3.0.x p0_b labels Jan 24, 2025

dataroaring previously approved these changes Jan 24, 2025

View reviewed changes

mymeiyi dismissed stale reviews from dataroaring and zhannngchen via 30b28e7 January 24, 2025 10:27

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jan 24, 2025

fix

3a18587

mymeiyi force-pushed the fix-delete-from branch from 30b28e7 to 3a18587 Compare January 24, 2025 10:33

dataroaring approved these changes Jan 24, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 24, 2025

bobhan1 approved these changes Jan 25, 2025

View reviewed changes

bobhan1 reviewed Jan 25, 2025

View reviewed changes

dataroaring approved these changes Jan 26, 2025

View reviewed changes

gavinchou approved these changes Jan 26, 2025

View reviewed changes

gavinchou merged commit c9fb76b into apache:master Jan 26, 2025
30 of 36 checks passed

github-actions bot mentioned this pull request Jan 26, 2025

branch-3.0: [fix](delete) fix insert into cols should be corresponding to the query output for delete from command #47406 #47454

Merged

dataroaring pushed a commit that referenced this pull request Jan 26, 2025

branch-3.0 [fix](delete) fix insert into cols should be corresponding…

2ebf871

… to the query output for delete from command #47406 (#47454) Cherry-picked from #47406 Co-authored-by: meiyi <[email protected]>

dataroaring added dev/3.0.4-merged and removed dev/3.0.x labels Jan 26, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[fix](delete) fix insert into cols should be corresponding to the query output for delete from command #47406

[fix](delete) fix insert into cols should be corresponding to the query output for delete from command #47406

mymeiyi commented Jan 24, 2025

hello-stephen commented Jan 24, 2025

zhannngchen left a comment

github-actions bot commented Jan 24, 2025

github-actions bot commented Jan 24, 2025

mymeiyi commented Jan 24, 2025

doris-robot commented Jan 24, 2025

doris-robot commented Jan 24, 2025

doris-robot commented Jan 24, 2025

dataroaring left a comment

mymeiyi commented Jan 24, 2025

doris-robot commented Jan 24, 2025

doris-robot commented Jan 24, 2025

doris-robot commented Jan 24, 2025

dataroaring left a comment

github-actions bot commented Jan 24, 2025

bobhan1 Jan 25, 2025

dataroaring left a comment

dataroaring left a comment

		@@ -475,7 +475,8 @@ public LogicalPlan completeQueryPlan(ConnectContext ctx, LogicalPlan logicalQuer
		expr = new UnboundAlias(new TinyIntLiteral(((byte) 1)), Column.DELETE_SIGN);
		} else if (column.getName().equalsIgnoreCase(Column.SEQUENCE_COL)

[fix](delete) fix insert into cols should be corresponding to the query output for delete from command #47406

[fix](delete) fix insert into cols should be corresponding to the query output for delete from command #47406

Conversation

mymeiyi commented Jan 24, 2025

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

hello-stephen commented Jan 24, 2025

zhannngchen left a comment

Choose a reason for hiding this comment

github-actions bot commented Jan 24, 2025

github-actions bot commented Jan 24, 2025

mymeiyi commented Jan 24, 2025

doris-robot commented Jan 24, 2025

doris-robot commented Jan 24, 2025

doris-robot commented Jan 24, 2025

dataroaring left a comment

Choose a reason for hiding this comment

mymeiyi commented Jan 24, 2025

doris-robot commented Jan 24, 2025

doris-robot commented Jan 24, 2025

doris-robot commented Jan 24, 2025

dataroaring left a comment

Choose a reason for hiding this comment

github-actions bot commented Jan 24, 2025

bobhan1 Jan 25, 2025

Choose a reason for hiding this comment

dataroaring left a comment

Choose a reason for hiding this comment

dataroaring left a comment

Choose a reason for hiding this comment