Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

branch-3.0: [Enhancement](jdbc catalog) Support reading some array types of PostgreSQL #44160 #45219

Merged
merged 1 commit into from
Dec 20, 2024
Merged

branch-3.0: [Enhancement](jdbc catalog) Support reading some array types of PostgreSQL #44160 #45219

merged 1 commit into from
Dec 20, 2024

Conversation

github-actions[bot]
Copy link
Contributor

Cherry-picked from #44160

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Dec 10, 2024
@doris-robot
Copy link

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40330 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit eef87794de09f28b610f045f5428f4c1afb8f879, data reload: false

------ Round 1 ----------------------------------
q1	17638	7419	7221	7221
q2	2063	165	179	165
q3	10857	1072	1145	1072
q4	10546	757	780	757
q5	7728	2833	2771	2771
q6	233	150	144	144
q7	963	621	612	612
q8	9591	1934	2004	1934
q9	8087	6372	6409	6372
q10	6980	2296	2289	2289
q11	456	265	267	265
q12	395	226	215	215
q13	17787	2946	2954	2946
q14	231	199	214	199
q15	570	522	524	522
q16	674	592	601	592
q17	966	558	491	491
q18	7216	6489	6532	6489
q19	1704	1052	1072	1052
q20	461	194	192	192
q21	4013	3106	3050	3050
q22	1042	996	980	980
Total cold run time: 110201 ms
Total hot run time: 40330 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7327	7201	7252	7201
q2	325	233	224	224
q3	2888	2901	2880	2880
q4	2029	1742	1793	1742
q5	5665	5676	5668	5668
q6	224	142	144	142
q7	2179	1781	1784	1781
q8	3319	3527	3408	3408
q9	8843	8837	8772	8772
q10	3537	3501	3481	3481
q11	580	533	497	497
q12	819	601	630	601
q13	16461	3121	3125	3121
q14	297	275	265	265
q15	572	539	515	515
q16	723	678	685	678
q17	1854	1605	1592	1592
q18	8266	7725	7563	7563
q19	1737	1558	1519	1519
q20	2050	1867	1845	1845
q21	5426	5299	5221	5221
q22	1106	1006	1010	1006
Total cold run time: 76227 ms
Total hot run time: 59722 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 194284 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit eef87794de09f28b610f045f5428f4c1afb8f879, data reload: false

query1	1229	916	948	916
query2	6232	2070	2025	2025
query3	10821	4049	3962	3962
query4	66434	29048	23335	23335
query5	5508	441	445	441
query6	474	185	175	175
query7	6033	331	317	317
query8	333	226	228	226
query9	9143	2665	2644	2644
query10	513	275	257	257
query11	17848	15167	15909	15167
query12	173	106	112	106
query13	1563	459	425	425
query14	10782	6395	6505	6395
query15	214	184	179	179
query16	7214	521	487	487
query17	1167	564	573	564
query18	1817	329	310	310
query19	211	162	162	162
query20	119	112	115	112
query21	66	44	53	44
query22	4704	4478	4331	4331
query23	34712	34015	33841	33841
query24	5863	2908	2840	2840
query25	534	405	391	391
query26	665	164	160	160
query27	1814	294	307	294
query28	4368	2523	2520	2520
query29	675	441	419	419
query30	247	169	169	169
query31	1008	821	824	821
query32	64	53	53	53
query33	478	277	286	277
query34	922	504	486	486
query35	859	750	735	735
query36	1099	943	947	943
query37	116	72	77	72
query38	4201	4034	4085	4034
query39	1525	1469	1609	1469
query40	136	81	82	81
query41	49	48	46	46
query42	113	100	93	93
query43	530	488	487	487
query44	1160	788	830	788
query45	183	169	166	166
query46	1142	720	720	720
query47	1972	1968	1912	1912
query48	472	371	384	371
query49	736	394	374	374
query50	834	434	417	417
query51	7329	7136	6991	6991
query52	98	91	87	87
query53	251	180	183	180
query54	536	451	438	438
query55	75	75	74	74
query56	248	225	228	225
query57	1183	1089	1122	1089
query58	205	198	195	195
query59	3258	2775	2920	2775
query60	275	247	245	245
query61	106	107	102	102
query62	768	668	659	659
query63	208	183	189	183
query64	1475	652	633	633
query65	3235	3148	3231	3148
query66	720	293	298	293
query67	15730	15201	15351	15201
query68	4584	557	535	535
query69	409	247	252	247
query70	1202	1002	1133	1002
query71	348	252	263	252
query72	6551	3983	4156	3983
query73	749	337	333	333
query74	10194	8984	8933	8933
query75	3271	2636	2641	2636
query76	1889	1009	1022	1009
query77	496	259	260	259
query78	10618	9753	9532	9532
query79	3031	595	597	595
query80	2262	418	429	418
query81	565	239	235	235
query82	698	117	117	117
query83	261	143	143	143
query84	288	84	78	78
query85	1241	305	293	293
query86	454	300	303	300
query87	4416	4308	4287	4287
query88	5106	2368	2382	2368
query89	413	291	277	277
query90	2090	184	186	184
query91	172	144	141	141
query92	63	47	47	47
query93	4079	530	537	530
query94	888	289	304	289
query95	345	248	249	248
query96	617	275	284	275
query97	3342	3171	3156	3156
query98	220	196	193	193
query99	1865	1313	1312	1312
Total cold run time: 327502 ms
Total hot run time: 194284 ms

@zy-kkk @morningman
[Enhancement](jdbc catalog) Support reading some array types of Postg…
15e3937 
…reSQL (#44160)

### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

This PR supports reading some PostgreSQL array types, including:

```
{
                "int2", "int4", "int8", "smallserial", "serial",
                "bigserial", "float4", "float8", "timestamp", "timestamptz",
                "date", "bool", "bpchar", "varchar", "text"
        }
```


### Release note

[feat](jdbc catalog) Support reading some array types of PostgreSQL
@morningman morningman force-pushed the auto-pick-44160-branch-3.0 branch from eef8779 to 15e3937 Compare December 20, 2024 01:40
@morningman
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40238 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 15e3937236fba82e05d95de767fbb7b3813729ff, data reload: false

------ Round 1 ----------------------------------
q1	17575	7329	7236	7236
q2	2052	176	153	153
q3	10732	1051	1179	1051
q4	10532	730	816	730
q5	7745	2803	2793	2793
q6	232	145	143	143
q7	947	614	607	607
q8	9575	1979	1952	1952
q9	7596	6402	6369	6369
q10	7018	2280	2311	2280
q11	453	251	252	251
q12	402	222	209	209
q13	17814	2966	3069	2966
q14	262	224	220	220
q15	577	534	518	518
q16	665	615	591	591
q17	981	600	547	547
q18	7329	6682	6353	6353
q19	1377	997	1053	997
q20	482	192	191	191
q21	3952	3272	3097	3097
q22	1102	1016	984	984
Total cold run time: 109400 ms
Total hot run time: 40238 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7328	7238	7215	7215
q2	326	236	234	234
q3	2879	2916	2918	2916
q4	2009	1776	1798	1776
q5	5647	5659	5651	5651
q6	223	142	144	142
q7	2183	1759	1794	1759
q8	3309	3536	3457	3457
q9	8775	8832	8803	8803
q10	3544	3502	3508	3502
q11	586	500	490	490
q12	786	626	607	607
q13	16517	3114	3119	3114
q14	307	270	269	269
q15	576	529	530	529
q16	713	669	694	669
q17	1858	1634	1615	1615
q18	8086	7711	7459	7459
q19	6509	1606	1549	1549
q20	2096	1828	1854	1828
q21	5536	5202	5284	5202
q22	1144	1043	989	989
Total cold run time: 80937 ms
Total hot run time: 59775 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 195567 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 15e3937236fba82e05d95de767fbb7b3813729ff, data reload: false

query1	1332	951	916	916
query2	6251	2053	2046	2046
query3	10951	4235	3987	3987
query4	65962	28375	23562	23562
query5	5321	445	426	426
query6	416	178	175	175
query7	5685	306	318	306
query8	305	231	241	231
query9	9234	2702	2682	2682
query10	501	266	263	263
query11	17773	15147	15850	15147
query12	154	100	104	100
query13	1575	454	415	415
query14	10467	6843	7554	6843
query15	221	171	178	171
query16	7365	525	554	525
query17	1047	585	578	578
query18	1885	322	326	322
query19	241	158	159	158
query20	118	115	111	111
query21	64	45	45	45
query22	4546	4496	4248	4248
query23	34588	34091	34102	34091
query24	6092	3002	2973	2973
query25	550	413	400	400
query26	674	174	171	171
query27	1798	312	311	311
query28	4129	2563	2526	2526
query29	721	460	454	454
query30	245	170	161	161
query31	1038	825	813	813
query32	64	56	53	53
query33	471	281	286	281
query34	902	502	515	502
query35	846	720	740	720
query36	1085	964	955	955
query37	119	70	74	70
query38	4092	4046	4065	4046
query39	1507	1479	1488	1479
query40	141	80	80	80
query41	48	47	48	47
query42	113	96	101	96
query43	537	511	500	500
query44	1201	791	793	791
query45	189	167	168	167
query46	1173	761	756	756
query47	1964	1851	1870	1851
query48	470	384	372	372
query49	723	393	364	364
query50	845	424	419	419
query51	7232	7128	7159	7128
query52	92	90	87	87
query53	262	184	194	184
query54	573	454	446	446
query55	78	73	73	73
query56	256	236	242	236
query57	1182	1096	1087	1087
query58	203	202	196	196
query59	3363	2843	2832	2832
query60	261	248	248	248
query61	106	109	108	108
query62	768	667	663	663
query63	205	179	185	179
query64	1495	656	618	618
query65	3267	3169	3245	3169
query66	719	299	299	299
query67	15947	15232	15299	15232
query68	4564	541	539	539
query69	432	248	258	248
query70	1170	1096	1076	1076
query71	418	251	246	246
query72	6574	3953	3892	3892
query73	764	344	340	340
query74	10264	8837	9026	8837
query75	3318	2650	2594	2594
query76	1968	1010	1043	1010
query77	488	270	269	269
query78	10538	9754	9574	9574
query79	4757	592	600	592
query80	1965	419	421	419
query81	563	235	236	235
query82	882	119	116	116
query83	256	143	140	140
query84	292	80	80	80
query85	1722	314	291	291
query86	495	310	301	301
query87	4412	4314	4242	4242
query88	5201	2400	2406	2400
query89	421	290	304	290
query90	2009	183	187	183
query91	181	145	142	142
query92	68	47	58	47
query93	5598	549	531	531
query94	874	296	284	284
query95	347	255	251	251
query96	646	282	277	277
query97	3312	3151	3129	3129
query98	229	207	204	204
query99	1585	1303	1297	1297
Total cold run time: 330061 ms
Total hot run time: 195567 ms

@morningman morningman merged commit 85e2e99 into branch-3.0 Dec 20, 2024
27 of 30 checks passed
@github-actions github-actions bot deleted the auto-pick-44160-branch-3.0 branch December 20, 2024 03:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@doris-robot @morningman @dataroaring @zy-kkk