Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[refactor](execenv) remove shared ptr from exec env #46034

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

yiguolei
Copy link
Contributor

What problem does this PR solve?

ExecEnv should be the last object to deconstructed, so that it should not own any shared ptr. If it own any shared ptr, then we could not make sure the deconstruct sequence.

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@yiguolei
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32766 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bfca486003d0d956bf3c59687b482018fedf2a8c, data reload: false

------ Round 1 ----------------------------------
q1	17610	6381	6046	6046
q2	2047	307	173	173
q3	10415	1255	779	779
q4	10209	891	448	448
q5	7521	2201	2013	2013
q6	214	180	147	147
q7	913	776	609	609
q8	9242	1408	1229	1229
q9	5237	4902	4858	4858
q10	6759	2303	1864	1864
q11	489	286	259	259
q12	356	362	235	235
q13	17756	3651	2987	2987
q14	255	236	217	217
q15	566	515	500	500
q16	637	626	588	588
q17	607	878	334	334
q18	7250	6455	6379	6379
q19	2163	978	559	559
q20	303	322	192	192
q21	2901	2313	2038	2038
q22	368	335	312	312
Total cold run time: 103818 ms
Total hot run time: 32766 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6304	6295	6227	6227
q2	238	336	233	233
q3	2291	2697	2292	2292
q4	1513	1872	1373	1373
q5	4394	4768	4875	4768
q6	195	180	152	152
q7	2138	1983	1863	1863
q8	2661	2876	2724	2724
q9	7406	7349	7414	7349
q10	3091	3392	2830	2830
q11	581	503	496	496
q12	684	798	635	635
q13	3378	3776	3115	3115
q14	296	318	274	274
q15	591	516	515	515
q16	635	691	659	659
q17	1274	1768	1264	1264
q18	7927	7597	7529	7529
q19	870	1244	1112	1112
q20	2082	2006	1916	1916
q21	5748	5217	5028	5028
q22	622	592	587	587
Total cold run time: 54919 ms
Total hot run time: 52941 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197304 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bfca486003d0d956bf3c59687b482018fedf2a8c, data reload: false

query1	1352	971	908	908
query2	6498	2319	2446	2319
query3	10978	4546	4533	4533
query4	33129	23775	23634	23634
query5	3606	603	459	459
query6	272	217	203	203
query7	4003	506	311	311
query8	312	256	246	246
query9	9332	2649	2656	2649
query10	472	314	261	261
query11	17886	15666	15677	15666
query12	159	110	106	106
query13	1715	566	406	406
query14	12211	6937	7585	6937
query15	243	201	185	185
query16	7963	679	493	493
query17	1510	766	625	625
query18	2045	404	322	322
query19	210	188	154	154
query20	124	126	114	114
query21	200	123	109	109
query22	4678	4561	4632	4561
query23	34623	33570	33792	33570
query24	6169	2341	2352	2341
query25	503	472	384	384
query26	744	283	151	151
query27	2230	469	329	329
query28	5422	2473	2480	2473
query29	638	550	437	437
query30	207	187	155	155
query31	1011	933	837	837
query32	78	60	58	58
query33	490	359	297	297
query34	759	884	508	508
query35	858	886	768	768
query36	996	1068	989	989
query37	124	105	73	73
query38	4436	4357	4183	4183
query39	1552	1499	1471	1471
query40	218	116	99	99
query41	45	43	44	43
query42	118	103	106	103
query43	549	556	514	514
query44	1391	846	825	825
query45	190	176	167	167
query46	909	1079	666	666
query47	1999	1991	1948	1948
query48	398	420	344	344
query49	714	479	394	394
query50	649	693	410	410
query51	7294	7464	7127	7127
query52	106	99	92	92
query53	238	256	191	191
query54	476	507	414	414
query55	87	84	84	84
query56	262	262	280	262
query57	1249	1267	1149	1149
query58	246	229	232	229
query59	3310	3557	3212	3212
query60	276	265	260	260
query61	117	106	134	106
query62	889	851	795	795
query63	228	190	204	190
query64	3095	1052	691	691
query65	3493	3314	3265	3265
query66	778	412	319	319
query67	16673	15836	15532	15532
query68	9586	824	554	554
query69	485	293	253	253
query70	1244	1151	1163	1151
query71	428	316	254	254
query72	6256	3878	3837	3837
query73	666	749	352	352
query74	10039	9385	8982	8982
query75	4629	3177	2628	2628
query76	4920	1187	772	772
query77	891	379	272	272
query78	10164	10286	9397	9397
query79	2864	908	598	598
query80	740	510	432	432
query81	482	271	225	225
query82	620	152	118	118
query83	189	171	152	152
query84	349	88	75	75
query85	787	368	319	319
query86	352	322	301	301
query87	4475	4430	4521	4430
query88	3272	2180	2151	2151
query89	431	331	297	297
query90	1877	186	191	186
query91	124	131	103	103
query92	72	55	52	52
query93	1793	895	541	541
query94	659	390	300	300
query95	344	266	249	249
query96	493	611	282	282
query97	2813	2822	2747	2747
query98	225	202	196	196
query99	1680	1573	1442	1442
Total cold run time: 299684 ms
Total hot run time: 197304 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.06 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bfca486003d0d956bf3c59687b482018fedf2a8c, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.03	0.03
query3	0.23	0.07	0.06
query4	1.61	0.10	0.10
query5	0.41	0.41	0.41
query6	1.16	0.65	0.64
query7	0.03	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.50	0.53
query10	0.55	0.60	0.55
query11	0.15	0.10	0.10
query12	0.13	0.11	0.10
query13	0.61	0.61	0.59
query14	2.84	2.76	2.76
query15	0.88	0.82	0.82
query16	0.40	0.39	0.37
query17	1.07	1.07	1.07
query18	0.23	0.21	0.21
query19	1.97	1.88	2.03
query20	0.01	0.01	0.01
query21	15.37	0.95	0.59
query22	0.77	0.80	0.63
query23	15.31	1.47	0.60
query24	4.10	1.69	2.42
query25	0.18	0.15	0.08
query26	0.24	0.14	0.13
query27	0.06	0.06	0.05
query28	14.78	1.40	1.05
query29	12.56	3.87	3.22
query30	0.25	0.09	0.06
query31	2.81	0.59	0.37
query32	3.24	0.54	0.46
query33	3.05	3.10	3.22
query34	17.04	5.10	4.48
query35	4.53	4.48	4.45
query36	0.67	0.50	0.51
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.02
query40	0.17	0.14	0.13
query41	0.08	0.03	0.03
query42	0.04	0.03	0.02
query43	0.03	0.04	0.03
Total cold run time: 108.45 s
Total hot run time: 32.06 s

@yiguolei
Copy link
Contributor Author

run buildall

1 similar comment
@yiguolei
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 33124 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e7a7fcfc5c7d6ac6f402a3462e2b33d3149c9b6f, data reload: false

------ Round 1 ----------------------------------
q1	17580	6180	6689	6180
q2	2050	315	183	183
q3	10417	1263	784	784
q4	10546	878	438	438
q5	9484	2213	2016	2016
q6	213	177	145	145
q7	910	780	610	610
q8	9243	1404	1231	1231
q9	5258	4978	4965	4965
q10	6774	2347	1869	1869
q11	474	285	254	254
q12	351	372	222	222
q13	17768	3594	2940	2940
q14	232	239	213	213
q15	565	509	498	498
q16	635	616	596	596
q17	595	869	355	355
q18	7054	6489	6516	6489
q19	2305	987	552	552
q20	322	316	193	193
q21	3049	2207	2081	2081
q22	373	339	310	310
Total cold run time: 106198 ms
Total hot run time: 33124 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6280	6284	6243	6243
q2	235	322	232	232
q3	2302	2713	2327	2327
q4	1408	1902	1350	1350
q5	4384	4868	4979	4868
q6	180	173	141	141
q7	2088	2027	1891	1891
q8	2647	2838	2686	2686
q9	7360	7230	7288	7230
q10	3102	3284	2775	2775
q11	590	512	527	512
q12	696	771	634	634
q13	3397	3795	3138	3138
q14	287	316	276	276
q15	587	530	507	507
q16	643	700	666	666
q17	1231	1728	1252	1252
q18	7739	7522	7403	7403
q19	906	977	1168	977
q20	1995	2051	1858	1858
q21	5863	5280	5040	5040
q22	638	646	603	603
Total cold run time: 54558 ms
Total hot run time: 52609 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197069 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e7a7fcfc5c7d6ac6f402a3462e2b33d3149c9b6f, data reload: false

query1	1283	968	895	895
query2	6489	2307	2301	2301
query3	10992	4659	4695	4659
query4	32906	24170	23523	23523
query5	4351	633	465	465
query6	294	198	182	182
query7	3998	492	313	313
query8	309	256	246	246
query9	9582	2775	2765	2765
query10	471	311	250	250
query11	18055	15597	15100	15100
query12	153	107	104	104
query13	1593	544	416	416
query14	9663	6762	7411	6762
query15	249	208	204	204
query16	8041	630	461	461
query17	1581	743	589	589
query18	2108	430	324	324
query19	251	188	163	163
query20	128	119	120	119
query21	210	124	111	111
query22	4611	4567	4553	4553
query23	35320	33678	34061	33678
query24	6706	2307	2361	2307
query25	474	451	388	388
query26	758	275	160	160
query27	2134	477	348	348
query28	5961	2496	2519	2496
query29	555	582	432	432
query30	209	186	152	152
query31	976	928	874	874
query32	72	65	59	59
query33	478	379	307	307
query34	772	871	523	523
query35	793	832	774	774
query36	1021	1049	979	979
query37	120	98	78	78
query38	4327	4343	4168	4168
query39	1544	1452	1501	1452
query40	207	114	98	98
query41	45	43	43	43
query42	122	110	103	103
query43	520	519	503	503
query44	1391	870	826	826
query45	188	178	176	176
query46	906	1059	662	662
query47	2048	2004	1950	1950
query48	397	446	332	332
query49	736	485	418	418
query50	638	673	398	398
query51	7365	7110	7231	7110
query52	105	99	89	89
query53	233	261	200	200
query54	482	503	419	419
query55	87	81	80	80
query56	270	257	264	257
query57	1256	1270	1149	1149
query58	242	237	224	224
query59	3112	3284	3120	3120
query60	280	269	256	256
query61	110	110	105	105
query62	845	827	807	807
query63	229	198	194	194
query64	3128	1036	682	682
query65	3371	3301	3306	3301
query66	793	412	293	293
query67	16616	15865	15631	15631
query68	9275	776	519	519
query69	481	294	249	249
query70	1204	1167	1137	1137
query71	435	295	260	260
query72	6236	3974	3930	3930
query73	670	765	369	369
query74	10097	9060	8929	8929
query75	4589	3228	2689	2689
query76	5155	1188	783	783
query77	974	368	273	273
query78	10120	10185	9705	9705
query79	4491	897	582	582
query80	716	526	470	470
query81	483	266	231	231
query82	309	152	121	121
query83	196	162	147	147
query84	286	89	75	75
query85	745	357	303	303
query86	347	317	307	307
query87	4605	4689	4371	4371
query88	3817	2213	2194	2194
query89	430	332	300	300
query90	2125	192	190	190
query91	138	135	103	103
query92	65	56	53	53
query93	2526	937	523	523
query94	677	389	290	290
query95	343	271	256	256
query96	494	609	278	278
query97	2781	2851	2683	2683
query98	218	212	198	198
query99	1669	1570	1445	1445
Total cold run time: 301957 ms
Total hot run time: 197069 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.12 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e7a7fcfc5c7d6ac6f402a3462e2b33d3149c9b6f, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.03
query3	0.24	0.07	0.07
query4	1.62	0.10	0.10
query5	0.41	0.41	0.42
query6	1.16	0.65	0.65
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.59	0.50	0.52
query10	0.56	0.58	0.55
query11	0.14	0.10	0.11
query12	0.14	0.12	0.11
query13	0.62	0.60	0.60
query14	2.82	2.71	2.85
query15	0.91	0.83	0.82
query16	0.39	0.40	0.38
query17	1.01	1.03	1.06
query18	0.22	0.22	0.21
query19	1.91	1.87	1.99
query20	0.01	0.01	0.02
query21	15.38	0.91	0.58
query22	0.76	0.91	0.65
query23	15.17	1.40	0.54
query24	3.03	0.71	1.57
query25	0.11	0.12	0.17
query26	0.29	0.15	0.14
query27	0.05	0.05	0.06
query28	13.90	1.53	1.05
query29	12.64	3.98	3.32
query30	0.25	0.09	0.07
query31	2.82	0.60	0.38
query32	3.23	0.56	0.47
query33	3.07	3.10	3.09
query34	16.92	5.08	4.47
query35	4.50	4.45	4.45
query36	0.64	0.50	0.49
query37	0.10	0.07	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.02
query40	0.16	0.13	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.19 s
Total hot run time: 31.12 s

@yiguolei
Copy link
Contributor Author

run beut

3 similar comments
@yiguolei
Copy link
Contributor Author

run beut

@yiguolei
Copy link
Contributor Author

run beut

@yiguolei
Copy link
Contributor Author

run beut

@yiguolei
Copy link
Contributor Author

run beut

@yiguolei
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.91% (10122/26016)
Line Coverage: 29.90% (85545/286093)
Region Coverage: 29.03% (43726/150609)
Branch Coverage: 25.57% (22307/87256)
Coverage Report: http://coverage.selectdb-in.cc/coverage/bfff8f1ecdbb79177aa2d142ac565db13a3450e4_bfff8f1ecdbb79177aa2d142ac565db13a3450e4/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 32976 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bfff8f1ecdbb79177aa2d142ac565db13a3450e4, data reload: false

------ Round 1 ----------------------------------
q1	17573	6202	6043	6043
q2	2045	312	188	188
q3	10392	1283	765	765
q4	10225	911	440	440
q5	7562	2220	2012	2012
q6	204	185	149	149
q7	904	764	613	613
q8	9251	1431	1236	1236
q9	5311	4910	4931	4910
q10	6780	2322	1887	1887
q11	483	283	273	273
q12	354	372	226	226
q13	17786	3595	3029	3029
q14	231	232	209	209
q15	577	511	496	496
q16	637	607	592	592
q17	596	869	363	363
q18	7180	6459	6432	6432
q19	1643	990	567	567
q20	330	324	190	190
q21	2894	2305	2045	2045
q22	369	343	311	311
Total cold run time: 103327 ms
Total hot run time: 32976 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6291	6279	6228	6228
q2	264	343	227	227
q3	2285	2673	2317	2317
q4	1439	1800	1383	1383
q5	4398	4831	4846	4831
q6	193	177	144	144
q7	2096	2033	1812	1812
q8	2754	2835	2679	2679
q9	7379	7297	7411	7297
q10	3087	3386	2818	2818
q11	577	511	508	508
q12	645	767	658	658
q13	3455	3753	3195	3195
q14	288	311	279	279
q15	580	514	512	512
q16	666	698	631	631
q17	1243	1744	1288	1288
q18	7721	7493	7480	7480
q19	881	1155	1149	1149
q20	2003	2025	1998	1998
q21	5788	5361	5085	5085
q22	644	622	615	615
Total cold run time: 54677 ms
Total hot run time: 53134 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197826 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bfff8f1ecdbb79177aa2d142ac565db13a3450e4, data reload: false

query1	1301	958	948	948
query2	6393	2358	2328	2328
query3	11021	4776	4692	4692
query4	33092	24221	26485	24221
query5	4324	622	473	473
query6	282	215	201	201
query7	3994	514	304	304
query8	313	237	225	225
query9	9499	2713	2729	2713
query10	452	319	265	265
query11	18320	15435	15309	15309
query12	160	107	103	103
query13	1562	580	421	421
query14	9609	7064	8292	7064
query15	277	219	195	195
query16	8136	634	510	510
query17	1570	803	613	613
query18	2177	429	329	329
query19	231	201	167	167
query20	127	120	120	120
query21	208	135	115	115
query22	4468	4609	4413	4413
query23	34865	33479	34001	33479
query24	6817	2295	2412	2295
query25	491	493	390	390
query26	1036	285	156	156
query27	2383	488	330	330
query28	5384	2483	2483	2483
query29	669	563	428	428
query30	208	183	151	151
query31	979	932	850	850
query32	88	61	58	58
query33	532	379	297	297
query34	795	862	532	532
query35	802	835	807	807
query36	1016	1076	967	967
query37	112	94	79	79
query38	4107	4198	4196	4196
query39	1522	1464	1476	1464
query40	220	115	103	103
query41	44	45	43	43
query42	113	102	111	102
query43	526	530	487	487
query44	1376	835	832	832
query45	192	176	173	173
query46	886	1061	682	682
query47	1976	2008	1924	1924
query48	379	423	324	324
query49	731	485	388	388
query50	682	698	417	417
query51	7320	7297	7235	7235
query52	106	103	95	95
query53	236	259	190	190
query54	494	508	426	426
query55	84	84	80	80
query56	263	260	245	245
query57	1268	1249	1164	1164
query58	245	237	259	237
query59	3212	3353	3133	3133
query60	284	271	260	260
query61	107	112	110	110
query62	889	809	759	759
query63	228	192	200	192
query64	3654	1025	689	689
query65	3419	3305	3295	3295
query66	795	401	304	304
query67	16591	15880	15625	15625
query68	9496	771	511	511
query69	487	290	250	250
query70	1204	1126	1128	1126
query71	430	297	252	252
query72	6013	3842	3795	3795
query73	677	774	367	367
query74	9849	9159	8972	8972
query75	4607	3193	2630	2630
query76	5113	1192	776	776
query77	893	369	279	279
query78	10269	10420	9395	9395
query79	3274	921	587	587
query80	743	527	437	437
query81	493	274	234	234
query82	602	156	125	125
query83	187	161	152	152
query84	288	91	77	77
query85	787	384	315	315
query86	352	371	267	267
query87	4648	4577	4524	4524
query88	3554	2251	2207	2207
query89	430	338	294	294
query90	1811	190	194	190
query91	138	130	105	105
query92	64	56	50	50
query93	1201	888	550	550
query94	663	387	290	290
query95	330	270	265	265
query96	485	606	277	277
query97	2791	2823	2688	2688
query98	229	202	203	202
query99	1666	1575	1432	1432
Total cold run time: 299416 ms
Total hot run time: 197826 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.43 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bfff8f1ecdbb79177aa2d142ac565db13a3450e4, data reload: false

query1	0.03	0.03	0.03
query2	0.08	0.03	0.03
query3	0.24	0.07	0.07
query4	1.60	0.11	0.11
query5	0.44	0.43	0.42
query6	1.14	0.65	0.65
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.60	0.51	0.50
query10	0.56	0.58	0.57
query11	0.15	0.10	0.10
query12	0.14	0.11	0.11
query13	0.62	0.60	0.59
query14	2.73	2.86	2.75
query15	0.92	0.83	0.82
query16	0.38	0.39	0.37
query17	0.98	0.99	0.96
query18	0.23	0.22	0.20
query19	1.84	1.76	2.02
query20	0.02	0.00	0.02
query21	15.38	0.94	0.58
query22	0.76	0.81	0.64
query23	15.23	1.48	0.51
query24	2.93	1.31	2.33
query25	0.19	0.06	0.16
query26	0.21	0.14	0.14
query27	0.06	0.04	0.03
query28	14.56	1.47	1.04
query29	12.66	3.99	3.29
query30	0.25	0.09	0.07
query31	2.82	0.59	0.37
query32	3.23	0.54	0.46
query33	3.13	3.08	3.12
query34	16.51	5.07	4.52
query35	4.53	4.47	4.46
query36	0.67	0.49	0.47
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.16	0.13	0.13
query41	0.09	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.38 s
Total hot run time: 31.43 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants