You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I had searched in the issues and found no similar issues.
Version
2.1.5
What's Wrong?
When inserting data into an auto-partitioned table, it is possible for the same partition name to be assigned multiple different partition IDs, indirectly leading to the creation of a large number of redundant tablets. In my environment, an insertion that should have created 100 tablets ended up creating over 9,000 tablets, causing instability in the cluster.
table:
CREATE TABLE base_info_m (
c_month varchar(255) NOT NULL
-- 其他列...
)ENGINE=OLAP
DUPLICATE KEY(c_month)
AUTO PARTITION BY LIST(c_month)
(
PARTITION p2024116 VALUES IN("202411")
)
DISTRIBUTED BY HASH('c_month') BUCKETS 100;
insert sql:
insert into base_info_m select * from t where t.c_month='202411'
fe log:
2024-12-18 18:18:44,131 INFO (thrift-server-pool-12189|559823) [InternalCatalog.createPartitionWithIndices():2106] succeed in creating partition[255404482-p2024116], table : [255230533-base_info_m]
2024-12-18 18:18:44,131 INFO (thrift-server-pool-12189|559823) [InternalCatalog.addPartition():1692] table[base_info_m] add partition[p2024116] which already exists
2024-12-18 18:18:44,133 INFO (thrift-server-pool-11499|499577) [InternalCatalog.createPartitionWithIndices():2106] succeed in creating partition[255298217-p2024116], table : [255230533-base_info_m]
2024-12-18 18:18:44,133 INFO (thrift-server-pool-11499|499577) [InternalCatalog.addPartition():1692] table[base_info_m] add partition[p2024116] which already exists
2024-12-18 18:18:44,150 INFO (thrift-server-pool-12021|559655) [InternalCatalog.createPartitionWithIndices():2106] succeed in creating partition[255316262-p2024116], table : [255230533-base_info_m]
2024-12-18 18:18:44,150 INFO (thrift-server-pool-12021|559655) [InternalCatalog.addPartition():1692] table[base_info_m] add partition[p2024116] which already exists
.....
One of the BE (Backend) logs
I20241218 18:18:42.194964 3360682 tablet_manager.cpp:272] begin to create tablet. tablet_id=255298230, table_id=255230533, partition_id=255298217, replica_id=255298233, stores.size=12, first store=/data02/doris-storage
I20241218 18:18:42.195721 3360680 tablet_manager.cpp:272] begin to create tablet. tablet_id=255298254, table_id=255230533, partition_id=255298217, replica_id=255298256, stores.size=12, first store=/data10/doris-storage
I20241218 18:18:42.196261 3360682 tablet_manager.cpp:272] begin to create tablet. tablet_id=255298278, table_id=255230533, partition_id=255298217, replica_id=255298279, stores.size=12, first store=/data06/doris-storage
I20241218 18:18:42.196411 3360681 tablet_manager.cpp:272] begin to create tablet. tablet_id=255298298, table_id=255230533, partition_id=255298217, replica_id=255298301, stores.size=12, first store=/data11/doris-storage
I20241218 18:18:42.197140 3360680 tablet_manager.cpp:272] begin to create tablet. tablet_id=255298322, table_id=255230533, partition_id=255298217, replica_id=255298324, stores.size=12, first store=/data08/doris-storage
I20241218 18:18:42.197643 3360681 tablet_manager.cpp:272] begin to create tablet. tablet_id=255298346, table_id=255230533, partition_id=255298217, replica_id=255298347, stores.size=12, first store=/data07/doris-storage
I20241218 18:18:42.197715 3360682 tablet_manager.cpp:272] begin to create tablet. tablet_id=255298366, table_id=255230533, partition_id=255298217, replica_id=255298369, stores.size=12, first store=/data03/doris-storage
I20241218 18:18:42.198566 3360680 tablet_manager.cpp:272] begin to create tablet. tablet_id=255298390, table_id=255230533, partition_id=255298217, replica_id=255298392, stores.size=12, first store=/data01/doris-storage
I20241218 18:18:42.198913 3360681 tablet_manager.cpp:272] begin to create tablet. tablet_id=255298414, table_id=255230533, partition_id=255298217, replica_id=255298415, stores.size=12, first store=/data04/doris-storage
I20241218 18:18:42.199162 3360682 tablet_manager.cpp:272] begin to create tablet. tablet_id=255298434, table_id=255230533, partition_id=255298217, replica_id=255298437, stores.size=12, first store=/data09/doris-storage
I20241218 18:18:42.200104 3360680 tablet_manager.cpp:272] begin to create tablet. tablet_id=255298458, table_id=255230533, partition_id=255298217, replica_id=255298460, stores.size=12, first store=/data05/doris-storage
I20241218 18:18:42.200398 3360682 tablet_manager.cpp:272] begin to create tablet. tablet_id=255298482, table_id=255230533, partition_id=255298217, replica_id=255298483, stores.size=12, first store=/data12/doris-storage
I20241218 18:18:42.200505 3360681 tablet_manager.cpp:272] begin to create tablet. tablet_id=255298502, table_id=255230533, partition_id=255298217, replica_id=255298505, stores.size=12, first store=/data02/doris-storage
I20241218 18:18:42.201678 3360682 tablet_manager.cpp:272] begin to create tablet. tablet_id=255298526, table_id=255230533, partition_id=255298217, replica_id=255298528, stores.size=12, first store=/data10/doris-storage
I20241218 18:18:42.201699 3360680 tablet_manager.cpp:272] begin to create tablet. tablet_id=255298550, table_id=255230533, partition_id=255298217, replica_id=255298551, stores.size=12, first store=/data06/doris-storage
I20241218 18:18:42.201918 3360681 tablet_manager.cpp:272] begin to create tablet. tablet_id=255298570, table_id=255230533, partition_id=255298217, replica_id=255298573, stores.size=12, first store=/data11/doris-storage
I20241218 18:18:42.202903 3360680 tablet_manager.cpp:272] begin to create tablet. tablet_id=255298594, table_id=255230533, partition_id=255298217, replica_id=255298596, stores.size=12, first store=/data08/doris-storage
I20241218 18:18:42.437721 3360681 tablet_manager.cpp:272] begin to create tablet. tablet_id=255404483, table_id=255230533, partition_id=255404482, replica_id=255404485, stores.size=12, first store=/data02/doris-storage
I20241218 18:18:42.437844 3360682 tablet_manager.cpp:272] begin to create tablet. tablet_id=255404507, table_id=255230533, partition_id=255404482, replica_id=255404508, stores.size=12, first store=/data10/doris-storage
I20241218 18:18:42.439217 3360681 tablet_manager.cpp:272] begin to create tablet. tablet_id=255404527, table_id=255230533, partition_id=255404482, replica_id=255404530, stores.size=12, first store=/data06/doris-storage
I20241218 18:18:42.439227 3360682 tablet_manager.cpp:272] begin to create tablet. tablet_id=255404551, table_id=255230533, partition_id=255404482, replica_id=255404553, stores.size=12, first store=/data11/doris-storage
I20241218 18:18:42.440524 3360681 tablet_manager.cpp:272] begin to create tablet. tablet_id=255404575, table_id=255230533, partition_id=255404482, replica_id=255404576, stores.size=12, first store=/data08/doris-storage
I20241218 18:18:42.440696 3360682 tablet_manager.cpp:272] begin to create tablet. tablet_id=255404595, table_id=255230533, partition_id=255404482, replica_id=255404598, stores.size=12, first store=/data07/doris-storage
I20241218 18:18:42.442117 3360681 tablet_manager.cpp:272] begin to create tablet. tablet_id=255404643, table_id=255230533, partition_id=255404482, replica_id=255404644, stores.size=12, first store=/data01/doris-storage
I20241218 18:18:42.442091 3360682 tablet_manager.cpp:272] begin to create tablet. tablet_id=255404619, table_id=255230533, partition_id=255404482, replica_id=255404621, stores.size=12, first store=/data03/doris-storage
I20241218 18:18:42.443413 3360682 tablet_manager.cpp:272] begin to create tablet. tablet_id=255404663, table_id=255230533, partition_id=255404482, replica_id=255404666, stores.size=12, first store=/data04/doris-storage
I20241218 18:18:42.443660 3360681 tablet_manager.cpp:272] begin to create tablet. tablet_id=255404687, table_id=255230533, partition_id=255404482, replica_id=255404689, stores.size=12, first store=/data09/doris-storage
I20241218 18:18:42.444985 3360682 tablet_manager.cpp:272] begin to create tablet. tablet_id=255404711, table_id=255230533, partition_id=255404482, replica_id=255404712, stores.size=12, first store=/data05/doris-storage
I20241218 18:18:42.445149 3360681 tablet_manager.cpp:272] begin to create tablet. tablet_id=255404731, table_id=255230533, partition_id=255404482, replica_id=255404734, stores.size=12, first store=/data12/doris-storage
I20241218 18:18:42.446177 3360682 tablet_manager.cpp:272] begin to create tablet. tablet_id=255404755, table_id=255230533, partition_id=255404482, replica_id=255404757, stores.size=12, first store=/data02/doris-storage
I20241218 18:18:42.446630 3360681 tablet_manager.cpp:272] begin to create tablet. tablet_id=255404779, table_id=255230533, partition_id=255404482, replica_id=255404780, stores.size=12, first store=/data10/doris-storage
I20241218 18:18:42.447506 3360682 tablet_manager.cpp:272] begin to create tablet. tablet_id=255404799, table_id=255230533, partition_id=255404482, replica_id=255404802, stores.size=12, first store=/data06/doris-storage
I20241218 18:18:42.447930 3360681 tablet_manager.cpp:272] begin to create tablet. tablet_id=255404823, table_id=255230533, partition_id=255404482, replica_id=255404825, stores.size=12, first store=/data11/doris-storage
I20241218 18:18:42.448966 3360682 tablet_manager.cpp:272] begin to create tablet. tablet_id=255404847, table_id=255230533, partition_id=255404482, replica_id=255404848, stores.size=12, first store=/data08/doris-storage
I20241218 18:18:42.450199 3360682 tablet_manager.cpp:272] begin to create tablet. tablet_id=255404867, table_id=255230533, partition_id=255404482, replica_id=255404870, stores.size=12, first store=/data07/doris-storage
What You Expected?
partition p2024116 should not be recreated repeatedly.
Search before asking
Version
2.1.5
What's Wrong?
When inserting data into an auto-partitioned table, it is possible for the same partition name to be assigned multiple different partition IDs, indirectly leading to the creation of a large number of redundant tablets. In my environment, an insertion that should have created 100 tablets ended up creating over 9,000 tablets, causing instability in the cluster.
table:
insert sql:
fe log:
One of the BE (Backend) logs
What You Expected?
partition p2024116 should not be recreated repeatedly.
How to Reproduce?
No response
Anything Else?
No response
Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: