配置冷热存储分离疑惑 #45597
goeason-world
started this conversation in
General
配置冷热存储分离疑惑
#45597
Replies: 2 comments
-
搞明白了, |
Beta Was this translation helpful? Give feedback.
0 replies
-
是的可以通过show tablets from table来查看是否已经存储到s3,也可以通过show partitions from table查看分区的冷热时间。 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
我这里有一个测试样例
CREATE RESOURCE "tencent_cos"
PROPERTIES (
"type" = "s3",
"s3.endpoint" = "cos.ap-guangzhou.myqcloud.com",
"s3.region" = "ap-guangzhou",
"s3.bucket" = "xxxxxxxxxxx",
"s3.access_key" = "xxxxxxxxxxx",
"s3.secret_key" = "xxxxxxxxxxx",
"s3.use_aws_sdk_default_behavior" = "false",
"s3.use_path_style" = "false"
);
CREATE STORAGE POLICY cold_storage_policy
PROPERTIES(
"storage_resource" = "tencent_cos",
"cooldown_ttl" = "1d"
);
CREATE TABLE my_table_cos (
id INT,
date DATE,
value DOUBLE
)
DUPLICATE KEY(id, date)
PARTITION BY RANGE (date) (
PARTITION p202301 VALUES LESS THAN ('2023-02-01'),
PARTITION p202302 VALUES LESS THAN ('2023-03-01'),
PARTITION p202303 VALUES LESS THAN ('2023-04-01'),
PARTITION p202304 VALUES LESS THAN ('2023-05-01'),
PARTITION p202305 VALUES LESS THAN ('2023-06-01'),
PARTITION p202306 VALUES LESS THAN ('2023-07-01'),
PARTITION p202307 VALUES LESS THAN ('2023-08-01'),
PARTITION p202308 VALUES LESS THAN ('2023-09-01'),
PARTITION p202309 VALUES LESS THAN ('2023-10-01'),
PARTITION p202310 VALUES LESS THAN ('2023-11-01'),
PARTITION p202311 VALUES LESS THAN ('2023-12-01'),
PARTITION p202312 VALUES LESS THAN ('2024-01-01'),
PARTITION p202401 VALUES LESS THAN ('2024-02-01'),
PARTITION p202402 VALUES LESS THAN ('2024-03-01'),
PARTITION p202403 VALUES LESS THAN ('2024-04-01'),
PARTITION p202404 VALUES LESS THAN ('2024-05-01'),
PARTITION p202405 VALUES LESS THAN ('2024-06-01'),
PARTITION p202406 VALUES LESS THAN ('2024-07-01'),
PARTITION p202407 VALUES LESS THAN ('2024-08-01'),
PARTITION p202408 VALUES LESS THAN ('2024-09-01'),
PARTITION p202409 VALUES LESS THAN ('2024-10-01'),
PARTITION p202410 VALUES LESS THAN ('2024-11-01'),
PARTITION p202411 VALUES LESS THAN ('2024-12-01'),
PARTITION p202412 VALUES LESS THAN ('2025-01-01')
)
DISTRIBUTED BY HASH(id) BUCKETS 10
PROPERTIES (
"dynamic_partition.enable" = "true",
"storage_policy" = "cold_storage_policy",
"dynamic_partition.time_unit" = "MONTH",
"dynamic_partition.start" = "-48",
"dynamic_partition.end" = "3",
"dynamic_partition.prefix" = "p",
"dynamic_partition.buckets" = "10",
"replication_num" = "3"
);
我关闭服务后,手动将服务器的时间调整到3天后,即
date -s "+3 days"
启动服务,发现cos上已经有相关目录,之前为空,但依然无法判断,分区数据热的数据,怎么区分在本地卷还是在cos卷上
mysql> SELECT substring(date, 1, 7) AS partition_month, count() AS total_rows, sum(value) as total_value FROM my_table_cos GROUP BY partition_month ORDER BY partition_month;
+-----------------+------------+--------------------+
| partition_month | total_rows | total_value |
+-----------------+------------+--------------------+
| 2023-01 | 38217 | 1935533.0699999991 |
| 2023-02 | 38743 | 1955229.1399999985 |
| 2023-03 | 21467 | 1078427.160000001 |
| 2023-04 | 10542 | 529884.4199999995 |
| 2023-05 | 2522 | 128274.25999999997 |
| 2023-06 | 2513 | 123743.10000000003 |
| 2023-07 | 2524 | 126989.36000000002 |
| 2023-08 | 2448 | 122921.20999999998 |
| 2023-09 | 2505 | 128078.24999999997 |
| 2023-10 | 2535 | 127403.13000000003 |
| 2023-11 | 2502 | 126660.61000000004 |
| 2023-12 | 2482 | 121976.15000000005 |
| 2024-01 | 4604 | 233188.35 |
| 2024-02 | 2308 | 118334.12 |
| 2024-03 | 2298 | 116566.07000000002 |
| 2024-04 | 2300 | 114688.14 |
| 2024-05 | 2324 | 117612.52000000006 |
| 2024-06 | 2294 | 113606.74000000009 |
| 2024-07 | 2280 | 116766.52999999998 |
| 2024-08 | 2404 | 123040.32999999994 |
| 2024-09 | 2296 | 114863.36000000007 |
| 2024-10 | 2281 | 115320.52999999996 |
| 2024-11 | 2330 | 117626.92999999996 |
| 2024-12 | 2281 | 113436.41000000003 |
| 2025-01 | 5030 | 251946.04000000004 |
| 2025-02 | 2494 | 125585.30999999998 |
| 2025-03 | 2476 | 126977.76000000007 |
+-----------------+------------+--------------------+
请问有什么办法可以判断,数据存储在哪里吗?
通过
SHOW PROC '/dbs/zhazhaxiong/my_table_cos/partitions';
看不出什么端倪
Beta Was this translation helpful? Give feedback.
All reactions