Я использую Hudi с Flink. У меня есть таблица MySQL с именем stu4.
Без каталога все работает. но с каталогом dfs я не могу прочитать данные из исходной таблицы.
таблица в hudi:
create table stu4(
id bigint not null,
name string,
school string,
PRIMARY KEY (id) NOT ENFORCED
) with (
'connector' = 'jdbc',
'url' = 'jdbc:mysql://hadoop2:3306/test',
'username' = 'root',
'password' = '*******',
'table-name' = 'stu4'
);
без каталога:
select * from stu4;
2024-10-22 17:26:12,202 WARN org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The configuration directory ('/bigdata/flink/conf') already contains a LOG4J config file.If you want to use logback, then please delete or rename the log configuration file.
2024-10-22 17:26:12,223 INFO org.apache.hadoop.yarn.client.DefaultNoHARMFailoverProxyProvider [] - Connecting to ResourceManager at hadoop1/10.211.55.20:8032
2024-10-22 17:26:12,223 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
2024-10-22 17:26:12,231 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Found Web Interface hadoop1:43613 of application 'application_1729525785030_0003'.
+----+----------------------+--------------------------------+
| op | id | name |
+----+----------------------+--------------------------------+
| +I | 3 | 1 |
| +I | 4 | 2 |
| +I | 5 | 3 |
+----+----------------------+--------------------------------+
Received a total of 3 rows
с каталогом:
select * from stu4;
2024-10-22 17:30:25,787 WARN org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The configuration directory ('/bigdata/flink/conf') already contains a LOG4J config file.If you want to use logback, then please delete or rename the log configuration file.
2024-10-22 17:30:25,803 INFO org.apache.hadoop.yarn.client.DefaultNoHARMFailoverProxyProvider [] - Connecting to ResourceManager at hadoop1/10.211.55.20:8032
2024-10-22 17:30:25,804 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
2024-10-22 17:30:25,807 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Found Web Interface hadoop1:43613 of application 'application_1729525785030_0003'.
+----+----------------------+--------------------------------+
| op | id | name |
+----+----------------------+--------------------------------+
Received a total of 0 row
конфигурация каталога:
CREATE CATALOG hoodie_catalog
WITH (
'type'='hudi',
'catalog.path' = '/tmp/hudi_catalog',
'mode'='dfs'
);
USE CATALOG hoodie_catalog;
Подробнее здесь: https://stackoverflow.com/questions/791 ... -not-works
