Искровой набор данных.фильтр китайское имя столбца, org.apache.spark.sql.catalyst.parser.ParseException ⇐ JAVA

[Anonymous]

У меня есть таблица с китайским именем столбца 人员, используйте org.apache.spark.sql.Dataset для обработки данных.
Когда я фильтрую с английским именем столбца, запуск выполняется успешно.

Код: Выделить всё

dataset.filter(" ( (name = 'name1') ) ")

, queryExecution — это:

Код: Выделить всё

== Parsed Logical Plan ==
'Filter ('name = name1)
+- Project [人员#541, name#542, 1 AS col1#547]
+- Project [人员#541, name#542]
+- Project [cast(人员#537 as string) AS 人员#541, cast(name#538 as string) AS name#542]
+- Relation [人员#537,name#538] JDBCRelation(`test1111`) [numPartitions=1]

== Analyzed Logical Plan ==
string, name: string, col1: int
Filter (name#542 = name1)
+- Project [人员#541, name#542, 1 AS col1#547]
+- Project [人员#541, name#542]
+- Project [cast(人员#537 as string) AS 人员#541, cast(name#538 as string) AS name#542]
+- Relation [人员#537,name#538] JDBCRelation(`test1111`) [numPartitions=1]

== Optimized Logical Plan ==
Project [人员#537, name#538, 1 AS col1#547]
+- Filter (isnotnull(name#538) AND (name#538 = name1))
+- Relation [人员#537,name#538] JDBCRelation(`test1111`) [numPartitions=1]

== Physical Plan ==
*(1) Project [人员#537, name#538, 1 AS col1#547]
+- *(1) Scan JDBCRelation(`test1111`) [numPartitions=1] [人员#537,name#538] PushedFilters: [*IsNotNull(name), *EqualTo(name,name1)], ReadSchema: struct

Но если фильтр с китайским именем столбца запускается неудачно.

Код: Выделить всё

dataset.filter(" ( (`人员` = '111') ) ")

, queryExecution — это:

Код: Выделить всё

== Parsed Logical Plan ==
'Filter ('人员 = 111)
+- Project [人员#576, name#577, 1 AS col1#582]
+- Project [人员#576, name#577]
+- Project [cast(人员#572 as string) AS 人员#576, cast(name#573 as string) AS name#577]
+- Relation [人员#572,name#573] JDBCRelation(`test1111`) [numPartitions=1]

== Analyzed Logical Plan ==
人员: string, name: string, col1: int
Filter (人员#576 = 111)
+- Project [人员#576, name#577, 1 AS col1#582]
+- Project [人员#576, name#577]
+- Project [cast(人员#572 as string) AS 人员#576, cast(name#573 as string) AS name#577]
+- Relation [人员#572,name#573] JDBCRelation(`test1111`) [numPartitions=1]

== Optimized Logical Plan ==
Project [人员#572, name#573, 1 AS col1#582]
+- Filter (isnotnull(人员#572) AND (人员#572 = 111))
+- Relation [人员#572,name#573] JDBCRelation(`test1111`) [numPartitions=1]

== Physical Plan ==
org.apache.spark.sql.catalyst.parser.ParseException:
Syntax error at or near '人'(line 1, pos 0)

== SQL ==
人员
^^^

Версия Spark — 3.3.1
У кого-нибудь есть идеи?
Я думаю, это не Неизвестный язык Универсальная китайская проблема.
Понятия не имею.

Подробнее здесь: https://stackoverflow.com/questions/787 ... rser-parse