Синтаксическая ошибка при или рядом с ним: '(строка 1, POS 2) - parse_syntax_error - == sql == ⇐ Python

[Anonymous]

Я пытаюсь читать два файла json за раз из ведра AWS S3. Получение ошибок parse_syntax_error
Но когда я читаю отдельный файл, он работает нормально. Br />file_paths = [
"s3://test_bt/data/batch_date=20241217/20241217/data_0_0_0.json.gz",
"s3://test_bt/data/batch_date=20241217/20241217/data_0_1_0.json.gz"
]

df_single = spark.read.json(*file_paths)

df_single.show()
< /code>
ошибка: < /p>
Spark Error Class: PARSE_SYNTAX_ERROR; Traceback (most recent call last):
File "/tmp/TEST.py", line 10, in
df_single = spark.read.json(*s3_paths)
File "/opt/amazon/spark/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 254, in json
self._set_opts(
File "/opt/amazon/spark/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 50, in _set_opts
self.schema(schema) # type: ignore[attr-defined]
File "/opt/amazon/spark/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 118, in schema
self._jreader = self._jreader.schema(schema)
File "/opt/amazon/spark/python/lib/py4j-0.10.9.5-src.zip/py4j/java_gateway.py", line 1321, in __call__
return_value = get_return_value(
File "/opt/amazon/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 196, in deco
raise converted from None
pyspark.sql.utils.ParseException:
Syntax error at or near ':'(line 1, pos 2)

== SQL ==
s3://test_bt/data/batch_date=20241217/20241217/data_0_1_0.json.gz
--^^^


Подробнее здесь: https://stackoverflow.com/questions/794 ... -error-sql