Как разрешить скрытый конфликт зависимостей Python между «apache-beam» и «google-cloud-bigquery»? ⇐ Python

[Anonymous]

Я не могу использовать пакеты «apache-beam» и «google-cloud-bigquery» в одном проекте Python, но предупреждений о конфликтах зависимостей также нет. Мне неясно, как это отладить, чтобы оба пакета могли сосуществовать в моем проекте.
Я подозреваю, что виноваты любые зависимости C, которые есть у pyarrow, но мне это не ясно. как это исправить, кроме как (а) хранить код конвейера в отдельном репозитории для кода заданий или (б) играть в «ударь крота» с версиями пакета и Python, пока они не синхронизируются.
Шаги воспроизведения:

• В новой среде запустите: pip install google-cloud-bigquery
  < li>Запустите python -X errorhandler -c "import google.cloud.bigquery", он ничего не должен вывести.
• Затем установите apache-beam: pip install apache-beam< /code>
• Запустите python -X errorhandler -c "import google.cloud.bigquery" второй раз.

Вы должны увидеть что-то похожее на следующую ошибку SegmentationFault:
Fatal Python error: Segmentation fault

Current thread 0x0000000102466e00 (most recent call first):
File "", line 241 in _call_with_frames_removed
File "", line 1233 in create_module
File "", line 573 in module_from_spec
File "", line 676 in _load_unlocked
File "", line 1149 in _find_and_load_unlocked
File "", line 1178 in _find_and_load
File "/Users/jamesroseman/workspace/new-project/.venv/lib/python3.11/site-packages/pyarrow/__init__.py", line 65 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1149 in _find_and_load_unlocked
File "", line 1178 in _find_and_load
File "/Users/jamesroseman/workspace/new-project/.venv/lib/python3.11/site-packages/google/cloud/bigquery_storage_v1/reader.py", line 34 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1149 in _find_and_load_unlocked
File "", line 1178 in _find_and_load
File "", line 241 in _call_with_frames_removed
File "", line 1234 in _handle_fromlist
File "/Users/jamesroseman/workspace/new-project/.venv/lib/python3.11/site-packages/google/cloud/bigquery_storage_v1/client.py", line 26 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1149 in _find_and_load_unlocked
File "", line 1178 in _find_and_load
File "", line 241 in _call_with_frames_removed
File "", line 1234 in _handle_fromlist
File "/Users/jamesroseman/workspace/new-project/.venv/lib/python3.11/site-packages/google/cloud/bigquery_storage_v1/__init__.py", line 23 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1149 in _find_and_load_unlocked
File "", line 1178 in _find_and_load
File "", line 241 in _call_with_frames_removed
File "", line 1128 in _find_and_load_unlocked
File "", line 1178 in _find_and_load
File "", line 241 in _call_with_frames_removed
File "", line 1128 in _find_and_load_unlocked
File "", line 1178 in _find_and_load
File "", line 241 in _call_with_frames_removed
File "", line 1128 in _find_and_load_unlocked
File "", line 1178 in _find_and_load
File "/Users/jamesroseman/workspace/new-project/.venv/lib/python3.11/site-packages/google/cloud/bigquery/client.py", line 61 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1149 in _find_and_load_unlocked
File "", line 1178 in _find_and_load
File "/Users/jamesroseman/workspace/new-project/.venv/lib/python3.11/site-packages/google/cloud/bigquery/__init__.py", line 36 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1149 in _find_and_load_unlocked
File "", line 1178 in _find_and_load
File "", line 1 in

Extension modules: zstandard.backend_c, charset_normalizer.md, requests.packages.charset_normalizer.md, requests.packages.chardet.md, google._upb._message, grpc._cython.cygrpc, fastavro._logical_readers, fastavro._schema, fastavro._read, fastavro._logical_writers, fastavro._validation, fastavro._write (total: 12)
Segmentation fault: 11


Подробнее здесь: https://stackoverflow.com/questions/791 ... m-and-goog