Я пытаюсь разместить модель MMDetection на сервере Databricks Serving (в Azure). Модель обучена на 15.4 LTS ML. Однако во время обновления конечной точки обслуживания он жалуется на GLIBC_2.32:
An error occurred while loading the model: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmcv/_ext.cpython-311-x86_64-linux-gnu.so)
Обычно я просто запускаю apt-get install glibc или что-то в этом роде в сценарии запуска. Однако в документации Databricks указано, что сценарии инициализации не поддерживаются.
Вопрос
Итак, как обойти эту ошибку и развернуть MMDet модель на Databrciks?
Ранее я развернул множество моделей PyTorch Lightning (то есть не использовал OpenMMLab), и glibc не был проблемой для обслуживания Databricks.
Эскизы решений
Я вижу два возможных пути к потенциальному решению.
Может быть, есть волшебный трюк разрешить установку отсутствующих библиотек на обслуживающей конечной точке?
В качестве альтернативы, возможно, мы сможем избавиться от основной причины проблемы, а именно mmcv, и эта ошибка немного похожа на opencv проблема для меня? Модель, которую я пытаюсь использовать, — это CoDETR с конфигурацией, основанной на этой.
Оба варианта кажутся мне одинаково маловероятными. Может быть, есть другой способ?
Выдержка из использованных пакетов ключей:
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 130, in _load_model_closure
[mlrpj] [2024-12-30 09:50:06 +0000] model = load_model_fn(path)
[mlrpj] [2024-12-30 09:50:06 +0000] ^^^^^^^^^^^^^^^^^^^
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracing/provider.py", line 309, in wrapper
[mlrpj] [2024-12-30 09:50:06 +0000] is_func_called, result = True, f(*args, **kwargs)
[mlrpj] [2024-12-30 09:50:06 +0000] ^^^^^^^^^^^^^^^^^^
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/__init__.py", line 1067, in load_model
[mlrpj] [2024-12-30 09:50:06 +0000] model_impl = importlib.import_module(conf[MAIN])._load_pyfunc(data_path)
[mlrpj] [2024-12-30 09:50:06 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 561, in _load_pyfunc
[mlrpj] [2024-12-30 09:50:06 +0000] context, python_model, signature = _load_context_model_and_signature(model_path, model_config)
[mlrpj] [2024-12-30 09:50:06 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 555, in _load_context_model_and_signature
[mlrpj] [2024-12-30 09:50:06 +0000] python_model.load_context(context=context)
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/computer_vision/engine/mmdet/utils.py", line 44, in load_context
[mlrpj] [2024-12-30 09:50:06 +0000] from mmdet.apis import DetInferencer
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/apis/__init__.py", line 2, in
[mlrpj] [2024-12-30 09:50:06 +0000] from .det_inferencer import DetInferencer
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/apis/det_inferencer.py", line 22, in
[mlrpj] [2024-12-30 09:50:06 +0000] from mmdet.evaluation import INSTANCE_OFFSET
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/evaluation/__init__.py", line 4, in
[mlrpj] [2024-12-30 09:50:06 +0000] from .metrics import * # noqa: F401,F403
[mlrpj] [2024-12-30 09:50:06 +0000] ^^^^^^^^^^^^^^^^^^^^^^
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/evaluation/metrics/__init__.py", line 5, in
[mlrpj] [2024-12-30 09:50:06 +0000] from .coco_metric import CocoMetric
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/evaluation/metrics/coco_metric.py", line 16, in
[mlrpj] [2024-12-30 09:50:06 +0000] from mmdet.datasets.api_wrappers import COCO, COCOeval, COCOevalMP
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/datasets/__init__.py", line 31, in
[mlrpj] [2024-12-30 09:50:06 +0000] from .utils import get_loading_pipeline
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/datasets/utils.py", line 5, in
[mlrpj] [2024-12-30 09:50:06 +0000] from mmdet.datasets.transforms import LoadAnnotations, LoadPanopticAnnotations
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/datasets/transforms/__init__.py", line 6, in
[mlrpj] [2024-12-30 09:50:06 +0000] from .formatting import (ImageToTensor, PackDetInputs, PackReIDInputs,
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/datasets/transforms/formatting.py", line 11, in
[mlrpj] [2024-12-30 09:50:06 +0000] from mmdet.structures.bbox import BaseBoxes
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/structures/bbox/__init__.py", line 2, in
[mlrpj] [2024-12-30 09:50:06 +0000] from .base_boxes import BaseBoxes
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/structures/bbox/base_boxes.py", line 9, in
[mlrpj] [2024-12-30 09:50:06 +0000] from mmdet.structures.mask.structures import BitmapMasks, PolygonMasks
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/structures/mask/__init__.py", line 3, in
[mlrpj] [2024-12-30 09:50:06 +0000] from .structures import (BaseInstanceMasks, BitmapMasks, PolygonMasks,
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/structures/mask/structures.py", line 12, in
[mlrpj] [2024-12-30 09:50:06 +0000] from mmcv.ops.roi_align import roi_align
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmcv/ops/__init__.py", line 3, in
[mlrpj] [2024-12-30 09:50:06 +0000] from .active_rotated_filter import active_rotated_filter
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmcv/ops/active_rotated_filter.py", line 10, in
[mlrpj] [2024-12-30 09:50:06 +0000] ext_module = ext_loader.load_ext(
[mlrpj] [2024-12-30 09:50:06 +0000] ^^^^^^^^^^^^^^^^^^^^
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmcv/utils/ext_loader.py", line 13, in load_ext
[mlrpj] [2024-12-30 09:50:06 +0000] ext = importlib.import_module('mmcv.' + name)
[mlrpj] [2024-12-30 09:50:06 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/importlib/__init__.py", line 126, in import_module
[mlrpj] [2024-12-30 09:50:06 +0000] return _bootstrap._gcd_import(name[level:], package, level)
[mlrpj] [2024-12-30 09:50:06 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[mlrpj] [2024-12-30 09:50:06 +0000] ImportError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmcv/_ext.cpython-311-x86_64-linux-gnu.so)
[mlrpj] [2024-12-30 09:50:06 +0000] [595] [INFO] Worker exiting (pid: 595)
[mlrpj] [2024-12-30 09:50:07 +0000] An error occurred while loading the model: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmcv/_ext.cpython-311-x86_64-linux-gnu.so)
[mlrpj] [2024-12-30 09:50:07 +0000] Traceback (most recent call last):
[mlrpj] [2024-12-30 09:50:07 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 212, in get_model_option_or_exit
[mlrpj] [2024-12-30 09:50:07 +0000] self.model = self.model_future.result()
[mlrpj] [2024-12-30 09:50:07 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[mlrpj] [2024-12-30 09:50:07 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 449, in result
[mlrpj] [2024-12-30 09:50:07 +0000] return self.__get_result()
[mlrpj] [2024-12-30 09:50:07 +0000] ^^^^^^^^^^^^^^^^^^^
[mlrpj] [2024-12-30 09:50:07 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
[mlrpj] [2024-12-30 09:50:07 +0000] raise self._exception
[mlrpj] [2024-12-30 09:50:07 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/thread.py", line 58, in run
[mlrpj] [2024-12-30 09:50:07 +0000] result = self.fn(*self.args, **self.kwargs)
[mlrpj] [2024-12-30 09:50:07 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[mlrpj] [2024-12-30 09:50:07 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 130, in _load_model_closure
[mlrpj] [2024-12-30 09:50:07 +0000] model = load_model_fn(path)
[mlrpj] [2024-12-30 09:50:07 +0000] ^^^^^^^^^^^^^^^^^^^
[mlrpj] [2024-12-30 09:50:07 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracing/provider.py", line 309, in wrapper
[mlrpj] [2024-12-30 09:50:07 +0000] is_func_called, result = True, f(*args, **kwargs)
[mlrpj] [2024-12-30 09:50:07 +0000] ^^^^^^^^^^^^^^^^^^
[mlrpj] [2024-12-30 09:50:07 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/__init__.py", line 1067, in load_model
[mlrpj] [2024-12-30 09:50:07 +0000] model_impl = importlib.import_module(conf[MAIN])._load_pyfunc(data_path)
[mlrpj] [2024-12-30 09:50:07 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[mlrpj] [2024-12-30 09:50:07 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 561, in _load_pyfunc
[mlrpj] [2024-12-30 09:50:07 +0000] context, python_model, signature = _load_context_model_and_signature(model_path, model_config)
[mlrpj] [2024-12-30 09:50:07 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[mlrpj] [2024-12-30 09:50:07 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 555, in _load_context_model_and_signature
[mlrpj] [2024-12-30 09:50:07 +0000] python_model.load_context(context=context)
[mlrpj] [2024-12-30 09:50:07 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/computer_vision/engine/mmdet/utils.p
Обновление от 7 января
Вот обновленная информация о моих исследованиях.
< li>Возможным и трудным способом решения было бы преобразование модели в ONNX с помощью mmdeploy, но CoDETR на данный момент им еще не поддерживается.
Пока Databricks в Azure не разрешает использовать собственные изображения при показе, это интересно, их можно настроить на AWS.
Я пытаюсь разместить модель MMDetection на сервере Databricks Serving (в Azure). Модель обучена на 15.4 LTS ML. Однако во время обновления конечной точки обслуживания он жалуется на GLIBC_2.32: [code]An error occurred while loading the model: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmcv/_ext.cpython-311-x86_64-linux-gnu.so) [/code] Обычно я просто запускаю apt-get install glibc или что-то в этом роде в сценарии запуска. Однако в документации Databricks указано, что сценарии инициализации не поддерживаются. Вопрос Итак, как обойти эту ошибку и развернуть MMDet модель на Databrciks? Ранее я развернул множество моделей PyTorch Lightning (то есть не использовал OpenMMLab), и glibc не был проблемой для обслуживания Databricks. Эскизы решений Я вижу два возможных пути к потенциальному решению. [list] [*]Может быть, есть волшебный трюк разрешить установку отсутствующих библиотек на обслуживающей конечной точке? [*]В качестве альтернативы, возможно, мы сможем избавиться от основной причины проблемы, а именно mmcv, и эта ошибка немного похожа на opencv проблема для меня? Модель, которую я пытаюсь использовать, — это CoDETR с конфигурацией, основанной на этой. [/list] Оба варианта кажутся мне одинаково маловероятными. Может быть, есть другой способ? Выдержка из использованных пакетов ключей: [code]pip install --upgrade pip pip install uv uv pip install torch==2.1.0 torchvision==0.16.0 numpy==1.26.4 openmim "mmengine==0.10.5" uv pip install "mmcv==2.1.0" -f https://download.openmmlab.com/mmcv/dist/cu121/torch2.1.0/index.html uv pip install albumentations==1.4.18 pycocotools==2.0.7 mlflow python-snappy==0.7.3 uv pip install lightning==2.2.2 mmdet==3.3.0 [/code] Полная трассировка стека: [code][mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 130, in _load_model_closure [mlrpj] [2024-12-30 09:50:06 +0000] model = load_model_fn(path) [mlrpj] [2024-12-30 09:50:06 +0000] ^^^^^^^^^^^^^^^^^^^ [mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracing/provider.py", line 309, in wrapper [mlrpj] [2024-12-30 09:50:06 +0000] is_func_called, result = True, f(*args, **kwargs) [mlrpj] [2024-12-30 09:50:06 +0000] ^^^^^^^^^^^^^^^^^^ [mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/__init__.py", line 1067, in load_model [mlrpj] [2024-12-30 09:50:06 +0000] model_impl = importlib.import_module(conf[MAIN])._load_pyfunc(data_path) [mlrpj] [2024-12-30 09:50:06 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 561, in _load_pyfunc [mlrpj] [2024-12-30 09:50:06 +0000] context, python_model, signature = _load_context_model_and_signature(model_path, model_config) [mlrpj] [2024-12-30 09:50:06 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 555, in _load_context_model_and_signature [mlrpj] [2024-12-30 09:50:06 +0000] python_model.load_context(context=context) [mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/computer_vision/engine/mmdet/utils.py", line 44, in load_context [mlrpj] [2024-12-30 09:50:06 +0000] from mmdet.apis import DetInferencer [mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/apis/__init__.py", line 2, in [mlrpj] [2024-12-30 09:50:06 +0000] from .det_inferencer import DetInferencer [mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/apis/det_inferencer.py", line 22, in [mlrpj] [2024-12-30 09:50:06 +0000] from mmdet.evaluation import INSTANCE_OFFSET [mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/evaluation/__init__.py", line 4, in [mlrpj] [2024-12-30 09:50:06 +0000] from .metrics import * # noqa: F401,F403 [mlrpj] [2024-12-30 09:50:06 +0000] ^^^^^^^^^^^^^^^^^^^^^^ [mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/evaluation/metrics/__init__.py", line 5, in [mlrpj] [2024-12-30 09:50:06 +0000] from .coco_metric import CocoMetric [mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/evaluation/metrics/coco_metric.py", line 16, in [mlrpj] [2024-12-30 09:50:06 +0000] from mmdet.datasets.api_wrappers import COCO, COCOeval, COCOevalMP [mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/datasets/__init__.py", line 31, in [mlrpj] [2024-12-30 09:50:06 +0000] from .utils import get_loading_pipeline [mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/datasets/utils.py", line 5, in [mlrpj] [2024-12-30 09:50:06 +0000] from mmdet.datasets.transforms import LoadAnnotations, LoadPanopticAnnotations [mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/datasets/transforms/__init__.py", line 6, in [mlrpj] [2024-12-30 09:50:06 +0000] from .formatting import (ImageToTensor, PackDetInputs, PackReIDInputs, [mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/datasets/transforms/formatting.py", line 11, in [mlrpj] [2024-12-30 09:50:06 +0000] from mmdet.structures.bbox import BaseBoxes [mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/structures/bbox/__init__.py", line 2, in [mlrpj] [2024-12-30 09:50:06 +0000] from .base_boxes import BaseBoxes [mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/structures/bbox/base_boxes.py", line 9, in [mlrpj] [2024-12-30 09:50:06 +0000] from mmdet.structures.mask.structures import BitmapMasks, PolygonMasks [mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/structures/mask/__init__.py", line 3, in [mlrpj] [2024-12-30 09:50:06 +0000] from .structures import (BaseInstanceMasks, BitmapMasks, PolygonMasks, [mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmdet/structures/mask/structures.py", line 12, in [mlrpj] [2024-12-30 09:50:06 +0000] from mmcv.ops.roi_align import roi_align [mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmcv/ops/__init__.py", line 3, in [mlrpj] [2024-12-30 09:50:06 +0000] from .active_rotated_filter import active_rotated_filter [mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmcv/ops/active_rotated_filter.py", line 10, in [mlrpj] [2024-12-30 09:50:06 +0000] ext_module = ext_loader.load_ext( [mlrpj] [2024-12-30 09:50:06 +0000] ^^^^^^^^^^^^^^^^^^^^ [mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmcv/utils/ext_loader.py", line 13, in load_ext [mlrpj] [2024-12-30 09:50:06 +0000] ext = importlib.import_module('mmcv.' + name) [mlrpj] [2024-12-30 09:50:06 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [mlrpj] [2024-12-30 09:50:06 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/importlib/__init__.py", line 126, in import_module [mlrpj] [2024-12-30 09:50:06 +0000] return _bootstrap._gcd_import(name[level:], package, level) [mlrpj] [2024-12-30 09:50:06 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [mlrpj] [2024-12-30 09:50:06 +0000] ImportError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmcv/_ext.cpython-311-x86_64-linux-gnu.so) [mlrpj] [2024-12-30 09:50:06 +0000] [595] [INFO] Worker exiting (pid: 595) [mlrpj] [2024-12-30 09:50:07 +0000] An error occurred while loading the model: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mmcv/_ext.cpython-311-x86_64-linux-gnu.so) [mlrpj] [2024-12-30 09:50:07 +0000] Traceback (most recent call last): [mlrpj] [2024-12-30 09:50:07 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 212, in get_model_option_or_exit [mlrpj] [2024-12-30 09:50:07 +0000] self.model = self.model_future.result() [mlrpj] [2024-12-30 09:50:07 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^ [mlrpj] [2024-12-30 09:50:07 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 449, in result [mlrpj] [2024-12-30 09:50:07 +0000] return self.__get_result() [mlrpj] [2024-12-30 09:50:07 +0000] ^^^^^^^^^^^^^^^^^^^ [mlrpj] [2024-12-30 09:50:07 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result [mlrpj] [2024-12-30 09:50:07 +0000] raise self._exception [mlrpj] [2024-12-30 09:50:07 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/concurrent/futures/thread.py", line 58, in run [mlrpj] [2024-12-30 09:50:07 +0000] result = self.fn(*self.args, **self.kwargs) [mlrpj] [2024-12-30 09:50:07 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [mlrpj] [2024-12-30 09:50:07 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflowserving/scoring_server/__init__.py", line 130, in _load_model_closure [mlrpj] [2024-12-30 09:50:07 +0000] model = load_model_fn(path) [mlrpj] [2024-12-30 09:50:07 +0000] ^^^^^^^^^^^^^^^^^^^ [mlrpj] [2024-12-30 09:50:07 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/tracing/provider.py", line 309, in wrapper [mlrpj] [2024-12-30 09:50:07 +0000] is_func_called, result = True, f(*args, **kwargs) [mlrpj] [2024-12-30 09:50:07 +0000] ^^^^^^^^^^^^^^^^^^ [mlrpj] [2024-12-30 09:50:07 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/__init__.py", line 1067, in load_model [mlrpj] [2024-12-30 09:50:07 +0000] model_impl = importlib.import_module(conf[MAIN])._load_pyfunc(data_path) [mlrpj] [2024-12-30 09:50:07 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [mlrpj] [2024-12-30 09:50:07 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 561, in _load_pyfunc [mlrpj] [2024-12-30 09:50:07 +0000] context, python_model, signature = _load_context_model_and_signature(model_path, model_config) [mlrpj] [2024-12-30 09:50:07 +0000] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [mlrpj] [2024-12-30 09:50:07 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 555, in _load_context_model_and_signature [mlrpj] [2024-12-30 09:50:07 +0000] python_model.load_context(context=context) [mlrpj] [2024-12-30 09:50:07 +0000] File "/opt/conda/envs/mlflow-env/lib/python3.11/site-packages/computer_vision/engine/mmdet/utils.p [/code] Обновление от 7 января Вот обновленная информация о моих исследованиях. [list] < li>Возможным и трудным способом решения было бы преобразование модели в ONNX с помощью mmdeploy, но CoDETR на данный момент им еще не поддерживается. [*]Пока Databricks в Azure не разрешает использовать собственные изображения при показе, это интересно, их можно настроить на AWS. [/list]
Мобильная версия