Создайте колесо PIP для OpenCV, созданного из источника, чтобы предотвратить перезапись библиотечной зависимости ⇐ Python

[Anonymous]

У меня есть Dockerfile, который строит OpenCV из Source с CUDA. Сама сборка преуспевает, но PIP не распознает эту пользовательскую установку. В результате, когда я позже устанавливаю пакет Python, который зависит от OpenCV, PIP получает предварительно построенное колесо OpenCV-Python, которое затем переопределяет (или скрывает) мою сборку источника. Согласно этому предложению (https://stackoverflow.com/a/62642547/13045595 по @jkr), правильное исправление состоит в том, чтобы упаковать мою пользовательскую сборку в виде колеса и установить ее, чтобы PIP рассматривает зависимость как удовлетворенную. Я попробовал это, и хотя сценарий сборки колес заканчивается без ошибок, установленное колесо на самом деле не работает. Не могли бы вы просмотреть сценарий строительства колес и/или DockerFile и сообщить мне, что нужно настроить или оптимизировать?
# syntax=docker/dockerfile:1
# Requires Docker BuildKit for cache mounts (build with DOCKER_BUILDKIT=1)
FROM nvidia/cuda:12.1.0-cudnn8-devel-ubuntu22.04

# Environment setup
ENV DEBIAN_FRONTEND=noninteractive \
LANG=en_US.UTF-8 \
LC_ALL=en_US.UTF-8 \
PYTHONIOENCODING=UTF-8 \
NVIDIA_DRIVER_CAPABILITIES=all \
CUDA_HOME=/usr/local/cuda \
PATH=/usr/local/cuda/bin:$PATH \
LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH \
CCACHE_DIR=/ccache

# Create ccache directory
RUN mkdir -p /ccache

# Set locale
RUN apt-get update && apt-get install -y --no-install-recommends \
locales \
&& locale-gen en_US.UTF-8 \
&& update-locale LANG=en_US.UTF-8

# Essential system dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
# Build tools
build-essential \
cmake \
pkg-config \
ninja-build \
ccache \
# GCC-10 for CUDA 12.x compatibility
gcc-10 \
g++-10 \
# Version control
git \
# Utilities
wget \
curl \
unzip \
vim \
nano \
htop \
software-properties-common \
&& add-apt-repository ppa:deadsnakes/ppa -y \
&& apt-get update \
&& apt-get install -y --no-install-recommends \
python3.9 \
python3.9-dev \
python3.9-venv \
python3.9-distutils \
python3-pip \
# OpenCV dependencies
libgtk-3-dev \
libgtk2.0-dev \
libavcodec-dev \
libavformat-dev \
libswscale-dev \
libswresample-dev \
libtbb-dev \
libjpeg-dev \
libpng-dev \
libtiff-dev \
libwebp-dev \
libopenexr-dev \
libdc1394-dev \
libgstreamer1.0-dev \
libgstreamer-plugins-base1.0-dev \
libv4l-dev \
libxvidcore-dev \
libx264-dev \
libfdk-aac-dev \
libmp3lame-dev \
libtheora-dev \
libvorbis-dev \
libxine2-dev \
libopencore-amrnb-dev \
libopencore-amrwb-dev \
# Math libraries
libopenblas-dev \
liblapack-dev \
libatlas-base-dev \
libeigen3-dev \
libhdf5-dev \
# OpenGL support
libgl1-mesa-glx \
libglu1-mesa-dev \
libglew-dev \
# Qt5 for OpenCV GUI
qtbase5-dev \
qtchooser \
qt5-qmake \
qtbase5-dev-tools \
# Additional utilities
libprotobuf-dev \
protobuf-compiler \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*

# Set Python 3.9 as default
RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.9 1 \
&& update-alternatives --install /usr/bin/python python /usr/bin/python3.9 1 \
&& python3.9 -m pip install --upgrade pip setuptools wheel

# Install numpy first (required for OpenCV Python bindings)
# Also install wheel building tools
RUN python3 -m pip install --no-cache-dir \
numpy==1.26.4 \
wheel \
setuptools \
build \
auditwheel \
patchelf

# Remove any pre-existing OpenCV packages that might conflict
RUN pip3 uninstall -y opencv-python opencv-python-headless opencv-contrib-python opencv-contrib-python-headless || true

# Set GCC-10 as default for OpenCV build
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 100 \
&& update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-10 100

# Configure ccache
RUN ccache --set-config=cache_dir=/ccache \
&& ccache --set-config=max_size=15G \
&& ccache --set-config=compression=true \
&& ccache --set-config=compression_level=6

# OpenCV 4.8.0 with CUDA support and NONFREE modules
ARG OPENCV_VERSION=4.8.0
ARG CUDA_ARCH_BIN="7.5;8.0;8.6;8.9"

# opencv and opencv-contrib :
# including NONFREE code -could be used or not-
# Use BuildKit cache mount for ccache to speed up rebuilds
RUN --mount=type=cache,target=/ccache \
cd /opt/ &&\
wget https://github.com/opencv/opencv/archiv ... RSION}.zip -O opencv.zip &&\
unzip -qq opencv.zip &&\
rm opencv.zip &&\
wget https://github.com/opencv/opencv_contri ... RSION}.zip -O opencv-co.zip &&\
unzip -qq opencv-co.zip &&\
rm opencv-co.zip &&\
mkdir /opt/opencv-${OPENCV_VERSION}/build && cd /opt/opencv-${OPENCV_VERSION}/build &&\
# Configure OpenCV with proper CUDA linking:\
# - Use stubs for linking but set RPATH to real CUDA libs for runtime\
# - This prevents runtime failures from stub library references\
cmake \
-D BUILD_opencv_java=OFF \
-D WITH_CUDA=ON \
-D BUILD_opencv_dnn=ON \
-D CUDA_ARCH_BIN="${CUDA_ARCH_BIN}" \
-D WITH_CUBLAS=ON \
-D WITH_CUDNN=ON \
-D OPENCV_DNN_CUDA=ON \
-D ENABLE_FAST_MATH=ON \
-D CUDA_FAST_MATH=ON \
-D WITH_CUFFT=ON \
-D WITH_OPENGL=ON \
-D WITH_QT=ON \
-D WITH_IPP=ON \
-D WITH_TBB=ON \
-D WITH_EIGEN=ON \
-D WITH_OPENEXR=ON \
-D BUILD_TESTS=OFF \
-D BUILD_PERF_TESTS=OFF \
-D BUILD_DOCS=OFF \
-D BUILD_EXAMPLES=OFF \
-D WITH_OPENCL=ON \
-D WITH_OPENMP=ON \
-D WITH_FFMPEG=ON \
-D WITH_V4L=ON \
-D WITH_GSTREAMER=ON \
-D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_C_COMPILER_LAUNCHER=ccache \
-D CMAKE_CXX_COMPILER_LAUNCHER=ccache \
-D CMAKE_CUDA_COMPILER_LAUNCHER=ccache \
-D OPENCV_EXTRA_MODULES_PATH=/opt/opencv_contrib-${OPENCV_VERSION}/modules \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D PYTHON3_EXECUTABLE=$(which python3) \
-D PYTHON3_INCLUDE_DIR=$(python3 -c "import sysconfig; print(sysconfig.get_paths()['include'])") \
-D PYTHON3_LIBRARY=$(python3 -c "import sysconfig; cfg=sysconfig.get_config_vars(); print(cfg['LIBDIR'] + '/' + cfg['LDLIBRARY'])") \
-D PYTHON3_PACKAGES_PATH=$(python3 -c "import sysconfig; print(sysconfig.get_paths()['platlib'])") \
-D PYTHON3_NUMPY_INCLUDE_DIRS=$(python3 -c "import numpy; print(numpy.get_include())") \
-D BUILD_opencv_python3=ON \
-D CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda \
-D CMAKE_LIBRARY_PATH=/usr/local/cuda/lib64/stubs \
-D CMAKE_INSTALL_RPATH=/usr/local/cuda/lib64 \
-D CMAKE_BUILD_RPATH=/usr/local/cuda/lib64 \
-D CMAKE_INSTALL_RPATH_USE_LINK_PATH=ON \
-D OPENCV_ENABLE_NONFREE=ON \
.. &&\
make -j$(nproc) && \
make install && \
ldconfig

# Copy constraints file to prevent PyPI OpenCV packages from overwriting our build
COPY constraints.txt /tmp/constraints.txt

# Build and install OpenCV as a proper Python wheel
COPY build_opencv_wheel.py /tmp/
RUN cd /tmp && \
python3 build_opencv_wheel.py ${OPENCV_VERSION} --output-dir /tmp/opencv_wheel --install && \
# Save the wheel for potential reuse
cp /tmp/opencv_wheel/dist/*.whl /tmp/ 2>/dev/null || true && \
# Clean up build directories
rm -rf /opt/opencv-${OPENCV_VERSION} && \
rm -rf /opt/opencv_contrib-${OPENCV_VERSION} && \
rm -rf /tmp/opencv_wheel && \
rm /tmp/build_opencv_wheel.py

# Quick verification that OpenCV is installed
RUN python3 -c "import cv2; print(f'OpenCV installed: {cv2.__version__}')" && \
pip3 list | grep opencv
< /code>
build_opencv_wheel.py
#!/usr/bin/env python3
"""
Build a proper Python wheel for OpenCV after compilation.
This script creates a wheel package from the compiled OpenCV libraries.
"""

import os
import sys
import shutil
import subprocess
from pathlib import Path
from setuptools import setup, find_packages
import sysconfig

def find_cv2_module():
"""Find the compiled cv2 module."""
# Check both platlib and purelib
for key in ('platlib', 'purelib'):
site_packages = Path(sysconfig.get_paths()[key])
cv2_path = site_packages / 'cv2'
if cv2_path.exists() and cv2_path.is_dir():
return cv2_path

# Common locations where cv2.so might be after make install
possible_paths = [
Path('/usr/local/lib/python3.9/dist-packages/cv2'),
Path('/usr/local/lib/python3.9/site-packages/cv2'),
Path('/usr/lib/python3/dist-packages/cv2'),
Path('/usr/lib/python3.9/dist-packages/cv2'),
Path('/usr/lib/python3.9/site-packages/cv2'),
]

for path in possible_paths:
if path.exists() and path.is_dir():
return path

# Try to find cv2.*.so files
for path in [Path('/usr/local/lib'), Path('/usr/lib'), Path('/usr/local')]:
cv2_files = list(path.glob('**/cv2*.so'))
if cv2_files:
return cv2_files[0].parent

raise FileNotFoundError("Could not find compiled cv2 module")

def create_wheel(opencv_version='4.8.0', output_dir='/tmp/opencv_wheel'):
"""Create a wheel from the compiled OpenCV."""

# Find the cv2 module
cv2_path = find_cv2_module()
print(f"Found cv2 module at: {cv2_path}")

# Create temporary build directory
build_dir = Path(output_dir)
build_dir.mkdir(parents=True, exist_ok=True)

# Copy cv2 module directly to build directory (top-level package)
cv2_dest = build_dir / 'cv2'
if cv2_dest.exists():
shutil.rmtree(cv2_dest)
shutil.copytree(cv2_path, cv2_dest)

# Ensure cv2 has proper version info
version_file = cv2_dest / '__version__.py'
version_file.write_text(f'__version__ = "{opencv_version}+cuda12.1"\n')

# Create setup.py
setup_py = build_dir / 'setup.py'
setup_content = f'''
from setuptools import setup, find_packages

setup(
name='opencv-contrib-python',
version='{opencv_version}+cuda12.1',
description='OpenCV Python bindings with contrib modules and CUDA support (custom build)',
long_description='Custom build of OpenCV {opencv_version} with CUDA support and contrib modules',
author='OpenCV Team',
author_email='',
url='https://opencv.org',
license='Apache 2.0',
packages=['cv2', 'cv2.data', 'cv2.misc', 'cv2.mat_wrapper', 'cv2.utils'],
package_data={{
'cv2': [
'*.so',
'*.pyd',
'config*.py',
'__version__.py',
'data/*.xml',
'data/*.dat',
'misc/**/*.json',
'mat_wrapper/*.json',
'utils/**/*.py'
]
}},
include_package_data=True,
install_requires=['numpy>=1.19.3'],
python_requires='>=3.6',
classifiers=[
'Development Status :: 5 - Production/Stable',
'Intended Audience :: Developers',
'Intended Audience :: Science/Research',
'License :: OSI Approved :: Apache Software License',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.9',
'Programming Language :: C++',
'Topic :: Scientific/Engineering',
'Topic :: Scientific/Engineering :: Image Recognition',
'Topic :: Software Development :: Libraries :: Python Modules',
],
zip_safe=False,
)
'''
setup_py.write_text(setup_content)

# Create MANIFEST.in to include all necessary files
manifest = build_dir / 'MANIFEST.in'
manifest.write_text('''
recursive-include cv2 *.so *.pyd *.py
recursive-include cv2/data *
recursive-include cv2/misc *
recursive-include cv2/mat_wrapper *
recursive-include cv2/utils *
''')

# Build the wheel using modern build module
os.chdir(build_dir)
result = subprocess.run(
[sys.executable, '-m', 'build', '--wheel', '--outdir', 'dist'],
capture_output=True,
text=True
)

if result.returncode != 0:
print(f"Error building wheel: {result.stderr}")
# Fallback to legacy method if build module fails
print("Falling back to legacy setup.py bdist_wheel...")
result = subprocess.run(
[sys.executable, 'setup.py', 'bdist_wheel'],
capture_output=True,
text=True
)
if result.returncode != 0:
print(f"Error with fallback build: {result.stderr}")
return None

# Find the built wheel
dist_dir = build_dir / 'dist'
wheels = list(dist_dir.glob('*.whl'))

if not wheels:
print("No wheel file found after build")
return None

wheel_file = wheels[0]
print(f"Successfully built wheel: {wheel_file}")

# Skip auditwheel for internal use (often fails with CUDA libs)
# Uncomment if you need manylinux compatibility for distribution
# try:
# repaired_dir = build_dir / 'wheelhouse'
# repaired_dir.mkdir(exist_ok=True)
#
# result = subprocess.run(
# ['auditwheel', 'repair', str(wheel_file),
# '--plat', 'manylinux_2_17_x86_64',
# '-w', str(repaired_dir)],
# capture_output=True,
# text=True
# )
#
# if result.returncode == 0:
# repaired_wheels = list(repaired_dir.glob('*.whl'))
# if repaired_wheels:
# print(f"Repaired wheel: {repaired_wheels[0]}")
# return repaired_wheels[0]
# else:
# print(f"Auditwheel repair failed: {result.stderr}")
# print("Using original wheel")
# except Exception as e:
# print(f"Auditwheel not available or failed: {e}")
# print("Using original wheel")

return wheel_file

def main():
import argparse

parser = argparse.ArgumentParser(
description='Build a Python wheel for compiled OpenCV'
)
parser.add_argument(
'version',
nargs='?',
default='4.8.0',
help='OpenCV version without +cuda suffix (default: 4.8.0)'
)
parser.add_argument(
'--output-dir',
default='/tmp/opencv_wheel',
help='Output directory for wheel build (default: /tmp/opencv_wheel)'
)
parser.add_argument(
'--install',
action='store_true',
help='Install the wheel after building'
)

args = parser.parse_args()

# Build the wheel
wheel_file = create_wheel(args.version, args.output_dir)

if wheel_file and args.install:
print(f"Installing wheel: {wheel_file}")
result = subprocess.run(
[sys.executable, '-m', 'pip', 'install',
str(wheel_file), '--force-reinstall'],
capture_output=True,
text=True
)
if result.returncode == 0:
print("Wheel installed successfully")
# Verify installation
subprocess.run([sys.executable, '-c',
'import cv2; print(f"OpenCV {cv2.__version__} installed")'])
else:
print(f"Installation failed: {result.stderr}")
return 1

return 0

if __name__ == '__main__':
sys.exit(main())
< /code>
constraints.txt
# Block all PyPI OpenCV packages - we use our custom build
opencv-python==99.99.99 # Impossible version to prevent installation
opencv-python-headless==99.99.99 # Impossible version to prevent installation
opencv-contrib-python==99.99.99 # Impossible version to prevent installation
opencv-contrib-python-headless==99.99.99 # Impossible version to prevent installation


Подробнее здесь: https://stackoverflow.com/questions/797 ... -with-libr