Yocto How to stop cmake to look in a native sysroot path for linking? - python-3.x

Question

I'm trying to add dlib python module into my image so far this is the recipe i'm working on...
# python3-dlib_19.21.1.bb
SUMMARY = "A toolkit for making real world machine learning and data analysis applications"
HOMEPAGE = "https://pypi.python.org/pypi/dlib"
PYPI_PACKAGE = "dlib"
LICENSE = "Boost-Software"
SRC_URI[md5sum] = "1e7e357d7d54e86267ef60f606cb40e1"
LIC_FILES_CHKSUM = "file://dlib/LICENSE.txt;md5=2c7a3fa82e66676005cd4ee2608fd7d2 \
file://dlib/external/libpng/LICENSE;md5=243135ddedf702158f9170807cbcfb66 \
file://dlib/external/pybind11/LICENSE;md5=beb87117af69fd10fbf9fb14c22a2e62 \
file://python_examples/LICENSE_FOR_EXAMPLE_PROGRAMS.txt;md5=064f53ab40ea2b6a4bba1324149e4fde \
"
DEPENDS = "cmake-native"
inherit pypi setuptools3
but when do_compile() ran i got the following error message:
| [100%] Building CXX object CMakeFiles/_dlib_pybind11.dir/src/line.cpp.o
| [100%] Linking CXX shared module build_dunfell/tmp/work/aarch64-poky-linux/python3-dlib/19.21.1-r0/build/lib.linux-x86_64-3.8/_dlib_pybind11.cpython-38-aarch64-linux-gnu.so
| build_dunfell/tmp/work/aarch64-poky-linux/python3-dlib/19.21.1-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux/../../libexec/aarch64-poky-linux/gcc/aarch64-poky-linux/8.3.0/ld: build_dunfell/tmp/work/aarch64-poky-linux/python3-dlib/19.21.1-r0/recipe-sysroot-native/usr/lib/libsqlite3.so: error adding symbols: file in wrong format
| collect2: error: ld returned 1 exit status
| CMakeFiles/_dlib_pybind11.dir/build.make:445: recipe for target 'build_dunfell/tmp/work/aarch64-poky-linux/python3-dlib/19.21.1-r0/build/lib.linux-x86_64-3.8/_dlib_pybind11.cpython-38-aarch64-linux-gnu.so' failed
| make[2]: *** [build_dunfell/tmp/work/aarch64-poky-linux/python3-dlib/19.21.1-r0/build/lib.linux-x86_64-3.8/_dlib_pybind11.cpython-38-aarch64-linux-gnu.so] Error 1
| CMakeFiles/Makefile2:116: recipe for target 'CMakeFiles/_dlib_pybind11.dir/all' failed
| make[1]: *** [CMakeFiles/_dlib_pybind11.dir/all] Error 2
| Makefile:83: recipe for target 'all' failed
| make: *** [all] Error 2
| Traceback (most recent call last):
| File "build_dunfell/tmp/work/aarch64-poky-linux/python3-dlib/19.21.1-r0/dlib-19.21.1/setup.py", line 223, in <module>
| setup(
| File "build_dunfell/tmp/work/aarch64-poky-linux/python3-dlib/19.21.1-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/setuptools/__init__.py", line 144, in setup
| return distutils.core.setup(**attrs)
| File "build_dunfell/tmp/work/aarch64-poky-linux/python3-dlib/19.21.1-r0/recipe-sysroot-native/usr/lib/python3.8/distutils/core.py", line 148, in setup
| dist.run_commands()
| File "build_dunfell/tmp/work/aarch64-poky-linux/python3-dlib/19.21.1-r0/recipe-sysroot-native/usr/lib/python3.8/distutils/dist.py", line 966, in run_commands
| self.run_command(cmd)
| File "build_dunfell/tmp/work/aarch64-poky-linux/python3-dlib/19.21.1-r0/recipe-sysroot-native/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
| cmd_obj.run()
| File "build_dunfell/tmp/work/aarch64-poky-linux/python3-dlib/19.21.1-r0/recipe-sysroot-native/usr/lib/python3.8/distutils/command/build.py", line 135, in run
| self.run_command(cmd_name)
| File "build_dunfell/tmp/work/aarch64-poky-linux/python3-dlib/19.21.1-r0/recipe-sysroot-native/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
| self.distribution.run_command(command)
| File "build_dunfell/tmp/work/aarch64-poky-linux/python3-dlib/19.21.1-r0/recipe-sysroot-native/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
| cmd_obj.run()
| File "build_dunfell/tmp/work/aarch64-poky-linux/python3-dlib/19.21.1-r0/dlib-19.21.1/setup.py", line 135, in run
| self.build_extension(ext)
| File "build_dunfell/tmp/work/aarch64-poky-linux/python3-dlib/19.21.1-r0/dlib-19.21.1/setup.py", line 175, in build_extension
| subprocess.check_call(cmake_build, cwd=build_folder)
| File "build_dunfell/tmp/work/aarch64-poky-linux/python3-dlib/19.21.1-r0/recipe-sysroot-native/usr/lib/python3.8/subprocess.py", line 364, in check_call
| raise CalledProcessError(retcode, cmd)
| subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j8']' returned non-zero exit status 2.
| WARNING: exit code 1 from a shell command.
|
it looks like the problem occurs when cmake is trying to link the shared object _dlib_pybind11.cpython-38-aarch64-linux-gnu.so. Linker complains with the following error
adding symbols: file in wrong format
i think the linker is mixing target objects with native ones, but i'm not used to cmake so i don't know how to stop it to use the path of recipe-sysroot-native and use my target sysroot, if i remove cmake-native and use only DEPENDS="cmake", setup.py fails because it can not find cmake at all...
hope you can help me or give me any idea that i can try.
BR,
Update 1
just as #Nayfe pointed out dlib/setup.py has an --set option to pass CMAKE config variables, below is a recipe that already worked for me, i can import dlib without problems.
SUMMARY = "A toolkit for making real world machine learning and data analysis applications"
HOMEPAGE = "https://pypi.python.org/pypi/dlib"
PYPI_PACKAGE = "dlib"
LICENSE = "Boost-Software"
SRC_URI[md5sum] = "1e7e357d7d54e86267ef60f606cb40e1"
LIC_FILES_CHKSUM = "file://dlib/LICENSE.txt;md5=2c7a3fa82e66676005cd4ee2608fd7d2 \
file://dlib/external/libpng/LICENSE;md5=243135ddedf702158f9170807cbcfb66 \
file://dlib/external/pybind11/LICENSE;md5=beb87117af69fd10fbf9fb14c22a2e62 \
file://python_examples/LICENSE_FOR_EXAMPLE_PROGRAMS.txt;md5=064f53ab40ea2b6a4bba1324149e4fde \
"
DEPENDS = "sqlite3 "
inherit pypi cmake setuptools3
INSANE_SKIP_${PN} = "already-stripped"
DISTUTILS_BUILD_ARGS_append = " \
--set CMAKE_TOOLCHAIN_FILE=${WORKDIR}/toolchain.cmake \
"

Answer 1

Maybe recipe can be optimized but the following works
python3-dlib_19.21.bb
SUMMARY = "A toolkit for making real world machine learning and data analysis applications"
HOMEPAGE = "http://dlib.net/"
SECTION = "devel/python"
LICENSE = "BSL-1.0"
LIC_FILES_CHKSUM = "file://dlib/LICENSE.txt;md5=2c7a3fa82e66676005cd4ee2608fd7d2"
DEPENDS = "sqlite3"
SRC_URI = "git://github.com/davisking/dlib"
SRCREV = "9117bd784328d9ac40ffa1f9cf487633a8a715d7"
S = "${WORKDIR}/git"
DISTUTILS_BUILD_ARGS_append = " \
--set CMAKE_INSTALL_PREFIX:PATH=${prefix} \
--set CMAKE_INSTALL_BINDIR:PATH=${#os.path.relpath(d.getVar('bindir'), d.getVar('prefix') + '/')} \
--set CMAKE_INSTALL_SBINDIR:PATH=${#os.path.relpath(d.getVar('sbindir'), d.getVar('prefix') + '/')} \
--set CMAKE_INSTALL_LIBEXECDIR:PATH=${#os.path.relpath(d.getVar('libexecdir'), d.getVar('prefix') + '/')} \
--set CMAKE_INSTALL_SYSCONFDIR:PATH=${sysconfdir} \
--set CMAKE_INSTALL_SHAREDSTATEDIR:PATH=${#os.path.relpath(d.getVar('sharedstatedir'), d. getVar('prefix') + '/')} \
--set CMAKE_INSTALL_LOCALSTATEDIR:PATH=${localstatedir} \
--set CMAKE_INSTALL_LIBDIR:PATH=${#os.path.relpath(d.getVar('libdir'), d.getVar('prefix') + '/')} \
--set CMAKE_INSTALL_INCLUDEDIR:PATH=${#os.path.relpath(d.getVar('includedir'), d.getVar('prefix') + '/')} \
--set CMAKE_INSTALL_DATAROOTDIR:PATH=${#os.path.relpath(d.getVar('datadir'), d.getVar('prefix') + '/')} \
--set PYTHON_EXECUTABLE:PATH=${PYTHON} \
--set Python_EXECUTABLE:PATH=${PYTHON} \
--set Python3_EXECUTABLE:PATH=${PYTHON} \
--set LIB_SUFFIX=${#d.getVar('baselib').replace('lib', '')} \
--set CMAKE_INSTALL_SO_NO_EXE=0 \
--set CMAKE_TOOLCHAIN_FILE=${WORKDIR}/toolchain.cmake \
--set CMAKE_NO_SYSTEM_FROM_IMPORTED=1 \
--set CMAKE_LIBRARY_OUTPUT_DIRECTORY=${B} \
"
do_configure() {
distutils3_do_configure
}
do_compile() {
distutils3_do_compile
}
do_install() {
distutils3_do_install
}
inherit cmake setuptools3 python3native
INSANE_SKIP_${PN} = "already-stripped"
RDEPENDS_${PN} += "python3-core"