Unable to build Tensorflow from source MacOS High Sierra - python-3.x

I've followed all the steps in the official guide. Except I built it using:
$ bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=- msse4.1 --copt=-msse4.2 --config=opt -k //tensorflow/tools/pip_package:build_pip_package
And during ./config I've set the right paths and disabled Google Cloud Platform, Hadoop, XLA, VERBS, OpenCL, CUDA, MPI support.
Hardware:
Macbook Pro 13 inch (mid 2014)
CPU: Intel Core i5 (4278U)
RAM: 8GB
Software:
High Sierra (10.13.2)
Clang Version: clang-900.0.39.2
Bazel Version: 0.9.0
Conda Version: 4.4.3
Python: 3.6.3
All the packages are upto date. This worked perfectly fine 2 months ago on this machine. For some strange reasons it doesn't build anymore now. I'm just posting a part of the error list here:
WARNING: Config values are not defined in any .rc file: opt
ERROR: Skipping 'msse4.1': no such target '//:msse4.1': target 'msse4.1' not declared in package '' defined by /Users/rakshithgb/Documents/Tensorflow/tensorflow/BUILD
WARNING: Target pattern parsing failed.
ERROR: /private/var/tmp/_bazel_rakshithgb/fde7bc60972656b0c2db4fd0b79e24fb/external/com_googlesource_code_re2/BUILD:96:1: First argument of 'load' must be a label and start with either '//', ':', or '#'. Use --incompatible_load_argument_is_label=false to temporarily disable this check.
ERROR: /private/var/tmp/_bazel_rakshithgb/fde7bc60972656b0c2db4fd0b79e24fb/external/com_googlesource_code_re2/BUILD:98:1: name 're2_test' is not defined (did you mean 'ios_test'?)
ERROR: /private/var/tmp/_bazel_rakshithgb/fde7bc60972656b0c2db4fd0b79e24fb/external/com_googlesource_code_re2/BUILD:100:1: name 're2_test' is not defined (did you mean 'ios_test'?)
And it ends like this:
ERROR: /Users/rakshithgb/Documents/Tensorflow/tensorflow/tensorflow/core/kernels/BUILD:550:1: Target '#local_config_sycl//sycl:using_sycl' contains an error and its package is in error and referenced by '//tensorflow/core/kernels:debug_ops'
WARNING: errors encountered while analyzing target '//tensorflow/tools/pip_package:build_pip_package': it will not be built
INFO: Analysed target //tensorflow/tools/pip_package:build_pip_package (203 packages loaded).
INFO: Found 0 targets...
ERROR: command succeeded, but there were errors parsing the target pattern
INFO: Elapsed time: 12.763s, Critical Path: 0.02s
FAILED: Build did NOT complete successfully
Has anyone else had this issue? How do I fix it? I've uploaded the entire error log on GitHub Tensorflow issue page. #15622

Ok it looks like the new bazel version isn't compatible with the current Tensorflow release. It looks like the fix will be issued in the next release. According to this thread on GitHub - #15492
The temporary fix that worked for me was to build it using --incompatible_load_argument_is_label=false in the bazel command. So my build command now looks like this:
$ bazel build --config=opt --incompatible_load_argument_is_label=false //tensorflow/tools/pip_package:build_pip_package

Related

unknown import: `wasi_snapshot_preview1::sock_setsockopt` has not been defined when running Rust WASI binary

Title says it all - I am attempting to run a very basic Rust program using hyper_wasi, and everything compiles seemingly ok using cargo wasi run. Unfortunately when it actually executes, I get:
Error: failed to run main module `target/wasm32-wasi/debug/<my project name here>.wasm`
Caused by:
0: failed to instantiate "target/wasm32-wasi/debug/<my project name here>.wasm"
1: unknown import: `wasi_snapshot_preview1::sock_setsockopt` has not been defined
Execution environment is a rather older Mac (2013 MBP with Big Sur), clang version:
Homebrew clang version 15.0.3
Target: x86_64-apple-darwin20.6.0
Thread model: posix
InstalledDir: /usr/local/opt/llvm/bin
If anyone can point me at what I might be missing I'd appreciate it.

`Undefined symbol: ossl_sha3_512_functions` when trying to build the Kadena chainweb-node Haskell project

I've been trying to compile the Kadena chainweb-node project from source (via the docs found here) in a docker container running the arm64v8/ubuntu base image on an Apple M1 host machine and I keep getting errors. Thanks to the super helpful folks in the #haskell IRC channel, I've made it over a number of small hurdles, but the build still fails with:
Undefined symbol: ossl_sha3_512_functions
I can not find any resources on how to fix this but one of the #haskell community members informed me that these symbols are not provided by the openssl 3.0 API. That person created an issue here: https://github.com/larskuhtz/hs-hashes/issues/14
I've found that I can use the docker base image haskell:8 which comes preinstalled with ghc, cabal, and openssl 1.1.1 to cabal build chainweb-node successfully using a slightly different set of apt-get dependencies. However, when trying to cabal install the binaries so that I can run chainweb-node, I get:
Failed to build chainweb-2.14.1. The failure occurred during the configure
step.
Build log (
/root/.cabal/logs/ghc-8.10.7/chainweb-2.14.1-36aedf5adc1967eb17358e6434b6fd51bc7e64082f6c12e0df40ddafece6ff69.log
):
[1 of 1] Compiling Main ( /tmp/cabal-install.-838/dist-newstyle/tmp/src-838/chainweb-2.14.1/dist/setup/setup.hs, /tmp/cabal-install.-838/dist-newstyle/tmp/src-838/chainweb-2.14.1/dist/setup/Main.o )
Linking /tmp/cabal-install.-838/dist-newstyle/tmp/src-838/chainweb-2.14.1/dist/setup/setup ...
Configuring chainweb-2.14.1...
setup: Encountered missing or private dependencies:
criterion -any,
data-ordlist >=0.4.7,
resource-pool >=0.2,
retry >=0.7,
statistics >=0.15,
tasty-json >=0.1
cabal: Failed to build chainweb-2.14.1. See the build log above for details.
I've tried cabal install criterion and the other missing deps, one by one, but the cabal install gives me the same errors.
Can someone tell me what I'm doing wrong and help me get a working chainweb-node binary built from source and installed using docker?

trying to build pytorch 1.0.0 cuda 10.2 with support for old gpu (3.0)

I'm playing with a couple of projects that explicitly require pytorch == 1.0.0, but I have an old graphics card that only supports cuda 3.0 so I'm using the cpu, which is very slow, being the graphics card a dual gpu I decided to give a try and build pytorch from the sources with support for 3.0 (I have planned to update the pc but is not gonna happen anytime soon).
I am using docker to do the build, in particular I tried to modify an existing Dockerfile from build-pytorch, on the host system I am using debian/sid and there is cuda 10.2 cudnn 7.6 installed, I'm not sure if I can downgrade cuda, and I don't know if the versions in the container must be exactly the same as the host (like for nvidia drivers).
Gist of the modified Dockerfile
The first thing I noticed when updating the versions is that package cuda-cublas-dev-10-2 was not found, the latest version was 10-0,
CUBLAS packaging changed in CUDA 10.1 to be outside of the toolkit installation path
If I install cublas version 10-0 or if I don't install it obviously no header files are found (error below), if I install the recommended libcublas-dev version the build continues for a while, with some warnings (below) , but then it stops with the error below.
I searched for the error online but I did not find anything specific, if I understand correctly there is a function declared more than once and when it is called the choice is ambiguous, but I have not yet investigated looking at the sources.
I would like to know if anyone has run into this error before and knows how to fix it.
libcublas-dev installed error:
[ 67%] Building NVCC (Device) object caffe2/CMakeFiles/caffe2_gpu.dir/__/aten/src/ATen/native/sparse/cuda/caffe2_gpu_generated_SparseCUDABlas.cu.o
/pytorch/aten/src/ATen/native/sparse/cuda/SparseCUDABlas.cu(58): error: more than one instance of function "at::native::sparse::cuda::cusparseGetErrorString" matches the argument list:
function "cusparseGetErrorString(cusparseStatus_t)"
function "at::native::sparse::cuda::cusparseGetErrorString(cusparseStatus_t)"
argument types are: (cusparseStatus_t)
1 error detected in the compilation of "/tmp/tmpxft_00004ccc_00000000-6_SparseCUDABlas.cpp1.ii".
CMake Error at caffe2_gpu_generated_SparseCUDABlas.cu.o.Release.cmake:279 (message):
Error generating file
/pytorch/build/caffe2/CMakeFiles/caffe2_gpu.dir/__/aten/src/ATen/native/sparse/cuda/./caffe2_gpu_generated_SparseCUDABlas.cu.o
caffe2/CMakeFiles/caffe2_gpu.dir/build.make:1260: recipe for target 'caffe2/CMakeFiles/caffe2_gpu.dir/__/aten/src/ATen/native/sparse/cuda/caffe2_gpu_generated_SparseCUDABlas.cu.o' failed
warnings:
ptxas warning : Too big maxrregcount value specified 96, will be ignored
missing header error:
Scanning dependencies of target caffe2_pybind11_state
[ 59%] Building CXX object caffe2/CMakeFiles/caffe2_pybind11_state.dir/python/pybind_state.cc.o
In file included from /pytorch/aten/src/THC/THC.h:4:0,
from /pytorch/torch/lib/THD/../THD/base/TensorDescriptor.h:6,
from /pytorch/torch/lib/THD/../THD/base/TensorDescriptor.hpp:6,
from /pytorch/torch/lib/THD/../THD/THD.h:14,
from /pytorch/torch/lib/THD/base/DataChannelRequest.h:3,
from /pytorch/torch/lib/THD/base/DataChannelRequest.hpp:6,
from /pytorch/torch/lib/THD/base/DataChannelRequest.cpp:1:
/pytorch/build/caffe2/aten/src/THC/THCGeneral.h:17:23: fatal error: cublas_v2.h: No such file or directory
compilation terminated.
make[2]: *** [caffe2/torch/lib/THD/CMakeFiles/THD.dir/base/DataChannelRequest.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
Apparently the problem was that both libcusparse and aten/src/ATen/native/sparse/cuda/SparseCUDABlas.cu implement cusparseGetErrorString() and for version >= 10.2 the one in the library should be used.
--- aten/src/ATen/native/sparse/cuda/SparseCUDABlas.cu.orig 2020-11-16 12:13:17.680023134 +0000
+++ aten/src/ATen/native/sparse/cuda/SparseCUDABlas.cu 2020-11-16 12:13:45.158407583 +0000
## -9,7 +9,7 ##
namespace at { namespace native { namespace sparse { namespace cuda {
-
+#if 0
std::string cusparseGetErrorString(cusparseStatus_t status) {
switch(status)
{
## -51,6 +51,7 ##
}
}
}
+#endif
inline void CUSPARSE_CHECK(cusparseStatus_t status)
{
I haven't tried yet if it works at runtime but the build is successful.

Yocto bitbake build error when adding fftw package

I am using Yocto Warrior release to build linux for Dart-imx8m SOM. Documentation can be found here : https://variwiki.com/index.php?title=DART-MX8M_Yocto&release=RELEASE_WARRIOR_V1.1_DART-MX8M.
I want to add fftw package whose recipe is in meta-oe layer. Whenever I add this package in my local.conf file, I get an error with bitbake regarding a dnf related task.
I add the package like this in my local.conf file : IMAGE_INSTALL_append = " fftw"
I get the following error when building image with bitbake fsl-image-gui :
ERROR: fsl-image-gui-1.0-r0 do_rootfs: Could not invoke dnf. Command '/home/safetyn/safetynYoctoGenerator/warrior/var-fsl-yocto/build_xwayland/tmp/work/imx8mq_var_dart-poky-linux/fsl-image-gui/1.0-r0/recipe-sysroot-native/usr/bin/dnf -v --rpmverbosity=info -y -c /home/safetyn/safetynYoctoGenerator/warrior/var-fsl-yocto/build_xwayland/tmp/work/imx8mq_var_dart-poky-linux/fsl-image-gui/1.0-r0/rootfs/etc/dnf/dnf.conf --setopt=reposdir=/home/safetyn/safetynYoctoGenerator/warrior/var-fsl-yocto/build_xwayland/tmp/work/imx8mq_var_dart-poky-linux/fsl-image-gui/1.0-r0/rootfs/etc/yum.repos.d --installroot=/home/safetyn/safetynYoctoGenerator/warrior/var-fsl-yocto/build_xwayland/tmp/work/imx8mq_var_dart-poky-linux/fsl-image-gui/1.0-r0/rootfs --setopt=logdir=/home/safetyn/safetynYoctoGenerator/warrior/var-fsl-yocto/build_xwayland/tmp/work/imx8mq_var_dart-poky-linux/fsl-image-gui/1.0-r0/temp --repofrompath=oe-repo,/home/safetyn/safetynYoctoGenerator/warrior/var-fsl-yocto/build_xwayland/tmp/work/imx8mq_var_dart-poky-linux/fsl-image-gui/1.0-r0/oe-rootfs-repo --nogpgcheck install packagegroup-core-tools-profile packagegroup-fsl-tools-audio packagegroup-core-ssh-dropbear tree rpm packagegroup-fsl-tools-benchmark boost gstreamer1.0 packagegroup-fsl-tools-gpu packagegroup-fsl-gstreamer1.0 psplash gpsd packagegroup-base-extended collectd kernel-image-4.19.35-imx8mq+gdc7a2fd packagegroup-core-tools-testapps packagegroup-core-boot tcf-agent packagegroup-fsl-tools-gpu-external packagegroup-core-full-cmdline packagegroup-tools-bluetooth packagegroup-fsl-tools-testapps opencv fftw libsdl-1.2-0 packagegroup-fsl-gstreamer1.0-full lftp packagegroup-core-tools-debug libgpiod2 dnf weston-init run-postinsts mosquitto packagegroup-core-nfs-server sshfs-fuse spitools kernel-devicetree android-tools htop poco tmux openssh-sftp-server weston-xwayland xterm locale-base-en-us locale-base-en-gb' returned 1:
DNF version: 4.1.0
cachedir: /home/safetyn/safetynYoctoGenerator/warrior/var-fsl-yocto/build_xwayland/tmp/work/imx8mq_var_dart-poky-linux/fsl-image-gui/1.0-r0/rootfs/var/cache/dnf
Added oe-repo repo from /home/safetyn/safetynYoctoGenerator/warrior/var-fsl-yocto/build_xwayland/tmp/work/imx8mq_var_dart-poky-linux/fsl-image-gui/1.0-r0/oe-rootfs-repo
repo: using cache for: oe-repo
not found other for:
not found modules for:
not found deltainfo for:
not found updateinfo for:
oe-repo: using metadata from Wed 28 Oct 2020 09:16:57 AM UTC.
Last metadata expiration check: 0:00:01 ago on Wed 28 Oct 2020 09:16:57 AM UTC.
No module defaults found
No match for argument: fftw
Error: Unable to find a match
ERROR: fsl-image-gui-1.0-r0 do_rootfs:
ERROR: fsl-image-gui-1.0-r0 do_rootfs: Function failed: do_rootfs
ERROR: Logfile of failure stored in: /home/safetyn/safetynYoctoGenerator/warrior/var-fsl-yocto/build_xwayland/tmp/work/imx8mq_var_dart-poky-linux/fsl-image-gui/1.0-r0/temp/log.do_rootfs.2614
ERROR: Task (/home/safetyn/safetynYoctoGenerator/warrior/var-fsl-yocto/sources/meta-fsl-bsp-release/imx/meta-sdk/recipes-fsl/images/fsl-image-gui.bb:do_rootfs) failed with exit code '1'
Woud you have any clue where this error come from ?
Thanks,
Paul
Edit :
Found a way to make it compile for those who are intereseted. (inspired by Yocto build for a static library fails with error "No Match Found")
I edited my *.bb file and added ALLOW_EMPTY_${PN} = "1"
I am not sure I got it completely but it looks like dnf package manager needs folders created for all added packages even if empty.
https://www.yoctoproject.org/docs/latest/ref-manual/ref-manual.html#var-ALLOW_EMPTY
The fftw recipe is set up to create a few different packages (RPM) like libfftw, libfftwl, libfftwf, fftw-wisdom, fftwl-wisdom, fftwf-wisdom, and fftw-wisdom-to-conf.
You probably want to add one or more of those. It seems there is no actual fftw package.
It is important to remember that IMAGE_INSTALL and RDEPEND lists items from the package namespace, while DEPENDS lists items from the recipe namespace.
If you are unsure about which package you want to install you can inspect the packages-split folder for fftw in tmp/work to see which files are included in which package.

pathspec 'gnulib' did not match any file(s) known to git while building Linux image with Yocto

I am building a Linux image using Yocto SDK for QorIQ T1042 PowerPC processor on my Ubuntu 16.04.
My problem is, when I try to build a full image using "bitbake fsl-image-full", there is an error says "pathspec 'gnulib' did not match any file(s) known to git."
I researched many forums but couldn't find any answer. Here is the full log from terminal.
|./bootstrap: Bootstrapping from checked-out netcf sources...
| ./bootstrap: consider installing git-merge-changelog from gnulib
| ./bootstrap: getting gnulib files...
| error: pathspec 'gnulib' did not match any file(s) known to git.
| WARNING: /media/hwlab/depo/CalismaBolgesi/YoctoInstallationPoint/yocto/build_mpx1042/tmp/work/ppce5500-fsl-linux/netcf/0.2.8+gitAUTOINC+9158278ad3-r0/temp/run.do_configure.24998:1 exit 1 from
| ./bootstrap --force
| ERROR: Function failed: do_configure (log file is located at /media/hwlab/depo/CalismaBolgesi/YoctoInstallationPoint/yocto/build_mpx1042/tmp/work/ppce5500-fsl-linux/netcf/0.2.8+gitAUTOINC+9158278ad3-r0/temp/log.do_configure.24998)
ERROR: Task 4195 (/media/hwlab/depo/CalismaBolgesi/YoctoInstallationPoint/yocto/sources/meta-openembedded/meta-networking/recipes-support/netcf/netcf_git.bb, do_configure) failed with exit code '1'
NOTE: Tasks Summary: Attempted 2036 tasks of which 2031 didn't need to be rerun and 1 failed.
Waiting for 0 running tasks to finish:
This issue was fixed more than 3 years ago in meta-opembedded: https://git.openembedded.org/meta-openembedded/commit/?id=093942197237a0b79802d20ebcf3ae0ea35a66bc
Consider updating your layers (or if they come from a vendor, asking them to update the SDK) or backporting the patch.

Resources