I have run the tutorials and created my own neural network implementation in tensorflow successfully. I then decided to go one bit further an add my own op because I needed to do some of my own preprocessing on the data. I followed the tutorial on the tensorflow site to add an op. I successfully built tensorflow after writing my own c++ file. Then, when I try to use it from my code, I get
'module' object has no attribute 'sec_since_midnight'
My code does get reflected in bazel-genfiles/tensorflow/python/ops/gen_user_ops.py so the wrapper does get generated for it correctly. It just looks like I can't see the tensorflow/python/user_ops/user_ops.py which is what imports that file.
Now when I when I go through the testing of this module, I get the following odd behavior. It should not pass because the expected vector I give it does not match what the result should be. But maybe the test never gets executed despite saying passed?
INFO: Found 1 test target...
Target //tensorflow/python:sec_since_midnight_op_test up-to-date:
bazel-bin/tensorflow/python/sec_since_midnight_op_test
INFO: Elapsed time: 6.131s, Critical Path: 5.36s
//tensorflow/python:sec_since_midnight_op_test (1/0 cached) PASSED
Executed 0 out of 1 tests: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
Hmmm. Well, I uninstalled tensorflow and then I reinstalled from what I just built and what I wrote was suddenly recognized. I have seen this behavior twice in a row now where an uninstall is necessary. So to sum, the steps after adding my own op are:
$ pip uninstall tensorflow
$ bazel build -c opt //tensorflow/tools/pip_package:build_pip_package
# To build with GPU support:
$ bazel build -c opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
$ bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
# The name of the .whl file will depend on your platform.
$ pip install /tmp/tensorflow_pkg/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
Related
The issue is I want to use colmap on a set of images, but am not sure the proper way to go about it in linux in this particular circumstance; colmap works fine in another conda environment, but I don't understand the differences, and am uncertain what my problems are due to. I am asking (please) for clarity in understanding how to approach this, or solutions if you happen to have one.
I have tried two ways of installing colmap, colmap's offical linux instillation guide, https://colmap.github.io/install.html, and by running sudo apt install colmap.When doing a simple sudo apt install, everything generally works, but with the code I am trying to use I run into an error where the feature matching fails (EX1).
However, when I uninstall this colmap and try doing it via the other method, it fails at "cmake .. ", which to solve this I have tried specifying my gcc and g++ (CC=/usr/bin/gcc-8 CXX=/usr/bin/g++-9 cmake ..) but this had no effect. The error in this case was a long list of errors similar to (EX2), which seems to indicate an issue with my cuda. Cuda works fine with me normally in base, but I am wondering if maybe being in a conda environment (which I am installing colmap in) would effect its ability to link to targets?
Something else I had heard is that the precompiled version of colmap won't work on linux, so it is neccesary to follow the method in colmap's linux guide rather than doing sudo apt install colmap, but I am confused on that.
I'm very new to linux so generally any advice on how to approach solving this would be helpful!
EX1)
`Exhaustive feature matching
==============================================================================
Shader not supported by your hardware!
ERROR: SiftGPU not fully supported
ERROR: SiftGPU not fully supported
==== running: mkdir colmap_sparse
==== running: colmap mapper --database_path colmap.db --image_path "images" --output_path colmap_sparse
==============================================================================
Loading database
==============================================================================
Loading cameras... 1 in 0.000s
Loading matches... 0 in 0.000s
Loading images... 10 in 0.000s (connected 0)
Building correspondence graph... in 0.000s (ignored 0)
Elapsed time: 0.000 [minutes]
WARNING: No images with matches found in the database.
`
------------------------------------------------
`
**EX2)**
`CMake Error at cmake/CMakeHelper.cmake:121 (add_library):
Target "colmap" links to target "CUDA::cudart" but the target was not
found. Perhaps a find_package() call is missing for an IMPORTED target, or
an ALIAS target is missing?
Call Stack (most recent call first):
src/CMakeLists.txt:65 (COLMAP_ADD_STATIC_LIBRARY)`
OS: Ubuntu 20.04 LT
GPU: RTX A2000
Cuda: 11.8
I have successfully installed pytorch from source using command git clone --recursive https://github.com/pytorch/pytorch.git on my Windows 11 with CPU. But I cannot run the pretrained DL model. It gives error on line: from caffe2.python import workspace. Even though I have workspace on pytorch/caffe2/python/workspace. Please guide if there is anything else I need to do?
Please enable BUILD_CAFFE2 while building PyTorch from source if not already done.
SYSTEM: Ryzen 5800x, rx 6700xt, 32 gigs of RAM, Ubuntu 22.04.1
I'm attempting to install Stable-Diffusion by following https://youtu.be/d_CgaHyA_n4
When attempting to run the SD script, I get the "hipErrorNoBinaryForGpu: Unable to find code object for all current devices!" error.
I believe this is caused by PyTorch not working as expected. When validating Pytorchs' installation with "The Master Test", I get the same error:
"hipErrorNoBinaryForGpu: Unable to find code object for all current devices!"
Aborted (core dumped)
I believe that it is install correctly as using the conda list command tells me that torch 1.12.0a0+git2a932eb and torchvision 0.13.0a0+f5afae5 are installed. Interestingly, when I change the command ever-so-slightly to torch.cuda.is_available, (without the parentheses), I get the following output: <function is_available at 0x7f42278788b0>. Granted, I'm not sure what this is telling me. Following the "Verification" step resulted in the expected array of random numbers. However, failed the GPU driver check.
Thank you in advance.
Try running the following command:
export HSA_OVERRIDE_GFX_VERSION=10.3.0
This made it work on my machine using an RX 6600 XT, with which I got the same error running it, before exporting the variable.
I was struggling with ROCm enabled Tensorflow, the tensorflow-rocm, package. Without setting up the environment variable
export HSA_OVERRIDE_GFX_VERSION=10.3.0
, tensorflow-rocm crashes.
After setting it up, it works with 6700XT as well (at least it does not crash.) But still having problem with finding libraries.
It seems ROCm has a lot to catch up.
I have a custom Python Flask App and I build it for my embedded target using Yocto and Bitbake.
I also have a set of test cases that I run on my Build Machine against the app using PyTest. I'd like the build to fail if the tests fail.
I'm looking for a Yocto way of running these tests as a custom task as part of my recipe. So far by Google search has ( unusually ) come up empty.
Here is what I have implemented so far - it works but it requires the system Python3 and various pip installs. Ideally the requirements would be built in Yocto but only for the Host machine and not the target. I haven't figured out how to do that yet.
# Run the pytest test cases against the app code after it is installed
python do_run_pytest_testsuite(){
# Change to the testing directory
import os
testDir = "%s"%(d.getVar('WORKDIR', True))
os.chdir(testDir)
# Run pytest execute the test suite
from subprocess import Popen, PIPE
with open("%s/TestOutput.txt"%(testDir), "w") as testReportFile:
with Popen(['/usr/bin/python3','-m','pytest','-v','tests/test_app.py'], stdout=PIPE, bufsize=1, universal_newlines=True) as proc:
for line in proc.stdout:
testReportFile.write(line)
# Get the return code
if not proc.returncode == 0:
# Force Failure
bb.fatal("Test Cases Failed! ( %s )"%(testDir))
}
addtask run_pytest_testsuite before do_install after do_configure
How can I accomplish this self-contained within the Yocto environment and not install any PyTest dependencies for the target board.
First I would have a look at poky/meta/recipes-devtools/python to get an idea what python recipes are available (mind which release you are working on).
Then you can add a dependency on the native version of the recipes e.g. DEPENDS = "python3-native python3-yaml-native" (or whatever packages you need to run your python application)
Then add a task that runs your application
do_run_testsuite(){
python3 ${WORKDIR}/test/test_app.py
}
addtask do_run_testsuite before do_install after do_configure
Maybe also check the openembedded python layer
If you don't find all the dependencies you need, it is relatively easy to add a pip package to your own layer (just have a look at the recipes in the mentioned layers).
I used a virtualbox with ubuntu-16, to install caffe in python 2.6. As I wanted to use py2exe, I needed to change python version to 3.6. When this was done, caffe import code stopped working. Here is the error message:
ImportError: libcaffe.so.1.0.0-rc5: cannot open shared object file: No such file
or directory
Do I have to rebuild caffe? Or is something else that need to be changed?
Here is the full image of the error:
Yes, you need to rebuild Caffe. The Python 2 and Python 3 libraries are not compatible. Not all of the file names are the same (different / added software organization). This rebuild requires that you place a single version of Python first in all search paths: make, compile, load, ...
When I need to do this, I check with my SysOps: there's always something I forget.