I got tired of all the reminders that TensorFlow is not optimized for my CPU and so finally compiled it from source. In fact I did it twice and made two .whl files, once using
bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package
and once using
bazel build --config=mkl --config=opt //tensorflow/tools/pip_package:build_pip_package
since I was not sure how much difference the Intel MKL would make. Now I have setup two identical anaconda environments, one using each whl file.
What is the quickest way to determine which of the .whl packages performs better on my system? If someone can point me to a standard Benchmark package/command in tensorflow that would be great (please note that I do not have GPU support).
You can run the below benchmark and check the performance.
https://github.com/tensorflow/benchmarks.git
git clone the above code to your terminal and then run the tf_cnn_benchmark.py benchmark code.
Thanks
Related
Platform: Precision 5820, 32G, rtx4000; Win 10 Pro, Arcgis Pro 2.6 concurrent license;
Issue:
I installed the deep learning tools following the guidelines provided here:
deeplearninginstallation
tersorflow was not found after installation so I manually installed the 2.1.0 version. I now have arcgis 1.8.2, pro 2.6, fastai 1.0.60, python 3.6.12, pytorch 1.4.0, tensorflow-gpu 2.1.0; environment check in arcgis pro python seemed fine.
However, after I select toolbox-image analyst-deeplearning-traindeeplearningmodel, the program seems to go into a hang, with most buttons disabled/unresponsive, this would continue until I force terminate the program. I also ran into "tool not licensed" twice, which was gone after I restarted the program; and a "name 'CallBackHandler' is not defined" once, which was also gone after I restarted.
I tried runing the command from the arcgis pro python prompt:
TrainDeepLearningModel(r"**", r"**", 40, "RETINANET", 16, "# #", None, "RESNET50", None, 10, "STOP_TRAINING", "FREEZE_MODEL")
executing the command would also send the program into a hang similar to the previous one. Monitor shows that ram and GPU usage haven't changed much, so I left the program running for an hour before forcibly terminating it.
I'd greatly appreciate it if anyone can tell me what the issues are here. I'll post any other env parameters if anyone requires. Cheers.
I got the tool up and running now by running conda install -c pytorch -c fastai fastai=1.0.54 pytorch=1.1.0 torchvision scikit-image and removing all the conflicting specifications in the cloned arcgispro-py3 env that I had. Now I still don't understand what went wrong. Presumably one or more packages in the env was conflicting. But seeing as I'm not a python expert, I couldn't identify the exact issue.
Before this I tried the versions stated here deeplearning install guide, but wasn't able to get pass tensorflow-gpu because python kept checking conflications. Now I actually don't have tensorflow-gpu in the env. I have tensorflow 2.1.0, keras-applications 1.0.8/base 2.3.1/preprocesing 1.1.0 (no keras-gpu), scikit-image 0.17.2, pillow 6.2.1, fastai 1.0.54, pytorch 1.1.0, libtiff 4.0.10. Some are different from what the guideline provided.
Thing is when I ran the process, CPU usage was up and GPU wasn't despite the fact that I specified GPU as the processing core. But I have much more pressing things to do right now like getting the analysis finished. So I'll probably tweek the env around a little after I'm done with this bit and see what happens. Meanwhile, anyone's input is still welcome.
I am trying to use Theano on the Azure DSVM which is a preconfigured VM for Data Science. Is anyone aware of such a VM and does it support Theano out of the box?
WARNING (theano.configdefaults): g++ not detected ! Theano will be unable to execute optimized C-implementations (for both CPU and GPU) and will default to Python implementations. Performance will be severely degraded. To remove this warning, set Theano flags cxx to an empty string.
Anyone with some experience in this regard?
We recently released a new version of the Data Science Virtual Machine for Linux (Ubuntu) edition which has Theano (and several other popular deep learning tools) pre-installed that you can use on either NC-Series GPU Azure VM or any CPU only VM.
More info:
http://aka.ms/dsvm/ubuntu
http://aka.ms/dsvm/ubuntu/docs
I have experienced the same issue since Theano has dependency on g++. So what i did is the following:
1) Install anaconda from here.
3) Open anaconda command prompt and write the following command. This will install Theano with all its dependencies including g++ and others.
conda install theano
I need to run a program which use VTK5 on my Archlinux PC, but I found it really hard to install VTK5, there is only VTK6(not compatible with VTK5) in official repo, and when I try to install it from AUR, it returns "Makepg was unable to build vtk5", then I try to install through source code, the result is that I was unable to install the VTK Python module...
Is there anybody who has any experience or idea about it?
I have not installed on Archlinux specifically, but on different linux machines. If you compile from source and are interested in python, remember to select the option python wrapping when running cmake. Btw, once built, you will have to update both the pythonpath and the ldlibrarypath.
You can also have a try at enthought canopy, which distributes a complete installation with numpy, scipy, vtk http://docs.enthought.com/canopy/quick-start/install_linux.html
I've installed CUDA on my OSX Yosemite. I've downloaded the accelerate-examples package and compiled it with cabal install. It compiled correctly. When I ran the examples, though, I noticed they do not offer a option to run under CUDA. For example:
vh:accelerate-crystal apple1$ ./accelerate-crystal
EKG monitor started at: http://localhost:8000
accelerate-crystal (c) [2011..2013] The Accelerate Team
Usage: accelerate-crystal [OPTIONS]
Available backends:
* interpreter reference implementation (sequential)
This makes them run slow (and, obviously, beat the purpose). How do I enable CUDA on the compiled examples?
I think you just need to use the -fcuda flag when you Cabal install. This will install the acclerate-cuda package and enable the CUDA backend for all the examples (they seem to use the CUDA backend by default if it's enabled).
Does anybody know how to build Octave for x64 Windows? The 2GB data limitation for x32 is too limiting for many problems that require analysis on large data sets.
http://wiki.octave.org/Octave_for_Microsoft_Windows has information on installing Octave on Windows and links to building it from source using different methods.
GNU Octave is primarily developed on GNU/Linux and other POSIX conformal systems. The ports of GNU Octave to Windows use different approaches to get most of the original Octave and adapt it to Microsoft Windows idiosyncrasies...
Windows support is experimental.
According to http://www.gnu.org/software/octave/doc/interpreter/Compiling-Octave-with-64_002dbit-Indexing.html
To use arrays larger than 2 GB, Octave has to be configured with the option --enable-64. This option is experimental...
Compiling Octave for 64 bit is experimental on Linux. It might cause a lot of headache to try an experimental feature in a port of the software. It would be better to use a true Linux installation for now. If you feel adventurous, try compiling it in http://www.cygwin.com/
I have installed Octave-4.0.0 into windows 7,8 and 10 in x64 platforms. All works perfectly well.
Just follow these steps
Download Octave-4.0.0_0-installer.exe from https://ftp.gnu.org/gnu/octave/windows/
Install the same - just follow the steps in the installer.
Find the build_packages.m file in C:\Octave\Octave 4.0.0\src
Open it in Octave and find
try install general-1.3.4.tar.gz, and try install signal-1.3.1.tar.gz, the versions are wrong.
Replace with 2.0.0 and 1.3.2 respectively.
In the build_packages.m file find
pkg ('install', pkgname, '-noauto').
Change it to
pkg ('install', pkgname).
Skip this and you will have to load the packages you require every time you use Octave. Lesser load for octave though. Sometimes it may take a while for the packages to get installed, kindly wait.
Run build_packages.m
load the packages
e.g. to load the general package - pkg load general
Note that the signal package is dependent on the control package.
I found that the plot function got octave stuck. The answer for the same is to type in at the command window
pkg rebuild -noauto oct2mat
Found this solution in Plot window not responding
Hope this works for u too. :)
I found Sreepad's ans is CORRECT. I use octave on win 10 64-bit OS.
octave 4.0.0 is ok as Sreepad said, But Octave 4.2.1 is not OK on Win 10 64-bit OS.