I am doing a Differential Privacy Course and have to work with syft. When running the below command, I encountered a error
I have already installed syft package required for the analysis. And I am working in Anaconda on Windows 10
from syft.frameworks.torch.differential_privacy import pate
WARNING:tf_encrypted:Falling back to insecure randomness since the required custom op could not be found for the installed version of TensorFlow (1.13.1). Fix this by compiling custom ops.
It is not an error it is a warning. It comes from tf_encrypted which is used in tensorflow. The problem is that on Windows secure randomness is not supported (according to https://github.com/tf-encrypted/tf-encrypted/issues/513). And as you use PyTorch you can just ignore it.
Related
Platform: Precision 5820, 32G, rtx4000; Win 10 Pro, Arcgis Pro 2.6 concurrent license;
Issue:
I installed the deep learning tools following the guidelines provided here:
deeplearninginstallation
tersorflow was not found after installation so I manually installed the 2.1.0 version. I now have arcgis 1.8.2, pro 2.6, fastai 1.0.60, python 3.6.12, pytorch 1.4.0, tensorflow-gpu 2.1.0; environment check in arcgis pro python seemed fine.
However, after I select toolbox-image analyst-deeplearning-traindeeplearningmodel, the program seems to go into a hang, with most buttons disabled/unresponsive, this would continue until I force terminate the program. I also ran into "tool not licensed" twice, which was gone after I restarted the program; and a "name 'CallBackHandler' is not defined" once, which was also gone after I restarted.
I tried runing the command from the arcgis pro python prompt:
TrainDeepLearningModel(r"**", r"**", 40, "RETINANET", 16, "# #", None, "RESNET50", None, 10, "STOP_TRAINING", "FREEZE_MODEL")
executing the command would also send the program into a hang similar to the previous one. Monitor shows that ram and GPU usage haven't changed much, so I left the program running for an hour before forcibly terminating it.
I'd greatly appreciate it if anyone can tell me what the issues are here. I'll post any other env parameters if anyone requires. Cheers.
I got the tool up and running now by running conda install -c pytorch -c fastai fastai=1.0.54 pytorch=1.1.0 torchvision scikit-image and removing all the conflicting specifications in the cloned arcgispro-py3 env that I had. Now I still don't understand what went wrong. Presumably one or more packages in the env was conflicting. But seeing as I'm not a python expert, I couldn't identify the exact issue.
Before this I tried the versions stated here deeplearning install guide, but wasn't able to get pass tensorflow-gpu because python kept checking conflications. Now I actually don't have tensorflow-gpu in the env. I have tensorflow 2.1.0, keras-applications 1.0.8/base 2.3.1/preprocesing 1.1.0 (no keras-gpu), scikit-image 0.17.2, pillow 6.2.1, fastai 1.0.54, pytorch 1.1.0, libtiff 4.0.10. Some are different from what the guideline provided.
Thing is when I ran the process, CPU usage was up and GPU wasn't despite the fact that I specified GPU as the processing core. But I have much more pressing things to do right now like getting the analysis finished. So I'll probably tweek the env around a little after I'm done with this bit and see what happens. Meanwhile, anyone's input is still welcome.
I'm having an error running simple code using cv2 module.
It's just:
import cv2
img = cv2.imread('sudoku.png',0)
cv2.imshow('image',img)
And it fails with the following error:
QObject::moveToThread: Current thread (0x1b74720) is not the object's thread (0x1e57d70).
Cannot move to target thread (0x1b74720)
I googled this error and tried a lot of things but it doesn't help. I tried installing without pip, I tried using step-by-step installation (from official OpenCV) but nothing helps.
When I run:
cv2.__version__
It returns 3.4.3
As noted already, the basis for this problem is discussed in opencv-python issue 46, and results from the duplication of the following libraries both on the host and the opencv-python distro libQtDBus libQtCore and libQtGui.
It has been lately addressed in the newest release of opencv-python. It is not a fix to the source code, rather the fix is to force pip to compile the newly available source via
pip install --no-binary opencv-python opencv-python
This will cause opencv-python to use the same libraries as the host, so the conflict no longer exists.
According to this issue posted on the OpenCV GitHub, this is a known issue that the developer states is damn near impossible to fix. It is apparently caused by a conflict in any Qt installations on the system with the Qt that is shipped with OpenCV. There are some suggestions floating around to remove the libqt5x11extras5 package from the system. This may fix it for some but anyone running these libraries on a Linux distribution that uses a window manager based on Qt will render their desktop environment unusable by removing this package (having tried it myself).
You can try building OpenCV from source using the WITH_GTK=ON option when running cmake which will use GTK instead of Qt, circumventing the conflict. However, this is hard to make use of in Python when using virtual environments.
I haven't spent the time to fully appreciate this problem, but as I understand this is caused by multiple conflicting versions of some plugin in the environment. I tried installing building opencv-python but there were errors with that approach. Another suggestion is to change your import order, but I've had mixed success with that, and I couldn't get it to work on a project today.
But I found a workaround that worked for me. Install opencv-python-headless instead of opencv-python. This will avoid installing the conflicting plugins. It may not work for you depending on what features of opencv you need.
$ pip uninstall opencv-python
$ pip install opencv-python-headless
The error was fixed on my system, by simply updating one library.
To find out where it is coming from, assuming Linux, try the following,
LD_DEBUG=files python -c "import cv2"
or,
LD_DEBUG=files python -c "import cv2 ; img = cv2.imread('myimage.png',0) ; cv2.imshow('image',img) ; cv2.waitKey(0)"
On my machine, it failed in one of the blas libraries. I updated that library and the code now runs without error.
This is an old bug you can find it discussed in a number of online communities.
My test code is as follows. Notably the error has not reoccurred, and apparently has nothing to do with Qt.
import cv2
img = cv2.imread('sudoku.png',0)
cv2.imshow('image',img)
cv2.waitKey(0)
To me, the solution to this problem was removing Anaconda, then installing pip followed by installing OpenCV with a simple pip install command.
I have been trying to get Keras working on my laptop running El Capitan but when I attempt to import it I get the following message
Using TensorFlow backend.
Illegal instruction: 4
I've looked for solutions, and have tried updating theano, installing mxnet-mkl, and running an older version of numpy (1.13) to avoid a FutureWarning error. None of these has seemed to fix the issue, though. I feel like I must be missing something somewhere.
Try installing an older version of TensorFlow (1.5 seems to work). AVX2 instructions are now enabled by default in latest TF pip wheels, meaning if your processor doesn't support these instructions, then you will get an illegal instruction error.
Another solution is to install TensorFlow from source, which should detect your processor and configure it accordingly.
I am getting this error:
./google/protobuf/generated_message_util.h:86: const string&
google::protobuf::internal::GetEmptyStringAlreadyInited(): Assertion
`empty_string_ != __null' failed.
I have been trying to fix this all day long.
So the flow in my project goes like this:
I have a C++ project that I compile using CMake on Ubuntu 16.10. After some processing I want to send the data to a Python script that uses Keras for machine learning.
Now both of these scripts work perfectly independently. I wanted to send the data directly to the Python script. So I have embedded Python in C++ by referring this link
This is again working for a Python script that does not import keras.
Now when I import keras in this Python script, I get the above error, though both of them work properly independetly.
First I had got an error for the mismatch of libprotobuf and protoc versions which looked like:
Using TensorFlow backend.
[libprotobuf FATAL google/protobuf/stubs/common.cc:67] This program requires version 3.5.0 of the Protocol Buffer runtime library,
but the installed version is 3.0.0. Please update your library. If
you compiled the program yourself, make sure that your headers are
from the same version of Protocol Buffers as your link-time library.
(Version verification failed in "google/protobuf/descriptor.pb.cc".)
terminate called after throwing an instance of 'google::protobuf::FatalException'
what(): This program requires version 3.5.0 of the Protocol Buffer runtime library, but the installed version is 3.0.0. Please
update your library. If you compiled the program yourself, make sure
that your headers are from the same version of Protocol Buffers as
your link-time library. (Version verification failed in
"google/protobuf/descriptor.pb.cc".)
Aborted (core dumped)
I had installed protoc using apt-get (latest available on apt-get is 3.0.0) as I found out on one of the answers that you need not have protoc 3.5.0 for it to work. pip3 install protobuf installs protobuf 3.5.0 which resulted in the mismatch error message.
So I uninstalled this Python protobuf version and installed the older version using pip3 install protobuf==3.0.0. This removed the mismatch error message.
But now I am getting the aforementioned error message. I have also figured out that the error message is occuring due to Tensorflow dependency of Keras
How do I fix this?
Please help
I'm running Canopy 64-bit on a mac. After installing scikit-learn (using the package manager), I tried to import svm, and got the following error.
ImportError: dlopen(/Users/johnsaccount/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/sklearn/svm/liblinear.so, 2): Library not loaded: /System#rpath/libBLAS.dylib
Referenced from: /Users/johnsaccount/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/sklearn/svm/liblinear.so
Reason: image not found
Any ideas on what might be the problem?
Jsen,
My apologies for this error, it was my mistake when building that egg. The update should now be available and fix that error (the fixed version is 0.13.1-3, both 0.13.1-1 and 0.13.1-2 are broken on os x).
Thanks for the report and sorry for the inconvenience.
I can reproduce this, apparently due to a build error on OSX (it works ok on Windows). We'll let you know when it is fixed.