I'm trying to embed python in a c/c++ application, using the embedded python zip file. I also want to use site-packages, like numpy. I manipulate the sys path myself, so it'll include all paths necessary.
What I have found out for now is:
If I use Py_Initialize/Py_Finalize more than once, my application will crash the 2nd time I run a script when using numpy, due to numpy being imported twice during the same process and Py_Finalize doesn't clean up the mess numpy leaves behind.
If I use sub interpreters, the same happens, because, again, numpy gets imported each time "import" is called in a sup interpreter.
If I use a single interpreter (calling Py_Initialize at application start and then calling Py_Finalize at application shutdown), Py_RunFile() will run smoothly as long as I do not use site-packages. If I do import a site-package (e.g. import numpy) Py_RunFile will hang, though. The application will work, because the python interpreter is running in its own thread, but the interpreter itself will hang.
I know that the site-package *.pyd files are loaded:
'xxx.exe' (Win32): Loaded 'C:\tools\python37x64\DLLs\_ctypes.pyd'. Symbols loaded.
'xxx.exe' (Win32): Loaded 'C:\tools\python37x64\Lib\site-packages\numpy\.libs\libopenblas.TXA6YQSD3GCQQC22GEQ54J2UDCXDXHWN.gfortran-win_amd64.dll'. Module was built without symbols.
'xxx.exe' (Win32): Loaded 'C:\tools\python37x64\Lib\site-packages\numpy\core\_multiarray_umath.cp37-win_amd64.pyd'. Module was built without symbols.
'xxx.exe' (Win32): Loaded 'C:\tools\python37x64\Lib\site-packages\numpy\core\_multiarray_tests.cp37-win_amd64.pyd'. Module was built without symbols.
'xxx.exe' (Win32): Loaded 'C:\tools\python37x64\Lib\site-packages\numpy\linalg\lapack_lite.cp37-win_amd64.pyd'. Module was built without symbols.
'xxx.exe' (Win32): Loaded 'C:\tools\python37x64\Lib\site-packages\numpy\linalg\_umath_linalg.cp37-win_amd64.pyd'. Module was built without symbols.
'xxx.exe' (Win32): Loaded 'C:\tools\python37x64\DLLs\_bz2.pyd'. Symbols loaded.
'xxx.exe' (Win32): Loaded 'C:\tools\python37x64\DLLs\_lzma.pyd'. Symbols loaded.
'xxx.exe' (Win32): Loaded 'C:\tools\python37x64\DLLs\_decimal.pyd'. Symbols loaded.
'xxx.exe' (Win32): Loaded 'C:\tools\python37x64\Lib\site-packages\numpy\fft\fftpack_lite.cp37-win_amd64.pyd'. Module was built without symbols.
'xxx.exe' (Win32): Loaded 'C:\tools\python37x64\Lib\site-packages\numpy\random\mtrand.cp37-win_amd64.pyd'. Module was built without symbols.
but then nothing happens.
So, can anybody tell me what I'm doing wrong that my interpreter hangs when I try to use site-packages?
edit:
I don't call any other python specific functions, other than those to manipulate sys.path and Py_Initialize
edit 2:
I have found this discussion.
But that guy had the problem that the program would hang if he used sub interpreters. But using sub interpreters for me works once and then it crashes (as mentioned above). It seems, though, as if the same line is leading to a deadlock here:
python37.dll!_PyCOND_WAIT_MS(_PyCOND_T * cv, _RTL_CRITICAL_SECTION * cs, unsigned long ms) Line 187 C
python37.dll!take_gil(_ts * tstate) Line 208 C
python37.dll!PyEval_RestoreThread(_ts * tstate) Line 273 C
python37.dll!PyGILState_Ensure() Line 1067 C
_multiarray_umath.cp37-win_amd64.pyd!00007ffdab6ee940() Unknown
Related
I've created a script in python that is using PyQt5. Now everything works when I'm launching the file from my editor - Studio Code in this case.
I'm having a problem when I try to deploy an exe using PyInstaller or py2exe it gives me error on missing dll's when it's building. It still finishes building anyway, but when I try to run the exe file the cmd window just flashes for a brief moment and nothing more happens.
I suppose this is due to missing dll's and if not than I have to sort this out first anyway.
I've tried searching for the dll's I'm missing on my computer and some I couldn't find at all e.g. Qt53DInput and one I could find in what I suppose is application made in Qt - Qt5Multimedia .
I'm currently using python 3.7.4 which I've already tried reinstalling. I think I didn't try to reinstall PyQt5, should I try doing that?
from PyQt5 import QtWidgets, uic, QtGui, QtCore, QtQuick
import sys
import os.path
import datetime
import shutil
I guess the only important part of my script in this case are the imports, so I've included them here.
python -m PyInstaller --paths C:\Users\bonana\AppData\Local\Programs\Python\Python37-32\Lib\site-packages\PyQt5\Qt\bin my_code.py
Somewhere I've read that it's good idea to include this path, but since the dll are not to be found anywhere, including this folder, then obviously didn't help at all.
This is the warning about the dll's I'm getting:
Looking for dynamic libraries
32805 WARNING: lib not found: Qt5MultimediaQuick.dll dependency of C:\Users\bonana\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PyQt5\Qt\qml\QtMultimedia\declarative_multimedia.dll
58554 WARNING: lib not found: Qt53DInput.dll dependency of C:\Users\bonana\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PyQt5\Qt\qml\QtQuick\Scene3D\qtquickscene3dplugin.dll
58844 WARNING: lib not found: Qt53DAnimation.dll dependency of C:\Users\bonana\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PyQt5\Qt\qml\QtQuick\Scene3D\qtquickscene3dplugin.dll
58989 WARNING: lib not found: Qt53DRender.dll dependency of C:\Users\bonana\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PyQt5\Qt\qml\QtQuick\Scene3D\qtquickscene3dplugin.dll
59284 WARNING: lib not found: Qt53DLogic.dll dependency of C:\Users\bonana\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PyQt5\Qt\qml\QtQuick\Scene3D\qtquickscene3dplugin.dll
59440 WARNING: lib not found: Qt53DCore.dll dependency of C:\Users\bonana\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PyQt5\Qt\qml\QtQuick\Scene3D\qtquickscene3dplugin.dll
60365 WARNING: lib not found: Qt53DQuickScene2D.dll dependency of C:\Users\bonana\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PyQt5\Qt\qml\QtQuick\Scene2D\qtquickscene2dplugin.dll
60680 WARNING: lib not found: Qt53DRender.dll dependency of C:\Users\bonana\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PyQt5\Qt\qml\QtQuick\Scene2D\qtquickscene2dplugin.dll
60992 WARNING: lib not found: Qt53DCore.dll dependency of C:\Users\bonana\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PyQt5\Qt\qml\QtQuick\Scene2D\qtquickscene2dplugin.dll
84256 WARNING: lib not found: api-ms-win-core-winrt-l1-1-0.dll dependency of C:\Users\bonana\AppData\Local\Programs\Python\Python37-32\Lib\site-packages\PyQt5\Qt\bin\qt5bluetooth.dll
84545 WARNING: lib not found: api-ms-win-core-winrt-string-l1-1-0.dll dependency of C:\Users\bonana\AppData\Local\Programs\Python\Python37-32\Lib\site-packages\PyQt5\Qt\bin\qt5bluetooth.dll
Maybe it's not complete answear to why PyInstaller or py2exe is not working, but I've found way to deploy my exe properly. I've used fman build system. It didn't work right off the bat though. I had to install windows 10 sdk and make small change to my code:
QFileDialog.getOpenFileName(appctxt)
This no longer worked, because for some reason it needed to be changed to:
QtWidgets.QFileDialog.getOpenFileName(appctxt)
Other than that everything including creation of an installer worked without a problem.
I am trying to create executable python file using pyinstaller, but while loading hooks, it shows error like this,
24021 INFO: Removing import of PySide from module PIL.ImageQt
24021 INFO: Loading module hook "hook-pytz.py"...
24506 INFO: Loading module hook "hook-encodings.py"...
24600 INFO: Loading module hook "hook-pandas.py"...
25037 INFO: Loading module hook "hook-lib2to3.py"...
25131 INFO: Loading module hook "hook-lxml.etree.py"...
25131 INFO: Loading module hook "hook-pycparser.py"...
25396 INFO: Loading module hook "hook-setuptools.py"...
25506 WARNING: Hidden import "setuptools.msvc" not found!
25506 INFO: Loading module hook "hook-distutils.py"...
25521 INFO: Loading module hook "hook-nltk.py"...
Unable to find "C:\nltk_data" when adding binary and data files.
I have tried coping nltk_data from Appdata to C drive. but same error.
This answer worked for me... it modifies the code in hook-nltk.py to only include the path if it exists.
hook-nltk.py can be found in your PyInstaller location within the hooks folder (something like <'path-to-python-installation'>\Lib\site-packages\PyInstaller\hooks)
I have been working on this issue for a few days not and don't have hair left. For some reason nltk and pyinstaller do not work well together.
So my first solution to this issue is to use something other than nltk if it is possible to code the solution without nltk.
If you must use NLTK, I solved this by forcing the nltk_data path into datas.
Locate your nltk_data path. Mine was in
C:\Users\user-name\AppData\Roaming\nltk_data
In hook-nltk.py (within pyinstaller directory) I commented out and added lines to look like this.
import nltk
from PyInstaller.utils.hooks import collect_data_files
datas = collect_data_files('nltk', False)
'''
for p in nltk.data.path:
datas.append((p, "nltk_data"))
'''
datas.append(("C:\\Users\\nedhu\\AppData\\Roaming\\nltk_data", "nltk_data"))
hiddenimports = ["nltk.chunk.named_entity"]
There is a deeper problem with pyinstaller looping through the datas list of paths, but this solution works as a patch.
I solved the problems editing the pyinstaller nltk-hook. After much research, I decided to go it alone in the code structure. I solved my problem by commenting on the lines:
datas=[]
'''for p in nltk.data.path:
datas.append((p, "nltk_data"))'''
hiddenimports = ["nltk.chunk.named_entity"]
What's more, you need to rename the file: pyi_rth__nltk.cpython-36.pyc to pyi_rth_nltk.cpython-36.pyc . This file have 1 more underline. Warning with the python version.
I know that questions have already been raised on this topic, but I believe that all decisions were strictly individual. The essence of the problem:
There is a calculator.py script and a kalkulator.py These files are in the same directory, the calculator file is executable, this is the file that I convert into an EXE using PyInstaller. Inside the file, I import several PyQt5 libraries, and the second file is Python: from kalkulator import *
The second file kalkulator.py contains the GUI of the program created with QtDesigner
When i run the command pyinstaller --onefile --debug calculator.py everything goes fine.
But when I run the EXE program file in debug mode, I see the following error:
The photo shows that the following error occurs.
"DLL Load Failed: The specified procedure could not be found."
I can assume that the problem is that when assembling, PyInstaller does not see the file kalkulator.py it contains a graphical interface.
What can you say about this problem, how do you think you can solve it?
I use:
Python 3.5
PyQt5
PyInstaller 3.3.1
I managed to solve my problem! I changed the version of Python 3.5.0 to 3.6.0, and also reinstalled PyQt5.
I have successfully build sphinxbase. When I tried compiling sphinxtrain, it says
"The program can't start because sphinxbase.dll is missing from your computer. try reinstalling the program"
I checked sphinxtrain/bin/release and saw that I have sphinxbase.dll in that folder. What could be the problem?
This is whats showing in the output window
'agg_seg.exe' (Win32): Loaded 'D:\Sphinx\SphinxTrain\bin\Debug\agg_seg.exe'. Symbols loaded.
'agg_seg.exe' (Win32): Loaded 'C:\Windows\SysWOW64\ntdll.dll'. Cannot find or open the PDB file.
'agg_seg.exe' (Win32): Loaded 'C:\Windows\SysWOW64\kernel32.dll'. Cannot find or open the PDB file.
'agg_seg.exe' (Win32): Loaded 'C:\Windows\SysWOW64\KernelBase.dll'. Cannot find or open the PDB file.
'agg_seg.exe' (Win32): Loaded 'D:\Sphinx\SphinxTrain\bin\Debug\sphinxbase.dll'. Symbols loaded.
'agg_seg.exe' (Win32): Loaded 'C:\Windows\SysWOW64\winmm.dll'. Cannot find or open the PDB file.
'agg_seg.exe' (Win32): Loaded 'C:\Windows\SysWOW64\msvcrt.dll'. Cannot find or open the PDB file.
'agg_seg.exe' (Win32): Loaded 'C:\Windows\SysWOW64\user32.dll'. Cannot find or open the PDB file.
'agg_seg.exe' (Win32): Loaded 'C:\Windows\SysWOW64\gdi32.dll'. Cannot find or open the PDB file.
'agg_seg.exe' (Win32): Loaded 'C:\Windows\SysWOW64\lpk.dll'. Cannot find or open the PDB file.
'agg_seg.exe' (Win32): Loaded 'C:\Windows\SysWOW64\usp10.dll'. Cannot find or open the PDB file.
'agg_seg.exe' (Win32): Loaded 'C:\Windows\SysWOW64\advapi32.dll'. Cannot find or open the PDB file.
'agg_seg.exe' (Win32): Loaded 'C:\Windows\SysWOW64\sechost.dll'. Cannot find or open the PDB file.
'agg_seg.exe' (Win32): Loaded 'C:\Windows\SysWOW64\rpcrt4.dll'. Cannot find or open the PDB file.
'agg_seg.exe' (Win32): Loaded 'C:\Windows\SysWOW64\sspicli.dll'. Cannot find or open the PDB file.
'agg_seg.exe' (Win32): Loaded 'C:\Windows\SysWOW64\cryptbase.dll'. Cannot find or open the PDB file.
'agg_seg.exe' (Win32): Loaded 'C:\Windows\SysWOW64\msvcr110d.dll'. Symbols loaded.
'agg_seg.exe' (Win32): Loaded 'C:\Windows\SysWOW64\imm32.dll'. Cannot find or open the PDB file.
'agg_seg.exe' (Win32): Loaded 'C:\Windows\SysWOW64\msctf.dll'. Cannot find or open the PDB file.
'agg_seg.exe' (Win32): Loaded 'C:\Windows\SysWOW64\nvinit.dll'. Cannot find or open the PDB file.
The program '[6444] agg_seg.exe' has exited with code -1 (0xffffffff).
If you want to run binary from Debug folder, you need to copy debug version of sphinxbase.dll from sphinxbase build sphinxbase/bin/Debug to that sphinxtrain/bin/Debug folder. dll must be in the current folder.
I'm getting this error when running my app from Visual Studio 2012.
Development Specs:
Cuda 5.0
VS C++ 2012
Quadro K4000
I've seen there are other posts related here, here and here but none has a good answer to my problem. I'm checking all the return codes from CUDA API, CuRAND and CuBLAS and all are returning SUCCESS. I looked in the code where was the problem raising and it happens to be when I create a timer object for the GPU.
I'm using the timer that comes with CUDA SDK
struct GpuTimer{
cudaEvent_t start;
cudaEvent_t stop;
GpuTimer(){
checkCudaErr(
cudaEventCreate(&start)
);
checkCudaErr(
cudaEventCreate(&stop)
);
}
~GpuTimer(){
checkCudaErr(
cudaEventDestroy(start)
);
checkCudaErr(
cudaEventDestroy(stop)
);
}
void Start(){
checkCudaErr(
cudaEventRecord(start, 0)
);
}
void Stop(){
checkCudaErr(
cudaEventRecord(stop, 0)
);
}
float Elapsed(){
float elapsed;
checkCudaErr(
cudaEventSynchronize(stop)
);
checkCudaErr(
cudaEventElapsedTime(&elapsed, start, stop)
);
return elapsed;
}
};
so in the main function I have
int main(args)
{
...
...
...
GpuTimer t;
...
...
}
and exactly after that line is executed I get
'App.exe' (Win32): Loaded 'C:\Windows\System32\nvcuda.dll'. Module was built without symbols.
'App.exe' (Win32): Loaded 'C:\Windows\System32\user32.dll'. Cannot find or open the PDB file.
'App.exe' (Win32): Loaded 'C:\Windows\System32\gdi32.dll'. Cannot find or open the PDB file.
'App.exe' (Win32): Loaded 'C:\Windows\System32\lpk.dll'. Cannot find or open the PDB file.
'App.exe' (Win32): Loaded 'C:\Windows\System32\usp10.dll'. Cannot find or open the PDB file.
'App.exe' (Win32): Loaded 'C:\Windows\System32\msvcrt.dll'. Cannot find or open the PDB file.
'App.exe' (Win32): Loaded 'C:\Windows\System32\setupapi.dll'. Cannot find or open the PDB file.
'App.exe' (Win32): Loaded 'C:\Windows\System32\cfgmgr32.dll'. Cannot find or open the PDB file.
'App.exe' (Win32): Loaded 'C:\Windows\System32\rpcrt4.dll'. Cannot find or open the PDB file.
'App.exe' (Win32): Loaded 'C:\Windows\System32\advapi32.dll'. Cannot find or open the PDB file.
'App.exe' (Win32): Loaded 'C:\Windows\System32\sechost.dll'. Cannot find or open the PDB file.
'App.exe' (Win32): Loaded 'C:\Windows\System32\oleaut32.dll'. Cannot find or open the PDB file.
'App.exe' (Win32): Loaded 'C:\Windows\System32\ole32.dll'. Cannot find or open the PDB file.
'App.exe' (Win32): Loaded 'C:\Windows\System32\devobj.dll'. Cannot find or open the PDB file.
'App.exe' (Win32): Loaded 'C:\Windows\System32\shell32.dll'. Cannot find or open the PDB file.
'App.exe' (Win32): Loaded 'C:\Windows\System32\shlwapi.dll'. Cannot find or open the PDB file.
'App.exe' (Win32): Loaded 'C:\Windows\System32\imm32.dll'. Cannot find or open the PDB file.
'App.exe' (Win32): Loaded 'C:\Windows\System32\msctf.dll'. Cannot find or open the PDB file.
'App.exe' (Win32): Loaded 'C:\Windows\System32\dwmapi.dll'. Cannot find or open the PDB file.
'App.exe' (Win32): Unloaded 'C:\Windows\System32\dwmapi.dll'
'App.exe' (Win32): Loaded 'C:\Windows\System32\nvapi64.dll'. Cannot find or open the PDB file.
'App.exe' (Win32): Loaded 'C:\Windows\System32\version.dll'. Cannot find or open the PDB file.
'App.exe' (Win32): Loaded 'C:\Windows\System32\wintrust.dll'. Cannot find or open the PDB file.
'App.exe' (Win32): Loaded 'C:\Windows\System32\crypt32.dll'. Cannot find or open the PDB file.
'App.exe' (Win32): Loaded 'C:\Windows\System32\msasn1.dll'. Cannot find or open the PDB file.
First-chance exception at 0x000007FEFDA29E5D in App.exe: Microsoft C++ exception: cudaError_enum at memory location 0x000000000018EA00. //Repeated 20 times at least
finally, just after launching the application I saw this messages in the output console (on VS2012)
'App.exe' (Win32): Loaded 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v5.0\bin\cudart64_50_35.dll'. Module was built without symbols.
'App.exe' (Win32): Loaded 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v5.0\bin\cublas64_50_35.dll'. Module was built without symbols.
'App.exe' (Win32): Loaded 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v5.0\bin\curand64_50_35.dll'. Module was built without symbols.
The application runs just fine and results look fine to me but I would like to know what could be causing this errors/exceptions and how to solve them, or if I should just ignore them.
The observation you are making has to do with an exception that is caught and handled properly within the CUDA libraries. It is, in some cases, a normal part of CUDA GPU operation. As you have observed, your application returns no API errors and runs correctly. If you were not within the VS environment that can report this, you would not observe this at all.
This is considered normal behavior under CUDA 5.0. I believe there were some attempts to eliminate it in CUDA 5.5. You might wish to try that, although it's not considered an issue either way.
You might also be interested in this question/answer.