I am trying to save videos from multiple cameras connected to a Raspberry Pi using OpenCV. I have the Video saving working when there is a single camera without any multiprocessing. But when I try to have each camera on its own process nothing happens. I do not get any errors or log messages or anything, the process does not start.
Here is, what I think, the relevant code.
def init():
log(LOG_CODE_APP_START)
# video_capture(0) <-- works perfectly!
for index in range(0, 9):
camera_check = cv.VideoCapture(index)
if camera_check is None or not camera_check.isOpened():
continue
else:
camera_index.append(index)
print(camera_index) # <-- prints [0,1]
processes = []
for camera in camera_index:
video_process = multiprocessing.Process(name=str(camera), target=video_capture, kwargs={'camera': camera})
processes.append(video_process)
for process in processes:
print("Starting processes")
process.start()
def video_capture(camera=None):
start_time = time.time()
while True:
capture = cv.VideoCapture(camera)
capture.set(cv.CAP_PROP_FRAME_WIDTH, 800)
capture.set(cv.CAP_PROP_FRAME_HEIGHT, 600)
log(LOG_CODE_NEW_VIDEO)
file_name = 'camera_' + str(camera + 1) + '_' + '_' + get_time_string() + '_' + '.avi'
fourcc = cv.VideoWriter_fourcc(*'XVID')
height, width = get_video_stream_height_width(capture)
out = cv.VideoWriter(file_name, fourcc, 20.0, (width, height))
while int(time.time() - start_time) < CAPTURE_TIME:
ret, frame = capture.read()
if ret:
out.write(frame)
capture.release()
out.release()
start_time = time.time()
try:
if not os.path.exists(VIDEO_STORE):
os.mkdir(VIDEO_STORE)
shutil.move(file_name, VIDEO_STORE + '/' + file_name)
log(LOG_CODE_VIDEO_SAVED)
except Exception as e:
print("Exception")
log_error(e)
Change your script without multiprocessing to accept a camera number as a command-line argument.
Then try launching multiple instances of that script.
I have a surveillance program that opens 16 threads to check on to 16 different cameras with live video, but when after some time (20/30 seconds) cameras starts to lose frames and I need to reconnect them.
The thing is that if I only open 10 streams at once it might lose a frame from time to time, but if I open 16 streams it begins to fail every 20 seconds or so.
I've checked system params, and CPU doesnt go more than 50% and RAM is on 4GB, so this doesnt look like the problem.
Also I've checked the network and it's fine, no delay and no packet loss.
Sometime I get error while decoding MB 8 7, bytestream -5
This is the thread I use to get the streams.
"""
Class that continuously gets frames from a VideoCapture object
with a dedicated thread.
"""
def __init__(self, idcam,ipcamara, user, passwd,resolucionh,resolucionw):
self.id = idcam
self.ip = ipcamara
self.user = user
self.passwd = passwd
self.Q =[]
self.scale_percent = 50
self.i = 0
def start(self):
self.conectar()
Thread(target=self.get, args=()).start()
return self
def get(self):
while True:
if(self.stream.isOpened() and self.online):
(status, preFrame) = self.stream.read()
if(status):
self.frame = (preFrame)
else:
self.stream.release()
self.online = False
else:
graba_log('Camara '+str(self.id),'Desconexion')
self.conectar()
time.sleep(1)
def conectar(self):
print('Iniciando conexion con camara '+str(self.id))
self.stream = cv2.VideoCapture("rtsp://"+self.user+":"+self.passwd+"#"+self.ip)
self.online = True
def recuperarFrame(self):
return self.frame ```
I have to stitch the images captured from many (9) cameras. Initially, I tried to capture the frames from 2 cameras with rate 15 FPS. Then, I connected 4 cameras (I also used externally powered USB hub to provide enough power) but I could only see only one stream.
For testing, I used the following script:
import numpy as np
import cv2
import imutils
index = 0
arr = []
while True:
cap = cv2.VideoCapture(index)
if not cap.read()[0]:
break
else:
arr.append(index)
cap.release()
index += 1
video_captures = [cv2.VideoCapture(idx) for idx in arr]
while True:
# Capture frame-by-frame
frames = []
frames_preview = []
for i in arr:
# skip webcam capture
if i == 1: continue
ret, frame = video_captures[i].read()
if ret:
frames.append(frame)
small = cv2.resize(frame, (0, 0), fx=0.25, fy=0.25)
frames_preview.append(small)
for i, frame in enumerate(frames_preview):
cv2.imshow('Cam {}'.format(i), frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
# When everything is done, release the capture
for video_capture in video_captures:
video_capture.release()
cv2.destroyAllWindows()
Is there any limit for the number of cameras? Does anyone know what is the right way to capture frames from multiple cameras?
To capture multiple streams with OpenCV, I recommend using threading which can improve performance by alleviating the heavy I/O operations to a separate thread. Since accessing the webcam/IP/RTSP stream using cv2.VideoCapture().read() is a blocking operation, our main program is stuck until the frame is read from the camera device. If you have multiple streams, this latency will definitely be visible. To remedy this problem, we can use threading to spawn another thread to handle retrieving the frames using a deque in parallel instead of relying on a single thread to obtain the frames in sequential order. Threading allows frames to be continuously read without impacting the performance of our main program. The idea to capture a single stream using threading and OpenCV, is from a previous answer in Python OpenCV multithreading streaming from camera.
But if you want to capture multiple streams, OpenCV alone is not enough. You can use OpenCV in combination with a GUI framework to stitch each image onto a nice display. I will use PyQt4 as the framework, qdarkstyle for GUI CSS, and imutils for OpenCV convenience functions.
Here is a very stripped down version of the camera GUI I currently use without the placeholder images, credential admin login page, and camera switching ability. I've kept the automatic camera reconnect feature incase the internet dies or the camera connection is lost. I only have 8 cameras as shown in the image above, but it is very simple to add in another camera and should not impact performance. This camera GUI currently performs at about ~60 FPS so it is real-time. You can easily rearrange the layout using PyQt layouts so feel free to modify the code! Remember to change the stream links!
from PyQt4 import QtCore, QtGui
import qdarkstyle
from threading import Thread
from collections import deque
from datetime import datetime
import time
import sys
import cv2
import imutils
class CameraWidget(QtGui.QWidget):
"""Independent camera feed
Uses threading to grab IP camera frames in the background
#param width - Width of the video frame
#param height - Height of the video frame
#param stream_link - IP/RTSP/Webcam link
#param aspect_ratio - Whether to maintain frame aspect ratio or force into fraame
"""
def __init__(self, width, height, stream_link=0, aspect_ratio=False, parent=None, deque_size=1):
super(CameraWidget, self).__init__(parent)
# Initialize deque used to store frames read from the stream
self.deque = deque(maxlen=deque_size)
# Slight offset is needed since PyQt layouts have a built in padding
# So add offset to counter the padding
self.offset = 16
self.screen_width = width - self.offset
self.screen_height = height - self.offset
self.maintain_aspect_ratio = aspect_ratio
self.camera_stream_link = stream_link
# Flag to check if camera is valid/working
self.online = False
self.capture = None
self.video_frame = QtGui.QLabel()
self.load_network_stream()
# Start background frame grabbing
self.get_frame_thread = Thread(target=self.get_frame, args=())
self.get_frame_thread.daemon = True
self.get_frame_thread.start()
# Periodically set video frame to display
self.timer = QtCore.QTimer()
self.timer.timeout.connect(self.set_frame)
self.timer.start(.5)
print('Started camera: {}'.format(self.camera_stream_link))
def load_network_stream(self):
"""Verifies stream link and open new stream if valid"""
def load_network_stream_thread():
if self.verify_network_stream(self.camera_stream_link):
self.capture = cv2.VideoCapture(self.camera_stream_link)
self.online = True
self.load_stream_thread = Thread(target=load_network_stream_thread, args=())
self.load_stream_thread.daemon = True
self.load_stream_thread.start()
def verify_network_stream(self, link):
"""Attempts to receive a frame from given link"""
cap = cv2.VideoCapture(link)
if not cap.isOpened():
return False
cap.release()
return True
def get_frame(self):
"""Reads frame, resizes, and converts image to pixmap"""
while True:
try:
if self.capture.isOpened() and self.online:
# Read next frame from stream and insert into deque
status, frame = self.capture.read()
if status:
self.deque.append(frame)
else:
self.capture.release()
self.online = False
else:
# Attempt to reconnect
print('attempting to reconnect', self.camera_stream_link)
self.load_network_stream()
self.spin(2)
self.spin(.001)
except AttributeError:
pass
def spin(self, seconds):
"""Pause for set amount of seconds, replaces time.sleep so program doesnt stall"""
time_end = time.time() + seconds
while time.time() < time_end:
QtGui.QApplication.processEvents()
def set_frame(self):
"""Sets pixmap image to video frame"""
if not self.online:
self.spin(1)
return
if self.deque and self.online:
# Grab latest frame
frame = self.deque[-1]
# Keep frame aspect ratio
if self.maintain_aspect_ratio:
self.frame = imutils.resize(frame, width=self.screen_width)
# Force resize
else:
self.frame = cv2.resize(frame, (self.screen_width, self.screen_height))
# Add timestamp to cameras
cv2.rectangle(self.frame, (self.screen_width-190,0), (self.screen_width,50), color=(0,0,0), thickness=-1)
cv2.putText(self.frame, datetime.now().strftime('%H:%M:%S'), (self.screen_width-185,37), cv2.FONT_HERSHEY_SIMPLEX, 1.2, (255,255,255), lineType=cv2.LINE_AA)
# Convert to pixmap and set to video frame
self.img = QtGui.QImage(self.frame, self.frame.shape[1], self.frame.shape[0], QtGui.QImage.Format_RGB888).rgbSwapped()
self.pix = QtGui.QPixmap.fromImage(self.img)
self.video_frame.setPixmap(self.pix)
def get_video_frame(self):
return self.video_frame
def exit_application():
"""Exit program event handler"""
sys.exit(1)
if __name__ == '__main__':
# Create main application window
app = QtGui.QApplication([])
app.setStyleSheet(qdarkstyle.load_stylesheet_pyqt())
app.setStyle(QtGui.QStyleFactory.create("Cleanlooks"))
mw = QtGui.QMainWindow()
mw.setWindowTitle('Camera GUI')
mw.setWindowFlags(QtCore.Qt.FramelessWindowHint)
cw = QtGui.QWidget()
ml = QtGui.QGridLayout()
cw.setLayout(ml)
mw.setCentralWidget(cw)
mw.showMaximized()
# Dynamically determine screen width/height
screen_width = QtGui.QApplication.desktop().screenGeometry().width()
screen_height = QtGui.QApplication.desktop().screenGeometry().height()
# Create Camera Widgets
username = 'Your camera username!'
password = 'Your camera password!'
# Stream links
camera0 = 'rtsp://{}:{}#192.168.1.43:554/cam/realmonitor?channel=1&subtype=0'.format(username, password)
camera1 = 'rtsp://{}:{}#192.168.1.45/axis-media/media.amp'.format(username, password)
camera2 = 'rtsp://{}:{}#192.168.1.47:554/cam/realmonitor?channel=1&subtype=0'.format(username, password)
camera3 = 'rtsp://{}:{}#192.168.1.40:554/cam/realmonitor?channel=1&subtype=0'.format(username, password)
camera4 = 'rtsp://{}:{}#192.168.1.44:554/cam/realmonitor?channel=1&subtype=0'.format(username, password)
camera5 = 'rtsp://{}:{}#192.168.1.42:554/cam/realmonitor?channel=1&subtype=0'.format(username, password)
camera6 = 'rtsp://{}:{}#192.168.1.46:554/cam/realmonitor?channel=1&subtype=0'.format(username, password)
camera7 = 'rtsp://{}:{}#192.168.1.41:554/cam/realmonitor?channel=1&subtype=0'.format(username, password)
# Create camera widgets
print('Creating Camera Widgets...')
zero = CameraWidget(screen_width//3, screen_height//3, camera0)
one = CameraWidget(screen_width//3, screen_height//3, camera1)
two = CameraWidget(screen_width//3, screen_height//3, camera2)
three = CameraWidget(screen_width//3, screen_height//3, camera3)
four = CameraWidget(screen_width//3, screen_height//3, camera4)
five = CameraWidget(screen_width//3, screen_height//3, camera5)
six = CameraWidget(screen_width//3, screen_height//3, camera6)
seven = CameraWidget(screen_width//3, screen_height//3, camera7)
# Add widgets to layout
print('Adding widgets to layout...')
ml.addWidget(zero.get_video_frame(),0,0,1,1)
ml.addWidget(one.get_video_frame(),0,1,1,1)
ml.addWidget(two.get_video_frame(),0,2,1,1)
ml.addWidget(three.get_video_frame(),1,0,1,1)
ml.addWidget(four.get_video_frame(),1,1,1,1)
ml.addWidget(five.get_video_frame(),1,2,1,1)
ml.addWidget(six.get_video_frame(),2,0,1,1)
ml.addWidget(seven.get_video_frame(),2,1,1,1)
print('Verifying camera credentials...')
mw.show()
QtGui.QShortcut(QtGui.QKeySequence('Ctrl+Q'), mw, exit_application)
if(sys.flags.interactive != 1) or not hasattr(QtCore, 'PYQT_VERSION'):
QtGui.QApplication.instance().exec_()
Related camera/IP/RTSP, FPS, video, threading, and multiprocessing posts
Python OpenCV streaming from camera - multithreading, timestamps
Video Streaming from IP Camera in Python Using OpenCV cv2.VideoCapture
How to capture multiple camera streams with OpenCV?
OpenCV real time streaming video capture is slow. How to drop frames or get synced with real time?
Storing RTSP stream as video file with OpenCV VideoWriter
OpenCV video saving
Python OpenCV multiprocessing cv2.VideoCapture mp4
I'm trying to make a python gui that has 2 buttons, one for recording audio and one to stop recording. the code runs on a raspberry pi 3 with a usb mic connected already.
I've done the recording button, and now I'm stuck on the stop recording button.
the recording button runs in a while loop and I need that the 'stop' button will simply stop the while loop and save the recording to a file.
from sense_hat import SenseHat
import time
import Tkinter
import ttk
import pyaudio
import wave
sense = SenseHat()
sense.clear()
snsui = Tkinter.Tk()
snsui.style = ttk.Style()
snsui.style.theme_use("classic")
snsui.geometry('130x150')
snsui.resizable(0,0)
#img = PhotoImage(file='/home/pi/sensehat/fav.ico')
#snsui.Tkinter.call('wm', 'iconphoto', root._w, img)
#snsui.iconbitmap(default = "/home/pi/sensehat/image.png")
snsui.title("Sensors UI")
snsui.configure(background='light gray')
snsui.style.configure("TButton", padding=6, relief="flat",
background="turquoise",fieldbackground="black", foreground='royal blue')
form_1 = pyaudio.paInt16 # 16-bit resolution
chans = 1 # 1 channel
samp_rate = 44100 # 44.1kHz sampling rate
chunk = 4096 # 2^12 samples for buffer
record_secs = 3 # seconds to record
dev_index = 2 # device index found by p.get_device_info_by_index(ii)
audio = pyaudio.PyAudio() # create pyaudio instantiation
# create pyaudio stream
stream = audio.open(format = form_1,rate = samp_rate,channels = chans, \
input_device_index = dev_index,input = True, \
frames_per_buffer=chunk)
recording = True
def record(recording,s,a,c,ch,sr):
if recording == False:
recording = True
print('start recording')
else:
recording = False
print('trying to stop')
frames = []
# loop through stream and append audio chunks to frame array
# ii in range(0,int((samp_rate/chunk)*record_secs)):
while recording:
data = stream.read(chunk)
frames.append(data)
print("finished recording")
# stop the stream, close it, and terminate the pyaudio instantiation
stream.stop_stream()
stream.close()
audio.terminate()
wav_output_filename = 'test1.wav' # name of .wav file
# save the audio frames as .wav file
wavefile = wave.open(wav_output_filename,'wb')
wavefile.setnchannels(chans)
wavefile.setsampwidth(audio.get_sample_size(form_1))
wavefile.setframerate(samp_rate)
wavefile.writeframes(b''.join(frames))
wavefile.close()
B_recording = ttk.Button(snsui, text =" Record ", command = record(recording,stream,audio,chunk,chans,samp_rate))
B_recording.pack()
snsui.mainloop()
You can create boolean flag which value is changed by stop button. Then just add if statement to your while loop, and break when the flag have desired value.
I am doing a graphics project and have to use pixel plotting function only. I came across pygame but it doesnt seem to provide its calls to be used in threads. Please suggest me workaround this or any other python library that can be used with threads.
#! /usr/bin/env python
import threading
import time
class plotterthread(threading.Thread):
def __init__(self,screen,queue):
threading.Thread.__init__(self)
self.screen = screen
self.__queue__ = queue
self._colorset = False
self.kill = False
def setcolor(self,r,g,b):
self._r = r
self._g = g
self._b = b
self._colorset = True
def run(self):
try:
while not self.__queue__.empty() and not self.kill:
(x,y) = self.__queue__.get(timeout=1)
if not self._colorset:
#this is where error occures
#this calls pygame.set_at() method
self.screen.plotpixel((x,y))
else:
self.screen.plotpixel((x,y),(self._r,self._g,self._b))
except:
pass
class plotter:
def __init__(self,screen,min,max,queue,ppt=50):
self.screen = screen
self.min = min
self.max = max
self.ppt = ppt # pixels per thread
self.coords = queue
def plot(self):
noOfcoords = self.coords.qsize()
noOfthreads = int(noOfcoords/self.ppt)
if noOfthreads > self.max:
noOfthreads = self.max
elif noOfthreads < self.min:
noOfthreads = self.min
self.threadpool = []
for i in range(noOfthreads):
pthread = plotterthread(screen = self.screen,queue = self.coords)
self.threadpool.append(pthread)
pthread.start()
while True:
try:
livethreads = 0
for t in self.threadpool:
if t is not None and t.isAlive():
livethreads += 1
t.join(timeout=1)
if livethreads == 0:
break
except KeyboardInterrupt:
for threads in self.threadpool:
if threads is not None and threads.isAlive():
threads.kill = True
break
Here, screen is the instance of the window on which I am trying to draw.
queue is the Queue which contains all the calculated coordinates to be plotted.
All I wanted to do was first calculate all the pixels and push them into a queue and then share the queue with threads to plot them in parallel to speed up the process.