I have to stitch the images captured from many (9) cameras. Initially, I tried to capture the frames from 2 cameras with rate 15 FPS. Then, I connected 4 cameras (I also used externally powered USB hub to provide enough power) but I could only see only one stream.
For testing, I used the following script:
import numpy as np
import cv2
import imutils
index = 0
arr = []
while True:
cap = cv2.VideoCapture(index)
if not cap.read()[0]:
break
else:
arr.append(index)
cap.release()
index += 1
video_captures = [cv2.VideoCapture(idx) for idx in arr]
while True:
# Capture frame-by-frame
frames = []
frames_preview = []
for i in arr:
# skip webcam capture
if i == 1: continue
ret, frame = video_captures[i].read()
if ret:
frames.append(frame)
small = cv2.resize(frame, (0, 0), fx=0.25, fy=0.25)
frames_preview.append(small)
for i, frame in enumerate(frames_preview):
cv2.imshow('Cam {}'.format(i), frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
# When everything is done, release the capture
for video_capture in video_captures:
video_capture.release()
cv2.destroyAllWindows()
Is there any limit for the number of cameras? Does anyone know what is the right way to capture frames from multiple cameras?
To capture multiple streams with OpenCV, I recommend using threading which can improve performance by alleviating the heavy I/O operations to a separate thread. Since accessing the webcam/IP/RTSP stream using cv2.VideoCapture().read() is a blocking operation, our main program is stuck until the frame is read from the camera device. If you have multiple streams, this latency will definitely be visible. To remedy this problem, we can use threading to spawn another thread to handle retrieving the frames using a deque in parallel instead of relying on a single thread to obtain the frames in sequential order. Threading allows frames to be continuously read without impacting the performance of our main program. The idea to capture a single stream using threading and OpenCV, is from a previous answer in Python OpenCV multithreading streaming from camera.
But if you want to capture multiple streams, OpenCV alone is not enough. You can use OpenCV in combination with a GUI framework to stitch each image onto a nice display. I will use PyQt4 as the framework, qdarkstyle for GUI CSS, and imutils for OpenCV convenience functions.
Here is a very stripped down version of the camera GUI I currently use without the placeholder images, credential admin login page, and camera switching ability. I've kept the automatic camera reconnect feature incase the internet dies or the camera connection is lost. I only have 8 cameras as shown in the image above, but it is very simple to add in another camera and should not impact performance. This camera GUI currently performs at about ~60 FPS so it is real-time. You can easily rearrange the layout using PyQt layouts so feel free to modify the code! Remember to change the stream links!
from PyQt4 import QtCore, QtGui
import qdarkstyle
from threading import Thread
from collections import deque
from datetime import datetime
import time
import sys
import cv2
import imutils
class CameraWidget(QtGui.QWidget):
"""Independent camera feed
Uses threading to grab IP camera frames in the background
#param width - Width of the video frame
#param height - Height of the video frame
#param stream_link - IP/RTSP/Webcam link
#param aspect_ratio - Whether to maintain frame aspect ratio or force into fraame
"""
def __init__(self, width, height, stream_link=0, aspect_ratio=False, parent=None, deque_size=1):
super(CameraWidget, self).__init__(parent)
# Initialize deque used to store frames read from the stream
self.deque = deque(maxlen=deque_size)
# Slight offset is needed since PyQt layouts have a built in padding
# So add offset to counter the padding
self.offset = 16
self.screen_width = width - self.offset
self.screen_height = height - self.offset
self.maintain_aspect_ratio = aspect_ratio
self.camera_stream_link = stream_link
# Flag to check if camera is valid/working
self.online = False
self.capture = None
self.video_frame = QtGui.QLabel()
self.load_network_stream()
# Start background frame grabbing
self.get_frame_thread = Thread(target=self.get_frame, args=())
self.get_frame_thread.daemon = True
self.get_frame_thread.start()
# Periodically set video frame to display
self.timer = QtCore.QTimer()
self.timer.timeout.connect(self.set_frame)
self.timer.start(.5)
print('Started camera: {}'.format(self.camera_stream_link))
def load_network_stream(self):
"""Verifies stream link and open new stream if valid"""
def load_network_stream_thread():
if self.verify_network_stream(self.camera_stream_link):
self.capture = cv2.VideoCapture(self.camera_stream_link)
self.online = True
self.load_stream_thread = Thread(target=load_network_stream_thread, args=())
self.load_stream_thread.daemon = True
self.load_stream_thread.start()
def verify_network_stream(self, link):
"""Attempts to receive a frame from given link"""
cap = cv2.VideoCapture(link)
if not cap.isOpened():
return False
cap.release()
return True
def get_frame(self):
"""Reads frame, resizes, and converts image to pixmap"""
while True:
try:
if self.capture.isOpened() and self.online:
# Read next frame from stream and insert into deque
status, frame = self.capture.read()
if status:
self.deque.append(frame)
else:
self.capture.release()
self.online = False
else:
# Attempt to reconnect
print('attempting to reconnect', self.camera_stream_link)
self.load_network_stream()
self.spin(2)
self.spin(.001)
except AttributeError:
pass
def spin(self, seconds):
"""Pause for set amount of seconds, replaces time.sleep so program doesnt stall"""
time_end = time.time() + seconds
while time.time() < time_end:
QtGui.QApplication.processEvents()
def set_frame(self):
"""Sets pixmap image to video frame"""
if not self.online:
self.spin(1)
return
if self.deque and self.online:
# Grab latest frame
frame = self.deque[-1]
# Keep frame aspect ratio
if self.maintain_aspect_ratio:
self.frame = imutils.resize(frame, width=self.screen_width)
# Force resize
else:
self.frame = cv2.resize(frame, (self.screen_width, self.screen_height))
# Add timestamp to cameras
cv2.rectangle(self.frame, (self.screen_width-190,0), (self.screen_width,50), color=(0,0,0), thickness=-1)
cv2.putText(self.frame, datetime.now().strftime('%H:%M:%S'), (self.screen_width-185,37), cv2.FONT_HERSHEY_SIMPLEX, 1.2, (255,255,255), lineType=cv2.LINE_AA)
# Convert to pixmap and set to video frame
self.img = QtGui.QImage(self.frame, self.frame.shape[1], self.frame.shape[0], QtGui.QImage.Format_RGB888).rgbSwapped()
self.pix = QtGui.QPixmap.fromImage(self.img)
self.video_frame.setPixmap(self.pix)
def get_video_frame(self):
return self.video_frame
def exit_application():
"""Exit program event handler"""
sys.exit(1)
if __name__ == '__main__':
# Create main application window
app = QtGui.QApplication([])
app.setStyleSheet(qdarkstyle.load_stylesheet_pyqt())
app.setStyle(QtGui.QStyleFactory.create("Cleanlooks"))
mw = QtGui.QMainWindow()
mw.setWindowTitle('Camera GUI')
mw.setWindowFlags(QtCore.Qt.FramelessWindowHint)
cw = QtGui.QWidget()
ml = QtGui.QGridLayout()
cw.setLayout(ml)
mw.setCentralWidget(cw)
mw.showMaximized()
# Dynamically determine screen width/height
screen_width = QtGui.QApplication.desktop().screenGeometry().width()
screen_height = QtGui.QApplication.desktop().screenGeometry().height()
# Create Camera Widgets
username = 'Your camera username!'
password = 'Your camera password!'
# Stream links
camera0 = 'rtsp://{}:{}#192.168.1.43:554/cam/realmonitor?channel=1&subtype=0'.format(username, password)
camera1 = 'rtsp://{}:{}#192.168.1.45/axis-media/media.amp'.format(username, password)
camera2 = 'rtsp://{}:{}#192.168.1.47:554/cam/realmonitor?channel=1&subtype=0'.format(username, password)
camera3 = 'rtsp://{}:{}#192.168.1.40:554/cam/realmonitor?channel=1&subtype=0'.format(username, password)
camera4 = 'rtsp://{}:{}#192.168.1.44:554/cam/realmonitor?channel=1&subtype=0'.format(username, password)
camera5 = 'rtsp://{}:{}#192.168.1.42:554/cam/realmonitor?channel=1&subtype=0'.format(username, password)
camera6 = 'rtsp://{}:{}#192.168.1.46:554/cam/realmonitor?channel=1&subtype=0'.format(username, password)
camera7 = 'rtsp://{}:{}#192.168.1.41:554/cam/realmonitor?channel=1&subtype=0'.format(username, password)
# Create camera widgets
print('Creating Camera Widgets...')
zero = CameraWidget(screen_width//3, screen_height//3, camera0)
one = CameraWidget(screen_width//3, screen_height//3, camera1)
two = CameraWidget(screen_width//3, screen_height//3, camera2)
three = CameraWidget(screen_width//3, screen_height//3, camera3)
four = CameraWidget(screen_width//3, screen_height//3, camera4)
five = CameraWidget(screen_width//3, screen_height//3, camera5)
six = CameraWidget(screen_width//3, screen_height//3, camera6)
seven = CameraWidget(screen_width//3, screen_height//3, camera7)
# Add widgets to layout
print('Adding widgets to layout...')
ml.addWidget(zero.get_video_frame(),0,0,1,1)
ml.addWidget(one.get_video_frame(),0,1,1,1)
ml.addWidget(two.get_video_frame(),0,2,1,1)
ml.addWidget(three.get_video_frame(),1,0,1,1)
ml.addWidget(four.get_video_frame(),1,1,1,1)
ml.addWidget(five.get_video_frame(),1,2,1,1)
ml.addWidget(six.get_video_frame(),2,0,1,1)
ml.addWidget(seven.get_video_frame(),2,1,1,1)
print('Verifying camera credentials...')
mw.show()
QtGui.QShortcut(QtGui.QKeySequence('Ctrl+Q'), mw, exit_application)
if(sys.flags.interactive != 1) or not hasattr(QtCore, 'PYQT_VERSION'):
QtGui.QApplication.instance().exec_()
Related camera/IP/RTSP, FPS, video, threading, and multiprocessing posts
Python OpenCV streaming from camera - multithreading, timestamps
Video Streaming from IP Camera in Python Using OpenCV cv2.VideoCapture
How to capture multiple camera streams with OpenCV?
OpenCV real time streaming video capture is slow. How to drop frames or get synced with real time?
Storing RTSP stream as video file with OpenCV VideoWriter
OpenCV video saving
Python OpenCV multiprocessing cv2.VideoCapture mp4
I am trying to use openCV to record videos from my IP Cameras for security use. I'd like to set the videos to record within a given time span (Eg from 10am to 10 pm) then sleep until the next day at the next time. The code below tries to emulate this process over a shorter time period to allow for easy testing.
import cv2
# import numpy as np
from datetime import datetime, timedelta
import time
# capture IP camera's live feed
cap = cv2.VideoCapture('rtsp://admin:#10.187.1.146:554/user=admin_password=tlJwpbo6_channel=1_stream=0')
ret, initialFrame = cap.read()
# Settings for the video recording.
fourcc = cv2.VideoWriter_fourcc(*'XVID')
fps = 22
# Get the frame's size
fshape = initialFrame.shape
fheight = fshape[0] # int(fshape[0] * (75 / 100))
fwidth = fshape[1] # int(fshape[1] * (75 / 100))
frameSize = (fwidth,fheight)
if cap.isOpened:
print ("The IP Camera's feed is now streaming\n")
today = datetime.today().strftime('%Y-%b-%d')
try:
#Loop to view the camera's live feed
while datetime.today().strftime('%Y-%b-%d') == today:
vidOut = cv2.VideoWriter('Cam01_'+str(today)+ " " + str(datetime.today().strftime('%H:%M')) +'.avi',fourcc,fps,frameSize)
print ("Recording for " + today)
recStart = datetime.now()
recStop = recStart + timedelta(seconds= 60*3)
print ("Recording for duration of 10 mins \n\n Press Ctrl+C on the keyboard to quit \n\n")
while recStart <= datetime.now() and datetime.now() <= recStop:
# read frame
ret, frame = cap.read()
# Write frame to video file
vidOut.write(frame)
print ("Recording ended at " + datetime.now().strftime("%Y-%B-%d, %A %H:%M"))
print ("Next Recording at " + (datetime.now() + timedelta(seconds= 60*3)).strftime("%Y-%B-%d, %A %H:%M"))
vidOut.release() # End video write
cap.release() # Release IP Camera
# cv2.destroyAllWindows() # Close all Windows that were launched by cv2.
print ('Waiting 3 mins before next recording')
time.sleep(60*3)
continue
except KeyboardInterrupt:
print ("\n Process Interrupted by User \n")
print ("\n Recording ended at " + datetime.now().strftime("%Y-%B-%d, %A %H:%M"))
print ("Next Recording at " + (recStart + timedelta(seconds= 600)).strftime("%Y-%B-%d, %A %H:%M"))
vidOut.release() # End video write
cap.release() # Release IP Camera
else:
print("The video stream couldn't be reached")
I expected that after after the first video is done recording and is released, the videos after that would have content in them. However, only the first video file after running the code has frames in it. the rest are just empty files.
In the while loop you call cap.release() which destroys the reference to the camera, so there are no new frames to save. Either (re)open the videoCapture at the start of the while loop, or don't close it. It is also good practice to check if the cap.read() was successful.
I am trying to capture image in every 5 seconds using opencv through my laptop's built-in webcam. I am using time.sleep(5) for the required pause. In every run, the first image seems to be correctly saved but after that rest all the images are being saved as an unsupported format (when i am opening them). Also I am unable to break the loop by pressing 'q'.
Below is my code.
import numpy as np
import cv2
import time
cap = cv2.VideoCapture(0)
framerate = cap.get(5)
x=1
while(True):
# Capture frame-by-frame
ret, frame = cap.read()
cap.release()
# Our operations on the frame come here
filename = 'C:/Users/shjohari/Desktop/Retail_AI/MySection/to_predict/image' + str(int(x)) + ".png"
x=x+1
cv2.imwrite(filename, frame)
time.sleep(5)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()
Any help is appreciated.
Just a small change, solved my issue.
I included the below code inside the loop :P
cap = cv2.VideoCapture(0)
framerate = cap.get(5)
I am building a motion detector application.So for the motion detection algorithm to work, I converted the frames to grayscale so now the application is able to detect the motions.But when I try to put a text on the frame trying to post a message like "MOVING", even the text has turned gray and is hardly visible.How do I draw colored text on a video frame?
Below is my motion detection application code
import cv2
import numpy as np
from skimage.measure import compare_ssim
from twilio.rest import Client
#we can compare two images using Structural Similarity
#so a small change in pixel value won't prompt this method to term both images as dissimilar
#the closer the value is to 1,the more similar two images are
def ssim(A, B):
return compare_ssim(A, B, data_range=A.max() - A.min())
#capture a video either from a file or a live video stream
cap = cv2.VideoCapture(0)
first_frame = True
prev_frame = None
current_frame = None
#we keep a count of the frames
frame_counter = 0
while True:
if frame_counter == 0:
#prev_frame will always trail behind the current_frame
prev_frame = current_frame
#get a frame from the video
ret, current_frame = cap.read()
#if we reach the end of the video in case of a video file,stop reading
if current_frame is None:
break
#convert the image to grayscale
current_frame = cv2.cvtColor(current_frame,cv2.COLOR_BGR2GRAY)
if first_frame:
#for the first time prev_frame and current_frame will be the same
prev_frame = current_frame
first_frame = False
if frame_counter == 9:
#compare two images based on SSIM
ssim_val = ssim(current_frame, prev_frame)
print(ssim_val)
#if there is a major drop in the SSIM value ie it has detected an object
if ssim_val < 0.8:
# Here I want to put a colored text to the screen
cv2.putText(current_frame, "MOVING", (100, 300),
cv2.FONT_HERSHEY_TRIPLEX, 4, (255, 0, 0))
frame_counter = -1
#show the video as a series of frames
cv2.imshow("Motion Detection",current_frame) #(name of the window,image file)
frame_counter += 1
key = cv2.waitKey(1) & 0xFF #cv2.waitKey(1) returns a value of -1 which is masked using & 0xFF to get char value
if key == ord('q'): #gives ASCII value of 'q'
break
#release the resources allocated to the video file or video stream
cap.release()
#destroy all the windows
cv2.destroyAllWindows()
I searched online and I got this piece of code which basically suggests to convert grayscale back to BGR
backtorgb = cv2.cvtColor(current_frame, cv2.COLOR_GRAY2RGB)
But this didn't work.I even took a copy of the current frame before it being converted to grayscale frame and then tried to write on the copied color frame but still the text comes gray and not colored.What should I do?
I made a motion detector solely software-based by referring a few books and then adding my own code. The added code saves a frame of the detected motion locally on the Raspberry Pi and also uploads the same to my Google Drive. Another set of code, sends an email to my email-address informing me of the motion detected.
The problem is that when the file is being saved and uploaded, the Open CV output window freezes until the above processes finish. I tried multiprocessing and multi-threading on python but it didn't help. Is there any way I could improve my logic in a way that it doesn't freeze the output window?
EDIT : The issue was somewhat fixed by removing join() from the two processes. There is a very slight lag but I think that's good enough. Thanks to everyone who replied :)
from pydrive.drive import GoogleDrive
from pydrive.auth import GoogleAuth
import cv2
from multiprocessing import Process
import numpy as np
import datetime
import time
import smtplib
# make count 0
count = 0
def sf(t2):
cv2.imwrite("/home/pi/Desktop/StoredImages/frame%d.jpg" % count, t2)
# 'google drive authentication' stuff
gauth = GoogleAuth()
# try to load saved client credentials
gauth.LoadCredentialsFile("mycreds.txt")
if gauth.credentials is None:
# authenticate if not there
gauth.LocalWebserverAuth()
elif gauth.access_token_expired:
# Refresh them if expired
gauth.Refresh()
else:
# Initialize the saved creds
gauth.Authorize()
# Save the current credentials to a file
gauth.SaveCredentialsFile("mycreds.txt")
drive = GoogleDrive(gauth)
def upload_file():
file1 = drive.CreateFile({'parent':'/home/pi/Desktop/StoredImages/'})
file1.SetContentFile('/home/pi/Desktop/StoredImages/frame%d.jpg' % count)
file1.Upload()
# 'sending an email' stuff
server = smtplib.SMTP('smtp.gmail.com',587)
server.starttls()
server.login("Removed Intentionally","Removed Intentionally")
msg = "Motion Detected! For more details check your Drive."
# capture Video from the camera module
cap = cv2.VideoCapture(0)
# stores the present date and time
lastUploaded = datetime.datetime.now()
# kernel is created for the dilation process
k = np.ones((3,3),np.uint8) # creates a 3X3 Matrix filled with ones and
# has the data type uint8 (unsigned integer)
# which can contain values from 0 to 255
# first two subsequent frames captured
t0 = cap.read()[1]
t1 = cap.read()[1]
# initially motion detected 0 times
motionCounter = 0
while True:
# difference between two subsequent frames
d=cv2.absdiff(t1,t0)
# stores present date and time
timestamp = datetime.datetime.now()
# converting difference to grayscale
grey = cv2.cvtColor(d,cv2.COLOR_BGR2GRAY)
# grayscale converted to gaussian blur
blur = cv2.GaussianBlur(grey,(3,3),0)
# gaussian blur converted to binary image
ret, th = cv2.threshold(blur, 15, 155, cv2.THRESH_BINARY)
# dilating the image before using the contour function
dilated = cv2.dilate(th,k,iterations=2)
# contour function to find edges
_, contours, heierarchy = cv2.findContours(dilated,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
# copying the original frame to a temporary frame for display
t2 = t0
# drawing green edges around the area with movement
cv2.drawContours(t2, contours, -1, (0,255,0), 2)
# showing output in a new window
cv2.imshow('Output',t2)
# going through each and every contour in the image
for c in contours:
# if contour is lesser than a threshold size, ignore
if cv2.contourArea(c) < 5000:
continue
# if motion occurred after 2 secs
if (timestamp - lastUploaded).seconds >= 2.0:
motionCounter += 1
# if 8 motions occured in 2 secs
if motionCounter >= 8:
# write to a temporary file location using threads
new_process = Process(target=sf, args=(t2,))
new_process.start()
new_process.join()
# upload the temporary pic to Google drive using threads
new_process = Process(target=upload_file)
new_process.start()
new_process.join()
# sending a mail about motion detected
server.sendmail("Removed Intentionally","Removed Intentionally",msg)
# increasing count by 1 and resetting everything
count=count+1
lastUploaded = timestamp
motionCounter = 0
# making the next frame the previous and reading a new frame
t0 = t1
t1 = cap.read()[1]
# esc key breaks the entire loop
if cv2.waitKey(5) == 27:
break
# stops the video stream and exits the window
cap.release()
cv2.destroyAllWindows()
# stops the email server connection
server.quit()
I think you used the multiprocessing in a wrong way. Your code
# write to a temporary file location using threads
new_process = Process(target=sf, args=(t2,))
new_process.start()
new_process.join()
will actually create and start a process, but then it will also wait for it (new_process.join()) to finish. So basically you want to start a parallel running process, but then you wait for it to finish.
Better would be to create and start the processes at the beginning of your program and wait for them to finish at the very end of your program.
Also create a queue for each process (also in the multiprocessing module).
Each process should run in an endless loop and wait for a queue. In your main thread, you feed each process' queue with what it should do (store a file locally, store file remotely)
At the end of your program, you should send your processes a final indication to leave their endless loop, so your new_process.join() statement in the main thread will pick up the fact, that the processes have ended.