Multi-object tracking initialization in Opencv using multiTracker object - python-3.x

Question

I am using multiTracker in cv2 to track multiple objects. My code is built based on this link and this one. I want to initialize all the bounding boxes at once at any point during the video. However, I have trouble doing this. Here is the code:
import imutils
import cv2
from random import randint
trackerName = 'csrt'
videoPath = "C:/Users/Art/testVideo.mp4"
OPENCV_OBJECT_TRACKERS = {
"csrt": cv2.TrackerCSRT_create,
"kcf": cv2.TrackerKCF_create,
"boosting": cv2.TrackerBoosting_create,
"mil": cv2.TrackerMIL_create,
"tld": cv2.TrackerTLD_create,
"medianflow": cv2.TrackerMedianFlow_create,
"mosse": cv2.TrackerMOSSE_create
}
trackers = cv2.MultiTracker_create()
cap = cv2.VideoCapture(videoPath)
while cap.isOpened():
ret, frame = cap.read()
if frame is None:
break
# for fast processing resize the frame
frame = imutils.resize(frame, width=600)
(success, boxes) = trackers.update(frame)
for box in boxes:
(x, y, w, h) = [int(v) for v in box]
cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)
cv2.imshow("Frame", frame)
key = cv2.waitKey(1) & 0xFF
# if the 'p' key is selected, we select a bounding box to track
if key == ord("p"):
boxes = []
colors = []
boxFlag = True
while boxFlag:
box = cv2.selectROI('MultiTracker', frame, fromCenter=False,
showCrosshair=True)
boxes.append(box)
colors.append((randint(64, 255), randint(64, 255), randint(64,
255)))
print("Press q to quit selecting boxes and start tracking")
print("Press any other key to select next object")
if key == ord("q"): # q is pressed
boxFlag = False
# Initialize MultiTracker
for box in boxes:
tracker = OPENCV_OBJECT_TRACKERS[trackerName]()
trackers.add(tracker, frame, box)
cap.release()
cv2.destroyAllWindows()
However, there are some problems. First, when I click on 'p' key to select bounding boxes, the video pauses and another window opens that shows the frame at which video was paused and I can select bounding boxes on the new window only. Also, when I press 'q' key, nothing will happen and basically it stays there forever. My question is how I can fix this problem, and be able to initialize tracking after I select all of the bounding boxes.

Answer 1

I figured it out and thought this might be useful for someone else. I made some changes to the previous code. I also realized that cv2 has selectROIs method which can be useful if someone wants to select multiple bounding boxes at once. Here is the updated code:
import imutils
import cv2
from random import randint
trackerName = 'csrt'
videoPath = "C:/Users/Art/testVideo.mp4"
OPENCV_OBJECT_TRACKERS = {
"csrt": cv2.TrackerCSRT_create,
"kcf": cv2.TrackerKCF_create,
"boosting": cv2.TrackerBoosting_create,
"mil": cv2.TrackerMIL_create,
"tld": cv2.TrackerTLD_create,
"medianflow": cv2.TrackerMedianFlow_create,
"mosse": cv2.TrackerMOSSE_create
}
# initialize OpenCV's special multi-object tracker
trackers = cv2.MultiTracker_create()
cap = cv2.VideoCapture(videoPath)
while cap.isOpened():
ret, frame = cap.read()
if frame is None:
break
frame = imutils.resize(frame, width=600)
(success, boxes) = trackers.update(frame)
# loop over the bounding boxes and draw them on the frame
for box in boxes:
(x, y, w, h) = [int(v) for v in box]
cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)
cv2.imshow("Frame", frame)
key = cv2.waitKey(1) & 0xFF
# if the 's' key is selected, we are going to "select" a bounding
# box to track
if key == ord("s"):
colors = []
# select the bounding box of the object we want to track (make
# sure you press ENTER or SPACE after selecting the ROI)
box = cv2.selectROIs("Frame", frame, fromCenter=False,
showCrosshair=True)
box = tuple(map(tuple, box))
for bb in box:
tracker = OPENCV_OBJECT_TRACKERS[trackerName]()
trackers.add(tracker, frame, bb)
# if you want to reset bounding box, select the 'r' key
elif key == ord("r"):
trackers.clear()
trackers = cv2.MultiTracker_create()
box = cv2.selectROIs("Frame", frame, fromCenter=False,
showCrosshair=True)
box = tuple(map(tuple, box))
for bb in box:
tracker = OPENCV_OBJECT_TRACKERS[trackerName]()
trackers.add(tracker, frame, bb)
elif key == ord("q"):
break
cap.release()
cv2.destroyAllWindows()
After selecting each bounding box, you need to click on "Enter" or "Space" button to finalize it and start selecting a new bounding box. Once you are done with bounding box selection, press "Esc" key to end ROI selection and start tracking. Also, if you need to reset bounding boxes for any reason, press "r" button.

Answer 2

Maybe You should try the "cv2.selectROIs" function before the "while cap.isOpened():" loop, without loop, because it looks like "cv2.selectROIs" function works as loop itself, while you are selecting ROIs.