Retrieve exif data from thousands of images fast - optimising function - python-3.x

Question

I've written script that retrieves specific fields of exif data from thousands of images in a directory (including subdirectories) and saves the info to a csv file:
import os
from PIL import Image
from PIL.ExifTags import TAGS
import csv
from os.path import join
####SET THESE!###
imgpath = 'C:/x/y' #Path to folder of images
csvname = 'EXIF_data.csv' #Name of saved csv
###
def get_exif(fn):
ret = {}
i = Image.open(fn)
info = i._getexif()
for tag, value in info.items():
decoded = TAGS.get(tag, tag)
ret[decoded] = value
return ret
exif_list = []
path_list = []
filename_list = []
DTO_list = []
MN_list = []
for root, dirs, files in os.walk(imgpath, topdown=True):
for name in files:
if name.endswith('.JPG'):
pat = join(root, name)
pat.replace(os.sep,"/")
exif = get_exif(pat)
path_list.append(pat)
filename_list.append(name)
DTO_list.append(exif['DateTimeOriginal'])
MN_list.append(exif['MakerNote'])
zipped = zip(path_list, filename_list, DTO_list, MN_list)
with open(csvname, "w", newline='') as f:
writer = csv.writer(f)
writer.writerow(('Paths','Filenames','DateAndTime','MakerNotes'))
for row in zipped:
writer.writerow(row)
However, it is quite slow. I've attempted to optimise the script for performance + readabilty by using list and dictionary comprehensions.
import os
from os import walk #Necessary for recursive mode
from PIL import Image #Opens images and retrieves exif
from PIL.ExifTags import TAGS #Convert exif tags from digits to names
import csv #Write to csv
from os.path import join #Join directory and filename for path
####SET THESE!###
imgpath = 'C:/Users/au309263/Documents/imagesorting_testphotos/Finse/FINSE01' #Path to folder of images. The script searches subdirectories as well
csvname = 'PLC_Speedtest2.csv' #Name of saved csv
###
def get_exif(fn): #Defining a function that opens an image, retrieves the exif data, corrects the exif tags from digits to names and puts the data into a dictionary
i = Image.open(fn)
info = i._getexif()
ret = {TAGS.get(tag, tag): value for tag, value in info.items()}
return ret
Paths = [join(root, f).replace(os.sep,"/") for root, dirs, files in walk(imgpath, topdown=True) for f in files if f.endswith('.JPG' or '.jpg')] #Creates list of paths for images
Filenames = [f for root, dirs, files in walk(imgpath, topdown=True) for f in files if f.endswith('.JPG' or '.jpg')] #Creates list of filenames for images
ExifData = list(map(get_exif, Paths)) #Runs the get_exif function on each of the images specified in the Paths list. List converts the map-object to a list.
MakerNotes = [i['MakerNote'] for i in ExifData] #Creates list of MakerNotes from exif data for images
DateAndTime = [i['DateTimeOriginal'] for i in ExifData] #Creates list of Date and Time from exif data for images
zipped = zip(Paths, Filenames, DateAndTime, MakerNotes) #Combines the four lists to be written into a csv.
with open(csvname, "w", newline='') as f: #Writes a csv-file with the exif data
writer = csv.writer(f)
writer.writerow(('Paths','Filenames','DateAndTime','MakerNotes'))
for row in zipped:
writer.writerow(row)
However, this has not changed the performance.
I've timed the specific regions of the code and found that specifically opening each image and getting the exif data from each image in the get_exif function is what takes time.
To make the script faster,I'm wondering if:
1) It is possible to optimise on the performance of the function?, 2) It is possible to retrive exif data without opening the image?, 3) list(map(fn,x)) is the fastest way of applying the funtion?

Answer 1

If read the docs in the right way PIL.Image.open() does not only extracts the EXIF data from the file but also reads and decodes the entire image, which probably is the bottleneck here. The first thing I would do would be to change to a library or routine that only works on the EXIF data and does not care for the image content. ExifRead or piexif might be worth a try.