Read parquet files from S3 folder using wildcard - apache-spark

Question

I have S3 folders as below, each with parquet files:
s3://bucket/folder1/folder2/2020-02-26-12/key=Boston_20200226/
s3://bucket/folder1/folder2/2020-02-26-12/key=Springfield_20200223/
s3://bucket/folder1/folder2/2020-02-26-12/key=Toledo_20200226/
s3://bucket/folder1/folder2/2020-02-26-12/key=Philadelphia_20191203/
My goal is to be able to open the parquet files from '*_20200226' folders only.
I use a FOR loop to first gather a list/array of all files and then pass it to the READ operation into a DF in spark 2.4.
s3_files = []
PREFIX = "folder1/folder2/"
min_datetime = current_datetime - timedelta(hours=72)
while current_datetime >= min_datetime:
each_hour_prefix = min_datetime.strftime('%Y-%m-%d-%H')
if any(fname.key.endswith('.parquet') for fname in s3_bucket.objects.filter(Prefix=(PREFIX + each_hour_prefix))):
s3_files.append('s3://{bucket}/{prefix}'.format(bucket=INPUT_BUCKET_NAME, prefix=(PREFIX + each_hour_prefix + '/*')))
min_datetime = min_datetime + timedelta(hours=1)
spark.read.option('basePath',('s3://' + INPUT_BUCKET_NAME)).schema(fileSchema).parquet(*s3_files)
where fileSchema is the schema struct of the parquet files, s3_files is a array of all files I picked up by perusing through S3 folders above. The above FOR loop works but my goal is to read Boston_20200226 and Toledo_20200226 folders only. Is it possible to do wildcard searches like "folder1/folder2/2020-02-26-12/key=**_20200226*" or perhaps change the 'read.parquet' command in some way?
Thanks in advance.
Update:
I resorted to a rudimentary way of perusing through all the folders and only finding files that match pattern = '20200226'(not the most efficient way). I collect the keys in a list and then read each parquet file in a DF and perform a union at the end. Everything works fine except the 'key' column is not read in the final DF. It is part of the partitionBy() code that created these parquet files. Any idea on how can capture the 'key'?