Is it possible to read files in order of Date Modified? - excel

Question

My macro will need to read through a very large directory of files and parse data from them. This directory is updated periodically so I am trying to optimize my program to only read files that were added since the last time the program was run.
So far using FileSystemObject it seems I can only read files alphabetically, if I'm not mistaken.
The best solution I have so far is to read all the files every time, create an array containing the file information, sort by DateModified, then open only the files I need. I'm curious to see if I can skip this step by reading files in order of DateModified.
Thanks in advance.

Answer 1

Shell does seem to be a good option here - although I haven't compared performance against a FSO. You could, for example, consider the forfiles command which allows you to retrieve files modified after a specified date?
Some sample code for that would be as follows:
Public Sub RunMe()
Dim fileNames As Collection
Dim path As String
Dim dat As Date
Dim file As Variant
'Set the path and 'find after' date.
path = "c:\user\documents"
dat = #1/1/2018#
'Fetch the files, setting mask as required.
'This example is fetching all .txt files.
Set fileNames = GetFilesModifiedAfter(path, dat, "*.txt")
'Process the list of files.
If Not fileNames Is Nothing Then
For Each file In fileNames
' ... do stuff here.
Debug.Print path & "\" & file
Next
End If
End Sub
Private Function GetFilesModifiedAfter( _
path As String, _
after As Date, _
Optional mask As String) As Collection
Dim cmd As String
Dim piped() As String
Dim result As Collection
Dim i As Long
'Build the command string.
'Date must be formatted as MM/DD/YYYY.
cmd = "cmd.exe /s /c forfiles /p " & _
"""" & path & """" & _
" /d +" & Format(after, "mm/dd/yyyy")
'Add the mask if passed-in.
If mask <> vbNullString Then cmd = cmd & " /m " & mask
'Execute the command and split by /r/n.
piped = Split(CreateObject("WScript.Shell").Exec(cmd).StdOut.ReadAll, vbCrLf)
'Leave if nothing is returned.
If UBound(piped) = -1 Then Exit Function
'Poplate the result collection,
'and remove the leading and trailing inverted commas.
Set result = New Collection
For i = 0 To UBound(piped)
If Len(piped(i)) > 2 Then
result.Add Mid(piped(i), 2, Len(piped(i)) - 2)
End If
Next
'Return the result collection.
Set GetFilesModifiedAfter = result
End Function
Update
I've just done some testing and it seems FSO is quicker, certainly on Folders containing fewer than 100 files. It'd be interesting to run this on really large folders (say a thousand files) as instinctively I feel Shell might have a performance advantage. However, for now, here's the FSO version:
Private Function GetFilesModifiedAfter2( _
path As String, _
after As Date, _
mask As String) As Collection
Dim fso As Object, file As Object
Dim result As Collection
'Instance of objects.
Set fso = CreateObject("Scripting.FileSystemObject")
Set result = New Collection
'Iterate the files and test date last modified property.
For Each file In fso.GetFolder(path & "\").Files
If file.Name Like mask And file.DateLastModified > after Then
result.Add file.Name
End If
Next
'Return the result collection.
Set GetFilesModifiedAfter2 = result
End Function