Consolidation VBA of files in directory via ms-access - excel

Question

I have 33 linked tables all in same directory(sharepoint site). i would like some code to copy in all the tables in this directory and create a consolidated table of this data
all tables are same format and live in same folder.
i have tried Union and other ways but these have been restricted with the number of columns i could use.
The excel template has columns up to GG

Answer 1

You can try using Dir to loop through the directory, and use the DoCmd.TransferSpreadsheet to import the spreadsheets in that particular directory. Replace "YourTable" with your table name and "\yourdirectory\" with the location of the SP folder.
Sub ExcelImport()
Dim MyObj As Object, MySource As Object, file As Variant
file = Dir("\\yourdirectory\")
While (file <> "")
If InStr(file, "xls") Or InStr(file, "xlsx") > 0 Then
DoCmd.TransferSpreadsheet acImport, acSpreadsheetTypeExcel12, "YourTable", file, True
End If
file = Dir
Wend
End Sub

Answer 2

I'd do a similar thing to that mentioned by Ryan, but wouldn't import direct into the table I would want to query (mainly because there could be loads of housekeeping and/or data quality issues you may need to deal with first).
I'd first create a staging table with multiple alphanumeric fields (GG would take you to 189 fields; I'd call them F1-F189), PLUS a field to store the filename (and perhaps a couple of extra fields to handle the avoidance of duplicate records or to handle in-table updates - a datetimestamp and a process flag).
You can then have a loop that imports each file in turn into the staging table, and then within the loop execute a query (referring to a pre-written Access query is best as you will need to set up a Parameter to handle the filename; plus, you may want to tweak it over time) that will insert the records from the staging table into your proper destination table with the filename from your FOR-NEXT loop.
Within the loop, after you have imported each file you may then want to perform some housekeeping before moving onto the next file.
Below is something I've pretty much pasted in from what I use - it is messy, and it does use bits that are adapted from someone else's code, so I will credit that where I remember and apologies in advance for those I don't:
Sub import_ASSGMU()
' Macro loops through the specified directory (strPath)
' and imports Excel files to staging table in the Access Database
' (tbl_import_generic)
' before using an Access query to insert into the cleaned-up destination table
' (tbl_transformed_ASSGMU)
Dim strPath As String
strPath = "c:\data" ' YOU MUST SET YOUR OWN PATH HERE
If Right(strPath, 1) <> "\" Then strPath = strPath & "\"
' this bit is if you want to move files out of the folder after they are processed
' Dim ImportSuccessPath As String
' ImportSuccessPath = "c:\data\Processed" 'hard-coded sub-folder name
' If Right(ImportSuccessPath, 1) <> "\" Then ImportSuccessPath = ImportSuccessPath & "\"
Dim strFile As String 'Filename
Dim strFileList() As Variant 'File Array
Dim intFile As Integer 'File Number
'Loop through the folder & build file list - file name prefix is HARDCODED
strFile = Dir(strPath & "ASSG_MU_*.xls*")
' this takes all the filenames and adds them to an array
While strFile <> ""
'add files to the list
intFile = intFile + 1
ReDim Preserve strFileList(1 To intFile)
strFileList(intFile) = strFile
strFile = Dir()
Wend
'see if any files were found
If intFile = 0 Then
'MsgBox "No files found"
Exit Sub
End If
' sorts the array so files are in filename order
' needs a procedure from here:
' http://www.access-programmers.co.uk/forums/showthread.php?t=194737
Call SortArray(strFileList())
' cycle through the list of files in the array & import to Access
For intFile = 1 To UBound(strFileList)
' empty staging table
DoCmd.SetWarnings False
DoCmd.RunSQL "DELETE FROM tbl_import_generic;"
DoCmd.SetWarnings True
' delete records from same filename previously imported
'DoCmd.SetWarnings False
' DoCmd.RunSQL "DELETE FROM tbl_transformed_ASSGMU WHERE import_file = """ & strFileList(intFile) & """;"
'DoCmd.SetWarnings True
' allows import of xls AND xlsx
If Right(strFileList(intFile), 1) = "s" Then
DoCmd.TransferSpreadsheet acImport, 8, "tbl_import_generic", strPath & strFileList(intFile), False, "A1:GG50000"
ElseIf Right(strFileList(intFile), 1) = "x" Then
DoCmd.TransferSpreadsheet acImport, 10, "tbl_import_generic", strPath & strFileList(intFile), False, "A1:GG50000"
End If
'DoCmd.Echo False, ""
DoCmd.SetWarnings False
Set db = CurrentDb
Dim qdf As DAO.QueryDef
' create an Access INSERT query with a name that matches the next code line that will take records from the tbl_import_generic and write them to the destination table. At this stage you can embed some hygeine factors or parse data in the Access query
Set qdf = db.QueryDefs("qry_transform_ASSGMU")
' create a Parameter called param_filename and use it to populate a
' destination field - this will then put the name of the imported
' file in your destination table
qdf!param_filename = strFileList(intFile)
qdf.Execute 'dbFailOnError ' useful when testing your query
DoCmd.SetWarnings True
' if you want to "move" files out of the folder when processed
' copy processed file to Processed folder & delete file from Import
' FileCopy strPath & strFileList(intFile), ImportSuccessPath & strFileList(intFile)
' Kill strPath & strFileList(intFile)
Next
MsgBox UBound(strFileList) & " file(s) were imported."
End Sub
Have a play around with it. Hope this is useful to you.