VBA Code to consolidate .csv data - excel

I am trying to consolidate a specific range of date from many csv files. What I want to do is to select this range and paste it into a master sheet in a separate workbook. The .csv files are all arranged in one folder and all have 1 sheet in the same format. The range can be dynamic so this code will need to be able to select all the rows below a cell or be able to delete blank rows from a larger range. I would appreciate any help.
Thanks

I've used a batch file to something like this in the past. This code does not handle deleting Aggregate.csv if the batch file is ran again.
#ECHO OFF
Set loc=C:\Test\
Copy %loc%*.csv %loc%\Aggregate.csv
Once in Excel, you can delete all the header rows and filter date ranges with VBA. You can also use VBA's Shell method to aggregate with Copy.
Edit:
You can also create a data source in Other Data Sources > MS Query in order to query Aggregate.csv with Microsoft Text Driver using date ranges, etc.

Some pointers how to go about the solution:
First, enumerate the files using the FileSystemObject.
Set fso = CreateObject("Scripting.FileSystemObject")
set fld = fso.GetFolder("path\to\my\folder")
For Each file in fld.Files
If file.Name Like "*.csv" Then LoadFile file.Name
Next
Declare fso, fld, file as Object. LoadFile will be a function you have to write, which processes a single file. It will look approximately like this:
Sub LoadFile(filename as String)
dim buffer() as variant
dim wb as workbook, ws as worksheet
dim i as Long, beginrow as long, endrow as long
Set wb = Workbooks.Open(filename)
Set ws = wb.Worksheets(1) ' .csv always has 1 worksheet
buffer = ws.Range("A1:A10000") ' put a sensible upper bound here
for i = 1 to 10000
' replace (..first..) and (..last..) with your search interval
if buffer(i, 1) <= (..first..) Then beginrow = i
if buffer(i, 1) < (..last..) Then endrow=i
next
' now beginrow and endrow hold the interval to cut
ws.Cells(beginrow, 1).Resize(endrow-beginrow+1, column_count).Copy destination:=(... to be determined ..)
wb.Close
End Sub
The function opens the file; then searches the first column for the interval to copy; then copies the cells and closes the file.
The code is not runnable as-is, but should hopefully give you the right ideas.

Related

Skipping column names - combining tables from multiple workbooks

I am trying to combine a table from multiple workbooks and create a new master workbook which contains all the extracted tables. My current code successfully copies and pastes data from each workbook but still have a few issues and couldn't figure them out by myself.
First, I want to skip the first row, which is just variable names, starting from the second source file. I still need it from the first source file so that my master workbook can have column names in the first row. I tried to achieve this using some loops but it didn't work. Which part do I need to update in order to do this?
Second, is there a way to create an additional column in the master workbook as a flag that shows the source file of data when I copy and paste a table from each individual source file to the master?
For example, if a source excel file is named "file123", the flag column would contain "file123" as its value.
Lastly, in the line of my code where it pastes the copied value,
MaWS.Range("A12785").End(xlUp).Offset(1, 0).PasteSpecial Paste:=xlPasteValues
I randomly assigned a big number "A12785" but how do I determine this value? Can I just keep it random?
FYI, here is my current code.
My master workbook is named "Master" and has a sheet named "Summary"
My source files have the table in the "data" tab.
Option Explicit
Sub Merge()
Dim SrPath As String
Dim MaPath As String
Dim SrName As String
Dim MaName As String
Dim SrTemplate As String
Dim MaTemplate As String
Dim SrWS As Worksheet
Dim MaWS As Worksheet
'Define folders and filenames
SrPath = "C:\Users\Documents\Test\"
MaPath = "C:\Users\Documents\Test\Master\"
SrTemplate = "*.xlsx" '
MaTemplate = "Master.xlsm"
'Open the template file and get the Worksheet to put the data into
MaName = Dir(MaPath & MaTemplate)
Workbooks.Open SumPath & SumName
Set MaWS = ActiveWorkbook.Worksheets("Summary")
'Open each source file, copying the data from each into the template file
SrName = Dir(SrPath & SrTemplate) 'Retrieve the first file
Do While SrName <> ""
'Open the source file and get the worksheet with the data we want.
Workbooks.Open SrPath & SrName
Set SrWS = ActiveWorkbook.Worksheets("data")
'Copy the data from the source and paste at the end of Summary sheet
SrWS.Range("A1:N35").Copy
sumWS.Range("A12785").End(xlUp).Offset(1, 0).PasteSpecial Paste:=xlPasteValues
'Close the current sourcefile and get the next
Workbooks(SrName).Close SaveChanges:=False 'close
MyName = Dir 'Get next file
Loop
'Now all sourcefiles are copied into the Template file. Close and save it
Workbooks(MaName).Close SaveChanges:=True
End Sub
Thank you.

Excel VBA that will get the specific data from different workbook

Sub VBA_Read_External_Workbook()
'''''Define Object for Target Workbook
Dim Target_Workbook As Workbook
Dim Source_Workbook As Workbook
Dim Target_Path As String
'''''Assign the Workbook File Name along with its Path
'''''Change path of the Target File name
Target_Path = "C:\Users\User\Desktop\Excel VBA\Working Sample Folder\MAY 2017 Summary- Atlas work.xlsx"
Set Target_Workbook = Workbooks.Open(Target_Path)
Set Source_Workbook = ThisWorkbook
'''''With Target_Workbook object now, it is possible to pull any data from it
'''''Read Data from Target File
Target_Data = Target_Workbook.Sheets(1).Range("A1:B3")
Source_Workbook.Sheets(2).Range("A1:B3") = Target_Data
'''''Update Target File
Source_data = Source_Workbook.Sheets(1).Range("A1:B3")
Target_Workbook.Sheets(1).Range("A1:B3") = Source_data
'''''Close Target Workbook
Source_Workbook.Save
Target_Workbook.Save
Target_Workbook.Close False
'''''Process Completed
MsgBox "Task Completed"
End Sub
I've been modifying this code that i found in this website to use it for copying the specific data in a specific format.
What I need some guidance is to add a loop to get the data from the files that will be put in a folder.Files that needed to be read
So my questions
Basically, I already set the specific range of data that needed to be copied and paste on my destination files. But instead of keep changing the target path, is there a way to put a loop that it will auto jump to the next workbook and get those values?
I found out that using this method to transfer the data, it doesn't transfer the data nature which for example if it is in time format at the source file, when the VBA execute and update the destination file, the value is not in the same format and all are pasted in general format.
Is it possible to loop the update where it will auto jump to the next row to paste the data?
I tried to google some of the VBA codes but the answer is very vague.
Appreciate any input from your experiences.
1)Basically, I already set the specific range of data that needed to be copied and paste on my destination files. But instead of keep changing the target path, is there a way to put a loop that it will auto jump to the next workbook and get those values?
This will get you started
Dim MyFolder As String
Dim StrFile As String
Dim flName As String
'~~> Change this to the relevant folder
MyFolder = "c:\MyFolder\"
StrFile = Dir(MyFolder & "*.xls*")
'~~> Loop through all excel files in the folder
Do While Len(StrFile) > 0
flName = MyFolder & StrFile
'~~> Open the workbook
Set wb = Workbooks.Open(flName)
'
'~~> Rest of your code
'
wb.Close (False)
StrFile = Dir
Loop
2)I found out that using this method to transfer the data, it doesn't transfer the data nature which for example if it is in time format at the source file, when the VBA execute and update the destination file, the value is not in the same format and all are pasted in general format.
The code is directly setting the value and hence the formats are not copied. You need to .Copy and .Pastespecial instead of directly setting the value. Record a macro to see how .Copy and .Pastespecial work. or read up on
Range.PasteSpecial Method (Excel)
3)Is it possible to loop the update where it will auto jump to the next row to paste the data?
Find the last row and then do a copy paste to that row. Please see the below link to find the last row.
Finding Last Row

Copying data from two files and feeding it into a comparison sheet

I have two documents titled Old Cal and New Cal. I want to be able to take the tables from each separate file and copy them into a new comparison file automatically. The only issue is, there will continuously be new documents that need to be inputted into a new comparison sheet every time the data is collected. For example, this time the files are called Old Cal and New Cal, but then after those comparisons I will have two brand new files named Blue and Red with completely new information that I will need to compare on another new page. I need help writing a macro that can automatically populate the comparison sheet I need to make between the two documents every time even though there will be different files every time. Let me know if more info is needed, any help is welcome. THANKS!
As Tim Williams says using Application.GetOpenFilename() is a great solution. It will open the standard excel open file dialog, but will not actually open the file, it just gives you the file name.
To actually open the workbook you can use Application.Workbooks.Open(). Here is a snippet of code that you can modify for your use.
Dim file1 as String
Dim file2 as String
Dim book1 as Workbook
Dim book2 as Workbook
Dim comp as Workbook
Dim targetSheet as Worksheet
Dim copySheet as Worksheet
Dim lastRow as Long
Dim lastCol as Long
Set comp = Application.ActiveWorkbook
file1 = Application.getOpenFilename
file2 = Application.getOpenFilename
If file1 <> "False" AND file2 <> "False" Then
Set book1 = Application.Workbooks.Open(file1)
Set book2 = Application.Workbooks.Open(file2)
'Copy the contents from the first sheet of each of the files to a new sheet in a new sheet
comp.Worksheets.add After:=comp.Worksheets(comp.Worksheets.count)
Set targetSheet = comp.Worksheets(comp.Worksheets.count)
Set copySheet = book1.Worksheets(1)
lastRow = copySheet.Cells.SpecialCells(xlCellTypeLastCell).Row
lastCol = copySheet.Cells.SpecialCells(xlCellTypeLastCell).Column
copySheet.Range(copySheet.Cells(1, 1), copySheet.Cells(lastRow, lastCol)).Copy
targetSheet.Cells(1, 1).PasteSpecial
Set copySheet = book2.Worksheets(1)
lastRow = copySheet.Cells.SpecialCells(xlCellTypeLastCell).Row
lastCol = copySheet.Cells.SpecialCells(xlCellTypeLastCell).Column
copySheet.Range(copySheet.Cells(1, 1), copySheet.Cells(lastRow, lastCol)).Copy
lastRow = targetSheet.Cells.SpecialCells(xlCellTypeLastCell).Row
targetSheet.Cells(lastRow, 1).PasteSpecial
book1.Close
book2.Close
End If
This probably won't solve you're problem exactly, but it should be a good starting point. Right now it will copy the first sheet of both workbooks (one ontop of the other) onto a new sheet in the original workbook.
Hope this helps, and welcome to SO

Creating graph in Excel using VBA/macro

I created a macro in Excel for creating bar graphs automatically.
Whenever I run it, it gives "smr run time error" and I am not able to figure out what is wrong with my code.
Sub CreateGraph()
'
' CreateGraph Macro
''Initialize variables
Dim lastRow As Integer
Dim xlsPath As String
Dim xlsFile As String
xlsPath = "H:\"
xlsFile = "text.xls"
Workbooks.Open Filename:=xlsPath & xlsFile
ActiveWindow.SmallScroll Down:=-81
Range("A1:B" & lastRow).Select
ActiveSheet.Shapes.AddChart.Select
ActiveChart.SetSourceData Source:=Range("'TEST'!$A$1:$B" & lastRow)
ActiveChart.ChartType = xlBarClustered
ActiveChart.Axes(xlCategory).Select
ActiveSheet.ChartObjects("Chart 2").Activate
ActiveChart.Axes(xlCategory).ReversePlotOrder = True
Range("Q111").Select
ActiveWorkbook.Save
ActiveWorkbook.Close
End Sub
Can anyone help me in solving this puzzle please.
Also for running any macro automatically from SAS, I always have to change the Excel options for "enable all macros" which I suppose is not good. I have seen people creating and running macros without doing this. Can you please tell me how can I run the macros with enabling all macros option in Excel.
The code within this version of the answer is essentially unchanged from the previous version. However, the text has been rewritten to (1) describe my experience of this type of project, (2) answer the true question and (3) better explain the solution.
My experience of this type of project
I have been involved in five such projects. In each case, the client believed they required the automatic creation of charts but detailed discussion revealed that that this was not the requirement. The clients all published a substantial number of charts per month but most of the charts were the same as last month but with new data. They needed to automate the provision of new data for the charts. Every month some charts were revised but this was humans agreeing better ways of presenting the data. They wanted the 90% of charts that were unchanged to go through without any effort and implementation of the revisions to be as easy as possible.
In this case, the questioner publishes 100 charts per month in the form of an Excel workbook. The data for these charts comes from an Access database. The solution allows for the charts to be changed easily but this is to ease the programming and not to provide more than has been requested.
Release Template.xls
The solution requires a hand-crafted workbook named Release Template.xls. This workbook will contain all the charts and the Month 1 data. The solution creates a copy of this workbook named Release YYMM.xls in which the Month 1 data has been overwritten by the MM/YY data.
Release Template.xls contains a worksheet, Params, which will be deleted from the release version. This worksheet has a title row and one data row per chart. There are five columns: Sheet Name, Range, Number of Rows, Number of Columns and SQL command.
Sheet Name and Range define the location of the source data for the chart.
Number of Rows and Number of Columns define the size of the range. These values should be generated from the range (or vice versa) but this generation is not difficult and its inclusion would complicate the answer for little advantage.
SQL command is the command to be used to extract the data for the chart from the database. The code below assumes the SQL command generates a Recordset containing data ready to drop into the worksheet.
These parameters could be in the Access database but I believe they fit more logically in the workbook. These parameters control getting data out of the Access database and into the Excel workbook. If a chart is changed such that it requires new data, these parameters must be changed to match but no change is required to the code.
Envelope
When this code was tested, it was within an Access Module. It could probably be transferred to a form but that has not been tested. There MUST be a reference to the "Microsoft Excel 11.0 Object Library".
This envelope should be suitable for any similar problem.
Option Compare Database
Option Explicit
Sub Control()
' This list includes the variables for the envelope and the generation code
Dim DestFileName As String
Dim Path As String
Dim xlApp As Excel.Application
Dim xlWB As Excel.Workbook
' I have my Excel file and my Access database in the same folder.
' This statement gets me the name of the folder holding my database.
' You may need to define a different path.
Path = Application.CurrentProject.Path
' Create path and file name of "Resource YYMM.xls"
DestFileName = Path & "\" & "Resource " & Format(Date, "yymm") & ".xls"
' Create copy of "Resource Template.xls".
FileCopy Path & "\Resource Template.xls", DestFileName
Set xlApp = New Excel.Application
With xlApp
.Visible = True ' This slows the macro but helps with debugging
' .Visible = False
Set xlWB = .Workbooks.Open(DestFileName)
With xlWB
' Code to amend "Resource YYMM.xls" goes here
.Save ' Save the amended workbook
.Close ' Close the amended workbook
End With
Set xlWB = Nothing ' Clear reference to workbook
.Quit ' Quit Excel
End With Set xlApp = Nothing ' Clear reference to Excel
End Sub
Code to generate copy data to workbook
This code assumes it is possible to create SQL statments that will generate Recordsets of data ready to drop into the workbook.
This code has been partially tested. The tests parameters defined ranges in the workbook which matches the size of the parameters. The data loaded into Params() was written to these ranges.
Dim DestSheetName As String
Dim NumCols As Integer
Dim NumRows As Integer
Dim OutData() as Variant
Dim Params() as Variant
Dim RngDest As String
Dim RowParamCrnt As Integer
Dim RowParamMax As Integer
Dim SQLCommand As String
With .Sheets("Params")
' Find last used row in worksheet
RowParamMax = .Cells(Rows.Count,"A").End(xlUp).Row
' Read entire worksheet into array Params
Params = .Range(.Cells(1, 1), .Cells(RowParamMax, 5)).Value
xlApp.DisplayAlerts = False ' Surpress delete confirmation
.Delete ' Delete parameters sheet
xlApp.DisplayAlerts = True
End With
' Params is an array with two dimensions. Dimension 1 is the row.
' Dimension 2 is the column. Loading Params from the range is
' equivalent to:
' ReDim Params( 1 to RowParamMax, 1 to 5)
' Copy data from worksheet to array
For RowParamCrnt = 2 To RowParamMax
DestSheetName = Params(RowParamCrnt, 1)
DestRng = Params(RowParamCrnt, 2)
NumRows = Params(RowParamCrnt, 3)
NumCols = Params(RowParamCrnt, 4)
SQLCommand = Params(RowParamCrnt, 5)
' Use the SQL command to create a Recordset containing the data
' for the chart.
' Check the Recordset's dimensions against NumRows and NumCols
ReDim OutData(1 to NumRows, 1 to NumCols)
' Note (repeat Note): the first dimension is for rows and the
' second dimension is for columns. This is required for arrays
' to be read from or to a worksheet.
' Move the data out of the Recordset into array OutData.
.Sheets(DestSheetName).Range(DestRng).Value = OutData
Next

Excel: Check Sheet Dependencies within a Workbook?

I'm in the process of refactoring a huge workbook woth a lot of legacy parts, redundant computations, cross-dependencies etc.
Basically, I'm trying to remove unneeded sheets and implement some proper information flow within the workbook. Is there a good way to extract the dependencies between the sheets (with VBA)?
Thanks
Martin
You can use ShowPrecedents and NavigateArrow.
here is some pseudocode
for each oCell in oSht containing a formula
ocell.showprecedents
do until nomoreprecedents
i=i+1
Set oPrec = oCell.NavigateArrow(True, 1, i)
If not oPrec.Parent Is oSht Then
' off-sheet precedent
endif
loop
next ocell
I came up with a little sub to do this. It moves all the sheets into seperate workbooks and prints out the dependencies. The advantage over using showPrecedents is that it captures all links including names, embedded forms/diagramms etc.
Word of warning: Moving worksheets isn't undo-able, save your workbook before running this and close (without saving) and re-open afterwards.
Sub printDependencies()
' Changes workbook structure - save before running this
Dim wbs As VBA.Collection, wb As Workbook, ws As Worksheets
Dim i As Integer, s As String, wc As Integer
Set ws = ThisWorkbook.Worksheets
Set wbs = New VBA.Collection
wbs.Add ThisWorkbook, ThisWorkbook.FullName
For i = ws.Count To 2 Step -1
ws(i).Move
wc = Application.Workbooks.Count
wbs.Add Application.Workbooks(wc), Application.Workbooks(wc).FullName
Next
Dim wb As Workbook
For Each wb In wbs
For Each s In wb.LinkSources(xlExcelLinks)
Debug.Print wb.Worksheets(1).Name & "<-" & wbs(s).Worksheets(1).Name
Next
Next
End Sub
The code isn't very polished or user-friendly, but it works.
You can follow the steps at "Find external references that are used in cells" topic of the following link:
Find external references in a worbook
But instead of enter the "[" you should enter the name of the sheet you're trying to find its dependencies. It will display a large list of every single cell referencing the sheet, but at the end it works. Haven't find the way to group by Sheet.

Resources