I want to combine certain line items of data into one tab - excel

I run an audit at work which can return several million line items of data. So excel gives me many tabs of 65,536 line items on each tab. I have the vba code which will combine all data onto one tab but again this won't work because of the row limits for each tab. Therefore I only want certain line items being pulled out.
I only want to include the line items where the Field Name is Position is Using Time. I am new to VBA and assume that I should specify that within the Loop?
So (within the code listed below) I need to include For every time the Field Name equals Position is Using Time I want that entire line item to be dropped into the new Audit Trail tab. I have included an example of the line item I am looking for within each tab, and also a copy of the code I use for combining smaller audit trails.
Any help is very much appreciated! Thank you.
Example of line item I need to pull from the data
Sub CopyFromWorksheets()
Dim wrk As Workbook 'Workbook object
Dim sht As Worksheet 'Object for handling worksheets in loop
Dim trg As Worksheet 'Audit Trail Worksheet
Dim rng As Range 'Range object
Dim colCount As Integer 'Column count in tables in the worksheets
Set wrk = ActiveWorkbook 'Working in active workbook
'Add new worksheet as the last worksheet
Set trg = wrk.Worksheets.Add(After:=wrk.Worksheets(wrk.Worksheets.Count))
'Rename the new worksheet
trg.Name = "Audit Trail"
'Get column headers from the first worksheet
'Column count first
Set sht = wrk.Worksheets(1)
colCount = sht.Cells(1, 255).End(xlToLeft).Column
'Retrieve Headers
With trg.Cells(1, 1).Resize(1, colCount)
.Value = sht.Cells(1, 1).Resize(1, colCount).Value
'Set font as bold
.Font.Bold = True
End With
'Start loop
For Each sht In wrk.Worksheets
'If worksheet in loop is the last one, stop execution (it is Master worksheet)
If sht.Index = wrk.Worksheets.Count Then
Exit For
End If
'Data range in worksheet - starts from 1st row as headers are only in first tab.
Set rng = sht.Range(sht.Cells(1, 1), sht.Cells(65536, 1).End(xlUp).Resize(, colCount))
'Put data into the Master worksheet
trg.Cells(65536, 1).End(xlUp).Offset(1).Resize(rng.Rows.Count, rng.Columns.Count).Value = rng.Value
Next sht
'Fit the columns in Master worksheet
trg.Columns.AutoFit
End Sub

The 64K row limit went away with Excel 2007. It's over a million rows since that version. With Power Pivot (XL 2010 and later), you can load a LOT more rows than that into the data model. Looping through all that data with VBA will be painfully slow. Using modern tools like Power Query, you can combine the data from all sheets and set filters. That will run a LOT faster.

Related

variable referencing a cell date value not passing to a copy paste filter

I have a spreadsheet that operators input data in, with the A column being the date, and the data is input by row. The A column is a formula that adds +1 to the date in the previous cell, going all the way down recursively to auto-populate the date as the sheet is filled out.
I have to have a report printed out at the end of every day, and I am trying to use VBA to filter the rows out by a date that the operator inputs on another sheet in cell B2. I need the macro to grab that date value, and pass it as a variable to the filter in order to pull the 12 rows of that date and paste it into a new sheet. Unfortunately, the value it pulls is not being passed, and when I put a MsgBox command in there, it shows it's pulling 12:00 AM and not a date. When using the Date variable, it also breaks the filter on the bottom macro below (trying 2 different versions just to get this working).
I'm not good with VBA, so my macros were pulled from example websites and I tailored them to what I need.
This is one macro I have tried:
Sub For_RangeCopy()
Dim rDate As Date
Dim rSheet As Worksheet
Set rSheet = ThisWorkbook.Worksheets("EOS")
rDate = CDate(rSheet.Range("B2").Value)
MsgBox (rDate)
' Get the worksheets
Dim shRead As Worksheet
Set shRead = ThisWorkbook.Worksheets("Bi-Hourly Report")
Dim shWrite As Worksheet
Set shWrite = ThisWorkbook.Worksheets("Report")
' Get the range
Dim rg As Range
Set rg = shRead.Range("A1").CurrentRegion
With shWrite
' Clear the data in output worksheet
.Cells.ClearContents
' Set the cell formats
'.Columns(1).NumberFormat = "dd/mm/yyyy"
'.Columns(3).NumberFormat = "$#,##0;[Red]$#,##0"
'.Columns(4).NumberFormat = "0"
'.Columns(5).NumberFormat = "$#,##0;[Red]$#,##0"
End With
' Read through the data
Dim i As Long, row As Long
row = 1
For i = 1 To rg.Rows.Count
If rg.Cells(i, 1).Value2 = rDate Or i = 1 Then
' Copy using Range.Copy
rg.Rows(i).Copy
shWrite.Range("A" & row).PasteSpecial xlPasteValues
' move to the next output row
row = row + 1
End If
Next i
End Sub
And here is another Macro I have tried to use. This one actually gives me the 3 header rows which I don't need, but I don't mind, this paste is a reference for the report layout anyway, so the operators won't see this sheet. But this macro does give me the first block of the date range: 1/1/2023. I do know that the "rgCriteria As String" is likely incorrect, but that is how I get anything useful from this macro. If I change that rgCriteria to a Date, it breaks the rgData.AdvancedFilter command, and I haven't learned enough VBA to know why. And my boss wants this done today, although here I am posting here, thus it's not getting done today.
Sub AdvancedFilterExample()
' Get the worksheets
Dim rSheet As Worksheet
Set rSheet = ThisWorkbook.Worksheets("EOS")
Dim shRead As Worksheet, shWrite As Worksheet
Set shRead = ThisWorkbook.Worksheets("Bi-Hourly Report")
Set shWrite = ThisWorkbook.Worksheets("Report")
' Clear any existing data
shWrite.Cells.Clear
' Remove the any existing filters
If shRead.FilterMode = True Then
shRead.ShowAllData
End If
' Get the source data range
Dim rgData As Range, rgCriteria As String
Set rgData = shRead.Range("A1").CurrentRegion
' IMPORTANT: Do not have any blank rows in the criteria range
'Set rgCriteria = rSheet.Range("B2")
rgCriteria = rSheet.Range("B2").Value
MsgBox (rgCriteria)
' Apply the filter
rgData.AdvancedFilter Action:=xlFilterCopy, CriteriaRange:=rgCriteria _
, CopyToRange:=shWrite.Range("A1")
End Sub
I don't know which method of filtering and pasting is best for my situation, but I do know that the faster is better. I'm copying entire rows, and it needs to be efficient because this log contains a lot of data. I only need one of these macros to work, but I will be heavily modifying them and chaining them together with about 5 other filter/copy/paste sequences to follow, along with printOut commands after that, and finalized by clearing the sheets it pastes to, and then re-enabling all the functionality of the sheet (calculations, displaystatusbar, events, and screenupdating) all to make it quicker while the macro is running. All of these reports will be run using the macro with a button click.
Any thoughts or suggestions would be greatly appreciated. I've been struggling with this for a couple of weeks now. I'm at a loss and turning to the community that has helped me with a TON of questions over the past 20 or so years just by a Google search!
Other information:
I'm using Office 365 on a Windows 10/11 machine. The headers of the sheet it filters does contain merged cells as the header is rows 1-3, there is a lot of data in this sheet that grows through the year. 12 rows per day for an entire year. These macros are written in a Module aptly named "Module 1" if that helps. I do have this workbook, and the original log saved on OneDrive that can be shared.
When using Advanced Filter your criteria range should have headers which match your data table.
Sub AdvancedFilterExample()
Dim rSheet As Worksheet, shRead As Worksheet, shWrite As Worksheet
Dim rgData As Range, rgCriteria As Range
Set rSheet = ThisWorkbook.Worksheets("EOS")
Set shRead = ThisWorkbook.Worksheets("Bi-Hourly Report")
Set shWrite = ThisWorkbook.Worksheets("Report")
Set rgData = shRead.Range("A1").CurrentRegion 'source data range
'## criteria range needs to include a matching date header...
Set rgCriteria = rSheet.Range("B3:B4") 'eg. "Date" in B3, date value in B4
shWrite.Cells.Clear ' Clear any existing data
If shRead.FilterMode = True Then shRead.ShowAllData ' Remove the any existing filters
rgData.AdvancedFilter Action:=xlFilterCopy, _
CriteriaRange:=rgCriteria, _
CopyToRange:=shWrite.Range("A1")
End Sub

Setting value using filtered data

I'm trying to open a sheet (Archive) from my inventory sheet, filter the data in the second sheet and then copy the filtered data to a sheet on the inventory. Everything is working except that the filtered data only copies the data from rows in the first contiguous range. My code is as follows
Dim LastRow As Long
Dim nOoFrOWS As Long
Dim oSht As Worksheet
Workbooks.Open ("C:\Inventory\Archive.xlsm") '<- at opening a workbook it becomes the active one
Set oSht = ActiveWorkbook.Worksheets("Archive") '<-- set the destination worksheet in the activeworkbook
With ActiveSheet
.ListObjects("Archive").Range.AutoFilter Field:=12, Criteria1:=mOrder
nOoFrOWS = .AutoFilter.Range.Columns(1).SpecialCells(xlCellTypeVisible).Cells.Count - 1 '# of rows in Inventory
End With
Unload Me
ThisWorkbook.Sheets("RAM").Range("A2:K" & nOoFrOWS).Value = oSht.Range("Archive[[QTY]:[RTK]]").SpecialCells(xlCellTypeVisible).Cells.Value
oSht.Parent.Close False
What am I doing wrong?
edit: I don't know if it is pertinent, but the range in the archive (from which I am copying) is not the entire table. I have more rows, but These are all I need for this application.
Also, is there a way to do this without the clipboard by using .value or am I stuck with using the copy paste method?
As your working with a table you can copy the visible cells in the databodyrange.
No need to activate or select anything - just work with the referenced files & sheets.
Sub Test()
Dim wrkBk As Workbook
Dim mOrder As Long
mOrder = 5
'You can reference the workbook without it being active.
Set wrkBk = Workbooks.Open("C:\Inventory\Archive.xlsm")
With wrkBk.Worksheets("Archive").ListObjects("Archive")
.Range.AutoFilter Field:=12, Criteria1:=mOrder
'Copy the DataBodyRange (Range would include the headers).
.DataBodyRange.Resize(, 11).SpecialCells(xlVisible).Copy Destination:=ThisWorkbook.Worksheets("RAM").Range("A2")
End With
End Sub

ms excel copy multiple spread out cells in worksheet to separate sheet in rows for multiple sheets

I have a workbook with over 100 separate sheets and
Each sheet is the same form with cells having different data. I need to be able to go through each sheet and copy the cells in the form with data and input it in a
Select sheet of rolls. Each roll would represent the data
From one sheet.
Basicly. Say for example. The form on each of the 100
Worksheets was an address book with the typical data for a simple address book. And each worksheet was exactly the same. How would be able to cycle through each of the 100 sheets and copy the information in the
Cells into one sheet with each row have the data from
The worksheets sepeatery. This way I could transfer the
Info into a database.
Thanks for you time
You can use a macro to do this:
Public Sub CopyToOneSheet()
Dim DestSht As Worksheet
Dim sht As Worksheet
Set DestSht = ThisWorkbook.Worksheets("DestinationSheetName") 'Change to your sheet name where everything will be copied.
'Add a header to your distination sheet
DestSht.Cells(1, 1).Value = "Everything will be copied here."
'Loop through each sheet.
For Each sht In ThisWorkbook.Worksheets
'Not going to copy the destination sheet.
If sht.Name <> DestSht.Name Then
'Copy used range to first available row on destination sheet.
sht.Range("A2:Z" & sht.Cells(Rows.Count, 1).End(xlUp).Row).Copy Destination:=DestSht.Range("A" & DestSht.Cells(Rows.Count, 1).End(xlUp).Row + 1)
End If
Next sht
Set DestSht = Nothing
Set sht = Nothing
End Sub

How to Run a Macro without using Excel

Iam a DB Guy and i dont know anything about VB.
I have a Macro in Excel and in Excel i have cross tabular records.
My macro will convert Crosstabular records to tabular records.
But My requirement is i want to Run the Same Macro outside the excel.
.VBS file should be there and whenever we run the .VBS it should pick excel from some place and convert the crosstab records to tabular records and save at some different location.
I have created a Code for the same by googling and Somebody Please review my below code and help me with the Proper code.
Sub RunMacro()
Dim xlApp 'As Excel.Application
Dim xlBook 'As Workbook
Dim xlSheet 'As Worksheet
Dim wsCrossTab 'As Worksheet
Dim wsList 'As Worksheet
Dim iLastCol 'As Long
Dim iLastRow 'As Long
Dim iLastRowList 'As Long
Dim rngCTab 'As Range 'Used for range in Sheet1 cross tab sheet
Dim rngList 'As Range 'Destination range for the list
Dim I 'As Long
Set xlApp = CreateObject("Excel.Application")
Set xlBook = xlApp.Workbooks.Open("D:\Source.xls")
CrossTabToList()
xlBook.SaveAs "D:\Results.xls"
xlApp.Quit
End Sub
Sub CrossTabToList()
Set wsCrossTab = Worksheets("Tabular")
Set wsList = Worksheets.Add
'Find the last row in Sheet1 with the cross tab
iLastRow = wsCrossTab.Cells(Rows.Count, "A").End(xlUp).Row
'Set the initial value for the row in the destination worksheet
iLastRowList = 2
'Find the last column in Sheet1 with the cross tab
iLastCol = wsCrossTab.Range("A8").End(xlToRight).Column
'Create a new sheet and set the heading titles
wsList.Range("A1:C1") = Array("CATEGORY", "SUBCATEGORY", "VALUE")
'Start looping through the cross tab data
For I = 2 To iLastRow
Set rngCTab = wsCrossTab.Range("A" & I) 'initial value A2
Set rngList = wsList.Range("A" & iLastRowList) 'initial value A2
'Copy individual names in Col A (A2 initially) into as many rows as there are data columns in the cross tab (less 1 for Col A).
rngCTab.Copy rngList.Resize(iLastCol - 1)
'Move up a I rows less one and across one column (using offset function) to select heading row. Copy.
rngCTab.Offset(-(I - 1), 1).Resize(, iLastCol - 1).Copy
'Paste transpose to columns in the list sheet alongside the names
rngList.Offset(0,1).PasteSpecial Transpose:=True
'Staying on same row (2 initially) copy the data from the cross tab
rngCTab.Offset(, 1).Resize(, iLastCol - 1).Copy
'Past transpose as column in list sheet
rngList.Offset(0, 2).PasteSpecial Transpose:=True
'Set the new last row in list sheet to be just below the last name copied
iLastRowList = iLastRowList + (iLastCol - 1)
'increment I by 1
Next I
Application.DisplayAlerts = False
Sheets("Tabular").Select
ActiveWindow.SelectedSheets.Delete
Application.DisplayAlerts = True
Sheets("Sheet1").Select
Sheets("Sheet1").Name = "Results"
objwkbk.SaveAs "D:\Results.xls"
End Sub
Thanks,
Praveen
As i mentioned i am not a Java Developer or Coding guy,i am a Database person ,i dont know anything about Java .I want to use the above code as .VBS file.I want somebody to correct my above code to use it in a .VBS File.If you can do that it will be really appreciated.
Thanks in Advance.
That's a very good idea. VBA in an Excel file can confuse users, so I try to avoid that whenever possible.
I recommend storing your procedure in an Access file. There's a little work involved in converting it, but this should get you started:
Make a new Access db
In your new db, make a new VBA module. Paste your code in there.
Add your most current version of Microsoft Excel Object Library.
Make whatever other changes are necessary to get the code in working order again (you'll have to do a bit of trial and error. Run the code repeatedly and deal with the error messages as they pop up)
Change your Sub to a Function (you need to do this to call it from a Macro)
Make a new Macro. Add the action RunCode with the argument RunMacro()
In the future, all you will have to do is open the db and click on the macro to run the code.

Excel VBA: Moving Columns between Workbooks

Currently I have spreadsheets coming in that are formatted incorrectly. Our client sent out to his suppliers an old spreadsheet where columns are laid out differently than what they are currently setup as. Normally we would tell them to correct it, but some of these spreadsheets have over 220k rows and 33 columns. They're updating it for the future, but asking them to have their clients redo their tables is a no-go. I've written a script that will copy a column, and place it into the corresponding static column in another workbook. This works okay but I feel there is more that could be done.
Name of open workbook copying from varies.
Name of workbook copied to: C:\User\(Name)\UCOR\Catalogs\PSX-Toolset v1.503-EN.xls
What I would like is help writing a macro that will do the following from open workbook:
1.) Select an entire column minus Row 1 to the first blank row. - This goes from B to AH
2.) Paste that column into PSX-Toolset workbook, worksheet name "Item Data" - Static Assigned Columns
3.) Perform a Save As on PSX-Toolset as (Catalog-PSX-<Workbook Copied From>.xls)
Lastly, I'd like to know if it's possible to do the above, but mapping heading cells. Unfortunately the cell names are not identical.
Untested:
Sub MapAndCopyColumns()
Dim i As Integer, rng As Range
Dim shtSrc As Worksheet, wbDest As Workbook
Dim shtDest As Worksheet
Dim iNew
Set shtSrc = ActiveSheet
Set wbDest = Workbooks.Open("C:\User\(Name)\UCOR\Catalogs\PSX-Toolset v1.503-EN.xls")
Set shtDest = wbDest.Sheets("Item Data")
For i = 2 To 34
Set rng = shtSrc.Cells(2, i)
If rng.Value <> "" Then
If rng.Offset(1, 0).Value <> "" Then
Set rng = Range(rng, rng.End(xlDown))
End If
'map old position >> new position
' mapping table has 2 columns of numbers: "old" and "new"
iNew = Application.VLookup(i, _
ThisWorkbook.Sheets("Mapping").Range("A2:B40"), 2, False)
If Not IsError(iNew) Then
'copy if the column has an entry in the mapping table
rng.Copy shtDest.Cells(2, iNew)
End If
End If
Next i
wbDest.SaveAs "C:\wheretosaveto\Catalog-PSX-" & shtSrc.Parent.Name
End Sub
How I learned most of my vba is through 'record macro'. You start recording, do what you want to do yourself, stop recording and then look at the generated code.
Usually you can improve the code by eliminating a lot of redundant lines, but it should at least expose all the commands you need to complete your goal.

Resources