EXCEL VBA: Mapping of data is too slow - excel

I am running sub where it compares two cells (B and D/or string Received) from one sheet ("DATA") with two cells (C, H) from another sheet ("Incoming_report"), and if they match it transposes I, G cells from Incoming to Data.
It is done by combining two cells from Incoming_report sheet and writing new value in Z column for example "123456" from C and H to f.e. "123456Received" (there another 5 statuses (Received, Rejected, Sent...., but I need the ones only that was Received)
Then I am taking from Data Sheet B column for example 123456 and only Received (there might be another 5 statuses, but I only need the one that was received).
That makes all sence to me and works pretty good, but I have to work with more than 500k rows in each sheet. What happens - 500,000 times two cells are combined and searched in Z column in another sheet among another 500,000 for possible match, if nothing found then N/A, and then 2 combination, 3rd, 4th... till 500,000. I added the Display status bar and I see how slowly it goes (only 900 rows per minute, so for one minor mapping it would take more than 10 hours). Here is the sub itself, can anyone share ideas how to improve it to make it work faster? Thanks a million.
Sub incoming_fetch()
Application.ScreenUpdating = False
Dim incr As Long
Dim x As String
n = Sheets("Data").Range("A" & Rows.Count).End(xlUp).Row
Z = Sheets("Incoming_report").Range("D" & Rows.Count).End(xlUp).Row
For i2 = 2 To Z
Sheets("Incoming_report").Range("Z" & i2).Value = Sheets("Incoming_report").Range("C" & i2).Value & Sheets("Incoming_report").Range("H" & i2).Value
Next i2
For i = 3 To n
Application.DisplayStatusBar = True
Application.StatusBar = i
x = Sheets("Data").Range("B" & i).Value & "Received"
If Sheets("Incoming_report").Range("Z:Z").Find(x, lookat:=xlWhole, LookIn:=xlFormulas) Is Nothing Then
Sheets("Data").Range("L" & i) = "N/A"
Sheets("Data").Range("M" & i) = "N/A"
Else
incr = Sheets("Incoming_report").Range("Z:Z").Find(x, lookat:=xlWhole, LookIn:=xlFormulas).Row
Sheets("DATA").Range("L" & i) = Sheets("Incoming_report").Range("I" & incr)
Sheets("DATA").Range("M" & i) = Sheets("Incoming_report").Range("G" & incr)
End If
Next i
End Sub

EDIT2: fixed source columns :
Sub incoming_fetch()
Dim i As Long, n As Long, z As Long, num As Long
Dim x As String
Dim shtIn As Worksheet, shtData As Worksheet
Dim dict As Object, arrC, arrH, arrG, arrI, v, arr, r1, r2
Dim t
Set dict = CreateObject("scripting.dictionary")
Set shtIn = Sheets("Incoming_report")
Set shtData = Sheets("Data")
n = shtData.Range("A" & Rows.Count).End(xlUp).Row
z = shtIn.Range("D" & Rows.Count).End(xlUp).Row
t = Timer
'get all values from Cols C, H, L, M
arrC = shtIn.Range(shtIn.Range("C2"), shtIn.Range("C" & z)).Value
arrH = shtIn.Range(shtIn.Range("H2"), shtIn.Range("H" & z)).Value
arrG = shtIn.Range(shtIn.Range("G2"), shtIn.Range("G" & z)).Value
arrI = shtIn.Range(shtIn.Range("I2"), shtIn.Range("I" & z)).Value
Debug.Print "Get Arrays: " & Timer - t
t = Timer
'create a lookup dictionary of all the ColC values
' (where ColH = "Received")
num = UBound(arrC, 1)
For i = 1 To num
v = arrC(i, 1)
If arrH(i, 1) = "Received" And Len(v) > 0 Then
dict(v) = Array(arrI(i, 1), arrG(i, 1))
End If
Next i
'free up some memory
Erase arrC: Erase arrH: Erase arrI: Erase arrG
Debug.Print "Filled dict: " & Timer - t
t = Timer
Application.ScreenUpdating = False
Application.Calculation = xlCalculationManual
On Error GoTo haveError
For i = 3 To n
If i Mod 500 = 0 Then Application.StatusBar = i
x = shtData.Range("B" & i).Value
If dict.exists(x) Then
arr = dict(x)
r1 = arr(0)
r2 = arr(1)
Else
r1 = "N/A": r2 = "N/A"
End If
With shtData
.Range("L" & i) = r1
.Range("M" & i) = r2
End With
Next i
Debug.Print "Done: " & Timer - t
haveError:
Application.ScreenUpdating = True
Application.Calculation = xlCalculationAutomatic
End Sub

Related

Need help in speeding up a looping macro

I am using a Macro to identify same details entered (in Column B, C and D) for multiple students (Column A).
This macro is working fine on approx 10,000 rows. But when I use it on full database ex: 1,00,000 rows it just got hanged.
Any help to in speeding up this macro will be very helpful.
`
Option Explicit
Sub test()
Dim lRow As Long, i As Long, j As Long
Dim c As Integer
Dim students() As Variant
Dim results() As Variant
lRow = Cells(Rows.Count, 1).End(xlUp).Row
ReDim results(lRow - 1)
students = Range("A1:D" & lRow).Value
For i = 2 To lRow
found = False
For j = 2 To lRow
If students(i, 1) = students(j, 1) Then GoTo skipStudent
For c = 1 To 4
If students(j, c) = "-" Or students(i, c) = "-" Then GoTo skipColumn
If students(i, c) = students(j, c) Then
results(i - 1) = results(i - 1) & "Same " & students(1, c) & " information with " & students(j, 1) & vbCrLf
End If
skipColumn:
Next
skipStudent:
Next
Next
Application.ScreenUpdating = False
For i = 2 To lRow
Cells(i, 5).Value = results(i - 1)
Next
Application.ScreenUpdating = True
End Sub
`

Copy-Paste above row's Range if a specific range is empty and another is not

I have a table in an active worksheet.
I am trying to:
Scan Columns(A:M) of Row 6 to see if all cells are empty
If yes, then scan Columns (N:R) of Row 6 to see if all cells are empty
If 2. is false, then copy above row's Columns (A:I) in Row 6
Repeat 1-3 but on Row 7
This process should repeat until the rows of the table end.
I would like to incorporate ActiveSheet.ListObjects(1).Name or something similar to duplicate the sheet without having to tweak the code.
How I can make this as efficient and as risk free as possible? My code works but it's really too much.
Sub CopyPasteRow()
Dim lr As Long
Dim x As Long
Dim y As Long
Dim a As Long
lr = Cells(Rows.Count, 1).End(xlUp).Row
a = 0
For x = 6 To lr
For y = 1 To 13
If Not IsEmpty(Cells(x, y)) Then
a = a + 1
End If
Next y
If a = 0 Then
For y = 14 To 18
If Not IsEmpty(Cells(x, y)) Then
a = a + 1
End If
Next y
Else
a = 0
End If
If a <> 0 Then
For y = 1 To 13
Cells(x, y).Value = Cells(x - 1, y).Value
Next y
End If
a = 0
Next x
End Sub
This is the final code based on #CHill60 code. It got me 99% where I wanted.
Sub CopyPasteRow()
Dim lr As Long
Dim x As Long
Dim a As Long
Dim r As Range, r2 As Range, r3 As Range
lr = Cells(Rows.Count, 1).End(xlUp).Row
For x = 6 To lr
'check columns A to M for this row are empty
Set r = ActiveSheet.Range("A" & CStr(x) & ":M" & CStr(x))
'check columns N to R for this row are empty
Set r2 = ActiveSheet.Range("N" & CStr(x) & ":R" & CStr(x))
If WorksheetFunction.CountA(r) = 0 And WorksheetFunction.CountA(r2) <> 0 Then
'copy the data into columns A to M
Set r3 = ActiveSheet.Range("A" & CStr(x) & ":I" & CStr(x))
r3.Value = r3.Offset(-1, 0).Value
End If
Next x
End Sub
Instead of looking at individual cells, look at Ranges instead. Consider this snippet of code
Sub demo()
Dim x As Long
For x = 6 To 8
Dim r As Range
Set r = Sheets(1).Range("A" & CStr(x) & ":M" & CStr(x))
Debug.Print r.Address, MyIsEmpty(r)
Next x
End Sub
I have a function for checking for empty ranges
Public Function MyIsEmpty(rng As Range) As Boolean
MyIsEmpty = WorksheetFunction.CountA(rng) = 0
End Function
I use this because the cell might "look" empty, but actually contain a formula.
Note I've explicitly said which sheet I want the Cells from - users have a habit of clicking places other than where you think they should be! :laugh:
Edit after OP comment:
E.g. your function might look like this
Sub CopyPasteRow()
Dim lr As Long
Dim x As Long
Dim a As Long
Dim r As Range, r2 As Range
lr = Cells(Rows.Count, 1).End(xlUp).Row
For x = 6 To lr
a = 0
'check columns A to M for this row are empty
Set r = Sheets(1).Range("A" & CStr(x) & ":M" & CStr(x))
If Not MyIsEmpty(r) Then
a = a + 1
End If
If a = 0 Then
'check columns N to R for this row are empty
Set r2 = Sheets(1).Range("N" & CStr(x) & ":R" & CStr(x))
If Not MyIsEmpty(r2) Then
a = a + 1
End If
Else
a = 0
End If
If a <> 0 Then
'copy the data into columns A to M
'You might have to adjust the ranges here
r.Value = r2.Value
End If
Next x
End Sub
where you have a source range and a target range - you appear to be putting the values in the previous row so my value of r is probably wrong in this example - you could use r.Offset(-1,0).Value = r2.Value
I'm also not sure what you are trying to do with the variable a If that is meant to be a "flag" then consider using a Boolean instead - it only has the values True or False

Copy value and paste under matching column near respective row

I have a column with certain values which are also the headers for some columns. I want to check where the column values match and paste the value from the first column into the column with the same column name. I have around 1200 values in the first column. I want to loop through those values and paste the matching values in the corresponding row.
[![Data][1]][1]
Here is my sheet with my data that I want to work on. How I want my final sheet to look like is as follows:
Weeks | W1 | W2 | W3 | W4 | W5 | W6
W1 W1
W3 W3
Any help for the same would be highly appreciated.
Sub Weeks()
Application.ScreenUpdating = False
Dim i As Long, j As Long, Mas As Worksheet
Set Mas = Sheets("Master Sheet")
For i = 5 To 1200
If Mas.Range("B" & i) <> "" Then
If Mas.Range("AO" & i) = "Missing week" Then
Mas.Range("AV" & i) = ""
Mas.Range("AW" & i) = ""
Mas.Range("AX" & i) = ""
Mas.Range("AY" & i) = ""
Mas.Range("AZ" & i) = ""
Mas.Range("BA" & i) = ""
Else
For j = 5 To 1200
If Mas.Range("AO" & i) = "W1" Then
Mas.Range("AV" & j) = "W1"
GoTo Nexti
ElseIf Mas.Range("AO" & i) = "W2" Then
Mas.Range("AW" & j) = "W2"
GoTo Nexti
ElseIf Mas.Range("AO" & i) = "W3" Then
Mas.Range("AX" & j) = "W3"
GoTo Nexti
ElseIf Mas.Range("AO" & i) = "W4" Then
Mas.Range("AY" & j) = "W4"
GoTo Nexti
ElseIf Mas.Range("AO" & i) = "W5" Then
Mas.Range("AZ" & j) = "W5"
GoTo Nexti
ElseIf Mas.Range("AO" & i) = "W6" Then
Mas.Range("BA" & j) = "W6"
GoTo Nexti
End If
Next j
End If
End If
Nexti:
Next i
End Sub
This is the code I tried so far but it does not show any output.
This is how you use the dictionary to achieve your goal:
Option Explicit
Sub Weeks()
Dim Mas As Worksheet: Set Mas = ThisWorkbook.Sheets("Master Sheet")
With Mas
'Calculate thelast row
Dim i As Long
Dim LastRow As Long: LastRow .Cells(.Rows.Count, "AO").End(xlUp).Row
'insert your data into an array
Dim arr As Variant: arr = .Range("AO5:BA" & LastRow).Value
'Generate a dictionary with the headers
'this needs the library Microsoft Scripting Runtime under Tools->References
Dim Headers As Dictionary: Set Headers = LoadHeaders(.Range("AO4:BA4").Value)
'Now loop through the array
For i = 1 To UBound(arr)
If Headers.Exists(arr(i, 1)) Then arr(i, Headers(arr(i, 1))) = arr(i, 1)
arr(i, 1) = vbNullString
Next i
.Range("AO5:BA" & LastRow).Value = arr
End With
End Sub
Private Function LoadHeaders(arr As Variant) As Dictionary
Set LoadHeaders = New Dictionary
Dim i As Long
For i = 1 To UBound(arr, 2)
LoadHeaders.Add arr(1, i), i
Next i
End Function
You won't even need the Application.ScreenUpdating because it does only one operation in Excel, will take a second or two to end this procedure.
Place this code in a module and link it to a button on the sheet.
'data
Cells.Clear
wks = Array("W1", "W2", "W3", "W4", "W5", "W6", "Missing week")
For i = 1 To 30
Cells(i, 2) = wks(Int(Rnd * 7))
Next i
'code
Set weeks = [b:b]
For Each wk In weeks
If Len(wk) = 2 Then Cells(wk.Row, Right(wk, 1) + 2) = wk
Next wk
weeks contains the column that has the list of column headings. The For..Each statement loops through each entry in this list.
Then it checks each entries string length. If it is length 2, it assumes the entry is valid (i.e of the form 'Wx' with x between 1 and 6), and then uses the inbuilt Right function to find the value of x, and then adds the appropriate entry into the appropriate column.

Adding recursive loop/function into a sort in VBA

I have a bubble sort that only works with the first element.
This is solved by reevaluating my array elements and placing them accordingly, which happens if I run the whole thing time and time again.
I'd like to add a recursive loop that's set to break when the sort is done.
I tried adding a function, but I'm not solid enough on my syntax to combine it with my sub. What is a basic recursion loop for this code? Function not expressly required, just something that will let me recall my sub.
Private Sub SortEverything_Click()
Dim everything() As Range
Dim check As Range
Dim count As Range
Dim sorting As Range
Dim holder As Range
Dim middleman As Range
Dim firstman As Range
Dim Temp1 As String
Dim Temp2 As String
Dim lr As Long
Dim x As Long
Dim y As Long
Dim z As Long
Dim q As Long
Dim everyrow As Long
Dim everycol As Long
Dim firstrow As Long
Dim firstcol As Long
y = 0
z = 0
q = 0
With ThisWorkbook.Sheets("Names and Vendors")
lr = .Cells(.Rows.count, "B").End(xlUp).Row
'Counts number of RMs to size the "everything" array
For z = 2 To lr
Set count = .Range("B" & z)
If IsEmpty(count) = False Then
count.Select
q = q + 1
End If
Next z
ReDim everything(q - 1) As Range 'Resizes array
'Loops all RM info into array by each distinct range
For x = 2 To lr
Set check = .Range("A" & x & ":H" & x)
'ensures subcomponents are added to range
If IsEmpty(.Range("B" & 1 + x)) = True Then
Do While IsEmpty(.Range("B" & 1 + x)) = True And x < lr
Set check = Union(check, .Range("A" & 1 + x & ":H" & 1 + x))
check.Select
x = x + 1
Loop
End If
Set everything(y) = check
y = y + 1
check.Select
Next x
'This For has been commented out so that it doesn't run more than once
'For y = 0 To q - 1
'sorting allows us to copy/paste into a helper range line-by-line as the program loops
'firstman is the helper range. firstrow and firstcol return the dimensions of the everything(y) so that we can resize things
Set sorting = everything(0)
Set firstman = .Range("B20")
Set firstman = firstman.Resize(sorting.Rows.count, sorting.Columns.count)
firstman.Value = sorting.Value
firstrow = firstman.Rows.count
firstcol = firstman.Columns.count
'Returns the name of the RM listed to compare to the one below it
sorting.Offset(0, 1).Select
ActiveCell.Select
Temp1 = "" & ActiveCell.Value
For x = 1 To q - 1
'Checks whether a selected component has subcomponents and identifies its dimensions
sorting.Select
Set holder = everything(x)
holder.Offset(0, 1).Select
everyrow = Selection.Rows.count
everycol = Selection.Columns.count
'Returns the name of the material being compared to the referenced material in everything(y)
ActiveCell.Select
Temp2 = "" & ActiveCell.Value
If Temp2 > Temp1 Then 'If the RM we're on comes alphabetically after the name of the one we're checking against, then
If everyrow > 1 Then 'Handles if everything(x) has subcomponents
'Resize the other helper range to be the same as the range with subcomponents and paste the values into it
Set middleman = .Range("A1").Offset(0, everything(x).Columns.count)
Set middleman = middleman.Resize(everyrow, everycol)
middleman.Select
middleman.Value = holder.Value
'Resize the range we're pasting into in the master table so it can take the new range, then paste
Set sorting = sorting.Resize(everyrow, everycol)
sorting.Select
sorting.Value = holder.Value
'Resize the holder column to the same size as everything(y).
'Then paste everything(y) into the space BELOW the one we've just shifted upwards
Set holder = holder.Resize(firstrow, firstcol)
Set holder = holder.Offset(everyrow - 1, 0)
holder.Select
holder.Value = firstman.Value
Set sorting = sorting.Offset(everyrow, 0)
Else
Set middleman = .Range("A1").Offset(0, everything(x).Columns.count)
Set middleman = middleman.Resize(firstrow, firstcol)
middleman.Select
middleman.Value = holder.Value
Set sorting = sorting.Resize(everyrow, everycol)
sorting.Select
sorting.Value = holder.Value
Set holder = holder.Resize(firstrow, firstcol)
'Set firstman = firstman.Resize(everyrow, everycol)
holder.Select
holder = firstman.Value
Set sorting = sorting.Offset(1, 0)
End If
End If
Next x
'Next y
'This is where my inexperience shows. The recursion should go here, but I'm not sure how to do so.
'PopulateArray (everything)
End With
End Sub
Public Function PopulateArray(myArray()) As Variant
Dim myArray() As Range
Dim check As Range
Dim count As Range
Dim sorting As Range
Dim holder As Range
Dim middleman As Range
Dim firstman As Range
Dim Temp1 As String
Dim Temp2 As String
Dim lr As Long
Dim x As Long
Dim y As Long
Dim z As Long
Dim q As Long
y = 0
z = 0
q = 0
With ThisWorkbook.Sheets("Names and Vendors")
lr = .Cells(.Rows.count, "B").End(xlUp).Row
'Counts number of RMs to size the "myArray" array
For z = 2 To lr
Set count = .Range("B" & z)
If IsEmpty(count) = False Then
count.Select
q = q + 1
End If
Next z
ReDim myArray(q - 1) As Range 'Resizes array
'Loops all RM info into array by each distinct range
For x = 2 To lr
Set check = .Range("A" & x & ":H" & x)
'ensures subcomponents are added to range
If IsEmpty(.Range("B" & 1 + x)) = True Then
Do While IsEmpty(.Range("B" & 1 + x)) = True And x < lr
Set check = Union(check, .Range("A" & 1 + x & ":H" & 1 + x))
check.Select
x = x + 1
Loop
End If
Set myArray(y) = check
y = y + 1
check.Select
Next x
End With
End Function
Found out what I needed to do. Put the whole thing under a Do loop and then added the following lines to it:
'checking to see if array is completely alphabetized
For Each cell In .Range("B2:B" & lr)
'Returns first check value
If IsEmpty(cell) = False Then
cell.Select
check1 = "" & cell.Value
x = cell.Row
.Range("A14").Value = check1
'Returns next check value
For z = x + 1 To lr
Set checking = .Range("B" & z)
If IsEmpty(checking) = False Then
checking.Select
check2 = "" & .Range("B" & z).Value
.Range("A15").Value = check2
Exit For
End If
Next z
Else
End If
If check2 > check1 Then
Exit For
End If
Next cell
'If the last two values are sorted, then the whole thing is sorted and we can stop the recursion
If check2 < check1 Or check1 = check2 Then
Exit Do
End If

Index/Match in VBA

I tried several ways, but I still have a problem in my code.
What I want to do (this example in Q2 on Sheet4):
=INDEX('Sheet8'!K:K,MATCH('Sheet4'!P2,'Sheet8'!A:A,0))
I'd like to do it for all rows with content in column K on Sheet 4 so I'll probably need "For i = 1..."
What I tried:
For i = 1 To LastRowShort
row_mtch = Application.WorksheetFunction.Match(Sheet4.Cells("Q????").Value, Sheet8.Range("A1:A"), 0)
Sheet4.Range("R" & i).Value = Application.WorksheetFunction.Index(Sheet8.Range("K1:K" & LastRowShort), row_mtch)
Next i
Thanks a lot!
Andy
Entire Module:
Sub MissingBoth()
Application.ScreenUpdating = False
Dim MyRange, CopyRange As Range
Dim LastRow As Long
Dim LastRowSheet4 As Long
Dim LastRowSheet8 As Long
Set src4 = Sheet2
Set dst4 = Sheet4
LastRow = src4.Cells(Cells.Rows.Count, "D").End(xlUp).Row
LastRowSheet8 = Worksheets("Sheet8").Cells(Cells.Rows.Count, "B").End(xlUp).Row
LastRowSheet4 = Worksheets("Sheet4").Cells(Cells.Rows.Count, "K").End(xlUp).Row
src4.Unprotect
dst4.Unprotect
If src4.FilterMode = True Then
src4.ShowAllData
End If
dst4.Cells.ClearFormats
dst4.Cells.Clear
'Find content in the "Type of Rack" cells
j = 3
For i = 10 To LastRow
If src4.Cells(i, "CL").Value = "" And src4.Cells(i, "GV").Value = "" Then
src4.Cells(i, "CL").EntireRow.Copy dst4.Cells(j, 1)
j = j + 1
End If
Next i
src4.Range("A6:GW7").Copy Destination:=dst4.Range("A1:GW2")
'Copy every column EXCEPT the following
dst4.Range("GW1,CM1:GU1, U1:CK1,R1:S1,P1,J1:M1").EntireColumn.Delete
For i = 1 To LastRowSheet4
For i2 = 1 To LastRowSheet8
If Worksheets("Sheet4").Range("P" & i).Value = Worksheets("Sheet8").Range("A" & i2).Value Then
Worksheets("Sheet4").Range("Q" & i).Value = Worksheets("Sheet8").Range("K" & i2).Value
End If
Next i2
Next i
dst4.Columns("A:AX").EntireColumn.AutoFit
dst4.Rows("1:500").RowHeight = 15
dst4.Columns("N:O").Interior.Color = vbYellow
dst4.Rows("1:2").Interior.ColorIndex = 15
dst4.Range("B:I").EntireColumn.Hidden = True
Application.ScreenUpdating = True
End Sub
Have you tried something like the following code:
For i = 1 To LastRowSheet4
For i2 = 1 To LastRowSheet8
If Worksheets("Sheet4").Range("P" & i).Value = Worksheets("Sheet8").Range("A" & i2).Value Then
Worksheets("Sheet4").Range("Q" & i).Value = Worksheets("Sheet8").Range("K" & i2).Value
End If
Next i2
Next i
You will just need to define the upper ends of both sheets (LastRowSheet4 and LastRowSheet8) and this should work.
Thanks for your help. I solved the problem with recording a macro and modifying it:
Sheet4.Cells(3, 17).FormulaR1C1 = _
"=INDEX('TS-48 Matrix'!C[-7],MATCH('Missing Both'!RC[-1],'TS-48 Matrix'!C[-16],0))"
Range("Q3").AutoFill Destination:=Range("Q3:Q" & lastRowSheet4)

Resources