I have a code that works as following:
Refresh a query that has roughly 10.000 rows.
Split the data in various 2D Arrays, depending on some criteria (I have in total a dozen different 2D Arrays).
Paste each 2D Array in a different sheet.
When I run the macro the first time it takes ~18 seconds, on the second run ~30 seconds, on the third run ~35 seconds, on the fourth run ~45 seconds and so on. The data from the query is exactly the same at every run (it doesn't change so frequently, or it can change of at most one or two lines).
Can somebody explain me if there are some kind of memory issues I'm not aware of? I also tried to set the matrix equal to Nothing after is pasted on the sheet but this didn't change the slowdown at every run. Any help would be appreciated.
P.S. the code is long and I feel it would be useless to paste it all here. It works more or less as follows:
Sub GetMatrix()
Dim Matrix As Variant, IndexMatrix As Long, i As Long, NoRows As Long
IndexMatrix = 0
ReDim Matrix(IndexMatrix, 2)
NoRows = Application.CountA(Range("A:A"))
For i = 2 To NoRows
If Cells(i, 1) = "Something" Then
Matrix(IndexMatrix, 0) = "Something"
Matrix(IndexMatrix, 1) = "Something"
Matrix(IndexMatrix, 2) = "Something"
IndexMatrix = IndexMatrix + 1
ReDim Preserve Matrix(IndexMatrix, 2)
End If
Next
GetMatrix = Matrix
End Sub
Try the next adapted function, please:
Function GetMatrix() As Variant
Dim sh As Worksheet, arr As Variant, Matrix As Variant
Dim IndexMatrix As Long, i As Long, LastRow As Long
Set sh = ActiveSheet
LastRow = sh.Range("A" & Rows.count).End(xlUp).Row
arr = sh.Range("A1:A" & LastRow).Value
ReDim Matrix(2, UBound(arr)) 'to admit redim preserve (only on the last dimension) at the end
For i = 2 To LastRow
If arr(i, 1) = "Something" Then
Matrix(0, IndexMatrix) = "Something"
Matrix(1, IndexMatrix) = "Something"
Matrix(2, IndexMatrix) = "Something"
IndexMatrix = IndexMatrix + 1
End If
Next
ReDim Preserve Matrix(2, IndexMatrix - 1)
GetMatrix = Matrix
End Function
Related
I have my column B with Strings like "6L2AAB".
My column D with Strings like "E3"
I'd like to produce in my column J the concatenation of B&D, for instance "6L2AABE3", for each row
My code throws a "13" error
.Range("J:J").Value = .Range("B:B").Value & "" & .Range("D:D").Value
Is there a way to do this without a loop ? Thanks
Edit: added loop-based approach to compare timing. Loop is faster!
Your code doesn't work because (eg) .Range("B:B").Value returns a 2-dimensional array, and you can't concatenate the contents of two arrays using &
You can use the worksheet's Evaluate method:
Sub tester()
Dim t, i As Long, arr1, arr2, arr3, x As Long
t = Timer
With ActiveSheet
.Range("J:J").Value = .Evaluate("=B:B&D:D")
End With
Debug.Print "Evaluate", Timer - t
t = Timer
With ActiveSheet
arr1 = .Range("B:B").Value 'read input values
arr2 = .Range("D:D").Value
ReDim arr3(1 To UBound(arr1), 1 To 1) 'size array for output
For i = 1 To UBound(arr1, 1) 'loop and concatenate
arr3(i, 1) = arr1(i, 1) & arr2(i, 1)
Next i
.Range("J:J").Value = arr3 'populate output to sheet
End With
Debug.Print "Loop", Timer - t
End Sub
Maybe don't run it on the whole column unless you really need that though.
#Tim Williams It really depends on the number of rows, if you are working with larger amounts of data loops start to slow slightly and evaluate will jump ahead in processing speed just slightly. But there is another option that is faster than both, the .formula method. See Tim's code with it added in below using > 1 million rows with .formula added.
Average for me:
Formula 1.664063
Evaluate 3.675781
Loop 3.824219
Sub tester()
Dim t, i As Long, arr1, arr2, arr3, x As Long
t = Timer
With ActiveSheet
.Range("C:C").Formula = "=A:A&B:B"
End With
Debug.Print "Formula", Timer - t
t = Timer
With ActiveSheet
.Range("C:C").Value = .Evaluate("=A:A&B:B")
End With
Debug.Print "Evaluate", Timer - t
t = Timer
With ActiveSheet
arr1 = .Range("A:A").Value 'read input values
arr2 = .Range("B:B").Value
ReDim arr3(1 To UBound(arr1), 1 To 1) 'size array for output
For i = 1 To UBound(arr1, 1) 'loop and concatenate
arr3(i, 1) = arr1(i, 1) & arr2(i, 1)
Next i
.Range("C:C").Value = arr3 'populate output to sheet
End With
Debug.Print "Loop", Timer - t
End Sub
I am doing some code to put a range into an array so i can create plots by analyzing the data inside that array. I am trying to use a general code for the range since the input can be different depending on the type of analysis i want to perform. Tried to find a solution for this in other questions without success.
Dim DieBankArray As Variant
last_row = Sheets("Tabela CT geral").Range("A2").End(xlDown).Row 'Last row of the data set
For i = 0 To last_row - 2 '-2 to exclude the first line and another value because the array first position is 0, not 1
DieBankArray(i) = Range("A" & i + 2)
Next
The return is a type mismatch error that i can't understand...
Here's one approach:
Function RangeTo1DArray(rngStart As Range)
Dim rv(), arr, r As Long, n As Long
'read the source data to an array for better performance
With rngStart.Parent
arr = .Range(rngStart, .Cells(Rows.Count, rngStart.Column).End(xlUp)).Value
End With
n = UBound(arr, 1)
ReDim rv(0 To n - 1)
'Fill the output array. Note: purposefully not using transpose()
' to avoid its limitations
For r = 1 To n
rv(r - 1) = arr(r, 1)
Next r
RangeTo1DArray= rv
End Function
Ok, i used the Redim and it worked just fine.
What i couldn't understand is that there's a need to set the correct size of an array to read/write data. I thought a simple Dim as Variant should be enough to store the data at my will without need to set a correct size each time i want to use an array.
The code after ReDim:
Dim DieBankArray As Variant
last_row = Sheets("Tabela CT geral").Range("A2").End(xlDown).Row 'Last row of the data set
ReDim DieBankArray(A2 To last_row - 2)
For i = 0 To last_row - 2 '-2 to exclude the first line and another value because the array first position is 0, not 1
DieBankArray(i) = Range("A" & i + 2)
Next
I am attempting to run a VBA macro that iterates down about 67,000 rows with 100 columns in each row. For each of the cells in these rows, the value is compared against a column with 87 entries in another sheet. There are no errors noted when the code is run but Excel crashes every time. The odd thing is that the code seems to work; I have it set to mark each row in which a match is found and it does so before crashing. I have attempted to run it many times and it has gotten through between 800 and 11,000 rows before crashing, depending on the attempt.
My first suspect was memory overflow due to the volume of calculations but my system shows CPU utilization at 100% and memory usage around 50% while running this code:
Sub Verify()
Dim codes As String
Dim field As Object
For i = 2 To Sheets("DSaudit").Rows.Count
For Each field In Sheets("Dsaudit").Range(Cells(i, 12), Cells(i, 111))
r = 1
While r <= 87
codes = ThisWorkbook.Sheets("287 Denominator CPT").Cells(r, 1).Value
If field = codes Then
Cells(i, 112).Value = "True"
r = 88
Else
r = r + 1
End If
Wend
Next field
i = i + 1
Next i
End Sub
It should also be noted that I am still very new to VBA so it's likely I've made some sort of egregious rookie mistake. Can I make some alterations to this code to avoid a crash or should I scrap it and take a more efficient approach?
When ever possible iterate variant arrays. This limits the number of times vba needs to access the worksheet.
Every time the veil between vba and Excel is pierced cost time. This only pierces that veil 3 times not 9,031,385,088
Sub Verify()
With Sheets("DSaudit")
'Get last row of Data
Dim lastrow As Long
lastrow = .Cells(.Rows.Count, 12).End(xlUp).Row 'if column 12 ends before the last row of data change to column that has them all.
'Load Array with input Values
Dim rng As Variant
rng = .Range(.Cells(2, 12), .Cells(lastrow, 111)).Value
'Create output array
Dim outpt As Variant
ReDim outpt(1 To UBound(rng, 1), 1 To 1)
'Create Match array
Dim mtch As Variant
mtch = Worksheets("287 Denominator CPT").Range("A1:A87").Value
'Loop through first dimension(Row)
Dim i As Long
For i = LBound(rng, 1) To UBound(rng, 1)
'Loop second dimension(Column)
Dim j As Long
For j = LBound(rng, 2) To UBound(rng, 2)
'Loop Match array
Dim k As Long
For k = LBound(mtch, 1) To UBound(mtch, 1)
'If eqaul set value in output and exit the inner loop
If mtch(k, 1) = rng(i, j) Then
outpt(i, 1) = "True"
Exit For
End If
Next k
'If filled true then exit this for
If outpt(i, 1) = "True" Then Exit For
Next j
Next i
'Assign the values to the cells.
.Cells(2, 112).Resize(UBound(outpt, 1), 1).Value = outpt
End With
End Sub
I am trying to speed up a currently working automated workbook.
PHP sends a string similar to the below to VBA:
1[|:#:|]text-one[|:#:|]code-one[|:#:|]qty-one[{:#:}]
2[|:#:|]text-two[|:#:|]code-two[|:#:|]qty-two[{:#:}]
where
[|:#|] represents "new column"
[{:#:}] represents "new row"
When it is parsed by the VBA this is the output:
I currently use the following VBA code to parse this into a workbook:
myArray = Split(myReply, "[{:#:}]")
myRow = 1
For Each element In myArray
myRow = myRow + 1
subArray = Split(element, "[|:#:|]")
myCol = 2
For Each subelement In subArray
myCol = myCol + 1
Cells(myRow, myCol).Value = subelement
Next subelement
Next element
I am about to start optimising the code in this workbook and I am aware I can do something like (pseudo code):
for each element....
Range("C2:F2").Value = Split(element, "[|:#:|]") 'Example row number would be incremental
However is there a way to do it so that I can split into the entire Range?
For example, If I know there are 29 "rows" within the data that has been returned, I would like to be able to use split to place the data into all the rows.
I imagine the syntax would be something similar to the below, however this doesn't seem to work:
Range("C2:F29").Value = Split(Split(element, "[|:#:|]"),"[{:#:}]")
The optimal thing to do is to do everything in native VBA code and not interact with the Excel sheet until the end. Writing to sheet is a time consuming operation, so this procedure does it once and once only, writing the whole two-dimensional array at once, rather than writing it line by line. Therefore no need to disable screen updating, calculation, or anything else.
Function phpStringTo2DArray(ByVal phpString As String) As Variant
Dim iRow As Long
Dim iCol As Long
Dim nCol As Long
Dim nRow As Long
Dim nColMax As Long
Dim lines() As String
Dim splitLines() As Variant
Dim elements() As String
lines = Split(phpString, "[{:#:}]")
nRow = UBound(lines) - LBound(lines) + 1
ReDim splitLines(1 To nRow)
For iRow = 1 To nRow
splitLines(iRow) = Split(lines(iRow - 1), "[|:#:|]")
nCol = UBound(splitLines(iRow)) - LBound(splitLines(iRow)) + 1
' in case rows have different number of columns:
If nCol > nColMax Then nColMax = nCol
Next iRow
Erase lines
'We now have a (Variant) array of arrays. Convert this to a regular 2D array.
ReDim elements(1 To nRow, 1 To nColMax)
For iRow = 1 To nRow
nCol = UBound(splitLines(iRow)) - LBound(splitLines(iRow)) + 1
For iCol = 1 To nCol
elements(iRow, iCol) = splitLines(iRow)(iCol - 1)
Next iCol
Next iRow
Erase splitLines
phpStringTo2DArray = elements
End Function
Example usage:
Dim s As String
Dim v As Variant
s = "1[|:#:|]text-one[|:#:|]code-one[|:#:|]qty-one[{:#:}]2[|:#:|]text-two[|:#:|]code-two[|:#:|]qty-two[{:#:}]"
v = phpStringTo2DArray(s)
'Write to sheet
Range("A1").Resize(UBound(v, 1), UBound(v, 2)) = v
If you want to ignore the final line break [{:#:}], could add this line at the top of the function:
If Right(phpString, 7) = "[{:#:}]" Then phpString = Left(phpString, Len(phpString) - 7)
This wasn't as easy as I originally thought. I can get rid of one loop easily. But there's still an if test, so it doesn't break on empty strings etc. I feel a guru could make this even more efficient.
My worry is that for you this process is taking a lot of time. If you are trying to speed things up, your code doesn't look too horribly inefficient.
More likely if it's running slowly, is that the application.calculation & application.screenUpdating settings are set incorrectly.
Sub takePHP(myString As String)
'This sub takes specially formatted strings from a PHP script,
'and parses into rows and columns
Dim myRows As Variant
Dim myCols As Variant
Dim subRow As Variant
Application.ScreenUpdating = False
Application.Calculation = xlCalculateManual
myRows = Split(myString, "[{:#:}]")
x = 1
For Each subRow In myRows
bob = Split(subRow, "[|:#:|]")
If UBound(bob) <> -1 Then
Range(Cells(x, 1), Cells(x, UBound(bob) + 1)).Value = bob
x = x + 1
End If
Next
Application.Calculation = xlCalculationAutomatic
Application.ScreenUpdating = True
End Sub
I've been struggling with this code here (probably very simple mistake), would anyone mind pointing out where my issues are? My overall goal is to allow this subroutine to accept a range of variable size, however I can't seem to get it to work for a fixed size.
If I manually allocate the array, things work as expected but when I allocate with a range that's where things go wrong. The output comes back untouched, which leads me to believe that I'm not doing something correctly with the allocation. Also I'm getting errors when I try to pass ws.UsedRange as oppose to a fixed range.
Private Sub InsertionSort(ByRef a(), ByVal lo0 As Long, ByVal hi0 As Long)
Dim i As Long, j As Long, v As Long
For i = lo0 + 1 To hi0
v = a(i)
j = i
Do While j > lo0
If Not a(j - 1) > v Then Exit Do
a(j) = a(j - 1)
j = j - 1
Loop
a(j) = v
Next i
End Sub
Sub runSort()
Dim ws As Worksheet
Set ws = ActiveWorkbook.ActiveSheet
Dim myArr() As Variant
Dim rangeUse As Range
With ws.Range("D17:K17")
ReDim myArr(1 To 1, 1 To ws.Range("D17:K17").Columns.Count)
myArr = ws.Range("D17:K17").Value
End With
Call InsertionSort(myArr, LBound(myArr), UBound(myArr))
Range("D19:K19") = myArr
End Sub
Any help would be appreciated! TIA
So considerating you only want to sort your 2-dimensional array row by row, this might be a useful starting point. You can always change With ws.Range("A2:A3") to With Selection. If you do so, you have the Range you selected with your cursor.
With ws.Range("A2:A3")
myArr = .Value
For i = 1 To .Rows.Count
ReDim tmpArr(1 To .Columns.Count)
For j = 1 To .Columns.Count
tmpArr(j) = myArr(i, j)
Next j
Call InsertionSort(tmpArr, 1, .Columns.Count)
For j = 1 To .Columns.Count
myArr(i, j) = tmpArr(j)
Next j
Next i
.Offset(RowOffset:=10) = myArr
End With
Detailed Description
You don't have to redim myArray because if you set it to a range, it automatically scales.
tmpArr is each row of your range. If you select your range with the cursor some rows might be shorter or longer than others, thats why we redim that one. Edit This doesn't work just yet, because .Columns.Count refers to the whole range, not just the row. If you have different column counts then you'd have to change that.
For j = 1 To .Columns.Count
tmpArr(j) = myArr(i, j)
Next j
Unfortunately we cannot use tmpArr = myArr(i) because only one dimension of a multidimensional array cannot be accessed like this in VBA.
Call InsertionSort(tmpArr, 1, .Columns.Count) calles your Insertion Sort algorithm and sorts one row at a time.
After tmpArray got sorted, we have to set myArray(i) to the new values with the same loop we already used:
For j = 1 To .Columns.Count
myArr(i, j) = tmpArr(j)
Next j
Now we sorted all the rows in our Range, now we can put it back on the sheet, 10 rows beneath the first row of the specified range with .Offset(RowOffset:=10) = myArr
I hope that this helps you! While testing I saw that you might have a little bug in your InsertionSort algorithm. If the first value is the smalles, it just blindly gets copied into all the other fields of the array :)