Excel VBA correct/efficient formula population - excel

Good Afternoon,
I have been developing solutions in VBA for going on 6 years now. A lot of what I have learned along the way has been sourced from SO in that time, so finally I get a chance to say THANK YOU!
Today I have come to the conclusion that I need to post a question that has been bothering me for years. So - one of the things I do most often in VBA for Excel is to populate defined ranges with results from formulas applied to the range in question. Until now I have used the below method to accomplish this:
Sub CalculateField()
Application.ScreenUpdating = False
Application.DisplayAlerts = False
'''''''''''''''''''''''''''''''''
Dim RecordStop As Long
Set MyRange = ActiveSheet.Range("A:A")
RecordStop = Application.WorksheetFunction.CountA(MyRange)
'Apply Formula
Range("M2:M" & RecordStop).FormulaR1C1 = "=IF(RC[-7]=1,IF(ISERROR(VLOOKUP(CONCATENATE(RC[-2],""|"",1),R1C12:R[-1]C[-1],1,FA LSE)),0,1),0)"
'Drop Formula, keep values
Range("M2:M" & RecordStop).Value = Range("M2:M" & RecordStop).Value
''''''''''''''''''''''''''''''''''
Application.ScreenUpdating = True
Application.DisplayAlerts = True
End Sub
What I have found is that, although this method works, with larger files/more complex formula the hit I take in performance is HUGE! Now, I know that there must be a more efficient way to derive the results of the formula and inject them directly into the sheet where one wants the results to come back to as values. In the above method I first calculate the results then overwrite the "Formulas" with the "values"(results). I am currently working on a file that will need 21 calculated fields and average between 100k - 400k records.
I fear the time has come for me to learn a better way, the proper more professional way to do this. I have searched here and via the mighty google for examples on how to do this. I have found posts on SO describing what I think is part of the solution, but have had difficulty finding a specific simple example I can follow to implement. I think the correct method involves one or all of the following:
Arrays
Dictionaries
Collections
... I hope you guys can point me in the right direction. This, once solved, I feel will greatly up my game in VBA development. I have been hesitant to delve into this as I am self educated and never really thought I would get this far as it is not my primary role, but would love to get past this now so I would greatly appreciate your assistance.
BTW; I am pretty good at self learning, so if by chance there is already a post that can get me there just point the way. Again, in searching I could not find what I was looking for so I apologize if that was a result of not knowing how to (words to describe) look for the answer.

I might start with this before tinkering a bunch.
If you don't get a performance boost, I'd then consider pulling your source data into Arrays, modify them in code, then write your results in a one line of code write back to your cells.
Here's a great link for working the Array route: VBA Arrays And Worksheet Ranges
Dim PrevCalc As XlCalculation
With Application
PrevCalc = .Calculation
.Calculation = xlCalculationManual
.Cursor = xlWait
.Calculate
.EnableEvents = False
.ScreenUpdating = False
End With
Dim RecordStop As Long
Set MyRange = ActiveSheet.Range("A:A")
RecordStop = Application.WorksheetFunction.CountA(MyRange)
'Apply Formula
With Range("M2:M" & RecordStop)
.FormulaR1C1 = "=IF(RC[-7]=1,IF(ISERROR(VLOOKUP(CONCATENATE(RC[-2],""|"",1),R1C12:R[-1]C[-1],1,FA LSE)),0,1),0)"
'Calc only the target
.Calculate
'Drop Formula, keep values
.Value = .Value
End With 'Range("M2:M" & RecordStop)
With Application
.Cursor = xlDefault
.Calculate
.Calculation = PrevCalc
'.ScreenUpdating = True 'Not Needed...
.EnableEvents = True
End With

Related

Pause VBA until #GETTING DATA Power Pivot is complete

I've found a lot of questions and answers about issues that feel very close to what I'm working on, but not quite. I have an Excel workbook with a large Data Model connected to two slicers. I need to cycle through every entry in the slicer, allow the workbook to catch up on loading a large number of cube formulas, then copy one particular worksheet over into another.
I've written VBA which does all of this, but I can't for the life of me get the VBA to wait for the workbook to finish uploading before it continues with the rest of the script. I can rule out background refresh-based solutions, which don't apply to OLAP. Various solutions I've found online which recommend waiting for calculations to be complete don't seem to work, the script just barrels right through those lines. The only solution I've seen which seems to apply here involved identifying every cell which would be updated as a result of the slicer change and looping through them until they no longer say #GETTING DATA. For my workbook, this would be hundreds of cells to identify and check and feels very unsustainable.
Even telling the script to Applcation.Wait seems to wait for the selected amount of time during which the workbook pauses getting data.
Setting different values of a slicer connected to a Data Model and automating some output feels like it should be such a common task that we have a solution for it. Any help would be much appreciated.
Running Office 365
Sub generate_all_forecasts()
'Cycle through all products and push forecast values to fcst_output'
Application.ScreenUpdating = True
Dim SC_products As SlicerCache
Dim selection, product_array As Variant
Dim push As Boolean
Set SC_products = ThisWorkbook.SlicerCaches("Slicer_PRODUCT_GROUPING_WRITTEN") 'The product slicer on the Inputs worksheet'
product_array = Range("product_array") 'Named range product_array on Tbl_Codes worksheet'
For Each p In product_array 'For each product'
push = WorksheetFunction.Index(Range("fcst_push_array"), WorksheetFunction.Match(p, product_array, 0)) 'Check if the product has been selected for this run'
If push = True Then
If p = "Major Medical Plan" Then 'If "Major Medical" '
selection = Array("[Query1 1].[PRODUCT_GROUPING_WRITTEN].&[Major Medical Plan - CMM]", _
"[Query1 1].[PRODUCT_GROUPING_WRITTEN].&[Major Medical Plan - GMM]") 'selection will be both CMM and GMM'
Else
selection = Array("[Query1 1].[PRODUCT_GROUPING_WRITTEN].&[" & p & "]") 'Otherwse selection is the single product'
End If
SC_products.VisibleSlicerItemsList = selection 'Change slicer to current selection'
'This is where the script needs to pause until #GETTING DATA is complete'
Application.Run "push_to_output" 'Run the forecast update macro'
End If
Next p
Worksheets("Fcst_Output").Range("B2:B1381").Value = "" 'Clear prior month's comments'
Application.ScreenUpdating = True
End Sub
Solutions which have not worked: Wait time after change slicer in
power pivot, Getting vba to wait before proceeding, Wait
until Application.Calculate has finished
The "solution" I really don't want to use: Force VBA to wait until
power pivot finishes refreshing
Thanks to Tragamor for linking to a thread where they already had a working answer. I included the following immediately after the slicer selection in my VBA and it appears to properly wait until all data fetching is complete:
Dim CalculationState As Long
With Application
CalculationState = .Calculation
.Calculation = xlCalculationAutomatic
.CalculateUntilAsyncQueriesDone
Do Until .CalculationState = xlDone
DoEvents
Loop
.Calculation = CalculationState
End With
Find your Query Properties, and set 'Enable background refresh' to False (Microsoft use True as default).
Then in your code you need to call for RefreshAll and wait for the data to load with the DoEvents. If you want your update of data to run on open you can use this in 'ThisWorkbook' Object.
Private Sub Workbook_Open()
For Each q In ThisWorkbook.Connections
q.Refresh
DoEvents
Next
End Sub

Macro slows down after change in formula

I used to have this formula below copied into multiple cells, but at some point it wasn't pulling the data properly so I had to modify it to the formula below it. The SUMPRODUCT formula finds data from another spreadsheet based off the part number and pulls it over to the new spreadsheet.
=IFERROR(VLOOKUP($B14,'G:\Locations_NA\TUS\LO\[DMSU MACRO DATA.xlsm]DDCPIV'!$A:$W,13,FALSE),0)
=IFERROR(SUMPRODUCT(('G:\Locations_NA\TUS\LO\[DMSU MACRO DATA.xlsm]DDC SD'!$F$4:$AT$10000)*('G:\Locations_NA\TUS\LO\[DMSU MACRO DATA.xlsm]DDC SD'!$A$4:$A$10000=$B14)*('G:\Locations_NA\TUS\LO\[DMSU MACRO DATA.xlsm]DDC SD'!$F$3:$AT$3=$AD$13)),0)
I run a macro that someone else made which, besides doing other things, fills down this formula a couple hundred rows. Now when I run it with the new formula it takes ages compared to how long it used to take. Is there a better way I can go about this to speed it up?
I would recommend these three basic things, that will help you without modifying much your code.
First: Change VLOOKUP or SUMPRODUCT to "XLOOKUP". This is a faster new formula
it will be something like this
=IFERROR(XLOOKUP(R14C12, _
'G:\Locations_NA\TUS\LO\[DMSU MACRO DATA.xlsm]DDCPIV'!$A:$A, _
'G:\Locations_NA\TUS\LO\[DMSU MACRO DATA.xlsm]DDCPIV'!$W:$W,_
"Error",0),0)
Second: Try to set the formula by a bunch of cells, not one by one (I'm assuming this, if you share more of your code we can check.
one by one is:
Range("A1").FormulaR1C1 =IFERROR(XLOOKUP(R14C12 ...
Range("A2").FormulaR1C1 =IFERROR(XLOOKUP(R15C12 ...
Bunch of cell is:
Range("A1:A5").FormulaR1C1 =IFERROR(XLOOKUP(R14C12 ...
Third: as ENIAC just said, disable UpdatingScreen and other Automatic Updates that Excel has. This will be like this:
Sub Example()
'Beginning Code
With Application
.ScreenUpdating = False
.Calculation = xlCalculationManual
.EnableEvents = False
.DisplayAlerts = False
End With
'Your code Here
'End Code
With Application
.ScreenUpdating = True
.Calculation = xlCalculationAutomatic
.EnableEvents = True
.DisplayAlerts = True
End With
End sub
Now, the best way, in my opinion, is to work with Arrays and Dictionaries, that system is much faster, very much.

Range.PasteSpecial causes an Application ScreenUpdate in Excel VBA

I have written a VBA Macro for Excel that looks like the following:
Application.ScreenUpdating = False
Dim copyingRange, pastingRange As Range
copyingRange.Cells.Copy
pastingRange.PasteSpecial xlPasteValues
Application.ScreenUpdating = True
It is my understanding that Application.ScreenUpdating = False will force the application to stop screen updates however this is not the behavior I am seeing. I am copying a range from one sheet to another and the application switches views to and from very quickly, an undesirable effect.
Taking out the line pastingRange.PasteSpecial xlPasteValues causes the screen flickering to go away but obviously results in the macro not doing what I want.
Is the pastingRange.PasteSpecial xlPasteValues the problem here? I must be misunderstanding something here?
From comment, so this can be listed as Answered:
You can avoid copy/paste since you just want values and write:
pastingrange.value = copyingrange.value

Excel slow with other spreadsheets open

I have a very simple VBA code running in workbook A that is rather slow when another specific workbook B is open. I don't have access to any macros inside workbook B, so I can't know for sure if anything wierd is happening there.
I've looked around and came with this minimal code condensing the typical solutions I've found in stackoverflow and many Excel specific sites, but it still takes a long time to run.
' Code in workbook A
Sub slow_simple_macro()
With Application
.Calculation = xlCalculationManual
.EnableEvents = False
.ScreenUpdating = False
.DisplayStatusBar = False
End with
Workbooks("workbookA.xlsm").Sheets("Sheet1").Range("A1") = "Slow" ' This line takes about half a second to run when workbook B is open
With Application
.Calculation = xlCalculationAutomatic
.EnableEvents = True
.ScreenUpdating = True
.DisplayStatusBar = True
End with
End Sub
I'm guessing that disabling events is not enough to prevent something inside workbook B from running. Are there any other Application flags I can setup to make it run faster? Any other ideas?
You can see while disabling some functions of excel like calculation mode, screenupdates,status bar would impact on application of Excel because "Excel is the one of The object model that is a large hierarchy of all the objects used in VBA" (Source:www.globaliconnect.com).
So if you want run by turn off application functions , you should be wait till you set back to its original or else same problem would happen.
Better simplify code as possible unless if its required
Sub slow_simple_macro()
Workbooks("workbookA.xlsm").Sheets("Sheet1").Range("A1") = "Slow"
End Sub
Hope you find it as helpful. Have a good day.

Removing an Autofilter that is set by variable

I'm having some trouble removing an autofilter that is set by a variable. I have tried everything I could think of/find online. Nothing seems to be able to remove it. Any suggestions?
Dim CashRange As Range
Set CashRange = Range("L2:L50000")
CashRange.AutoFilter 1, "CASH"
CashLabeledRows = ActiveSheet.AutoFilter.Range.Columns(1).SpecialCells(xlCellTypeVisible).Count - 1
CashRange.Parent.AutoFilter.Range.Cells.Interior.ColorIndex = 6
CashRange.Parent.AutoFilterMode = False
Have a look at this examples, maybe they will be helpfull.
If ActiveSheet.FilterMode = True Then 'checks if there are any filters turned on!?
If ActiveSheet.AutoFilterMode = True Then 'if there are filters at all(what so ever)
ActiveSheet.ShowAllData 'Turns off all filters(select all), but not remove them
This is not sequence code, Every line is just for it self. But it can be combined with each other.
Not the optimal answer but after some testing over different data types present in the form, I determined that the filter issue was only present when "CASH" was not present in any cell of the L column.
I decided the best way to combat this for the time being was to do a FIND on the column for the CASH value. Then based on the response, I had a GoTo defined for before and after the above vba. Basically skipping if it was unnecessary.
Thanks everyone for your time and input. Code posted below for reference.
Dim t As Range
Set t = Columns("L").Find("CASH", LookAt:=xlWhole)
If t Is Nothing Then
GoTo AfterCashYellow
ElseIf Not t Is Nothing Then
GoTo CashYellowFill
End If

Resources