How to Force a Function (UDF) Update in MS Excel VBA - excel

This is my first post because in several years of learning VBA, I always been able to find the answers to all my question because some one else beat me to it and because stackexchange rocks. However, I have finally ran into something that I cannot find any answers for...
BACKGROUND:
I have a very complicated Excel workbook that is very VBA intensive (lots of loops and automatic subs). I'm very cautious about referencing cells from VBA because if someone ever shifts a cell without updating the VBA code, problems will ensue. However, I have a couple calculations that need to be done that are far too complex for an in-cell formula, so I've created functions for those things. That avoids any tampering issues and gives me the power I need.
PROBLEM:
But the problem is, since the workbook is so complicated, if I make the UDFs volatile, they will lock up the workbook because it's just looping through way to much data.
But if I don't make them volatile, the function values will not update in the cells.
HOPES AND DREAMS:
I would like to know how to force my UDF to run without being volatile.
NOTE:
I've tried everything I can find with no success, but I am open to ideas. ".calculate" does not work (unless I'm using it wrong). ".calculate = ...automatic" doesn't work either. Creating a "volatile" variable and I toggle externally also doesn't work.
Another solution would be to temporarily turn on function volatility from a different module/sub, but I don't know of a way to do that.
Any help would be greatly appreciated.
Thanks!

(As K.Davis said and xlLogan approved in the comments above) Application.Calculate or (if not working) Application.CalculateFull can help in many cases.
But sometimes it may be necessary to (additionally) mark some UDF(s) as volatile (which xlLogan explicitely did/could not do), especially (like in another case I had) when the dependencies are bi-directional between sheet formulas and VBA code (and like in my case not dependent on some sheet change, but e.g. on the excel filename, that may have been changed during two openings and is recognized and processed during the Workbook_Open() event):
Sub Foo()
Application.Volatile '=> so this sub/function is not considered entirely dependent (deterministic)
' on some cells only, but on other "background" conditions as well
...
End Sub
Also explicitely disabling and enabling events during certain (potentially expensive) recalculation triggering operations/changes (e.g. also possbile via user dialogs/buttons) could help to manage performance issues, e.g. using (some often applicable functionality):
'used with EventsEnable()
Sub EventsDisable()
With Application: .EnableEvents = False: .ScreenUpdating = False: .Calculation = xlCalculationManual: End With
End Sub
'used with EventsDisable()
Sub EventsEnable()
With Application: .EnableEvents = True: .ScreenUpdating = True: .Calculation = xlCalculationAutomatic: End With
End Sub

Related

Is there a way using sheet protection to allow a user to enter values but protect formats?

I have a worksheet containing a column of numbers. The column is formatted with a background color, number formats, etc. The column is unlocked. I protect the sheet manually by right-clicking on the tab and selecting Protect. In the Protect Sheet dialog, 'format cells' is unchecked. I interpret to mean that the user should not be able to format cells. Yet when the user pastes into the column, formats are pasted along with values.
If I protect the sheet in VBA using
sh.Protect UserInterfaceOnly:=True
I get the same result: formats are pasted. I do not specify AllowFormattingCells:=False because the default is False.
I have seen posts suggesting that formats can be restored by copying and pasting them from a shadow area. I have used this solution before I started protecting worksheets and found it overly complex. I had hoped this was something protection would handle. If there is a way to handle it, I'd like to do it in VBA.
This question is a little old, but I had the same one, and so have many people in the past. With a bit of browsing I came up with a solution which seems quite clean and appears to work. Am I missing something?
Private Sub Worksheet_Change(ByVal rngTarget As Range)
Dim vPaste As Variant
With Application.CommandBars("Standard").Controls("&Undo")
If Not .Enabled Then Exit Sub
If .ListCount < 1 Then Exit Sub
If .List(1) <> "Paste" Then Exit Sub
End With
vPaste = rngTarget.Value2
On Error Resume Next
Application.EnableEvents = False
Application.Undo
rngTarget.Value2 = vPaste
Application.EnableEvents = True
On Error GoTo 0
End Sub
This could go in Workbook_SheetChange, but not if you already have code in Worksheet_Change, because the worksheet event handler gets called before the workbook event handler. But this code can go in a module to keep things tidy.
There is no built-in protection option you can use to achieve your desired result.
The only thing that works in this case is the clunky workaround that you mention, i.e. use a Worksheet_Change event that ensures the correct format after a cell has been modified.
Since there are many different ways to paste content, i.e. via various menus, ribbon commands, keyboard shortcuts, etc., any VBA solution that tries to intercept pasting will become very complex, much more complex than the change event that restores the format to its original state.
Another option might be user education and training (so they know to paste values only), although user behaviour may be the toughest element to change in the whole scenario.

Inserting Row in Excel Takes a long time

I have an excel document (10 sheets). 1 sheet has at this point about 1600 rows and about 148 columns of information. There are no formula's in this sheet.
At first I thought it was a problem with my Macro. But it turns out manually inserting a row (or deleting) gives the same problem: a 14 second delay (for ONE ROW!). I've tried deleting all conditional formatting, Data Validation, (deleting hidden graphs that get there data from a different sheet). I've tried turning of Auto Calculation (which I also do in the Macro), I've tried inserting/deleting a row in "Design Mode"> But nothing seems to help.
One version of my file (I have different back-ups) seems to have fixed it. But I cannot reproduce this in my current file. I don't know what the fix was.
I've scoured the internet for solutions but have yet to find one. I'm running on excel 2010. Who knows the trick I need? Or is updating to excel 2016 my best option?
cheers!
The three primary causes of recalculation lag are calculation, event handling (due to a Worksheet_Change) and (to a lesser degree) screen updating.
Put this 'helper' sub procedure in a public module code sheet.
Public Sub appTGGL(Optional bTGGL As Boolean = True)
With Application
.ScreenUpdating = bTGGL
.EnableEvents = bTGGL
.DisplayAlerts = bTGGL
.AutoRecover.Enabled = bTGGL 'no interruptions with an auto-save
.Calculation = IIf(bTGGL, xlCalculationAutomatic, xlCalculationManual)
.CutCopyMode = False
.StatusBar = vbNullString
End With
Debug.Print Timer
End Sub
Use it like this,
sub main()
appTGGL btggl:=false
'do everything you have to do here
appTGGL
end sub
You can add or remove application environment settings as you see fit.
If the problem turns out to be a badly written Worksheet_Change then post it here or on code review for improvement,
quick update: somehow the problem is fixed when I remove to other sheets. These sheets are not connected by formula references in any ways. Both sheets have (flat) information implanted by macro's, but after the macro is done it's just flat information. I've tried disabling al relevant macro's but this didn't solve the issue. Only deleting the other sheets: If i deleted 1 of the sheets the time would go from 14 sec to 7 seconds, and if I deleted the other as well it would go to 0 seconds.
It seems as though somehow these sheets are connected by some kind of calculation (outside of the macros) but I cannot imagine what this might be. Any Ideas?

Does making some codes in the same line makes the macro faster in VBA?

I've tried every suggestion on this site to make my macro faster.
Halt calculations
Disable the screen updating
Disable status bar updating
Telling the Excel to ignore events
and so much more. Even in loops, I've tried many methods of making my macro smooth.
And I saw an article that I can use colon to make a multiple lines of codes into single line.
From:
Dim x as Integer
x = 1
If x = 0 Then
Exit Sub
End If
To:
Dim x as Integer: x = 1: If x = 0 Then Exit Sub
Does making some codes in the same line makes the macro faster?
No, writing the code on the same line does not make the code run faster.
Code can be slow because of inefficient coding techniques, like looping through all cells in a column, reading each cell, calculating it, and writing the result back to another cell in each step of the loop.
Writing that code in one line is possible, but it won't make the code run faster.
What WILL make the code run faster is reading the data range into an array, then looping over the array, writing the output of any calculation into another array and then finally writing the result array back to the spreadsheet as one write operation.
This is simply a different line break and does not affect execution time.
You already noted a couple of tricks you applied:
Application.ScreenUpdating = False
Application.DisplayAlerts = False
Application.Calculation = xlCalculationManual
Application.EnableEvents = False
Since you haven't posted full code and "Please optimize my code" is not a question, there's the following general tips to speed up Execution:
When referencing the same Object multiple times in a row, use With and End With instead. This is faster than specifying the full path to the object each time, since it prevents certain IO operations to access the right block in memory.
Use as little references to Worksheet / Range objects as possible, in general. A lot of times it's faster to use an Array than a Range. See Chip Pearson's article on this.
Looping is slow. Using native functions is faster. Make use of Autofilter instead of checking each cell, for example.
Do not use Copy and Paste, instead use the Destination parameter of the Copy method
Declare your objects as accurately as possible: Use Dim ws As Worksheet instead of Dim ws or Dim ws As Object.
Never use .Select and .Activate - See this Question
No, such changes does not boost Your code.
It also depends what Your macro do, but my case was that I needed to use sth similar to vlookup in my code, so at the first version I used ForEach loop and if statment - when I was to match 10 000 records with about 3k values it took about 15 minutes. So I decided to use dictionary (reference does not break acro on other machines) and it really boost my code. Now 20 k records with 6k values takes about 2 minutes.

Excel VBA Slow Calculation Loop Iterations

Not sure if this is a code-specific issue or not, so I will be general for now.
I have a somewhat complicated macro that begins by reading financial market data that is manually inputted by a user into a worksheet, and then proceeds to process that market data, generating the required market curves, etc., and then calculate the certain valuations of interest.
The process requires a lot of looping since there are thousands of instruments that need to be valued. However, I noticed that every now and then the macro will loop extremely slowly - on the order of about 2-3 seconds per iteration. When I have the Excel workbook up, I can see down at the bottom it is saying "Calcuating (4 Processors x% complete)".
To resolve the issue I have to manually force Excel to shut down; usually this fixes the problem and the next time I run the program it works fine.
I am running Windows 8 (not 8.1) and Excel 2013. I've heard that this combination is particularly prone to crashing/bugs (I've experienced this several times myself where Excel will take a very long time to process basic requests such as font formatting or will spontaneously crash for no apparent reason).
However, I'd like to ask the community to see if the problem is more universal/known.
Thanks!
As a general tip for creating fast excel macros: wherever prossible don't loop through cells, you will get much better performance using a with statement on a range object, or where you need to operate on the data in a more elaborate way try copying your range of data into a 2 dimensional array, looping through an array will be orders of magnitude faster than looping though cells in a worksheet, you can then dump the array back to the range.
Try application.visible = false in the beginning of your code. Then make sure to do application.visible = true at the end. Should help.
Try to change the cursor to xlBeam
It changes everything in terms of speed in Windows 8.1
Sub CurseurDefault(zz As Boolean)
If zz = True Then
Application.Cursor = xlDefault
'Call ShowCursor(True)
Else
Application.Cursor = xlIBeam
'Call ShowCursor(False)
End If
End Sub

How do I determine whether calculation was completed, or detect interrupted calculation?

I have a rather large workbook that takes a really long time to calculate. It used to be quite a challenge to get it to calculate all the way, since Excel is so eager to silently abort calculation if you so much as look at it.
To help alleviate the problem, I created some VBA code to initiate the the calculation, which is initiated by a form, and the result is that it is not quite as easy to interrupt the calculation process, but it is still possible. (I can easily do this by clicking the close X on the form, but I imagine there are other ways)
Rather than taking more steps to try and make it harder to interrupt calculation, I'd like to have the code detect whether calculation is complete, so it can notify the user rather than just blindly forging on into the rest of the steps in my code. So far, I can't find any way to do that.
I've seen references to Application.CalculationState, but the value is xlDone after I interrupt calculation, even if I interrupt the calculation after a few seconds (it normally takes around an hour).
I can't think of a way to do this by checking the value of cells, since I don't know which one is calculated last. I see that there is a way to mark cells as "dirty" but I haven't been able to find a way to check the dirtiness of a cell. And I don't know if that's even the right path to take, since I'd likely have to check every cell in every sheet.
The act of interrupting calculation does not raise an error, so my ON ERROR doesn't get triggered.
Is there anything I'm missing? Any ideas?
Any ideas?
I think the trick you need to implement (if you're application runs in Excel 2007 or later) is to handle this with the Application.AfterCalculate event, which is raised after both calculation is complete and there are no outstanding queries.
If you've never worked with events in VBA before, there is a good overview from cpearson.com.
The (MSDN) solution by Charles Williams above worked for me where I had 1000's of VLOOKUP's that neeeded to recalculate as the code was changing the lookup value because of an iteration loop. Results were skewed as calculations were not running to 100% completion.
At the beginning of my subroutine the code executes
Application.Calculation = xlManual
This eliminated unnecessary calculations by Excel until I was ready.
Now at the critical point the code executes
Application.Calculation = xlAutomatic
ThisWorkbook.ForceFullCalculation = True
Application.Calculate
Having forced Excel to perform a full calculation, the code could then saved the result and move onto the next iteration ... but before doing so
ThisWorkbook.ForceFullCalculation = False
Application.Calculation = xlManual
Remembering at the very end
Application.Calculation = xlAutomatic
I've never actually used it but I think this might work to prevent calculation from being interrupted.
Application.CalculationInterruptKey = xlNoKey
I think I'm hearing that you need a way to monitor whether each step within the calculations being performed was executed.
Assuming that you're not interested in re-engineering the workbook to use methods that are easier to track than spreadsheet calculations (such as volatile calculations within VBA or Pivot Tables), this may work for you:
Within VB, you can utilize .EnableCalculation and .Calculate to set an entire worksheet as "Dirty" (needing calculation) and then recalculate. The key difference between this and your current process is that we will perform these actions one worksheet at a time in manual mode. By initiating the calculations one worksheet at a time from within VBA, you will be able to perform additional intermediate actions that can be used to track how far you got in the calculation process.
Please note that this approach assumes a fairly linear workbook structure such that your workbook will produce the correct results if we first recalculate Sheet1, then Sheet2, Sheet3, and so on, in whatever order you wish. If your formula dependencies are more "spaghetti" than linear, this probably won't work for you. It also assumes you are working in Excel 2000 or later.
For example, you could write a VBA routine that accomplishes the following steps.
You will need to know your dependencies in order to know which calculations must come before others, and start with the worksheet in a "clean" state where no calculations are currently pending.
Step 1: Set the active sheet to the first worksheet where recalculation is needed
Step 2: Set the calculation mode to manual as follows:
Application.Calculation = xlCalculationManual
Step 3: "Dirty" the entire active sheet as follows:
With ActiveSheet
.EnableCalculation = False
.EnableCalculation = True
Step 4: Initiate a recalculation for this worksheet only (not the entire workbook) using:
.Calculate
End With
Note that if the calculation mode were set to automatic, Step 3 would initiate a re-calculation across the entire workbook. By using manual mode and With, we are constraining that calculation to the current sheet.
Now you have dirtied and re-calculated the first sheet (hurray!). Now, by embedding Steps 3 and 4 above into a For/Each or For/Next loop, you can repeat the process for each worksheet in your workbook. Again, make sure you know the order in which your worksheets need to be calculated (if an order is needed).
Now for the big finish - by creating a counter variable within your loop, you can track how far you got in the calculations by updating your counter variable value each time you complete a worksheet calculation. For example, after you recalculate a worksheet, you can set the counter value to current value + 1 and store the results either in a global variable (so that it will persist even after your VBA routine ends), or in a cell within your worksheet. That way, you can check this value later to see how many worksheets were updated before the calculations finished or were interrupted.
If you have relatively few worksheets in your workbooks, the same approach could be applied to one range at a time rather than a sheet.
I won't go into detail about how to construct a "counter", loops, or global variables here, but if needed, this information can be easily found using your favorite search engine. I would also highly recommend re-enabling automatic calculations once you are done as it is easy to forget that it's been set to manual mode.
I hope this works for you - for more information on calculation modes and recalculation, this is a helpful link:
http://msdn.microsoft.com/en-us/library/bb687891.aspx
Perhaps the following would work:
Do Until Application.CalculationState = xlDone
DoEvents
Loop
Can't say I've tested it, nor that I know how robust the functionality of Application.CalculationState really is to determine whether 'complete' calculation occurred, as opposed to something interrupting the process and flagging the calculation state as done.
Private sub SomeCodeThatGeneratesFormulas
Application.Calculation = xlCalculation.xlCalculationManual
'...Some formulas are copied here'
Application.OnTime DateTime.DateAdd ("s",.01,DateTime.Now), "Module1.CalculateFullRebuildAndSubsequentSteps" 'By using Application.OnTime, this method will be called in a way that locks the end-user out of providing inputs into Excel until the calculation itself is complete.
end sub
public sub CalculateFullRebuildAndSubsequentSteps
Application.CalculateFullRebuild
'...Do next steps, i.e. paste as values'
end sub
On the status bar, right hand side, it will say Calculating (N processors) X% (where N is the number of processors on your computer and X% is how much it has completed) when recalculating. If you don't see text there, it's not recalculating.
I'm using Office 2010, but it should be there in all versions. It's just kinda subtle so it's easy to miss.
Arrays in Excel can be a bit stupid. That is that in order to accomplish some tasks people avoid to use intermediate columns/rows to store (temporary) data, so arrays have to recalculate staff from the beginning every time, thus getting really slow. My Suggestion would be:
fix arrays to avoid multiple searches. Use hidden cells or even hidden sheets
Avoid using A:A and rather use A1:A1000 specially in excel 2007 or later
use formulas to equal zero or error (ex: NA()) while previous items aren't calculated, so you can clearly see if an operation is done at all.
some VBA could be used to inject formulas in place one step at a time, perform calculations, then proceed to next step, but this could mean lots of work...

Resources