VBA subroutine slows down a lot after first execution - excel

I have a subroutine that generates a report of performance of different portfolios within 5 families. The thing is that the portfolios in question are never the same and the amount in each family neither. So, I copy paste a template (that is formated and...) and add the formated row (containing the formula and...) in the right family for each portfolio in the report. Everything works just fine, the code is not optimal and perfect of course, but it works fine for what we need. The problem is not the code itself, it is that when I execute the code the first time, it goes really fast (like 1 second)... but from the second time, the code slows down dramatically (almost 30 second for a basic task identical to the first one). I tried all the manual calculation, not refreshing the screen and ... but it is really not where the problem comes from. It looks like a memory leak to me, but I cannot find where is the problem! Why would the code runs very fast but sooooo much slower right after... Whatever the length of the report and the content of the file, I would need to close excel and reopen it for each report.
**Not sure if I am clear, but it is not because the code makes the excel file larger or something, because after the first (fast) execution, if I save the workbook, close and reopen it, the (new) first execution will again be very fast, but if I would have done the same excat thing without closing and reopening it would have been very slow...^!^!
Dim Family As String
Dim FamilyN As String
Dim FamilyP As String
Dim NumberOfFamily As Integer
Dim i As Integer
Dim zone As Integer
Sheets("RapportTemplate").Cells.Copy Destination:=Sheets("Rapport").Cells
Sheets("Rapport").Activate
i = 3
NumberOfFamily = 0
FamilyP = Sheets("RawDataMV").Cells(i, 4)
While (Sheets("RawDataMV").Cells(i, 3) <> "") And (i < 100)
Family = Sheets("RawDataMV").Cells(i, 4)
FamilyN = Sheets("RawDataMV").Cells(i + 1, 4)
If (Sheets("RawDataMV").Cells(i, 3) <> "TOTAL") And _
(Sheets("RawDataMV").Cells(i, 2) <> "Total") Then
If (Family <> FamilyP) Then
NumberOfFamily = NumberOfFamily + 1
End If
With Sheets("Rapport")
.Rows(i + 8 + (NumberOfFamily * 3)).EntireRow.Insert
.Rows(1).Copy Destination:=Sheets("Rapport").Rows(i + 8 + (NumberOfFamily * 3))
.Cells(i + 8 + (NumberOfFamily * 3), 6).Value = Sheets("RawDataMV").Cells(i, 2).Value
.Cells(i + 8 + (NumberOfFamily * 3), 7).Value = Sheets("RawDataMV").Cells(i, 3).Value
End With
End If
i = i + 1
FamilyP = Family
Wend
For i = 2 To 10
If Sheets("Controle").Cells(16, i).Value = "" Then
Sheets("Rapport").Cells(1, i + 11).EntireColumn.Hidden = True
Else
Sheets("Rapport").Cells(1, i + 11).EntireColumn.Hidden = False
End If
Next i
Sheets("Rapport").Cells(1, 1).EntireRow.Hidden = True
'Define printing area
zone = Sheets("Rapport").Cells(4, 3).End(xlDown).Row
Sheets("Rapport").PageSetup.PrintArea = "$D$4:$Y$" & zone
Sheets("Rapport").Calculate
Sheets("RANK").Calculate
Sheets("SommaireGroupeMV").Calculate
Sheets("SommaireGroupeAlpha").Calculate
Application.CutCopyMode = False
End Sub

I do not have laptop with me at the moment but you may try several things:
use option explicit to make sure you declare all variables before using them;
from what I remember native vba type for numbers is not integer but long, and integers are converted to long, to save the computation time use long instead of integers;
your Family variables are defined as strings but you store in them whole cells and not their values i.e. =cells() instead of =cells().value;
a rule of a thumb is to use cells(rows.count, 4).end(xlup).row
instead of cells(3, 4).end(xldown).row.;
conditional formatting may slow down things a lot;
use for each loop on a range if possible instead of while, or even copy range to variant array and iterate over that (that is the fastest solution);
use early binding rahter of late binding, i.e., define objects in a proper type as soon a possible;
do not show printing area (page breaks etc.);
try to do some pofiling and look for the bottlenecks - see finding excel vba bottlenecks;
paste only values if you do not need formats;
clear clipboard after each copy/paste;
set objects to Nothing after finishing using them;
use Value2 instead of Value - that will ignore formatting and take only numeric value instead of formatted value;
use sheet objects and refer to them, for example
Dim sh_raw As Sheet, sh_rap As Sheet
set sh_raw = Sheets("RawDataMV")
set sh_rap = Sheets("Rapport")
and then use sh_raw instead of Sheets("RawDataMV") everywhere;

I had the same problem, but I finally figured it out. This is going to sound ridiculous, but it has everything to do with print page setup. Apparently Excel recalculates it every time you update a cell and this is what's causing the slowdown.
Try using
Sheets("Rapport").DisplayPageBreaks = False
at the beginning of your routine, before any calculations and
Sheets("Rapport").DisplayPageBreaks = True
at the end of it.

I had the same problem. I am far from expert programer. The above answers helped my program but did not solve the problem. I'm running excel 2013 on a 5 year old lap top. Open the program without running it, go to File>OptionsAdvanced, Scroll down to Data and uncheck "Disable undo for large Pivot table refresh...." and "Disable undo for large data Model operation". You could also try leaving them checked but decreasing their value. One or both of these seem to be creating a ever increase file that slows the macro and eventual grinds it to a stop. I assume closing excel clears the files they create so that's why it runs fast when excel is closed and reopened at least for a while. Someone with more knowledge will have to explain what these changes will do and what the consequences are of unchecking them. It appears these changes will be applied to any new spread sheets you create. Maybe these changes would not be necessary if I had a newer more powerful computer.

Related

Clean data in excel that comes in varying formats

I have an excel table that contain values in these formats. The tables span over 30000 entries.
I need to clean this data so that only the numbers directly after V- are left. This would mean that when the value is SV-51140r3_rule, V-4407..., I would only want 4407 to remain and when the value is SV-245744r822811_rule, I would only want 245744 to remain. I have about 10 formulas that can handle these variations, but it requires a lot of manual labor. I've also used the text to column feature of excel to clean this data as well, but it takes about 30 minutes to an hour to go through the whole document. I'm looking for ways that I can streamline this process so that one formula or function can handle all of these different variations. I'm open to using VBA but don't have a whole lot of experience with it and I am unable to use Pandas or any IDE or programming language. Help please!!
I've used text to columns to clean data that way and I've used a variation of this formula
=IFERROR(RIGHT(A631,LEN(A631)-FIND("#",SUBSTITUTE(A631,"-","#",LEN(A631)-LEN(SUBSTITUTE(A631,"-",""))))),A631)
Depending on your version of Excel, either of these should work. If you have the ability to use the Let function, it will improve your performance, as this outstanding article articulates.
If you're on a really old version of excel, you'll need to hit ctl shift enter to make array formula work.
While these look daunting, all these functions are doing is finding the last V (by this function) =SUBSTITUTE(RIGHT(SUBSTITUTE(A2,"V",REPT("πŸ„",999)),999),"πŸ„","") and then looping through each character and only returning numbers.
Obviously the mushroom πŸ„ could be any character that one would consider improbable to appear in the actual data.
Old School
=TEXTJOIN("",TRUE,IF(ISNUMBER(MID(MID(SUBSTITUTE(RIGHT(SUBSTITUTE(A2,"V",REPT("πŸ„",999)),999),"πŸ„",""),
FIND("-",SUBSTITUTE(RIGHT(SUBSTITUTE(A2,"V",REPT("πŸ„",999)),999),"πŸ„","")),9^9),
FILTER(COLUMN($1:$1),COLUMN($1:$1)<=LEN(MID(SUBSTITUTE(RIGHT(SUBSTITUTE(A2,"V",REPT("πŸ„",999)),999),"πŸ„",""),
FIND("-",SUBSTITUTE(RIGHT(SUBSTITUTE(A2,"V",REPT("πŸ„",999)),999),"πŸ„","")),9^9))),1)+0),
MID(MID(SUBSTITUTE(RIGHT(SUBSTITUTE(A2,"V",REPT("πŸ„",999)),999),"πŸ„",""),
FIND("-",SUBSTITUTE(RIGHT(SUBSTITUTE(A2,"V",REPT("πŸ„",999)),999),"πŸ„","")),9^9),
FILTER(COLUMN($1:$1),COLUMN($1:$1)<=LEN(MID(SUBSTITUTE(RIGHT(SUBSTITUTE(A2,"V",REPT("πŸ„",999)),999),"πŸ„",""),
FIND("-",SUBSTITUTE(RIGHT(SUBSTITUTE(A2,"V",REPT("πŸ„",999)),999),"πŸ„","")),9^9))),1),""))
Let Function
(use this if you can)
=LET(zText,SUBSTITUTE(RIGHT(SUBSTITUTE(A2,"V",REPT("πŸ„",999)),999),"πŸ„",""),
TEXTJOIN("",TRUE,IF(ISNUMBER(MID(MID(zText,FIND("-",zText),9^9),
FILTER(COLUMN($1:$1),COLUMN($1:$1)<=LEN(MID(zText,FIND("-",zText),9^9))),1)+0),
MID(MID(zText,FIND("-",zText),9^9),
FILTER(COLUMN($1:$1),COLUMN($1:$1)<=LEN(MID(zText,FIND("-",zText),9^9))),1),"")))
VBA Custom Function
You could also use a VBA custom function to accomplish what you want.
Function getNumbersAfterCharcter(aCell As Range, aCharacter As String) As String
Const errorValue = "#NoValuesInText"
Dim i As Long, theValue As String
For i = Len(aCell.Value) To 1 Step -1
theValue = Mid(aCell.Value, i, 1)
If IsNumeric(theValue) Then
getNumbersAfterCharcter = Mid(aCell.Value, i, 1) & getNumbersAfterCharcter
ElseIf theValue = aCharacter Then
Exit Function
End If
Next i
If getNumbersAfterCharcter = "" Then getNumbersAfterCharcter = errorValue
End Function

MS Project 2016 - VBA Cycle Time Issues (Extraction to Excel)

I have numerous macros that seem to be slowing down when interfacing with MS Project (e.g., reading data and processing it) in MS Office 2016. It could be my underlying code, mechanisms for obtaining/storing the data, but I'm not sure and would appreciate input.
I was just writing a new macro for extracting MSP data into an array (held in memory I thought this would be fastest, but it seemed to struggle to even do this). The idea being that the array memory access 'should' be faster than bridging from MS Excel to the Project referenced when needing to extract/place data? Perhaps this is incorrect.
Aim: Hold 3 parts of each MS Project Resource in Array, eventually iterate over array and place into Excel for additional use/processing.
The initial 'read' never quite completed (at 926 of my 1300 resources) before I broke the code to review.
Ideas on the below, or for interfacing with the MSP 2016 Reference library? It seems to have become very slow compared to just working within MS Excel.
Subroutine:
Private Sub cb_IMSResourceImport_Click()
Dim Prj As Project
Set Prj = GetObject(Me.cboMaintainToProject.Value)
Dim ResourceMatrix() As String
Prj.Application.WindowActivate Prj.Name
ReDim ResourceMatrix(Prj.Resources.Count, 2)
ActiveWorkbook.Sheets("Resource Table").Range("A2:C" & C`Str(ActiveWorkbook.ActiveSheet.UsedRange.Rows.Count)).ClearContents
For i = 1 To UBound(ResourceMatrix)
ResourceMatrix(i - 1, 0) = Prj.Resources(i).ID
ResourceMatrix(i - 1, 1) = Prj.Resources(i).Name
ResourceMatrix(i - 1, 2) = Prj.Resources(i).Code
Next i
For i = 0 To UBound(ResourceMatrix)
ActiveWorkbook.Sheets("Resource Table").Cells(i + 1, 1).Value = ResourceMatrix(i, 0)
ActiveWorkbook.Sheets("Resource Table").Cells(i + 1, 2).Value = ResourceMatrix(i, 1)
ActiveWorkbook.Sheets("Resource Table").Cells(i + 1, 3).Value = ResourceMatrix(i, 2)
Next i
End Sub
Using Project 2013 and a mocked-up schedule with 2000 resources, your code took less than 2 minutes to run. That said, if you have many schedules open, calculation left on, etc. it could take longer.
However, there is another way to copy data such as this from Project to Excel that is very fastβ€”use the clipboard.
Private Sub ExtractResources()
Dim prj As MSProject.Project
Set prj = GetObject(Me.cboMaintainToProject.Value)
Dim msp As MSProject.Application
Set msp = prj.Application
msp.WindowActivate prj.Name
ActiveWorkbook.Sheets("Resource Table").Range("A2:C" & CStr(ActiveWorkbook.ActiveSheet.UsedRange.Rows.Count)).ClearContents
msp.ViewApply "Resource Sheet"
msp.SelectSheet
msp.EditCopy
ActiveWorkbook.Sheets("Resource Table").Range("A2").PasteSpecial xlPasteValues
End Sub
Note: Modify the Resource View (or create your own) to define which columns to export out of Project. If you want the ID column to be exported, check the definition of the table used with the resource view to make sure the first column is not locked, otherwise it won't be included in the SelectSheet method.

Excel error 1004 "Unable to get .... property of WorksheetFunction class" appearing inconsistently

I have a VBA function within a spreadsheet which operates on another spreadsheet that is opened in an earlier stage of my macro. The macro used to work fine but just recently has started causing a 1004 error ("Unable to get RoundDown property of the WorksheetFunction class") when it runs.
I believe I understand what the error would be caused by (a problem running RoundDown) but I cannot see why it is getting triggered in my macro and the odd part is that when I go into Debug mode and step through the code in the VBE the error does not recur (despite nothing obviously changing).
Does anyone have a similar experience of this sort of error occuring inconsistently and know what I could do to resolve it?
I'm reasonably VBA/Excel-savvy, but any suggestions on further steps to diagnose it would be appreciated. I am wondering if there is some issue with the opened spreadsheet not being ready but I cannot see how.
The code is here. The error occurs on the line marked with a comment.
Public Function GetDatesA(sWorkbookname As String, sSheetname As String, sCell As String) As Variant
Dim vDateList() As Variant
Dim currentCell As Range
Dim n As Long
Set currentCell = Workbooks(sWorkbookname).Worksheets(sSheetname).Range(sCell)
n = 0
Do
If Trim(currentCell.Value) = "" Then
Exit Do
Else
ReDim Preserve vDateList(0 To 1, 0 To n)
vDateList(0, n) = WorksheetFunction.RoundDown(currentCell.Value, 0) 'error occcurs on this line
vDateList(1, n) = currentCell.Column
'Debug.Print currentCell.Value
End If
Set currentCell = currentCell.Offset(0, 1)
n = n + 1
Loop While currentCell.Column < XL_LAST_COLUMN
GetDatesA = vDateList
End Function
Other details are:
Excel version: 2010
File being opened resides locally on my C: drive; my macro is in a spreadsheet on the network
File format for both files is .xls (i.e. Excel 2003) - I don't have the option of changing this
Windows 7 (not that I think it would be relevant)
Two points I've tried already are:
Substitute a different worksheet function (e.g. Min(currentCell)) and that also causes the same problem
Having the file open already seems to stop the problem - I wonder if there is some way that the workbook which is being opened (rather than my main workbook with the macro in it) is not enabled for macros and this is interfering. But even if this is the cause I'm not sure how to get around it!
Any ideas?
This error occurs often when any argument passed to the worksheet function is not of the correct type or simply doesn't make sense.
For example, I've had this problem when calling WorksheetFunction.Asin with an argument bigger than 1. In your case, I'd guess currentCell.Value is a non-numeric value or one not according to your region settings regarding numbers.
Yes, the error message is really misguiding.
I got the "Unable to get * property of WorksheetFunction Class" error using Transpose, MMult,MDterm, and MInverse functions.
I was able to get my code to run by putting "Option Base 1" in the Declarations (before the actual code) section of the particular Module in the Editer.
Excel assumes "Option Base 0" which will add an extra row and column of empty cells. This will cause the error to occur and isn't immediately obvious to see.
I have come accross this before, and for me it was becase the criteria range made no sense, as Andre said above.
See example formula below:
.Cells(11, i).Formula = Application.WorksheetFunction.CountIfs(Sheets("Sheet1").Range("AC8:C" & n), "S")
Have a look at the Range... it makes no sense. Amended the range from "AC8:C" to "AC8:AC" and it will work perfectly

VBA Code in Excel randomly stops executing. No error messages occur

Essentially, I have an Updata button that takes information from two columns, in two spreadsheets (within 1 book). The overall goal of this code is to take all the values from one column, and then append the values from the other column below it.
Worksheets("Overall Flow").Range("A4:A1004").Value = Worksheets("Active").Range("A2:A1002").Value
Dim i As Integer
For i = 4 To 1004
If Worksheets("Overall Flow").Range("A" & Trim(str(i))) = "" Then
Worksheets("Overall Flow").Range("A" & Trim(str(i)) & ":A" & Trim(str(1000 + i))).Value = Worksheets("Inactive").Range("A2:A1002").Value
i = 1005
End If
Next
For some reason, the first line executes, and then finishes. When I put break points, then do step-by-step, no other steps happen afterwards.
When I run the first line individually, it appears to work fine, but not when:
Worksheets("Overall Flow").Range("A" & Trim(str(i)) & ":A" & Trim(str(1000 + i))).Value = Worksheets("Inactive").Range("A2:A1002").Value
or
Worksheets("Overall Flow").Range("A4:A1004").Value = Worksheets("Inactive").Range("A2:A1002").Value
is present aftwards.
Solution to this is very unusual.
CTRL+BREAK CTRL+BREAK CTRL+BREAK ESC
It just happened to me againg after long time, I was looking for a solution and I came here then this sequence came back to my mind and I tried.
It worked for me, I hope this will help someone.
Update: Tweaked code (now with error checking!)
Main points concerning the current code:
When copying the ACTIVE range, check for last consecutive cell used. This is faster and more effecient than a loop.
Why are you trimming a number you know will not contain spaces?
There's no need to set i = 1005, just use Exit For. This is more effecient and clear to the reader what the intention is. I don't use this in the code below since I avoided looping altogether.
Here's a different way you can do this without any looping, which I think is more clear and effecient. Try this and see if it works for you:
Sub test()
Dim lastRow As Long, offSet As Long
lastRow = Worksheets("Active").Range("A2").End(xlDown).row
'Sanity checks
If IsEmpty(Worksheets("Active").Range("A2")) = True Then offSet = 1: lastRow = 2
If lastRow > 1001 Then lastRow = 1002
Worksheets("Overall Flow").Range("A4:A" & lastRow + 2).Value = _
Worksheets("Active").Range("A2:A" & lastRow).Value
If lastRow < 1002 Then
Worksheets("Overall Flow").Range("A" & lastRow + (3 - offSet) & _
":A1004").Value = Worksheets("Inactive").Range("A2:A1002").Value
End If
End Sub
Notes:
Sanity check 1 is for if A2 is blank in the Active sheet.
Sanity check 2 is for if there are cells beyond A1002 with values in Active sheet.
This is what I am using to test your code. Since I don't know what's in the spreadsheets, I can't reproduce exactly what you're seeing so I'm first putting dummy data into the ranges.
For me it is running fine every time, and I've tried it on 2 different computers - Excel 2003, and Excel 2010.
I set a breakpoint and stepped with F8, and also Shift F8 and both worked fine.
Something may be different with your data (i.e. the first cell being copied over from the inactive sheet is blank and therefore execution stops after processing the first cell -- check that column A4 is not blank), or perhaps some memory has gotten corrupted from having Office being killed.
In a Module I have:
Sub test()
Worksheets("Active").Range("A2:A1002").Value = "active"
Worksheets("Active").Range("A5").Value = ""
Worksheets("Inactive").Range("A2:A1002").Value = "inactive"
Worksheets("Overall Flow").Range("A4:A1004").Value = Worksheets("Active").Range("A2:A1002").Value
Dim i As Integer
For i = 4 To 1004
If Worksheets("Overall Flow").Range("A" & Trim(Str(i))) = "" Then
Worksheets("Overall Flow").Range("A" & Trim(Str(i)) & ":A" & Trim(Str(1000 + i))).Value = Worksheets("Inactive").Range("A2:A1002").Value
i = 1005
End If
Next
End Sub
Have you tried the same code on another computer?
I had this issue and I tracked it down to custom VBA functions used in Conditional Formatting that was processed while application.screenupdating was still set to True.
I'm not sure how consistent this behaviour is but, when a custom VBA function is referred to in a conditional formatting rule, when the screen updates, it will not step through the code even when employing break points or the debug.assert method. Here's the breakdown of what happened:
Context:
2 open workbooks.
Conditional formatting and custom function in question were in workbook1.
The code I was attempting to execute was in workbook2.
Process
I call a procedure in workbook2.
Workbook2's procedure reaches a line executing an autofilter command.
Autofilter command triggers a screen update in all open workbooks (any command that triggers a Worksheet_Change or Worksheet_Calculate event can apply here).
Screen update processes the conditional formatting rules, including the rule in workbook1 calling workbook1's custom function.
Custom function is run in a 'silent' state (i.e. with no interaction with user, ignoring break points and "debug.assert" calls; this appears to be by design as part of the conditional formatting feature)
Custom function finishes execution and ceases all other active code execution.
I fixed my problem by adding a Application.ScreenUpdating = False line at the start to prevent screen updates and, by extension, conditional format processing (but it's best to keep custom functions away from conditional formatting to begin with).
I'm not sure if this is relevant to your situation at all but I hope it helps somebody.
It has already been mentioned in transistor1's answer, but only as a side comment.
I had a similar problem, that VBA code simply stopped executing in the middle of a function. Just before that it also jumped back a few lines of code. No Error Message was shown.
I closed all open Excel programs, and upon reopening the File everything worked fine again.
So my confirmed Answer to this problem is: Corrupted Memory, restart Excel.
Edit: after doing this, I also encountered the Problem that Visual Basic Editor crashed when I tried uncommenting a particular line. So I created a New Excel file and copied my code. Now I don't have any problems anymore.
I ran into the same problem. I had a sub routine that gave random errors throughout the code without giving error messages. By pressing F8, the code would resume.
I found someone had posted a Subroutine he called "ThatCleverDevil" I do not remember the resource or who posted it. It would warn you an error was about to occur. The routine is posted below.
I split the code into component sub-routines. The short snippits ran with no interruption or erros. I created a subroutine that called each snippit. Errors resumed.
They would run individually, but not all together.
RESOLUTION: Between called sub-routines, I ran the following line of code:
Application.Wait Second(Now) + 1
The code then ran without error.
Thanks to whomever it was that wrote ThatCleverDevil. And special thanks to the coder who wrote about Application.Wait.
Sub ThatCleverDevil()
On Error GoTo err
MsgBox "About to error"
err.Raise 12345
MsgBox "Got here after the error"
Exit Sub
err:
Stop: Resume
End Sub
Robert
VBA simply is prone to this issue. I have used it for years in corproate workflows because it is so hardcoded into lots of things, but if possible I would just consider alternatives. If this an ad-hoc project R will be faster and offer more flexibility. If this is more production oriented and meant to handle large volumes I would consider informatica.
To improve the performance I called the function DoEvents inside the loop. It solved the problem for me.

Excel UDF calculation should return 'original' value

I have created a VSTO plugin with my own RTD implementation that I am calling from my Excel sheets. To avoid having to use the full-fledged RTD syntax in the cells, I have created a UDF that hides that API from the sheet.
The RTD server I created can be enabled and disabled through a button in a custom Ribbon component.
The behavior I want to achieve is as follows:
If the server is disabled and a reference to my function is entered in a cell, I want the cell to display Disabled.
If the server is disabled, but the function had been entered in a cell when it was enabled (and the cell thus displays a value), I want the cell to keep displaying that value.
If the server is enabled, I want the cell to display Loading.
Sounds easy enough. Here is an example of the - non functional - code:
Public Function RetrieveData(id as Long)
Dim result as String
// This returns either 'Disabled' or 'Loading'
result = Application.Worksheet.Function.RTD("SERVERNAME", "", id)
RetrieveData = result
If(result = "Disabled") Then
// Obviously, this recurses (and fails), so that's not an option
If(Not IsEmpty(Application.Caller.Value2)) Then
// So does this
RetrieveData = Application.Caller.Value2
End If
End If
End Function
The function will be called in thousands of cells, so storing the 'original' values in another data structure would be a major overhead and I would like to avoid it. Also, the RTD server does not know the values, since it also does not keep a history of it, more or less for the same reason.
I was thinking that there might be some way to exit the function which would force it to not change the displayed value, but so far I have been unable to find anything like that.
EDIT:
Due to popular demand, some additional info on why I want to do all this:
As I said, the function will be called in thousands of cells and the RTD server needs to retrieve quite a bit of information. This can be quite hard on both network and CPU. To allow the user to decide for himself whether he wants this load on his machine, they can disable the updates from the server. In that case, they should still be able to calculate the sheets with the values currently in the fields, yet no updates are pushed into them. Once new data is required, the server can be enabled and the fields will be updated.
Again, since we are talking about quite a bit of data here, I would rather not store it somewhere in the sheet. Plus, the data should be usable even if the workbook is closed and loaded again.
Different tack=new answer.
A few things I've discovered the hard way, that you might find useful:
1.
In a UDF, returning the RTD call like this
' excel equivalent: =RTD("GeodesiX.RTD",,"status","Tokyo")
result = excel.WorksheetFunction.rtd( _
"GeodesiX.RTD", _
Nothing, _
"geocode", _
request, _
location)
behaves as if you'd inserted the commented function in the cell, and NOT the value returned by the RTD. In other words, "result" is an object of type "RTD-function-call" and not the RTD's answer. Conversely, doing this:
' excel equivalent: =RTD("GeodesiX.RTD",,"status","Tokyo")
result = excel.WorksheetFunction.rtd( _
"GeodesiX.RTD", _
Nothing, _
"geocode", _
request, _
location).ToDouble ' or ToString or whetever
returns the actual value, equivalent to typing "3.1418" in the cell. This is an important difference; in the first case the cell continues to participate in RTD feeding, in the second case it just gets a constant value. This might be a solution for you.
2.
MS VSTO makes it look as though writing an Office Addin is a piece of cake... until you actually try to build an industrial, distributable solution. Getting all the privileges and authorities right for a Setup is a nightmare, and it gets exponentially worse if you have the bright idea of supporting more than one version of Excel. I've been using Addin Express for some years. It hides all this MS nastiness and let's me focus on coding my addin. Their support is first-rate too, worth a look. (No, I am not affiliated or anything like that).
3.
Be aware that Excel can and will call Connect / RefreshData / RTD at any time, even when you're in the middle of something - there's some subtle multi-tasking going on behind the scenes. You'll need to decorate your code with the appropriate Synclock blocks to protect your data structures.
4.
When you receive data (presumably asynchronously on a separate thread) you absolutely MUST callback Excel on the thread on which you were intially called (by Excel). If you don't, it'll work fine for a while and then you'll start getting mysterious, unsolvable crashes and worse, orphan Excels in the background. Here's an example of the relevant code to do this:
Imports System.Threading
...
Private _Context As SynchronizationContext = Nothing
...
Sub New
_Context = SynchronizationContext.Current
If _Context Is Nothing Then
_Context = New SynchronizationContext ' try valiantly to continue
End If
...
Private Delegate Sub CallBackDelegate(ByVal GeodesicCompleted)
Private Sub GeodesicComplete(ByVal query As Query) _
Handles geodesic.Completed ' Called by asynchronous thread
Dim cbd As New CallBackDelegate(AddressOf GeodesicCompleted)
_Context.Post(Function() cbd.DynamicInvoke(query), Nothing)
End Sub
Private Sub GeodesicCompleted(ByVal query As Query)
SyncLock query
If query.Status = "OK" Then
Select Case query.Type
Case Geodesics.Query.QueryType.Directions
GeodesicCompletedTravel(query)
Case Geodesics.Query.QueryType.Geocode
GeodesicCompletedGeocode(query)
End Select
End If
' If it's not resolved, it stays "queued",
' so as never to enter the queue again in this session
query.Queued = Not query.Resolved
End SyncLock
For Each topic As AddinExpress.RTD.ADXRTDTopic In query.Topics
AddinExpress.RTD.ADXRTDServerModule.CurrentInstance.UpdateTopic(topic)
Next
End Sub
5.
I've done something apparently akin to what you're asking in this addin. There, I asynchronously fetch geocode data from Google and serve it up with an RTD shadowed by a UDF. As the call to GoogleMaps is very expensive, I tried 101 ways and several month's of evenings to keep the value in the cell, like what you're attempting, without success. I haven't timed anything, but my gut feeling is that a call to Excel like "Application.Caller.Value" is an order of magnitude slower than a dictionary lookup.
In the end I created a cache component which saves and re-loads values already obtained from a very-hidden spreadsheet which I create on the fly in Workbook OnSave. The data is stored in a Dictionary(of string, myQuery), where each myQuery holds all the relevant info.
It works well, fulfils the requirement for working offline and even for 20'000+ formulas it appears instantaneous.
HTH.
Edit: Out of curiosity, I tested my hunch that calling Excel is much more expensive than doing a dictionary lookup. It turns out that not only was the hunch correct, but frighteningly so.
Public Sub TimeTest()
Dim sw As New Stopwatch
Dim row As Integer
Dim val As Object
Dim sheet As Microsoft.Office.Interop.Excel.Worksheet
Dim dict As New Dictionary(Of Integer, Integer)
Const iterations As Integer = 100000
Const elements As Integer = 10000
For i = 1 To elements + 1
dict.Add(i, i)
Next
sheet = _ExcelWorkbook.ActiveSheet
sw.Reset()
sw.Start()
For i As Integer = 1 To iterations
row = 1 + Rnd() * elements
Next
sw.Stop()
Debug.WriteLine("Empty loop " & (sw.ElapsedMilliseconds * 1000) / iterations & " uS")
sw.Reset()
sw.Start()
For i As Integer = 1 To iterations
row = 1 + Rnd() * elements
val = sheet.Cells(row, 1).value
Next
sw.Stop()
Debug.WriteLine("Get cell value " & (sw.ElapsedMilliseconds * 1000) / iterations & " uS")
sw.Reset()
sw.Start()
For i As Integer = 1 To iterations
row = 1 + Rnd() * elements
val = dict(row)
Next
sw.Stop()
Debug.WriteLine("Get dict value " & (sw.ElapsedMilliseconds * 1000) / iterations & " uS")
End Sub
Results:
Empty loop 0.07 uS
Get cell value 899.77 uS
Get dict value 0.15 uS
Looking up a value in a 10'000 element Dictionary(Of Integer, Integer) is over 11'000 times faster than fetching a cell value from Excel.
Q.E.D.
Maybe... Try making your UDF wrapper function non-volatile, that way it won't get called unless one of its arguments changes.
This might be a problem when you enable the server, you'll have to trick Excel into calling your UDF again, it depends on what you're trying to do.
Perhaps explain the complete function you're trying to implement?
You could try Application.Caller.Text This has the drawback of returning the formatted value from the rendering layer as text, but seems to avoid the circular reference problem.Note: I have not tested this hack under all possible circumstances ...

Resources