I have a VB.net application which needs to read and write Excel workbooks, using OLEDB and the MS Office Access Database driver. Over time, I have learned a vast amount here on StackOverflow which allowed me to get everything working the way it should be. Those topics covered everything from connectionstrings to the "magic" TypeGuessRows setting / 255 character limitation, now located in the registry. Over time, that issue by itself has morphed due to the evolutions by Microsoft over the years, so I won't reference those (hundreds? thousands?) of posts here. Suffice it to say that I was able to piece it all together and get it working.
For reference: my connectionstring looks like this:
"Provider=Microsoft.ACE.OLEDB.12.0;Data Source='#DBQ';Extended Properties='Excel 12.0 Xml;HDR=YES';"
where '#DBQ' is replaced at runtime by the appropriate pathname of the selected Excel workbook. Note that it does not contain TypeGuessRows (since that has moved to the registry), and IMEX=(whatever) has not effect either, so it does not appear. The result is that everything works quite well, AS LONG AS TYPEGUESSROWS is set to zero in the registry.
Using regedit to manually hunt down and set TypeGuessRows=0 in all occurrences (one of the things I learned about here on StackOverflow), this has worked and avoids the 255-char truncation perfectly.
The problem is that whenever there is a Windows update, it gets set back to the default value, 8. There is no warning or announcement of this; I only know about it when I start seeing the "truncated at 255 characters" symptom showing up again. So then I have to go back and use RegEdit again.
The obvious solution would be that the app does this setting on its own, but that seems to require manipulation of the Security features, which is something I'd prefer to avoid entirely as it seems to be more complex than I want to deal with presently.
So, I thought, "why not just scan the appropriate registry keys, and report if they are not zero", then issue a message to the user to get it sorted using regedit.
I wrote the following function, which looks at all of the various registry keys I have found which contain TypeGuessRows:
Public Function CheckRegistry() As Integer
Dim Val1 As Integer = 0, Val2 As Integer = 0
Dim KeysToCheck As String() = New String() {
"HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\Jet\4.0\Engines\Excel",
"HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\Jet\4.0\Engines\Lotus",
"HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\Office\14.0\Access Connectivity Engine\Engines\Excel",
"HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office\ClickToRun\REGISTRY\MACHINE\Software\Wow6432Node\Microsoft\Office\16.0\Access Connectivity Engine\Engines\Excel"
}
'
' try two methods for checking the registry values.
'
' method 1 uses the My.Computer class
'
For Each Key As String In KeysToCheck
Dim aVal As Object = My.Computer.Registry.GetValue(Key, "TypeGuessRows", CType("BOGUS", Object))
If IsNothing(aVal) Then ' did we hit Nothing?
Continue For ' have to skip it
ElseIf aVal.Equals(CType("BOGUS", Object)) Then ' did we hit one not exist?
Val1 = -9999 ' set a crazy value
End If
Val1 += CInt(aVal) ' otherwise, get its value and add to total
Next
'
' method 2 uses the RegistryKey class
'
Dim aKey As Microsoft.Win32.RegistryKey
For Each Key As String In KeysToCheck
Dim LM As Microsoft.Win32.RegistryKey = Microsoft.Win32.Registry.LocalMachine ' point to HKLM
Key = Key.TrimStart("HKEY_LOCAL_MACHINE\".ToCharArray) ' peel off the start (a convenience)
aKey = LM.OpenSubKey(Key, Microsoft.Win32.RegistryKeyPermissionCheck.ReadSubTree) ' try to read the subkey
If IsNothing(aKey) Then Continue For ' did we hit Nothing? have to skip it
'If CInt(aKey.GetValue("TypeGuessRows", 0)) <> 0 Then aKey.SetValue("TypeGuessRows", 0)
Val2 += CInt(aKey.GetValue("TypeGuessRows", 0)) ' otherwise, get its value and add to total
Next
Return Val1 + Val2 ' return the sum of both methods
End Function
This works well enough, EXCEPT FOR THE LAST ONE, the one with "...ClickToRun..." in the key name.
No matter what, this ALWAYS returns as Nothing, so the attempt to check it fails. Even in the first method, where I specifically ask it to return a "BOGUS" object, it always returns as Nothing.
The problem is that it is THIS KEY which gets updated by Windows Update. Thus far I have never seen any of the other three get reset.
If I "walk the tree" of this key (i.e., try looking at "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office"), it works fine until I hit the ClickToRun branch, at which point it just returns Nothing.
So, I guess I have two questions:
Is there a way to read this key so that it does not return as Nothing?
Can someone provide a (hopefully not too messy / complex) guide to getting the Security settings correct to allow "ReadWriteSubTree" for the permission check? N.B. if I try that now, I get Security exception, telling me that the app does not have the appropriate permission. That's why the aKey.SetValue line is commented out.
Further info:
the machine is Win 10 Pro, 64-bit, with appropriate drivers installed. The app is set to "Enable ClickToRun security settings", and "full trust application". I have played with those settings and the appropriate entries in app.manifest, but thus far that only causes (much) other grief and difficulties elsewhere. That's why I'd prefer to not mess with it.
Any help would be really appreciated.
After some sleep, some more thought and in particular, some hints found [here]
Reading 64bit Registry from a 32bit application I managed to at least get the ability to read the registry keys.
The problem was that I was trying to "read" 64-bit registry keys from my 32-bit app.
Now I can do what I really wanted to do, which was to detect when the TypeGuessRows issue has cropped up again, and warn the user to do something about it.
Here is the modified code:
Public Function CheckRegistry() As Integer
Dim CheckVal As Integer = 0
Dim KeysToCheck As String() = New String() {
"SOFTWARE\WOW6432Node\Microsoft\Jet\4.0\Engines\Excel",
"SOFTWARE\WOW6432Node\Microsoft\Jet\4.0\Engines\Lotus",
"SOFTWARE\WOW6432Node\Microsoft\Office\14.0\Access Connectivity Engine\Engines\Excel",
"SOFTWARE\Microsoft\Office\ClickToRun\REGISTRY\MACHINE\Software\Wow6432Node\Microsoft\Office\16.0\Access Connectivity Engine\Engines\Excel"
}
Dim LMachine As Microsoft.Win32.RegistryKey =
Microsoft.Win32.RegistryKey.OpenBaseKey(Microsoft.Win32.RegistryHive.LocalMachine,
Microsoft.Win32.RegistryView.Registry64)
Dim LKey As Microsoft.Win32.RegistryKey
For Each Key As String In KeysToCheck
Try
LKey = LMachine.OpenSubKey(Key, Microsoft.Win32.RegistryKeyPermissionCheck.ReadSubTree)
CheckVal += CInt(LKey.GetValue("TypeGuessRows", 0))
Catch ex As Exception
LMachine.Close()
Return -9999
End Try
Next
LMachine.Close()
Return CheckVal
End Function
In testing this out, it can successfully check all four of the keys shown. The try-catch is never fired, but I left it there for safety anyway.
I hope that someone else might find this useful.
Related
I can seem to do everything else just fine with my scanner, using WIA and VBA, but when I try to set my scanner to detect the page size, I get an error message "The parameter is incorrect." I've searched extensively for a solution without any luck. Someone else asked a similar question here 11 years ago (Set page size using WIA (with scanner)), but the answer didn't really address my question and later comments claimed the solution didn't work. Here's a code snippet I created to demonstrate my problem. In the snippet I first set another property (the "pages" property) to show that I can set properties, but the "page size" property doesn't work.
Sub SetScannerToAutoSize(DeviceID As String)
Dim wiaScanner As WIA.Device
Dim wiaDM As New WIA.DeviceManager
Set wiaScanner = wiaDM.DeviceInfos(DeviceID).Connect ' DeviceID is "{6BDD1FC6-810F-11D0-BEC7-08002BE2092F}\0003"
wiaScanner.Properties.Item(wiaDocumentHandlingSelect).Value = WIA_FEEDER + WIA_DUPLEX ' This works fine setting it to do double-sided scanning
wiaScanner.Properties.Item(wiaPages).Value = WIA_ALL_PAGES ' Works -- wiaPages = "3096", WIA_ALL_PAGES = 1 Tells the scanner to scan all the pages in the feeder.
'
' Next line results in "Run-time error '-2147024809 (80070057)': - The parameter is incorrect"
'
wiaScanner.Properties.Item(wiaPageSize).Value = WIA_PAGE_AUTO ' Doesn't work -- wiaPageSize = "3097", WIA_PAGE_AUTO = 100 'I expected it to tell the scanner to detect the page size
End Sub
According to the device info for the scanner, it is capable of performing this feature. It's a "Brother ADS-1700W LAN" scanner. I also tried it on my EPSON WF-2750 scanner with the same results.
I hope someone can tell me what I'm doing wrong. I need to deliver my app soon.
I'm also trying to get the SDK from Brother, but so far no luck.
I am coding a Manager in Excel-VBA with several buttons.
One of them is to generate a tab using another Excel file (let me call it T) as input.
Some properties of T:
~90MB size
~350K lines
Contains sales data of the last 14 months (unordered).
Relevant columns:
year/month
total-money
seller-name
family-product
client-name
There is not id columns (like: cod-client, cod-vendor, etc.)
Main relation:
Sellers sells many Products to many Clients
I am generating a new Excel tab with data from T of the last year/month grouped by Seller.
Important notes:
T is the only available input/source.
If two or more Sellers sells the same Product to the same Client, the total-money should be counted to all of those Sellers.
This is enough, now you know what I have already coded.
My code works, but, it takes about 4 minutes of runtime.
I have already coded some other buttons using smaller sources (not greater than 2MB) which runs in 5 seconds.
Considering T size, 4 minutes runtime could be acceptable.
But I'm not proud of it, at least not yet.
My code is mainly based on Scripting.Dictionary to map data from T, and then I use for each key in obj ... next key to set the grouped data to the new created tab.
I'm not sure, but here are my thoughts:
If N is the total keys in a Scripting.Dictionary, and I need to check for obj.Exists(str) before aggregating total-money. It will run N string compares to return false.
Similarly it will run maximun N string compares when I do Set seller = obj(seller_name).
I want to be wrong with my thoughts. But if I'm not wrong, my next step (and last hope) to reduce the runtime of this function is to code my own class object with Tries.
I will only start coding tomorrow, what I want is just some confirmation if I am in the right way, or some advices if I am in the wrong way of doing it.
Do you have any suggestions? Thanks in advance.
Memory Limit Exceeded
In short:
The main problem was because I used a dynamic programming approach of storing information (preprocessing) to make the execution time faster.
My code now runs in ~ 13 seconds.
There are things we learn the hard way. But I'm glad I found the answer.
Using the Task Manager I was able to see my code reaching 100% memory usage.
The DP approach I mentioned above using Scripting.Dictionary reached 100% really faster.
The DP approach I mentioned above using my own cls_trie implementation also reached 100%, but later than the first.
This explains the ~4-5 min compared to ~2-3 min total runtime of above attempts.
In the Task Manager I could also see that the CPU usage never hited 2%.
Solution was simple, I had to balance CPU and Memory usages.
I changed some DP approaches to simple for-loops with if-conditions.
The CPU usage now hits ~15%.
The Memory usage now hits ~65%.
I know this is relative to the CPU and Memory capacity of each machine. But in the client machine it is also running in no more than 15 seconds now.
I created one GitHub repository with my cls_trie implementation and added one excel file with an example usage.
I'm new to the excel-vba world (4 months working with it right now). There might probably have some ways to improve my cls_trie implementation, I'm openned to suggestions:
Option Explicit
Public Keys As Collection
Public Children As Variant
Public IsLeaf As Boolean
Public tObject As Variant
Public tValue As Variant
Public Sub Init()
Set Keys = New Collection
ReDim Children(0 To 255) As cls_trie
IsLeaf = False
Set tObject = Nothing
tValue = 0
End Sub
Public Function GetNodeAt(index As Integer) As cls_trie
Set GetNodeAt = Children(index)
End Function
Public Sub CreateNodeAt(index As Integer)
Set Children(index) = New cls_trie
Children(index).Init
End Sub
'''
'Following function will retrieve node for a given key,
'creating a entire new branch if necessary
'''
Public Function GetNode(ByRef key As Variant) As cls_trie
Dim node As cls_trie
Dim b() As Byte
Dim i As Integer
Dim pos As Integer
b = CStr(key)
Set node = Me
For i = 0 To UBound(b) Step 2
pos = b(i) Mod 256
If (node.GetNodeAt(pos) Is Nothing) Then
node.CreateNodeAt pos
End If
Set node = node.GetNodeAt(pos)
Next
If (node.IsLeaf) Then
'already existed
Else
node.IsLeaf = True
Keys.Add key
End If
Set GetNode = node
End Function
'''
'Following function will get the value for a given key
'Creating the key if necessary
'''
Public Function GetValue(ByRef key As Variant) As Variant
Dim node As cls_trie
Set node = GetNode(key)
GetValue = node.tValue
End Function
'''
'Following sub will set a value to a given key
'Creating the key if necessary
'''
Public Sub SetValue(ByRef key As Variant, value As Variant)
Dim node As cls_trie
Set node = GetNode(key)
node.tValue = value
End Sub
'''
'Following sub will sum up a value for a given key
'Creating the key if necessary
'''
Public Sub SumValue(ByRef key As Variant, value As Variant)
Dim node As cls_trie
Set node = GetNode(key)
node.tValue = node.tValue + value
End Sub
'''
'Following function will validate if given key exists
'''
Public Function Exists(ByRef key As Variant) As Boolean
Dim node As cls_trie
Dim b() As Byte
Dim i As Integer
b = CStr(key)
Set node = Me
For i = 0 To UBound(b) Step 2
Set node = node.GetNodeAt(b(i) Mod 256)
If (node Is Nothing) Then
Exists = False
Exit Function
End If
Next
Exists = node.IsLeaf
End Function
'''
'Following function will get another Trie from given key
'Creating both key and trie if necessary
'''
Public Function GetTrie(ByRef key As Variant) As cls_trie
Dim node As cls_trie
Set node = GetNode(key)
If (node.tObject Is Nothing) Then
Set node.tObject = New cls_trie
node.tObject.Init
End If
Set GetTrie = node.tObject
End Function
You can see in the above code:
I hadn't implemented any delete method because I didn't need it till now. But it would be easy to implement.
I limited myself to 256 children because in this project the text I'm working on is basically lowercase and uppercase [a-z] letters and numbers, and the probability that two text get mapped to the same branch node tends zero.
as a great coder said, everyone likes his own code even if other's code is too beautiful to be disliked [1]
My conclusion
I will probably never more use Scripting.Dictionary, even if it is proven that somehow it could be better than my cls_trie implementation.
Thank you all for the help.
I'm convinced that you've already found the right solution because there wasn't any update for last two years.
Anyhow, I want to mention (maybe it will help someone else) that your bottleneck isn't the Dictionary or Binary Tree. Even with millions of rows the processing in memory is blazingly fast if you have sufficient amount of RAM.
The botlleneck is usually the reading of data from worksheet and writing it back to the worksheet. Here the arrays come very userfull.
Just read the data from worksheet into the Variant Array.
You don't have to work with that array right away. If it is more comfortable for you to work with dictionary, just transfer all the data from array into dictionary and work with it. Since this process is entirely made in memory, don't worry about the performance penalisation.
When you are finished with data processing in dictionary, put all data from dictionary back to the array and write that array into a new worksheet at one shot.
Worksheets("New Sheet").Range("A1").Value = MyArray
I'm pretty sure it will take only few seconds
In VBA, I am trying to determine whether or not a cell contains an error value, e.g. due to an invalid function. My existing code uses the Excel.WorksheetFunction.IsError method, but I recently came across a case which caused a false positive. The cell does not contain an error value, but Excel.WorksheetFunction.IsError returns true. Another method, VBA.Information.IsError, does not exhibit this behavior; it correctly returns false.
I was able to determine that the issue only occurs when the cell contains a value exceeding 255 characters in length. Here is a routine to verify the behavior:
Sub IsErrorBehavior()
Dim i As Long
Dim str As String
Dim r1 As Range, r2 As Range
Dim xlWSFuncCheck1 As Boolean, xlWSFuncCheck2 As Boolean
Dim vbaInfCheck1 As Boolean, vbaInfCheck2 As Boolean
str = WorksheetFunction.Rept("A", 255)
Set r1 = Range("A1")
r1.Value = str
xlWSFuncCheck1 = Excel.WorksheetFunction.IsError(r1)
vbaInfCheck1 = VBA.Information.IsError(r1)
str = str & "A"
Set r2 = Range("A2")
r2.Value = str
xlWSFuncCheck2 = Excel.WorksheetFunction.IsError(r2)
vbaInfCheck2 = VBA.Information.IsError(r2)
End Sub
The above routine was written in Excel 2010 and verified in Excel 2007. The target application is currently running under Excel 2007.
Question 1: Is this a bug or is there a reasonable explanation for the behavior of Excel.WorksheetFunction.IsError in this case?
I will be switching to the VBA.Information.IsError method, but now I'm a bit concerned that I may encounter bugs with it as well. Which leads me to...
Question 2: Is there a more reliable way to check for errors in a specific cell?
Not a complete response for Excel 2007, but might give some ideas for other peers to follow up.
Question 1: Is this a bug or is there a reasonable explanation for the behavior of Excel.WorksheetFunction.IsError in this case?
There used to be an old known bug with the 255 character limit that was supposed to be fixed in newer versions. Your issue is of very similar nature so it could be the same source, although it is not possible to say for sure.
It seems that setting a Value of a Range equal to a String of more than 255 characters corrupts the Range itself, and makes the IsError check on the Range false.
Question 2: Is there a more reliable way to check for errors in a specific cell?
I would do it as in CPearson's website, namely checking r2.Value instead of r2, and using the IsError function (which is the VBA IsError) that has more stable performance. In general, checking a Range object for error might be tricky. In this question a while ago I was getting funny results because of type casting that goes on in the background. I guess this might be the case here as well. In any case, being explicit helps (your example works as expected if r2.value is used instead).
I encountered this error, too, and in our case it was independent of the use or omission of ".value".
Suppose, in cell A1, you enter
=1-BINOM.DIST(35, 76, 0.1, TRUE)
The resulting probability is minuscule, so Excel will return 0.000000 as the cell value.
However, when you then use VBA to test
IsError(ActiveSheet.Range("A1").Value)
VBA will return TRUE. Probably this is due to some internal rounding error checking where Excel realizes that the return value of the formula is to close to zero to be correctly calculated.
This is an unfortunate situation since Excel will display the return value as a cell value without any error or warning, while at the same time VBA will return TRUE on IsError(). I don't know whether this bug is present in other Excel functions as well, such as
=1-NORM.S.VERT(42, TRUE)
but would advise caution when working with IsError().
I have a VBA function within a spreadsheet which operates on another spreadsheet that is opened in an earlier stage of my macro. The macro used to work fine but just recently has started causing a 1004 error ("Unable to get RoundDown property of the WorksheetFunction class") when it runs.
I believe I understand what the error would be caused by (a problem running RoundDown) but I cannot see why it is getting triggered in my macro and the odd part is that when I go into Debug mode and step through the code in the VBE the error does not recur (despite nothing obviously changing).
Does anyone have a similar experience of this sort of error occuring inconsistently and know what I could do to resolve it?
I'm reasonably VBA/Excel-savvy, but any suggestions on further steps to diagnose it would be appreciated. I am wondering if there is some issue with the opened spreadsheet not being ready but I cannot see how.
The code is here. The error occurs on the line marked with a comment.
Public Function GetDatesA(sWorkbookname As String, sSheetname As String, sCell As String) As Variant
Dim vDateList() As Variant
Dim currentCell As Range
Dim n As Long
Set currentCell = Workbooks(sWorkbookname).Worksheets(sSheetname).Range(sCell)
n = 0
Do
If Trim(currentCell.Value) = "" Then
Exit Do
Else
ReDim Preserve vDateList(0 To 1, 0 To n)
vDateList(0, n) = WorksheetFunction.RoundDown(currentCell.Value, 0) 'error occcurs on this line
vDateList(1, n) = currentCell.Column
'Debug.Print currentCell.Value
End If
Set currentCell = currentCell.Offset(0, 1)
n = n + 1
Loop While currentCell.Column < XL_LAST_COLUMN
GetDatesA = vDateList
End Function
Other details are:
Excel version: 2010
File being opened resides locally on my C: drive; my macro is in a spreadsheet on the network
File format for both files is .xls (i.e. Excel 2003) - I don't have the option of changing this
Windows 7 (not that I think it would be relevant)
Two points I've tried already are:
Substitute a different worksheet function (e.g. Min(currentCell)) and that also causes the same problem
Having the file open already seems to stop the problem - I wonder if there is some way that the workbook which is being opened (rather than my main workbook with the macro in it) is not enabled for macros and this is interfering. But even if this is the cause I'm not sure how to get around it!
Any ideas?
This error occurs often when any argument passed to the worksheet function is not of the correct type or simply doesn't make sense.
For example, I've had this problem when calling WorksheetFunction.Asin with an argument bigger than 1. In your case, I'd guess currentCell.Value is a non-numeric value or one not according to your region settings regarding numbers.
Yes, the error message is really misguiding.
I got the "Unable to get * property of WorksheetFunction Class" error using Transpose, MMult,MDterm, and MInverse functions.
I was able to get my code to run by putting "Option Base 1" in the Declarations (before the actual code) section of the particular Module in the Editer.
Excel assumes "Option Base 0" which will add an extra row and column of empty cells. This will cause the error to occur and isn't immediately obvious to see.
I have come accross this before, and for me it was becase the criteria range made no sense, as Andre said above.
See example formula below:
.Cells(11, i).Formula = Application.WorksheetFunction.CountIfs(Sheets("Sheet1").Range("AC8:C" & n), "S")
Have a look at the Range... it makes no sense. Amended the range from "AC8:C" to "AC8:AC" and it will work perfectly
I have created a VSTO plugin with my own RTD implementation that I am calling from my Excel sheets. To avoid having to use the full-fledged RTD syntax in the cells, I have created a UDF that hides that API from the sheet.
The RTD server I created can be enabled and disabled through a button in a custom Ribbon component.
The behavior I want to achieve is as follows:
If the server is disabled and a reference to my function is entered in a cell, I want the cell to display Disabled.
If the server is disabled, but the function had been entered in a cell when it was enabled (and the cell thus displays a value), I want the cell to keep displaying that value.
If the server is enabled, I want the cell to display Loading.
Sounds easy enough. Here is an example of the - non functional - code:
Public Function RetrieveData(id as Long)
Dim result as String
// This returns either 'Disabled' or 'Loading'
result = Application.Worksheet.Function.RTD("SERVERNAME", "", id)
RetrieveData = result
If(result = "Disabled") Then
// Obviously, this recurses (and fails), so that's not an option
If(Not IsEmpty(Application.Caller.Value2)) Then
// So does this
RetrieveData = Application.Caller.Value2
End If
End If
End Function
The function will be called in thousands of cells, so storing the 'original' values in another data structure would be a major overhead and I would like to avoid it. Also, the RTD server does not know the values, since it also does not keep a history of it, more or less for the same reason.
I was thinking that there might be some way to exit the function which would force it to not change the displayed value, but so far I have been unable to find anything like that.
EDIT:
Due to popular demand, some additional info on why I want to do all this:
As I said, the function will be called in thousands of cells and the RTD server needs to retrieve quite a bit of information. This can be quite hard on both network and CPU. To allow the user to decide for himself whether he wants this load on his machine, they can disable the updates from the server. In that case, they should still be able to calculate the sheets with the values currently in the fields, yet no updates are pushed into them. Once new data is required, the server can be enabled and the fields will be updated.
Again, since we are talking about quite a bit of data here, I would rather not store it somewhere in the sheet. Plus, the data should be usable even if the workbook is closed and loaded again.
Different tack=new answer.
A few things I've discovered the hard way, that you might find useful:
1.
In a UDF, returning the RTD call like this
' excel equivalent: =RTD("GeodesiX.RTD",,"status","Tokyo")
result = excel.WorksheetFunction.rtd( _
"GeodesiX.RTD", _
Nothing, _
"geocode", _
request, _
location)
behaves as if you'd inserted the commented function in the cell, and NOT the value returned by the RTD. In other words, "result" is an object of type "RTD-function-call" and not the RTD's answer. Conversely, doing this:
' excel equivalent: =RTD("GeodesiX.RTD",,"status","Tokyo")
result = excel.WorksheetFunction.rtd( _
"GeodesiX.RTD", _
Nothing, _
"geocode", _
request, _
location).ToDouble ' or ToString or whetever
returns the actual value, equivalent to typing "3.1418" in the cell. This is an important difference; in the first case the cell continues to participate in RTD feeding, in the second case it just gets a constant value. This might be a solution for you.
2.
MS VSTO makes it look as though writing an Office Addin is a piece of cake... until you actually try to build an industrial, distributable solution. Getting all the privileges and authorities right for a Setup is a nightmare, and it gets exponentially worse if you have the bright idea of supporting more than one version of Excel. I've been using Addin Express for some years. It hides all this MS nastiness and let's me focus on coding my addin. Their support is first-rate too, worth a look. (No, I am not affiliated or anything like that).
3.
Be aware that Excel can and will call Connect / RefreshData / RTD at any time, even when you're in the middle of something - there's some subtle multi-tasking going on behind the scenes. You'll need to decorate your code with the appropriate Synclock blocks to protect your data structures.
4.
When you receive data (presumably asynchronously on a separate thread) you absolutely MUST callback Excel on the thread on which you were intially called (by Excel). If you don't, it'll work fine for a while and then you'll start getting mysterious, unsolvable crashes and worse, orphan Excels in the background. Here's an example of the relevant code to do this:
Imports System.Threading
...
Private _Context As SynchronizationContext = Nothing
...
Sub New
_Context = SynchronizationContext.Current
If _Context Is Nothing Then
_Context = New SynchronizationContext ' try valiantly to continue
End If
...
Private Delegate Sub CallBackDelegate(ByVal GeodesicCompleted)
Private Sub GeodesicComplete(ByVal query As Query) _
Handles geodesic.Completed ' Called by asynchronous thread
Dim cbd As New CallBackDelegate(AddressOf GeodesicCompleted)
_Context.Post(Function() cbd.DynamicInvoke(query), Nothing)
End Sub
Private Sub GeodesicCompleted(ByVal query As Query)
SyncLock query
If query.Status = "OK" Then
Select Case query.Type
Case Geodesics.Query.QueryType.Directions
GeodesicCompletedTravel(query)
Case Geodesics.Query.QueryType.Geocode
GeodesicCompletedGeocode(query)
End Select
End If
' If it's not resolved, it stays "queued",
' so as never to enter the queue again in this session
query.Queued = Not query.Resolved
End SyncLock
For Each topic As AddinExpress.RTD.ADXRTDTopic In query.Topics
AddinExpress.RTD.ADXRTDServerModule.CurrentInstance.UpdateTopic(topic)
Next
End Sub
5.
I've done something apparently akin to what you're asking in this addin. There, I asynchronously fetch geocode data from Google and serve it up with an RTD shadowed by a UDF. As the call to GoogleMaps is very expensive, I tried 101 ways and several month's of evenings to keep the value in the cell, like what you're attempting, without success. I haven't timed anything, but my gut feeling is that a call to Excel like "Application.Caller.Value" is an order of magnitude slower than a dictionary lookup.
In the end I created a cache component which saves and re-loads values already obtained from a very-hidden spreadsheet which I create on the fly in Workbook OnSave. The data is stored in a Dictionary(of string, myQuery), where each myQuery holds all the relevant info.
It works well, fulfils the requirement for working offline and even for 20'000+ formulas it appears instantaneous.
HTH.
Edit: Out of curiosity, I tested my hunch that calling Excel is much more expensive than doing a dictionary lookup. It turns out that not only was the hunch correct, but frighteningly so.
Public Sub TimeTest()
Dim sw As New Stopwatch
Dim row As Integer
Dim val As Object
Dim sheet As Microsoft.Office.Interop.Excel.Worksheet
Dim dict As New Dictionary(Of Integer, Integer)
Const iterations As Integer = 100000
Const elements As Integer = 10000
For i = 1 To elements + 1
dict.Add(i, i)
Next
sheet = _ExcelWorkbook.ActiveSheet
sw.Reset()
sw.Start()
For i As Integer = 1 To iterations
row = 1 + Rnd() * elements
Next
sw.Stop()
Debug.WriteLine("Empty loop " & (sw.ElapsedMilliseconds * 1000) / iterations & " uS")
sw.Reset()
sw.Start()
For i As Integer = 1 To iterations
row = 1 + Rnd() * elements
val = sheet.Cells(row, 1).value
Next
sw.Stop()
Debug.WriteLine("Get cell value " & (sw.ElapsedMilliseconds * 1000) / iterations & " uS")
sw.Reset()
sw.Start()
For i As Integer = 1 To iterations
row = 1 + Rnd() * elements
val = dict(row)
Next
sw.Stop()
Debug.WriteLine("Get dict value " & (sw.ElapsedMilliseconds * 1000) / iterations & " uS")
End Sub
Results:
Empty loop 0.07 uS
Get cell value 899.77 uS
Get dict value 0.15 uS
Looking up a value in a 10'000 element Dictionary(Of Integer, Integer) is over 11'000 times faster than fetching a cell value from Excel.
Q.E.D.
Maybe... Try making your UDF wrapper function non-volatile, that way it won't get called unless one of its arguments changes.
This might be a problem when you enable the server, you'll have to trick Excel into calling your UDF again, it depends on what you're trying to do.
Perhaps explain the complete function you're trying to implement?
You could try Application.Caller.Text This has the drawback of returning the formatted value from the rendering layer as text, but seems to avoid the circular reference problem.Note: I have not tested this hack under all possible circumstances ...