I have written a fairly complex application that uses Excel (2016) as front-end application for data stored in a MS Access database, using the ADO 6.1 library. I have noticed during macro code execution that SQL transactions triggered by my Excel application can take quite long to complete, and often the execution of the next line of code in my Excel macro depends on this SQL transaction first being completed. Unfortunately, macro code execution and SQL transactions are asynchronous operations, which means that the macro jumps to the next line of code even though the SQL transaction hasn't been completed.
My current work-around is to use a Sleep() function using the Windows API to insert a fixed delay, but this is a really ugly solution as it does reduce the performance of my application and depends very much on the CPU load, so it may sometimes work, sometimes not.
I haven't been able to find so far a solution to this problem, I can't find any hints on the Internet either.
Using Application.CalculateUntilAsyncQueriesDone doesn't help here either.
Does anyone have an idea or a hint how to halt macro code execution in Excel until an SQL transaction has been completed? Is there a method in ADO to check the completion of an SQL transaction?
Is your query within the Data/connections section?
I had this problem to, I turned off "Enable Background refresh" and added "DoEvents" to the VBA, this forces the data connection to refresh before it allows the code to continue. Downside to this is it makes excel feel like its locked up, But this resolved my issue.
Sub Button1_Click()
ActiveWorkbook.Connections("ScrapData").Refresh
DoEvents
....Other code....
End With
Related
I am handling RTD feeds again and remembering the difficulties, but now we have multi-core machines and multi-threading. maybe anyone can advise.
As I understand/rememeber: pushing data into Excel is not one(obvious reasons) so it sends a friendly nod to say your parcel is ready come and get it. Then when Excel has done its nails and feels in the mood, it might get the data.
So this kind of architecture is right back on the dance-floor and hopefully I can make it work.
Problem.
I need to run a bot that examines the data and responds differently;
1. looping or running a winapi timer even is still enough to keep excel busy and no data, or so it seems.
Definitely, executing bot logic , however small, will bring on an Excel fainting fit.
Tried responding via calculation event. Very hit-and-miss and definitely not up to the job. There is no logic obvious as to when and why it fires or does not other than a " bad hair day"
Tried winapi timer looking at the newly acquired data every second comparing to old in a separate data structure, running some EMAs and making a decision. No dice.
The timer is enough to put a delay up to 10 or even 20 seconds between occasional delivery of data.
Options I am thinking about:
1. The timer running outside of the excel environment and lookin in a the data. e.g an AQddon via pia etc. What I don't know is whether this Addon, perhaps inc C# or vb.net, could utilze multithreading, via tasks I think and do its bit without "scarin the panties of her ladyship"?
2. I remember hearing that XLL UDFs could be asynchronous, does anyone know if this is a potential option?
Any ideas?
I built a package in SSIS that uses a script task to open an Excel file, format, and refresh some data in Excel. I would like to have Excel visible when the script task is running to see if Excel gets hung up which occurs all the time. Is this possible? I am converting a process that is calling Excel via a shell script to using SSIS to call Excel instead. I guess a second question is, is that a bad idea?
Why this is a bad idea
Generally speaking, administrators are tasked with maximizing the amount of "uptime" a server or service on the server has. The more software that gets installed on the machine, the greater the odds of service interruptions and outages due to patching. To be able to manipulate Excel in the mechanism you described, you're going to force the installation of MS Office on that machine. That will cost you a software license and the amount of patching required is going to blow holes in whatever SLAs those admins might be required to adhere to.
Memory leaks. Along with the whole patching bit, in the past at least, there were issues with programmatically manipulating Excel and it basically boiled down to it was easy to end up with memory leaks (I gotta make you understand. Allocated memory but never given it up, never let the allocated memory go down). Over time, the compounded effect is that running this package will result in less and less system memory available and the only way to reclaim it is through a reboot, which gets back to SLAs.
The reason you want to see what Excel is doing is so that you can monitor execution because it "gets hung up which occurs all the time". That doesn't sound like a stable process. Again, no admin is going to want an unstable process running on the servers. Something is not right in the cycle of events. Whether it's your code that opens Excel, the macros it runs, etc, something in there is awry and that's why you need to inspect the process. This is akin to putting a bandaid on a shotgun wound. Stop shooting yourself and you won't require bandages.
The task that you're attempting to perform is "open an Excel file, format, and refresh some data in Excel" SSIS can natively push data into Excel. If you preformat the file, develop your SSIS to write to the formatted file and just copy it off, that should work. It's not graceful but it works. There are better methods of providing formatted data but without knowing your infrastructure, I don't know if SSRS, SharePoint, Excel Services, Power Pivot, etc are viable options.
Why you won't be able to see Excel
Generally speaking, the account that runs SQL Agent is probably going to be fairly powerful. To prevent things like a shatter attack, from Windows 2008+ services are restricted in what they can do. For the service account to be able to interact with the desktop, you have to move it into the user tier of apps which might not be a good thing if you, or your DBA/admins, are risk adverse.
For more information, please to enjoy the following links
InteractWithDesktop
http://lostechies.com/keithdahlby/2011/08/13/allowing-a-windows-service-to-interact-with-desktop-without-localsystem/
https://serverfault.com/questions/576144/allow-service-to-interact-with-desktop
https://superuser.com/questions/415204/how-do-i-allow-interactive-services-in-windows-7
That said, if all of the stars are aligned and you accept the risk, of Allow Service to Interact with the Desktop, the answer is exactly as Sam indicated. In your unshown code, you need to set the Visible property to true.
As you go off and allow interactivity with the desktop and someone leaves some "testing" code in the package that gets deployed to production with MessageBox.Show("Click OK to continue"); be aware that if nobody notices this dialog box sitting there, you'll have a job waiting to complete for a very long time.
Regarding your first question, I understand that you want to debug your script task.
You can make Excel visible by adding the following line of code in your script task (assuming C# is the coding language):
// Create your Excel app
var excelApp = new Excel.Application();
// Make the Excel window visible to spot any issues
excelApp.Visible = true;
Don't forget to remove/comment that line after debugging.
Regarding your second question, I don't that this is a bad idea if you properly handle how Excel is opened and closed, in order to avoid memory issues.
I'd like to be able to schedule an Excel macro (VBA) to run in the middle of the night (after a file is ready) to create a customized workbook (multiple sheets, pivot tables, charts, filters, outlines, custom formatting, etc.). Currently, the macro is fired up manually the next day. Furthermore, it needs to run unattended on a server (laptop goes home at night!). Anybody successfully do something like this? Please, no Unix-side hacks (e.g., Perl modules) - need full access to VBA features, including database functions. Thanks!
Well you have some options.
First, for all Excel has to be installed on server.
Then you create a sheduled task to call a program.
In this case you can write e.g. a vbscript or .NET program to call the app, load the document and starts its content (your VBA). That should work at all.
Or you move the VBA code to a program and target Excel with your code, but prolly more work.
If you do this with .NET you have prolly best success. e.g. you can add an eventlog for successful run, etc.
If you can leave Excel running on the server all the time, you can use Application.OnTime to schedule the next runs of a particular macro (once it's run, reschedule another in the macro code). When I worked in banking we used this all the time to run night-time jobs.
If you cannot leave Excel running, I have to say you may be in a world of pain. It's possible to start Excel using an AT job (scheduled task) but you may have headaches getting it to run under the correct user privileges and if you use any addins you'll experience regular disasters where they failed to load and stopped Excel from starting up. At the end of the day, Excel isn't really meant to be run on servers (it's actually a violation of the terms of use) and starting/running/stopping it is not going to be a reliable system even if you do get it to work.
I have an RTD server that gets the values from a realtime source. The problem is that the users have pretty large excel sheets close to 20,000 RTD formulas. So when the user opens the sheet, all the RTD formulas get fired resulting in sending 20,000 queries to the server. This works for now, but the server can perform much better if i can group the queries and send it to the server.
My idea was to maintain a flag. When the calculation starts, the flag will be set to false, and when the calculation ends i can reset it. When an RTD formula is called, if the flag is unset, i will not send the query to RTD server, but pool it. When the flag is set to true i can combine the pooled queries and send it to the server
I am not sure how to get the notification as when excel starts and stops calculating. Please help. Also if you know any other approach for solving this problem, it would be great. I am using Excel 2007, C# 3.5
Please help. Thank you very much.
Rashmi
Thanks,
Since you're using RTD, I wonder if this could work:
You make calls to the back-end in timed batches. You start a timer in the first RTD call - a short time, maybe 500ms, then build up a batch of work from all the calls made to your RTD server until the timer expires, then send the batch to the back-end and await the response, while starting a new batch on the client. When the batch response comes, you notify Excel that the topics have been updated, and when Excel calls RefreshData you return the individual items out of your batch response. This way your batching uses the async-ness of RTD effectively, but you are not tied to Excel's recalculation events.
Hope this makes sense.
I have a number of users I support that are asking for things to happen automatically ( well more automagically but that's another point!).
One want events to happen every 120 secs ( see my other question ) and also another wants 1 thing to happen say at 5pm each business day. This has to be on the Excel sheet so therefore VBA as addins etc will be a no no, as it needs to be self contained.
I have a big dislike of using Application.OnTime I think its dangerous and unreliable, what does everyone else think?
EDIT:
Cross post is at VBA Macro On Timer style to run code every set number of seconds, i.e. 120 seconds
Application.OnTime is absolutely 100% reliable and is most definitely not dangerous. However, it is only exposed via VBA and you are regarding VBA as a "no no" for some reason here, so this option would appear to be unavailable to you.
I would generally not use OnTime for long-term scheduling, such as scheduling Excel to execute a command each day at 5pm. The problem is that if the user closes Excel, then the OnTime scheduling is lost. What you would need, in this case, is to use the Task Scheduler, or create your own application or windows service to open Excel and execute your commands.
For scheduling an event to occur every 120 seconds, however, using Application.OnTime would be perfect for this -- you would simply need to re-schedule OnTime to occur again in 120 seconds each time that OnTime calls back, because OnTime only fires once per scheduling, not on a repeat basis. I would absolutely use VBA for this task. If you don't want VBA commencing the action, that is fine: just have the VBA contained in a workbook which is then opened by your program or via the Task Scheduler. From that point onward, the VBA code can fire every 120 seconds.
Make sense?
You're right. an "infinite" interval of "onTime" calling itself, creates an infinite recursion.
It will cause a stack overflow after few thousand/million/billion function calls, and it will "leak" memory.