Dynamic Worksheet Title Reference Optimization - excel

Scenario: I have a workbook that needs to be able to dynamically reference different worksheets based on user input. I was able to achieve this by using several INDIRECT functions and some other basic functions such as INDEX.
My problem is that the workbook is taking several minutes to load on a fairly powerful computer with no other strenuous processes running.
Question: Is there a simple way to optimize workbooks? I tried looking online for some standard methods to run through to catch unnecessarily strenuous processes to no avail. I am willing to use VBA if it would help speed the sheet up.
Thanks

Don't use INDIRECT or any other volatile functions (OFFSET, RAND, NOW, TODAY) in your workbooks if you can help it, because they get recalculated automatically pretty much at the drop of a hat...even if you’re doing something to the workbook that has absolutely nothing to do with those formulas whatsoever, such as:
Entering new data in a completely unrelated cell
Editing any existing, completely unrelated cells
Deleting or inserting a row or column anywhere in the workbook
Performing certain AutoFilter actions
Double-clicking a row or column divider (in Automatic calculation
mode)
Adding, editing, or deleting a defined name
Renaming a worksheet
Changing the position of a worksheet in relation to other worksheets
Hiding or unhiding rows (but not columns)
This means that if you have lots and lots of volatile functions like OFFSET or INDIRECT in your formulas, those formulas are going to be recalculating pretty much constantly. And if those formulas do some pretty resource-intensive number-crunching, Excel will be re-crunching those numbers pretty much every time you touch the mouse or keyboard. Yikes!
But that’s not the half of it: After Excel recalculates those functions, it merrily starts recalculating all cells downstream of them—regardless of whether those downstream cells have volatile functions in them. And regardless of whether anything actually changed as a result. And I mean all cells...not just the cells that directly point to your volatile functions but all the cells that point to those cells, and then the next bunch of cells that point to that last bunch, and so on and so on, all the way through your workbook. If you’ve used volatile functions at key points within a spreadsheet, pretty much the entire spreadsheet could be recalculating constantly. Double yikes!
But even this is not the worst of it! The problem isn’t limited to the workbook you happen to be doing something with at the time. Rather, every volatile function in every open workbook gets recalculated when you make a change in any workbook anywhere—even if there aren’t any formula links between those workbooks. So if you’ve got a very large spreadsheet model with volatile functions open—and then you start typing, say, a shopping list in another, it could take minutes for you to add each item to that shopping list, as everything you type in the shopping list triggers a new avalanche of unnecessary and pointless recalculation in the large spreadsheet model. Triple yikes!
The fact that each and every cell downstream of any volatile functions get recalculated is an important point to get your head around. Many people think that just the volatile functions themselves get recalculated. They are mistaken. It’s the subsequent forced recalculation of all cells downstream of all volatile functions in all open workbooks that causes you grief. In fact, even just one volatile function in one workbook could cause you grief if you have a very long calculation chain hanging off it.
What to use instead?
Use INDEX, or even better, CHOOSE.
Note that while INDEX isn’t fully volatile, it ain’t perfect: It still gets recalculated whenever the workbook opens. This is known as is semi-volatility, and it means that you still might notice recalculation delays when initially opening a complicated model that uses INDEX. Which is one of the reasons I go for CHOOSE
=CHOOSE(index_num,value1,value2,...)
Here’s the translation from Microsoft-speak into Jeff-speak:
=CHOOSE(Which area do you want?, First area, Second area, ...)
You need to translate your dropdown choice into a number in order to drive the Which area argument. But that’s no problem; you just use exactly the same setup as you do for INDEX—a lookup table that converts the output of the dropdown into an index number that tells CHOOSE which range you want.
If your input areas are turned into Tables, then great...you can just use the Table names.

Related

Count total mentions of multiple values from closed workbook

I am using this formula to pull data from another workbook. This is repeated hundreds of times to count how many times a particular domain (there are hundreds of domains) has been used in the other sheet.
=SUMPRODUCT(COUNTIF(INDIRECT({"'E:\[OtherSheet.xlsx]Sheet1'!A1:ZZ500";"'E:\[OtherSheet.xlsx]Sheet2'!A1:ZZ500";"'E:\[OtherSheet.xlsx]Sheet3'!A1:ZZ500";"'E:\[OtherSheet.xlsx]Sheet4'!A1:ZZ500";"'E:\[OtherSheet.xlsx]Sheet5'!A1:ZZ500";"'E:\[OtherSheet.xlsx]Sheet6'!A1:ZZ500";"'E:\[OtherSheet.xlsx]Sheet7'!A1:ZZ500";"'E:\[OtherSheet.xlsx]Sheet8'!A1:ZZ500";"'E:\[OtherSheet.xlsx]Sheet9'!A1:ZZ500"}),D2))
This counts the number of times "D2" appears in the other workbook (across nine different sheets). This code is copied for "D3" "D4" and so on, hundreds of times.
There are a number of problems.
1) The other workbook has to be open, otherwise the data will not update (even with the exact path, INDIRECT will not allow updates).
2) If both sheets are open it updates every time you do anything, even copying and pasting cells, or inserting new cells. "Manual" formula updates are not able to be used as there are other dynamic variables besides this one (that don't slow down Excel) that need to be recalculated often.
3) From what I understand, INDIRECT and other related functions are single-threaded, which, even with a i7-8700k processor and 32gb RAM, with 64-bit Excel, makes the recalculation insanely slow.
4) I tried lowering the "scanned area" dramatically, but it doesn't speed it up. ZZ10000 vs ZZ500 doesn't make a difference, they're both equally slow.
A workaround is to keep one sheet open, update it, then open both when I need to have an 'overall' view. If one sheet is closed, Excel works fine as it is not having to constantly recalculate. Preferably I'd like to keep both sheets open without such a dramatic slow down.
From dozens of hours of research, I've sort of come to the conclusion that it is not possible to do this a fast way without using VBA. But I can't find any VBA function to replicate the code above.
I'm open to non-VBA based solutions as well.
You could use python's pandas module to solve this issue. It is much easier to work with using pandas's dataframe and you have no issue accessing a workbook even if it is closed. Just my two cents worth.

Providing a list of files in Excel of a directory is slowing down excel

I need to provide a current list of files in a directory in an Excel workbook and everything is working as required, just too slowly. I really only need the list to check it is current once upon opening the workbook. It takes around 11 seconds to do this which is acceptable but the problem is it keeps rechecking this every time I carry out even minor edits to the workbook (I guess due to the fact that it is brought in as an Excel table). I determined the lag in my workbook using the rangetimer() function that is provided and it is the only thing taking a long time to calculate. I should also state that the table containing the list of files is finally used in a cell on another worksheet to provide a data validation drop-down list but don't believe this is really the issue.
I did some Googling on reducing Excel calculation times and discovered that there are some Excel functions that are definitely culprits for increasing calculation times (described as volatile) and three of these (NOW,INDEX and ROW) are used in providing the functionality I would like in this part of the workbook.
I have tried two solutions so far:
1. Force Full Calculation set to True in VBA properties window
2. Switched calculations to manual. I set this back to automatic once I identified that this part of the workbook was the issue as I don't want manual calculation generally.
The formula I have in the 'refers to' box of the named range (TutorFileList) is:
'''=FILES("\O008DC01\Shared\Tutor Qualifications*")&T(NOW())'''
The formula I have in each cell of the excel table is:
'''=IFERROR(INDEX(TutorFileList,ROW()-1),"")'''
What I would like to have is the ~11secs of calculated time to find these files reduced down to just one check of the networked directory rather than it taking 11secs of automatic recalculation every time the workbook is modified.
If there is a more efficient way to achieve what I am doing I am prepared to redesign things but I do need the functionality of a drop-down list of files in the specific directory in a cell.
Many thanks for assistance from anyone on this.
I have resolved my issue by reducing the number of rows back to around 200 instead of 500 rows. This brings the calculation lag back to about a second which I can live with.

Excel VBA UDF Recalculation Indicator Parameter

I have a UDF function within a xlam workbook that connects to a database and executes a query, based on the parameters entered. This function is going to be used hundreds of times within different workbooks. Since it is going to be used in a series of different workbooks, that is the reason why I used a xam workbook.
The issue is that using the UDF function so many times, slows the Recalculation down to much since it is recalculating when it doesn't need to. I was thinking about adding another indicator parameter in the UDF that if set to zero, it will just return the value of the cell before the Recalculation started. Is there anyway to do this? I was thinking about adding a conditional statement that would use something like application.caller.value to get the value.

How to improve Excel performance in workbook with external links and conditional formatting

I have 5 workbooks with 540 column x 50 row blocks of data.
I also have a 'roll-up' workbook that lists all this data on a single worksheet with links. There are 6 conditional formatting rules.
The roll-up workbook takes ~30 seconds to update links on open and takes 1-2 seconds each time i modify data and move around the worksheet.
I feel like Excel should be able to work with this data in a performant manner.
Am I doing anything wrong? Should i have set things up differently?
I've tried using arrays of links and individualy linked cells and not noticed a difference.
-- EDIT --
When I remove the conditional formatting, the 30 second refresh takes 1 second. Additionally, some Data Validation dropdowns speed up from 6 seconds to immediate. I'll look at ways of removing the Conditional Formatting (like the VBA idea below).
I don't think the problem is in the conditional formatting but rather, it is in the fact that you need to open data from 5 different workbooks. Trying to consolidated those workbooks first would improve the speed. Refreshing data connections takes 30 sec.
Alternatively, instead of a linked cell, try to use designated data connections (as you would in Power Query, for example). This way a refresh wouldn't happen automatically but the responsiveness of the main spreadsheet would improve.
From experience, conditional formatting can really slow a spreadsheet down. I had one much smaller template that became unusable due a lot of conditional formatting, so I had to redesign it with VBA code instead. In that case this was workable as the cells didn't change much after the initial set-up, so the script only needs to be run once. It would not be so workable if the data is likely to change often.
You might consider putting the formatting in a VBA script, depending on how often the data (and hence the formating) is likely to change.

Point to a relative other Sheet in Excel

I'm trying to find a way to from a Cell get the data from a cell in the Sheet that lies to the Left (down in the tray) of the current Sheet.
I know how to call to other sheets via
=Sheet1!A1
But now I need something best explained with
=Sheet[-1]!A1
Any ideas?
Using the tab order as a fundamental part of your calculations is a complicated and risky approach to Excel calculations. Excel offers many alternatives which you'd be better off using:
A simplified version of belisarius's suggestion is: =INDIRECT(A1 & "!A2") where cell A1 has the name of your datasource sheet and A2 has the name of your target cell in your datasource sheet. If you know the name of your sheet of interest (or can look it up in some way), use this method.
If you need to do this often, you might want to export the data into an actual database (i.e. MS Access). Then you can make standard SQL queries and import the results into your Excel file.
If you absolutely want to go the VBA route, then you'd have to write some code that:
3a. Grabs all the names of the active workbook and stores them in an array.
3b. Identifies the index number of the currently active workbook in that array. Subtract 1 from that index to get the sheet to the left.
3c. Gets the cell value from that sheet.
You can also get freaky with Named Ranges. In Excel 2003, go to Insert->Name->Define, add a new Named Range and you can use that name in your calculations instead of referring to the cell by row and column.
Edit
The whole Idea with this one, is that
you have the Sheets arranged, and are
able to move them around, and that
shall change the calculations. – Gnutt
1 hour ago
Please, please, don't do that. For starters, this isn't a standard method of interaction with a spreadsheet. Your end-users will likely be confused and may not even ask for clarification.
You'll want to explore the idea of data validation:
Using Data->Validation, make a drop-down menu listing all the sheets in the workbook (if the names of all the sheets are static, you can just hardcode them, otherwise, you'll need some VBA to pull them).
Then the user just picks the sheet of their choice and indirect() will automatically update everything.
Alternatively, you can also check out Tools->Scenarios. I don't know anybody who uses this feature, but you might be a good candidate for it. Basically, it lets you see the results of calculations using different datasets (i.e. "scenarios") so the user can go back and forth between them.
Using either of the 2 methods above, there's a good chance you can avoid VBA entirely, thus saving users that annoying warning message when they open your file.
=INDIRECT("Sheet"&TEXT(VALUE(MID(CELL("filename",A8),FIND("]",CELL("filename",A8))+1,256))-1,"#")&"!A1")
Caveats:
Your workbook must be saved previously
A8 may be replaced by a reference to ANY non-error cell
I know it's not seen here as good practice, but I want to do something similar. And it does replicate database functionality to an extent but I don't have the time or support to build one from scratch when there's something already half in place.
The reason I want to be able to do this is to create a summary table that links to all the worksheets in the workbook, and automatically extends if you insert a new worksheet. This is to manage a large sales / reporting spreadsheet with lots of different business units that all have the same structure (ie use the same worksheet format to report the same outcomes for different people. There is a high turnover. I want to have several summary sheets reporting different aspects of the source sheets. This is very time consuming to manage if recreating all of the tables each time.
You should be able to use the row() as an index marker to define the information that you want using something like REPLACE, OFFSET or INDEX but you can't as they only refer to 2D arrays.
Whereas Excel treats 3-D references as arrays for statistical functions it does not seem to do the same for reference functions. You might have SUM(sheetX:sheetY!A1) and be able to add a sheet in between, there is not (eg) a INDEX(sheetX:sheetY!A1,n) function. I've tried experimenting using these 2D functions as part of array formulas, and defining the 3D reference as an array or a named range... well it was worth a go :).
So I believe it's a valid action. I also believe there must be a way to do it, but for now I'm falling back on a UDF that has the risk of errors caused by calculation issues, or manipulating a Workbook_SheetChange function or similar. Or creating a single master sheet to control all the others which is populated by using a subroutine based on an array of all workbooks.
these functions work well for me. They get the worksheet index (the parent of
the range you call them with), add or subtract from that index, and then create
a range from that (relative) sheet and the address passed in.
Function relativeSheet(r As Range, iRelative As Integer)
Application.Volatile
relativeSheet = Sheets(r.Cells(1, 1).Parent.Index + iRelative).Range(r.Address)
End Function
Function prevSheet(r As Range)
prevSheet = relativeSheet(r, -1)
End Function
Function nextSheet(r As Range)
nextSheet = relativeSheet(r, 1)
End Function

Resources