I'm autofilling some columns in Excel (one at a time). These column use a UDF I wrote in ExcelDna. Using taskmanager, I notice that only half of the cores are being used. Excel settings is set to "use all processors on this computer." So I can't figure out why only half the cores are in use. Thoughts?
This page contains some insightful explainations about multi-processing in Excel 2007+
http://msdn.microsoft.com/en-us/library/office/aa730921(v=office.12).aspx#office2007excelperf_ExcelPerformanceImprovements
*Ref: "Multithreaded Calculations"
"Some Excel features do not use multithreaded calculation, for example:
Data table calculation (but structured references to tables do use MTC).
User-defined functions (but XLL functions can be multithread-enabled).
XLM functions.
INDIRECT, CELL functions that use either the format2 or address options.
GETPIVOTDATA and other functions referring to PivotTables or cubes.
Range.Calculate and Range.CalculateRowMajorOrder.
Cells in circular reference loops. "
Since it doesn't specifically describe "autofill" - I'll take a crack at it from a concurrency perspective. Please forgive if some of this is speculation..
Certain functions require an "in-order" task that cannot easily be split across processors, we might suspect that fill is one of them. (It requires sequential operation in some of it's modes.. example: stepping 1,1.2,1.4,etc). In this example, processor 2 can't just start from the middle of the page without performing a new/custom independant calculation. Special functions would have to be designed. Perhaps they decided not to code for these kinds of scenerios. With formulas I can't see this even being possible, because you could be creating a formula tree.
Other operations are indepentant based on the formula trees (see link). This strongly implies Excel won't multi-process a complext formula tree either .. so if you have numerous formulas connected, they (that tree) will be only be processsed, in order, by one processor.
Granted, there's all kinds of complex work-arounds for these situations.. (perhaps similiar to compilers, SQL servers, etc) but the documentation above is highly suggestive that Microsoft designed it to streamline independant tasks only.
Related
I am currently trying to optimize a VBA code with extreme execution time (56 hours and more on high end PC). The main problem is nested loops (up to 8) due to complicated iterative calculations and referencing ranges in formulas.
I have two questions:
1)
What loops are most preferred in VBA (based on their hardware strain for lack of better word)
2)
Is there a way to reference variable range of cells in a formula? For example in formula:
=AGGREGATE(15,6,SQRT((R2C10:R500000C10-RC[10])^2+(R2C11:R500000C11-RC[11])^2+(R2C12:R500000C12-RC[12])^2),1)
I am referencing relative cell: "R500000C10". There are two main issues: For one, the half milion cells might not capture all data - though in most cases there is less active cells than that, in same cases that might be an issue. Second, it takes forever to scan through so many cells (same reference is used in multiple of VLOOKUPs).
The first issue could be solved by referencing the whole column, but that makes the second problem worst.
I would like to reference it in a similar way that xlDown works:
Range().End(xlDown).Select
The problem there is I would need to reference it within the Active.Cell.FormulaR1C1. Is there a way to do that?
Thanks!
Well you wrote that you have a code with 8 loops and try to optimize it maybe you can edit and put part of that code, because your description is very general, however, with information that you put i speculate that about your point
1)it depend what do you want to do? because each "loop" have their own use for specifyc scenaries
2)there are tricks to do that but all depend what are you trying to do and why you want to use .formulaR1C1
again im speculating but you can try tu separate your loops in indenpendent function, and maybe you are putting a formula inside cells, and question is why? you can create functoins and Sub in your VBA code and do your "extrem execution" without use xtra "resours"
What I would like to do:
I would like to use operator overloading in Excel to run custom functions on my custom data types. For example, when evaluating a formula, I want Excel to run my function instead of the '+' operator when the calculation involves one of my custom data types.
Why I want to do it:
In analytical chemistry, every number has an uncertainty attached to it and is written:
13.56 (±0.02) mm
I would like to create a custom data type that keeps the magnitude and the uncertainty of the number together in the same cell.
Additionally, I want to implement operator overloading, so when I write
=A1+A2
and either A1 or A2 contains an uncertainty-type number, my custom function runs instead of the default
'+' operator to calculate the uncertainty.
This would make my spreadsheets much cleaner, as currently, I have to write such a statement as
=ADD_UNC(A1, A2)
Which is fine for very simple equations, but becomes a pain when the operation I am trying to form is even slightly non-trivial.
=MULT_UNC(A3, ADD_UNC(MULT_UNC(A5, A1, A2), A3)
vs.
=A3*((A1*A2)+A3)
Why I assume this is possible:
I know in real, full-blown programming languages such as C#, operator overloading is very common and very easy to perform.
Thank you for your help,
Michael
Not possible in VBA. VBA was intended to provide scripts which help with automation. You see, we call them macro. VBA is not built on top of modular classes or objects. Your VBE writes direct P-code the moment you type/ hit enter in the editor. VBA is awesome and packs alot of features but expecting these kind of facilities in VBA is a bit of stretch. No possibility to have this feature even in future. and Just a suggestion, never worry too much about code-cosmetics, they are useless overhead.
Here is a hack using the Worksheet_Change event. You can essentially place in a cell that contains a "+" character anything you want, thereby effectively disengaging the "+" signs normal function.
Private Sub Worksheet_Change(ByVal Target As Range)
If UBound(Split(CStr(Target), "+")) > 0 Then
Target = "Overloaded"
Else:
Target = "Not overloaded"
End If
End Sub
Consider below set up:
Whenever you enter value under Time, it will be mapped on the right side under Process Time using the correct line legend depending on what column you enter your input.
I know my question would be a bit off topic or too broad, but I was hoping that someone may share their idea on how to accomplish the same other than VBA. I am currently doing this using Change_Event but this may be hard to maintain as steps may vary as well as the time intervals.
I also come-up with another solution using Conditional Formatting (same as the available template that Excel 2013 offers - Project Planner) but my customer wish to (as much as possible) preserve the legends.
So before I go on optimizing the current routine I've written, I want to consult on the communities opinion if I am on the right track on using brute force VBA (automating inserting and deleting lines) or if there is a simpler way. I just felt that maybe I am overdoing things (especially on the freeform line)
I had already a solution for this without VBA. But of course the legend symbols must be character glyphs. So they are not as flexible as you need. But maybe you get stimuli how programming this in VBA from this.
There is only one Formula in F4 copied cross down.
=IF(AND(SUM($B$3:$E3)<F$2,SUM($B$4:$E4)>=F$2),REPT(CHOOSE(MATCH(0,$B4:$E4,-1),"—","-","~","═"),4),"")
Row 3 must be empty. All times must be multiples of 10. And there is only one kind of process possible in each step. The cells F4:O10 must be formatted with a proportional font.
I would like to create a report that look like this picture below.
My data has around 500,000 cells (it will continue to grow larger)
Right now, I'm using countifs function from excel but it takes a very long time to calculate. (cannot turnoff automatic calculate)
The main value is collected as date and the range of date is about 3 years, so I have to put a lot of formula to cover all range of value.
result
The picture below is the datasource the top one cannot be changed. , while the bottom is the one I created by myself (can change). I use weeknum to change date to week number.
data
Are there any better formula or any ways to make this file faster? Every kinds of suggestions are welcome!
I was thinking about using Pivot Table, but I don't know how to make pivot table from this kind of datasource.
PS. VBA is the last option.
You can download example file here: https://www.mediafire.com/?t21s8ngn9mlme2d
I will post this answer with the disclaimer that it is entirely dependent on the size of the data set. That turning on and off the auto calculate is the best way, but your question doesn't let me do that, so keep reading.
Your question made me curious, so I gave it a try and timed it. I essentially set up two columns of over 100,000 rand numbers choosing from 1-1000 and then tried to do a countif on the two columns if they were equal. I made a macro that I can run that turns off the autocalculate, inserts the start time, calculates, and then inserts the finish time. I highlighted in yellow the time difference.
First I tried your way, two criteria, countifs:
Then I tried to combine (concatenate) the two columns to see if I could make it easier by only having one countif criteria and data set. It doesn't. see result below:
Finally, realizing what was going on. I decided to make the criteria only match the FIRST value in the number to look for. I was essentially reducing the number of characters to check per cell. This had a positive result. See below:
Therefore my suggestion is to limit the length of the words you are comparing in anyway possible. You are mostly looking at dates, so you might have to get creative, but this seems to be the best way possible without going to manual calculation.
I have worked with Excel sheets of a similar size. Especially if you are using the data on a regular basis, I would heartily recommend switching to a proper database SQL based, Access, or whatever fits your purpose. I does wonders for the speed and also you won't run into the size limits of Excel. :-)
You can import the data you have now fairly easy.
I am happy as a clam with my postgresql db.
Is there anyway to rewrite this formula to speed up Excel processing?
My spreadsheet has become terribly slow!
=SUMPRODUCT((Sheet1!J:J=Sheet2!A2)*(Sheet1!G:G="Windows XP")*(Sheet1!B:B="Desktop")*(Sheet1!M:M<>"Refresh >=Q2 2014")*(Sheet1!M:M<>"Release 2013")*(Sheet1!M:M<>"Release 2014")*(Sheet1!M:M<>"N/A NVM")*(Sheet1!M:M="No")*(Sheet1!M:M="N/A"))
As written your formula will always return zero because the last two conditions are mutually exclusive - did you mean those last two to be <> rather than = (or did you refer to the wrong columns)?
In any case I can see from the use of whole columns that you must be using Excel 2007 or later (your current formula would give an error otherwise) in which case COUNTIFS will be much faster, i.e. assuming the last two conditions should be adjusted as I suggested try this version:
=COUNTIFS(Sheet1!J:J,Sheet2!A2,Sheet1!G:G,"Windows XP",Sheet1!B:B,"Desktop",Sheet1!M:M,"<>Refresh >=Q2 2014",Sheet1!M:M,"<>Release 2013",Sheet1!M:M,"<>Release 2014",Sheet1!M:M,"<>N/A NVM",Sheet1!M:M,"<>No",Sheet1!M:M,"<>N/A")
If you do need to use SUMPRODUCT then restrict the ranges rather than using whole columns
I don't think that there is really a chance to speed up Excel Formula. But you could save your File in binary code (.xlsb). Losing some compatibility but improving performance.
You also could stop auto (re-)calculations of ther Formula, then you have to manualy refresh. This will let you edit the file much smoother.