I am new with vba, and I want to fill a huge "microsoft treeview control, version 6.0" with 15000 different folders in excel on this format:
/folderOne
/folderOne/subfolderOne
/folderOne/subfolderTwo
/folderTwo/subfolderOne
I have used the Pradeep Kumar's solution, but it makes excel crash if I try to make it with too many rows (ok with 1000 rows, but not with 2000 rows) which is as follow:
Sub Button1_Click()
LoadTreeView1 TreeView1, 1, 1000
End Sub
Private Sub LoadTreeView1(TV As TreeView, min As Integer, max As Integer)
Dim i As Integer, RootNode As Node
TV.Nodes.Clear
Set RootNode = TV.Nodes.Add(, , "ROOT", "ROOT")
RootNode.Expanded = True
For i = min To max
AddNode TV, RootNode, Cells(i, 1)
Next
End Sub
Private Sub AddNode(TV As TreeView, RootNode As Node, Path As String)
Dim ParentNode As Node, NodeKey As String
Dim PathNodes() As String
On Error GoTo ErrH
PathNodes = Split(Path, "/")
NodeKey = RootNode.Key
For i = 1 To UBound(PathNodes)
Set ParentNode = TV.Nodes(NodeKey)
NodeKey = NodeKey & "/" & PathNodes(i)
TV.Nodes.Add ParentNode, tvwChild, NodeKey, PathNodes(i)
ParentNode.Expanded = True
Next
Exit Sub
ErrH:
If Err.Number = 35601 Then
Set ParentNode = RootNode
Resume
End If
Resume Next
End Sub
My treeview looks good on excel, similar to the one below but my issue is that i can't put enough data in it.
I have exported my list into access, but the process is different, and I am a bit lost because as I said, I am new with vba.
Thank you for your help
Main strategy in using treeview controls is to upload data of expanded rows on demand.
firstly, you're loading just high-level treeview elements (clienti list)
after user clicks the node, the method of filling corresponding node with child elements is called
Idea is that nobody would ever scroll through all list of elements in treeview.
These details could be difficult if you're new in VBA, so may be it is better to use linked lists. E.g.
list1: contains clienti list
list2: contains dates
after selecting element in list1, list2 recordsource is dynamically changed in order to provide list of only and only elements, that correspond to list1
This solution exceeds limitations of memory for treeview control, and suitable if you have definite depth of taxonomy of your objects.
Related
I am brand new to VBA and would need your help on a personal project.
I want to create a custom Gantt chart on Excel VBA.
Excel version is Microsoft® Excel® for Microsoft 365 MSO (Version 2208 Build 16.0.15601.20446) 64-bit
One aspect of the project I am struggling with is to manage "Parent-activities". Lets say that I have an activity identified as "Child", that can not start before other activities identified as "Parents" finish.
I would like to build a Public dictionary, each entry/key being a "Child", each child having a collection of "Parents".
My issue is that my code is very buggy, sometimes it works, sometimes not and I can not tell why...
Here is what I tried. I would really appreciate if you could flag any issue.
The Parent List is buggy (sometimes works, sometimes not).
The Parent Count function does not work, I guess I dont have the right syntax.
Public Parents_Dict As New Scripting.Dictionary 'I want this dictionary accessible from any Sub and I want to keep its content throughout the project
Public Child As Variant 'This will store the ID of the Child depending on the context
Public Parent As Variant 'This will store the ID of the Parent depending on context
'I have a button "Manage Parent" that triggers a user form and save the Child ID
Sub Open_Parents_Manager() 'We want to open the user form to manage parent activities
'We find the activity ID of the cell that was active when we opened the form
'We store this ID as the Variable "Child" that is recognize in this whole module
Child = CStr(ActiveSheet.Cells(ActiveCell.Row, 1).Value)
Manage_Parents.Show vbModeless 'VbModeless allow to interact with the speadsheet while the user form is still open
End Sub
'In this user form I have a Listbox that should list the Parents of the selected Child
Sub UserForm_Activate()
'We want the title of the box to mention Child ID to help the user confirm they are working on the right activity
Me.Caption = "Manage parent activities of " & Child 'Mention the ID of the Child as a title of the window
'We want to display the list of the current parents of this activity
Call Display_Parents_List
'We want to display the number of parents of this activity
Call Count_Parents
End Sub
Sub Display_Parents_List()
UserForm.Parents_ListBox.Clear
Dim Val As Variant
On Error Resume Next 'Override error if no dictionary exists
For Each Val In Parents_Dict(Child)
UserForm.Parents_ListBox.AddItem Val 'We add each item in the collection in our ListBox
Next Val
End Sub
'In this user form I also have a label that should return the number of parents for the selected child
Sub Count_Parents() 'We inform the user of how many parents this Child has
If Parents_Dict.Exists(Child) Then
UserForm.Label1.Caption = Parents_Dict(Child).Count 'This one does not work
Else
UserForm.Label1.Caption = "Currently, this activity has no parent"
End If
End Sub
'Finally I have button to add parents.
Sub Add_Parent()
Parent = "TP9" 'for testing I change the values manually here
If Parents_Dict.Exists(Child) Then
On Error Resume Next
Parents_Dict(Child).Add Item:=Parent, Key:=Parent 'Collection can have multiple times the same item but not under the same Key, Using Parent as Item & Key avoids duplicates
On Error GoTo 0
Else
Dim newList As Collection
Set newList = New Collection
newList.Add Item:=Parent, Key:=Parent
Parents_Dict.Add Key:=Child, Item:=newList
End If
End Sub
Let me explain.
I have a state similarity implementation in excel. One module, called a state, contains many public dictionaries at the top level.
I fill each dictionary with a huge number of object classes - mostly just data from sheets. Simple practice. The problem starts after the macro has worked normally, it leaves all these dictionaries in memory, in the task manager the Excel occupies from 2GB - this is also normal.
State Module - Standalone module
public Dict1 as Dictionary
public Dict2 as Dictionary
'Persists sheets data
public Dict3 as Dictionary
public Dict4 as Dictionary
'For persists renaming some objs
public Dict5 as Dictionary
public Dict6 as Dictionary
public Dict7 as Dictionary
Class Module - Data Interface Example - clsData
Public Name as string
Public Prop1 as string
Public Prop2 as Integer
Public Prop3 as Date
Public Prop4 as string
Public Value as double
The code below is just an example for stackoverflow. In my modules I am grabbing data from a sheet with the Range.CurrentRegion and iterate lbound to ubound.
Another data grabber
Function DataGrabberFromSheet(ByRef CurrentDict as Dict) as String
Dim i as long
Dim data as variant
Dim DataObj as clsData
set CurrentDict = New Dictionary <--- That's recreate dict obj and start clear old data for some how, but i do not need that anymore.
data = Sheet1.Range("A1:Q5000").Values
for i = 1 to 5000
set DataObj = new clsData
DataObj.Name = data(i, 1)
DataObj.Prop1 = data(i,2)
...
call CurrentDict.add(DataObj.Name, DataObj)
next
'For logging, it just an example:
If Success then
DataGrabberFromSheet = "Success"
else
DataGrabberFromSheet = "Bad"
endif
end Function
And so I decide in the debbuger to forcibly stop the program by pressing the stop button. At this point, Excel freezes for a long time, because I already have 50 or more of these dictionaries in the RAM, some of them have 200k elements each. Sometimes it takes about 300 seconds, and sometimes it instantly clears it somehow.
Old data erase by item, I think, but I do not need that data anymore. Can excel just skip that part of memory and just rewrite a new data.
How can I instantly redefine these dictionaries when the macro is rerun and not have to wait for that long cleanup? data is no longer needed at this point
Classic internet methods optimisations not solved that problem:
Application.Calculation = xlCalculationManual
Application.ScreenUpdating = False
...
I think this is a known issue with VBA - clearing a large number of objects takes a long time: regardless of whether they're stored in a dictionary, collection, or an array.
For example:
Dim arr() As clsData
Sub Tester()
Const NUM As Long = 120000
Dim i As Long, obj As clsData, t
t = Timer
Debug.Print "---------"
ReDim arr(1 To NUM)
Debug.Print "Reset", Timer - t
t = Timer
For i = 1 To NUM
Set obj = New clsData
obj.Prop1 = "Item" & i
obj.Prop2 = "Item" & i
obj.Prop3 = "Item" & i
obj.Prop4 = "Item" & i
Set arr(i) = obj
Next i
Debug.Print "Fill", Timer - t
End Sub
Where clsData is just:
Public Prop1
Public Prop2
Public Prop3
Public Prop4
Output from first run (after clicking "Stop" in VBE):
Reset 0
Fill 0.34375
Second run:
Reset 8.601563 <<<<<
Fill 0.3554688
I have the below code to populate a listbox, therefore I want to remove duplicates from my combobox. I Don't know how to do it:
Private Sub CommandButton1_Click()
Dim ws_suivi As Worksheet
Set ws_suivi = ActiveWorkbook.Worksheets("suivi")
Fin_Liste_suivi = ws_suivi.Range("A65530").End(xlUp).Row
For i = 2 To Fin_Liste_suivi
UserForm_SDE.ComboBox_Type_Rapp.AddItem ws_suivi.Range("AD" & i)
Next
UserForm_SDE.Show
End Sub
It is often worth searching to see if a Library for VBA exists that will save you reinventing the wheel.
It is a particular annoyance of VBA that whilst we have such useful structures as Collections and Scripting.Dictionaries there is no easy way to get information into such objects or to do much processing of the data once those objects are populated.
I had a project which had a lot of processing of arrays/scripting.dictionariews and to make my life a little easier I created a VBA library in C# called Kvp (for Key Value Pairs) which is a bit like a Scripting.Dictionary on steriods.
You can download the library, source code, documentation for the Kvp object from here
Once you have added a reference to the Kvp library you can declare a Kvp object in the standard way.
Dim myKvp as Kvp
Set myKvp=New Kvp
You can then add a 1D range from an excel spreadsheet in a single statement
myKvp.AddByIndexFromArray <excel range>.Value
which gives a Kvp of long integers vs cell values
The OP wishes a list of unique values. To do this with a Kvp we can use the Mirror method to create a Kvp of the unique values.
Dim myMirroredKvp as Kvp
set myMirroredKvp=myKvp.Mirror
The Mirror method returns a Two item Kvp where item 0 is a Kvp of unique items vs the first Key at which the item was found and item 1 is a Kvp of original Keys vs value where the values are a duplicate.
You can then get an array of the keys using the GetKeys method
Dim myUniqueValues as Variant
myUniqueValues = myMirroredKvp.GetItem(0).GetKeys
Or should you want the items sorted in reverse order
myUniqueValues - myMirroredKvp.GetItem(0).GetKeysDescending
The above can be shortened to
myUniqueValues = myKvp.Mirror.GetItem(0).GetKeysDescending
I've found the Kvp library quite useful. I hope you do to!!
While you could load the list to a Dictionary, you might find it simpler to try using WorksheetFunction.CountIf to check if the item is further up your list (and has, thus, already been included):
If (i=2) OR (WorksheetFunction.CountIf(ws_suivi.Range(ws_suivi.Cells(2,30),ws_suivi.Cells(i-1,30)), ws_suivi.cells(i,30).Value)<1) Then
UserForm_SDE.ComboBox_Type_Rapp.AddItem ws_suivi.Range("AD" & i)
End If
As a side-note: Since Excel 2007 increased the Row Limit from 65536 (216) to 1048576 (220), you may want to change Fin_Liste_suivi = ws_suivi.Range("A65530").End(xlUp).Row to Fin_Liste_suivi = ws_suivi.Cells(ws_suivi.Rows.Count, 1).End(xlUp).Row
I found :
Dim Valeur As String
Dim i As Integer
Dim j As Integer
'For each element in the list
For i = 0 To lst_ref.ListCount - 1
Valeur = Combobox.List(i)
For j = i + 1 To Combobox.ListCount - 1
'If the element exist, delete it
If Valeur = Combobox.List(j) Then
Call Combobox.RemoveItem(j)
End If
Next j
Next i
It take the beggining of the combobox and check if the value is red again in to the end of the combobox.
I am coding a Manager in Excel-VBA with several buttons.
One of them is to generate a tab using another Excel file (let me call it T) as input.
Some properties of T:
~90MB size
~350K lines
Contains sales data of the last 14 months (unordered).
Relevant columns:
year/month
total-money
seller-name
family-product
client-name
There is not id columns (like: cod-client, cod-vendor, etc.)
Main relation:
Sellers sells many Products to many Clients
I am generating a new Excel tab with data from T of the last year/month grouped by Seller.
Important notes:
T is the only available input/source.
If two or more Sellers sells the same Product to the same Client, the total-money should be counted to all of those Sellers.
This is enough, now you know what I have already coded.
My code works, but, it takes about 4 minutes of runtime.
I have already coded some other buttons using smaller sources (not greater than 2MB) which runs in 5 seconds.
Considering T size, 4 minutes runtime could be acceptable.
But I'm not proud of it, at least not yet.
My code is mainly based on Scripting.Dictionary to map data from T, and then I use for each key in obj ... next key to set the grouped data to the new created tab.
I'm not sure, but here are my thoughts:
If N is the total keys in a Scripting.Dictionary, and I need to check for obj.Exists(str) before aggregating total-money. It will run N string compares to return false.
Similarly it will run maximun N string compares when I do Set seller = obj(seller_name).
I want to be wrong with my thoughts. But if I'm not wrong, my next step (and last hope) to reduce the runtime of this function is to code my own class object with Tries.
I will only start coding tomorrow, what I want is just some confirmation if I am in the right way, or some advices if I am in the wrong way of doing it.
Do you have any suggestions? Thanks in advance.
Memory Limit Exceeded
In short:
The main problem was because I used a dynamic programming approach of storing information (preprocessing) to make the execution time faster.
My code now runs in ~ 13 seconds.
There are things we learn the hard way. But I'm glad I found the answer.
Using the Task Manager I was able to see my code reaching 100% memory usage.
The DP approach I mentioned above using Scripting.Dictionary reached 100% really faster.
The DP approach I mentioned above using my own cls_trie implementation also reached 100%, but later than the first.
This explains the ~4-5 min compared to ~2-3 min total runtime of above attempts.
In the Task Manager I could also see that the CPU usage never hited 2%.
Solution was simple, I had to balance CPU and Memory usages.
I changed some DP approaches to simple for-loops with if-conditions.
The CPU usage now hits ~15%.
The Memory usage now hits ~65%.
I know this is relative to the CPU and Memory capacity of each machine. But in the client machine it is also running in no more than 15 seconds now.
I created one GitHub repository with my cls_trie implementation and added one excel file with an example usage.
I'm new to the excel-vba world (4 months working with it right now). There might probably have some ways to improve my cls_trie implementation, I'm openned to suggestions:
Option Explicit
Public Keys As Collection
Public Children As Variant
Public IsLeaf As Boolean
Public tObject As Variant
Public tValue As Variant
Public Sub Init()
Set Keys = New Collection
ReDim Children(0 To 255) As cls_trie
IsLeaf = False
Set tObject = Nothing
tValue = 0
End Sub
Public Function GetNodeAt(index As Integer) As cls_trie
Set GetNodeAt = Children(index)
End Function
Public Sub CreateNodeAt(index As Integer)
Set Children(index) = New cls_trie
Children(index).Init
End Sub
'''
'Following function will retrieve node for a given key,
'creating a entire new branch if necessary
'''
Public Function GetNode(ByRef key As Variant) As cls_trie
Dim node As cls_trie
Dim b() As Byte
Dim i As Integer
Dim pos As Integer
b = CStr(key)
Set node = Me
For i = 0 To UBound(b) Step 2
pos = b(i) Mod 256
If (node.GetNodeAt(pos) Is Nothing) Then
node.CreateNodeAt pos
End If
Set node = node.GetNodeAt(pos)
Next
If (node.IsLeaf) Then
'already existed
Else
node.IsLeaf = True
Keys.Add key
End If
Set GetNode = node
End Function
'''
'Following function will get the value for a given key
'Creating the key if necessary
'''
Public Function GetValue(ByRef key As Variant) As Variant
Dim node As cls_trie
Set node = GetNode(key)
GetValue = node.tValue
End Function
'''
'Following sub will set a value to a given key
'Creating the key if necessary
'''
Public Sub SetValue(ByRef key As Variant, value As Variant)
Dim node As cls_trie
Set node = GetNode(key)
node.tValue = value
End Sub
'''
'Following sub will sum up a value for a given key
'Creating the key if necessary
'''
Public Sub SumValue(ByRef key As Variant, value As Variant)
Dim node As cls_trie
Set node = GetNode(key)
node.tValue = node.tValue + value
End Sub
'''
'Following function will validate if given key exists
'''
Public Function Exists(ByRef key As Variant) As Boolean
Dim node As cls_trie
Dim b() As Byte
Dim i As Integer
b = CStr(key)
Set node = Me
For i = 0 To UBound(b) Step 2
Set node = node.GetNodeAt(b(i) Mod 256)
If (node Is Nothing) Then
Exists = False
Exit Function
End If
Next
Exists = node.IsLeaf
End Function
'''
'Following function will get another Trie from given key
'Creating both key and trie if necessary
'''
Public Function GetTrie(ByRef key As Variant) As cls_trie
Dim node As cls_trie
Set node = GetNode(key)
If (node.tObject Is Nothing) Then
Set node.tObject = New cls_trie
node.tObject.Init
End If
Set GetTrie = node.tObject
End Function
You can see in the above code:
I hadn't implemented any delete method because I didn't need it till now. But it would be easy to implement.
I limited myself to 256 children because in this project the text I'm working on is basically lowercase and uppercase [a-z] letters and numbers, and the probability that two text get mapped to the same branch node tends zero.
as a great coder said, everyone likes his own code even if other's code is too beautiful to be disliked [1]
My conclusion
I will probably never more use Scripting.Dictionary, even if it is proven that somehow it could be better than my cls_trie implementation.
Thank you all for the help.
I'm convinced that you've already found the right solution because there wasn't any update for last two years.
Anyhow, I want to mention (maybe it will help someone else) that your bottleneck isn't the Dictionary or Binary Tree. Even with millions of rows the processing in memory is blazingly fast if you have sufficient amount of RAM.
The botlleneck is usually the reading of data from worksheet and writing it back to the worksheet. Here the arrays come very userfull.
Just read the data from worksheet into the Variant Array.
You don't have to work with that array right away. If it is more comfortable for you to work with dictionary, just transfer all the data from array into dictionary and work with it. Since this process is entirely made in memory, don't worry about the performance penalisation.
When you are finished with data processing in dictionary, put all data from dictionary back to the array and write that array into a new worksheet at one shot.
Worksheets("New Sheet").Range("A1").Value = MyArray
I'm pretty sure it will take only few seconds
I'm using LibreOffice Version: 4.4.3.2 Build ID: 40m0(Build:2) Locale: en_AU
I have a Basic Module
At the top of this module before any sub or functions I have
Type InitHeadings
MySort_By As Integer
MyCharacter As Integer
MyInitiative As Integer
MyRolled As Integer
MyTotal As Integer
End Type
...
Global InitiativeColumn As New InitHeadings
But when I run a sub, set a breakpoint and 'watch' the InitiativeColumn Object only the first two fields are shown.
The rest of my code relevant to this struct as the documentation calls them is below. I don't reference it anywhere else. Can anyone tell me why the first two would work but not the rest? I have two other structs in this code and both also ignore the last three fields. Is this a Bug?
Sub Main
'Initialise Doc and Sheet Objects
Dim Doc As Object
Doc = ThisComponent
StatsSheet = Doc.Sheets.getByName("Stats")
InitiativeSheet = Doc.Sheets.getByName("Initiative")
CombatSheet = Doc.Sheets.getByName("Combat")
'LOAD HEADING NAMES
'Initiative Sheet
For Column = 0 to 25 'Columns A to Z
MyHeadingName = InitiativeSheet.getCellByPosition(Column,0).String
Select Case MyHeadingName
Case "Sort By"
InitiativeColumn.MySort_By = Column
Case "Character"
InitiativeColumn.MyCharacter = Column
Case "Initiative"
InitiativeColumn.MyInitiative = Column
Case "Rolled"
InitiativeColumn.MyRolled = Column
Case "Total"
InitiativeColumn.MyTotal = Column
End Select
Next Column
End Sub
Sub MyInitiativeButton
'Iterate over a range of cells:
For Row = 1 To 25 'Rows 2 to 26
'Column 3 is column D the "Rolled" column
InitiativeSheet.getCellByPosition(InitiativeColumn.MyRolled,Row).VALUE = Roledice(1,20,0)
Next Row
End Sub
It looks like a bug, and seems to have been reported here. The problem did not occur when I tested it in a newer version (LO 5.1.0.3).
This is only an issue for the debugger window. The values are still there:
Sub TestStructs
InitiativeColumn.MySort_By = 5
InitiativeColumn.MyCharacter = 5
InitiativeColumn.MyTotal = 5
InitiativeColumn.DoesntExist = 5
End Sub
This code works fine until the line InitiativeColumn.DoesntExist = 5, whereupon it crashes.
Now the Global problem that you mentioned in the comments is really a problem. Considering the standard programming advice that global variables are bad, I think it's wise to consider alternatives.
Instead of a subroutine, could you perhaps use a Function that returns InitiativeColumn? If not, then assigning the variable as you suggested seems a viable workaround. Personally for LO macros I prefer Python or Java since they have classes.