Lua weak tables memory leak - memory-leaks

I don't use often weak tables. However now I need to manage certain attributes for my objects which should be stored somewhere else. Thats when weak tables come in handy. My issue is, that they don't work es expected. I need weak keys, so that the entire key/value pair is removed, when the key is no longer referenced and I need strong values, since what is stored are tables with meta information which is only used inside that table, which also have a reference to the key, but somehow those pairs are never collected.
Code example:
local key = { }
local value = {
ref = key,
somevalue = "Still exists"
}
local tab = setmetatable({}, { __mode = "k" })
tab[key] = value
function printtab()
for k, v in pairs(tab) do
print(v.somevalue)
end
end
printtab()
key = nil
value = nil
print("Delete values")
collectgarbage()
printtab()
Expected output:
Still exists
Delete values
Got:
Still exists
Delete values
Still exists
Why is the key/value pair not deleted? The only reference to value is effectivly a weak reference inside tab, and the reference inside value is not relevant, since the value itself is not used anywhere.

Ephemeron tables are supported since Lua 5.2.
The Lua 5.2 manual says:
A table with weak keys and strong values is also called an ephemeron table. In an ephemeron table, a value is considered reachable only if its key is reachable. In particular, if the only reference to a key comes through its value, the pair is removed.
Lua 5.1 does not support ephemeron tables correctly.

You are making too many assumptions about the garbage collector. Your data will be collected eventually. In this particular example it should work if you call collectgarbage() twice, but if you have some loops in your weak table it might take even longer.
EDIT: this actually only matters when you're waiting for the __cg event
I went over your code in more detail and noticed you have another problem.
Your value is referencing the key as well, creating a loop that is probably just too much for the GC of your Lua version to handle. In PUC Lua 5.3 this works as expected, but in LuaJIT the loop seems to keep the value from being collected.
This actually makes a lot of sense if you think about it; from what I can tell, the whole thing works by first removing weak elements from a table when they're not referenced anywhere else and thus leave them to be collected normally the next time the GC runs.
However, when this step runs, the key is still in the table, so the (not weak) value is a valid reference in the GCs eyes, as it is accessible from the code. So the GC kind of deadlocks itself into not being able to remove the key-value pair.
Possible solutions would be:
Don't save a reference to the key in the value
Make the value a weak table as well so it doesn't count as a reference either
Upgrade to another Lua version
Wrap the reference in a weak-valued single-element array

you can change the code like this. Then you will get the expected output. tips: do not reference key variable when you want it to be week.
local key = { }
local value = {
-- ref = key,
somevalue = "Still exists"
}
local tab = setmetatable({}, { __mode = "k" })
tab[key] = value
function printtab()
for k, v in pairs(tab) do
print(v.somevalue)
end
end
printtab()
key = nil
value = nil
print("Delete values")
collectgarbage()
printtab()

Related

Delete element in the middle of mapping

I need to delete an item in the middle of the Everscale solidity mapping containing struct:
struct Example {
string data;
uint64 validFrom;
uint64 valiUntil;
}
mapping(uint64 => Example) example;
example[1668161798] = Example("Start", 1668161798, 1668162798);
...
example[1668163798] = Example("Middle", 1668163798, 1668164798); // <-- Need to delete this one
...
example[1668165798] = Example("End", 1668165798, 1668166798);
Question 1
What is the best way to do this in terms of:
Gas consumption?
Storage?
Is it using the delete instruction work from the Ethereum example, or is it better to rebuild and reassign the mapping?
delete example[1668163798];
Question 2
What happens to the data contained in the mapping's item after using delete? Is there any garbage collector that wipes them out to minimize the storage?
What will happen if I reassign new data on the same index after deletion?
delete example[1668163798];
is the right way to do it. "delete" assigns the default value of the type for the variable it is applied to. For the mapping key, it removes the pair from the dictionary, thus freeing the storage space.
assigning a new value to the previously deleted key is no different from adding any other (key, value) pair to the dictionary; it works just fine.

Building a std::map and issue with using std::emplace

Code:
std::map<CString, S_DISCUSSION_HIST_ITEM> mapHistory;
// History list is in ascending date order
for (auto& sHistItem : m_listDiscussionItemHist)
{
if (m_bFullHistoryMode)
mapHistory.emplace(sHistItem.strName, sHistItem);
else if (sHistItem.eSchool == m_eActiveSchool)
mapHistory.emplace(sHistItem.strName, sHistItem);
}
// The map is sorted by Name (so reset by date later)
// The map has the latest assignment info for each Name now
Observation:
I now understand that std::emplace behaves like this:
The insertion only takes place if no other element in the container has a key equivalent to the one being emplaced (keys in a map container are unique).
Therefore my code is flawed. What I was hoping to acheive (in pseudo code) is:
For Each History Item
Is the name in the map?
No, so add to map with sHitItem
Yes, so replace the sHistItem with this one
End Loop
By the end of this loop iteration I want to have the most recent sHitItem, for each person. But as it is, it is only adding an entry into the map if the name does not exist.
What is the simplest way to get around this?
Use insert_or_assign method if the item is assignable. It will be assigned if it already exists. Or use [] operator followed by assignment, it will default-construct item if it does not exist.
For non-assignable types I'm afraid there's no convenient way.

Accessing a VBA dictionary entry with a non-existent key [duplicate]

I am using a dictionary object from the MS Scripting Runtime library to store a series of arrays and perform operations on the array cells as necessary. There is a for loop to go through the process of creating all of these entries. My issue is that when using the .exists property, it is returning True even before the item has been added.
Closer debugging indicates that the key is being added to the dictionary at the beginning of the for loop, even though no .add command is used and will not be used until the end of the loop.
I have tried a few different configurations, but here is a simple example that fails:
Dim dTotals As Dictionary
Set dTotals = New Dictionary
dTotals.CompareMode = BinaryCompare
For Each cell In rAppID
If Not dTotals.Exists(cell) Then
Set rAppIDCells = Find_Range(cell, rAppID)
Set rAppIDValues = rAppIDCells.Offset(0, 6)
dAppIDTotal = WorksheetFunction.Sum(rAppIDValues)
dTotals.Add Key:=cell.Value, Item:=dAppIDTotal
End If
Next cell
Where each cell contains a string / unique id. At the If statement, the code is returning false, even on the first iteration.
In the official documentation‌​ for the scripting runtime it says "If key is not found when attempting to return an existing item, a new key is created and its corresponding item is left empty."
...and yea, when you're debugging in a loop, it appears to pop right out of the sky before the '.exists' function is even called. All is well...
Instead of attempting to add the item that just got added, as in:
dTotals.Add Key:=cell.Value, Item:=dAppIDTotal
...just set the empty object currently at your key to your new one:
dTotals(cell.Value) = dAppIDTotal
So your code block becomes:
If Not dTotals.Exists(cell) Then
Set rAppIDCells = Find_Range(cell, rAppID)
Set rAppIDValues = rAppIDCells.Offset(0, 6)
dAppIDTotal = WorksheetFunction.Sum(rAppIDValues)
dTotals(cell.Value) = dAppIDTotal
End If
Voila. I tend to rediscover this "feature" on every revisit to VBA. You may also notice the effects of it if you are having a memory leak caused by adding new keys that you do not intend to store.
I had this problem manifest itself while debugging when I had a watch that attempted to return the "missing" key's item. Actually, further frustrated debugging had the same problem when I literally had a watch for the [scriptingdictonaryObject].exists() condtional); I suggest that the "missing" key is added because of the watch. When I removed the watch and instead created a temporary worksheet to copy the array to while running, the unwanted keys were no longer added.

How can I use a string in a table?

I need to use a string value as a table, in order to restore points to a player when they reconnect to a game server.
This string value is their profile ID, which never changes and I need to put data inside the string value (Kills, deaths, head shots) in order to effectively restore these points. I have had a quick look on the internet but I have not found much because I don't know what this specific thing is actually called.
To make it easier, here's what I have so far:
if (not Omega.Playertable) then
Omega.Playertable = {};
System.LogAlways("Set static record table on first connect");
end
local ID = g_gameRules.game:GetProfileId(player.id);
if (not Omega.Playertable.ID) then
table.insert(Omega.Playertable, ID);
Omega.Playertable.g_gameRules.game:GetProfileId(player.id).Kills=0;
Omega.Playertable.g_gameRules.game:GetProfileId(player.id).Deaths=0;
Omega.Playertable.g_gameRules.game:GetProfileId(player.id).Headshots=0;
else
local Kills=Omega.Playertable.g_gameRules.game:GetProfileId(player.id).Kills;
local Deaths=Omega.Playertable.g_gameRules.game:GetProfileId(player.id).Deaths;
local Headshots=Omega.Playertable.g_gameRules.game:GetProfileId(player.id).Headshots;
g_gameRules.game:SetSynchedEntityValue(playerId, 101, Kills);
g_gameRules.game:SetSynchedEntityValue(playerId, 100, Deaths);
g_gameRules.game:SetSynchedEntityValue(playerId, 102, Headshots);
end
As you can see, I've tried adding their ID into the table and adding info based on this. I cannot get the system to read the 'ID' value that I set before, so I tried adding the code that gets the ID instead, and it doesn't work. The ID is unique to each player so I cannot use a simple number system for this.
Could someone point out to me what I have done wrong here? If I manage to fix the problem, I will answer my own question on here so that it can be helpful to other users.
It seems to me that you are using the wrong table indexing syntax.
Indexing a table by a variables value in Lua is done with the [] syntax.
Furthermore, in Lua Foo.bar is syntactic sugar for Foo["bar"] both formats are interchangeable, but the . variant has limitations on which characters you can use with it. For example Foo["\n.*#%!"] is a valid table index, but you certainly can't write this: Foo.\n.*#%!
Also table.insert(t, v) inserts v at the end of the array part of the table. That means if you do this
foo = {};
foo.X = "Some value";
table.insert(foo, "X");
This is what you get
{
X = "Some value"
[1] = "X"
}
That means, if I apply this to the code you gave us, this is what you probably had in mind:
if (not Omega.Playertable) then
Omega.Playertable = {};
System.LogAlways("Set static record table on first connect");
end
local ID = g_gameRules.game:GetProfileId(player.id);
if (not Omega.Playertable[ID]) then
Omega.Playertable[ID] = {};
Omega.Playertable[ID].Kills=0;
Omega.Playertable[ID].Deaths=0;
Omega.Playertable[ID].Headshots=0;
else
local Kills = Omega.Playertable[ID].Kills;
local Deaths = Omega.Playertable[ID].Deaths;
local Headshots = Omega.Playertable[ID].Headshots;
g_gameRules.game:SetSynchedEntityValue(playerId, 101, Kills);
g_gameRules.game:SetSynchedEntityValue(playerId, 100, Deaths);
g_gameRules.game:SetSynchedEntityValue(playerId, 102, Headshots);
end
Try this:
s="35638846.12.34.45"
id,kills,deaths,headshots=s:match("(.-)%.(.-)%.(.-)%.(.-)$")
print(id,kills,deaths,headshots)
But note that these values are strings. If you're using them as numbers, use tonumber to convert them.

Lua weak references

I'm working on a project in Lua where I will be creating tables and storing them in a master table, to be erased at a later time. I will pass around references to these tables to other sibling tables.
master = {}
table.insert(master, {name = 'hello'})
table.insert(master, {name = 'world', pre = master[1]})
The problem that occurs is that when I wish to erase the reference from the master table, the reference still remains in master[2] here. Obviously my first solution was to make the tables have weak values. (through .__mode on a metatable, not shown here)
This worked, and would work, so long as I would never store a singly-referenced table within these tables.
table.insert(master, {name = 'goodbye', pre = master[2], some_table = {123}})
The third element, some_table would eventually be collected, because the tables have weak values, and this table (some_table) is not referenced anywhere else. This is undesired behavior. My latest solution involves creating "weak reference objects" to the tables within the master table. A naive implementation follows:
function WeakRef(t)
r = {__mode = 'v', __index = t, __newindex = t}
setmetatable(r, r)
return r
end
These weak reference objects act similarly to boost::weak_ptrs and accomplish my goal, but I am uncertain if they are the best solution to my problem.
Is there a better way; a more elegant solution?
Is my design, which requires this master table, perhaps flawed?
Given that:
You want master to be the "one place" where you define whether an object exists or not
Your objects can have links between them
Then probably the simplest architecture is reserving one of the members of each object as a "middle man" in charge of managing the references to others. Here're the steps:
Make master a regular table (not weak)
On each physical object, create a weak table called links (or whatever name suits your logic better)
Make all links tables weak. Use them to store references to other objects.
And this is a possible implementation. I've tried it in Lua 5.1:
local function newWeakTable()
return setmetatable({}, {__mode = "v"})
end
local master = {}
-- create two physical objects
local obj1 = { name = "obj1", links = newWeakTable() }
local obj2 = { name = "obj2", links = newWeakTable() }
-- link them
obj2.links.pre = obj1
-- insert them into master
table.insert(master, obj1)
table.insert(master, obj2)
-- master has 2 objects, and they are linked
assert(#master == 2)
assert(obj2.links.pre == obj1)
-- remove obj1 from master, and remove the variable reference
table.remove(master, 1)
obj1 = nil
-- run gc manually
collectgarbage("collect")
-- master has only 1 object now, and the link has dissapeared
assert(#master == 1)
assert(obj2.links.pre == nil)
print("Everything went as expected")

Resources