Memory leak where CPython extension returns a 'PyList_New' instance to Python, which is never deallocated - memory-leaks

Question

I've been trying to debug a memory leak for a few days and I'm running out of ideas.
High-level: I've written a CPython extension that allows querying against binary data files, and it returns the results as a Python list of objects. Usage is similar to this psuedocode:
for config in configurations:
s = Strategy(config)
for date in alldates:
data = extension.getData(date)
# do analysis on 'data', capture/save statistics
I've used tracemalloc, memory_profiler, objgraph, sys.getrefcount, and gc.get_referrers to try to find the root cause, and these tools all point to this extension as source of an exorbitant amount of memory (many gigs). For context, a single record in the binary file is 64 bytes, there are typically 390 records per day, so each date iteration is working with ~24K bytes. Now, there are many iterations happening (synchronously), but in each iteration the data is used as a local variable, so I expected each subsequent assignment to deallocate the previous object. The results from memory_profile suggest otherwise...
Line # Mem usage Increment Occurences Line Contents
============================================================
86 33.7 MiB 33.7 MiB 1 #profile
87 def evaluate(self, date: int, filterConfidence: bool, limitToMaxPositions: bool, verbose: bool) -> None:
92 112.7 MiB 0.0 MiB 101 for symbol in self.symbols:
93 111.7 MiB 0.0 MiB 100 fromdate: int = TradingDays.getAdjacentDay(date, -(self.config.analysisPeriod - 1))
94 111.7 MiB 0.0 MiB 100 throughdate: int = date
95
96 111.7 MiB 0.0 MiB 100 maxtime: int = self.config.maxTimeToGain
97 111.7 MiB 0.0 MiB 100 target: float = self.config.profitTarget
98 111.7 MiB 0.0 MiB 100 islong: bool = self.config.isLongStrategy
99
100 111.7 MiB 0.8 MiB 100 avgtime: Optional[int] = FileStore.getAverageTime(symbol, maxtime, target, islong, fromdate, throughdate, verbose)
101 111.7 MiB 0.0 MiB 100 if avgtime is None:
102 110.7 MiB 0.0 MiB 11 continue
103
104 112.7 MiB 78.3 MiB 89 weightedModel: WeightedModel = self.testAverageTimes(symbol, avgtime, fromdate, throughdate)
105 112.7 MiB 0.0 MiB 89 if weightedModel is not None:
106 112.7 MiB 0.0 MiB 88 self.watchlist.append(weightedModel)
107 112.7 MiB 0.0 MiB 88 self.averageTimes[symbol] = avgtime
108
109 112.7 MiB 0.0 MiB 1 if verbose:
110 print('\nFull Evaluation Results')
111 print(self.getWatchlistTableString())
112
113 112.7 MiB 0.0 MiB 1 self.watchlist.sort(key=WeightedModel.sortKey, reverse=True)
114
115 112.7 MiB 0.0 MiB 1 if filterConfidence:
116 112.7 MiB 0.0 MiB 91 self.watchlist = [ m for m in self.watchlist if m.getConfidence() >= self.config.winRate ]
117
118 112.7 MiB 0.0 MiB 1 if limitToMaxPositions:
119 self.watchlist = self.watchlist[:self.config.maxPositions]
120
121 112.7 MiB 0.0 MiB 1 return
This is from the first iteration of the evaluate function (there are 30 iterations total). Line 104 is where it seems to be accumulating memory. What's strange is that the weightedModel contains only basic stats about the data queried, and that data is stored in a loop-local variable. I can't figure out why the memory used is not cleaned up after each inner iteration.
I've tried to del the objects in question after an iteration completes, but it has no effect. The refcount does seem high for the containing objects, and gc.get_referrers shows an object as referring to itself (?).
I'm happy to provide additional information/code, but I've tried so many things at this point a braindump would be a complete mess :) I'm hoping someone with more experience might be able to help me focus my thought process.
Cheers!

Answer 1

Found it! The leak was one layer deeper, where the extension function builds an instance of a Python object.
This was the leaky version:
PyObject* obj = PyObject_CallObject(PRICEBAR_CLASS_DEF, args);
PyObject_SetAttrString(obj, "id", PyLong_FromLong(bar->id));
# a bunch of other attrs...
return obj;
This is the fixed version:
PyObject* obj = PyObject_CallObject(PRICEBAR_CLASS_DEF, args);
PyObject* id = PyLong_FromLong(bar->id);
# others...
PyObject_SetAttrString(obj, "id", id);
# others...
Py_DECREF(id);
# others...
return obj;
For some reason I had it in my head that the PyLong_FromLong function did NOT increment the ref count of the resulting object, but this is apparently not true. This is how I wound up with an extra reference count for every bar object that was created.