Does RandomForestsClassifier have a `decision_function`? - scikit-learn

Question

SVC in sklearn has a decision_function function, but why doesn't random forests? Secondly, how can I emulate the decision_function for a RandomForestsClassifier?

Answer 1

Yes, it has a decision function.
The function is coded directly into the Forest of Randomised Trees.
Simply put RandomForest learners use this trivial decision function strategy:
Rule a): The Forest ( the ensemble ) votes on result.
Rule b): Each Randomised Tree contains a series of decisions ( represented by Node(s).
Rule c): Each Node contains one or more simple conditions, based on which the decision making process moves farther from the root-Node towards the terminal-Leaf, according to the specified values of the Example FeatureVECTOR-input values.
Rule d): Terminal-Leaf contains a value, that a particular Randomised Tree presents to the ensemble voting ( Ad-a) ).
While this is not an exact RandomForestClassifier Tree-representation ( a Node can have more than one condition there - check the Scikit documentation on parameters ) but a fairly well illustrating the principles:
VOTE[12] = IN[0] < 6.85417 ? 1 : 2
| |
| 2:[IN[5]<183]
| |
| IN[5] < 183 ? 5 : 6
| | |
| | 6:[IN[10]<1.00118]
| | |
| | IN[10] < 1.00118 ? 13 : 14
| | | |
| | | 14:
| | | |
| | |
| | |
| |
| 5:[IN[6]<187]
| |
| IN[6] < 187 ? 11 : 12
| | |
1:
IN[10] < 1.00054 ? 3 : 4
| |
| 4:
| |
|
3:
voter[12]:
0:[inp0<6.85417] yes=1,no=2,missing=1
1:[inp10<1.00054] yes=3,no=4,missing=3
3:[inp21<0.974632] yes=7,no=8,missing=7
7:[inp22<1.01021] yes=15,no=16,missing=15
15:[inp15<0.994931] yes=31,no=32,missing=31
31:[inp12<0.999151] yes=63,no=64,missing=63
63:[inp23<0.957624] yes=111,no=112,missing=111
111:leaf=0.163636
112:leaf=-0.36
64:leaf=0.323077
32:[inp19<0.993949] yes=65,no=66,missing=65
65:[inp23<0.931146] yes=113,no=114,missing=113
113:leaf=-0
114:[inp23<0.972193] yes=161,no=162,missing=161
161:leaf=-0.421782
162:leaf=-0.133333
66:[inp2<61] yes=115,no=116,missing=115
115:leaf=0.381818
116:leaf=-0.388235
16:[inp17<0.985065] yes=33,no=34,missing=33
How to emulate decision function?
One can build another RandomForest-interpreter to traverse all the Trees and vote on resulting "ensemble compromise".
That will mimick the decision function ( there is no other there ) and would provide the same results as the RandomForest Trees were constructed to work this very way.