Or why you shouldn’t belief predict_proba strategies
In earlier articles, I identified the significance of realizing how certain a mannequin is about its predictions.
For classification issues, it isn’t useful to solely know the ultimate class. We’d like extra data to make well-informed choices in downstream processes. A classification mannequin that solely outputs the ultimate class covers essential data. We have no idea how certain the mannequin is and the way a lot we are able to belief its prediction.
How can we obtain extra belief within the mannequin?
Two approaches can provide us extra perception into classification issues.
We might flip our level prediction right into a prediction set. The objective of the prediction set is to ensure that it accommodates the true class with a given chance. The scale of the prediction set then tells us how certain our mannequin is about its prediction. The less courses the prediction set accommodates, the surer the mannequin is.
