What is the difference between conformal prediction and uncertainty estimation

Question

Recently I am seeing the topic of Conformal Prediction to be very trendy on social media and research. Awesome Conformal Prediction

But what is the main difference between conformal prediction and uncertainty estimation?

Answer 1

The main difference is that Conformal Prediction is a much better framework because it is:

1. distribution free
2. works with any model without ...
3. provided mathematical guarantees of validity for final samples of any size.

In comparison Bayesian frameworks produce posterior in the wrong place (unless using synthetic data where the data distribution is exactly known), interfere with the underlying models and do not have any mathematical guarantees of validity.

Answer 2

Conformal Prediction is a distribution-free statistically rigorous uncertainty uncertainty quantification framework in the form of prediction sets for classification and prediction intervals for regression (Vovk, V.; Gammerman, A.; and Shafer, G. 2005. Algorithmic Learning in a Random World. Berlin, Heidelberg:Springer-Verlag. ISBN 0387001522., and Shafer 2005).

Bayesian method provides a natural and principled way for modeling uncertainty. In Deep learning, epistemic uncertainty concerns the probabilistic estimation of model parameters. There are number of approaches have been developed to measure uncertainty in neural networks, such as approx. Bayesian deep learning techniques such as Monte-Carlo dropout (Gal, Y.; and Ghahramani, Z. 2016), DropWeights (Ghoshal, B.; and Tucker, A. 2019), and an ensemble of deep models (Lakshminarayanan, Pritzel, and Blundell 2017).