Some years ago, my former manager told me a phrase that would later make sense and guide my research interests for the next years: "I do not care that your model does not know everything, as long as I know what it does not know."
Deep learning models are often overconfident and can predict outputs with high probability even for inputs with patterns different from those used during training and validation.
This overconfidence is a problem since it creates an illusion of security in the predictions. Uncertainty analysis helps developers and users understand more about the knowledge of the models.
Imagine that you are a biologist specializing in identifying butterflies. One day, a student arrives with a butterfly you have never seen before. What do you do? You could give your best guess as to which group that butterfly belongs to, but you will also tell the student that you need more time to research this unknown species and give them a better answer when you have more knowledge.
Ideally, deep learning models should do this: give a prediction along with a warning about their knowledge—or lack thereof—about the input pattern.
In this post, we will discuss the uncertainties of deep learning models' predictions and how you can use them to your advantage.
The uncertainties in predictions, also called predictive uncertainties, can be divided into two types:1
- Epistemic Uncertainty
- This is due to the model itself and reflects its lack of knowledge. Given more data about the patterns that the model doesn't know, this uncertainty decreases.
- Aleatoric Uncertainty
- This is the uncertainty inherent in the data itself and can be caused, for example, by noise in the sensor that captures the data. This uncertainty will not decrease even with more data.
The predictive uncertainty of deep learning predictions can be estimated using different techniques, such as Monte Carlo Dropout (MC-Dropout), Deep Ensembles, Direct Modeling, and Error Propagation2.
- Direct Modeling treats the uncertainty as an output of the model. In this case, the model's head contains neurons to predict the uncertainty, and the loss used for training is designed for the model to learn the uncertainty. For example, the model can predict the mean and variance of the output, where the variance corresponds to the uncertainty. A negative log-likelihood loss can be used to train the model to maximize the probability that the output is drawn from a normal distribution with the predicted mean and variance. This technique leverages aleatoric uncertainty.
- Error Propagation is used for assessing both aleatoric and epistemic uncertainties. This method captures the model’s uncertainty by propagating the variance from each layer, which arises from layers such as dropout and batch normalization, to the model’s output.
- MC-Dropout is used for modeling epistemic uncertainty. During inference, MC-Dropout allows the use of the dropout layers, and the same input sample is forward-passed many times. The variance of the outputs of each forward pass is used to assess epistemic uncertainty.
- Deep Ensembles are also used for assessing epistemic uncertainty. An input sample is passed through the ensemble, and the variance in the outputs of the neural networks that compose the ensemble is used to evaluate the uncertainty.
MC-Dropout is a widely used method due to its easy implementation in models that contain dropout layers. In the following paragraphs, we will explain how it works and how it can be implemented.
Usually, dropout layers are used to prevent overfitting during training. However, if these layers are allowed during inference, the uncertainty of the predictions can be assessed. Because the dropout layers randomly drop out neurons in the model with a certain probability, each time the same input is forward passed through the model, a potentially different result is obtained. If the model is well-trained and has enough knowledge about the input pattern, the outputs will be equal or numerically very close to each other. If the model does not have enough knowledge about the input pattern, the outputs of each forward pass of the same input will result in very different outputs.
To calculate the uncertainty, the following steps are performed:
- Allow the dropout layers;
- Foward pass the same input many times (T times) through the deep learning model;
- Measure the uncertainties of the outputs by applying some metric to the values predicted by the model.
Many different metrics can be used for calculating the uncertainties of the predictions, based on the values of the predicted outputs.3 Some examples are:
- Mutual Information
- Entropy
- Variation Ratio
- Total Variance
The metrics mentioned above are applied to each output predicted. This means that for segmentation models, an uncertainty value is calculated for each value predicted to each different part (or pixel) of the image. An example is shown in the image below, which displays the uncertainties as a heatmap, where warmer colors (more yellow/red) indicate higher uncertainty.
For some tasks, such as in active learning, it is necessary to compare the amount of uncertainties of predictions referring to different inputs. When dealing with segmentation, where a map of uncertainties is generated, these values need to be aggregated somehow. One option for aggregating the uncertainty in this case is to calculate the mean value of the uncertainties corresponding to an image.
It is important to keep in mind that the predictions might have high uncertainty only for a small region of the image, and averaging the uncertainties for the entire image could hide the significance of this high uncertainty. Another consideration is that predictions of different classes of objects have different levels of uncertainty, and not accounting for these different classes when averaging the results could also suppress important information about the uncertainties of a predicted segmentation mask.
Now you have an idea of how to calculate uncertainties! But what can you do with that?
Incorrect predictions tend to have higher uncertainties than correct predictions. These unceratinties are much more reliable than the probabilities of the softmax in the last layer of classification and segmentation models. This way, the uncertainties can be used to guide the reliability of the predictions. It is important to remember that different datasets and different models have different levels of uncertainty. Therefore, a study of the uncertainty should be performed for each individual case, analyzing the uncertainties in the training and validation sets, and using these values to define acceptable uncertainty thresholds for use during inference.
Another use case is to guide the selection of data to be labeled when creating a dataset for training a model. Active learning is a family of methods that aim to select a small subset of a dataset that can be used for training a model to achieve performance similar to the performance achieved by the entire dataset. Being able to select a smaller dataset for labeling and training the model has the huge advantage of saving time and money with the tedious task of labeling datasets. For selecting the images, an acquisition function needs to be defined. One option is to use uncertainty as the acquisition function.
An example of an active learning application is the research that we developed for selecting a small subset of images for training a semantic segmentation model for underwater pipeline images. The image below explains the methodology utilized in our research.4
As can be seen in the image above, the methodology follows these steps:
- Select some images in the pool of unlabeled images for labeling:
- Initially, all the images are unlabeled and the pool of labeled images is empty;
- In the first iteration, a small percentage of images is randomly selected;
- In the next iterations, the unlabeled images are selected if the uncertainty corresponding to the predictions performed by the model trained in (3) are above the threshold defined in (5);
- Train a segmentation model
- Calculate the uncertainty of the labeled images using MC-dropout:
- Because we want to select images that contain a pattern that the model does not have enough knowledge of, we use the epistemic uncertainty;
- For calculating the uncertainty of the image, we calculate the average value of the uncertainty heatmap;
- Define a threshold based on the uncertainties calculated in (4)
- Go back to step (1)
Still using active learning, we developed research for adapting a model trained with synthetic images to be applied to real images, using the least effort for labeling images. The figure below summarizes our work.
We hope you are now curious to check out our papers!
If you would like to read them, here is the name of our paper on active learning:
"Uncertainty Driven Active Learning for Image Segmentation in Underwater Inspection.", by Luiza Ribeiro Marnet, Yury Brodskiy, Stella Grasshof, and Andrzej Wąsowski.
For the Sim2Real paper, here is the reference:
"Bridging the Sim-to-Real GAP for Underwater Image Segmentation", by Luiza Ribeiro Marnet, Stella Grasshof, Yury Brodskiy, and Andrzej Wąsowski.
We also released a dataset for underwater pipeline segmentation, which can be downloaded from this link
Footnotes
-
Kendall, Alex, and Yarin Gal. "What uncertainties do we need in bayesian deep learning for computer vision?." Advances in neural information processing systems 30 (2017). ↩
-
Feng, Di, et al. "A review and comparative study on probabilistic object detection in autonomous driving." IEEE Transactions on Intelligent Transportation Systems 23.8 (2021): 9961-9980. ↩
-
Gal, Yarin. "Uncertainty in deep learning." (2016). ↩
-
Ribeiro Marnet, Luiza, et al. "Uncertainty Driven Active Learning for Image Segmentation in Underwater Inspection." International Conference on Robotics, Computer Vision and Intelligent Systems. Cham: Springer Nature Switzerland, 2024. ↩