Parametric ReLU (PReLU)

[nleck]

Parametric ReLU (PReLU) - This is a variant of the ReLU (Rectified Linear Unit) that allows the slope of the negative part to be learned during training, unlike the Leaky ReLU which has a fixed slope. This can sometimes lead to better performance because the network can adjust the activation function to fit the data better.