Which activation function is commonly used in the hidden layers of a neural network to introduce non-linearity?Group of answer choicesSigmoidLinearSoftmaxReLU

The activation function commonly used in the hidden layers of a neural network to introduce non-linearity is ReLU, which stands for Rectified Linear Unit. ReLU is preferred because it allows for faster training and helps mitigate the vanishing gradient problem, making it more effective for deep networks.

Explanation of Options:

1. Sigmoid: This function maps input values to a range between 0 and 1, but it can cause vanishing gradients for deep networks.
2. Linear: Linear activation is essentially no activation, which means it does not introduce non-linearity, making it unsuitable for hidden layers.
3. Softmax: This function is typically used in the output layer for multi-class classification, converting logits into probabilities, but is not used in hidden layers.
4. ReLU: This function outputs the input directly if it is positive; otherwise, it outputs zero, helping capture non-linear patterns effectively.

In summary, for hidden layers, ReLU is the widely adopted choice due to its advantageous properties in deep learning contexts.