Question 1What task does Batch normalization do?1 pointWe normalize the input layer by adjusting and scaling the activations Reducing Internal Covariate Shift

Understanding Batch Normalization

1. Definition and Purpose
   Batch normalization is a technique used in training deep neural networks. Its primary purpose is to improve the speed, performance, and stability of the model.

2. Main Functions

   • Normalize Inputs: It normalizes the inputs to each layer by adjusting and scaling the activations. This helps in maintaining a stable distribution of activations throughout the training process.
   • Reduce Internal Covariate Shift: By normalizing the inputs, batch normalization reduces the so-called internal covariate shift, which refers to the changes in the distribution of network activations due to changes in the parameters during training.

3. Operational Steps
   During training, for each mini-batch, batch normalization computes the mean and variance of the activations, applies the normalization process, and then scales and shifts the normalized values using learnable parameters.

Conclusion

Batch normalization serves mainly to normalize activations to ensure stable distributions, thus enhancing the training efficiency of deep learning models. It is an essential component for modern neural network architectures, helping to mitigate issues related to training.

Final Answer

Batch normalization normalizes the input layer by adjusting and scaling the activations to reduce internal covariate shift.