This page (revision-1) was last changed on 29-Nov-2024 16:16 by UnknownAuthor

Only authorized users are allowed to rename pages.

Only authorized users are allowed to delete pages.

Page revision history

Version Date Modified Size Author Changes ... Change note

Page References

Incoming links Outgoing links
Dropout Regularization
Dropout Regularization

Version management

Difference between version and

At line 1 added 48 lines
!!! Overview [1]
[{$pagename}] is a [Regularization] method where randomly selected [Artificial Neurons] are ignored during training.
They are "dropped-out" randomly. This means that their contribution to the activation of downstream neurons is temporarily removed on the [Forward propagation] and any [weight] updates are also __not__ applied to the [Artificial Neurons] on the [Backpropagation].
As an [Artificial Neural network] learns, neuron [weights] settle into their context within the [Artificial Neural network]. [Weights] of neurons are tuned for specific features providing some specialization. Neighboring neurons become to rely on this specialization, which if taken too far can result in a fragile [Machine Learning model] too specialized to the [Training dataset] which is referred to as [Overfitting]. This reliant on context for a neuron during training is referred to complex co-adaptations.
You can imagine that if [Artificial Neurons] are randomly dropped out of the network during training, that other [Artificial Neurons] will have to step in and handle the representation required to make predictions for the missing [Artificial Neurons]. This is believed to result in multiple independent internal representations being learned by the network.
The effect is that the network becomes less sensitive to the specific [weights] of neurons. This in turn results in a network that is capable of better generalization and is less likely to [Overfitting].
!! More Information
There might be more information for this subject on one of the following:
[{ReferringPagesPlugin before='*' after='\n' }]
* [#1] - [Dropout Regularization in Deep Learning Models With Keras|https://machinelearningmastery.com/dropout-regularization-deep-learning-models-keras/|target='_blank'] - based on information obtained 2018-01-03-
This page (revision-1) was last changed on 29-Nov-2024 16:16 by UnknownAuthorTop