³We moving between these options, it is important to note that weight decay is applied each time weights are updated. If weights are updated after each pattern, a smaller value of weight decay should be used than if they are updated after a batch of n patterns or a whole epoch.