-专业IT技术社区(指牛堂) Neural Network Tuning Web程序 - 指牛堂-专业IT技术平台

Neural Network Tuning Web程序

标签:where   ted   bat   ber   log   cape   body   esc   amount   

Q1: assuming that we train the neural network with the same amount of training examples, how to set the optimal batch size and number of iterations? (where batch size * number of iterations = number of training examples shown to the neural network, with the same training example being potentially shown several times)

It has been observed in practice that when using a larger batch there is a significant degradation in the quality of the model, as measured by its ability to generalize. Large-batch methods tend to converge to sharp minimizers of the training and testing functions -- and that sharp minima lead to poorer generalization. In contrast, small-batch methods consistently converge to flat minimizers. Large-batch methods are almost invariably attracted to regions with sharp minima and that, unlike small batch methods, are unable to escape basins of these minimizers.

Neural Network Tuning

标签:where   ted   bat   ber   log   cape   body   esc   amount   



已有 0 条评论