Как накапливать градиенты для больших размеров пакетов в Keras ⇐ Python
-
Гость
Как накапливать градиенты для больших размеров пакетов в Keras
I am working with a very memory demanding CNN model for a task of classification.
This poses a big limit on the batch size that I can use during training.
One solution is to accumulate the gradients during training, meaning that the weights of the model are not updated after every single batch. Instead the same weights are used for several batches, while the gradients from each batch are accumulated and than averaged for a single weight-update action.
I'm using a Tensorflow backend Keras and I'm pretty sure that Keras has no off-the-shelf function/method to achieve this.
How can it be done for a Keras/tensorflow model?
Источник: https://stackoverflow.com/questions/552 ... s-in-keras
I am working with a very memory demanding CNN model for a task of classification.
This poses a big limit on the batch size that I can use during training.
One solution is to accumulate the gradients during training, meaning that the weights of the model are not updated after every single batch. Instead the same weights are used for several batches, while the gradients from each batch are accumulated and than averaged for a single weight-update action.
I'm using a Tensorflow backend Keras and I'm pretty sure that Keras has no off-the-shelf function/method to achieve this.
How can it be done for a Keras/tensorflow model?
Источник: https://stackoverflow.com/questions/552 ... s-in-keras
