pyml.neural_network.optimizer.adagrad.Adagrad#

class Adagrad(learning_rate=1.0, decay=0.0, epsilon=1e-07)[source]#

Bases: _Optimizer

Adagrad Optimizer for Neural Networks.

This optimizer adapts the learning rate of each parameter based on the historical gradient information. It uses squared gradients to scale the learning rate for each parameter separately.

Parameters:

learning_rate (float, optional) – The initial learning rate, by default 1.0
decay (float, optional) – Learning rate decay factor, by default 0.0
epsilon (float, optional) – Small value added to the denominator to prevent division by zero, by default 1e-7

Methods

`__init__`
`post_update_parameters`	Updates the iteration counter after each layer update
`pre_update_parameters`	Update the current learning rate based on decay.
`update_parameters`	Update the parameters of the given layer using the Adagrad optimization algorithm.

post_update_parameters()#

Updates the iteration counter after each layer update

Return type:: None

pre_update_parameters()#

Update the current learning rate based on decay.

This method calculates and updates the current learning rate based on the decay factor and the number of iterations performed.

Return type:: None

update_parameters(layer)[source]#

Update the parameters of the given layer using the Adagrad optimization algorithm.

Return type:: None
Parameters:: layer (_Transformation) – The layer to update.

Note

If the layer does not have cache arrays for weights and biases, this method initializes them with zeros. It then updates the cache with squared gradients and performs parameter updates using Adagrad.