Automatic Differentiation
A method used to efficiently and accurately compute the gradient of a function with respect to its inputs. The gradient provides information about the rate of change of the function in relation to its inputs and is used in optimization algorithms to update the parameters of a model.
Automatic differentiation is used in machine learning and artificial intelligence to compute gradients for training models. It is beneficial for training deep neural networks, where the computation graph can be complex, and the gradients are required for many optimization algorithms.