MathLiveGradient descent update rule
Gives the next iterate of gradient descent by moving the current point a small step in the direction opposite the gradient of the function being minimized. It is used to iteratively find a local minimum of a differentiable function.
\mathbf{x}_{n+1} = \mathbf{x}_n - \gamma \nabla F(\mathbf{x}_n)What the symbols mean
| current point (parameter vector) at iteration n | |
| updated point at iteration n+1 | |
| step size (learning rate), a positive scalar | |
| gradient of the objective function F evaluated at \mathbf{x}_n |
About this formula
- FieldComputer science
- TopicOptimization algorithms
- LevelUniversity
- Also calledgradient descent step, steepest descent update, method of steepest descent, hill-climbing update rule, gradient descent formula, iterative gradient update
- SourceWikidata Q1199743
Write it in Obsidian
MathLive is a free visual math editor for Obsidian. Run Find a formula, type “gradient descent update rule”, and it goes into your note as editable LaTeX. You can also scan formulas from screenshots, PDFs and handwriting.