In this article, we will talk about methods for solving mathematical optimization problems based on the use of the gradient of a function. The main goal is to collect in the article all the most important ideas that are somehow connected with this method and its various modifications.
Note from the author
At the time of writing, I defended my thesis, the task of which required me to have a deep understanding of theoretical fundamentally methods of mathematical optimization. Nevertheless, I still (like everyone else) have their eyes blown away from the terrible long formulas...[/h][/h]