Glossary

Gradient descent: an iterative algorithm for finding the loss function minimum. It moves in the direction of the negative gradient and gradually approximates the minimum.

Gradient of a vector-valued function: a vector consisting of derivatives of the answer for each argument that indicates the direction in which the function grows the fastest.

Practice

COPYpython
1import numpy as np
2
3def func(x):
4    # function to be minimized
5
6def gradient(x):
7    # func function gradient
8
9def gradient_descent(initialization, step_size, iterations):
10    x = initialization
11    for i in range(iterations):
12        x = x - step_size * gradient(x)
13    return x