simpleopt.opt - Simple optimization methods

The simpleopt.opt module provides implementations of the steepest descent and conjugate gradient methods for solving unconstrained optimization problems, as well as a simple class for defining and storing optimization problems.

class simpleopt.opt.ConjugateGradient(problem, x0, gtol, alpha, max_iter)

Bases: _Optimizer

Conjugate Gradient algorithm with Fletcher-Reeves rule and Armijo line search for solving unconstrained optimization problems.

All parameters of _Optimizer are valid in this optimization solver.

_solve()

Solve the optimization problem using the conjugate gradient algorithm.

class simpleopt.opt.Problem(f, gradf, dim, prob_type='min', method=None)

Bases: object

A class for formulating an optimization problem.

Initialize the problem class.

Parameters:

  • self (object) – Optimization problem

  • f (callable) – Function to be optimized

  • gradf (callable) – Gradient of the function to be optimized

  • dim (int) – Dimension of the optimization problem

  • prob_type (str) – Type of optimization problem (‘min’ or ‘max’)

  • method (str) – Optimization method to use (‘sd’ for steepest descent or ‘cg’ for conjugate gradient)

solve(x0=None, gtol=1e-06, alpha=1, max_iter=None)

Solve the optimization problem using the method specified in the constructor.

Parameters:

  • self (object) – Optimization problem

  • x0 (NDArray) – Initial point

  • gtol (float) – Tolerance for the gradient

  • alpha (float) – Step size

  • max_iter (int) – Maximum number of iterations

Returns:

sol: Optimal point

Return type:

NDArray

:return info : Dictionary with additional information about the optimization solve. :rtype info: dict

class simpleopt.opt.SteepestDescent(problem, x0, gtol, alpha, max_iter)

Bases: _Optimizer

Steepest descent algorithm with Armijo line search for solving unconstrained optimization problems.

All parameters of _Optimizer are valid in this optimization solver.

_solve()

Solve the optimization problem using the steepest descent algorithm.

class simpleopt.opt._Optimizer(problem, x0=None, gtol=1e-06, alpha=1, max_iter=None)

Bases: object

Base class for optimization algorithms.

Instantiate the optimizer with the problem to be solved and parameters. :param problem: Object of class Problem :type problem: object :param x0: Initial point :type x0: NDArray :param gtol: Tolerance for stopping the algorithm when the norm of the gradient is less than tol :type gtol: float :param alpha: Step size :type alpha: float :param max_iter: Maximum number of iterations :type max_iter: int

After instantiating the Optimizer, the object contains the following attributes:

  • sol: the approximate solution sol

  • converged: True if the algorithm converged, False otherwise.

  • iter_count: Number of iterations performed.

  • iter_fvalues: List of function values at each iteration.

Parameters:

  • problem (Problem) –

  • x0 (ndarray[Any, dtype[ScalarType]]) –

  • gtol (float) –

  • alpha (float) –

Line search algorithm with Armijo condition for finding the step size

that minimizes a function at a given point towards a given direction.

Parameters:

  • direction (NDArray) – Direction of search

  • x (NDArray) – Current point

  • alpha (float) – Initial step size

  • beta (float) – Step size reduction factor

  • max_iter (int) – Maximum number of iterations

Returns:

alpha: Optimal step size

Rtype alpha:

float

Returns:

converged: Boolean variable indicating whether the algorithm converged

Rtype converged:

bool

_solve()

Abstract method that solves the optimization problem.