Non Linear Optimization


The goals are:

1- To distinguish the problems by degree of difficulty.

2 - To know optimality conditions and methods for local optima.

3- To understand how the methods "work" for problems with and without constraints, and to be able to compare their merits and weaknesses and convergence rate.

4- To understand the application of some methods for special problems like least squares.

5- To  have a basic overview of derivative free  optimization methods.

General characterization





Responsible teacher

Marcos Alejandro Raydan


Weekly - 3

Total - Available soon

Teaching language



Basic linear algebra and basic calculus.


1. C. Audet and W. Hare, Derivative-Free and Blackbox Optimization, Springer, 2017.

2. D. P. Bertsekas, Nonlinear Programming, second edition, Athena Scientific, 1999.

3. J. Nocedal and S. J. Wright, Numerical Optimization, second edition, Springer, 2006.

Teaching method

Available soon

Evaluation method

The evaluation will be done through  two exams (one in the middle of the semester and the  other one at the end) and several homework assignments (approximately 6), each one including  a theoretical  and a practical part.

Each exam represents 30 percent of the final grade, and the average of the homework assigments represents the remaining 40 percent.

Subject matter

1. Unconstrained Problems

               1.1 Basics (convexity and optimality conditions).

               1.2 Gradient-type methods and Newton method for local optima.

               1.3 Quasi-Newton methods.

               1.4 Line search strategies.

               1.5 Trust region methods.

               1.6 Introduction to derivative-free optimization. 

  2. Constrained Optimization

               2.1 Basics (convex-constrained case and optimality conditions).

               2.2 Quadratic programming.

               2.3 Penalty, barrier and augmented Lagrangian methods.

 3. Least squares problems (nonlinear).

 4. A brief introduction to stochastic optimization.


Programs where the course is taught: