Top products from r/optimization

Hi OP,

I found myself in a similar situation to you. To add a bit of context, I wanted to learn optimization for the sake of application to DSP/machine learning and related domains in ECE. However, I also wanted sufficient intuition and awareness to understand and appreciate optimization it for it's own sake. Further, I wanted to know how to numerically implement methods in real-time (embedded platforms) to solve the formulated problems (Since my job involves firmware development). I am assuming from your question that you are interested in some practical implementation/simulations too.

​

< A SAMPLE PIPELINE >

Optimization problem formulation -> Enumerating solution methods to formulated problem -> Algorithm development (on MATLAB for instance) -> Numerical analysis and fixed-point modelling -> Software implementation -> Optimized software implementation.

​

So, building from my coursework during my Masters (Involving the standard LinAlg, S&P, Optimization, Statistical Signal Processing, Pattern Recognition, <some> Real Analysis and Numerical methods), I mapped out a curriculum for myself to achieve the goals I explained in paragraph 1. The Optimization/Numerical sections of the same is as below:

​

OPTIMIZATION MODELS:

1. Optimization Models by Calafiore and El Ghaoui (Excellent and thorough reference book)
   
2. Non-linear Programming by D.Bertsakas ( I agree that nonlinear programming is very relevant and will be very useful in the future too)
   
   

• Lectures from Ohio State: https://www.youtube.com/playlist?list=PL_Nk3YvgORJtKF6B-cRBCKxi9jWgM0pcx
  
  

1. Convex Optimization by S. Boyd and Vandenberghe (Another very good book for basics)
   
   

• Lectures by S. Boyd on the same: https://www.youtube.com/playlist?list=PL3940DD956CDF0622
  
  NUMERICAL METHODS:
  
  

1. Numerical Linear Algebra by L.N.Trefethen and D.Bau III (Very good explanation of concepts and algorithms and you might be able to find a free ebook version online)
   
2. Numerical Optimization by Jorge Nocedal and S.Wright (Both authors are very accomplished and the textbook is well regraded as a sound introduction to this subject)
   
3. Numerical Algorithms by Justin Solomon (He's a very good teacher whose presentation is digestible immediately)
   
   

• His Lectures are here: https://www.youtube.com/playlist?list=PLHrg69yaUAPeiLEsa-1KauSe2HaA0Wf6I
  
  ​
  
  Personally I think this might be a good starting point, and as other posters have mentioned, you will need to tailor it to your use-case. Remember that learning is always iterative and you can re-discover/go deeper once you've finished a first pass. Front-loading all the knowledge at once usually is impractical.
  
  ​
  
  All the best and hope this helped!

A good book that has a large number of sample models is "Model Building in Mathematical Programming" by H.P. Williams

IBM has a large collection of examples, which they provide example solutions for, in their documentation for their CPLEX Studio product stack.

If you are interested in constraint programming, the IBM collection has a number of relevant examples, and there are a lot of example you could find for the MiniZinc platform. Here is a collection of examples that come from a coursera MOOC that might be worth investing some time in.

If you are interested in Metaheuristic/Search types of optimization, the OptaPlanner tool has some great examples that are worth a look.

I saw the name of a book by Lange (https://www.amazon.com/Optimization-Springer-Texts-Statistics-Kenneth/dp/1461458374/) in a Quora question: (https://www.quora.com/What-are-the-major-subfields-of-optimization-theory-What-textbooks-are-good-for-learning-about-them)

Can you comment on that book? It seems uptodate, general, and introduces math too.