Reddit reviews Optimization Models

Hi OP,

I found myself in a similar situation to you. To add a bit of context, I wanted to learn optimization for the sake of application to DSP/machine learning and related domains in ECE. However, I also wanted sufficient intuition and awareness to understand and appreciate optimization it for it's own sake. Further, I wanted to know how to numerically implement methods in real-time (embedded platforms) to solve the formulated problems (Since my job involves firmware development). I am assuming from your question that you are interested in some practical implementation/simulations too.

​

< A SAMPLE PIPELINE >

Optimization problem formulation -> Enumerating solution methods to formulated problem -> Algorithm development (on MATLAB for instance) -> Numerical analysis and fixed-point modelling -> Software implementation -> Optimized software implementation.

​

So, building from my coursework during my Masters (Involving the standard LinAlg, S&P, Optimization, Statistical Signal Processing, Pattern Recognition, <some> Real Analysis and Numerical methods), I mapped out a curriculum for myself to achieve the goals I explained in paragraph 1. The Optimization/Numerical sections of the same is as below:

​

OPTIMIZATION MODELS:

1. Optimization Models by Calafiore and El Ghaoui (Excellent and thorough reference book)
   
2. Non-linear Programming by D.Bertsakas ( I agree that nonlinear programming is very relevant and will be very useful in the future too)
   
   

• Lectures from Ohio State: https://www.youtube.com/playlist?list=PL_Nk3YvgORJtKF6B-cRBCKxi9jWgM0pcx
  
  

1. Convex Optimization by S. Boyd and Vandenberghe (Another very good book for basics)
   
   

• Lectures by S. Boyd on the same: https://www.youtube.com/playlist?list=PL3940DD956CDF0622
  
  NUMERICAL METHODS:
  
  

1. Numerical Linear Algebra by L.N.Trefethen and D.Bau III (Very good explanation of concepts and algorithms and you might be able to find a free ebook version online)
   
2. Numerical Optimization by Jorge Nocedal and S.Wright (Both authors are very accomplished and the textbook is well regraded as a sound introduction to this subject)
   
3. Numerical Algorithms by Justin Solomon (He's a very good teacher whose presentation is digestible immediately)
   
   

• His Lectures are here: https://www.youtube.com/playlist?list=PLHrg69yaUAPeiLEsa-1KauSe2HaA0Wf6I
  
  ​
  
  Personally I think this might be a good starting point, and as other posters have mentioned, you will need to tailor it to your use-case. Remember that learning is always iterative and you can re-discover/go deeper once you've finished a first pass. Front-loading all the knowledge at once usually is impractical.
  
  ​
  
  All the best and hope this helped!