Reddit reviews Numerical Linear Algebra

Why not read an introductory text to numerical linear algebra like Trefethen and Bau?

This is the book I used. It's a solid read with lots of good problems and examples.

https://www.amazon.com/Numerical-Linear-Algebra-Lloyd-Trefethen/dp/0898713617

Well I'd recommend:

• The Elements of Statistical Learning - this is kind of the go-to standard book recommendation
  
• GitHub: Python Data Science Handbook
  
• GitHub: iPython notebooks for data science - really useful to explore concepts
  
• Numerical Linear Algebra - if you need a linear algebra refresh
  
• Machine Learning: A Probabilistic Perspective
  
• Data Science for Business - provides a high level overview of techniques
  
  For a more basic stats refresh before you dive in, pretty much any introductory textbook will be sufficient. For a very basic but quick and dirty refresh on basic stats you can get: Statistics in Plain English

Basically all reading in mathematics will help with this. What kind of applied mathematician? What is your background?

If I had to pick one medium-sized book, I’d say read Trefethen & Bau’s Numerical Linear Algebra.

The Nature of Computation

(I don't care for people who say this is computer science, not real math. It's math. And it's the greatest textbook ever written at that.)

Concrete Mathematics

Understanding Analysis

An Introduction to Statistical Learning

Numerical Linear Algebra

Introduction to Probability

Elements of Statistical Learning covers KDE pretty well. (It does have a pretty heavy linear algebra prereq. If it is getting too hairy, you may want to look at a numerical linear algebra book, like Trefethen and Bau)

Also Computational Statistics covers it well from what I remember. These are both really good books.

But both are really great books.

You might want to consider some kind of numerical linear algebra book like the very readable Trefethen and Bau.

While this topic isn't always included in an undergrad curriculum, it's hugely useful. It's critical for a bunch of more advanced areas like physical simulation, graphics optimization, and machine learning.

Numerical Linear Algebra by Nick Trefethen is a pretty friendly intro to graduate linear algebra/matrix theory from a numerical analysis angle:

http://www.amazon.com/Numerical-Linear-Algebra-Lloyd-Trefethen/dp/0898713617

Introduction to Numerical Analysis is very comprehensive, more advanced, but reads like an encyclopedia in a way. A good reference, though not very good as a lone textbook.

http://www.amazon.com/Introduction-Numerical-Analysis-J-Stoer/dp/038795452X

Hi OP,

I found myself in a similar situation to you. To add a bit of context, I wanted to learn optimization for the sake of application to DSP/machine learning and related domains in ECE. However, I also wanted sufficient intuition and awareness to understand and appreciate optimization it for it's own sake. Further, I wanted to know how to numerically implement methods in real-time (embedded platforms) to solve the formulated problems (Since my job involves firmware development). I am assuming from your question that you are interested in some practical implementation/simulations too.

​

< A SAMPLE PIPELINE >

Optimization problem formulation -> Enumerating solution methods to formulated problem -> Algorithm development (on MATLAB for instance) -> Numerical analysis and fixed-point modelling -> Software implementation -> Optimized software implementation.

​

So, building from my coursework during my Masters (Involving the standard LinAlg, S&P, Optimization, Statistical Signal Processing, Pattern Recognition, <some> Real Analysis and Numerical methods), I mapped out a curriculum for myself to achieve the goals I explained in paragraph 1. The Optimization/Numerical sections of the same is as below:

​

OPTIMIZATION MODELS:

1. Optimization Models by Calafiore and El Ghaoui (Excellent and thorough reference book)
   
2. Non-linear Programming by D.Bertsakas ( I agree that nonlinear programming is very relevant and will be very useful in the future too)
   
   

• Lectures from Ohio State: https://www.youtube.com/playlist?list=PL_Nk3YvgORJtKF6B-cRBCKxi9jWgM0pcx
  
  

1. Convex Optimization by S. Boyd and Vandenberghe (Another very good book for basics)
   
   

• Lectures by S. Boyd on the same: https://www.youtube.com/playlist?list=PL3940DD956CDF0622
  
  NUMERICAL METHODS:
  
  

1. Numerical Linear Algebra by L.N.Trefethen and D.Bau III (Very good explanation of concepts and algorithms and you might be able to find a free ebook version online)
   
2. Numerical Optimization by Jorge Nocedal and S.Wright (Both authors are very accomplished and the textbook is well regraded as a sound introduction to this subject)
   
3. Numerical Algorithms by Justin Solomon (He's a very good teacher whose presentation is digestible immediately)
   
   

• His Lectures are here: https://www.youtube.com/playlist?list=PLHrg69yaUAPeiLEsa-1KauSe2HaA0Wf6I
  
  ​
  
  Personally I think this might be a good starting point, and as other posters have mentioned, you will need to tailor it to your use-case. Remember that learning is always iterative and you can re-discover/go deeper once you've finished a first pass. Front-loading all the knowledge at once usually is impractical.
  
  ​
  
  All the best and hope this helped!

Try buying a new hardcover of this linear algebra book!

Earlier this year I finished my PhD in aero (researching computational fluid dynamics). I'll go ahead and reiterate a couple of the other recommendations in this thread, I think they've given you pretty good advice so far.

Numerical Recipes is great, and you can even read their older editions for free online. Don't worry about them being older, their content really hasn't changed much over the years beyond switching around the programming language. A word of warning, though. The code itself in these books come with rather restrictive licenses, and what it ends up meaning for you is you can copy their code and use it yourself, but you aren't allowed to share it (although I don't think this is carefully enforced). If you want to share code, you'll either have to pay for their license, or use their code only as inspiration for writing your own. If you pay close attention to their licensing, they don't even let you store on your computer more than one copy of any of their functions (again, I can't imagine they actually have a way of enforcing this, but it makes me disappointed they do things this way nevertheless), so it can get problematic fast.

If you want more reading material, I've only paged through it myself but Chapra and Canale's book seems like a nice intro text (if it wasn't your textbook already), and uses MATLAB. Reddy has a well-liked intro to finite element methods. Some more graduate level texts are Moin, LeVeque (he has a bunch of good ones), and Trefethen.

Project Euler is indeed great.

I would also recommend you learn some other (any other, really) programming language. MATLAB is a fine tool, but learning something else as well will make you a better programmer and help you be versatile. I don't really recommend you go and learn half a dozen other languages, or even learn every feature available one language--just getting reasonably comfortable with one will do. I'd say pick any of: C, C++, Fortran 90 (or higher), or Python, but there are others as well. Python is probably the easiest to get into and there are lots of packages that will give it a similar "feel" to Matlab, if you like. One nice way of learning (I think) is going through Project Euler in your language of choice.

Slightly more long term, take other numerical/computational courses. As you take them, think about what you like to use computation for (if you don't have a good idea already). If you like to analyze data, develop more or less "simple" simulations to direct design decisions, and don't care so much for heavy simulations, you'll get a better idea of what to look for in industry. If you like physics simulations and solving PDEs, you may lean toward the research end of things and possibly dumping Matlab altogether in favor of more portable and high performance tools.

Thanks for sharing I'll look into that one! Thanks:)

​

Edit: They actually write in that course "The book Numerical Linear Algebra by Trefethen and Bau is recommended." so It might be some further applications!

Not sure if it covers the same topics as Math 110, but this textbook is extremely friendly: https://www.amazon.com/Numerical-Linear-Algebra-Lloyd-Trefethen/dp/0898713617

Depends what you're interested in, but since we're in the ML subreddit it's probably about computation.

Numerical/computational linear algebra studies how to implement the ideas introduced in a 1st LA course on a finite-precision computer.

Linear programming, integer programming, non-linear optimization, and differential equations all heavily rely on linear algebra. The latter two mainly because of Taylor expansions which allow us to approximate functions in terms of linear and quadratic forms.

For ML you're probably best off skimming through the high level ideas in numerical linear algebra, and then diving into linear programming and non-linear optimization.