Reddit reviews Data Analysis: A Bayesian Tutorial

• Probability Theory: The Logic of Science
  
• Bayesian Spectrum Analysis and Parameter Estimation
  
• Maximum Entropy in Action
  
• Data Analysis--A Bayesian Tutorial

> statistics was the real killer.

Maybe it's because statistics is usually presented as a bunch of recipes with no unifying principles.

Try "Data Analysis: A Bayesian Tutorial" by Sivia:

http://www.amazon.com/Data-Analysis-Bayesian-Devinderjit-Sivia/dp/0198568320

Learning statistics without Bayes' theorem is like trying to learn mechanics without Newton's laws.

I recommend "Data Analysis: A Bayesian Tutorial". It's pretty short and easy to read and has examples and pseudocode for many of the discussed methods. Use whatever programming language you're most comfortable with (MATLAB does have nice built in functions for dealing with large matrices). Depending on the amount of data, I'd avoid excel and just load ASCII data files from your code if possible.

You do need statistics, but if you have a physics degree, you should be able to pick up the necessary statistics fairly quickly. I would recommend going through introductory statistics homework assignments to learn the very basics.

I'd also heavily recommend learning Bayesian statistics and understanding where the loss functions actually come from (ie why do we minimize the sum of squared errors in linear regression). The best book on introductory Bayesian statistics I've read was Data Analysis: A Bayesian tutorial.

It's aimed at physicists, but [Sivia's book] (http://www.amazon.com/Data-Analysis-Bayesian-Devinderjit-Sivia/dp/0198568320) is extremely good.

Otherwise as actual papers http://bayes.wustl.edu/sivia/how.many.lines.pdf or (ahem) something of my own.

Frequentist statistics does use Bayes' theorem, all of the measure theoretic results are identical between the philosophies. It is the inclusion of a priori knowledge (or information attempting to express a lack thereof) that demarcates the primary modeling differences.

If you would like a solid background in bayesian statistics I would recommend BDA3 by Andrew Gelman and Machine Learning: A Probabilistic Perspective by Kevin Murphy

One can of course not forget Hastie et al.'s Elements of Statistical Learning as well.

If you would like a general introduction, however, I would recommend the following text by Sivia.

Probability theory itself is consistently axiomatized under the Komolgorov axioms. But the philosophy regarding how to perform inference is not.

There is not an obvious inconsistency in the mathematical formulations, but there are inconsistencies with how each of the philosophies treats various issues.

A brief overview of differences is here:
http://www.stat.ufl.edu/archived/casella/Talks/BayesRefresher.pdf

In short though, there is nothing mathematically wrong with the Frequentist approach--but I would personally argue there are things that are philosophically wrong with certain applications of those methods--not the least of which are issues where generating processes are non-stationary (though similar issues can be stated for Bayesians) or where simply the formulation tends to lead practitioners to drawing mistaken conclusions by mistake. You can make a Rube Goldberg machine of computation and still have it preserve all information and be mathematically consistent, but the likelihood of humans misinterpreting it is much higher than a simpler framework.

Data analysis: a Bayesian tutorial is really nice. It starts off with continuous parameter estimation and then moves on to model selection. Unlike Peter Lee's book it feels like a clean break from classical stats.

Another book is D.S. Sivia's Data Analysis: A Bayesian Tutorial. It's more expensive than when I first got it, though (sorry I don't have a free reference). The examples in the beginning of the book are easily done in PyMC, as well!

This one is a quite good book: Sivia, Skilling - Data Analysis: A Bayesian Tutorial

It's quite pragmatical, as opposed to the forementioned Jaynes for instance.

i can recommend this book:

http://www.amazon.com/Data-Analysis-Bayesian-Devinderjit-Sivia/dp/0198568320/ref=pd_sim_b_4

it doesn't cover bayes networks, but it explains the bayes theorem and shows how it can be used.

Well, this is aimed at senior undergraduates and research students in science and engineering: http://www.amazon.com/Data-Analysis-A-Bayesian-Tutorial/dp/0198568320

But really everything you learn should result in you realizing what else you want to learn related to it. The same is true for statistics. Learn some, and that should let you think... oh I wonder if I can find any publications on....

Feigelson & Babu is a great read, with lots of applications using R.

If you're looking for something a bit more "cookbook" style then this book is good. The authors also have the solutions to all the problems on their website.

For general statistics and data analysis (without a focus on astrophysics) Sivia & Skilling is also good.

Get serious about statistics, it's a huge part of astrophysics. This is tricky, because stats / probability is often taught from a terrible, unintuitive, seemingly non-rigorous approach where you learn various prescriptions for different scenarios, like a cookbook.

I've found this text to be perhaps the most broadly useful thing when it comes to really understanding data analysis: http://www.amazon.com/Data-Analysis-A-Bayesian-Tutorial/dp/0198568320