Reddit reviews Theoretical Astrophysics (Theoretical Astrophysics (Paperback))

If you want a post-graduate level of understanding, it will be hard to learn the math past calculus that you will need with no instruction. Maybe impossible unless you are very gifted or studious. You'll need to learn more advanced math (taylor expansions, more advanced integration methods not always taught in calc I, multi-variable calculus, ordinary differential equations and linear algebra for starters). A layperson's understanding wouldn't require that much (maybe reading Sagan and Co. would be enough?), but it sounds like you aren't content with that. Maybe it would be good to start reading some journal articles and seeing what you can glean from them (introductions mostly), especially reviews of subjects you find interesting. If those are opaque, check a local university library for textbooks like Introduction to Modern Astrophysics, Padmanabhan's astrophysics I-III, Binney and Tremaine and things like that. There are text books more focused on specific subjects as well, but that's more a matter of personal interest. For me, Lewin and van der Klis is good, and so is Accretion Power in Astrophysics and the "CV Bible." You might notice Cambridge Astrophysics publishes quite a lot of quality astrophysics textbooks.

None of those are going to be legible without the math, though. There's not really anything between the "popular science" and "so you're taking a graduate course in astrophysics..." level texts that I've seen.