(Part 2) Top products from r/linuxadmin

A few of my favorite books I reference and recommend. Just a note, many of these are older and can be purchased used for much less. Also if you can afford it, get a Safari subscription. I use my work Safari subscription but this alone has saved me from my book buying habit. Now I only buy "must have" books. :)

Official Ubuntu Server book - I really like this book as the writing style helped me "get it" with Linux. Kyle Rankin has a very good method of showing you the technology and then a quick run down to get the server working, followed by some admin tips. It's a just scratching the surface type of book but it's enough to get you started. I rarely use Ubuntu now, but this book helped me understand DNS and other topics that were IMHO harder to grasp from other books.

As a bonus, this book also has an entire chapter dedicated to troubleshooting. While this sounds obvious, it's a great read as it talks about dividing the problem, how to approach the facts, etc. Stuff a seasoned admin would know but might be hard to explain to a new admin.

The Linux Command Line: A Complete Introduction - You can read this book for free on the site, but having a paper copy is nice. As mentioned, you should have a very solid understanding of the command line. In my experience, I have seen co-workers struggle with basic shell scripting and even understanding how to make a single line for loop. This book covers the basics, moving to shell scripting and I think it's a good solid reference guide as well.

DevOps Troubleshooting: Linux Server Best Practices - This book is referenced a few times here but I'll throw another comment for it. Another book from Kyle Rankin and has the same straight to the point writing style. It's very quick reference and simple enough that you could give this to a new sysadmin and he or she could get started with some of the basic tools. While the book covers a good selection of basic services and tools, it's easy to get deeper into a chapter and find it's only discussing a handful of troubleshooting steps. The idea with this book is it's a quick reference guide, and if you want to get deeper into troubleshooting or performance, take a look at other books on the market. Either way, this is a great book I keep on my desk or reference through Safari.

UNIX and Linux System Administration Handbook (4th Edition) - Another popular book based on the comments here. This is a pretty big book, thin pages, but it's like a small brick of UNIX/Linux knowledge. While it's starting to get dated, it does give a great reference to many topics in the system administration world. The chapters can dive deep into the subject and offer more than enough information to get started but also understand the technology. The e-mail chapter I thought was great as well as the DNS. I think of this book as a overall guide and if I want to know more, I would read a book just on the subject, that's if I need more information. One thing to point out is this book makes use of different OS's so it's filled with references to Solaris, different UNIX versions, etc. Not a problem but just keep in mind the author may be talking about something outside the scope of vanilla Linux.

Shell Scripting: Expert Recipes for Linux, Bash and more - I found this book to be a good extenstion of the Linux Command Line book, but there are many many other Bash/Shell scripting books out there. The author has many of the topics discussed on his site but the book is a good reference for scripting. I can't stress enough how important shell scripting is. While it's good to know a more formal language like Python/Perl/etc, you are almost certain bash will be on the machine you are working on.

Systems Performance: Enterprise and the Cloud - I can't comment on this book beyond the first chapter, I'm still reading it now but it's reading similar to Brendan Gregg's site, and that's a great thing. If you don't know who this guy is, he's one of the top performance guys in the Solaris and now Linux world. He has some great infographics on his site, which I use for reference.

Use method for Linux

Linux Performance

Example of Linux performance monitoring tools

Hope this helps!

A lot of what has been suggested is great for learning linux. Realize that "out there" very little is served out of a single box (and if it is you're doin it wrong). Production infrastructure likely looks and acts very very differently from your home linux workstation. Just because you know how to type sudo apt-get install apache2 does not mean you are ready for a full ops position... BUT - if you put in the wrench time and pay your dues, you will get there.

Here are some areas that would be good to build your knoweldgebase up in...

• First and foremost - you must build the ability to learn how to figure things out and build an intuition of what to inspect should something not be working. This comes from having a working knowledge of many different systems in a large heterogeneous environment. This will come with experience.
  
• Learn some of the rapid deployment frameworks - cobbler, puppet, cfengine, etc... No one sits around configuring each and every production machine from scratch.
  
• Now that you are familiar with (presumably) the installation and configuration of apache, start thinking about setting up caching/proxy infrastructure. Get a sense for what to use for load balancing v.s. caching v.s. increasing availability (and some combination of the three). Become familiar with things such as nginx, mod_proxy, haproxy, squid, varnish, mongrel, etc...
  
• You MUST know how dns works. Crickets bind and dns should be considered required reading. Any lack of understanding of how dns works is simply unacceptable for a proper sysadmin.
  
• this book is required reading, period.
  
• You must become familiar with centralized authentication mechanisms. Most systems utilize something called PAM. Learn how to configure PAM to reference slapd, AD, etc... Kerberos is our current preferred central authentication mechanism, you need to know how to bounce kerberos tickets around. Get slapd (OpenLDAP) up on its legs.
  
• When running a linux kernel, learn how to configure netfilter. Under linux, Netfilter is the thing responsible for routing, nat, and packet filtering. Understand that other kernels do not use netfilter (or commonly use something else). Become familiar with the common kernels firewall, routing, and forwarding system(s). Don't make the mistake of saying "the iptables firewall..." in the interview room! Iptables is not a firewall.
  
• Know your basic networking. Internet core protocols should be added to your list of required reading. Understand the differences between a hub, bridge, switch, and router. Learn how to "subnet", which means knowing your binary math! I cannot tell you how many times I have seen a messed up network because someone didn't know how to figure out /27 and keyed in the wrong values from a "subnet calculator". Along with networking do a bunch of reading/research on vlans, trunking and stp. Most people cannot tell you what a L2/3 managed switch is or how it differs from a "dumb" switch or router. Don't be one of those people! Learn how to configure routing protocols such as BGP, RIP and OSPF (also, learn basic computational graph theory). You may not end up doing a whole lot of networking, but it's really good stuff to know.
  
• Virtualization is important. You need to know the different forms of virtualization (desktop v.s. os-level v.s. para v.s. hyper virtualization). If you are keen to linux, you need to know how xen and kvm work (this is typically what commercial vps's typically use). Also look at vmware and virtualbox for desktop virt. For os-level virtualization, you need to know how to use LxC and jails.
  
• Learn how LVM works! Spend some time familiarizing yourself with LVM2 (linux), vinum (BSD), and ZFS's container framework (Solaris/BSD). Know how and when to use raid. Make sure you understand the implications of the different raid configurations.
  
• Learn common backup methodology. Raid is not backup, don't make this mistake.
  
• Get used to doing everything on the command line, and always think "what if I had to do this on 20,000 servers?".
  
  So off the top of my head there's a bunch of things you could study. I think that's quite a bit to get your head around, and a deep understanding of some of these topics will only come from working experience. There may be a LOT of work to do in some of those areas. Getting a fully functional xen (or kvm) based system up and on it's legs is not an easy task for the uninitiated. It is my opinion (and everyone else is free to disagree with me) that all good sysadmins/ops/engineers need to "grow up" in some area of lower level technical position. That can be a jr. admin position, the helldesk, or whatever else... This will give you the "systems" working experience that will let you branch into a full fledged admin/op position. Getting some certs under your belt can help you get in the door, but by all means isn't required. Cert's cost money and (the ones worth getting) take time. Personally, I tend to stray away from places that make a big deal out of certs... but that's just me.
  
  tl;dr: Learn how to learn. Pick something you don't know how to do and leverage a linux system to accomplish that goal - rinse and repeat.
  

I have the UNIX and LINUX System Administration Handbook It's awesome and has a pirate boat on the front, so you know it's good. It's great for best practices type stuff, and there's a little bit of sysadmin humor mixed in.

I also have the Linux Command Line and Shell Scripting Bible which is good for CLI reference.

Other than that, you can find a ton of stuff on the web. Is there anything in particular you are looking for?

I recently got a job as a junior admin and found Unix and Linux System Administration to be really good. There is also a nice CBT Nuggets series on Linux which is a great overview, especially when watched while reading LPI Linux Certification in a Nutshell as the book complements the videos.

If he already knows another programming language Dive Into Pythion is great for getting up to speed quickly.

Read "The Linux Command Line" from cover to cover (it's free). You'll easily be a beginner shell scripter by then and probably a better Linux user overall. The book is also an easy read.

From there go into "Unix Shell Programming". If you reduce the gap between the 2 books, the 2nd book will also be an easy (yet somewhat dry) read.

Try to build some small projects while going thru the material. If you use it you'll be more apt to retain the info.

That's what I did. Took me 2 months of reading afterhours of my full time job. By the time the 2 months was up I surprisingly knew more shell scripting than most Linux Admins I've met since. (Admittedly, a lot of people in IT really don't know as much as they try to make it seem.) The only thing missing was experience and projects to use as practice.

Edit: You probably would do really well getting a LinuxAcademy subscription for a year and go thru as much as possible.

thrilljockey's advice is excellent. To add to that, follow the guides at gentoo.org and archlinux.org.

Work through Practical Guide to Linux

This has improved my skills/knowledge in amazing ways.

Honestly, you are never going to find a way to shortcut you out of this situation. No one answer is going to be perfect and get you from A to B if your already at C. I had a similar experience with programming and web development.

I studied computer networking all my adult life and never thought I would be developing as my career at the moment. It is the burden of knowing too much and not having a clear direction. What I needed was more confidence in my skills which can only really develop over the years through experience.

You say you already know a lot of Linux and Bash concepts. CD/CI pipelines try to abstract a lot of OS related involvement since your code doesn’t need to know how low level kernel operations are happening.

What it sounds like you need is knowledge of OS concepts, not just Linux concepts. I say this because every OS has its own way of doing the same thing one way or another.

For example virtual memory, if you understand the concept of virtual memory in any OS rather than a specific OS’s semantics regarding Virtual memory then I think you would be better off in the long run.

If I am wrong and you are the master of the Linux environment, I believe you just need to deep dive into development strategies and the core principles of CD/CI. Once you have a foundation it doesn’t really mater if you are a Jenkins expert or CircleCI expert, all that matters is if you have a foundation to fall back on.

Edit: if you wanted my two cents on material here are some books I recommend.

The Practice of System and Network Administration

Operating Systems Concepts

UNIX and Linux System Administration Handbook

All of these are good recommendations.

Personally, I would recommend http://www.amazon.com/Linux-Bible-Christopher-Negus/dp/1118999878/.

It's very shallow but broad. Good start if you know next to nothing.

Disclaimer, I have zero Ruby experience, I also suck at explaining things.

Cucumber/Aruba appear to be testing frameworks, I have no idea how they work specifically but the general gist of Test Driven Development is you write a test for what the code should do, then you write the code to pass the test (functions as specified), developers write code, run tests, release code (in short).

So assuming they are wanting this formalised by you I am guessing they want some kind of automated testing/continuous integration server. This would essentially be setting up a Jenkins server with a bunch of projects (one per code base) that uses the tests (developer written).

Example:

No CI:

You have a git repo for "project-awesome", (you'll very likely find) developers branch off of master into a feature branch, work on their stuff then merge into master when done, this will then either get packaged up or deployed from git (either manually or by a config management system).

With CI (very basic):

master is the release branch, developers branch off of a development branch do their stuff and push back to the development branch when done, from here Jenkins see the branch has new commits so runs the tests on the code automatically and if they pass merges them into master which then gets deployed, if not it'll fire off an email to whoever committed the code and give them the failed test results.

The aim here is that testing is completely automated which means even if I'm working on feature X but my changes break feature Y it will be caught (in theory) by Jenkins meaning there is a reduced risk of breaking production but pushing bad code.

I just found this page which should hopefully cover some of the general idea better than I can (and it has pictures!), personally I'm a big fan of CI and use it for everything that makes sense (code, scripts, config management states). With that said it's a pretty daunting subject if you have no idea what you're doing (like me most of the time!) and you'll need to do a fair bit of reading and I would heartily recommend this book

Are you using NRPE? If so, it needs to run on every machine you want monitor. I recommend the book Building a Monitoring Infrastructure with Nagios (older, but still a good reference) https://www.amazon.com/dp/0132236931/ref=cm_sw_r_cp_apa_c8S6ybVPBSMDJ

Tools like this show you what the system is doing for the application. If you don't know what applications want to do, the output is going to be hard to understand. Perhaps work on some UNIX programming to see what UNIX looks like from an app's point of view?

http://www.amazon.com/gp/product/0321637739

UNIX Shells by Example by Ellie Quigley. Reviews here.

The Practice of System and Network Administration, Second Edition

UNIX and Linux System Administration Handbook - 4th Edition

[TCP/IP Illustrated, Volume 1: The Protocols - 2nd Edition] (http://www.amazon.com/gp/aw/d/0321336313/)

These should be part of every ops department's library, if not already in your own personal one.

Start with this.

• http://amzn.com/020161622X
  
  Then if you are doing any OOP development then spend some time learning design patterns

This should be your first stop https://www.amazon.com/Complete-Guide-English-Spelling-Rules/dp/0983187215

Yeah.

https://www.amazon.com/Seagate-256MB-Cache-Enterprise-ST12000NM0007/dp/B0759Q9FXZ - 2.22% Annualized Failure rate. $355.99 for 12 TB, 256MB Cache

https://www.amazon.com/d/Internal-Hard-Drives/Toshiba-14TB-SATA-7200RPM-Enterprise/B07DHY61JP - 0.33% Annualized Failure rate. $439.98 for 14 TB, 512MB Cache.

You get what you pay for rings true.

Might not be the best approach, but maybe start with an OS internals book . They're always relevant, even when the actual underlying algorithms of an OS change.

See OS Principles by Silberschatz.

I used to read the linux docs and not understand what half the terminology meant until reading an OS book. Although I used this book which is far more rigorous in terms of theory.

Ah, darksim, we meet again!

As far as a general guide, there's the usual O'Reilly books - Postfix, Spam Assassin, Sendmail if you're a poor SOB who has to administer a Sendmail box, Zimbra (sigh), etc.

If you're using AWS SES as a relay, they have tons of good documentation on their website.

Honestly though, 40% of what I learned was from co-workers who learned it the hard way, 58% was from googling things and maybe 2% was from a reference manual or two we had sitting around. Just implementing email infrastructure does amazing things for your knowledge of it, and figuring out what the hell is going on when your clients call you highly upset because the entire IP block they're in has been blacklisted.