Reddit reviews Chemical Applications Of Group Theory, 3Rd Ed

https://en.wikipedia.org/wiki/Schur_orthogonality_relations

The concept is that if a representation is reducible, the entire representation can be written as a direct sum of at least two irreducible representations. That particular formula is a consequence of writing the group representation as an h-dimensional vector space where the vectors are the orthogonal, irreducible representations and the vector components are matrix elements.

So in more plain english, that formula comes from orthogonality.

I would also add that this is something you really shouldn't worry about if your goal is any flavor of chemistry. Chemical Applications of Group Theory by Cotton is a book you should have if you're the type of person to ask these kind of questions. It goes way more into the actual math and machinery than what you need, so if something isn't explained in Cotton, it's fair to say that spending time trying to understand it will be a waste of time.

Also, that meaning for n is incorrect. It's usually true, but it's not a given that all vibrations will be either IR or Raman active. Especially when there's high symmetry. Not to mention that this formula is just the reduction formula and only predicts vibrations if you remove the proper 6/5 representations while using the proper coordinates.