We have already solved many problems in Quantum Mechanics using wavefunctions and differential operators. Since the eigenfunctions of Hermitian operators are orthogonal (and we normalize them) we can now use the standard linear algebra to solve quantum problems with vectors and matrices. To include the spin of electrons and nuclei in our discussion of atomic energy levels, we will need the matrix representation.
These topics are covered at very different levels in Gasiorowicz Chapter 14, Griffiths Chapters 3, 4 and, more rigorously, in Cohen-Tannoudji et al. Chapters II, IV and IX.