First order perturbation theory will give quite accurate answers if the energy shifts calculated are (nonzero and) much smaller than the zeroth order energy differences between eigenstates. If the first order correction is zero, we will go to second order. If the eigenstates are (nearly) degenerate to zeroth order, we will diagonalize the full Hamiltonian using only the (nearly) degenerate states.
Cases in which the Hamiltonian is time dependent will be handled later.
This material is covered in Gasiorowicz Chapter 16, in Cohen-Tannoudji et al. Chapter XI, and in Griffiths Chapters 6 and 7.