If you assume the Death Star does indeed have to orbit Yavin (rather than just scooting around under its own power, as it obviously is capable of doing), then the Imperial officer has a point. To orbit faster without flying off into interplanetary space, you have to orbit lower. And below a certain altitude, called the Roche limit, tidal forces generated by the primary object (Yavin in this case) will tend to rip the orbiting body apart.

And yes, the structural strength of the Death Star means it can orbit a bit below the idealised Roche limit without flying apart, but the officer has already stated that they are orbiting at maximum velocity, which implies that they are right at the actual limit below which the Death Star would be threatened.