Actually, it's even worse than that: the design of the clock makes it so that the cesium atoms doing the actual time keeping are in free-fall while they are being observed. So it is physically impossible for any gravitational influence to change the accuracy of the clock.