Recent advances in medical imaging technologies have made 4D image sequences available in clinical routine. As a consequence, image registration techniques are evolving from alignment of pairs of static volumetric images to spatio-temporal registration of dynamic (4D) images. Since the elastic image registration problem is ill-posed, additional prior information or constraints are usually required to regularize the problem. This work proposes to enforce local invertibility (diffeomorphism) of 4D deformations. A novel sufficient condition for local invertibility over continuous space and time is proposed and a practical regularization prior is designed from the theory. The method has been applied to an image registration (motion tracking) of a dynamic 4D CT image sequence. Results show that using proposed regularizer leads to deformations that are more plausible for respiratory motion than the standard approach without additional temporal regularization.