Computer-generated holography is computationally intensive, making it especially challenging for holographic displays where high-resolutions and video rates are needed. To this end, we propose a technique for directly calculating short-time Fourier transform coefficients without the need for a look-up table. Because point spread functions are sparse in this transform domain, only a small fraction of the coefficients need to be updated, enabling significant speed gains. Twenty-fold accelerations are reported over the reference implementation. This approach generalizes the notion of the phase-added stereogram, allowing for the calculation of an arbitrary number of Fourier coefficients per block, enabling high calculation speed with holograms of good visual quality, targeting minimal memory requirements.