Measuring the impact of compression on the reconstruction quality of holograms remains a challenge. A public subjectively-annotated holographic data set that allows for testing the performance of compression techniques and quality metrics is presented, in addition to a subjective visual quality assessment methodology. Moreover, the performance of the quality assessment procedures is compared for holographic, regular 2D and light field displays. For these experiments, a double-stimulus, multi-perspective, multi-depth testing methodology was designed and implemented. Analysis of the quality scores indicated that in the absence of a suitable holographic display and under the presented test conditions, non-holographic displays can be deployed to display numerically reconstructed holograms for visual quality assessment tasks.