Deep neural networks (DNNs) have performed remarkably in various computer vision tasks. Considering the great success of DNNs and their inspiration from biological vision, it is interesting to explore the extent to which these frameworks can reflect the brain responses in the visual cortex. In this paper, we first evaluate the effectiveness of different DNN models in predicting brain neural activities when observing natural images. Next, we examined how the brain-like performance of deep features degrades under image distortion and whether such degradation is aligned with human quality preferences. The outcomes reveal that although DNN models are still far from optimal in modeling brain responses, some models can represent a hierarchical data processing scheme in accordance with the visual cortex. In addition, deep features showed good consistency with the human opinion on the degradation impact of different distortion types, making them good candidates for the design of biologically-inspired image quality assessment models.
Mahmoudpour, S & Schelkens, P 2023, On the Agreement of Deep Neural Networks with the Brain in Encoding Visual Stimuli: Implications for Image Quality Assessment. in 2023 24th International Conference on Digital Signal Processing, DSP 2023. International Conference on Digital Signal Processing, DSP, vol. 2023-June, IEEE, pp. 1-5, 24th International Conference on Digital Signal Processing, Island of Rhodes, Greece, 11/06/23. https://doi.org/10.1109/DSP58604.2023.10167939
Mahmoudpour, S., & Schelkens, P. (2023). On the Agreement of Deep Neural Networks with the Brain in Encoding Visual Stimuli: Implications for Image Quality Assessment. In 2023 24th International Conference on Digital Signal Processing, DSP 2023 (pp. 1-5). (International Conference on Digital Signal Processing, DSP; Vol. 2023-June). IEEE. https://doi.org/10.1109/DSP58604.2023.10167939
@inproceedings{c86a40f459264e69b5c51a7dc1e45c5a,
title = "On the Agreement of Deep Neural Networks with the Brain in Encoding Visual Stimuli: Implications for Image Quality Assessment",
abstract = "Deep neural networks (DNNs) have performed remarkably in various computer vision tasks. Considering the great success of DNNs and their inspiration from biological vision, it is interesting to explore the extent to which these frameworks can reflect the brain responses in the visual cortex. In this paper, we first evaluate the effectiveness of different DNN models in predicting brain neural activities when observing natural images. Next, we examined how the brain-like performance of deep features degrades under image distortion and whether such degradation is aligned with human quality preferences. The outcomes reveal that although DNN models are still far from optimal in modeling brain responses, some models can represent a hierarchical data processing scheme in accordance with the visual cortex. In addition, deep features showed good consistency with the human opinion on the degradation impact of different distortion types, making them good candidates for the design of biologically-inspired image quality assessment models.",
author = "Saeed Mahmoudpour and Peter Schelkens",
note = "Funding Information: The research was funded by Research Foundation – Flanders (FWO) (G0B3521N) Publisher Copyright: {\textcopyright} 2023 IEEE.; 24th International Conference on Digital Signal Processing, 24th DSP 2023 ; Conference date: 11-06-2023 Through 13-06-2023",
year = "2023",
doi = "10.1109/DSP58604.2023.10167939",
language = "English",
series = "International Conference on Digital Signal Processing, DSP",
publisher = "IEEE",
pages = "1--5",
booktitle = "2023 24th International Conference on Digital Signal Processing, DSP 2023",
url = "https://2023.ic-dsp.org/",
}