This paper gives an overview and initial results for audio-visual scene analysis developed at VUB. An observational system for measuring parent-child interaction in laboratory has been developed. The measurement approach provides an alternative to the paper and pencil measures of trained coders that depend on visualizing the recorded scene and annotating it manually by reporting on parent-child behavior. Direct observation and analysis allows making decisions about the qualities of actual interactive behavior between parents and children, adding uniformity to the observation process and potentially ameliorating systematic bias that may affect human observers. For the long term a system that provides an automatic method to achieve multimedia understanding by incorporating psychological knowledge is studied. In particular the current project combines audiovisual scene analysis with psychological analysis for discovering synchrony in mother-child communication.