The goal of this project is to build a semantically rich, dense 3D model of street scenes in city centers. Such a model will probably be a key component for the next generation GPS systems, providing a more intuitive user interface and/or additional information. automatic extraction of such models is crucial if one wants to keep the system up to date without having to employ an army of image annotators. Most earlier approaches to 3D reconstruction have worked solely on low level image data, finding correspondences and backprojecting these into 3D. In contrast, we believe that in order to obtain high quality models, higher level knowledge can best be incorporated into the 3D reconstruction process from the very start, i.e. information of what the image actually represents needs to be extracted in combination. For instance, detecting cars or pedestrians allows to remove these occluding obkects and their textures from the 3D reconstructions and to focus attention on the relevant building behind. Also, the presence of cars increases the probability that that part of the scene contains a road, pedestrians tend to walk on the walkway, etc. On the other hand, as soon as 3D oinformation is becoming available, this is in return helpful in the interpretation of the scen content, as probabilities to find objects at different locations in the scene depend on the geometry, e.g. whether there is supporting horizontal plane available, like a road or a wolkway. Traffic signs are important patters and they can be expected to be found within a range of heights, next to the road. Similarly, the sky, which typically gives erroneous reconstructions either because it is featureless or because it contains moving clouds, can be detected and directly projected to infinity.

Runtime: 2007 - 2010