On June 13th 2024 at 16:00, Remco Royen will defend their PhD entitled “ADDRESSING LABELLING, COMPLEXITY, LATENCY, AND SCALABILITY IN DEEP LEARNING-BASED PROCESSING OF POINT CLOUDS”.

Everybody is invited to attend the presentation in room I.0.01, or digitally via this link.

In recent years, deep learning has gained widespread use, demonstrating its significance across various domains. Its ability to automatically learn intricate patterns from vast datasets has resulted in a transformative impact, driving advancements in technology, and reshaping the landscape of artificial intelligence applications. The ongoing development of increasingly sophisticated neural network architectures continues to push the boundaries of what is achievable across diverse sectors.

As a result, deep learning has become ubiquitous. However, certain limitations hinder its broad applicability. This thesis delves into four crucial challenges associated with deep learning-based point cloud processing: (i) the precise labeling of extensive datasets, (ii) the model complexity requirements, (iii) the latency introduced during inference, and (iv) the concept of scalability. The initial challenge stems from the necessity for extensive datasets with highly accurate annotations. Particularly in the 3D domain, obtaining such high-quality annotations proves challenging and, consequently, expensive. The second challenged arises from the development of more intricate and memory-intensive, facilitated by advancements in high-power-consuming graphics cards. While these methods achieve higher performance levels, they impose constraints on deployment, particularly for embedded devices. Furthermore, the escalating complexity of these networks is accompanied by an increased inference time, impeding real-time applications. Lastly, deep learning-based solutions lack the concept of scalability which have proven vital in traditional methods.

In this thesis, we tackle these challenges and propose diverse solutions within the deep learning paradigm. The thesis commences with the introduction of a rapid 3D LiDAR simulator, designed to mitigate the labeling problem by learning from perfectly annotated synthetic data. We demonstrate its applications in 3D denoising and semantic segmentation. A second contribution can be found within the domain of point cloud instance segmentation. Through the joint learning of prototypes and coefficients, we present an efficient and rapid method that demands relatively low GPU memory. To further improve our method, we introduce an enhanced block merging algorithm. As a third main contribution, we achieve deep learning-based quality scalability by learning embedded latent representations, demonstrating compelling results in applications such as image reconstruction, point cloud compression, and image semantic hashing. The final contribution introduces resolution-scalable 3D semantic segmentation of point clouds. When applied to resolutionscalable 3D sensors, it enables joint point cloud acquisition and processing.

Our proposed methods consistently outperform established benchmarks across diverse datasets, as demonstrated through comprehensive experimentation. The research findings have been disseminated in various reputable journals and conferences, and have led to a patent submission, highlighting their impact in both academic and industrial contexts.

Communications of the ACM – Holograms on the Horizon 22.12.2021

ETRO and IMEC built-up a reputation in the domain of computer-generated holography. This technology is key to providing high-end media content for holographic display and printing. Recently, ETRO presented a novel approach combining classical raytracing approaches in computer graphics and state-of-the-art computer-generated holography techniques. In an interview in Communications of the ACM, David Blinder discusses (alongside other researchers from MIT and the universities of Stanford and Cambridge) the challenges that are currently being faced.

Links to relevant literature:

• Signal processing challenges for digital holographic video display systems, Signal Processing: Image Communication 70 (2019) 114-130. https://doi.org/10.1016/j.image.2018.09.014
• “Photorealistic computer generated holography with global illumination and path tracing,” Opt. Lett. 46, 2188-2191 (2021). https://doi.org/10.1364/OL.422159

Picture taken in ETRO visual testing lab

On May 25 2022 at 10.30 Mathias Polfliet will defend his PhD entitled “Advances in Groupwise Image Registration”.

Everybody is invited to attend the presentation live (in room Prof. A. Queridozaal, Faculty building Erasmus MC, ‘s Gravendijkwal 230, 3015 CE Rotterdam) or online via this link.

This thesis deals with advances in groupwise image registration. Image registration remains an important task in medical image analysis. Whereas most methods are designed for the registration of two images (pairwise registration), there is an increasing interest in simultaneously aligning more than two images using groupwise registration given the increasing availability of medical imaging data, both at the individual and the population level. Groupwise image registration has shown promise in a number of applications dealing with large quantities of data, among others to increase registration accuracy and robustness, to improve the transformation smoothness and to reduce the methodological bias compared to pairwise registrations. However, directly comparing groupwise registrations to conventional repeated pairwise registrations is difficult due to several confounding factors impacting the algorithm. In this thesis, as a first contribution, we rigorously evaluate two registration methodologies in several experiments and investigate the differences in performance. Secondly, we fill a gap in current literature on efficient (dis)similarity measures for multimodal groupwise image registration. These two contributions are distributed over four chapters.

In Chapter 3, we investigate several registration approaches for the alignment of CT and MRI acquisitions of the mandible in patients with oral squamous cell carcinoma. A comparison is made between rigid and non-rigid approaches with symmetric and asymmetric transformation strategies. The results suggest improved performance in terms of registration accuracy for a symmetric transformation strategy compared to an asymmetric approach, however, the differences were not statistically significant (p=0.054). For this clinical application, we conclude that a rigid registration method is the recommended approach.

In Chapter 4, an investigation is performed on different template images for groupwise registrations based on mutual information. Here, template images are employed as a representative image to compare every image in the group to (in terms of its (dis)similarity). We show that the entropy of the template image can have a counter-intuitive contribution to the global dissimilarity value. Additionally, we show that equivalent performance in terms of registration accuracy can be achieved between groupwise and repeated pairwise approaches.

In Chapter 5, a novel similarity measure is introduced for multimodal groupwise registration. The conditional template entropy measures the negated average of the pairwise conditional entropy of each image of the group and a template image, which is constructed based on principal component analysis. We show improved or equivalent performance in terms of accuracy compared to other state-of-the-art (dis)similarity measures for multimodal groupwise registration and repeated pairwise registration. Furthermore, groupwise registration vastly outperform repeated pairwise registration in terms of transitive error, a measure which can be interpreted as a measure for the consistency of the transformations in a groupwise setting.

In Chapter 6, to further improve on the efficiency of multimodal groupwise registration, we propose a novel dissimilarity measure which is especially adept at registering large groups of images. The dissimilarity measure is formulated as the second smallest eigenvalue of the generalized eigenvalue problem posed in the description of Laplacian eigenmaps. We show little dependence of the measure in terms of computation time with respect to the number of images in the group, and equivalent or improved performance in terms of registration accuracy compared to state-of-the-art groupwise (dis)similarity measures.

To summarize, in this work we evaluate groupwise approaches compared to repeated pairwise approaches and show mostly equivalent performance in terms of registration accuracy and robustness and an improved transitivity for groupwise registration. Furthermore, we recommend to use the proposed dissimilarity measure based on Laplacian eigenmaps for large groups of images given its superior or equivalent registration accuracy compared to other measures but superior scaling in terms of execution time with respect to the number of images in the group.

Evaluate the variations of your blood volume with photoplethysmography – ETRO-VUB

Experiment and discover how light-based devices can provide information about your health. You’ll find out all about photoplethysmography and its biomedical applications. This non-invasive technology provides a lot of information about the human body.

On May 27th 2024 at 16:00, Yifei Da will defend their PhD entitled “DATA-DRIVEN CAUSAL MODELLING FOR DE-BIASING SENTIMENT ANALYSIS MODELS AND MULTIVARIATE STOCK PRICE MOVEMENT PREDICTION”.

Everybody is invited to attend the presentation in room I.0.01, or digitally via this link.

In this thesis, we address the stock price movement prediction problem by investigating the interdependencies between sentiments from financial news and international stock markets in stock forecasting. To provide reliable sentiment analysis results, especially to reduce bias, we studied sentiment analysis methods and detected, evaluated, and mitigated bias that was picked up on and amplified by large pre-trained models. To retrieve intra- and inter-market interdependencies, we adopt the Transfer Entropy theory to detect and incorporate the information flow between financial news sentiment and the dynamics of the stock markets. We contribute to these two sub-tasks by (i) proposing a new method for debiasing sentiment analysis models that leverages the causal mediation analysis to identify the parts of the model primarily responsible for the bias and apply targeted counterfactual training for model debiasing. Furthermore, (ii) a causal-enhanced multi-modality model for multivariate stock price movement prediction is proposed based on establishing an accurate information flow propagation between stocks and sentiments. To repeatedly validate the feasibility, the Dow Jones Industrial Average indexes of 13 countries and daily financial news data from the New York Times are used in stock Price and Return forecasting.