Below you will find frequently asked questions, divided over four different groups. First, a generic FAQ with information applying to a broad set of master degrees and then more specific FAQs applying to specific programs only.
“Signal Processing in the AI era” was the tagline of this year’s IEEE International Conference on Acoustics, Speech and Signal Processing, taking place in Rhodes, Greece.
In this context, Brent de Weerdt, Xiangyu Yang, Boris Joukovsky, Alex Stergiou and Nikos Deligiannis presented ETRO’s research during poster sessions and oral presentations, with novel ways to process and understand graph, video, and audio data. Nikos Deligiannis chaired a session on Graph Deep Learning, attended the IEEE T-IP Editorial Board Meeting, and had the opportunity to meet with collaborators from the VUB-Duke-Ugent-UCL joint lab.
Featured articles:
On February 5th 2025 at 16:00, Raees Kizhakkumkara Muhamad will defend their PhD entitled “COMPRESSION STRATEGIES FOR DIGITAL HOLOGRAPHY”.
Everybody is invited to attend the presentation in room I.0.01 or online via this link.
Holographic techniques sample a 2D interference pattern produced by coherent light waves reflected or transmitted from the different objects in the 3D scene. As a display technology, it provides all necessary visual cues for perceiving the scene by the brain without causing mismatches between accommodation and vergence of the eyes. Non destructive imaging with high resolution for biomedical and industrial inspection also utilizes holographic principles. Holographic microscopes are realizable with optically simpler setups than regular microscopy, opening new pathways for computational microscopy. Utilizing more complex arrangements, such as holographic tomography, allows for reconstructing the 3D refractive index profile of transmissive objects, resolving even sub-cellular structures with visible light. It represents the culmination of humanity’s effort to record and represent light information.
However, sampling interference patterns for high-end displays or highresolution microscopy result in an extensive digital footprint. Historically, for many multimedia use cases, the data transmission bottleneck dictated the fidelity of the consumed content, and one can expect holography to be no different. Compression algorithms can help mitigate the data load, trading off more computation for an effective increase in transmission capacity. The algorithms must be tailored for holograms used in practice and exhibit a computational complexity appropriate for the use case, particularly on the decoder side. This thesis presents compression strategies for effectively tackling such use cases for holography.
The performance of conventional image compression tools on metrological hologram content is first studied. We provide a novel analysis of compression artifacts on the retrieved metrological data obtained for digital holographic microscopy and tomography. First-generation holographic displays are poised to use binary representation due to difficulties in modulating pixels at the sizes required by holography. Context-based compression is adequate for lossless and near-lossless compression of such data. Here, one extracts from a pixel the redundant information from previously decoded neighbouring pixels using a generalized Markovian model. Another context-based framework utilizing linear (autoregressive) models is used to design a highly scalable lossless compression scheme for non-binary holographic data. For broadcast scenarios, it is desirable to have a compression system that can support view-selective decoding to minimize the transmission of unutilized information. For this purpose, we propose a short-time Fourier transform (STFT) based codec, which slices the hologram into independently decoded, spatio-angular chunks. Given a target mean-squared error, the optimization techniques efficiently round down the signal in the STFT domain by application of adaptive quantizers. All these compression schemes are applicable for a single frame of hologram data and feature lightweight decoding architectures while surpassing compression performances achieved by any existing solution on most tested holograms. To compress holographic videos with arbitrary motion, we utilize a novel motion compensation algorithm that can predict rotational motion in conjunction with the above-mentioned STFT framework. The Markovian and STFT frameworks discussed in this work have been adopted as part of the first international hologram compression standard, JPEG Pleno — Part 5: Holography (ISO/IEC 21794-5)
ETRO, the Vrije Universiteit Brussel (VUB) and imec are proud to announce that Prof. Nikos Deligiannis has been awarded a prestigious ERC Consolidator Grant from the European Research Council to make a groundbreaking contribution to science and society.
Project: IONIAN: Reinventing Multiterminal Coding for Intelligent Machines
Budget: €1,999,404
Professor Deligiannis’ IONIAN project focuses on reinventing multiterminal coding, a crucial technology for efficient communication and collaboration between intelligent machines. With the explosive growth of data, such as video and point cloud streams, current storage and communication technologies are under pressure, undermining the ability of intelligent machines to cooperatively perceive their environment. This project develops a groundbreaking compression and communication approach based on interpretable and explainable AI that breaks the limits of traditional compression and cooperative perception techniques.
IONIAN combines classical theories, such as distributed source coding, with modern deep learning techniques and explainable AI, focusing on three innovative pillars:
The goal of this project is to elevate the collaboration between intelligent systems, such as autonomous vehicles and mobile robots, to a higher level, with greater safety and trust as the result.
Remote editing with VS Code
Visual Studio Code is a widely-used, cross-platform Integrated Development Environment (IDE) that supports numerous programming languages and offers a vast array of extensions to enhance its functionality.
One notable extension enables development on remote machines via SSH, providing integrated access to a file explorer, terminal, and text editor on the configured remote system. This makes it a strong alternative to JupyterLab as a remote editor for e.g., the ETROFARM Slurm cluster.
Open the Extensions tab using the corresponding icon in the left toolbar.
Search for “SSH” and subsequently select the topmost “Remote – SSH” extension. Install this extension.
Depending on your programming language of choice, you might also be interested in extensions such as “Python”, “Python Debugger”, “Ruff” (a Python code linter) etc.
With the “Remote – SSH” extension installed, a new tab “Remote explorer” has been added to the left toolbar.
Add a new remote by pressing the + icon.
When asked for the SSH Connection Command:
ssh <username>@etroflock.etrovub.be
Secondly, it will ask where to store this information. This can be the default option.
The etroflock.etrovub.be remote has been created. Time to connect by triggering one of the 2 corresponding buttons.
During a brief instant the option will be displayed to edit the configuration. If you have missed it you can find this file via the gear button next to remote explorer – remote tunnels – ssh
Your config file should look like
Host etroflock.etrovub.be
HostName etroflock.etrovub.be
User jdoe
IdentityFile C:\users\jdoe\.ssh\id_rsa
If you have no experience with encryption you can e.g. use a rsa 2048 type of key. Please make sure you are using a private key in openssh format.
Upon our first connection attempt, it requests the platform of the remote host, being the Slurm cluster’s login node we are connecting to. This is a Linux machine.
It will also ask to confirm the SSH public key credential of the server.
We are now connected to the remote server. This can be seen in the Remote explorer tab as well as in the left corner of the bottom toolbar.
Time to open our file explorer via the “Explorer” tab in the left toolbar (Ctrl + Shift + E). Press the “Open Folder” button. It should by default suggest to open your home folder on the Slurm cluster (currently on /FARM/<username>).
If prompted, confirm that the remote server is (again) a Linux platform. Lastly, confirm that you trust the authors of the files in this folder as this is your own home folder.
Congratulations! Your remote file explorer and text editor on the Slurm cluster is now operational.
A remote terminal can be opened using Terminal -> New Terminal in the top toolbar, or via the Ctrl + Shift + ` shortcut.
An interactive terminal session is opened on the remote host as if it was a PuTTY (or other) SSH session.
With the remote file explorer, remote text editor and remote terminal sessions available, it is a logical next step to focus on running our code remotely on the machine. Luckily, this is typically as straightforward as pressing the “Run file” button on the active (Python) file.
We can observe that the code has indeed been executed on the remote machine. However, the configured remote machine is ETROflock, ETRO’s Slurm cluster’s login server that is scarce in compute resources and lacks and GPU’s.
Running our code on a Slurm compute node is more complicated as it involves requesting a Slurm job. This is currently a manual process but we are investigating if this can be automated in VScode by using a custom launch script.
For now, there are two possibilities to run code on the Slurm cluster from within VScode:
A Slurm job can be requested that immediately runs the code until completion (or an error or timeout). This is the recommended default approach for running Slurm jobs.
E.g. the same test.py code is run as a Slurm job by using the srun command.
We observe that the first command is run on ETROFLOCK (the Slurm login node) because that is immediately launched on the remote system. The second command is scheduled as a Slurm job and is run on ETROFARM (a Slurm compute node).
The second possibility involves starting a Slurm job with an interactive shell. Once this interactive shell is running on a compute node, we can manually launch the desired code within that shell. This solution might be preferred when developing and testing the functionality of the code as a Slurm job (and potential queue) must only be requested once per session.
Launching a Slurm job with interactive shell is possible using the following srun command parameters:
srun –pty bash -i
E.g. the same test.py code is run after an interactive shell has been requested using the srun command and has started.
Observe that after requesting the slurm job with interactive terminal we change from “steffen@ETROFLOCK” to “steffen@ETROFARM”. This indicates our interactive shell is indeed running on a compute node instead of the login node.
Executing the test.py script from within this shell again confirms that the code is indeed run on the compute node with hostname ETROFARM.
When finished, one should use the “exit” command. This closes the interactive shell and terminates the Slurm job, thereby releasing the allocated resources for new jobs.
Filezilla is a free and open-source file transfer tool that can be used to exchange files with e.g., ETROFARM Slurm cluster. It requires some brief configuration in order to be able to connect using the SSH public/private key credentials. The required configuration steps are provided in the following tutorial.
Open the Site Manager, via the leftmost icon in the toolbar. Contrary to using the Quickconnect function which only supports connecting via a password, the Site Manager also allows configuring SFTP authentication via SSH keys.
Add a New site with the following configuration:
Connect to this SFTP server and trust the etroflock’s public SSH key.
All future FileZilla SFTP connections to ETROflock can be easily launched from the Site manager tab.
On November 15th 2024 at 10:00, Eden Teshome Hunde will defend their PhD entitled “CROSS-LAYER DESIGN, IMPLEMENTATION AND EVALUATION OF IPV6 MULTICAST FOR RADIO DUTY CYCLED WIRELESS SENSOR AND ACTUATOR NETWORKS”.
Everybody is invited to attend the presentation in room D.2.01 or online via this link.
In this work, we study Bidirectional Multicast RPL Forwarding (BMRF) as this protocol relies on forwarding tables put in place by the well-known Routing Protocol for Low Power and Lossy Networks (RPL) and allows to combine the best ideas of existing multicast protocols. Through RPL, a routing tree towards the sink is installed for multihop routing from node to sink, and the nodes’ forwarding tables will also contain entries for reaching destinations in downward direction.
For downward forwarding IPv6 multicast packets, two methods exist. One is via link layer (LL), broadcasting a frame containing the IPv6 multicast packet. The other is to send several LL unicast frames containing that packet. BMRF allows a node to choose between these two methods. The best option will depend on the presence of a radio duty cycling (RDC) protocol. RDC is part of the medium access control (MAC) layer and puts the radio to sleep when no communication is needed. We investigate the influence of MAC/RDC protocols on BMRF’s performance.
We evaluate the performance of BMRF on non-synchronized WSANs that use Carrier Sense Multiple Access (CSMA) as MAC and ContikiMAC as RDC. We demonstrate that LL unicast outperforms LL broadcast in terms of packet delivery ratio (PDR), delay, and energy consumption in many settings.
We investigate the performance of BMRF on WSANs with synchronous MAC and RDC based on Time Slotted Channel hopping (TSCH). This is more challenging, as TSCH needs a schedule to tell which action must happen in each timeslot. The actions can be to send or to listen on a given channel or to be idle. Idleness allows the radio to switch OFF, providing RDC. The schedule is not part of the standard and must be proposed by the system designer. An elegant autonomous scheduling method called Orchestra is available to accommodate traffic in a RPL tree. We extend Orchestra with a novel scheduling rule for supporting LL downwards forwarding through LL broadcast. Comparing LL unicast with LL broadcast forwarding teaches us that LL unicast outperforms LL broadcast in terms of packet delivery ratio (PDR), but the latter can be beneficial to certain applications, especially those sensitive to delay.
Before conducting the two previous evaluation studies, we investigate the performance of simple convergecast traffic while considering ContikiMAC and TSCH with Orchestra under RPL on the real dual Zolertia Firefly Motes (one is observed and other one is observing mote). This study served two purposes; it reminds the reader of the characteristics of those protocols and allowed to fine-tune the dual motes.
We also contributed by adapting the Orchestra to bursty convergecast traffic. Simulation results demonstrate that the new scheduler slightly improves PDR and reduces delay compared to state-of-the-art solutions.
On November 7th 2024 at 16:00, Boris Joukovsky will defend their PhD entitled “ SIGNAL PROCESSING MEETS DEEP LEARNING: INTERPRETABLE AND EXPLAINABLE NEURAL NETWORKS FOR VIDEO ANALYSIS, SEQUENCE MODELING AND COMPRESSION”.
Everybody is invited to attend the presentation in room I.0.01 or online via this link.
There is growing use of deep learning for solving signal processing tasks, and deep neural networks (DNNs) often outperform traditional methods little domain knowledge needed. However, DNNs behave as black boxes, making it difficult to understand their decisions. The empirical approaches to design DNNs often lack theoretical guarantees and create high computational requirements, which poses risks for applications requiring trustworthy artificial intelligence (AI). This thesis addresses these issues, focusing on video processing and sequential problems across three domains: (1) efficient, model-based DNN designs, (2) generalization analysis and information-theory-driven learning, and (3) post-hoc explainability.
The first contributions consist of new deep learning models for successive frame reconstruction, foreground-background separation, and moving object detection in video. These models are based on the deep unfolding method, a hybrid approach that combines deep learning with optimization techniques, leveraging low-complexity prior knowledge of the data. The resulting networks require fewer parameters than standard DNNs. They outperform DNNs of comparable size, large semantic-based convolutional networks, as well the underlying non-learned optimization methods.
The second area focuses on the theoretical generalization of deep unfolding models. The generalization error of reweighted-RNN (the model that performs video reconstruction) is characterized using Rademacher complexity analysis. This is a first-of-its-kind result that bridges machine learning theory with deep unfolding RNNs.
Another contribution in this area aims to learn optimally compressed, quality-scalable representations of distributed signals: a scheme traditionally known as Wyner-Ziv coding (WZC). The proposed method shows that deep models can retrieve layered binning solutions akin to optimal WZC, which is promising to learn constructive coding schemes for distributed applications.
The third area introduces InteractionLIME, an algorithm to explain how deep models learn multi-view or multi-modal representations. It is the first model-agnostic explanation method design to identify the important feature pairs across inputs that affect the prediction. Experimental results demonstrate its effectiveness on contrastive vision and language models.
In conclusion, this thesis addresses important challenges in making deep learning models more interpretable, efficient, and theoretically grounded, particularly for video processing and sequential data, thereby contributing to the development of more trustworthy AI systems.
HealthTech TouchPoints event (17/10/2024): VUB and UZ Brussel showcased their HealthTech expertise to companies.
ETRO did a pitch and had a demo booth at the matchmaking fair after the event.
More background info LinkedIn posts:
Benyameen Keelson and Pieter Boonen sucessfully finished the LifeTech.brussels MedTech accelerator with their startup projects PADFLOW en KARMA.
Some extra infomation can be found here.
The introductory movies for the projects:
Benyameen:
Pieter:
On October 25th 2024 at 16:00, Yuqing Yang will defend their PhD entitled “CRAFTING EFFECTIVE VISUAL EXPLANATIONS BY ATTRIBUTING THE IMPACT OF DATASETS, ARCHITECTURES AND DATA COMPRESSION TECHNIQUES”.
Everybody is invited to attend the presentation in room D.2.01 or online via this link.
Explainable Artificial Intelligence (XAI) plays an important role in modern AI research, motivated by the desire for transparency and interpretability within AI-driven decision-making. As AI systems become more advanced and complicated, it becomes increasingly important to ensure they are reliable, responsible, and ethical. These imperatives are particularly acute in domains where stakes are high, such as medical diagnostics, autonomous driving, and security frameworks.
In computer vision, XAI aims to provide understandable, straightforward explanations for AI model predictions, allowing users to grasp the decision-making processes of these complex systems. Visualizations such as saliency maps are frequently employed to identify input data regions significantly impacting model predictions, thus enhancing user understanding of AI visual data analysis. However, there are still concerns about the effectiveness of visual explanations, especially regarding their robustness, trustworthiness, and human-friendliness.
Our research aims to advance this field by evaluating how various factors—such as the diversity of datasets, the architecture of models, and techniques for data compression—influence the effectiveness of visual explanations in AI applications. Through thorough analysis and careful refinement, we strive to enhance these explanations, ensuring they are both highly informative and accessible to users in diverse XAI applications.
During our evaluation process, we conduct a detailed investigation using both automatic metrics and subjective evaluation methods to assess the effectiveness of visual explanations thoroughly. Automatic metrics, such as task performance and localization accuracy, provide quantifiable measures of the effectiveness of these explanations in real-world scenarios. For subjective evaluation, we have developed a framework named SNIPPET, which enables a detailed and user-oriented assessment of visual explanations. Additionally, our research explores how these objective metrics correlate with subjective human judgments, aiming to integrate quantitative data with the more nuanced, qualitative feedback from users. Ultimately, our goal is to provide comprehensive insights into the practical aspects of XAI methodologies, particularly focusing on their implementation in the field of computer vision.
A BIG high Five to Loris Giordano, Franjo Mikic, Anastasia Koutalianou, Jonathan Vrijsen and Sevada Sahakian. All five have obtained the prestigious Predoctoral Mandate for Strategic Basic Research for the coming 4 years.
We are VERY proud of you!
On October 15th 2024 at 16:00, Xiangyu Yang will defend their PhD entitled “Leveraging Deep Learning Models for Big Data Analytics”.
Everybody is invited to attend the presentation in room D.0.05 or online via this link.
With the exponential growth of data generated daily from social media, e-commerce, and various digital interactions, the necessity to effectively harness and leverage this vast expanse of information is more critical than ever. In this context, Deep Learning (DL), a subfield of Artificial Intelligence (AI), has emerged as a transformative force, delivering unparalleled capacities in pattern recognition, data analysis, and predictive modeling. Deep learning takes large amounts of available data as fuel to train itself, and significantly impacts various fields ranging from healthcare to finance, enabling advanced applications in natural language processing (NLP), computer vision (CV), and recommender systems (RS).
This thesis delves into the essential role of AI in leveraging big data, focusing on information extraction from social media, deep learning model explainability, and the development of explainable recommender systems. With the vast, ever-growing volume of data, extracting meaningful insights from unstructured social media becomes increasingly complex, necessitating cutting-edge AI solutions. Concurrently, the reliance on deep learning models for critical decisions brings explainability to the forefront, emphasizing the importance of developing transparent methods that ensure user trust. Furthermore, the demand for recommender systems that provide understandable textual explanations has surged, highlighting the need for explainable systems that align with user preferences and decision-making processes.
This thesis advances the field through three key contributions. Initially, we establish two traffic-related datasets from social media, annotated for comprehensive traffic event detection. Employing BERT-based models, we tackle this detection problem via text classification and slot filling, proving these models’ efficacy in parsing social media for traffic-related information. Our second contribution intro- duces LRP-based methods to explain deep conditional random fields, with successful applications in fake news detection and image segmentation. Lastly, we present an innovative personalized explainable recommender system that integrates user and item context into a language model, producing textual explanations that enhance system transparency.
On October 9th 2024 at 16:30, Esther Rodrigo Bonet will defend their PhD entitled “EXPLAINABLE AND PHYSICS-GUIDED GRAPH DEEP LEARNING FOR AIR POLLUTION MODELLING”.
Everybody is invited to attend the presentation in room I.0.02.
Air pollution has become a worldwide concern due to its negative impact on the population’s health and well-being. To mitigate its effects, it is essential to monitor pollutant concentrations across regions and time accurately. Traditional solutions rely on physics-driven approaches, leveraging particle motion equations to predict pollutants’ shifts in time. Despite being reliable and easy to interpret, they are computationally expensive and require background domain knowledge. Alternatively, recent works have shown that data-driven approaches, especially deep learning models, significantly reduce the computational expense and provide accurate predictions; yet, at the cost of massive data and storage requirements and lower interpretability.
This PhD research develops innovative air pollution monitoring solutions focusing on high accuracy, manageable complexity, and high interpretability. To this end, the research proposes various graph-based deep learning solutions focusing on two key aspects, namely, physics-guided deep learning and explainability.
First, as there exist correlations among the data points in smart city data, we propose exploiting them using graph-based deep learning techniques. Specifically, we leverage generative models that have proven efficient in data generation tasks, namely, variational graph autoencoders. The proposed models employ graph convolutional operations and data fusion techniques to leverage the graph structure and the multi-modality of the data at hand. Additionally, we design physics-guided deep-learning models that follow well-studied physical equations. By updating the graph convolution operator of graph convolutional networks to leverage the physics convection-diffusion equation, we can physically guide the learning curve of our network.
The second key point relates to explainability. Specifically, we design novel explainability techniques for interpretable graph deep modeling. We explore existing explainability algorithms, including Lasso and a layer-wise relevance propagation approach, and go beyond them to our graph-based architectures, designing efficient and specifically tailored explanation tools. Our explanation techniques are able to provide insights and visualizations based on various input data sources.
Overall, the research has produced state-of-the-art models that combine the best of both (physics-guided) graph-deep-learning-based and explainable approaches for inferring, predicting, and explaining air pollution. The developed techniques can also be applied to various applications in modeling graphs on the Internet such as in recommender systems’ applications.
“Signal Processing in the AI era” was the tagline of this year’s IEEE International Conference on Acoustics, Speech and Signal Processing, taking place in Rhodes, Greece.
In this context, Brent de Weerdt, Xiangyu Yang, Boris Joukovsky, Alex Stergiou and Nikos Deligiannis presented ETRO’s research during poster sessions and oral presentations, with novel ways to process and understand graph, video, and audio data. Nikos Deligiannis chaired a session on Graph Deep Learning, attended the IEEE T-IP Editorial Board Meeting, and had the opportunity to meet with collaborators from the VUB-Duke-Ugent-UCL joint lab.
Featured articles:
On February 5th 2025 at 16:00, Raees Kizhakkumkara Muhamad will defend their PhD entitled “COMPRESSION STRATEGIES FOR DIGITAL HOLOGRAPHY”.
Everybody is invited to attend the presentation in room I.0.01 or online via this link.
Holographic techniques sample a 2D interference pattern produced by coherent light waves reflected or transmitted from the different objects in the 3D scene. As a display technology, it provides all necessary visual cues for perceiving the scene by the brain without causing mismatches between accommodation and vergence of the eyes. Non destructive imaging with high resolution for biomedical and industrial inspection also utilizes holographic principles. Holographic microscopes are realizable with optically simpler setups than regular microscopy, opening new pathways for computational microscopy. Utilizing more complex arrangements, such as holographic tomography, allows for reconstructing the 3D refractive index profile of transmissive objects, resolving even sub-cellular structures with visible light. It represents the culmination of humanity’s effort to record and represent light information.
However, sampling interference patterns for high-end displays or highresolution microscopy result in an extensive digital footprint. Historically, for many multimedia use cases, the data transmission bottleneck dictated the fidelity of the consumed content, and one can expect holography to be no different. Compression algorithms can help mitigate the data load, trading off more computation for an effective increase in transmission capacity. The algorithms must be tailored for holograms used in practice and exhibit a computational complexity appropriate for the use case, particularly on the decoder side. This thesis presents compression strategies for effectively tackling such use cases for holography.
The performance of conventional image compression tools on metrological hologram content is first studied. We provide a novel analysis of compression artifacts on the retrieved metrological data obtained for digital holographic microscopy and tomography. First-generation holographic displays are poised to use binary representation due to difficulties in modulating pixels at the sizes required by holography. Context-based compression is adequate for lossless and near-lossless compression of such data. Here, one extracts from a pixel the redundant information from previously decoded neighbouring pixels using a generalized Markovian model. Another context-based framework utilizing linear (autoregressive) models is used to design a highly scalable lossless compression scheme for non-binary holographic data. For broadcast scenarios, it is desirable to have a compression system that can support view-selective decoding to minimize the transmission of unutilized information. For this purpose, we propose a short-time Fourier transform (STFT) based codec, which slices the hologram into independently decoded, spatio-angular chunks. Given a target mean-squared error, the optimization techniques efficiently round down the signal in the STFT domain by application of adaptive quantizers. All these compression schemes are applicable for a single frame of hologram data and feature lightweight decoding architectures while surpassing compression performances achieved by any existing solution on most tested holograms. To compress holographic videos with arbitrary motion, we utilize a novel motion compensation algorithm that can predict rotational motion in conjunction with the above-mentioned STFT framework. The Markovian and STFT frameworks discussed in this work have been adopted as part of the first international hologram compression standard, JPEG Pleno — Part 5: Holography (ISO/IEC 21794-5)
ETRO, the Vrije Universiteit Brussel (VUB) and imec are proud to announce that Prof. Nikos Deligiannis has been awarded a prestigious ERC Consolidator Grant from the European Research Council to make a groundbreaking contribution to science and society.
Project: IONIAN: Reinventing Multiterminal Coding for Intelligent Machines
Budget: €1,999,404
Professor Deligiannis’ IONIAN project focuses on reinventing multiterminal coding, a crucial technology for efficient communication and collaboration between intelligent machines. With the explosive growth of data, such as video and point cloud streams, current storage and communication technologies are under pressure, undermining the ability of intelligent machines to cooperatively perceive their environment. This project develops a groundbreaking compression and communication approach based on interpretable and explainable AI that breaks the limits of traditional compression and cooperative perception techniques.
IONIAN combines classical theories, such as distributed source coding, with modern deep learning techniques and explainable AI, focusing on three innovative pillars:
The goal of this project is to elevate the collaboration between intelligent systems, such as autonomous vehicles and mobile robots, to a higher level, with greater safety and trust as the result.
Remote editing with VS Code
Visual Studio Code is a widely-used, cross-platform Integrated Development Environment (IDE) that supports numerous programming languages and offers a vast array of extensions to enhance its functionality.
One notable extension enables development on remote machines via SSH, providing integrated access to a file explorer, terminal, and text editor on the configured remote system. This makes it a strong alternative to JupyterLab as a remote editor for e.g., the ETROFARM Slurm cluster.
Open the Extensions tab using the corresponding icon in the left toolbar.
Search for “SSH” and subsequently select the topmost “Remote – SSH” extension. Install this extension.
Depending on your programming language of choice, you might also be interested in extensions such as “Python”, “Python Debugger”, “Ruff” (a Python code linter) etc.
With the “Remote – SSH” extension installed, a new tab “Remote explorer” has been added to the left toolbar.
Add a new remote by pressing the + icon.
When asked for the SSH Connection Command:
ssh <username>@etroflock.etrovub.be
Secondly, it will ask where to store this information. This can be the default option.
The etroflock.etrovub.be remote has been created. Time to connect by triggering one of the 2 corresponding buttons.
During a brief instant the option will be displayed to edit the configuration. If you have missed it you can find this file via the gear button next to remote explorer – remote tunnels – ssh
Your config file should look like
Host etroflock.etrovub.be
HostName etroflock.etrovub.be
User jdoe
IdentityFile C:\users\jdoe\.ssh\id_rsa
If you have no experience with encryption you can e.g. use a rsa 2048 type of key. Please make sure you are using a private key in openssh format.
Upon our first connection attempt, it requests the platform of the remote host, being the Slurm cluster’s login node we are connecting to. This is a Linux machine.
It will also ask to confirm the SSH public key credential of the server.
We are now connected to the remote server. This can be seen in the Remote explorer tab as well as in the left corner of the bottom toolbar.
Time to open our file explorer via the “Explorer” tab in the left toolbar (Ctrl + Shift + E). Press the “Open Folder” button. It should by default suggest to open your home folder on the Slurm cluster (currently on /FARM/<username>).
If prompted, confirm that the remote server is (again) a Linux platform. Lastly, confirm that you trust the authors of the files in this folder as this is your own home folder.
Congratulations! Your remote file explorer and text editor on the Slurm cluster is now operational.
A remote terminal can be opened using Terminal -> New Terminal in the top toolbar, or via the Ctrl + Shift + ` shortcut.
An interactive terminal session is opened on the remote host as if it was a PuTTY (or other) SSH session.
With the remote file explorer, remote text editor and remote terminal sessions available, it is a logical next step to focus on running our code remotely on the machine. Luckily, this is typically as straightforward as pressing the “Run file” button on the active (Python) file.
We can observe that the code has indeed been executed on the remote machine. However, the configured remote machine is ETROflock, ETRO’s Slurm cluster’s login server that is scarce in compute resources and lacks and GPU’s.
Running our code on a Slurm compute node is more complicated as it involves requesting a Slurm job. This is currently a manual process but we are investigating if this can be automated in VScode by using a custom launch script.
For now, there are two possibilities to run code on the Slurm cluster from within VScode:
A Slurm job can be requested that immediately runs the code until completion (or an error or timeout). This is the recommended default approach for running Slurm jobs.
E.g. the same test.py code is run as a Slurm job by using the srun command.
We observe that the first command is run on ETROFLOCK (the Slurm login node) because that is immediately launched on the remote system. The second command is scheduled as a Slurm job and is run on ETROFARM (a Slurm compute node).
The second possibility involves starting a Slurm job with an interactive shell. Once this interactive shell is running on a compute node, we can manually launch the desired code within that shell. This solution might be preferred when developing and testing the functionality of the code as a Slurm job (and potential queue) must only be requested once per session.
Launching a Slurm job with interactive shell is possible using the following srun command parameters:
srun –pty bash -i
E.g. the same test.py code is run after an interactive shell has been requested using the srun command and has started.
Observe that after requesting the slurm job with interactive terminal we change from “steffen@ETROFLOCK” to “steffen@ETROFARM”. This indicates our interactive shell is indeed running on a compute node instead of the login node.
Executing the test.py script from within this shell again confirms that the code is indeed run on the compute node with hostname ETROFARM.
When finished, one should use the “exit” command. This closes the interactive shell and terminates the Slurm job, thereby releasing the allocated resources for new jobs.
Filezilla is a free and open-source file transfer tool that can be used to exchange files with e.g., ETROFARM Slurm cluster. It requires some brief configuration in order to be able to connect using the SSH public/private key credentials. The required configuration steps are provided in the following tutorial.
Open the Site Manager, via the leftmost icon in the toolbar. Contrary to using the Quickconnect function which only supports connecting via a password, the Site Manager also allows configuring SFTP authentication via SSH keys.
Add a New site with the following configuration:
Connect to this SFTP server and trust the etroflock’s public SSH key.
All future FileZilla SFTP connections to ETROflock can be easily launched from the Site manager tab.
On November 15th 2024 at 10:00, Eden Teshome Hunde will defend their PhD entitled “CROSS-LAYER DESIGN, IMPLEMENTATION AND EVALUATION OF IPV6 MULTICAST FOR RADIO DUTY CYCLED WIRELESS SENSOR AND ACTUATOR NETWORKS”.
Everybody is invited to attend the presentation in room D.2.01 or online via this link.
In this work, we study Bidirectional Multicast RPL Forwarding (BMRF) as this protocol relies on forwarding tables put in place by the well-known Routing Protocol for Low Power and Lossy Networks (RPL) and allows to combine the best ideas of existing multicast protocols. Through RPL, a routing tree towards the sink is installed for multihop routing from node to sink, and the nodes’ forwarding tables will also contain entries for reaching destinations in downward direction.
For downward forwarding IPv6 multicast packets, two methods exist. One is via link layer (LL), broadcasting a frame containing the IPv6 multicast packet. The other is to send several LL unicast frames containing that packet. BMRF allows a node to choose between these two methods. The best option will depend on the presence of a radio duty cycling (RDC) protocol. RDC is part of the medium access control (MAC) layer and puts the radio to sleep when no communication is needed. We investigate the influence of MAC/RDC protocols on BMRF’s performance.
We evaluate the performance of BMRF on non-synchronized WSANs that use Carrier Sense Multiple Access (CSMA) as MAC and ContikiMAC as RDC. We demonstrate that LL unicast outperforms LL broadcast in terms of packet delivery ratio (PDR), delay, and energy consumption in many settings.
We investigate the performance of BMRF on WSANs with synchronous MAC and RDC based on Time Slotted Channel hopping (TSCH). This is more challenging, as TSCH needs a schedule to tell which action must happen in each timeslot. The actions can be to send or to listen on a given channel or to be idle. Idleness allows the radio to switch OFF, providing RDC. The schedule is not part of the standard and must be proposed by the system designer. An elegant autonomous scheduling method called Orchestra is available to accommodate traffic in a RPL tree. We extend Orchestra with a novel scheduling rule for supporting LL downwards forwarding through LL broadcast. Comparing LL unicast with LL broadcast forwarding teaches us that LL unicast outperforms LL broadcast in terms of packet delivery ratio (PDR), but the latter can be beneficial to certain applications, especially those sensitive to delay.
Before conducting the two previous evaluation studies, we investigate the performance of simple convergecast traffic while considering ContikiMAC and TSCH with Orchestra under RPL on the real dual Zolertia Firefly Motes (one is observed and other one is observing mote). This study served two purposes; it reminds the reader of the characteristics of those protocols and allowed to fine-tune the dual motes.
We also contributed by adapting the Orchestra to bursty convergecast traffic. Simulation results demonstrate that the new scheduler slightly improves PDR and reduces delay compared to state-of-the-art solutions.
On November 7th 2024 at 16:00, Boris Joukovsky will defend their PhD entitled “ SIGNAL PROCESSING MEETS DEEP LEARNING: INTERPRETABLE AND EXPLAINABLE NEURAL NETWORKS FOR VIDEO ANALYSIS, SEQUENCE MODELING AND COMPRESSION”.
Everybody is invited to attend the presentation in room I.0.01 or online via this link.
There is growing use of deep learning for solving signal processing tasks, and deep neural networks (DNNs) often outperform traditional methods little domain knowledge needed. However, DNNs behave as black boxes, making it difficult to understand their decisions. The empirical approaches to design DNNs often lack theoretical guarantees and create high computational requirements, which poses risks for applications requiring trustworthy artificial intelligence (AI). This thesis addresses these issues, focusing on video processing and sequential problems across three domains: (1) efficient, model-based DNN designs, (2) generalization analysis and information-theory-driven learning, and (3) post-hoc explainability.
The first contributions consist of new deep learning models for successive frame reconstruction, foreground-background separation, and moving object detection in video. These models are based on the deep unfolding method, a hybrid approach that combines deep learning with optimization techniques, leveraging low-complexity prior knowledge of the data. The resulting networks require fewer parameters than standard DNNs. They outperform DNNs of comparable size, large semantic-based convolutional networks, as well the underlying non-learned optimization methods.
The second area focuses on the theoretical generalization of deep unfolding models. The generalization error of reweighted-RNN (the model that performs video reconstruction) is characterized using Rademacher complexity analysis. This is a first-of-its-kind result that bridges machine learning theory with deep unfolding RNNs.
Another contribution in this area aims to learn optimally compressed, quality-scalable representations of distributed signals: a scheme traditionally known as Wyner-Ziv coding (WZC). The proposed method shows that deep models can retrieve layered binning solutions akin to optimal WZC, which is promising to learn constructive coding schemes for distributed applications.
The third area introduces InteractionLIME, an algorithm to explain how deep models learn multi-view or multi-modal representations. It is the first model-agnostic explanation method design to identify the important feature pairs across inputs that affect the prediction. Experimental results demonstrate its effectiveness on contrastive vision and language models.
In conclusion, this thesis addresses important challenges in making deep learning models more interpretable, efficient, and theoretically grounded, particularly for video processing and sequential data, thereby contributing to the development of more trustworthy AI systems.
HealthTech TouchPoints event (17/10/2024): VUB and UZ Brussel showcased their HealthTech expertise to companies.
ETRO did a pitch and had a demo booth at the matchmaking fair after the event.
More background info LinkedIn posts:
Benyameen Keelson and Pieter Boonen sucessfully finished the LifeTech.brussels MedTech accelerator with their startup projects PADFLOW en KARMA.
Some extra infomation can be found here.
The introductory movies for the projects:
Benyameen:
Pieter:
On October 25th 2024 at 16:00, Yuqing Yang will defend their PhD entitled “CRAFTING EFFECTIVE VISUAL EXPLANATIONS BY ATTRIBUTING THE IMPACT OF DATASETS, ARCHITECTURES AND DATA COMPRESSION TECHNIQUES”.
Everybody is invited to attend the presentation in room D.2.01 or online via this link.
Explainable Artificial Intelligence (XAI) plays an important role in modern AI research, motivated by the desire for transparency and interpretability within AI-driven decision-making. As AI systems become more advanced and complicated, it becomes increasingly important to ensure they are reliable, responsible, and ethical. These imperatives are particularly acute in domains where stakes are high, such as medical diagnostics, autonomous driving, and security frameworks.
In computer vision, XAI aims to provide understandable, straightforward explanations for AI model predictions, allowing users to grasp the decision-making processes of these complex systems. Visualizations such as saliency maps are frequently employed to identify input data regions significantly impacting model predictions, thus enhancing user understanding of AI visual data analysis. However, there are still concerns about the effectiveness of visual explanations, especially regarding their robustness, trustworthiness, and human-friendliness.
Our research aims to advance this field by evaluating how various factors—such as the diversity of datasets, the architecture of models, and techniques for data compression—influence the effectiveness of visual explanations in AI applications. Through thorough analysis and careful refinement, we strive to enhance these explanations, ensuring they are both highly informative and accessible to users in diverse XAI applications.
During our evaluation process, we conduct a detailed investigation using both automatic metrics and subjective evaluation methods to assess the effectiveness of visual explanations thoroughly. Automatic metrics, such as task performance and localization accuracy, provide quantifiable measures of the effectiveness of these explanations in real-world scenarios. For subjective evaluation, we have developed a framework named SNIPPET, which enables a detailed and user-oriented assessment of visual explanations. Additionally, our research explores how these objective metrics correlate with subjective human judgments, aiming to integrate quantitative data with the more nuanced, qualitative feedback from users. Ultimately, our goal is to provide comprehensive insights into the practical aspects of XAI methodologies, particularly focusing on their implementation in the field of computer vision.
A BIG high Five to Loris Giordano, Franjo Mikic, Anastasia Koutalianou, Jonathan Vrijsen and Sevada Sahakian. All five have obtained the prestigious Predoctoral Mandate for Strategic Basic Research for the coming 4 years.
We are VERY proud of you!
On October 15th 2024 at 16:00, Xiangyu Yang will defend their PhD entitled “Leveraging Deep Learning Models for Big Data Analytics”.
Everybody is invited to attend the presentation in room D.0.05 or online via this link.
With the exponential growth of data generated daily from social media, e-commerce, and various digital interactions, the necessity to effectively harness and leverage this vast expanse of information is more critical than ever. In this context, Deep Learning (DL), a subfield of Artificial Intelligence (AI), has emerged as a transformative force, delivering unparalleled capacities in pattern recognition, data analysis, and predictive modeling. Deep learning takes large amounts of available data as fuel to train itself, and significantly impacts various fields ranging from healthcare to finance, enabling advanced applications in natural language processing (NLP), computer vision (CV), and recommender systems (RS).
This thesis delves into the essential role of AI in leveraging big data, focusing on information extraction from social media, deep learning model explainability, and the development of explainable recommender systems. With the vast, ever-growing volume of data, extracting meaningful insights from unstructured social media becomes increasingly complex, necessitating cutting-edge AI solutions. Concurrently, the reliance on deep learning models for critical decisions brings explainability to the forefront, emphasizing the importance of developing transparent methods that ensure user trust. Furthermore, the demand for recommender systems that provide understandable textual explanations has surged, highlighting the need for explainable systems that align with user preferences and decision-making processes.
This thesis advances the field through three key contributions. Initially, we establish two traffic-related datasets from social media, annotated for comprehensive traffic event detection. Employing BERT-based models, we tackle this detection problem via text classification and slot filling, proving these models’ efficacy in parsing social media for traffic-related information. Our second contribution intro- duces LRP-based methods to explain deep conditional random fields, with successful applications in fake news detection and image segmentation. Lastly, we present an innovative personalized explainable recommender system that integrates user and item context into a language model, producing textual explanations that enhance system transparency.
On October 9th 2024 at 16:30, Esther Rodrigo Bonet will defend their PhD entitled “EXPLAINABLE AND PHYSICS-GUIDED GRAPH DEEP LEARNING FOR AIR POLLUTION MODELLING”.
Everybody is invited to attend the presentation in room I.0.02.
Air pollution has become a worldwide concern due to its negative impact on the population’s health and well-being. To mitigate its effects, it is essential to monitor pollutant concentrations across regions and time accurately. Traditional solutions rely on physics-driven approaches, leveraging particle motion equations to predict pollutants’ shifts in time. Despite being reliable and easy to interpret, they are computationally expensive and require background domain knowledge. Alternatively, recent works have shown that data-driven approaches, especially deep learning models, significantly reduce the computational expense and provide accurate predictions; yet, at the cost of massive data and storage requirements and lower interpretability.
This PhD research develops innovative air pollution monitoring solutions focusing on high accuracy, manageable complexity, and high interpretability. To this end, the research proposes various graph-based deep learning solutions focusing on two key aspects, namely, physics-guided deep learning and explainability.
First, as there exist correlations among the data points in smart city data, we propose exploiting them using graph-based deep learning techniques. Specifically, we leverage generative models that have proven efficient in data generation tasks, namely, variational graph autoencoders. The proposed models employ graph convolutional operations and data fusion techniques to leverage the graph structure and the multi-modality of the data at hand. Additionally, we design physics-guided deep-learning models that follow well-studied physical equations. By updating the graph convolution operator of graph convolutional networks to leverage the physics convection-diffusion equation, we can physically guide the learning curve of our network.
The second key point relates to explainability. Specifically, we design novel explainability techniques for interpretable graph deep modeling. We explore existing explainability algorithms, including Lasso and a layer-wise relevance propagation approach, and go beyond them to our graph-based architectures, designing efficient and specifically tailored explanation tools. Our explanation techniques are able to provide insights and visualizations based on various input data sources.
Overall, the research has produced state-of-the-art models that combine the best of both (physics-guided) graph-deep-learning-based and explainable approaches for inferring, predicting, and explaining air pollution. The developed techniques can also be applied to various applications in modeling graphs on the Internet such as in recommender systems’ applications.
“Signal Processing in the AI era” was the tagline of this year’s IEEE International Conference on Acoustics, Speech and Signal Processing, taking place in Rhodes, Greece.
In this context, Brent de Weerdt, Xiangyu Yang, Boris Joukovsky, Alex Stergiou and Nikos Deligiannis presented ETRO’s research during poster sessions and oral presentations, with novel ways to process and understand graph, video, and audio data. Nikos Deligiannis chaired a session on Graph Deep Learning, attended the IEEE T-IP Editorial Board Meeting, and had the opportunity to meet with collaborators from the VUB-Duke-Ugent-UCL joint lab.
Featured articles:
On February 5th 2025 at 16:00, Raees Kizhakkumkara Muhamad will defend their PhD entitled “COMPRESSION STRATEGIES FOR DIGITAL HOLOGRAPHY”.
Everybody is invited to attend the presentation in room I.0.01 or online via this link.
Holographic techniques sample a 2D interference pattern produced by coherent light waves reflected or transmitted from the different objects in the 3D scene. As a display technology, it provides all necessary visual cues for perceiving the scene by the brain without causing mismatches between accommodation and vergence of the eyes. Non destructive imaging with high resolution for biomedical and industrial inspection also utilizes holographic principles. Holographic microscopes are realizable with optically simpler setups than regular microscopy, opening new pathways for computational microscopy. Utilizing more complex arrangements, such as holographic tomography, allows for reconstructing the 3D refractive index profile of transmissive objects, resolving even sub-cellular structures with visible light. It represents the culmination of humanity’s effort to record and represent light information.
However, sampling interference patterns for high-end displays or highresolution microscopy result in an extensive digital footprint. Historically, for many multimedia use cases, the data transmission bottleneck dictated the fidelity of the consumed content, and one can expect holography to be no different. Compression algorithms can help mitigate the data load, trading off more computation for an effective increase in transmission capacity. The algorithms must be tailored for holograms used in practice and exhibit a computational complexity appropriate for the use case, particularly on the decoder side. This thesis presents compression strategies for effectively tackling such use cases for holography.
The performance of conventional image compression tools on metrological hologram content is first studied. We provide a novel analysis of compression artifacts on the retrieved metrological data obtained for digital holographic microscopy and tomography. First-generation holographic displays are poised to use binary representation due to difficulties in modulating pixels at the sizes required by holography. Context-based compression is adequate for lossless and near-lossless compression of such data. Here, one extracts from a pixel the redundant information from previously decoded neighbouring pixels using a generalized Markovian model. Another context-based framework utilizing linear (autoregressive) models is used to design a highly scalable lossless compression scheme for non-binary holographic data. For broadcast scenarios, it is desirable to have a compression system that can support view-selective decoding to minimize the transmission of unutilized information. For this purpose, we propose a short-time Fourier transform (STFT) based codec, which slices the hologram into independently decoded, spatio-angular chunks. Given a target mean-squared error, the optimization techniques efficiently round down the signal in the STFT domain by application of adaptive quantizers. All these compression schemes are applicable for a single frame of hologram data and feature lightweight decoding architectures while surpassing compression performances achieved by any existing solution on most tested holograms. To compress holographic videos with arbitrary motion, we utilize a novel motion compensation algorithm that can predict rotational motion in conjunction with the above-mentioned STFT framework. The Markovian and STFT frameworks discussed in this work have been adopted as part of the first international hologram compression standard, JPEG Pleno — Part 5: Holography (ISO/IEC 21794-5)
ETRO, the Vrije Universiteit Brussel (VUB) and imec are proud to announce that Prof. Nikos Deligiannis has been awarded a prestigious ERC Consolidator Grant from the European Research Council to make a groundbreaking contribution to science and society.
Project: IONIAN: Reinventing Multiterminal Coding for Intelligent Machines
Budget: €1,999,404
Professor Deligiannis’ IONIAN project focuses on reinventing multiterminal coding, a crucial technology for efficient communication and collaboration between intelligent machines. With the explosive growth of data, such as video and point cloud streams, current storage and communication technologies are under pressure, undermining the ability of intelligent machines to cooperatively perceive their environment. This project develops a groundbreaking compression and communication approach based on interpretable and explainable AI that breaks the limits of traditional compression and cooperative perception techniques.
IONIAN combines classical theories, such as distributed source coding, with modern deep learning techniques and explainable AI, focusing on three innovative pillars:
The goal of this project is to elevate the collaboration between intelligent systems, such as autonomous vehicles and mobile robots, to a higher level, with greater safety and trust as the result.
Remote editing with VS Code
Visual Studio Code is a widely-used, cross-platform Integrated Development Environment (IDE) that supports numerous programming languages and offers a vast array of extensions to enhance its functionality.
One notable extension enables development on remote machines via SSH, providing integrated access to a file explorer, terminal, and text editor on the configured remote system. This makes it a strong alternative to JupyterLab as a remote editor for e.g., the ETROFARM Slurm cluster.
Open the Extensions tab using the corresponding icon in the left toolbar.
Search for “SSH” and subsequently select the topmost “Remote – SSH” extension. Install this extension.
Depending on your programming language of choice, you might also be interested in extensions such as “Python”, “Python Debugger”, “Ruff” (a Python code linter) etc.
With the “Remote – SSH” extension installed, a new tab “Remote explorer” has been added to the left toolbar.
Add a new remote by pressing the + icon.
When asked for the SSH Connection Command:
ssh <username>@etroflock.etrovub.be
Secondly, it will ask where to store this information. This can be the default option.
The etroflock.etrovub.be remote has been created. Time to connect by triggering one of the 2 corresponding buttons.
During a brief instant the option will be displayed to edit the configuration. If you have missed it you can find this file via the gear button next to remote explorer – remote tunnels – ssh
Your config file should look like
Host etroflock.etrovub.be
HostName etroflock.etrovub.be
User jdoe
IdentityFile C:\users\jdoe\.ssh\id_rsa
If you have no experience with encryption you can e.g. use a rsa 2048 type of key. Please make sure you are using a private key in openssh format.
Upon our first connection attempt, it requests the platform of the remote host, being the Slurm cluster’s login node we are connecting to. This is a Linux machine.
It will also ask to confirm the SSH public key credential of the server.
We are now connected to the remote server. This can be seen in the Remote explorer tab as well as in the left corner of the bottom toolbar.
Time to open our file explorer via the “Explorer” tab in the left toolbar (Ctrl + Shift + E). Press the “Open Folder” button. It should by default suggest to open your home folder on the Slurm cluster (currently on /FARM/<username>).
If prompted, confirm that the remote server is (again) a Linux platform. Lastly, confirm that you trust the authors of the files in this folder as this is your own home folder.
Congratulations! Your remote file explorer and text editor on the Slurm cluster is now operational.
A remote terminal can be opened using Terminal -> New Terminal in the top toolbar, or via the Ctrl + Shift + ` shortcut.
An interactive terminal session is opened on the remote host as if it was a PuTTY (or other) SSH session.
With the remote file explorer, remote text editor and remote terminal sessions available, it is a logical next step to focus on running our code remotely on the machine. Luckily, this is typically as straightforward as pressing the “Run file” button on the active (Python) file.
We can observe that the code has indeed been executed on the remote machine. However, the configured remote machine is ETROflock, ETRO’s Slurm cluster’s login server that is scarce in compute resources and lacks and GPU’s.
Running our code on a Slurm compute node is more complicated as it involves requesting a Slurm job. This is currently a manual process but we are investigating if this can be automated in VScode by using a custom launch script.
For now, there are two possibilities to run code on the Slurm cluster from within VScode:
A Slurm job can be requested that immediately runs the code until completion (or an error or timeout). This is the recommended default approach for running Slurm jobs.
E.g. the same test.py code is run as a Slurm job by using the srun command.
We observe that the first command is run on ETROFLOCK (the Slurm login node) because that is immediately launched on the remote system. The second command is scheduled as a Slurm job and is run on ETROFARM (a Slurm compute node).
The second possibility involves starting a Slurm job with an interactive shell. Once this interactive shell is running on a compute node, we can manually launch the desired code within that shell. This solution might be preferred when developing and testing the functionality of the code as a Slurm job (and potential queue) must only be requested once per session.
Launching a Slurm job with interactive shell is possible using the following srun command parameters:
srun –pty bash -i
E.g. the same test.py code is run after an interactive shell has been requested using the srun command and has started.
Observe that after requesting the slurm job with interactive terminal we change from “steffen@ETROFLOCK” to “steffen@ETROFARM”. This indicates our interactive shell is indeed running on a compute node instead of the login node.
Executing the test.py script from within this shell again confirms that the code is indeed run on the compute node with hostname ETROFARM.
When finished, one should use the “exit” command. This closes the interactive shell and terminates the Slurm job, thereby releasing the allocated resources for new jobs.
Filezilla is a free and open-source file transfer tool that can be used to exchange files with e.g., ETROFARM Slurm cluster. It requires some brief configuration in order to be able to connect using the SSH public/private key credentials. The required configuration steps are provided in the following tutorial.
Open the Site Manager, via the leftmost icon in the toolbar. Contrary to using the Quickconnect function which only supports connecting via a password, the Site Manager also allows configuring SFTP authentication via SSH keys.
Add a New site with the following configuration:
Connect to this SFTP server and trust the etroflock’s public SSH key.
All future FileZilla SFTP connections to ETROflock can be easily launched from the Site manager tab.
On November 15th 2024 at 10:00, Eden Teshome Hunde will defend their PhD entitled “CROSS-LAYER DESIGN, IMPLEMENTATION AND EVALUATION OF IPV6 MULTICAST FOR RADIO DUTY CYCLED WIRELESS SENSOR AND ACTUATOR NETWORKS”.
Everybody is invited to attend the presentation in room D.2.01 or online via this link.
In this work, we study Bidirectional Multicast RPL Forwarding (BMRF) as this protocol relies on forwarding tables put in place by the well-known Routing Protocol for Low Power and Lossy Networks (RPL) and allows to combine the best ideas of existing multicast protocols. Through RPL, a routing tree towards the sink is installed for multihop routing from node to sink, and the nodes’ forwarding tables will also contain entries for reaching destinations in downward direction.
For downward forwarding IPv6 multicast packets, two methods exist. One is via link layer (LL), broadcasting a frame containing the IPv6 multicast packet. The other is to send several LL unicast frames containing that packet. BMRF allows a node to choose between these two methods. The best option will depend on the presence of a radio duty cycling (RDC) protocol. RDC is part of the medium access control (MAC) layer and puts the radio to sleep when no communication is needed. We investigate the influence of MAC/RDC protocols on BMRF’s performance.
We evaluate the performance of BMRF on non-synchronized WSANs that use Carrier Sense Multiple Access (CSMA) as MAC and ContikiMAC as RDC. We demonstrate that LL unicast outperforms LL broadcast in terms of packet delivery ratio (PDR), delay, and energy consumption in many settings.
We investigate the performance of BMRF on WSANs with synchronous MAC and RDC based on Time Slotted Channel hopping (TSCH). This is more challenging, as TSCH needs a schedule to tell which action must happen in each timeslot. The actions can be to send or to listen on a given channel or to be idle. Idleness allows the radio to switch OFF, providing RDC. The schedule is not part of the standard and must be proposed by the system designer. An elegant autonomous scheduling method called Orchestra is available to accommodate traffic in a RPL tree. We extend Orchestra with a novel scheduling rule for supporting LL downwards forwarding through LL broadcast. Comparing LL unicast with LL broadcast forwarding teaches us that LL unicast outperforms LL broadcast in terms of packet delivery ratio (PDR), but the latter can be beneficial to certain applications, especially those sensitive to delay.
Before conducting the two previous evaluation studies, we investigate the performance of simple convergecast traffic while considering ContikiMAC and TSCH with Orchestra under RPL on the real dual Zolertia Firefly Motes (one is observed and other one is observing mote). This study served two purposes; it reminds the reader of the characteristics of those protocols and allowed to fine-tune the dual motes.
We also contributed by adapting the Orchestra to bursty convergecast traffic. Simulation results demonstrate that the new scheduler slightly improves PDR and reduces delay compared to state-of-the-art solutions.
On November 7th 2024 at 16:00, Boris Joukovsky will defend their PhD entitled “ SIGNAL PROCESSING MEETS DEEP LEARNING: INTERPRETABLE AND EXPLAINABLE NEURAL NETWORKS FOR VIDEO ANALYSIS, SEQUENCE MODELING AND COMPRESSION”.
Everybody is invited to attend the presentation in room I.0.01 or online via this link.
There is growing use of deep learning for solving signal processing tasks, and deep neural networks (DNNs) often outperform traditional methods little domain knowledge needed. However, DNNs behave as black boxes, making it difficult to understand their decisions. The empirical approaches to design DNNs often lack theoretical guarantees and create high computational requirements, which poses risks for applications requiring trustworthy artificial intelligence (AI). This thesis addresses these issues, focusing on video processing and sequential problems across three domains: (1) efficient, model-based DNN designs, (2) generalization analysis and information-theory-driven learning, and (3) post-hoc explainability.
The first contributions consist of new deep learning models for successive frame reconstruction, foreground-background separation, and moving object detection in video. These models are based on the deep unfolding method, a hybrid approach that combines deep learning with optimization techniques, leveraging low-complexity prior knowledge of the data. The resulting networks require fewer parameters than standard DNNs. They outperform DNNs of comparable size, large semantic-based convolutional networks, as well the underlying non-learned optimization methods.
The second area focuses on the theoretical generalization of deep unfolding models. The generalization error of reweighted-RNN (the model that performs video reconstruction) is characterized using Rademacher complexity analysis. This is a first-of-its-kind result that bridges machine learning theory with deep unfolding RNNs.
Another contribution in this area aims to learn optimally compressed, quality-scalable representations of distributed signals: a scheme traditionally known as Wyner-Ziv coding (WZC). The proposed method shows that deep models can retrieve layered binning solutions akin to optimal WZC, which is promising to learn constructive coding schemes for distributed applications.
The third area introduces InteractionLIME, an algorithm to explain how deep models learn multi-view or multi-modal representations. It is the first model-agnostic explanation method design to identify the important feature pairs across inputs that affect the prediction. Experimental results demonstrate its effectiveness on contrastive vision and language models.
In conclusion, this thesis addresses important challenges in making deep learning models more interpretable, efficient, and theoretically grounded, particularly for video processing and sequential data, thereby contributing to the development of more trustworthy AI systems.
HealthTech TouchPoints event (17/10/2024): VUB and UZ Brussel showcased their HealthTech expertise to companies.
ETRO did a pitch and had a demo booth at the matchmaking fair after the event.
More background info LinkedIn posts:
Benyameen Keelson and Pieter Boonen sucessfully finished the LifeTech.brussels MedTech accelerator with their startup projects PADFLOW en KARMA.
Some extra infomation can be found here.
The introductory movies for the projects:
Benyameen:
Pieter:
On October 25th 2024 at 16:00, Yuqing Yang will defend their PhD entitled “CRAFTING EFFECTIVE VISUAL EXPLANATIONS BY ATTRIBUTING THE IMPACT OF DATASETS, ARCHITECTURES AND DATA COMPRESSION TECHNIQUES”.
Everybody is invited to attend the presentation in room D.2.01 or online via this link.
Explainable Artificial Intelligence (XAI) plays an important role in modern AI research, motivated by the desire for transparency and interpretability within AI-driven decision-making. As AI systems become more advanced and complicated, it becomes increasingly important to ensure they are reliable, responsible, and ethical. These imperatives are particularly acute in domains where stakes are high, such as medical diagnostics, autonomous driving, and security frameworks.
In computer vision, XAI aims to provide understandable, straightforward explanations for AI model predictions, allowing users to grasp the decision-making processes of these complex systems. Visualizations such as saliency maps are frequently employed to identify input data regions significantly impacting model predictions, thus enhancing user understanding of AI visual data analysis. However, there are still concerns about the effectiveness of visual explanations, especially regarding their robustness, trustworthiness, and human-friendliness.
Our research aims to advance this field by evaluating how various factors—such as the diversity of datasets, the architecture of models, and techniques for data compression—influence the effectiveness of visual explanations in AI applications. Through thorough analysis and careful refinement, we strive to enhance these explanations, ensuring they are both highly informative and accessible to users in diverse XAI applications.
During our evaluation process, we conduct a detailed investigation using both automatic metrics and subjective evaluation methods to assess the effectiveness of visual explanations thoroughly. Automatic metrics, such as task performance and localization accuracy, provide quantifiable measures of the effectiveness of these explanations in real-world scenarios. For subjective evaluation, we have developed a framework named SNIPPET, which enables a detailed and user-oriented assessment of visual explanations. Additionally, our research explores how these objective metrics correlate with subjective human judgments, aiming to integrate quantitative data with the more nuanced, qualitative feedback from users. Ultimately, our goal is to provide comprehensive insights into the practical aspects of XAI methodologies, particularly focusing on their implementation in the field of computer vision.
A BIG high Five to Loris Giordano, Franjo Mikic, Anastasia Koutalianou, Jonathan Vrijsen and Sevada Sahakian. All five have obtained the prestigious Predoctoral Mandate for Strategic Basic Research for the coming 4 years.
We are VERY proud of you!
On October 15th 2024 at 16:00, Xiangyu Yang will defend their PhD entitled “Leveraging Deep Learning Models for Big Data Analytics”.
Everybody is invited to attend the presentation in room D.0.05 or online via this link.
With the exponential growth of data generated daily from social media, e-commerce, and various digital interactions, the necessity to effectively harness and leverage this vast expanse of information is more critical than ever. In this context, Deep Learning (DL), a subfield of Artificial Intelligence (AI), has emerged as a transformative force, delivering unparalleled capacities in pattern recognition, data analysis, and predictive modeling. Deep learning takes large amounts of available data as fuel to train itself, and significantly impacts various fields ranging from healthcare to finance, enabling advanced applications in natural language processing (NLP), computer vision (CV), and recommender systems (RS).
This thesis delves into the essential role of AI in leveraging big data, focusing on information extraction from social media, deep learning model explainability, and the development of explainable recommender systems. With the vast, ever-growing volume of data, extracting meaningful insights from unstructured social media becomes increasingly complex, necessitating cutting-edge AI solutions. Concurrently, the reliance on deep learning models for critical decisions brings explainability to the forefront, emphasizing the importance of developing transparent methods that ensure user trust. Furthermore, the demand for recommender systems that provide understandable textual explanations has surged, highlighting the need for explainable systems that align with user preferences and decision-making processes.
This thesis advances the field through three key contributions. Initially, we establish two traffic-related datasets from social media, annotated for comprehensive traffic event detection. Employing BERT-based models, we tackle this detection problem via text classification and slot filling, proving these models’ efficacy in parsing social media for traffic-related information. Our second contribution intro- duces LRP-based methods to explain deep conditional random fields, with successful applications in fake news detection and image segmentation. Lastly, we present an innovative personalized explainable recommender system that integrates user and item context into a language model, producing textual explanations that enhance system transparency.
On October 9th 2024 at 16:30, Esther Rodrigo Bonet will defend their PhD entitled “EXPLAINABLE AND PHYSICS-GUIDED GRAPH DEEP LEARNING FOR AIR POLLUTION MODELLING”.
Everybody is invited to attend the presentation in room I.0.02.
Air pollution has become a worldwide concern due to its negative impact on the population’s health and well-being. To mitigate its effects, it is essential to monitor pollutant concentrations across regions and time accurately. Traditional solutions rely on physics-driven approaches, leveraging particle motion equations to predict pollutants’ shifts in time. Despite being reliable and easy to interpret, they are computationally expensive and require background domain knowledge. Alternatively, recent works have shown that data-driven approaches, especially deep learning models, significantly reduce the computational expense and provide accurate predictions; yet, at the cost of massive data and storage requirements and lower interpretability.
This PhD research develops innovative air pollution monitoring solutions focusing on high accuracy, manageable complexity, and high interpretability. To this end, the research proposes various graph-based deep learning solutions focusing on two key aspects, namely, physics-guided deep learning and explainability.
First, as there exist correlations among the data points in smart city data, we propose exploiting them using graph-based deep learning techniques. Specifically, we leverage generative models that have proven efficient in data generation tasks, namely, variational graph autoencoders. The proposed models employ graph convolutional operations and data fusion techniques to leverage the graph structure and the multi-modality of the data at hand. Additionally, we design physics-guided deep-learning models that follow well-studied physical equations. By updating the graph convolution operator of graph convolutional networks to leverage the physics convection-diffusion equation, we can physically guide the learning curve of our network.
The second key point relates to explainability. Specifically, we design novel explainability techniques for interpretable graph deep modeling. We explore existing explainability algorithms, including Lasso and a layer-wise relevance propagation approach, and go beyond them to our graph-based architectures, designing efficient and specifically tailored explanation tools. Our explanation techniques are able to provide insights and visualizations based on various input data sources.
Overall, the research has produced state-of-the-art models that combine the best of both (physics-guided) graph-deep-learning-based and explainable approaches for inferring, predicting, and explaining air pollution. The developed techniques can also be applied to various applications in modeling graphs on the Internet such as in recommender systems’ applications.