ETRO VUB
About ETRO  |  News  |  Events  |  Vacancies  |  Contact  
Home Research Education Industry Publications About ETRO

Master theses

Current and past ideas and concepts for Master Theses.

1-Bit Compressive Sensing for Efficient AI

Subject

Recent successful results in artificial intelligence are achieved with deep learning models that contain a large number of parameters and are trained using a massive amount of data. To speed up the training or protect privacy, distributed learning is introduced. This approach, however, entails high communication rates and latency because of the computed gradients that need to be shared among nodes at every iteration. To reduce the communication cost, the gradients need to be compressed. Some research results reveal the tradeoff between convergence performance and communication efficiency due to the aggregation errors caused by gradients’ sparsification, dimension reduction, quantization, gradients reconstruction, and noise.
Compressed sensing has become a field, opening up new prospects for signal acquisition and processing. It aims to reduce the number of samples required for successful signal reconstruction. In fact, compressed sensing aims to save sensing resources, transmission, and storage capacity and facilitate signal processing when certain data is unavailable. Particularly, the task of 1-bit compressive sensing is to reconstruct the signal from compressive sensing measurements quantized to one bit per measurement, which can reduce information transmission cost. So, we will explore whether distributed training can become efficient by using 1-Bit compressive sensing principles and techniques.

Kind of work

the students need to design an efficient scheme for distributed learning which incorporates 1-bit compressed sensing (CS). Then, they should formulate 1-bit CS-based distributed learning as an optimization problem and find optimal results. Finally, the students need to set up a distributed training using the designed 1-bit compressive sensing and evaluate distributed training performance.

Framework of the Thesis

[1] P. T. Boufounos and R. G. Baraniuk, ֿ-Bit compressive sensing,” CISS 2008, 42nd Annu. Conf. Inf. Sci. Syst., pp. 16–21, 2008, doi: 10.1109/CISS.2008.4558487.
[2] X. Fan, Y. Wang, Y. Huo, and Z. Tian, ֿ-Bit Compressive Sensing for Efficient Federated Learning Over the Air,” Mar. 2021, Accessed: Apr. 10, 2021. [Online]. Available: http://arxiv.org/abs/2103.16055.
[3] Y Yang, P. Xiao, B. Liao, N. Deligiannis, “A Robust Deep Unfolded Network for Sparse Signal Recovery from Noisy Binary Measurements,” European Signal Processing Conference (EUSIPCO), 2020.

Number of Students

1-2 students

Expected Student Profile

The student should have a background in machine learning, and python programming knowledge of deep learning (e.g., PyTorch, TensorFlow)

Promotor

Prof. Dr. Ir. Nikolaos Deligiannis

+32 (0)2 629 1683

ndeligia@etrovub.be

more info

Supervisors

Miss Yuqing Yang

+32 (0)2 629 2930

yyanga@etrovub.be

more info

Mr. Yiming Chen

+32 (0)2 629 2930

cyiming@etrovub.be

more info

- Contact person

- IRIS

- AVSP

- LAMI

- Contact person

- Thesis proposals

- ETRO Courses

- Contact person

- Spin-offs

- Know How

- Journals

- Conferences

- Books

- Vacancies

- News

- Events

- Press

Contact

ETRO Department

Tel: +32 2 629 29 30

©2022 • Vrije Universiteit Brussel • ETRO Dept. • Pleinlaan 2 • 1050 Brussels • Tel: +32 2 629 2930 (secretariat) • Fax: +32 2 629 2883 • WebmasterDisclaimer