ETRO VUB
About ETRO  |  News  |  Events  |  Vacancies  |  Contact  
Home Research Education Industry Publications About ETRO

Master theses

Current and past ideas and concepts for Master Theses.

Phased Learning: Boosting the performance of the neural network through importance sampling

Subject

One of the key components of the successful training of neural networks is the large scale dataset. The larger the dataset, the higher diversity can be among the different samples, and as a consequence, the higher performance can be achieved on the unseen data. High diversity also means that samples can have different levels of "difficulty", from the point of view of prediction.
In the conventional training, at the first iterations, when model performance is low, the gain that model can have by learning from "easy" samples, or by learning from the "difficult" samples are nearly equal, as the ability of the model to extract useful information is low. Those, wasting "difficult" samples with a high amount of information at the first iterations of training are unreasonable. They can be introduced to the network at the later stages of the training when the model will already be able to extract all of the information that "easy" samples can provide. At that point, the model will take advantage of the additional information that "difficult" samples can provide.
This kind of Phase Learning can provide a better generalization to any neural network, independent of architecture, loss function or optimization method.

Kind of work

Within this work, a student will investigate the image classification dataset from point of view of information that each sample can provide. Further, it will lead to the separation of the data into clusters with samples of the same difficulty. Those clusters will be used to train the neural network in stages, with the goal of achieving better generalization.

Framework of the Thesis

Related literature:
Vighnesh Birodkar, Hossein Mobahi, and Samy Bengio. Semantic redundancies in image classification datasets: The 10% you don’t need. arXiv preprint arXiv:1901.11409, 2019.
Y. Bengio, J. Louradour, R. Collobert, and J. Weston. Curriculum learning. In Proc. of ICML, New York, NY, USA, 2009. 2

Number of Students

1-2 students

Expected Student Profile

The student should have a background in machine learning, image processing, python programming language and in one of the deep learning framework (Pytorch or Tensorflow)
Knowledge of Information theory is a plus.

Promotor

Prof. Dr. Ir. Nikolaos Deligiannis

+32 (0)2 629 1683

ndeligia@etrovub.be

more info

Supervisor

Miss Lusine Abrahamyan

+32 (0)2 629 1611

alusine@etrovub.be

more info

- Contact person

- IRIS

- AVSP

- LAMI

- Contact person

- Thesis proposals

- ETRO Courses

- Contact person

- Spin-offs

- Know How

- Journals

- Conferences

- Books

- Vacancies

- News

- Events

- Press

Contact

ETRO Department

Tel: +32 2 629 29 30

©2022 • Vrije Universiteit Brussel • ETRO Dept. • Pleinlaan 2 • 1050 Brussels • Tel: +32 2 629 2930 (secretariat) • Fax: +32 2 629 2883 • WebmasterDisclaimer