66123 Saarbrücken (Germany)
Pruning deep neural networks for lottery tickets
Deep learning has achieved major breakthroughs in a variety of tasks. Yet, it comes at a considerable computational cost, which is exaggerated by the recent trend towards ever wider and deeper neural network architectures. Instead, many problems can be solved with the help of extremely sparse neural network architectures but finding and training them is a non-trivial task. According to the recent lottery ticket hypothesis, such sparse architectures can be identified by pruning large randomly initialized neural networks. In this seminar, we will present recent algorithmic advancements in this direction, gain theoretical insights into the existence of lottery tickets, identify open problems, and discuss common challenges in the quest for winning lottery tickets.
In this seminar, students will learn to present, discuss, and summarize papers related to the lottery ticket hypothesis. Specifically, each student will get a single topic assigned to them, consisting of two papers (a lead and follow-up paper). Each student will
In addition to your review you will have to submit 3 questions that you will ask the presenter of the paper.
Presentation: (40%). You will prepare and deliver a 30 min presentation (followed by 15 mins question/discussion) of the paper assigned to you. You will have the possibility to get feedback on your slides before the presentation.