Winter School

NLDL 2024 Winter School

The PhD winter school will consist of tutorials by experts in the field and is co-hosted by NORA as part of the NORA Research School. A preliminary program is provided below. 

Anyone who is interested can attend the winter school by registering for the conference on the Registration Page.


Getting formal ECTS Credits:  UiT The Arctic University of Norway awards 5 ECTS for the Winter School for students who register formally for the course at UiT before the 1st of November (for credits, there is a limit of 40 students). To obtain credits, participants will be required to present an ongoing research project (poster presentation) as part of the winter school and complete a home exam afterward.

Please read the 5 ECTS course description and application procedure here:

 FYS-8602 | UiT


Course Description

Deep learning is a rapidly growing segment of machine learning. It is increasingly used to deliver near-human level accuracy for many tasks such as image classification, voice recognition and natural language processing. Applications areas include facial recognition, scene detection, advanced medical and pharmaceutical research, and autonomous, self-driving vehicles. 

This 5-day course is build upon tutorials on specific topics on deep learning from perspectives such as Synthetic data, Generative Models, and Explainability. The course further encompasses among others keynote talks as well as special sessions on industry and diversity in AI as part of the NLDL conference program. 

In particular, the winter school will provide a study of several emerging topics of high relevance within advanced deep learning, from a basic understanding of the techniques to the latest state-of-the-art developments in the field. Synthetic data generation for addressing common problems of data scarcity, privacy-preserving data sharing, and bias through case studies, reliability of AI, and generative models will be treated in depth in the form of tutorials, as will high-performance computing. Additional directions within deep learning, complementing those already mentioned, will be covered at the introductory level via a series of keynote talks. In addition, the participants will be exposed to the latest advances and applications in deep learning by the oral presentations and poster presentations in the main conference program.


The course will thus consist of 5 full days of the NLDL conference including: tutorials, keynote sessions, oral presentations and poster presentations, as well as practical components.

Program

Venue: TEO-H1 1.820-AUD1

Monday January 8th:

09:00-09:45 Registration 

09:45-10:00 Welcome

10-13 Tutorial: Innovative Uses of Synthetic Data Tutorial (Mihaela van der Schaar) 

One of the biggest barriers to AI adoption is the difficulty to access high quality training data. Synthetic data has been widely recognised as a viable solution to this problem. It allows sharing, augmenting and de-biasing data for building performant and socially responsible AI algorithms. However, despite the significant progress in the theory and algorithm, the community still lacks a unified software that enables practical data sharing and access with synthetic data. This lab aims to bridge this gap by introducing synthcity, an open source Python library that implements an array of cutting edge synthetic data generators to address the problems of data generation due to its commonality in various applications.

13-14 Lunch 

14:00-16:00 Poster Presentations

16-18 Tutorial: Generative Modeling with Variational Autoencoders (Rogelio A. Mancisidor)

Variational Autoencoders (VAEs) are probabilistic generative models used in different learning tasks, such as clustering, classification, representation learning or learning conditional distributions to generate new data. This tutorial presents different ways to derive the evidence lower bound (ELBO) to gain a deeper understanding of the advantages (e.g., representation learning and generative modeling in a single model) and disadvantages (e.g., posterior collapse or lack of flexibility in the latent space) of optimizing the ELBO. In addition, we will discuss how we can extend VAEs to the multimodality domain to learn conditional distributions, which enables cross-modal generation. By the end of this tutorial, you will have a good understanding of the main advantages of VAEs and why VAEs are a popular choice of generative models, as well as insight into some of the challenges posed by the probabilistic aspect of VAEs. 

18:00-21:00 Icebreaker at the Vitensenter (Google Maps) 



January 9-11th: Main Conference Program


Friday January 12th:

09-11 Tutorial: Coding a Diffusion model from scratch (Filippo Maria Bianchi)

Renowned for their exceptional capabilities in synthesizing realistic images, audio, and video, DDPMs have marked a new era in generative modeling. In this introductory tutorial, we will dive into the world of Denoising Diffusion Probabilistic Models (DDPM) by unraveling the theory behind these groundbreaking generative models. In particular, we will closely look at the vanilla DDPM by implementing it from scratch using Pytorch and then using it to generate images from the popular Fashion MNIST dataset. At the end of the tutorial, you will have gained a robust understanding of DDPMs and also acquired the practical skills to build and apply these models to real data.

11-12 Lunch

12-14:30 Tutorial: Accelerated Deep Learning via High-performance Computing (Norwegian AI Cloud)


Welcome to HPC-intro tutorial (Sabry Razick UiO/NRIS)


The following link leads to the presentation

https://training.pages.sigma2.no/tutorials/gpus-on-hpc/

 

Accelerated Deep Learning on the supercomputer LUMI-G 


Part I: Hicham Agueny (UiB/NRIS) 

I.1-Introduction to supercomputer LUMI 

I.2-Basics of AMD-GPU topology 

I.3-Distributed Deep Learning with Horovod-TensorFlow on LUMI-G 


Part II: Magnar Bjørgve (UiT/NRIS) 

II.1-Distributed Jax on LUMI-G 

II.2-Basic tools for GPU monitoring DL application on LUMI-G 



Tutorial Speakers

Mihaela van der Schaar is the John Humphrey Plummer Professor of Machine Learning, Artificial Intelligence and Medicine at the University of Cambridge and a Fellow at The Alan Turing Institute in London. In addition to leading the van der Schaar Lab, Mihaela is founder and director of the Cambridge Centre for AI in Medicine (CCAIM). Mihaela was elected IEEE Fellow in 2009. She has received numerous awards, including the Oon Prize on Preventative Medicine from the University of Cambridge (2018), a National Science Foundation CAREER Award (2004), 3 IBM Faculty Awards, the IBM Exploratory Stream Analytics Innovation Award, the Philips Make a Difference Award and several best paper awards, including the IEEE Darlington Award.


Mihaela is personally credited as inventor on 35 USA patents (the majority of which are listed here), many of which are still frequently cited and adopted in standards. She has made over 45 contributions to international standards for which she received 3 ISO Awards. In 2019, a Nesta report determined that Mihaela was the most-cited female AI researcher in the U.K.

Rogelio A. Mancisidor received his B.Sc. and M.Sc. in Finance from BI Norwegian Business School and his Ph.D. in Machine Learning from UiT The Arctic University of Norway in 2021. During his Ph.D., Rogelio developed different unsupervised, supervised and semi-supervised models using variational autoencoders. He is currently an assistant professor in the Department of Data Science and Analytics at BI Norwegian Business School, and his current research interests include deep generative models, Bayesian modeling, variational inference, multimodal learning and text analytics.

Filippo Maria Bianchi is an Associate Professor at the Department of Mathematics and Statistics at UiT the Arctic University of Norway and holds a senior researcher position at NORCE, the Norwegian Research Centre. He serves as the vice-chair of the IEEE Task Force on Learning for Structured Data and is an active member of multiple groups, including the Graph Machine Learning Group in Lugano, the Northernmost Graph Machine Learning group, and the IEEE Task Force on Reservoir Computing. His research is centered at the intersection of machine learning, dynamical systems, and complex networks, with a particular focus on the application of his expertise in the fields of energy analytics and remote sensing.

Norwegian AI Cloud (NAIC) Tutorial Speakers

Sabry Razick Ph.D. Is a Chief engineer at the University of Oslo. He is the manager of the Norwegian AI Cloud at the moment and part of the NRIS consortium working with high performance computer systems. He also is a member of the CodeRefinery, which is a project dedicated to teaching essential tools for software development and scientific computing.

Hicham Agueny received his Ph.D. in theoretical and computational physics and chemistry. He is currently a senior engineer in scientific computing at the IT department, University of Bergen (UiB). Previously, he worked as a researcher for about seven years at the Department of Physics and Technology, UiB. His particular interest lies in heterogeneous computing involving GPU acceleration. For more details, please see https://orcid.org/my-orcid?orcid=0000-0003-2838-7422 

Magnar Bjørgve is an engineer working at the IT Department in Tromsø, contributing to the Norwegian Research Infrastructure Services (NRIS), particularly within the GPU-team. He's involved in important projects focused on optimizing computing processes and infrastructure, including the development of storage solutions for the Norwegian AI Cloud (NAIC). Magnar has a background applied mathematics, with a Master’s degree, and has recently submitted his PhD thesis in theoretical chemistry. His academic work includes developing computational methods and software. .

Evaluation Committee for the Course's exam: 

Contact : 

Winter School Chair: Puneet Sharma (puneet.sharma@uit.no)

For other inquiries: Robert Jenssen (robert.jenssen@uit.no) and Michael Kampffmeyer (michael.c.kampffmeyer@uit.no).