Deep Learning
Objectives
This curricular unit will provide the student with the skills to:
Understand:
- The foundations of deep learning
- Fundamentals of deep network computing
Optimization algorithms, activation functions, objective functions
- Different deep network architectures and their usefulness:
Dense, convolution, recurrent, generative models.
- Training and regularization of deep networks
- The importance of data characteristics and of training, validation and test sets
Be able to:
- Select appropriate models and loss functions for different problems.
- Use modern libraries for deep learning.
- Implement deep networks, optimize their hyper-parameters and train them.
- Evaluate the training of the models and the quality of the results.
Know:
- Types of problems solved with deep networks
- Architectures and regularization of deep networks
- Model selection methods and hyper-parameters
General characterization
Code
12423
Credits
6.0
Responsible teacher
João Alexandre Carvalho Pinheiro Leite, Ludwig Krippahl
Hours
Weekly - 4
Total - 48
Teaching language
Português
Prerequisites
-
Bibliography
Main textbook, mandatory:
Goodfellow, Ian, Bengio, Yoshua, Courville, Aaron: Deep Learning , MIT Press, 2016
Complementary reading:
Skansi, Sandro: Introduction to Deep Learning: From Logical Calculus to Artificial Intelligence , Springer, 2018
Géron, Aurélien: Hands-on machine learning with Scikit-Learn and TensorFlow: concepts, tools, and techniques to build intelligent systems O''''Reilly Media, Inc, 2017
Teaching method
Lectures will cover the fundamental topics of the subject matter, which the students should complement with the given bibliography. All lecture materials will be supplied for further study. Lectures will include some time for questions and discussion of the subject matter.
Tutorial classes will be dedicated to exercises and guidance in the practical assignments, focusing on selected topics.
Class schedules and materials will be supplied online, as well as additional information regarding the course.
Evaluation method
The evaluation of this curricular unit is made by two components: theoretical/problems (T) and project (P). Each component contributes with 50% to the final grade.
Both components are evaluated in an integer scale from 0 to 20.
To pass, the student must have:
(i) a grade of at least 9,5 points in the theoretical/problems component; and
(ii) a grade of at least 9.5 points in the project component;
The final grade is defined as the weighted average of the two components of evaluation, that is (0.5×T + 0.5×P), in an integer scale from 0 to 20 points.
Theoretical/problems component
This component is evaluated by 2 written tests. The grade of this component is the average of the 2 written tests.
Alternatively, this component can be evaluated by a written exam. The exam will be grade in two independent parts, each corresponding to one test and replacing its grade if better.
Project component
This component is evaluated by 2 mini-projects.
Some tutorial classes will be allocating to the mini-projects.
The mini-projects are done in groups of students but the evaluation of this component is individual.
Grading of the different evaluation components is rounded to the first decimal place. The final grade is rounded to the closest integer value.
Subject matter
1. Introduction: fundamentals of deep learning, nonlinear transformations and overfitting.
2. Artificial neural networks, backpropagation. and deep feedforward networks.
3. Implementation and training of deep neural networks
4. Optimization and regularization of feedforward networks. Training, testing and cross validation.
5. Convolution networks, theory and practice
6. Unsupervised deep learning with autoencoders
7. Representation and transfer learning
8. Generative models
9. Recurrent networks and problems with sequential data
10. Reinforcement learning
11. Structured probabilistic models
12. Practical aspects of deep network selection, application and optimization
13. Data and model visualization.
14. Open problems in deep learning