Deep Learning
Objectives
This curricular unit will provide the student with the skills to:
Understand:
- The foundations of deep learning
- Fundamentals of deep network computing
Optimization algorithms, activation functions, objective functions
- Different deep network architectures and their usefulness:
Dense, convolution, generative models.
- Training and regularization of deep networks
- The importance of data characteristics and training, validation, and test sets
Be able to:
- Select appropriate models and loss functions for different problems.
- Use modern frameworks for deep learning.
- Implement deep networks, optimize their hyper-parameters, and train them.
- Evaluate the training of the models and the quality of the results.
Know:
- Types of problems solved with deep networks
- Architectures and regularization of deep networks
- Model selection methods and hyper-parameters
General characterization
Code
12423
Credits
6.0
Responsible teacher
Claudia Alexandra Magalhães Soares, João Alexandre Carvalho Pinheiro Leite
Hours
Weekly - 4
Total - 48
Teaching language
Inglês
Prerequisites
-
Bibliography
These books will not cover the material thoroughly; alternative materials will be provided when necessary. The main book is Dive into Deep Learning.
Zhang, A., Lipton, Z. C., Li, M., & Smola, A. J. (2023). Dive into deep learning. CUP, https://d2l.ai
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning . MIT Press. https://deeplearningbook.org
Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction. MIT Press.
François-Lavet, V., Henderson, P., Islam, R., Bellemare, M. G., & Pineau, J. (2018). An introduction to deep reinforcement learning. Foundations and Trends® in Machine Learning, 11(3-4), 219-354.
Bronstein, M. M., Bruna, J., Cohen, T., & Veličković, P. (2021). Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104.13478.
Teaching method
Lectures will cover the fundamental topics of the subject matter, which the students should complement with the given bibliography. All lecture materials will be supplied for further study. Lectures will include some time for questions and discussion of the subject matter.
Tutorial classes will be dedicated to exercises and guidance in the practical assignments, focusing on selected topics.
Class schedules and materials will be supplied online, as well as additional information regarding the course.
Evaluation method
The evaluation of this curricular unit is made by two components: theoretical/problems (T) and project (P). Each component contributes with 50% to the final grade.
Both components are evaluated in an integer scale from 0 to 20.
To pass, the student must have:
(i) a grade of at least 9,5 points in the theoretical/problems component; and
(ii) a grade of at least 9.5 points in the project component;
The final grade is the weighted average of the two evaluation components (0.5×T + 0.5×P) on an integer scale from 0 to 20 points. If the final grade is higher than 17 points, there will be an oral test to defend the grade obtained. After the oral examination, the grade will not be less than 17 points. If a student with a computed final grade larger than 17 does not attend the oral examination, the final grade is automatically defined as 17 points.
Theoretical/problems component
Two written tests evaluate this component. The grade of this component is the average of the two written tests.
Alternatively, this component can be evaluated by a written exam. The exam will be graded in two independent parts, each corresponding to one test and replacing its grade if better.
An extra point will be awarded in the theoretical-practical component that will evaluate the delivery of a report transcribing and deepening the contents of one theoretical class.
Project component
Two mini-projects evaluate this component.
Some tutorial classes will be allocated to the mini-projects.
The mini-projects are done in groups of students, but this component is evaluated individually.
The grading of the different evaluation components is rounded to the first decimal place. The final grade is rounded to the closest integer value.
Subject matter
1. Introduction: fundamentals of deep learning, nonlinear transformations, and overfitting.
2. Artificial neural networks, backpropagation. and deep feedforward networks.
3. Implementation and training of deep neural networks
4. Optimization and regularization of feedforward networks. Training, testing, and cross-validation.
5. Convolution networks, theory, and practice
6. Unsupervised deep learning with autoencoders
7. Representation and transfer learning
8. Generative models
9. Reinforcement learning
10. Structured probabilistic models
11. Practical aspects of deep network selection, application, and optimization
12. Data and model visualization.
13. Open problems in deep learning