Deep Learning


This curricular unit will provide the student with the skills to:

- The foundations of deep learning
- Fundamentals of deep network computing
   Optimization algorithms, activation functions, objective functions
- Different deep network architectures and their usefulness:
   Dense, convolution, generative models.
- Training and regularization of deep networks
- The importance of data characteristics and training, validation, and test sets

Be able to:
- Select appropriate models and loss functions for different problems.
- Use modern frameworks for deep learning.
- Implement deep networks, optimize their hyper-parameters, and train them.
- Evaluate the training of the models and the quality of the results.

- Types of problems solved with deep networks
- Architectures and regularization of deep networks
- Model selection methods and hyper-parameters

General characterization





Responsible teacher

Claudia Alexandra Magalhães Soares, João Alexandre Carvalho Pinheiro Leite


Weekly - 4

Total - 48

Teaching language





These books will not cover the material thoroughly; alternative materials will be provided when necessary. The main book is Dive into Deep Learning.

Zhang, A., Lipton, Z. C., Li, M., & Smola, A. J. (2023). Dive into deep learning. CUP,

Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning . MIT Press.

Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction. MIT Press.

François-Lavet, V., Henderson, P., Islam, R., Bellemare, M. G., & Pineau, J. (2018). An introduction to deep reinforcement learning. Foundations and Trends® in Machine Learning11(3-4), 219-354.

Bronstein, M. M., Bruna, J., Cohen, T., & Veličković, P. (2021). Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104.13478.

Teaching method

Lectures will cover the fundamental topics of the subject matter, which the students should complement with the given bibliography. All lecture materials will be supplied for further study. Lectures will include some time for questions and discussion of the subject matter.

Tutorial classes will be dedicated to exercises and guidance in the practical assignments, focusing on selected topics.

Class schedules and materials will be supplied online, as well as additional information regarding the course.

Evaluation method

The evaluation of this curricular unit is made by two components: theoretical/problems (T) and project (P). Each component contributes with 50% to the final grade.

Both components are evaluated in an integer scale from 0 to 20.

To pass, the student must have:

(i)  a grade of at least 9,5 points in the theoretical/problems component; and

(ii) a grade of at least 9.5 points in the project component;

The final grade is the weighted average of the two evaluation components (0.5×T + 0.5×P) on an integer scale from 0 to 20 points. If the final grade is higher than 17 points, there will be an oral test to defend the grade obtained. After the oral examination, the grade will not be less than 17 points. If a student with a computed final grade larger than 17 does not attend the oral examination, the final grade is automatically defined as 17 points.

Theoretical/problems component

Two written tests evaluate this component. The grade of this component is the average of the two written tests.

Alternatively, this component can be evaluated by a written exam. The exam will be graded in two independent parts, each corresponding to one test and replacing its grade if better.

 An extra point will be awarded in the theoretical-practical component that will evaluate the delivery of a report transcribing and deepening the contents of one theoretical class.

Project component

Two mini-projects evaluate this component.

Some tutorial classes will be allocated to the mini-projects.

The mini-projects are done in groups of students, but this component is evaluated individually.

The grading of the different evaluation components is rounded to the first decimal place. The final grade is rounded to the closest integer value.



🤖 Language Model Policy


Language models can be handy for spell-checking and generating code structures, thus reducing coding time. However, this course does not primarily focus on these aspects. Sole reliance on language models is not enough for completing assignments, let alone for tests and exams.

In the real world, computer science Master graduates are hired to solve problems that less well-paid professionals cannot resolve using Large Language Models. This course aims to hone high-value skills that will benefit you in the near future.

All submitted work for any academic exercise should be the student''s original work. When preparing their work, students must carefully distinguish their own ideas and knowledge from information derived from other sources. The term "sources" encompasses not only published or computer-accessed primary and secondary material, but also information and opinions obtained directly from others. Therefore, any use of language models beyond spell-checking should be properly acknowledged.

It''s crucial to take the model''s notice seriously, as model responses are consecutive samplings of a probability distribution and can be inaccurate or misleading.



Subject matter

1. Introduction: fundamentals of deep learning, nonlinear transformations, and overfitting.
2. Artificial neural networks, backpropagation. and deep feedforward networks.
3. Implementation and training of deep neural networks
4. Optimization and regularization of feedforward networks. Training, testing, and cross-validation.
5. Convolution networks, theory, and practice
6. Unsupervised deep learning with autoencoders
7. Representation and transfer learning
8. Generative models
9. Reinforcement learning
10. Structured probabilistic models
11. Practical aspects of deep network selection, application, and optimization
12. Data and model visualization.
13. Open problems in deep learning


Programs where the course is taught: