Data Collection, Administrative Sources and Big Data
This curricular unit aims to present a set of methodologies that support data gathering in the process of producing official statistics, including data collection processes (mainly in the context in which data collection takes place through surveys based on a questionnaire), the use of administrative data and the use of sources associated with what is commonly referred to as Big Data.
At the end of the curricular unit the student should achieve the following learning objectives:
1. Identify and differentiate methods of data collection.
2. Know and select the most appropriate data collection methods and modes for each situation.
3. Be able to design a data collection methodology based on a survey.
4. Identify potential administrative sources and methodological aspects in their use
5. Recognize the potentialities of the use of big data in official statistics and the methodological aspects in their use
Pedro Miguel Pereira Simões Coelho
Weekly - Available soon
Total - Available soon
Portuguese. If there are Erasmus students, classes will be taught in English
European Association of Methodology. International Handbook of Survey Methodology, Eds. Edith D. de Leeuw, Josep J. Hox, Don A. Dillman, 2008.;
European Commission. Handbook of Recommended Practices for Questionnaire Development and Testing in the European Statistical System, 2006.;
Statistics Canada. Survey Methods and Practices, 2010;
Malhotra, Naresh K., Birks, David F. (2007). Marketing research: an applied approach. Third European edition. Harlow: Prentice Hall/Financial Times.;
Vilares, M. J.; Coelho, P. A Satisfação e a Lealdade do Cliente. Metodologias de Avaliação, Gestão e Análise., 2ª Edição, Escolar Editora.,2011.
Wallgren, B. Wallgren (2014). Register-based Statistics Statistical Methods for Administrative Data. John Wiley&Sons, Ltd.
United Nations (2007). Register-based statistics in the Nordic countries – Review of best practices with focus on population and social statistics. Available online: http://unstats.un.org/unsd/dnss/docViewer.aspx?docID=2764
United Nations (2011). Using Administrative and Secondary Sources for Official Statistics: A Handbook of Principles and Practices. Available online:
P. Christen (2012). Data Matching – Concepts and Techniques for Record Linkage, Entity Resolution, and duplicate Detection. Springer.
This course is based on theoretical-practical classes including the presentation of concepts and methodologies, practical aplications, methodological discussions, and project/exercise resolution.
The evaluation includes a final exam (60%) and an optional project (40%).
The project includes the development of a report that should be discussed with the professor. The project comprehends the design of a methodology to support a data collection for a real problem.
To ensure approval it is necessary to achieve a minimum score of 8.5 in each component of assessment.
- Introduction to data collection methodologies
- The collection of data with surveys
- Planning the steps of the data collection process
- Sampling designs
- Target populations and sampling frames
- Sources of error in statistical operations
- Methods and modes of data collection
- Design of questionnaires
- Specificities of electronic surveys
- The collection of data with administrative sources
- Statistics based on administrative information
- The nature of administrative data
- The transfer of administrative information into statistical information
- Joint use of data from administrative sources and statistical surveys
- Data linkage and integration
- Evaluation of the quality of administrative data
- Protection of privacy and confidentiality
- Big data in the statistical production process
- Big date and digital trails
- Overview of big data sources
- The use of big data in official statistics
- Privacy and data protection
- Examples of the use of big data in the production of official statistics
- Methodological Challenges
- Big Data Tools Overview
Programs where the course is taught:
- PostGraduate in Smart Cities
- PostGraduate in Digital Enterprise Management
- PostGraduate in Information Systems Governance
- PostGraduate in Information Management and Business Intelligence in Healthcare
- PostGraduate in Enterprise Information Systems
- Postgraduate Program in Statistical Systems with a specialization in Central Banks
- Postgraduate Program in Statistical Systems with a specialization in Official Statistics