Data Preprocessing

Objectives

Although data preprocessing, in the context of data analysis/mining is a critical step and takes the vast majority of time and efforts in an analytics project, the fact is that data preprocessing is still often neglected. The data preprocessing is usually a process loosely controlled, resulting in out of range values, e.g., impossible data combinations (e.g., Gender: Male; Pregnant: Yes), missing values, outliers, among many others. Moreover, any empirical analysis, ranging from simple hypothesis testing to develop neural networks for predictive purposes, will only yield as good results as the quality of the data provided. This course aims to present the most important rationale and methods in data preprocessing as a critical requirement for successful analytic tasks, providing the students the basic knowledge for their future data analysis¿ efforts.