How to Win a Data Science Competition: Learn from Top Kagglers

About the Course

Participating in predictive modelling competitions can help you gain practical experience, improve and harness your data modelling skills in various domains such as credit, insurance, marketing, natural language processing, sales’ forecasting and computer vision to name a few. At the same time you get to do it in a competitive context against thousands of participants where each one tries to build the most predictive algorithm. Pushing each other to the limit can result in better performance and smaller prediction errors. Being able to achieve high ranks consistently can help you accelerate your career in data science. In this course, you will learn to analyse and solve competitively such predictive modelling tasks.

Disclaimer: This is not a machine learning online course in the general sense. This course will teach you how to get high-rank solutions against thousands of competitors with focus on practical usage of machine learning methods rather than the theoretical underpinnings behind them

Course Objectives

Understand how to solve predictive modelling competitions efficiently and learn which of the skills obtained can be applicable to real-world task

Learn how to preprocess the data and generate new features from various sources such as text and images

Be taught advanced feature engineering techniques like generating mean-encodings, using aggregated statistical measures or finding nearest neighbors as a means to improve your predictions

Be able to form reliable cross validation methodologies that help you benchmark your solutions and avoid overfitting or underfitting when tested with unobserved (test) data

Gain experience of analysing and interpreting the data. You will become aware of inconsistencies, high noise levels, errors and other data-related issues such as leakages and you will learn how to overcome them

Acquire knowledge of different algorithms and learn how to efficiently tune their hyperparameters and achieve top performance

Learning Outcomes

1. Data Analysis

2. Feature Extraction

3. Feature Engineering

4. Xgboost

Course Syllabus

Week 1. Introduction & Recap. Feature Preprocessing and Generation with Respect to Models. Final Project Description

Introduction & Recap

This week we will introduce you to competitive data science. You will learn about competitions' mechanics, the difference between competitions and a real life data science, hardware and software that people usually use in competitions. We will also briefly recap major ML models frequently used in competitions.

Feature Preprocessing and Generation with Respect to Models

In this module we will summarize approaches to work with features: preprocessing, generation and extraction. We will see, that the choice of the machine learning model impacts both preprocessing we apply to the features and our approach to generation of new ones. We will also discuss feature extraction from text with Bag Of Words and Word2vec, and feature extraction from images with Convolution Neural Networks.

Final Project

Description This is just a reminder, that the final project in this course is better to start soon! The final project is in fact a competition, in this module you can find an information about it.

Week 2. Exploratory Data Analysis. Validation. Data Leakages

Exploratory Data Analysis

We will start this week with Exploratory Data Analysis (EDA). It is a very broad and exciting topic and an essential component of solving process. Besides regular videos you will find a walk through EDA process for Springleaf competition data and an example of prolific EDA for NumerAI competition with extraordinary findings.

Validation

In this module we will discuss various validation strategies. We will see that the strategy we choose depends on the competition setup and that correct validation scheme is one of the bricks for any winning solution.

Data Leakages

Finally, in this module we will cover something very unique to data science competitions. That is, we will see examples how it is sometimes possible to get a top position in a competition with a very little machine learning, just by exploiting a data leakage.

Week 3. Metrics Optimization. Advanced Feature Engineering I

Metrics Optimization

This week we will first study another component of the competitions: the evaluation metrics. We will recap the most prominent ones and then see, how we can efficiently optimize a metric given in a competition.

Advanced Feature Engineering I

In this module we will study a very powerful technique for feature generation. It has a lot of names, but here we call it "mean encodings". We will see the intuition behind them, how to construct them, regularize and extend them.

Week 4. Hyperparameter Optimization. Advanced feature engineering II. Ensembling

Hyperparameter Optimization

In this module we will talk about hyperparameter optimization process. We will also have a special video with practical tips and tricks, recorded by four instructors.

Advanced feature engineering II

In this module we will learn about a few more advanced feature engineering techniques.

Ensembling

Nowadays it is hard to find a competition won by a single model! Every winning solution incorporates ensembles of models. In this module we will talk about the main ensembling techniques in general, and, of course, how it is better to ensemble the models in practice.

Week 5. Competitions go through. Final Project

Competitions go through

For the 5th week we've prepared for you several "walk-through" videos. In these videos we discuss solutions to competitions we took prizes at. The video content is quite short this week to let you spend more time on the final project. Good luck!

Final Project

Final project for the course.