#!/usr/bin/env python
# coding: utf-8

# <img src="https://www.bestdesigns.co/uploads/inspiration_images/4350/990__1511457498_404_walmart.png" alt="WALMART LOGO" />
# 
# # Walmart : predict weekly sales
# 
# ## Company's Description 📇
# 
# Walmart Inc. is an American multinational retail corporation that operates a chain of hypermarkets, discount department stores, and grocery stores from the United States, headquartered in Bentonville, Arkansas. The company was founded by Sam Walton in 1962.
# 
# ## Project 🚧
# 
# Walmart's marketing service has asked you to build a machine learning model able to estimate the weekly sales in their stores, with the best precision possible on the predictions made. Such a model would help them understand better how the sales are influenced by economic indicators, and might be used to plan future marketing campaigns.
# 
# ## Goals 🎯
# 
# The project can be divided into three steps:
# 
# - Part 1 : make an EDA and all the necessary preprocessings to prepare data for machine learning
# - Part 2 : train a **linear regression model** (baseline)
# - Part 3 : avoid overfitting by training a **regularized regression model**
# 
# ## Scope of this project 🖼️
# 
# For this project, you'll work with a dataset that contains information about weekly sales achieved by different Walmart stores, and other variables such as the unemployment rate or the fuel price, that might be useful for predicting the amount of sales. The dataset has been taken from a Kaggle competition, but we made some changes compared to the original data. Please make sure that you're using **our** custom dataset (available on JULIE). 🤓

# ## Deliverable 📬
# 
# To complete this project, your team should: 
# 
# - Create some visualizations
# - Train at least one **linear regression model** on the dataset, that predicts the amount of weekly sales as a function of the other variables
# - Assess the performances of the model by using a metric that is relevant for regression problems
# - Interpret the coefficients of the model to identify what features are important for the prediction
# - Train at least one model with **regularization (Lasso or Ridge)** to reduce overfitting
# 

# ## Helpers 🦮
# 
# To help you achieve this project, here are a few tips that should help you: 
# 
# ### Part 1 : EDA and data preprocessing
# 
# Start your project by exploring your dataset : create figures, compute some statistics etc...
# 
# Then, you'll have to make some preprocessing on the dataset. You can follow the guidelines from the *preprocessing template*. There will also be some specific transformations to be planned on this dataset, for example on the *Date* column that can't be included as it is in the model. Below are some hints that might help you 🤓
# 
#  #### Preprocessing to be planned with pandas
# 
#  **Drop lines where target values are missing :**
#  - Here, the target variable (Y) corresponds to the column *Weekly_Sales*. One can see above that there are some missing values in this column.
#  - We never use imputation techniques on the target : it might create some bias in the predictions !
#  - Then, we will just drop the lines in the dataset for which the value in *Weekly_Sales* is missing.
#  
# **Create usable features from the *Date* column :**
# The *Date* column cannot be included as it is in the model. Either you can drop this column, or you will create new columns that contain the following numeric features : 
# - *year*
# - *month*
# - *day*
# - *day of week*
# 
# **Drop lines containing invalid values or outliers :**
# In this project, will be considered as outliers all the numeric features that don't fall within the range : $[\bar{X} - 3\sigma, \bar{X} + 3\sigma]$. This concerns the columns : *Temperature*, *Fuel_price*, *CPI* and *Unemployment*
#  
# 
# 
# **Target variable/target (Y) that we will try to predict, to separate from the others** : *Weekly_Sales*
# 
#  **------------**
# 
#  #### Preprocessings to be planned with scikit-learn
# 
#  **Explanatory variables (X)**
# We need to identify which columns contain categorical variables and which columns contain numerical variables, as they will be treated differently.
# 
#  - Categorical variables : Store, Holiday_Flag
#  - Numerical variables : Temperature, Fuel_Price, CPI, Unemployment, Year, Month, Day, DayOfWeek
# 
# ### Part 2 : Baseline model (linear regression)
# Once you've trained a first model, don't forget to assess its performances on the train and test sets. Are you satisfied with the results ?
# Besides, it would be interesting to analyze the values of the model's coefficients to know what features are important for the prediction. To do so, the `.coef_` attribute of scikit-learn's LinearRegression class might be useful. Please refer to the following link for more information 😉 https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html
# 
# ### Part 3 : Fight overfitting
# In this last part, you'll have to train a **regularized linear regression model**. You'll find below some useful classes in scikit-learn's documentation :
# - https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Ridge.html#sklearn.linear_model.Ridge
# - https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Lasso.html#sklearn.linear_model.Lasso
# 
# **Bonus question**
# 
# In regularized regression models, there's a hyperparameter called *the regularization strength* that can be fine-tuned to get the best generalized predictions on a given dataset. This fine-tuning can be done thanks to scikit-learn's GridSearchCV class : https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html
# 
# Also, you'll find here some examples of how to use GridSearchCV together with Ridge or Lasso models : https://alfurka.github.io/2018-11-18-grid-search/

# In[ ]:




