📉 Customer Churn Prediction — Kaggle Playground Series S6E3

Predicting customer churn in a telecom dataset using advanced feature engineering,
ensemble learning, and automated hyperparameter optimization.

---

🏆 Result

Metric	Score
Best AUC (Optuna-tuned)	0.9157
Optimization Framework	Optuna
Best Model	XGBoost (via Optuna selection)

---

🗂 Project Structure

kaggle-churn-s6e3/
│
├── data/                  # Raw and processed datasets
├── notebooks/             # Exploratory and modeling notebooks
│   ├── 01_eda.ipynb
│   ├── 02_feature_engineering.ipynb
│   └── 03_modeling.ipynb
├── src/                   # Reusable Python modules
│   ├── features.py
│   └── train.py
├── requirements.txt
└── README.md

---

🔬 Feature Engineering

This project places heavy emphasis on feature engineering as the primary driver of model performance. The following techniques were applied:

📐 PCA-Based Features

• Outlier detection via PCA — identifying anomalous customers in reconstructed feature space
• PCA-inspired features — manually crafted features motivated by principal component directions
• PCA components as direct features — top components included as model inputs

📊 Feature Utility Scoring

A custom Feature Utility Score function was implemented to evaluate and rank features using multiple methods, guiding feature selection and reducing noise.

🔗 Interaction Features

Pairwise interaction terms between high-utility features were generated to capture non-linear relationships the base model might miss.

🧩 K-Means Clustering

Customer segments derived from unsupervised K-Means clustering were added as categorical features, giving the model implicit behavioral groupings.

---

🤖 Modeling

Multiple ensemble models were evaluated and compared:

Model	Notes
XGBoost	Strong baseline, tuned via Optuna
LightGBM	Fast training, competitive AUC
Random Forest	Used for comparison and feature importance
AutoGluon	Automated ensemble stacking baseline

Optuna was used for both model selection and hyperparameter optimization, running trials across models and parameter spaces to find the best-performing configuration.

---

⚙️ How to Run

# 1. Clone the repository
git clone https://github.com/EhsanFarazmand/kaggle-churn-s6e3.git
cd kaggle-churn-s6e3

# 2. Install dependencies
pip install -r requirements.txt

# 3. Run the notebook
jupyter notebook notebooks/03_modeling.ipynb

---

📦 Requirements

scikit-learn
xgboost
lightgbm
autogluon
optuna
pandas
numpy
matplotlib
seaborn
jupyter

---

📎 References

• Kaggle Competition — Playground Series S6E3
• My Kaggle Notebook

---

👤 Author

Ehsan Farazmand — ML Engineer