Deep learning based modeling approach to train large dataset and predict patient survival
š§¾Description: Getting a rapid understanding of the context of a patientās overall health has been particularly important during the COVID-19 pandemic as healthcare workers around the world struggle with hospitals overloaded by patients in critical condition. Intensive Care Units (ICUs) often lack verified medical histories for incoming patients. A patient in distress or a patient who is brought in confused or unresponsive may not be able to provide information about chronic conditions such as heart disease, injuries, or diabetes. Medical records may take days to transfer, especially for a patient from another medical provider or system. Knowledge about chronic conditions can inform clinical decisions about patient care and ultimately improve patient's survival outcomes.
Source of the dataset: Click Here
š§ Problem Statement: The target feature is hospital_death which is a binary variable. The task is to classify this variable based on the other 185 features step-by-step by going through each day's task. The scoring metric is Accuracy/Area under ROC curve.
Key challange:
- Large dataset with over 90K rows and 186 columns.
- Lack of significant relations with target feature
- Handled Missing Values using MCAR techniques
- EDA of the dataset to find distributions and relations of features
- Upsampling using SMOTE
- Feature selection using chi2 statistics
- Baseline modeling using neural network -
Validation score - 50% AUC - Fine tuned neural network -
Validation score - 85% AUC - Explainable AI usign shap Kernel explainer
- Sub-samplling features and reducing data improved the modeling metric score
- Using Keras-Tuner validation AUC score improved to
85%from50%