Reproduce the MIMIC3 experimentation

[tsuttaket]

Hi,
I tried to reproduce the experimentation result on MIMIC3 dataset from the paper. However, I think there might be something wrong probably in the setting.

I got the results as the last 2 rows. The 3rd row "StageNET (Author's Saved Weights)" is pretty close to the paper's number; however when I retrained the model by calling
"python train.py --data_path='path/to/data/' --file_name='trained_model'"
I got the result as in the last line. (The best model picked was from epoch 9.)
Can you please help on how should I retrain the model.

	AUPRC	AUROC	min(Re, P+)
LSTM	0.280 (0.003)	0.897 (0.002)	0.324 (0.003)
StageNET (Paper's Result)	0.323 (0.002)	0.903 (0.002)	0.372 (0.003)
StageNET (Author's Saved Weights)	0.337471534	0.902723791	0.372237646
StageNET (Trained Model)	0.288944411	0.890060419	0.339553482

Thanks,
Thiti

[v1xerunt]

Hi Thiti,

I've uploaded a jupyter notebook file to this repo. The trained weights are directly generated from this file. I get the best model at epoch 10. For this issue, I guess there may have some randomness not controlled. By the way, this benchmark seems very sensitive to randomness. If you check the original benchmark paper, they report 0.323 AUPRC for the naive LSTM network. But in my implementation, I can only get 0.28 best AUPRC after fine-tuning the hyperparameters.

[tsuttaket]

Thank you so much for your help and suggestion.