Hi Rocio，can the preprocessed data be shared？

[heyang9]

I am working on a project on ICS security，but lack of experimental data. I hope to do the job based on your work. I noticed that there are only raw data here, so can the preprocessed data be shared? It will help a lot for me！

[Rocionightwater]

Dear Yang, In the folder 'results' you can find all the preprocessed datasets. E.g: " processed_data_LSTM/time-series-datasets <https://github.com/Rocionightwater/ML-NIDS-for-SCADA/tree/master/results/processed_data_LSTM/time-series-datasets>" corresponds to the data already preprocessed (data cleaning and data transformation) and ready to use for a LSTM algorithm. The same for Random Forest and SVM. If you are about to publish a paper, it would be great if you cite us. Let me know if you have any question. Best regards, Rocio 2018-01-10 10:24 GMT+01:00 Yang He <notifications@github.com>:

…

I am working on a project on ICS security，but lack of experimental data. I hope to do the job based on your work. I noticed that there are only raw data here, so can the preprocessed data be shared? It will help a lot for me！ — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AG_mZR21lJ0k_NO1uqMHa4KWNb2dErcAks5tJIHrgaJpZM4RZB5T> .

[heyang9]

Dear Rocio,

Thank you so much！ Sorry for not reading your results carefully.

I spent some time reading Ian Turnipseed's master thesis and your keynote to understand the details of the dataset better. But I couldn't find your master thesis online. Is it readable now? If I publish a paper, I will let you know and cite you!

Thanks again.
Best wishes,
Yang

[heyang9]

Hi Rocio,

I did some experiments on your results, and now hopes to verify its effectiveness. So I wonder what

changes you made to increase the features from 17 to 42 or 46... In the raw dataset, we can know what

an exact feature means, but now I couldn’t find out what it means.

Because I need to return those packets to the original condition, I have to know the features' meaning.

Best,

Yang

[Rocionightwater]

Hi Yang, The number of features increased because I used 4 different strategies to deal with the missing values (when preprocessing the datasets): (1) Keeping the last existent value, (2) adding zeros (in the place the missing data is) and adding indicators (add a 1 per missing value), (3) Gaussian mixture model and (4) Kmeans (to cluster the payloads depending on the missing values they payloads have). For (1) Keep last value strategy you will have the same amount of features (except for those that are categorical values, and I transform them to one hot encoding notation), for (2) Zeros and Indicators, you will have many more features since you are adding the indicators, and for (3) GMM and (4) Kmeans, the clusters are in one hot encoding notation. In conclusion, after preprocessing you will always have more features than when you didn't preprocess. I hope this can help you, Best regards, Rocio 2018-01-27 15:49 GMT+01:00 Yang He <notifications@github.com>:

…

Hi Rocio, I did some experiments on your results, and now hopes to verify its effectiveness. So I wonder what changes you made to increase the features from 17 to 42 or 46... In the raw dataset, we can know what an exact feature means, but now I couldn’t find out what it means. Because I need to return those packets to the original condition, I have to know the features' meaning. Best, Yang — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AG_mZUhy-SBVnW1K_o7SxqEScSZtdiO-ks5tOzeEgaJpZM4RZB5T> .