numpy.core._exceptions.UFuncTypeError: ufunc 'add' did not contain a loop with signature matching types (dtype('int64'), dtype('<U1')) -> None

[baoyixin]

Hi, thank you for sharing your excellent work!
After preprocessing the WADI dataset, I encountered the following error when running the run.py file.

I've identified the issue at this line: early = early_detection_evaluation(labels, indicator, args.delays). However, I'm unsure how to fix it and would sincerely appreciate your guidance. Thank you！

[huankoh]

I believe this is due to the parallel processing that we used - some experienced the same issue when implemented in practical scenarios.

One way to resolve the issue is to (1) not do early detection evaluation or (2) adjust the code to run without parallel processing (e.g., a simple loop)

[baoyixin]

Bow ***@***.*** &nbsp; Dear Huan Yee Koh, I hope this message finds you well. I would like to express my gratitude for your prompt and insightful response to my previous inquiry. It was incredibly helpful, and I was able to make significant progress in my research thanks to your guidance. However, I have encountered a new challenge that I hope you could provide some insight into. As part of my ongoing project, I am working with the SWaT dataset, specifically using the Normal_v1.csv file for training data and the Attack_v0.csv file for testing data. I intended to apply the preprocessing steps outlined in your CST-GL paper to ensure the data aligns with the methodologies you've presented. Upon trying to implement the preprocessing steps, I've realized that the code provided in your GitHub repository, particularly the generate_wadi_data.ipynb notebook, does not directly apply to the SWaT data due to some differences between the SWaT and WADI datasets. This discrepancy has posed a significant challenge to the replication of your research's preprocessing phase for the SWaT dataset. Given your expertise and familiarity with the SWaT dataset's nuances in the context of your CST-GL paper, I was hoping you might be able to offer some guidance or share any specific preprocessing scripts or tips tailored for the SWaT dataset. This assistance would be invaluable to my project and would greatly aid in applying the insights from your paper more accurately. Thank you very much for considering my request. I look forward to your response and any guidance you can provide. Best regards, Yixin Bao

…

------------------&nbsp;原始邮件&nbsp;------------------ 发件人: "huankoh/CST-GL" ***@***.***&gt;; 发送时间:&nbsp;2024年3月26日(星期二) 上午7:29 ***@***.***&gt;; ***@***.******@***.***&gt;; 主题:&nbsp;Re: [huankoh/CST-GL] numpy.core._exceptions.UFuncTypeError: ufunc 'add' did not contain a loop with signature matching types (dtype('int64'), dtype('<U1')) -&gt; None (Issue #5) I believe this is due to the parallel processing that we used - some experienced the same issue when implemented in practical scenarios. One way to resolve the issue is to (1) not do early detection evaluation or (2) adjust the code to run without parallel processing (e.g., a simple loop) — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***&gt;

[huankoh]

For SWaT dataset, we follow the same procedure as we did in WaDI. However, in this case you don't need to add the time of attack (i.e., the anomaly labels) just as we showed them in the notebook. This is because the labels are already provided in the original dataset, so we used them as is. Running through the notebook with a few adaptation to SWaT dataset, you should get the same pre-processed dataset as GDN (https://github.com/d-ailin/GDN). So you can adjust the interactive notebook we provided that is easier to read or simply use the script provided in GDN repo.

[baoyixin]

Hi，I hope this email finds you well. I would like to extend my sincere gratitude for your previous guidance and responses, which have been instrumental in my research progress. I am writing to seek further assistance regarding the processing of the SWaT dataset within the GDN framework, as per your instructions. Despite meticulously following the steps you outlined, I have encountered an issue where the expected output files, specifically train.npz, val.npz, and test.npz, are not being generated. This has left me at an impasse in my data preparation phase. Could you kindly provide any additional insights or suggestions that might help in resolving this issue? Alternatively, if there are any updated steps or modifications to the process that I might have overlooked, I would greatly appreciate your guidance. Thank you once again for your invaluable support. I am looking forward to your response and any assistance you can provide. Bow ***@***.*** &nbsp;

…

------------------&nbsp;原始邮件&nbsp;------------------ 发件人: "huankoh/CST-GL" ***@***.***&gt;; 发送时间:&nbsp;2024年3月26日(星期二) 下午3:17 ***@***.***&gt;; ***@***.******@***.***&gt;; 主题:&nbsp;Re: [huankoh/CST-GL] numpy.core._exceptions.UFuncTypeError: ufunc 'add' did not contain a loop with signature matching types (dtype('int64'), dtype('<U1')) -&gt; None (Issue #5) For SWaT dataset, we follow the same procedure as we did in WaDI. However, in this case you don't need to add the time of attack (i.e., the anomaly labels) just as we showed them in the notebook. This is because the labels are already provided in the original dataset, so we used them as is. Running through the notebook with a few adaptation to SWaT dataset, you should get the same pre-processed dataset as GDN (https://github.com/d-ailin/GDN). So you can adjust the interactive notebook we provided that is easier to read or simply use the script provided in GDN repo. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***&gt;