duplicated entries
Closed this issue · 6 comments
Hello, I am analyzing this dataset and find some duplicated entries as follows.
r4.large_i-059cc399590672c2b_terasort_hadoop_small_1
r4.large_i-0e6cc86a677b8793f_terasort_hadoop_small_1
What's the meaning of the token i-059cc399590672c2b and i-0e6cc86a677b8793f. Thanks. @oxhead
It's the instance name. The duplicate entries mean two separate experiments with the same configuration.
It's the instance name. The duplicate entries mean two separate experiments with the same configuration.
Got it, thx.
But I noticed that not all the configurations are run twice, some of them are only run by one time. Why?
Those duplicate entries come from duplicate runs of experiment scripts caused by interruption of experiments.
Those duplicate entries come from duplicate runs of experiment scripts caused by interruption of experiments.
thx~ But some duplicated runs still confuse me. The following two entries are both reports of a spark aggregation program running on the same instance(c3.2xlarge) with almost the same input size, yet their elapsed times are very different(almost 5x). So I wonder except for the timestamp, were their other conditions that are different between these two runs?
I believe their settings are the same. It can be attributed to performance variance in the cloud or the insufficient resource on c3.2xlarge (probably memory).
This example demonstrates that performance can varied a lot in cloud. In our paper, we point out that this kind of variance posts a challenge to Bayesian Optimization process .
I believe their settings are the same. It can be attributed to performance variance in the cloud or the insufficient resource on c3.2xlarge (probably memory).
This example demonstrates that performance can varied a lot in cloud. In our paper, we point out that this kind of variance posts a challenge to Bayesian Optimization process .
Got it. thx a lot. 👍