What is reference model? Can reference model and target be same?
macabdul9 opened this issue · 4 comments
What is reference model? Can reference model and target model be the same? @swj0419
Same question here. As far as I understood, you claimed in the paper that your method, unlike many other MIA methods, is reference free? You don't need a reference model for the calibration? Correct me if I'm wrong.
I guess, I could just put a small model there, and after getting the results, I only look at "Min_{ratio*100}% Prob" and ignore the rest, since the calculation of "Min_{ratio*100}% Prob" does not rely on the reference model? But an option for only calculating "Min_{ratio*100}% Prob" with reference_model=None could be handy.
Reference model should be a whitebox model that is trained by the caliberation dataset? And target model is the blackbox model of our interest?
I am not the authors of this paper, but maybe I can provide some insights:
@macabdul9 The reference model is built for "imitating the behavior of the target model", and (as far as I know) it should not be the same as the target model because in this scenario you do not need to "refer to" it. You can further read this paper.
@XinnuoXu I think the reference model is not necessary for the proposed approach, but the code does not provide this option. The reference model in the code is used for baseline methods.
@iCSawyer @swj0419 I am confused about the code. The paper states that there is no need for a reference model, but the code reference model is necessary. It is either the default GPT3 or specified by the user input. Is there any code implemented according to the paper that can be referenced? I am waiting for your suggestion，thanks!