sdv-dev/SDV

Interpret PARSynthesizer loss.

jalr4ever opened this issue · 3 comments

jalr4ever commented

Environment details

  • SDV version: 1.16.1
  • Python version: 3.8.19
  • Operating System: Ubuntu 22.04.4 LTS (GNU/Linux 5.15.0-119-generic x86_64)
  • GPU: Tesla V100-SXM2-32GB-LS

Problem description

Hi, SDV. Recently, I've been using PARSynthesizer for simulating time series data, but I'm quite confused about the model's performance because its loss changes from a positive number to 0 and eventually becomes negative. Does this mean that the model's performance is getting worse?

What I already tried

I know that SDV has provided a blog explaining the loss of CTGAN, which is very good, but PARSynthesizer and CTGAN are completely different models. I hope you can clarify this in detail for me.

srinify commented

Hi @jalr4ever for PARSynthesizer, the loss can definitely be negative. As long as it's still decreasing overall then it's a good sign :)

This synthesizer uses a custom loss function, which you can read more about in our team's published paper starting on page 7: https://arxiv.org/pdf/2207.14406

jalr4ever commented

Hi @jalr4ever for PARSynthesizer, the loss can definitely be negative. As long as it's still decreasing overall then it's a good sign :)

This synthesizer uses a custom loss function, which you can read more about in our team's published paper starting on page 7: https://arxiv.org/pdf/2207.14406

@srinify Okay, thank you for your reply; it has cleared up my confusion. 😀

srinify commented

Awesome! @jalr4ever glad to hear